News classification
Contact us
- Add: No. 9, North Fourth Ring Road, Haidian District, Beijing. It mainly includes face recognition, living detection, ID card recognition, bank card recognition, business card recognition, license plate recognition, OCR recognition, and intelligent recognition technology.
- Tel: 13146317170 廖经理
- Fax:
- Email: 398017534@qq.com
Using deep learning technology to turn black and white pictures into color pictures.
Using deep learning technology to turn black and white pictures into color pictures.
Using deep learning technology to turn black and white pictures into color pictures.
Objective task
Use a black and white photo to create its color version. Intuitively, this idea is quite simple. According to the content of the picture, know what its color should be. For example, the leaves are usually green, the sky is blue, the clouds are white, and so on. What we need to do is to enable the computer to complete it.
Previous work and existing problems
In the past, in-depth learning was mainly used. They used regression to predict the color of each pixel, but the results were not satisfactory. The color was dull and monotonous.
Figure 1 Results of a regression based model. Left: Input to the to
Figure 1 Results of a regression based model. Left: Input to the to
Previous works have used Mean Squared Error (MSE) as the loss function of the exercise model. The authors point out that MSE will try to "uniformly" produce color in order to achieve the minimum uniform error, which will lead to dull appearance. So the author changed his mind and changed the picture coloring into classification.
Input and output of model
The author used the LAB color space form (the most common way to make RGB). In the lab project, the L channel recorded light intensity values, and the other two channels recorded a set of opposing colors, green-red and blue-yellow. You can understand more about LAB here.
A very good reason for using LAB color space is that it can keep the light intensity value. The benefit is that black-and-white images can be used as L-channels, so that models do not have to learn how to persist in light intensity when predicting. (Application of RGB is necessary). In this way, the model only needs to learn how to color the image, so as to focus on the key steps.
The AB channel value of the model output can be directly applied to black and white images to get color images.
The model is a fairly standard convolutional neural network. Instead of pooling layer, the authors use upper/lower sampling layer.
Color quantization
After a brief introduction above, the author used a classification model to replace the regression model. As a result, the number of categories needs to be corrected. The author selects 313 AB pairs as the possible number of categories. Although it looks a little bit less, it can also use some ways to ensure that more colors can be presented. (at the end of the article)
loss function
The loss function used by the author is the standard Cross Entropy. Z is the real type of pixels, and Z^ is the output of the model.
Equation 1 Standard Cross Entropy
Equation 1 Standard Cross Entropy
The author also points out that the color value will be unbalanced in classification, and the use of interpolation entropy as a loss function can not play a good role in classification imbalance. Classes with fewer examples usually have higher weight. Unsaturated shades of grey and light blue are more common than others because they often appear in lee. So the author proposes a weight plan.
Equation 1.1 Weighing scheme
Equation 1.1 Weighing scheme
The author calculates the distribution of class P from ImageNet database. Q is the type number (313), and the 0.5 effect is good. It should be noted that the author has stopped smoothing the
So after thinking about weight, our loss function is as follows:
Equation 1.3 Loss function after weighing scheme
V () is the weight value of each class, and H and W are the height and width of the image respectively.
Predicting colors based on type
Using 313 color classifications directly makes the image very rough, so long as a few can match the real color.
In order to get more color range from the prediction model, the author adds one step of subsequent disposal.
Equation 2 Calculating color from class probabilities
Equation 2 Calculating color from class probabilities
H is a function, Z is the output of the model, T is a hyper-parameter, the author experimented with the following different values.
Figure 1 Effect of different values of T
This is a clever step, because the output of the model is a set of probabilistic values, which are better than taking only the color values corresponding to the maximum approximation rate (as in image classification), by trying to use the model to output all the information of a set of probabilistic values to calculate the color.
Exercise frame
Such a neural network is divided into two parts. First, exercise data is positively propagated through the model, and output is predicted. Then calculate the reciprocal of H to calculate the loss (loss).
Result
The results of the experiment are very bright, and in most cases are almost the same as ideals. But be aware that sometimes there is a bias from the real world, but semantically correct (the model gives the right color for the purpose).
Figure 2 Comparison between different colouring frameworks
Colouring Legacy Black & White Photos
Figure 2.1 Colouring photos that were originally taken in Black & White
conclusion
In this paper, we discuss a new method to stop coloring images by using a modified loss function. We discuss how to control vibrancy via hyperparameters and why class rebalancing plays an important role in natural image coloring