Python Project(#1) - Colorize Black & White Images with Python

This Deep Learning Project aims to provide colourizing black & white images with Python.

In image colourization, we take a black and white image as input and produce a coloured image. We will solve this project with OpenCV deep neural network.

Like RGB, Lab is another colour space. It is also three-channel colour space like RGB where the channels are:

• L channel: This channel represents the Lightness
• a channel: This channel represents green-red
• b channel: This channel represents blue-yellow

In this colour space, the grayscale part of the image is only encoded in L channel. Therefore Lab colour space is more favourable for our project.

Problem Statement:

We can formulate our problem statement as to predict a and b channels, given an input grayscale image.

In this deep learning project, we will use OpenCV DNN architecture which is trained on ImageNet dataset. The neural net is trained with the L channel of images as input data and a,b channels as target data.

Steps to implement Image Colorization Project:

For colourizing black and white images, we will be using a pre-trained Caffe model, a prototxt file, and a NumPy file.

The prototxt file defines the network and the numpy file stores the cluster centre points in numpy format.

1. Make a directory with name models.

1. mkdir models

2. Open the terminal and run following commands to download the Caffe model, prototxt file and the NumPy file.

1. wget https://github.com/richzhang/colorization/blob/master/colorization/resources/pts_in_hull.npy?raw=true -O ./pts_in_hull.npy
3. wget https://raw.githubusercontent.com/richzhang/colorization/master/colorization/models/colorization_deploy_v2.prototxt -O ./models/colorization_deploy_v2.prototxt
5. wget http://eecs.berkeley.edu/~rich.zhang/projects/2016_colorization/files/demo_v2/colorization_release_v2.caffemodel -O ./models/colorization_release_v2.caffemodel

3. Create a python file image_colorization.py and paste the code specified in the below steps

4. Imports:

1. import numpy as np
2. import cv2 as cv
3. import os.path

5. Read B&W image and load the caffemodel:

1. frame = cv.imread(“__test_image_path__”)
3. numpy_file = np.load('./pts_in_hull.npy')
5. Caffe_net = cv.dnn.readNetFromCaffe("./models/colorization_deploy_v2.prototxt", "./models/colorization_release_v2.caffemodel")

6. Add layers to the caffe model:

1. numpy_file = numpy_file.transpose().reshape(2, 313, 1, 1)
2. Caffe_net.getLayer(Caffe_net.getLayerId('class8_ab')).blobs = [numpy_file.astype(np.float32)]
3. Caffe_net.getLayer(Caffe_net.getLayerId('conv8_313_rh')).blobs = [np.full([1, 313], 2.606, np.float32)]

7. Extract L channel and resize it:

1. input_width = 224
2. input_height = 224
4. rgb_img = (frame[:,:,[2, 1, 0]] * 1.0 / 255).astype(np.float32)
5. lab_img = cv.cvtColor(rgb_img, cv.COLOR_RGB2Lab)
6. l_channel = lab_img[:,:,0]
8. l_channel_resize = cv.resize(l_channel, (input_width, input_height))
9. l_channel_resize -= 50

8. Predict the ab channel and save the result:

1. Caffe_net.setInput(cv.dnn.blobFromImage(l_channel_resize))
2. ab_channel = Caffe_net.forward()[0,:,:,:].transpose((1,2,0))
4. (original_height,original_width) = rgb_img.shape[:2]
5. ab_channel_us = cv.resize(ab_channel, (original_width, original_height))
6. lab_output = np.concatenate((l_channel[:,:,np.newaxis],ab_channel_us),axis=2)
7. bgr_output = np.clip(cv.cvtColor(lab_output, cv.COLOR_Lab2BGR), 0, 1)
9. cv.imwrite("./result.png", (bgr_output*255).astype(np.uint8))

Results:

Now execute the deep learning project – run the python file with path of a grayscale image to test our results.

1. python3 image_colorization.py

After running the above python file, the colorized image will be saved in your working directory as result.png

Code for GUI:

Make a new python file gui.py in the present directory. Paste the below code for image colorization interface.

1. import tkinter as tk
2. from tkinter import *
3. from tkinter import filedialog
4. from PIL import Image, ImageTk
5. import os
6. import numpy as np
7. import cv2 as cv
8. import os.path
11. numpy_file = np.load('./pts_in_hull.npy')
12. Caffe_net = cv.dnn.readNetFromCaffe("./models/colorization_deploy_v2.prototxt", "./models/colorization_release_v2.caffemodel")
13. numpy_file = numpy_file.transpose().reshape(2, 313, 1, 1)
15. class Window(Frame):
16. def __init__(self, master=None):
17. Frame.__init__(self, master)
19. self.master = master
20. self.pos = []
21. self.master.title("B&W Image Colorization")
22. self.pack(fill=BOTH, expand=1)
24. menu = Menu(self.master)
25. self.master.config(menu=menu)
27. file = Menu(menu)
28. file.add_command(label="Upload Image", command=self.uploadImage)
29. file.add_command(label="Color Image", command=self.color)
30. menu.add_cascade(label="File", menu=file)
32. self.canvas = tk.Canvas(self)
33. self.canvas.pack(fill=tk.BOTH, expand=True)
34. self.image = None
35. self.image2 = None
36. label1=Label(self,image=img)
37. label1.image=img
38. label1.place(x=400,y=370)
43. def uploadImage(self):
44. filename = filedialog.askopenfilename(initialdir=os.getcwd())
45. if not filename:
46. return
47. load = Image.open(filename)
49. load = load.resize((480, 360), Image.ANTIALIAS)
51. if self.image is None:
52. w, h = load.size
53. width, height = root.winfo_width(), root.winfo_height()
54. self.render = ImageTk.PhotoImage(load)
55. self.image = self.canvas.create_image((w / 2, h / 2), image=self.render)
57. else:
58. self.canvas.delete(self.image3)
59. w, h = load.size
60. width, height = root.winfo_screenmmwidth(), root.winfo_screenheight()
62. self.render2 = ImageTk.PhotoImage(load)
63. self.image2 = self.canvas.create_image((w / 2, h / 2), image=self.render2)
66. frame = cv.imread(filename)
68. Caffe_net.getLayer(Caffe_net.getLayerId('class8_ab')).blobs = [numpy_file.astype(np.float32)]
69. Caffe_net.getLayer(Caffe_net.getLayerId('conv8_313_rh')).blobs = [np.full([1, 313], 2.606, np.float32)]
71. input_width = 224
72. input_height = 224
74. rgb_img = (frame[:,:,[2, 1, 0]] * 1.0 / 255).astype(np.float32)
75. lab_img = cv.cvtColor(rgb_img, cv.COLOR_RGB2Lab)
76. l_channel = lab_img[:,:,0]
78. l_channel_resize = cv.resize(l_channel, (input_width, input_height))
79. l_channel_resize -= 50
81. Caffe_net.setInput(cv.dnn.blobFromImage(l_channel_resize))
82. ab_channel = Caffe_net.forward()[0,:,:,:].transpose((1,2,0))
84. (original_height,original_width) = rgb_img.shape[:2]
85. ab_channel_us = cv.resize(ab_channel, (original_width, original_height))
86. lab_output =
87. np.concatenate((l_channel[:,:,np.newaxis],ab_channel_us),axis=2)
88. bgr_output = np.clip(cv.cvtColor(lab_output, cv.COLOR_Lab2BGR), 0, 1)
91. cv.imwrite("./result.png", (bgr_output*255).astype(np.uint8))
93. def color(self):
95. load = Image.open("./result.png")
96. load = load.resize((480, 360), Image.ANTIALIAS)
98. if self.image is None:
99. w, h = load.size
100. self.render = ImageTk.PhotoImage(load)
101. self.image = self.canvas.create_image((w / 2, h/2), image=self.render)
102. root.geometry("%dx%d" % (w, h))
103. else:
104. w, h = load.size
105. width, height = root.winfo_screenmmwidth(), root.winfo_screenheight()
107. self.render3 = ImageTk.PhotoImage(load)
108. self.image3 = self.canvas.create_image((w / 2, h / 2), image=self.render3)
109. self.canvas.move(self.image3, 500, 0)
110. root = tk.Tk()
111. root.geometry("%dx%d" % (980, 600))
112. root.title("B&W Image Colorization GUI")
113. img = ImageTk.PhotoImage(Image.open("logo2.png"))
115. app = Window(root)
116. app.pack(fill=tk.BOTH, expand=1)
118. root.mainloop()

NOTE: IGNORE THIS 'select' COLOUR  CODES 

Summary:

This tutorial guides you on how to build a deep learning project to colourize black and white images. It first introduces you to Lab colour space and why it is favourable for our problem statement. Then step by step it describes how to implement black and white image colourizer.

Are you looking for more deep learning projects with source code? Please write in the comment section, we will develop and publish the project.