Convolutions and image processing

Facemorpher

I wanted to use this with Marilyn

github

Here is the doc.

It took me 2h to install it. But after that it was easy to use...

Must be a GPL problem...

sudo apt-get install g++

create a virtual environment and install into it

  • conda create --name opencv-env python=3.6 conda activate opencv-env
  • pip install opencv-contrib-python
  • pip install facemorpher

surprisingly the environment is really empty so...

  • conda install -c intel scikit-learn

install the kernel in jupyter

  • conda install -c anaconda ipykernel
  • python -m ipykernel install --user --name=opencv-env

what about this ?

https://github.com/spmallick/learnopencv/blob/master/FaceMorph/faceMorph.py

or

this project

or

In [ ]:
import numpy as np
import matplotlib.pyplot as plt
In [37]:
import facemorpher as fm
import imageio
import cv2
In [116]:
img = cv2.imread('./obama2.png')
X,Y = fm.locator.face_points(img).T
plt.imshow(img);
plt.plot(X,Y, 'ro');
In [138]:
pts = fm.locator.face_points(img)

plt.imshow(img);
X,Y = pts[:16].T
plt.plot(X,Y, 'ro');
X,Y = pts[16:48].T
plt.plot(X,Y, 'go');
X,Y = pts[48:59].T
plt.plot(X,Y, 'yo');
X,Y = pts[59:-2].T
plt.plot(X,Y, 'bo');
In [107]:
pp = fm.morpher(['./marilyn_w.png','./obama_w.png'],
                 num_frames= 60,
                out_video='xx.avi' )

./marilyn_w.png
./obama_w.png
xx.avi saved
In [63]:
Out[63]:
79
In [139]:
! ../.g

[master 3e7be5d] web
 1 file changed, 15 insertions(+), 17 deletions(-)
Counting objects: 4, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 26.83 KiB | 1.12 MiB/s, done.
Total 4 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/macbuse/macbuse.github.io.git
   405fc9b..3e7be5d  master -> master

Rewrite

  • using signal.convolve
  • skimage.io
In [103]:
from skimage import io
import scipy.signal

img = io.imread( "./marilyn.png", as_gray=True)

img = io.imread( "./obama2.png", as_gray=True)

dd = 4

kernel = -20*np.ones((5*dd,1*dd)) 
kernel[2*dd:4*dd] = 40
kernel /= kernel.sum()

#V = np.array([1,0,-1])
#kernel = np.vstack([V, -2*V, V])


img_conv = scipy.signal.convolve2d( img, 
                                 kernel, 
                                 boundary='fill')

output = np.copy(img_conv)
#the values are float
mask = output > .7
output[ mask] = 1.
output[~mask] = 0

plt.imshow(output);

Varying the parameter in mask

In [104]:
imgs = []
for k in np.linspace(.05,.5, 9):
    output = np.copy(img_conv)
    #the values are float
    mask = output > k
    output[ mask] = 1.
    output[~mask] = 0
    imgs.append(output)
    
num_rows, num_cols = 3,3
fig, axs = plt.subplots(num_rows, num_cols)
fig.set_size_inches(10,10)

axs = axs.ravel()
for ax,pic in zip(axs, imgs):
    ax.set_axis_off() 
    ax.imshow(pic)

Interpolating between endpoints

In [105]:
num_rows, num_cols = 3,3
fig, axs = plt.subplots(num_rows, num_cols)
fig.set_size_inches(14,14)

axs = axs.ravel()
    
for ax in axs:
    ax.set_axis_off() 
 
for t, ax in zip(np.linspace(0,1,9), axs):
    ax.imshow(t*imgs[-1] + (1-t)*imgs[0]);

Colors

In [115]:
x,y = imgs[0].shape

rgb = np.ones((x,y,3))


rgb[:,:,0] = 1*imgs[0]
rgb[:,:,1] = 1*imgs[-3]
rgb[:,:,2] = imgs[-1]

plt.imshow(rgb)
imageio.imsave('obama_w.png', rgb)

Lossy conversion from float64 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning.
In [1]:
! ../.g

[master 3ab9401] web
 2 files changed, 77 insertions(+), 1768 deletions(-)
Counting objects: 5, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 58.38 KiB | 979.00 KiB/s, done.
Total 5 (delta 4), reused 0 (delta 0)
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To https://github.com/macbuse/macbuse.github.io.git
   9d93664..3ab9401  master -> master

Convolution applied to a curve

There is a problem. Can u see why this doesn't work ?

this works

what is happening ?

In [69]:
import  matplotlib.animation
from matplotlib import animation

from IPython.display import HTML

##Animation code

def plot_images(img_list):
  def init():
    img.set_data(img_list[0])
    return (img,)

  def animate(i):
    img.set_data(img_list[i])
    return (img,)

  fig = plt.figure()
  ax = fig.gca()
  plt.axis('off')
  img = ax.imshow(img_list[0])

  anim = animation.FuncAnimation(fig, animate, 
                                 init_func=init,
                                 frames=len(img_list), 
                                 interval=200, 
                                 blit=True)
  return anim

Set up

  • random starting configuration G
  • the kernel K
In [173]:
! ../.g

[master 7641183] web
 1 file changed, 60 insertions(+), 83 deletions(-)
Counting objects: 4, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 209.26 KiB | 4.87 MiB/s, done.
Total 4 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/macbuse/macbuse.github.io.git
   4facb17..7641183  master -> master
In [ ]: