Happiness Detection in Images using Keras

Hello folks! Are you happy or are you not sure? Alright, let's build a model that will help you find out if you're happy or not. 

Well, let's start with some basic understanding of this tutorial and later dive deeper into the neural networks. We're very well known what  popular Computer Vision is. It is one of the most popular field of machine learning. Happiness Detection is also one of such field where we apply Computer Vision techniques. This is a binary classification type of problem where we'll building a model that will detect whether the input image is either smiling or not.

 

The dataset is already labeled as smiling or not smiling. We'll be using 600 images for training and 150 images as test dataset. Before we get our hands into the core part, let's first import some libraries.

	import numpy as np
	from keras import layers
	from keras.layers import Input, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D
	from keras.layers import AveragePooling2D, MaxPooling2D, Dropout, GlobalMaxPooling2D, GlobalAveragePooling2D
	from keras.models import Model
	from keras.preprocessing import image
	from keras.utils import layer_utils
	from keras.utils.data_utils import get_file
	from keras.applications.imagenet_utils import preprocess_input
	import pydot
	from IPython.display import SVG
	from keras.utils.vis_utils import model_to_dot
	from keras.utils import plot_model
	from kt_utils import *
	import keras.backend as K
	K.set_image_data_format('channels_last')
	import matplotlib.pyplot as plt
	from matplotlib.pyplot import imshow
	# %matplotlib inline ##this line is only necessary when you're executing in python notebook

view raw import.py hosted with ❤ by GitHub

Now let's know more about the data. 

	X_train_orig, Y_train_orig, X_test_orig, Y_test_orig, classes = load_dataset()
	# Normalize image vectors
	X_train = X_train_orig/255.
	X_test = X_test_orig/255.
	# Reshape
	Y_train = Y_train_orig.T
	Y_test = Y_test_orig.T
	print ("number of training examples = " + str(X_train.shape[0]))
	print ("number of test examples = " + str(X_test.shape[0]))
	print ("X_train shape: " + str(X_train.shape))
	print ("Y_train shape: " + str(Y_train.shape))
	print ("X_test shape: " + str(X_test.shape))
	print ("Y_test shape: " + str(Y_test.shape))

view raw view_data.py hosted with ❤ by GitHub

After the execution, you'll be able to look at the number of data we've taken for training and testing the prepared model. Now we'll be using Keras to build and compile our model. If you don't have enough knowledge about using keras library, you can refer to the references section for additional resources. Note: Here, Tensorflow is being used in the backend. For knowing how to install it in backend, follow a stackoverflow question here.

Here we are creating a simple convolutional network. The network architecture goes like this:

INPUT layer -> CONV -> BN -> RELU activatation -> MAXPOOL -> FLATTEN X (means convert it to a vector) + FULLYCONNECTED

	def HappyModel(input_shape):
	"""
	Implementation of the HappyModel.
	Arguments:
	input_shape -- shape of the images of the dataset
	(height, width, channels) as a tuple.
	Note that this does not include the 'batch' as a dimension.
	If you have a batch like 'X_train',
	then you can provide the input_shape using
	X_train.shape[1:]
	Returns:
	model -- a Model() instance in Keras
	"""
	### START CODE HERE ###
	# Feel free to use the suggested outline in the text above to get started, and run through the whole
	# exercise (including the later portions of this notebook) once. The come back also try out other
	# network architectures as well.
	X_input = Input(input_shape)
	# Zero-Padding: pads the border of X_input with zeroes
	X = ZeroPadding2D((3, 3))(X_input)
	# CONV -> BN -> RELU Block applied to X
	X = Conv2D(32, (7, 7), strides = (1, 1), name = 'conv0')(X)
	X = BatchNormalization(axis = 3, name = 'bn0')(X)
	X = Activation('relu')(X)
	# MAXPOOL
	X = MaxPooling2D((2, 2), name='max_pool')(X)
	# FLATTEN X (means convert it to a vector) + FULLYCONNECTED
	X = Flatten()(X)
	X = Dense(1, activation='sigmoid', name='fc')(X)
	# Create model. This creates your Keras model instance, you'll use this instance to train/test the model.
	model = Model(inputs = X_input, outputs = X, name='HappyModel')
	### END CODE HERE ###
	return model

view raw model.py hosted with ❤ by GitHub

have now built a function to describe your model. To train and test this model, there are four steps in Keras: 

1. Create the model by calling the function above

2. Compile the model by calling model.compile(optimizer = "...", loss = "...", metrics = ["accuracy"])

3. Train the model on train data by calling model.fit(x = ..., y = ..., epochs = ..., batch_size = ...)  

4. Test the model on test data by calling model.evaluate(x = ..., y = ...)

If you want to know more about model.compile(), model.fit(), model.evaluate() and their arguments, refer to the official Keras documentation.

By 'Create the model', we mean calling the function created above by passing X_train.shape[1:] as a parameter. The second step is to compile the model with a defined optimizer function, a loss function and a metric class. You can try and include 'sgd', 'adam' or others for optimizer (optimizer documentation). Similarly, the loss function that you can use in this case is only 'binary_cross_entropy' since "happiness detection" is a binary classification problem. 

	##creating model
	happyModel = HappyModel(X_train.shape[1:])
	##compiling the model
	happyModel.compile('adam', 'binary_crossentropy', metrics=['accuracy'])

view raw create_compile.py hosted with ❤ by GitHub

Now it's time we train the model. For training, we simply need to pass the training set, the number of epochs and the batch size as parameter. Epoch generally means the number of passes of the entire training dataset the machine learning algorithm has completed. Since we're defining the batch size as well, it's the number of iterations for passing the training dataset in a single batch.

	### START CODE HERE ### (1 line)
	happyModel.fit(X_train, Y_train, epochs=40, batch_size=50)
	### END CODE HERE ###

view raw train_model.py hosted with ❤ by GitHub

After this, we evaluate the model with the test data which we just completed training. As similar to training the model, we pass test data (150 images) and batch size as the parameter to the evaluate method. 

	preds = happyModel.evaluate(X_test, Y_test, batch_size=32)
	print()
	print ("Loss = " + str(preds[0]))
	print ("Test Accuracy = " + str(preds[1]))

view raw evaluate.py hosted with ❤ by GitHub

You can analyze your models accuracy and loss. If not very accurate, you can still improve it by making some changes. 

Tips for improving your model

If you have not yet achieved a very good accuracy (>= 80%), here are some things tips:

• Use blocks of CONV->BATCHNORM->RELU such as:

  X = Conv2D(32, (3, 3), strides = (1, 1), name = 'conv0')(X)
  X = BatchNormalization(axis = 3, name = 'bn0')(X)
  X = Activation('relu')(X)
  

  until your height and width dimensions are quite low and your number of channels quite large (≈32 for example).
  You can then flatten the volume and use a fully-connected layer.
• Use MAXPOOL after such blocks. It will help you lower the dimension in height and width.
• Change your optimizer. We find 'adam' works well.
• If you get memory issues, lower your batch_size (e.g. 12 )
• Run more epochs until you see the train accuracy no longer improves.
  

Now let's pass our own image to the model and predict. 

	img_path = 'images/pic5.jpeg' ##location to you image
	img = image.load_img(img_path, target_size=(64, 64))
	imshow(img)
	x = image.img_to_array(img)
	x = np.expand_dims(x, axis=0)
	x = preprocess_input(x)
	print(happyModel.predict(x))

view raw test_my_image.py hosted with ❤ by GitHub

Congratulations, you have successfully used Keras to build "Happiness Detection". You can take a look at a summary of your model by executing the code below.

	##prints the details of your layers in a table.
	happyModel.summary()
	##plots a graph of all the steps taken to build the model.
	##You can also save it in .png or .svg format
	plot_model(happyModel, to_file='HappyModel.png')
	SVG(model_to_dot(happyModel).create(prog='dot', format='svg'))

view raw model_summary.py hosted with ❤ by GitHub

You can clone notebook and other essentials from this link. You can find this article in my github page as well.

References:

• Coursera deeplearning.ai course: Convolutional Neural Network