IMAGE CLASSIFICATION USING SIFT+KMEANS+KNN PYTHON

Implement Bike vs. Horse Classification 

(Use Bag-of-visual words approach (SIFT/SURF + KMEANS + KNN))

In this post i am trying to classify the test image into either Bike class or the Horse Class.

Procedure (KNN):
1. Get the path of images in the training set.
2. Extract SIFT features from each and every image in the set.
3. Compute K-Means over the entire set of SIFT features, extracted from the
training set.
4. Compute the histogram of features.
5. Train the KNearest classifier with the features (samples) and their
corresponding class names (responses).
6. Get the path of image in the test set.
7. Implement step 2 to step 6 for the image in the test set.
8. Now give the Test feature vector and the K value (Number of neighbors.
to be considered for classification) to the trained classifier (KNearest).
9. Get the prediction.
10. Print the prediction on to the image in the test data set.

Approach (KNN):
The code was written in two parts both for KNN : 

Finding Features(Features.py) 

Classifier(class.py)---Training & Testing 

Both the files were attached to this link.

Steps followed in finding the features are :--( Features.py)

1.Importing the necessary libraries.

2.Fetching path of training dataset.

3.Get the training classes names and store them in a list. 

4.Get all the path to the images and save them in a list (image_paths) and
the corresponding label in another list (image_classes).
5. Create feature extraction and key point detector objects.
6. Stack all the descriptors vertically in a numpy array i.e. convert a list into
a vertical numpy array.
7. Perform the K-means clustering over the descriptors.
8. Calculate the histogram of features.
9. Now give more weightage to the clusters which occur more frequently in
the data set and scale the visual words (This is used to increase the
efficiency of classifier).
10. Now save the entire copy of visual vocabulary, feature (samples),
Image_classes (responses) on to the disk. (This will reduce the complexity
and confusion).

Features.py:

#!/usr/bin/env python

import argparse as ap
# Importing library that supports user friendly commandline interfaces
import cv2
# Importing the opencv library
import imutils
# Importing the library that supports basic image processing functions
import numpy as np
# Importing the array operations library for python
import os
# Importing the library which supports standard systems commands
from scipy.cluster.vq import *
# Importing the library which classifies set of observations into clusters
from sklearn.preprocessing import StandardScaler
# Importing the library that supports centering and scaling vectors


# Get the path of the training set
parser = ap.ArgumentParser()
parser.add_argument("-t", "--trainingSet", help="Path to Training Set", required="True")
args = vars(parser.parse_args())

# Get the training classes names and store them in a list
train_path = args["trainingSet"]
training_names = os.listdir(train_path)  # Listing the train_path directory

# Get all the path to the images and save them in a list
# image_paths and the corresponding label in image_paths
image_paths = []  # Inilialising the list
image_classes = []  # Inilialising the list
class_id = 0
for training_name in training_names:  # Iterate over the training_names list
    dir = os.path.join(train_path, training_name)
    class_path = imutils.imlist(dir)
    image_paths+=class_path
    image_classes+=[class_id]*len(class_path)
    class_id+=1

# Create feature extraction and keypoint detector objects
fea_det = cv2.FeatureDetector_create("SIFT")
des_ext = cv2.DescriptorExtractor_create("SIFT")

# List where all the descriptors are stored
des_list = []
# Reading the image and calculating the features and corresponding descriptors
for image_path in image_paths:
    im = cv2.imread(image_path)
    kpts = fea_det.detect(im)  # Computing the keypoints
    kpts, des = des_ext.compute(im, kpts)  # Computing the key points and the descriptors
    des_list.append((image_path, des))  # Appending all the descriptors into the single list

# Stack all the descriptors vertically in a numpy array
descriptors = des_list[0][1]
for image_path, descriptor in des_list[1:]:
    descriptors = np.vstack((descriptors, descriptor))  # Stacking the descriptors

# Perform k-means clustering
k = 500  # Number of clusters
voc, variance = kmeans(descriptors, k, 1)  # Perform Kmeans with default values

# Calculate the histogram of features
im_features = np.zeros((len(image_paths), k), "float32")
for i in xrange(len(image_paths)):
    words, distance = vq(des_list[i][1],voc)
    for w in words:
        im_features[i][w] += 1

# Perform Tf-Idf vectorization
nbr_occurences = np.sum( (im_features > 0) * 1, axis = 0)
# Calculating the number of occurrences
idf = np.array(np.log((1.0*len(image_paths)+1) / (1.0*nbr_occurences + 1)), 'float32')
# Giving weight to one that occurs more frequently

# Scaling the words
stdSlr = StandardScaler().fit(im_features)
im_features = stdSlr.transform(im_features)  # Scaling the visual words for better Prediction

# Saving the contents into a file
np.savetxt("samples.data",im_features)
np.savetxt("responses.data",np.array(image_classes))
np.save("training_names.data",training_names)
np.save("stdSlr.data",stdSlr)
np.save("voc.data",voc)

class.py: 

#!/usr/local/bin/python2.7

import argparse as ap
# Importing library that supports user friendly commandline interfaces
import cv2
# Importing the opencv library
import imutils
# Importing the library that supports basic image processing functions
import numpy as np
# Importing the array operations library for python
import os
# Importing the library which supports standard systems commands
from scipy.cluster.vq import *
# Importing the library which classifies set of observations into clusters

# Load the classifier, class names, scaler, number of clusters and vocabulary
samples = np.loadtxt('samples.data',np.float32)
responses = np.loadtxt('responses.data',np.float32)
classes_names = np.load('training_names.data.npy')
voc = np.load('voc.data.npy')
k = 500  # Loading the number of cluster

# Training the Knearest classifier with the test descriptors
clf = cv2.KNearest()
clf.train(samples,responses)  # Train model using the training samples and corresponding responses

# Get the path of the testing set
parser = ap.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument("-t", "--testingSet", help="Path to testing Set")
group.add_argument("-i", "--image", help="Path to image")
parser.add_argument('-v',"--visualize", action='store_true')
args = vars(parser.parse_args())

# Get the path of the testing image(s) and store them in a list
image_paths = []
if args["testingSet"]:
    test_path = args["testingSet"]
    try:
        testing_names = os.listdir(test_path)
    except OSError:
        print "No such directory {}\nCheck if the file exists".format(test_path)
        exit()
    for testing_name in testing_names:
        dir = os.path.join(test_path, testing_name)
        class_path = imutils.imlist(dir)
        image_paths+=class_path
else:
    image_paths = [args["image"]]
   
# Create feature extraction and keypoint detector objects
fea_det = cv2.FeatureDetector_create("SIFT")
des_ext = cv2.DescriptorExtractor_create("SIFT")

# List where all the descriptors are stored
des_list = []
for image_path in image_paths:
    im = cv2.imread(image_path)
    if im == None:
        print "No such file {}\nCheck if the file exists".format(image_path)
        exit()
    kpts = fea_det.detect(im)  # Computing the key points of test image
    kpts, des = des_ext.compute(im, kpts)  # Computing the descriptors of the test image
    des_list.append((image_path, des))   # Appending the descriptors to a single list
   
# Stack all the descriptors vertically in a numpy array
descriptors = des_list[0][1]
for image_path, descriptor in des_list[0:]:
    descriptors = np.vstack((descriptors, descriptor))   # Stacking the descriptors in to a numpy array

# Computing the histogram of features
test_features = np.zeros((len(image_paths), k), "float32")
for i in xrange(len(image_paths)):
    words, distance = vq(des_list[i][1],voc)
    for w in words:
        test_features[i][w] += 1  # Calculating the histogram of features

# Perform Tf-Idf vectorization
nbr_occurences = np.sum( (test_features > 0) * 1, axis = 0)  # Getting the number of occurrences of each word
idf = np.array(np.log((1.0*len(image_paths)+1) / (1.0*nbr_occurences + 1)), 'float32')
# Assigning weight to one that is occurring more frequently

# Perform the predictions
retval, results, neigh_resp, dists = clf.find_nearest(test_features,k=17)
# Finding the 17 nearest neighbours of the test image descriptor

if results[0][0] == 0:  # results[0][0] will have the predicted class
    prediction = "Horse"
else:
    prediction = "Bike"

# Visualize the results, if "visualize" flag set to true by the user
if args["visualize"]:
    image = cv2.imread(image_path)
    cv2.namedWindow("Image",cv2.WINDOW_AUTOSIZE)  # Creating a named window
    pt = (180,3*image.shape[0]//4)  # Framing the size of font on the image
    cv2.putText(image, prediction, pt ,cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,2, [0, 0, 255],2)  # Placing text on the image
    cv2.imshow("Image",image)  # Displaying the image
    cv2.waitKey()  # Wait for the keystroke of the user


I have attached the link to the code(in github):Feel free to use it for your purpose.
https://github.com/manojkiraneda/Machine-Perception/tree/master/Image%20Classification(ML)