Computer Vision with Mathematica

Written by on September 26, 2016 in Machine Learning, Mathematica, Programming with 0 Comments

Over the past several weeks, I have been discussing many interesting features of Mathematica. As a continuation, today, I would like to show some cool functionality in the domain of computer vision and machine learning.

The function ImageIdentify[] tries to identify the object in the given image. According to the documentation, Mathematica knows about more than 10,000 objects. To test this feature, I downloaded a few images from the internet. You can see the interaction below.

The first example involves a truck. Mathematica was able to correctly identify the truck.

A Truck

A Truck

For the second example, I downloaded the image of a fountain pen. Although we humans can correctly identify this object as a pen, I thought Mathematica might have some trouble because, the cover has been removed and kept by the side of the pen, making it seem like two different objects. But Mathematica had no problem!

A Fountain Pen

A Fountain Pen

The third example is a stapler. Mathematica got it correct this time too.

A Stapler

A Stapler

Mathematica could not identify the object in the next three examples. I thought it was odd because USB Pendrives are so common these days.

USB Drive

USB Drive

 

Another USB Drive

Another USB Drive

That is when I thought maybe I should teach Mathematica to correctly classify pendrives. The Classify[] function allows us to give classification examples to Mathematica, from which it will generate a suitable Classifier. This latter function can then be used to classify a new object based on what it has been taught.

The Classify[] function takes an array of <example, class> pairs, and returns a classifier. You can later apply the classifier on a new example to get the appropriate classification. The figure below shows my examples, and how it eventually correctly classified the new pen drive (not seen before).

Teaching the Classifier

Teaching the Classifier

That is good! Of course, things are not that easy in the real world. Many, many more examples have to be supplied.

For the last experiment, instead of working with images that contain single objects, I wanted to experiment with an image having multiple objects. I used my iPhone and photographed three objects, namely, scissors, fountain pen, and a paper cutting knife arranged side by side on my desk. I then asked Mathematica to identify all the objects in it. See what happened:

Multiple Objects in Image

Multiple Objects in Image

It appears that the fountain pen has been classfied as a sort of knife. I guess the similarity is there, but given that in our earlier example, it was able to correctly identify a fountain pen, I was a bit surprised. It could be due to the orientation of the pen, or lack of sufficient detail, I assume. Perhaps Mathematica  folks should consider doing more training on the fountain pen data set.

Next, I decided to take a look at the image processing side of Mathematica. The cool thing is that Mathematica has hundreds of functions in each of the domains it supports, and so if you have the patience and the inclination, you could go as far as you want (of course, I have my own wish-list for Mathematica, but that is a different topic).

Given the last image with multiple objects, let us ask Mathematica to detect the edges. We can use the EdgeDetect[] function. This function finds edges in an image, returning a binary image as result.

Edge Detection

Edge Detection

I then used the ImageKeyPoints[] function to detect the key features in the binary image and highlight their coordinates.

Image Key Points

Image Key Points

Interesting. So, we could go on like this, but let me stop here for now.

More interesting stuff in the next post!

Tags: , ,

Subscribe

If you enjoyed this article, subscribe now to receive more just like it.

Subscribe via RSS Feed

Leave a Reply

Your email address will not be published.

Top