Over the past several weeks, I have been discussing many interesting features of Mathematica. As a continuation, today, I would like to show some cool functionality in the domain of computer vision and machine learning.
The function ImageIdentify[] tries to identify the object in the given image. According to the documentation, Mathematica knows about more than 10,000 objects. To test this feature, I downloaded a few images from the internet. You can see the interaction below.
The first example involves a truck. Mathematica was able to correctly identify the truck.
A Truck
For the second example, I downloaded the image of a fountain pen. Although we humans can correctly identify this object as a pen, I thought Mathematica might have some trouble because, the cover has been removed and kept by the side of the pen, making it seem like two different objects. But Mathematica had no problem!
A Fountain Pen
The third example is a stapler. Mathematica got it correct this time too.
A Stapler
Mathematica could not identify the object in the next three examples. I thought it was odd because USB Pendrives are so common these days.
USB Drive
Another USB Drive
That is when I thought maybe I should teach Mathematica to correctly classify pendrives. The Classify[] function allows us to give classification examples to Mathematica, from which it will generate a suitable Classifier. This latter function can then be used to classify a new object based on what it has been taught.
The Classify[] function takes an array of <example, class> pairs, and returns a classifier. You can later apply the classifier on a new example to get the appropriate classification. The figure below shows my examples, and how it eventually correctly classified the new pen drive (not seen before).
Teaching the Classifier
That is good! Of course, things are not that easy in the real world. Many, many more examples have to be supplied.
For the last experiment, instead of working with images that contain single objects, I wanted to experiment with an image having multiple objects. I used my iPhone and photographed three objects, namely, scissors, fountain pen, and a paper cutting knife arranged side by side on my desk. I then asked Mathematica to identify all the objects in it. See what happened:
Multiple Objects in Image
It appears that the fountain pen has been classfied as a sort of knife. I guess the similarity is there, but given that in our earlier example, it was able to correctly identify a fountain pen, I was a bit surprised. It could be due to the orientation of the pen, or lack of sufficient detail, I assume. Perhaps Mathematica folks should consider doing more training on the fountain pen data set.
Next, I decided to take a look at the image processing side of Mathematica. The cool thing is that Mathematica has hundreds of functions in each of the domains it supports, and so if you have the patience and the inclination, you could go as far as you want (of course, I have my own wish-list for Mathematica, but that is a different topic).
Given the last image with multiple objects, let us ask Mathematica to detect the edges. We can use the EdgeDetect[] function. This function finds edges in an image, returning a binary image as result.
Edge Detection
I then used the ImageKeyPoints[] function to detect the key features in the binary image and highlight their coordinates.
Image Key Points
Interesting. So, we could go on like this, but let me stop here for now.
More interesting stuff in the next post!
Recent Comments