About 10 days ago, I received an update for Mathematica. The latest version is 11.3.0. As usual, I looked through the list of new features in this release. There are several new features, but one of them attracted my attention immediately: There is a new function FindTextualAnswer that, given a piece of text and a question, can identify the part of the text that answers the question.
I explored this function briefly and would like to share my findings in this post. First, I must point out that the documentation says this function is Experimental, implying that there could be changes in future releases.
Let us start with a simple example:
Example 1
The function takes at least 2 arguments. The first is a piece of text and the second is a question (we can include multiple questions as a list) based on the text. The function returns a text fragment from the first argument, which supposedly answers the question.
In the above example, When asked the question “Who is Vicky?”, the function correctly says “my pet dog”. Nice.
Note that there is no paraphrasing involved here; the answer is taken directly from the given text. Even that is a non-trivial task.
Let us try two more examples:
Some More Examples
Again, fairly interesting. The function seems capable of providing direct answers to questions based on the text.
We can ask for possible multiple answers along with their probability.
Answers with Probability
In this example, I asked for two possible answers. The result contains each answer along with the respective probability.
Let us see if the function can handle tricky questions.
A Tricky Example
Gotcha! Since there is no mention of Mary anywhere in the text, I was expecting something like Don’t know, but the function’s response really stumped me! How did it map Mary to Vicky?
Let us try one more:
Another Tricky Question
This is even more weird. There is no mention of John, or anyone being happy in the given text, but the function seems to make incorrect inferences.
The documentation says that the function uses statistical methods to find answers. There is also a mention somewhere that it can understand semantics concepts to an extent, but that it cannot apply logic.
Well, I think this is a good beginning, but there is significant scope for improving the functionality.
Hope future releases take care of that.
Why didn’t you ask for the probabilities of the answers to the trick questions? If the system generates a low probability answer, you can figre that the answer is wrong or irrelevant.