{"id":1444,"date":"2019-02-10T14:04:34","date_gmt":"2019-02-10T08:34:34","guid":{"rendered":"https:\/\/www.rangakrish.com\/?p=1444"},"modified":"2019-02-10T14:36:00","modified_gmt":"2019-02-10T09:06:00","slug":"coreference-resolution-in-stanford-corenlp","status":"publish","type":"post","link":"https:\/\/www.rangakrish.com\/index.php\/2019\/02\/10\/coreference-resolution-in-stanford-corenlp\/","title":{"rendered":"Coreference Resolution in Stanford CoreNLP"},"content":{"rendered":"

In the last article<\/strong><\/em><\/a>, I showed how we can use the neuralcoref<\/strong><\/em><\/a>\u00a0library along with spaCy<\/strong><\/em><\/a>\u00a0to do coreference resolution (examples involved anaphoric<\/strong><\/em> references). In today’s article, I want to try the same (well, almost) examples in Stanford CoreNLP<\/strong><\/em><\/a> engine and see how they compare.<\/p>\n

Since CoreNLP<\/strong><\/em> is a Java<\/strong><\/em> implementation, I chose to write the test program in Java<\/strong><\/em>. Here is the program<\/strong><\/em><\/a>\u00a0\u00a0<\/span>along with the examples.<\/p>\n

\"CoreNLP<\/a>
CoreNLP Corereference Example<\/strong><\/figcaption><\/figure>\n

You can download the CoreNLP<\/strong><\/em> library from here.<\/strong><\/em><\/a><\/p>\n

Let us start with the first sentence:<\/p>\n

“My sister has a dog and she loves him.”<\/span><\/p><\/blockquote>\n

When I run it through the program, this is the output:<\/p>\n

\"Example-1<\/a>
Example-1 Output<\/strong><\/figcaption><\/figure>\n

Strange. It shows that the program has identified “My sister”<\/strong><\/em> and “she”<\/strong><\/em> as pointing to the same entity, but there is no mention of “dog”<\/strong><\/em> and “him”<\/strong><\/em>. In contrast, neuralcoref<\/strong><\/em>\u00a0identified both the “mentions”<\/strong><\/em> correctly.<\/p>\n

Could it be because this is a compound sentence? Let us convert this into two simple sentences:<\/p>\n

“My sister has a dog. She loves him.”<\/span><\/p><\/blockquote>\n

Here is the output in this case:<\/p>\n

\"Example-2<\/a>
Example-2 Output<\/strong><\/figcaption><\/figure>\n

OK! It is interesting that the program has identified both the “mentions”<\/strong><\/em> this time.<\/p>\n

I am using the remaining sentences as they were in neuralcoref<\/strong><\/em>\u00a0example. Here is the third sentence:<\/p>\n

“My sister has a dog and she loves him. He is cute.”<\/span><\/p><\/blockquote>\n

The output is:<\/p>\n

\"Example-3<\/a>
Example-3 Output<\/strong><\/figcaption><\/figure>\n

Although the system has identified “him”<\/strong><\/em> and “he”<\/strong><\/em> as belonging together, it has omitted “dog”<\/strong><\/em> from the reference. Bug! Here also, neuralcoref<\/strong><\/em>\u00a0fared better.<\/p>\n

The next example is:<\/p>\n

“My sister has a dog and she loves her.”<\/span><\/p><\/blockquote>\n

The output is:<\/p>\n

\"Example-4<\/a>
Example-4 Output<\/strong><\/figcaption><\/figure>\n

The behavior is the same as with neuralcoref<\/strong><\/em>. “dog”<\/strong><\/em> is not paired with “her”<\/strong><\/em>.<\/p>\n

Consider the following statement that uses different genders and hence should be easier to resolve:<\/p>\n

“My brother has a dog and he loves her.”<\/span><\/p><\/blockquote>\n

This is what we get:<\/p>\n

\"Example-5<\/a>
Example-5 Output<\/strong><\/figcaption><\/figure>\n

That is nice! “dog”<\/strong><\/em> and “her”<\/strong><\/em> are correctly paired. neuralcoref<\/strong><\/em>\u00a0did not handle this case correctly.<\/p>\n

Let us look at the next sentence:<\/p>\n

“Mary and Julie are sisters. They love chocolates.”<\/span><\/p><\/blockquote>\n

neuralcoref<\/strong><\/em>\u00a0handled this case correctly. Strangely, when given this input, CoreNLP<\/strong><\/em> does not identify any “mentions”<\/strong><\/em> at all! I confess I am disappointed.<\/p>\n

Let us try the next one:<\/p>\n

“John and Mary are neighbours. She admires him because he works hard.”<\/span><\/p><\/blockquote>\n

This looks more complex than the earlier one. How does CoreNLP<\/strong><\/em> fare?<\/p>\n

\"Example-7<\/a>
Example-7 Output<\/strong><\/figcaption><\/figure>\n

OK, this one is handled correctly. That is a relief.<\/p>\n

The next one is tougher:<\/p>\n

“X and Y are neighbours. She admires him because he works hard.”<\/span><\/p><\/blockquote>\n

Here is the program’s output:<\/p>\n

\"Example-8<\/a>
Example-8 Output<\/strong><\/figcaption><\/figure>\n

Not surprised with the behavior. neuralcoref<\/strong><\/em>\u00a0wasn’t any better.<\/p>\n

Here is the last example:<\/p>\n

“The dog chased the cat. But it escaped.”<\/span><\/p><\/blockquote>\n

This is the corresponding output:<\/p>\n

\"Example-9<\/a>
Example-9 Output<\/strong><\/figcaption><\/figure>\n

Not correct. Again, the behavior is identical to neuralcoref<\/strong><\/em>.<\/p>\n

Let me summarize and show the performance of CoreNLP<\/strong><\/em> and neuralcoref<\/strong><\/em> with respect to the examples:<\/p>\n

\"Comparison<\/a>
Comparison Table<\/strong><\/figcaption><\/figure>\n

You can see that among the sentences tested in both the systems, neuralcoref<\/strong><\/em>\u00a0got 4 out of 8 correct, whereas CoreNLP<\/strong><\/em> got just 2 out of 8 correct. To clarify, I have marked the output as “Wrong”<\/strong><\/em> if the system did not FULLY<\/strong><\/em> identify the mentions<\/strong><\/em>.<\/p>\n

Overall, in the case of anaphora resolution, both the popular libraries have fared poorly in my opinion. I expected more out of CoreNLP<\/strong><\/em>, so that was a bigger disappointment!<\/p>\n

Looks like there is a lot more work to do in the area of coreference resolution!<\/p>\n

You can download my Java<\/strong><\/em> program from here.<\/strong><\/em><\/a><\/p>\n

Have a nice weekend!<\/p>\n","protected":false},"excerpt":{"rendered":"

In the last article, I showed how we can use the neuralcoref\u00a0library along with spaCy\u00a0to do coreference resolution (examples involved anaphoric references). In today’s article, I want to try the same (well, almost) examples in Stanford CoreNLP engine and see how they compare. Since CoreNLP is a Java implementation, I chose to write the test […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[107,17],"tags":[186,174,185,190],"class_list":["post-1444","post","type-post","status-publish","format-standard","hentry","category-natural-language-processing","category-programming","tag-coreference-resolution","tag-natural-language-processing","tag-neuralcoref","tag-stanford-corenlp"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9OLnF-ni","jetpack-related-posts":[{"id":1427,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/02\/03\/coreference-resolution-using-spacy\/","url_meta":{"origin":1444,"position":0},"title":"Coreference Resolution Using spaCy","author":"admin","date":"February 3, 2019","format":false,"excerpt":"According to Stanford NLP Group, \"Coreference resolution is the task of finding all expressions that refer to the same entity in a text\".\u00a0 You can also read this Wikipedia page. For example, in the sentence \"Tom dropped the glass jar by accident and broke it\", what does \"it\" refer to?\u2026","rel":"","context":"In "Machine Learning"","block_context":{"text":"Machine Learning","link":"https:\/\/www.rangakrish.com\/index.php\/category\/machine-learning\/"},"img":{"alt_text":"Loading the Coreference Model","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/02\/Loading-Model.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":1368,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/01\/08\/parsing-text-with-apache-opennlp\/","url_meta":{"origin":1444,"position":1},"title":"Parsing Text with Apache OpenNLP","author":"admin","date":"January 8, 2019","format":false,"excerpt":"In my earlier posts I have written about parsing text using spaCy\u00a0and MeaningCloud's parsing API. For today's article, I decided to take a look at OpenNLP, an open-source ML-based Java toolkit for parsing natural language text. OpenNLP is a fairly mature library and has been around since 2004 (source: Wikipedia).\u2026","rel":"","context":"In "Natural Language Processing"","block_context":{"text":"Natural Language Processing","link":"https:\/\/www.rangakrish.com\/index.php\/category\/natural-language-processing\/"},"img":{"alt_text":"Parse Tree","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/01\/Tree3.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":912,"url":"https:\/\/www.rangakrish.com\/index.php\/2018\/04\/22\/question-answering-using-dependency-trees\/","url_meta":{"origin":1444,"position":2},"title":"Question Answering\u00a0Using Dependency Trees","author":"admin","date":"April 22, 2018","format":false,"excerpt":"A few weeks ago I had written about my brief experiment with Mathematica's new feature, which provides answers to questions based on given text. After that post, I spent some time thinking about how to implement something similar. In today's post, I want to show you what I have been\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"Dependency Tree","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2018\/04\/Deptree-example.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1386,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/01\/13\/chunking-in-opennlp\/","url_meta":{"origin":1444,"position":3},"title":"Chunking in OpenNLP","author":"admin","date":"January 13, 2019","format":false,"excerpt":"In my previous post, I showed how to parse sentences using OpenNLP. Another useful feature supported by OpenNLP is \"chunking\u201d. That is the subject of today\u2019s article. Chunking stands between part-of-speech tagging and full parse in terms of the information it captures. POS tagging assigns part of speech to individual\u2026","rel":"","context":"In "Natural Language Processing"","block_context":{"text":"Natural Language Processing","link":"https:\/\/www.rangakrish.com\/index.php\/category\/natural-language-processing\/"},"img":{"alt_text":"Printing Chunked Tags with Probability","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/01\/Example3-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/01\/Example3-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/01\/Example3-1.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":2506,"url":"https:\/\/www.rangakrish.com\/index.php\/2021\/08\/21\/loop-command-in-ring-programming-language\/","url_meta":{"origin":1444,"position":4},"title":"“Loop” Command in Ring Programming Language","author":"admin","date":"August 21, 2021","format":false,"excerpt":"In my last article, I had written about how easy it is to execute Ring code from within C\/C++. Today's article takes a look at the \"Loop\" command. This command is very similar to \"continue\" in many languages such as C++, Java, Python. etc. It is used to skip the\u2026","rel":"","context":"In "Programming"","block_context":{"text":"Programming","link":"https:\/\/www.rangakrish.com\/index.php\/category\/programming\/"},"img":{"alt_text":"\"Continue\" in Java","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2021\/08\/javacode-300x183.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":1475,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/02\/24\/emotion-detection-using-paralleldots-api\/","url_meta":{"origin":1444,"position":5},"title":"Emotion Detection using ParallelDots API","author":"admin","date":"February 24, 2019","format":false,"excerpt":"Last week, I showed how we can use IBM Natural Language Understanding API to identify emotions from given text. Today, I would like to run through the same examples, but using ParallelDots API service. There are wrappers\u00a0in Java, Python, Ruby, C#, and PHP for accessing the REST service. However, I\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"The Code","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/02\/Code-1.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/02\/Code-1.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/02\/Code-1.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1444","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/comments?post=1444"}],"version-history":[{"count":0,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1444\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/media?parent=1444"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/categories?post=1444"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/tags?post=1444"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}