{"id":1640,"date":"2019-07-11T14:30:49","date_gmt":"2019-07-11T09:00:49","guid":{"rendered":"https:\/\/www.rangakrish.com\/?p=1640"},"modified":"2019-07-11T14:30:49","modified_gmt":"2019-07-11T09:00:49","slug":"information-extraction-using-spacys-pattern-matcher","status":"publish","type":"post","link":"https:\/\/www.rangakrish.com\/index.php\/2019\/07\/11\/information-extraction-using-spacys-pattern-matcher\/","title":{"rendered":"Information Extraction Using spaCy\u2019s Pattern Matcher"},"content":{"rendered":"

In the previous article<\/strong><\/em><\/a>, I explored the Deep Categorization<\/strong><\/em> capabilities of MeaningCloud<\/strong><\/em><\/a>. We saw how a powerful rule-based pattern matching language allowed us to map fragments of unstructured text to custom categories.<\/p>\n

In today\u2019s post, I want to go through spaCy\u2019s<\/strong><\/em><\/a>\u00a0pattern matching capabilities. The version I am using is 2.0.13. Some newer features are available in version 2.1, but I could not explore those.<\/p>\n

There are essentially two matchers: Token Matcher<\/strong><\/em> and Phrase Matcher<\/strong><\/em>. The Token Matcher<\/strong><\/em> is well suited to match token sequences, whereas the Phrase Matcher<\/strong><\/em> is optimised for matching\u00a0 <\/span>multi-word text, such as names of people and places.<\/p>\n

I am going to use the same example as last time – patient case description, with slight enhancement.\u00a0<\/span><\/p>\n

Instead of using the Matcher<\/strong><\/em> class directly, I wrote a simple wrapper class named HomeoProcessor<\/strong><\/em> to provide a convenient abstraction. See the following:<\/p>\n

\"HomeoProcessor<\/a>
HomeoProcessor Class<\/strong><\/figcaption><\/figure>\n

There is a method to add patterns and another to process the given text.<\/p>\n

Next step is to define the various patterns we want to handle. I have defined simple patterns for detecting patient\u2019s name, gender, age,<\/strong><\/em> and ailment<\/strong><\/em>.<\/p>\n

\"The<\/a>
The Patterns<\/strong><\/figcaption><\/figure>\n

I guess the pattern definitions are self explanatory. Here is how we instantiate the class and use it:<\/p>\n

\"Using<\/a>
Using the HomeoProcessor Class<\/strong><\/figcaption><\/figure>\n

We create an instance of the class first and bind the different patterns. We then invoke the process<\/strong><\/em> method, passing the case text. The function returns a dictionary of matched pattern labels along with the corresponding text.<\/p>\n

Here is the output for the two example cases:<\/p>\n

\"Output\"<\/a>
Output<\/strong><\/figcaption><\/figure>\n

Seems to work as expected. spaCy<\/strong><\/em> allows us to attach custom handlers<\/em><\/strong> when matching patterns. For my example, I did not feel the need for these. Nor did I feel the need for Phrase Matcher<\/strong> <\/em>in this case.<\/p>\n

Although I could get the Matcher<\/strong><\/em> to do what I wanted, I feel the usability and design could be improved. It would be nice to be able to seamlessly combine token and phrase patterns using a unified syntax (instead of using separate Token<\/strong><\/em> and Phrase<\/strong><\/em> matchers) to build complex pattern expressions. Maybe also introduce “AND”,<\/strong><\/em>\u00a0“OR”<\/strong><\/em>\u00a0and “NOT”<\/strong><\/em> operators? These are not available in ver 2.1 as well. Something to think about.<\/p>\n

Limitations apart, the nice thing is that this is available for free in a high-performance NLP engine that is continuously getting better.<\/p>\n

The Python<\/strong><\/em> program can be downloaded from here<\/strong><\/em><\/a>.<\/p>\n

Have a great day!<\/p>\n","protected":false},"excerpt":{"rendered":"

In the previous article, I explored the Deep Categorization capabilities of MeaningCloud. We saw how a powerful rule-based pattern matching language allowed us to map fragments of unstructured text to custom categories. In today\u2019s post, I want to go through spaCy\u2019s\u00a0pattern matching capabilities. The version I am using is 2.0.13. Some newer features are available […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[22,107,17,103],"tags":[212,62,187],"class_list":["post-1640","post","type-post","status-publish","format-standard","hentry","category-homeopathy","category-natural-language-processing","category-programming","category-python","tag-information-extraction","tag-pattern-matching","tag-spacy"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9OLnF-qs","jetpack-related-posts":[{"id":1650,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/07\/21\/custom-text-analysis-using-textrazors-prolog-engine\/","url_meta":{"origin":1640,"position":0},"title":"Custom Text Analysis Using TextRazor\u2019s Prolog Engine","author":"admin","date":"July 21, 2019","format":false,"excerpt":"This is the third part in the series on information extraction from unstructured text. In the first part, we saw how MeaningCloud\u00a0allows us to specify complex rules to identify custom categories through their Deep Categorization API. The second part\u00a0covered spaCy\u2019s\u00a0pattern matcher. Today, I would like to discuss how we can\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"Result of Processed Case","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/07\/Output-1.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/07\/Output-1.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/07\/Output-1.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/07\/Output-1.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":256,"url":"https:\/\/www.rangakrish.com\/index.php\/2016\/06\/09\/pattern-matching-with-optima-lisp-library\/","url_meta":{"origin":1640,"position":1},"title":"Pattern Matching with Optima Lisp Library","author":"admin","date":"June 9, 2016","format":false,"excerpt":"In the previous two posts, we looked at external tools and libraries that can be used along with Opusmodus for algorithmic composition. In this post, I want to introduce\u00a0another interesting Lisp library called Optima. Optima is a powerful library for pattern matching. Often, when we talk of pattern matching, the\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2084,"url":"https:\/\/www.rangakrish.com\/index.php\/2020\/08\/16\/pattern-matching-comparing-elixir-and-mathematica\/","url_meta":{"origin":1640,"position":2},"title":"Pattern Matching: Comparing Elixir and Mathematica","author":"admin","date":"August 16, 2020","format":false,"excerpt":"One of the things I like about Elixir\u00a0is its support for patterns at the core language level, not through library functions as in most other languages. This contributes to writing cleaner code, in my opinion. \u00a0 Another environment that I am familiar with, namely Mathematica, boasts of (arguably) the most\u2026","rel":"","context":"In "Elixir"","block_context":{"text":"Elixir","link":"https:\/\/www.rangakrish.com\/index.php\/category\/elixir\/"},"img":{"alt_text":"Symbolic Expressions","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/08\/pattern-mm.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/08\/pattern-mm.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/08\/pattern-mm.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":268,"url":"https:\/\/www.rangakrish.com\/index.php\/2016\/06\/29\/pattern-matching-with-optima-lisp-library-part-3\/","url_meta":{"origin":1640,"position":3},"title":"Pattern Matching with Optima Lisp Library – Part 3","author":"admin","date":"June 29, 2016","format":false,"excerpt":"In the previous two posts on this topic, I explained some of the basic pattern matching facilities of Optima\u00a0library. There are many\u00a0more advanced features in the library and I will try to discuss them in future posts. In today\u2019s post, I will outline a straightforward application of the library for\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"Optima Example","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/06\/Optima-Example-1024x622.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/06\/Optima-Example-1024x622.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/06\/Optima-Example-1024x622.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/06\/Optima-Example-1024x622.png?resize=700%2C400 2x"},"classes":[]},{"id":263,"url":"https:\/\/www.rangakrish.com\/index.php\/2016\/06\/17\/pattern-matching-with-optima-lisp-library-part-2\/","url_meta":{"origin":1640,"position":4},"title":"Pattern Matching with Optima Lisp Library – Part 2","author":"admin","date":"June 17, 2016","format":false,"excerpt":"Let us continue where we left off last time. List Patterns If the incoming argument is a list, then we can use two types of list patterns to match the list elements, namely, list and list*. (match '(a b c)\u00a0 ((list 'a 'b X) X)) => c This list pattern\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1817,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/12\/08\/using-definite-clause-grammars-dcg-for-information-extraction\/","url_meta":{"origin":1640,"position":5},"title":"Using Definite Clause Grammars (DCG) for Information Extraction","author":"admin","date":"December 8, 2019","format":false,"excerpt":"In the previous article, I showed how we can use ATNs for extracting key information from natural language text. I also pointed out in that article that Definite Clause Grammars (DCG) are a more compact formalism for doing this. That will be the focus of today's article. For a nice\u2026","rel":"","context":"In "Natural Language Processing"","block_context":{"text":"Natural Language Processing","link":"https:\/\/www.rangakrish.com\/index.php\/category\/natural-language-processing\/"},"img":{"alt_text":"Processing the Text","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/12\/Processing-file-code.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/12\/Processing-file-code.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/12\/Processing-file-code.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1640","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/comments?post=1640"}],"version-history":[{"count":0,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1640\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/media?parent=1640"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/categories?post=1640"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/tags?post=1640"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}