{"id":1613,"date":"2019-06-24T16:55:01","date_gmt":"2019-06-24T11:25:01","guid":{"rendered":"https:\/\/www.rangakrish.com\/?p=1613"},"modified":"2019-06-26T05:09:23","modified_gmt":"2019-06-25T23:39:23","slug":"text-analysis-using-meaningclouds-deep-categorization-api","status":"publish","type":"post","link":"https:\/\/www.rangakrish.com\/index.php\/2019\/06\/24\/text-analysis-using-meaningclouds-deep-categorization-api\/","title":{"rendered":"Text Analysis Using MeaningCloud\u2019s Deep Categorization API"},"content":{"rendered":"

MeaningCloud<\/strong><\/em><\/a>\u00a0announced their premium offering \u201cDeep Categorization\u201d<\/strong><\/em> service quite recently. You can read about it in this nice blog<\/strong><\/em><\/a>\u00a0by Blanca Galego.<\/p>\n

Compared to the canonical classification models that rely on machine learning using large data sets, MeaningCloud\u2019s<\/strong><\/em> deep categorization models use hand-crafted rules that take advantage of \u201cmorphosyntactic, semantic and contextual information\u201d<\/strong><\/em> to classify a piece of text.<\/p>\n

The company has several pre-defined deep categorization models, such as Intention Analysis<\/strong><\/em>, Voice-of-Customer Banking<\/strong><\/em>, Voice-of-Employee Exit Interview<\/strong><\/em>, and so on. Take a look at this<\/strong><\/em><\/a>\u00a0to learn more.<\/p>\n

This feature, as well as how to build custom models, is explained in detail in the documentation<\/strong><\/em><\/a>\u00a0section.<\/p>\n

After reading the extensive documentation, I was keen to get my hands dirty by trying it on some example. I am working on a homeopathy project that involves text analysis, so I decided to apply it in that context.<\/p>\n

Here is the problem: Given a homeopathy case record (corresponding to a patient), can we automatically identify the nature of ailment? Prescribing the correct remedy for an ailment is a very complex process in homeopathy, so if some kind of automation can facilitate this process, I would consider it to be a significant achievement.<\/p>\n

For this toy experiment, of course, I have taken a simple example with just two categories:<\/p>\n

acute-disease>common-cold<\/span><\/p>\n

acute-disease>diarrhoea<\/span><\/p>\n

Given a piece of text, we want the system to label it as one or both of the above categories.\u00a0<\/span><\/p>\n

First, consider the labels I have chosen for the categories. In homeopathy, common cold<\/strong><\/em> and diarrhoea<\/strong><\/em> are examples of acute<\/strong><\/em><\/a> diseases. Diabetes<\/strong><\/em> is an example of chronic<\/strong><\/em><\/a> disease. When developing deep categorization models, it is useful to take advantage of hierarchies like this, wherever appropriate.\u00a0<\/span><\/p>\n

Now, take the case of common cold<\/strong><\/em>. A case record that documents this acute ailment of a patient will most likely have expressions like \u201ccold\u201d<\/strong><\/em>, \u201crunny nose\u201d<\/strong><\/em>, \u201csneezing\u201d<\/strong><\/em>, \u201cheadache\u201d<\/strong><\/em>, \u201cfever\u201d<\/strong><\/em>, etc. We can capture this knowledge in terms of\u00a0\u201crules\u201d<\/strong><\/em>\u00a0involving pattern expressions for categorizing given text into \u201cacute-disease>common-cold\u201d<\/strong><\/em> category. Here are my two rules:<\/p>\n

\"Rules<\/a>
Rules for Common Cold<\/strong><\/figcaption><\/figure>\n

The first rule \u201csymptom1\u201d<\/strong><\/em> uses a \u201cmacro\u201d<\/strong><\/em> called \u201ccold-symptoms\u201d<\/strong><\/em>. Macros are a nice way to capture patterns that are likely to be used across multiple rules. Instead of repeating the same pattern expression again and again, defining it as a macro makes it concise, more readable and maintainable.<\/p>\n

Here is the macro definition:<\/p>\n

\"Macro:<\/a>
Macro: cold-symptoms<\/strong><\/figcaption><\/figure>\n

The two rules \u201csymptom1\u201d and\u00a0\u201csymptom2\u201d\u00a0<\/strong><\/em>together define the category \u201cacute-disease>common-cold\u201d<\/strong><\/em>.<\/p>\n

Let us now look at the other category \u201cacute-disease>diarrhoea\u201d<\/strong><\/em>. See this figure:<\/p>\n

\"Rule<\/a>
Rule for Diarrhoea<\/strong><\/figcaption><\/figure>\n

This pattern expression is slightly more complex than the ones used for common cold<\/strong><\/em>. To whet your curiosity, the last part of the pattern [putrid|watery|undigested stools]~4<\/b> matches the following cases:<\/p>\n

\u201cputrid stools\u201d<\/span><\/p>\n

\u201cwatery stools\u201d<\/span><\/p>\n

\u201cstools containing undigested food\u201d<\/span><\/p>\n

That is, the two sets of words may appear in any order within a span of 4 words. For a detailed description of the pattern syntax, see the documentation<\/strong><\/em><\/a>.\u00a0<\/span><\/p>\n

The following image shows our two categories:<\/p>\n

\"The<\/a>
The Two Categories in Our Model<\/strong><\/figcaption><\/figure>\n

Once the model has been defined, we need to \u201cBuild\u201d<\/strong><\/em> the model by clicking the \u201cBuild\u201d<\/strong><\/em> button under \u201cActions\u201d<\/strong><\/em> group on the left side (see above). If there are no errors, the model is compiled and we get a confirmation message:<\/p>\n

\"Successful<\/a>
Successful Build<\/strong><\/figcaption><\/figure>\n

Once the model is ready, we can test it with our sample data.The nice thing is that we can do this\u00a0<\/span>through the Test Console<\/strong><\/em>; there is no need to write any program to make API<\/strong><\/em> calls for this purpose. Clicking on the \u201cTest\u201d<\/strong><\/em> button under \u201cActions\u201d<\/strong><\/em> brings up the Test Console<\/strong><\/em> with our model already selected!<\/p>\n

\"Common<\/a>
Common Cold Sample Text<\/strong><\/figcaption><\/figure>\n

When you click “Raw”<\/strong><\/em> button at the bottom left, we get the result:<\/p>\n

\"Analysis<\/a>
Analysis Result<\/strong><\/figcaption><\/figure>\n

You can see that the system has identified the key terms in the text and also computed overall relevance score of 40 (by the way, we can configure weights for various rules).<\/p>\n

Let us now look at an example of diarrhoea<\/strong><\/em>.<\/p>\n

\"Diarrhoea<\/a>
Diarrhoea Example<\/strong><\/figcaption><\/figure>\n

And here is the analysis result:<\/p>\n

\"Analysis<\/a>
Analysis Result<\/strong><\/figcaption><\/figure>\n

One thing worth pointing out is that the rules operate at “sentence”<\/strong><\/em> level (uses “Split Sentences”<\/strong><\/em> option). What this means is that the patterns are checked sentence by sentence. I feel this is an important option you have to consider while designing a model.<\/p>\n

Another useful capability is the support for exporting and\u00a0importing our models. This is needed if we are keeping track of different versions of our model.<\/p>\n

Needless to say, I am quite impressed with this new \u201cDeep Categorization\u201d<\/strong><\/em> feature since it opens up new techniques for identifying \u201chidden\u201d<\/strong><\/em> categories in the given text.<\/p>\n

One gentle warning may be in order here. Writing \u201cgood\u201d<\/strong><\/em> (expressive and efficient) pattern expressions for complex scenarios is really hard, and is no different from writing a \u201cgood program\u201d<\/strong><\/em> in any language. If we aren\u2019t careful enough, the rules can become \u201cfragile\u201d<\/strong><\/em> and fail to match valid scenarios. Of course, in these situations, it would be nice to have a tool that can take some sample sentences and generate the rules and pattern expression(s) for us! (Hope the tech team at MeaningCloud<\/strong><\/em> is listening!)\u00a0<\/span><\/p>\n

I trust you found this article interesting. I intend to spend more time exploring the deep categorization pattern language so that I can apply it to tougher problems.<\/p>\n

You might also be interested in looking at MeaningCloud’s Text Parsing<\/strong><\/em> engine, described in my earlier blog<\/strong><\/em><\/a>.<\/p>\n

Have a nice day and great week ahead!<\/p>\n","protected":false},"excerpt":{"rendered":"

MeaningCloud\u00a0announced their premium offering \u201cDeep Categorization\u201d service quite recently. You can read about it in this nice blog\u00a0by Blanca Galego. Compared to the canonical classification models that rely on machine learning using large data sets, MeaningCloud\u2019s deep categorization models use hand-crafted rules that take advantage of \u201cmorphosyntactic, semantic and contextual information\u201d to classify a piece […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[22,107,17],"tags":[211,25],"class_list":["post-1613","post","type-post","status-publish","format-standard","hentry","category-homeopathy","category-natural-language-processing","category-programming","tag-deep-text-categorization","tag-homeopathy"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9OLnF-q1","jetpack-related-posts":[{"id":1640,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/07\/11\/information-extraction-using-spacys-pattern-matcher\/","url_meta":{"origin":1613,"position":0},"title":"Information Extraction Using spaCy\u2019s Pattern Matcher","author":"admin","date":"July 11, 2019","format":false,"excerpt":"In the previous article, I explored the Deep Categorization capabilities of MeaningCloud. We saw how a powerful rule-based pattern matching language allowed us to map fragments of unstructured text to custom categories. In today\u2019s post, I want to go through spaCy\u2019s\u00a0pattern matching capabilities. The version I am using is 2.0.13.\u2026","rel":"","context":"In "Homeopathy"","block_context":{"text":"Homeopathy","link":"https:\/\/www.rangakrish.com\/index.php\/category\/homeopathy\/"},"img":{"alt_text":"Output","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/07\/Output.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/07\/Output.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/07\/Output.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":1368,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/01\/08\/parsing-text-with-apache-opennlp\/","url_meta":{"origin":1613,"position":1},"title":"Parsing Text with Apache OpenNLP","author":"admin","date":"January 8, 2019","format":false,"excerpt":"In my earlier posts I have written about parsing text using spaCy\u00a0and MeaningCloud's parsing API. For today's article, I decided to take a look at OpenNLP, an open-source ML-based Java toolkit for parsing natural language text. OpenNLP is a fairly mature library and has been around since 2004 (source: Wikipedia).\u2026","rel":"","context":"In "Natural Language Processing"","block_context":{"text":"Natural Language Processing","link":"https:\/\/www.rangakrish.com\/index.php\/category\/natural-language-processing\/"},"img":{"alt_text":"Parse Tree","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/01\/Tree3.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":1285,"url":"https:\/\/www.rangakrish.com\/index.php\/2018\/12\/09\/parsing-text-with-meaningclouds-text-analytics-api\/","url_meta":{"origin":1613,"position":2},"title":"Parsing Text with MeaningCloud’s Text Analytics API","author":"admin","date":"December 9, 2018","format":false,"excerpt":"There is wide-spread interest in Natural Language Processing (NLP) today, and there are several API services available to cater to this demand. See this article for a fairly detailed list of services. All of them support multiple languages, including English. Today, I am going to share my experience in working\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"Get Words Function","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2018\/12\/Get-words.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2018\/12\/Get-words.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2018\/12\/Get-words.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3321,"url":"https:\/\/www.rangakrish.com\/index.php\/2024\/02\/11\/using-similarity-search-in-homeopathy-repertorization-software\/","url_meta":{"origin":1613,"position":3},"title":"Using Similarity Search in Homeopathy Repertorization Software","author":"admin","date":"February 11, 2024","format":false,"excerpt":"Homeopathy Repertorization software such as RadarOpus, MacRepertory, Vithoulkas Compass, and Synergy Homeopathic Software (SHS) are widely used by homeopaths to speed up case taking, repertorization and case analysis. All of them have a \u201cSearch\u201d feature, where the given text is searched (typically based on key words and synonyms) against one\u2026","rel":"","context":"In "Homeopathy"","block_context":{"text":"Homeopathy","link":"https:\/\/www.rangakrish.com\/index.php\/category\/homeopathy\/"},"img":{"alt_text":"Vithoulkas Compass Example","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/02\/VCompass-300x65.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/02\/VCompass-300x65.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/02\/VCompass-300x65.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3283,"url":"https:\/\/www.rangakrish.com\/index.php\/2023\/12\/26\/homeopathy-case-analysis-using-retrieval-augmented-generation\/","url_meta":{"origin":1613,"position":4},"title":"Homeopathy Case Analysis Using Retrieval-Augmented Generation","author":"admin","date":"December 26, 2023","format":false,"excerpt":"Homeopaths, after detailed case taking, usually \u2018\u201crepertorize\u201d\u00a0 the case using software such as RadarOpus, MacRepertory, Vithoulkas Compass, etc., and finally consult a Materia Medica to confirm the remedy selection. There are some highly experienced homeopaths who have the gift of identifying the correct remedy without even repertorizing the case, but\u2026","rel":"","context":"In "Homeopathy"","block_context":{"text":"Homeopathy","link":"https:\/\/www.rangakrish.com\/index.php\/category\/homeopathy\/"},"img":{"alt_text":"Example Chat","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2023\/12\/Two-300x95.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2023\/12\/Two-300x95.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2023\/12\/Two-300x95.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3660,"url":"https:\/\/www.rangakrish.com\/index.php\/2025\/04\/11\/using-claude-to-generate-rdf-triples\/","url_meta":{"origin":1613,"position":5},"title":"Using Claude to Generate RDF Triples","author":"admin","date":"April 11, 2025","format":false,"excerpt":"We all know that LLMs are now capable of generating structured data. I have used OpenAI models earlier to generate Tables and JSON data, but this time I wanted to try a more complex example.\u00a0 As someone interested in Homeopathy, I wanted to generate remedy descriptions as RDF triples, in\u2026","rel":"","context":"In "Homeopathy"","block_context":{"text":"Homeopathy","link":"https:\/\/www.rangakrish.com\/index.php\/category\/homeopathy\/"},"img":{"alt_text":"Lycopodium TTL Format","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/04\/Lyco-300x232.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/04\/Lyco-300x232.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/04\/Lyco-300x232.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1613","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/comments?post=1613"}],"version-history":[{"count":0,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1613\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/media?parent=1613"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/categories?post=1613"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/tags?post=1613"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}