{"id":1870,"date":"2020-01-19T16:26:17","date_gmt":"2020-01-19T10:56:17","guid":{"rendered":"https:\/\/www.rangakrish.com\/?p=1870"},"modified":"2020-01-19T16:26:17","modified_gmt":"2020-01-19T10:56:17","slug":"experimenting-with-text-simplification","status":"publish","type":"post","link":"https:\/\/www.rangakrish.com\/index.php\/2020\/01\/19\/experimenting-with-text-simplification\/","title":{"rendered":"Experimenting with Text Simplification"},"content":{"rendered":"

After my last book review<\/strong><\/em><\/a>, I decided to check out a few websites that claim to simplify English text and\/or help compute the measure of readability. In today\u2019s post, I am sharing the results of my experiment.<\/p>\n

www.simplish.org<\/a><\/h3>\n

This site has some interesting functionality. It does spelling check, grammar check, text simplification, and can even provide a summary of the given text.<\/p>\n

I gave the following text for simplification (taken from here<\/strong><\/em><\/a>):<\/p>\n

Long sentences (full of description and unnecessary circumlocution), while capable of containing more information than a much shorter sentence, have a tendency to indulge in bad habits like the passive voice, and oftimes the reader, upon reaching the end of a monstrously long and verbose sentence, doesn’t remember what the sentence was supposed to be about.<\/span><\/p><\/blockquote>\n

Here is the simplified<\/strong><\/em> text from the system:<\/p>\n

Long groups of words making sense (full of account and unnecessary long way round), while able of having in it more news given than a much shorter group of words making sense, have a tendency to give way to in bad dresses of religion like the actionless voice, and oftimes the reader, upon getting to the end of a cruelly long and full of words group of words making sense, doesn’t have in mind what the group of words making sense was took as probable to be about.<\/span><\/p><\/blockquote>\n

How is that? Forget simplification, the resulting text is not even grammatical!<\/p>\n

I tried a simpler sentence next:<\/p>\n

I deem it an honour to be bestowed this coveted title upon me.<\/span><\/p><\/blockquote>\n

This is what I got back in return:<\/p>\n

I be of opinion (of) it a great respect to be given this desired sign of position upon me.<\/span><\/p><\/blockquote>\n

Not quite inspiring.\u00a0<\/span><\/p>\n

I then asked the system to check the grammar of the above text. No problems reported.<\/p>\n

My next example was about the correct use of the phrase “<\/strong>cope with”<\/strong><\/em>. I have seen people wrongly use the phrase as “<\/strong>cope up with”<\/strong><\/em>.\u00a0<\/span><\/p>\n

For example, the sentence \u201cHe could not cope with the job stress\u201d<\/strong><\/em> is sometimes written as \u201cHe could not cope up with the job stress\u201d<\/strong><\/em>.<\/p>\n

When I asked the system to check the grammar of the text \u201cHe could not cope up with the job stress\u201d,<\/strong><\/em> I got the same text back, suggesting that it was grammatically correct.<\/p>\n

I definitely expected something better from the system.<\/p>\n

www.rewordify.com<\/a><\/h3>\n

I tried giving the two examples as earlier.<\/p>\n

Example-1:<\/p>\n

Long sentences (full of description and unnecessary circumlocution), while capable of containing more information than a much shorter sentence, have a tendency to indulge in bad habits like the passive voice, and oftimes the reader, upon reaching the end of a monstrously long and verbose sentence, doesn’t remember what the sentence was supposed to be about.<\/span><\/p><\/blockquote>\n

Here is the output from the system:<\/p>\n

Long sentences (full of description and unnecessary (the use of a lot of words, but not in a clear way)), while capable of containing more information than a much shorter sentence, have a habit\/desire to do bad habits like the (when you write (for example) that a ball was thrown by a child, rather than a child threw a ball), and oftimes the reader, upon reaching the end of a terribly long and talkative (or rambling) sentence, doesn’t remember what the sentence was supposed to be about.<\/span><\/p><\/blockquote>\n

Example-2:<\/p>\n

I deem it an honour to be bestowed this coveted title upon me.<\/span><\/p><\/blockquote>\n

Result:<\/p>\n

I think of\/consider it a honor to be given this jealously desired title upon me.<\/span><\/p><\/blockquote>\n

Not much different from www.simplish.com<\/strong><\/em>. I am not impressed.<\/p>\n

www.readable.com<\/a><\/h3>\n

This site does not provide text simplification service, but computes the \u201creadability\u201d<\/strong><\/em> of the given text. I had to sign up for the basic plan, which comes with a 7-day free trial.<\/p>\n

I entered the following text used earlier:<\/p>\n

Long sentences (full of description and unnecessary circumlocution), while capable of containing more information than a much shorter sentence, have a tendency to indulge in bad habits like the passive voice, and oftimes the reader, upon reaching the end of a monstrously long and verbose sentence, doesn’t remember what the sentence was supposed to be about.<\/span><\/p><\/blockquote>\n

I got the following metrics:<\/p>\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n
SMOG Index<\/span><\/td>\n22.9<\/span><\/td>\n<\/tr>\n
Automated Readability Index<\/span><\/td>\n31.4<\/span><\/td>\n<\/tr>\n
FORCAST Grade Level<\/span><\/td>\n11.4<\/span><\/td>\n<\/tr>\n
Powers Sumner Kearl Grade<\/span><\/td>\n10.1<\/span><\/td>\n<\/tr>\n
Rix Readability<\/span><\/td>\n13<\/span><\/td>\n<\/tr>\n
Readable Rating<\/span><\/td>\nE<\/span><\/td>\n<\/tr>\n
Flesch Reading Ease<\/span><\/td>\n3.5<\/span><\/td>\n<\/tr>\n
CEFR Level<\/span><\/td>\nB2<\/span><\/td>\n<\/tr>\n
IELTS Level<\/span><\/td>\n5-6<\/span><\/td>\n<\/tr>\n
Spache Score<\/span><\/td>\n10.2<\/span><\/td>\n<\/tr>\n
New Dale-Chall Score<\/span><\/td>\n7<\/span><\/td>\n<\/tr>\n
Lensear Write<\/span><\/td>\n53.6<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

There are some more metrics such as sentences with more than 30 syllables, words having more than 12 letters, and so on, but those are not important for our discussion here. The essential point is that this site provides various measures of readability of the given text and can be useful if we know how to write satisfying these metrics.<\/p>\n

Here is an example of a highly readable text:<\/p>\n

Yoga is good for the body and mind.<\/span><\/p><\/blockquote>\n

Here are the scores:<\/p>\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n
SMOG Index<\/span><\/td>\n3.1<\/span><\/td>\n<\/tr>\n
Automated Readability Index<\/span><\/td>\n-1.5<\/span><\/td>\n<\/tr>\n
FORCAST Grade Level<\/span><\/td>\n8.8<\/span><\/td>\n<\/tr>\n
Powers Sumner Kearl Grade<\/span><\/td>\n3.8<\/span><\/td>\n<\/tr>\n
Rix Readability<\/span><\/td>\n1<\/span><\/td>\n<\/tr>\n
Readable Rating<\/span><\/td>\nA<\/span><\/td>\n<\/tr>\n
Flesch Reading Ease<\/span><\/td>\n93.0<\/span><\/td>\n<\/tr>\n
CEFR Level<\/span><\/td>\nA2<\/span><\/td>\n<\/tr>\n
IELTS Level<\/span><\/td>\n2-4<\/span><\/td>\n<\/tr>\n
Spache Score<\/span><\/td>\n3.7<\/span><\/td>\n<\/tr>\n
New Dale-Chall Score<\/span><\/td>\n2.4<\/span><\/td>\n<\/tr>\n
Lensear Write<\/span><\/td>\n87.5<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The above sentence gets a grade of \u201cA\u201d,<\/strong><\/em> whereas the earlier one got a grade of \u201cE\u201d<\/strong><\/em>.<\/p>\n

Here is an example of an ungrammatical, nonsense text:<\/p>\n

A that it they he she.<\/span><\/p><\/blockquote>\n

Can you guess the scores I got?<\/p>\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n
SMOG Index<\/span><\/td>\n3.1<\/span><\/td>\n<\/tr>\n
Automated Readability Index<\/span><\/td>\n-5.9<\/span><\/td>\n<\/tr>\n
FORCAST Grade Level<\/span><\/td>\n5.0<\/span><\/td>\n<\/tr>\n
Powers Sumner Kearl Grade<\/span><\/td>\n3.6<\/span><\/td>\n<\/tr>\n
Rix Readability<\/span><\/td>\n1<\/span><\/td>\n<\/tr>\n
Readable Rating<\/span><\/td>\nA<\/span><\/td>\n<\/tr>\n
Flesch Reading Ease<\/span><\/td>\n116.1<\/span><\/td>\n<\/tr>\n
CEFR Level<\/span><\/td>\nA2<\/span><\/td>\n<\/tr>\n
IELTS Level<\/span><\/td>\n2-4<\/span><\/td>\n<\/tr>\n
Spache Score<\/span><\/td>\n1.4<\/span><\/td>\n<\/tr>\n
New Dale-Chall Score<\/span><\/td>\n0.3<\/span><\/td>\n<\/tr>\n
Lensear Write<\/span><\/td>\n150<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Isn\u2019t it weird that this also gets a \u201cGrade A\u201d<\/strong><\/em>? I am sure we have to use these metrics with care and not depend on them entirely.<\/p>\n

Well, I decided to stop my experimentation at this point. I guess there are some more sites that help in assessment of readability and\/or help with simplifying given text.\u00a0<\/span><\/p>\n

As a general remark on readability, technical articles with domain-dependent terms and definitions might be difficult for readers from other disciplines. For example someone from software domain might find an article on biology difficult to understand (and vice versa).<\/p>\n

What about syntactically correct, but semantically absurd sentences: \u201cThe milk drank the cat\u201d?<\/strong><\/em><\/p>\n

Or what about ambiguous sentences: \u201cI saw a man on the hill with a telescope\u201d?<\/strong><\/em><\/p>\n

Coming up with a measure of readability based on so many factors is definitely hard, but even harder is taking a sentence and simplifying it.\u00a0No wonder, then, text simplification as a research topic is enormously challenging!<\/span><\/p>\n

Have a great week ahead!<\/p>\n","protected":false},"excerpt":{"rendered":"

After my last book review, I decided to check out a few websites that claim to simplify English text and\/or help compute the measure of readability. In today\u2019s post, I am sharing the results of my experiment. www.simplish.org This site has some interesting functionality. It does spelling check, grammar check, text simplification, and can even […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[107],"tags":[225,226],"class_list":["post-1870","post","type-post","status-publish","format-standard","hentry","category-natural-language-processing","tag-readability-metrics","tag-text-simplification"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9OLnF-ua","jetpack-related-posts":[{"id":1863,"url":"https:\/\/www.rangakrish.com\/index.php\/2020\/01\/02\/book-review-automatic-text-simplification\/","url_meta":{"origin":1870,"position":0},"title":"Book Review – Automatic Text Simplification","author":"admin","date":"January 2, 2020","format":false,"excerpt":"Title: Automatic Text Simplification Author: Horacio Saggino Publisher: Morgan & Claypool Publishers Year: 2017 Automatic Text Simplification is an active area of research in NLP and has been going on for over 20 years. The idea is to transform a given text T1 into text T2 such that T2 is\u2026","rel":"","context":"In "Book Review"","block_context":{"text":"Book Review","link":"https:\/\/www.rangakrish.com\/index.php\/category\/book-review\/"},"img":{"alt_text":"Automatic Text Simplification","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/01\/IMG_1496-edited-225x300.jpeg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":2224,"url":"https:\/\/www.rangakrish.com\/index.php\/2020\/12\/06\/using-falcon-2-with-opusmodus\/","url_meta":{"origin":1870,"position":1},"title":"Using Falcon 2 with Opusmodus","author":"admin","date":"December 6, 2020","format":false,"excerpt":"The recently concluded Black Friday sale by UVI\u00a0was an irresistible temptation for me and I ended up investing in Falcon 2 and a few other instruments. Although I have the latest versions of Ableton Live, Propellerheads Reason, Renoise, and Reaper, I felt Falcon 2 would be a good addition to\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"Sample Code in Opusmodus","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/12\/code-248x300.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":179,"url":"https:\/\/www.rangakrish.com\/index.php\/2016\/03\/12\/converting-text-to-music\/","url_meta":{"origin":1870,"position":2},"title":"Converting Text to Music","author":"admin","date":"March 12, 2016","format":false,"excerpt":"Converting text to music can be fun. Opusmodus\u00a0has a few built-in functions to work with text. Look up the description of text-to-pitch, for example. As described in its documentation, (text-to-pitch '(\"To be, or not to be, that is the question\")) => (((g5 d5) (cs4 e4) (d5 f5) (cs5 d5 g5)\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"Text to Music","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/03\/Text-to-Music-204x300.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/03\/Text-to-Music-204x300.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/03\/Text-to-Music-204x300.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/03\/Text-to-Music-204x300.png?resize=700%2C400 2x"},"classes":[]},{"id":1517,"url":"https:\/\/www.rangakrish.com\/index.php\/2019\/03\/28\/ocr-using-azure-computer-vision-api\/","url_meta":{"origin":1870,"position":3},"title":"OCR Using Azure Computer Vision API","author":"admin","date":"March 28, 2019","format":false,"excerpt":"I have been getting some good feedback on Azure\u2019s Computer Vision API, in particular, the OCR functionality. Although I am not working on any project that requires this functionality at the moment, I thought it would be a good idea to check out the service - just to be \u201cfuture\u2026","rel":"","context":"In "Image Processing"","block_context":{"text":"Image Processing","link":"https:\/\/www.rangakrish.com\/index.php\/category\/image-processing\/"},"img":{"alt_text":"Converted Text","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/03\/Converted-1.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/03\/Converted-1.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2019\/03\/Converted-1.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3545,"url":"https:\/\/www.rangakrish.com\/index.php\/2024\/11\/09\/semantic-search-in-wolfram-mathematica\/","url_meta":{"origin":1870,"position":4},"title":"Semantic Search in Wolfram Mathematica","author":"admin","date":"November 9, 2024","format":false,"excerpt":"In an earlier article, I explained how to use OpenAI from Wolfram Mathematica ver 14.1. This latest release of Wolfram supports Semantic Search as well. In today\u2019s article, let me discuss this feature. As in the case of using LLMs, using Semantic Search requires an account with one of the\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"Remedy Description","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/11\/image1-300x225.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/11\/image1-300x225.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/11\/image1-300x225.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":912,"url":"https:\/\/www.rangakrish.com\/index.php\/2018\/04\/22\/question-answering-using-dependency-trees\/","url_meta":{"origin":1870,"position":5},"title":"Question Answering\u00a0Using Dependency Trees","author":"admin","date":"April 22, 2018","format":false,"excerpt":"A few weeks ago I had written about my brief experiment with Mathematica's new feature, which provides answers to questions based on given text. After that post, I spent some time thinking about how to implement something similar. In today's post, I want to show you what I have been\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"Dependency Tree","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2018\/04\/Deptree-example.png?resize=350%2C200","width":350,"height":200},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1870","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/comments?post=1870"}],"version-history":[{"count":0,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/1870\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/media?parent=1870"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/categories?post=1870"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/tags?post=1870"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}