{"id":3545,"date":"2024-11-09T19:23:02","date_gmt":"2024-11-09T13:53:02","guid":{"rendered":"https:\/\/www.rangakrish.com\/?p=3545"},"modified":"2024-11-09T19:23:02","modified_gmt":"2024-11-09T13:53:02","slug":"semantic-search-in-wolfram-mathematica","status":"publish","type":"post","link":"https:\/\/www.rangakrish.com\/index.php\/2024\/11\/09\/semantic-search-in-wolfram-mathematica\/","title":{"rendered":"Semantic Search in Wolfram Mathematica"},"content":{"rendered":"

In an earlier article<\/strong><\/em><\/a>, I explained how to use OpenAI<\/strong><\/em> from Wolfram Mathematica<\/strong><\/em> ver 14.1. This latest release of Wolfram<\/strong><\/em> supports Semantic Search<\/strong><\/em> as well. In today\u2019s article, let me discuss this feature.<\/p>\n

As in the case of using LLMs<\/strong><\/em>, using Semantic Search<\/strong><\/em> requires an account with one of the supported vendors. I will continue to use OpenAI<\/strong><\/em> for this functionality too.<\/p>\n

For this example, I am going to create three short text fragments, each describing a homeopathy remedy briefly. The remedies are Lycopodium<\/strong><\/em>, Argentum Nitricum<\/strong><\/em> and Natrum Muriaticum<\/strong><\/em>.<\/p>\n

\"Remedy<\/a>
Remedy Description<\/strong><\/figcaption><\/figure>\n

I am going to build a semantic search database for the above text fragments.\u00a0<\/span><\/p>\n

\"Creating<\/a>
Creating a Semantic Search Index<\/strong><\/figcaption><\/figure>\n

The above function returns an index object after storing two texts. I have given a convenient label, in this case the remedy name, for the texts. In addition, since the data size is quite small, I am creating an in-memory database instead of using a persistent file object. Finally, I am specifying the minimum and maximum token limits for each chunk. The reason I am using small values for these is because my text fragments are small.<\/p>\n

The function returns an Index<\/strong><\/em> object that has several properties:<\/p>\n

\"Index<\/a>
Index Properties<\/strong><\/figcaption><\/figure>\n

The \u201cDocumentCount\u201d<\/strong><\/em> property tells us how many documents have been stored in the database. The \u201cItemCount\u201d<\/strong><\/em> property tells us the number of chunks created for our text. The \u201cLabel\u201d<\/strong><\/em> property returns a list of the labels for each chunk in the database.<\/p>\n

OK, how do we search for some text in this database? Let us ask the system how many remedies have “sweet craving”<\/strong><\/em>? And which remedy has “aggravation from sweets”<\/strong><\/em>?<\/p>\n

\"Searching<\/a>
Searching using the Index<\/strong><\/figcaption><\/figure>\n

The answers are correct! Both Lycopodium<\/strong><\/em> and Argentum Nitricum<\/strong><\/em> have desire for sweets. And Argentum Nitricum<\/strong><\/em> patients typically have some aggravation from sweets.<\/p>\n

What if we want to add some more text to this database and create an updated index? That is easy:<\/p>\n

\"Adding<\/a>
Adding More Text<\/strong><\/figcaption><\/figure>\n

We use the function \u201cUpdateSemanticSearchIndex\u201d<\/strong><\/em> for this. To use this, I am passing the third text fragment and specify the same chunking limits as earlier.<\/p>\n

You can now see that the number documents is 3 and 5 more chunks have been created for Natrum Muriaticum<\/strong><\/em>.<\/p>\n

Let us do another search with the new remedy added. We know that Natrum Muriaticum<\/strong><\/em> patients have an affinity for salt. Let us check this:<\/p>\n

\"Searching<\/a>
Searching Updated Index<\/strong><\/figcaption><\/figure>\n

The search function correctly returns the remedy label. That is good!<\/p>\n

The above three functions in the latest release of Wolfram<\/strong><\/em> are quite useful for doing semantic search on text. It is easy to extend the example to work with larger documents and persisting the database in the local file system, or in the cloud.<\/p>\n

Have a nice weekend!<\/p>\n","protected":false},"excerpt":{"rendered":"

In an earlier article, I explained how to use OpenAI from Wolfram Mathematica ver 14.1. This latest release of Wolfram supports Semantic Search as well. In today\u2019s article, let me discuss this feature. As in the case of using LLMs, using Semantic Search requires an account with one of the supported vendors. I will continue […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[72,107,388,17],"tags":[407,408],"class_list":["post-3545","post","type-post","status-publish","format-standard","hentry","category-mathematica","category-natural-language-processing","category-openai","category-programming","tag-semantic-search","tag-wolfram-mathematica"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9OLnF-Vb","jetpack-related-posts":[{"id":3514,"url":"https:\/\/www.rangakrish.com\/index.php\/2024\/10\/02\/using-openai-from-mathematica-part-2\/","url_meta":{"origin":3545,"position":0},"title":"Using OpenAI from Mathematica: Part-2","author":"admin","date":"October 2, 2024","format":false,"excerpt":"I had written an earlier article\u00a0showing how to use OpenAI models from Mathematica ver 14.0.\u00a0 Wolfram Mathematica ver 14.1 was released recently, with several improvements in the area of LLMs. Of course, there are many other core additions as well, but our focus is on LLMs in this article. This\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"LLMSynthesize Function","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/10\/image1-300x71.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/10\/image1-300x71.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/10\/image1-300x71.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3381,"url":"https:\/\/www.rangakrish.com\/index.php\/2024\/05\/20\/using-openai-from-mathematica\/","url_meta":{"origin":3545,"position":1},"title":"Using OpenAI from Mathematica","author":"admin","date":"May 20, 2024","format":false,"excerpt":"Mathematica was among the first to integrate with OpenAI. The functionality is nicely exposed in terms of a few pre-defined functions. Let us explore some of the functionality in today\u2019s article. The simplest way to get started is to use LLMSynthesize\u00a0function: It can take a few seconds before you get\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"LLMSynthesize Function","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/05\/Example1-300x27.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/05\/Example1-300x27.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2024\/05\/Example1-300x27.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":3794,"url":"https:\/\/www.rangakrish.com\/index.php\/2025\/08\/19\/stock-analysis-with-wolfram-mathematica-part-2\/","url_meta":{"origin":3545,"position":2},"title":"Stock Analysis with Wolfram Mathematica – Part 2","author":"admin","date":"August 19, 2025","format":false,"excerpt":"In the previous article, I had explained how to get started using Wolfram Mathematica\u2019s stock analysis features. The focus was on working with the exchanges supported by Mathematica, for example BSE India. What if we have stock data from a different source and want to analyze it in Mathematica? In\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"Loading the CSV Data","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/08\/processdata-fn-300x149.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/08\/processdata-fn-300x149.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/08\/processdata-fn-300x149.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":2131,"url":"https:\/\/www.rangakrish.com\/index.php\/2020\/09\/13\/mathematica-using-textcases-to-extract-information-from-natural-language-text\/","url_meta":{"origin":3545,"position":3},"title":"Mathematica: Using TextCases to Extract Information from Natural Language Text\u00a0","author":"admin","date":"September 13, 2020","format":false,"excerpt":"Extracting meaningful information from unstructured, human readable text is a hot topic of research today and has important applications in many domains. I have written a few blogs related to this topic, for example, see this\u00a0and this. In today\u2019s article, I would like to show how Mathematica can be a\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"Extracting Sentences","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/09\/ex1-2-300x106.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":3830,"url":"https:\/\/www.rangakrish.com\/index.php\/2025\/10\/31\/analysing-weather-data-using-wolfram-mathematica\/","url_meta":{"origin":3545,"position":4},"title":"Analysing Weather Data Using Wolfram Mathematica","author":"admin","date":"October 31, 2025","format":false,"excerpt":"If you are interested in analysing weather data and making predictions such as rainfall, storm, etc., then Mathematica has nice functionality that you can use. In this article, let me cover a few basic functions in this category. Let us start by getting the coordinates of the place for which\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"Getting Location Coordinates","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/10\/coord-300x50.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":3774,"url":"https:\/\/www.rangakrish.com\/index.php\/2025\/08\/04\/stock-analysis-using-wolfram-mathematica\/","url_meta":{"origin":3545,"position":5},"title":"Stock Analysis Using Wolfram Mathematica","author":"admin","date":"August 4, 2025","format":false,"excerpt":"I have my own toolkit for analyzing stocks, but occasionally I use Mathematica as well. Given Mathematica\u2019s extensive support for numerical computation including Machine Learning, it can be a valuable resource for deep analysis of the stock market. In today\u2019s article, let me give an overview of some of the\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"Supported Exchanges","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/08\/exchanges-300x28.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/08\/exchanges-300x28.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2025\/08\/exchanges-300x28.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/3545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/comments?post=3545"}],"version-history":[{"count":0,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/3545\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/media?parent=3545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/categories?post=3545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/tags?post=3545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}