{"id":390,"date":"2016-10-18T10:44:39","date_gmt":"2016-10-18T10:44:39","guid":{"rendered":"http:\/\/www.rangakrish.com\/?p=390"},"modified":"2016-10-18T11:53:36","modified_gmt":"2016-10-18T11:53:36","slug":"torch7-and-neural-networks","status":"publish","type":"post","link":"https:\/\/www.rangakrish.com\/index.php\/2016\/10\/18\/torch7-and-neural-networks\/","title":{"rendered":"Torch7 and Neural Networks"},"content":{"rendered":"

This week I wanted to experiment with Torch7<\/a>, a popular Machine Learning framework implemented in C\/LuaJIT. Seamless CUDA support is another plus point in favour of Torch7.<\/p>\n

I downloaded and installed Torch7 and related packages, as described here<\/a>. It is important to also install cunn<\/strong><\/em>\u00a0and cutorch<\/strong><\/em>\u00a0packages if you need CUDA support. In my case, I did that.<\/p>\n

Before dashing off into more complex convolutional neural nets (CNNs), I wanted\u00a0to start with the basics and implement a simple fully-connected, feed-forward network. In Torch7, this is called a\u00a0Sequential Model<\/strong><\/em>.<\/p>\n

I decided to implement a 3-layer network, each with its transfer function. Each layer is of type Linear<\/strong><\/em>, the most common fully-connected layer. For the first two layers, I chose Sigmoid<\/strong><\/em> and Tanh<\/strong><\/em> transfer functions. For the output layer, I chose\u00a0SoftMax<\/strong><\/em> (called LogSoftMax<\/strong><\/em>\u00a0in Torch7).<\/p>\n

Instead of hard-coding each layer and transfer function, I wanted\u00a0to add a little bit of flexibility, by setting up a table that defines the layers and corresponding transfer functions. I wrote a Lua function to convert this table into the desired Sequential<\/strong><\/em>\u00a0model.<\/p>\n

\"Different<\/a>
Different Layers<\/strong><\/figcaption><\/figure>\n

As you can see, a simple trick is needed in the buildLayer<\/strong><\/em>\u00a0function to create the layer dynamically.<\/p>\n

Because I also wanted to test with CUDA support enabled, I wrote a wrapper function to convert a Tensor into CudaTensor as needed (inspired by this blog<\/a>).<\/p>\n

\"CUDA<\/a>
CUDA Wrapper<\/strong><\/figcaption><\/figure>\n

The rest of the code is fairly straightforward. First, I setup a flag to decide whether the code should run on CUDA. This depends on extra argument passed to the Lua file as well as on the availability of GPU on the machine. Next, I create the network, and then compute the elapsed time to trigger the network in the forward direction with random input data. The actual time taken is finally printed.<\/p>\n

\"Building<\/a>
Building and Testing the Network<\/strong><\/figcaption><\/figure>\n

The following figure shows the output I get without CUDA option enabled:<\/p>\n

\"Result<\/a>
Result Without CUDA<\/strong><\/figcaption><\/figure>\n

Now, with CUDA enabled (pass any dummy argument to the file), here is the output:<\/p>\n

\"Result<\/a>
Result With CUDA<\/strong><\/figcaption><\/figure>\n

You can see that when\u00a0I enable\u00a0CUDA, the program runs\u00a05 times faster!<\/p>\n

This experiment was done on a machine with the following configuration:<\/p>\n

i7-6700<\/strong><\/em><\/p>\n

3.4 GHz<\/strong><\/em><\/p>\n

64 GB RAM<\/strong><\/em><\/p>\n

NVIDIA GTX 1080 GPU with 8GB RAM<\/strong><\/em><\/p>\n

Ubuntu 14.04<\/strong><\/em><\/p>\n

Today’s\u00a0example does not attempt to train the network with real samples and test the prediction accuracy. That will be the focus of\u00a0a future\u00a0post.<\/p>\n

You can download the LUA code from here<\/a>.<\/p>\n

Thanks for visiting!<\/p>\n","protected":false},"excerpt":{"rendered":"

This week I wanted to experiment with Torch7, a popular Machine Learning framework implemented in C\/LuaJIT. Seamless CUDA support is another plus point in favour of Torch7. I downloaded and installed Torch7 and related packages, as described here. It is important to also install cunn\u00a0and cutorch\u00a0packages if you need CUDA support. In my case, I […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[78,17],"tags":[79,80,81],"class_list":["post-390","post","type-post","status-publish","format-standard","hentry","category-machine-learning","category-programming","tag-lua","tag-neural-net","tag-torch7"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9OLnF-6i","jetpack-related-posts":[{"id":304,"url":"https:\/\/www.rangakrish.com\/index.php\/2016\/08\/20\/cuda-and-mathematica\/","url_meta":{"origin":390,"position":0},"title":"CUDA and Mathematica","author":"admin","date":"August 20, 2016","format":false,"excerpt":"Recently I purchased a high-end desktop computer for my image processing project. Since many computations tend to take several hours to execute, I wanted to accelerate the calculations to the extent possible by adding a GPU. I chose NIVIDA's GeForce GTX 1080\u00a0processor-based card. Although I will be using C++ for\u2026","rel":"","context":"In "Mathematica"","block_context":{"text":"Mathematica","link":"https:\/\/www.rangakrish.com\/index.php\/category\/mathematica\/"},"img":{"alt_text":"Basic CUDA Check","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/08\/Mathematica-1-1024x771.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/08\/Mathematica-1-1024x771.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/08\/Mathematica-1-1024x771.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2016\/08\/Mathematica-1-1024x771.png?resize=700%2C400 2x"},"classes":[]},{"id":2736,"url":"https:\/\/www.rangakrish.com\/index.php\/2022\/03\/19\/book-review-introducing-blockchain-with-lisp\/","url_meta":{"origin":390,"position":1},"title":"Book Review: Introducing Blockchain with Lisp","author":"admin","date":"March 19, 2022","format":false,"excerpt":"Title: Introducing Blockchain with Lisp: Implement and Extend Blockchains with the Racket Language Author: Boro Sitnikovski Publisher: Apress Year: 2021 Blockchains are a hot topic these days and interestingly, I am getting involved in a project that uses Blockchain. Languages such as C++, Java, Go and Node.js are commonly used\u2026","rel":"","context":"In "Book Review"","block_context":{"text":"Book Review","link":"https:\/\/www.rangakrish.com\/index.php\/category\/book-review\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2022\/03\/Blockchain-Book-300x251.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":4265,"url":"https:\/\/www.rangakrish.com\/index.php\/2026\/03\/22\/counting-sentences-an-implementation-in-c20\/","url_meta":{"origin":390,"position":2},"title":"Counting Sentences: An Implementation in C++20","author":"admin","date":"March 22, 2026","format":false,"excerpt":"Counting the number of sentences in a given paragraph appears rather simple on the surface - look for the common punctuation marks: \u201c.?!\u201d. Only when you dig deeper, you will know that it is really not that simple. For example, consider this text: \u201cPeter met Dr.James at 3 p.m.\u201d How\u2026","rel":"","context":"In "C++"","block_context":{"text":"C++","link":"https:\/\/www.rangakrish.com\/index.php\/category\/c\/"},"img":{"alt_text":"Regular Expressions","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2026\/03\/regex-300x91.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2026\/03\/regex-300x91.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2026\/03\/regex-300x91.png?resize=525%2C300&ssl=1 1.5x"},"classes":[]},{"id":278,"url":"https:\/\/www.rangakrish.com\/index.php\/2016\/07\/07\/multimethods-in-julia\/","url_meta":{"origin":390,"position":3},"title":"Multimethods in Julia","author":"admin","date":"July 7, 2016","format":false,"excerpt":"I got interested in Julia programming language quite recently, primarily because of a project involving image processing and machine learning. The language is still evolving, but already has a rich set of features and a good collection of external libraries\u00a0covering many areas. One of the highlights of the language is\u2026","rel":"","context":"In "Julia"","block_context":{"text":"Julia","link":"https:\/\/www.rangakrish.com\/index.php\/category\/julia\/"},"img":{"alt_text":"Multimethods Example","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2015\/10\/Multimethods.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2057,"url":"https:\/\/www.rangakrish.com\/index.php\/2020\/07\/19\/calling-lisp-functions-from-elixir\/","url_meta":{"origin":390,"position":4},"title":"Calling Lisp Functions from Elixir","author":"admin","date":"July 19, 2020","format":false,"excerpt":"In the last article\u00a0I showed how we can simulate the idea of Lisp's \"closure\" in Elixir. Today, I would like to demonstrate how we can call Lisp functions from Elixir using the NIF interface. What is the need to integrate Elixir with another language? I can think of two reasons:\u2026","rel":"","context":"In "Elixir"","block_context":{"text":"Elixir","link":"https:\/\/www.rangakrish.com\/index.php\/category\/elixir\/"},"img":{"alt_text":"Using the Lisp Functions","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/07\/Session.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/07\/Session.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/07\/Session.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2020\/07\/Session.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":2351,"url":"https:\/\/www.rangakrish.com\/index.php\/2021\/03\/14\/calling-c-methods-from-lispworks-lisp-part-2\/","url_meta":{"origin":390,"position":5},"title":"Calling C# Methods from LispWorks Lisp – Part 2","author":"admin","date":"March 14, 2021","format":false,"excerpt":"In the last article, I showed how we can invoke C# methods from LispWorks Enterprise Edition, through the COM\/Automation interface. That approach relied on invoking the Automation methods dynamically, without depending on the Type library (*.tlb).\u00a0 In this article, I will discuss the other approach, which uses the Type library.\u2026","rel":"","context":"In "LISP"","block_context":{"text":"LISP","link":"https:\/\/www.rangakrish.com\/index.php\/category\/lisp\/"},"img":{"alt_text":"C# Code","src":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2021\/03\/Code2-236x300.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2021\/03\/Code2-236x300.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/www.rangakrish.com\/wp-content\/uploads\/2021\/03\/Code2-236x300.jpg?resize=525%2C300&ssl=1 1.5x"},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/390","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/comments?post=390"}],"version-history":[{"count":0,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/posts\/390\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/media?parent=390"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/categories?post=390"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rangakrish.com\/index.php\/wp-json\/wp\/v2\/tags?post=390"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}