Dimagog Blog2013-09-16T22:43:51-07:00http://dimagog.github.ioDmitry Kakurindmitry.kakurin@gmail.comClojure All The Way - Deep Knockout2013-07-31T00:00:00-07:00http://dimagog.github.io/blog/clojure/clojurescript/2013/07/31/clojure-all-the-way-deep-knockout<p>This is a fourth and final article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).</p>
<p>Building up on the <a href='/blog/clojure/clojurescript/2013/07/22/clojure-all-the-way-knockout'>previous post</a> we will achieve much deeper integration of Clojure data structures with <a href='http://knockoutjs.com/'>Knockout.js</a>. But it comes with a price …</p>
<h2 id='our_goal'>Our Goal</h2>
<p>By the end of the <a href='/blog/clojure/clojurescript/2013/07/22/clojure-all-the-way-knockout'>previous post</a> we’ve implemented <code>observable-ref</code> that completely hides Clojure-to-JavaScript conversion call to <code>clj->js</code> function, but <em>we know</em> it’s still there.</p>
<p>Maybe it’s silly to worry about it, but I still do. So what if we set our mind to get rid of it completely, and have “turtles all the way down”? That’s my goal for this post and it’s up to the reader to decide if it is achieved or not :-).</p>
<h2 id='know_the_enemy'>Know the Enemy</h2>
<p>Let’s have a detailed look at what our Clojure data structures are and what our “enemy” (for this post) <code>clj->js</code> does for us.</p>
<p>The data we are rendering (headers of HTTP request) is a Clojure <code>vector</code> of 2-element <code>vectors</code> of <code>strings</code>. In type-safe language like C# or Java it would be <code>Vector<Vector<string>></code>.</p>
<p>The <code>clj->js</code> function recursively converts our Clojure <code>vector</code> of <code>vectors</code> into JavaScript <code>Array</code> of <code>Arrays</code>. I.e. not only outer <code>vector</code> is converted, but each element of it (inner <code>vector</code> of <code>strings</code>) is also converted to <code>Array</code>. The recursion stops there because <code>string</code> representation is the same across ClojureScript and JavaScript and no conversion is needed.</p>
<h2 id='the_easy_part__inner_vectors'>The Easy part - Inner vectors</h2>
<p>Later in this post it will become clear why this is the easy part, but let’s start with inner <code>vectors</code>.</p>
<p><em>Note: Code samples include only interesting parts, <a href='#src'>full source code is available below</a>.</em></p>
<h3 id='shallow_outer_vector_conversion'>Shallow outer vector conversion</h3>
<p>We want to stop converting inner <code>vectors</code> into <code>Arrays</code>. I.e. we don’t need a deep recursive conversion of outer <code>vector</code> that <code>clj->js</code> provides. Instead we want to convert outer <code>vector</code> to <code>Array</code> and leave its elements intact. The result of this step should become <code>Array</code> of <code>vectors</code> of <code>strings</code>.</p>
<p>This is easy, we can use <code>into-array</code> function in place of <code>clj->js</code> in <code>observable-ref</code> function:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='kd'>defn </span><span class='nv'>observable-ref</span> <span class='p'>[</span><span class='nv'>r</span><span class='p'>]</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>state</span> <span class='p'>(</span><span class='nf'>ko/observable</span> <span class='p'>(</span><span class='nb'>into-array </span><span class='o'>@</span><span class='nv'>r</span><span class='p'>))]</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nf'>add-watch</span> <span class='nv'>r</span> <span class='nv'>state</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>obs</span> <span class='nv'>_</span> <span class='nv'>_</span> <span class='nv'>new</span><span class='p'>]</span> <span class='p'>(</span><span class='nf'>obs</span> <span class='p'>(</span><span class='nb'>into-array </span><span class='nv'>new</span><span class='p'>))))</span>
<span class='lineno'>4</span> <span class='nv'>state</span><span class='p'>))</span>
</code></pre></div>
<p>The only changes to this function compared to the <a href='/blog/clojure/clojurescript/2013/07/22/clojure-all-the-way-knockout'>previous post</a> are <code>into-array</code> calls in <em>lines 2 and 3</em> where <code>clj->js</code> calls used to be.</p>
<p>Of cause if we try to render HTML now it won’t work because our <a href='http://knockoutjs.com/'>KO</a> bindings look like this:</p>
<div class='highlight'><pre><code class='html'><span class='nt'><td</span> <span class='na'>data-bind=</span><span class='s'>"text: $data[0]"</span><span class='nt'>></td></span>
<span class='nt'><td</span> <span class='na'>data-bind=</span><span class='s'>"text: $data[1]"</span><span class='nt'>></td></span>
</code></pre></div>
<p>and <code>$data[0]</code> access syntax does not work for Clojure <code>vectors</code>.</p>
<h3 id='accessing_persistentvector_elements_from_javascript_code'>Accessing PersistentVector elements from JavaScript code</h3>
<p>If we examine <code>vector</code>’s “class” <code>PersistentVector</code> from pure JavaScript perspective we’ll see the <code>nth</code> method that we want to call to get <code>vector</code>’s elements. But it looks like this:</p>
<div class='highlight'><pre><code class='javascript'><span class='nx'>cljs$core$IIndexed$_nth$arity$2</span><span class='o'>:</span> <span class='kd'>function</span> <span class='p'>(</span><span class='nx'>coll</span><span class='p'>,</span> <span class='nx'>n</span><span class='p'>)</span>
</code></pre></div>
<p>We certainly <em>can</em> call it from JavaScript, and it works:</p>
<div class='highlight'><pre><code class='javascript'><span class='nx'>$data</span><span class='p'>.</span><span class='nx'>cljs$core$IIndexed$_nth$arity$2</span><span class='p'>(</span><span class='nx'>$data</span><span class='p'>,</span> <span class='mi'>0</span><span class='p'>)</span>
</code></pre></div>
<p>but it’s not very user-friendly to say the least.</p>
<p>Fortunately JavaScript is a very flexible and dynamic language. We can easily extend <code>PersistentVector</code> with helper <code>get</code> method that we need:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='nb'>aset </span><span class='p'>(</span><span class='nb'>aget </span><span class='p'>[]</span> <span class='s'>"__proto__"</span><span class='p'>)</span>
<span class='lineno'>2</span> <span class='s'>"get"</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>index</span><span class='p'>]</span>
<span class='lineno'>4</span> <span class='p'>(</span><span class='nf'>this-as</span> <span class='nv'>this</span>
<span class='lineno'>5</span> <span class='p'>(</span><span class='nb'>nth </span><span class='nv'>this</span> <span class='nv'>index</span><span class='p'>))))</span>
</code></pre></div>
<p>We use empty vector <code>[]</code> in <em>line 1</em> to get to <code>PersistentVector</code>’s <em>prototype</em>. And add a new method <code>get</code> that takes an <code>index</code> and returns element at that index for <code>this</code>. <code>this-as</code> in <em>line 4</em> gives us access to <code>this</code> from ClojureScript.</p>
<p>Now we can change our <a href='http://knockoutjs.com/'>KO</a> bindings in HTML file to use our new <code>get</code> method:</p>
<div class='highlight'><pre><code class='html'><span class='nt'><td</span> <span class='na'>data-bind=</span><span class='s'>"text: $data.get(0)"</span><span class='nt'>></td></span>
<span class='nt'><td</span> <span class='na'>data-bind=</span><span class='s'>"text: $data.get(1)"</span><span class='nt'>></td></span>
</code></pre></div>
<p>Not too bad, and rather straight-forward. At this point our Client works again and we can see the table of HTTP headers rendered correctly.</p>
<h2 id='the_hard_part__outer_vector'>The Hard part - Outer vector</h2>
<p>The only piece of data conversion logic we have left is <code>into-array</code> calls that convert our outer <code>vector</code> into JavaScript <code>Array</code>. If we get rid of it - we’ve reached our goal.</p>
<h3 id='why_its_hard'>Why it’s hard</h3>
<p>But this is where it becomes tricky. The problem is that we use <a href='http://knockoutjs.com/'>KO</a>’s <code>foreach</code> binding to iterate over our collection of headers:</p>
<div class='highlight'><pre><code class='clojure'><span class='nv'><tbody</span> <span class='nv'>data-bind=</span><span class='s'>"foreach: $root"</span><span class='nv'>></span>
</code></pre></div>
<p>And <code>foreach</code> expects it to be JavaScript <code>Array</code>. <strong>Period.</strong> No kidding.</p>
<p>I’ve tried to teach <a href='http://knockoutjs.com/'>KO</a> to iterate over custom collection, and <a href='https://groups.google.com/forum/#!topic/knockoutjs/P2XXc9q6k04'>I’ve asked KO group</a>. It’s not possible for now.</p>
<h3 id='should_we_give_up'>Should we give up?</h3>
<p>At this point we still have achieved a tangible improvement: the <em>elements</em> of our collection are not converted anymore, so even when we create a copy of outer <code>vector</code> it is a shallow copy: only the “skeleton” is copied, but elements are reused. And they potentially have a bigger memory footprint.</p>
<p>However there is a saying:</p>
<blockquote>
<p>If the mountain will not come to Muhammad, then Muhammad will go to the mountain.</p>
</blockquote>
<p>In our case: if <code>foreach</code> would not take anything but <code>Array</code> then <code>vector</code> should become an <code>Array</code>. Or at least pretend to be one.</p>
<p>Now how do we pretend? The access pattern from <a href='http://knockoutjs.com/'>KO</a> is to read <code>length</code> property first and then read fields <code>[0], [1] ... [length-1]</code>. We can easily add all these fields to our <code>vector</code> but it defeats the purpose: we would copy all <code>vector</code> elements into fields <code>0</code>, <code>1</code>, etc. It’s no better than <code>into-array</code> call.</p>
<p>If only there was a way to intercept field access calls in JavaScript …</p>
<h3 id='warning_danger_zone'>WARNING: Danger Zone</h3>
<p>To the best of my knowledge there is no portable way to intercept field access in JavaScript. So we’ll have to use <strong>JavaScript “Experimental Features”</strong> that are already implemented by some modern browsers (Firefox and Chrome support what we need), but are not standard yet and may not even be enabled by default.</p>
<p><strong>This is why the rest of this post is not practical [for now] but hopefully still entertaining.</strong></p>
<p>If you are willing to proceed and use Chrome, you must navigate to <a href='chrome://flags'>chrome://flags</a> URL (enter it manually in address bar, Chrome blocks navigation to special pages from random web sites :-), find <strong>“Enable Experimental JavaScript”</strong> feature and enable it.</p>
<h3 id='ecmascript_6_proxy_api'>ECMAScript 6 Proxy API</h3>
<p>ECMAScript 6 draft specifies <a href='http://wiki.ecmascript.org/doku.php?id=harmony:direct_proxies'>Direct Proxies</a> - a mechanism to create a proxies that:</p>
<blockquote>
<p>… enable ES programmers to represent virtualized objects (proxies). In particular, to enable writing generic abstractions that can intercept property access on ES objects.</p>
</blockquote>
<p>Exactly what we need!</p>
<h3 id='wrapping_outer_vector_in_a_proxy'>Wrapping outer vector in a Proxy</h3>
<p>A new helper function <code>vector-as-array</code> makes <code>vector</code> sort-of look like <code>Array</code> in a narrow sense required by <a href='http://knockoutjs.com/'>KO</a>. This is by no means a full proxy to emulate <code>Arrays</code>, just a bare minimum required for our purposes.</p>
<p><a href='http://knockoutjs.com/'>KO</a> only needs 2 things from <code>Array</code>: <code>length</code> field and <code>0</code>, <code>1</code>, etc. fields. Well, we can give it what it wants:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'> 1</span> <span class='p'>(</span><span class='kd'>defn </span><span class='nv'>vector-as-array</span>
<span class='lineno'> 2</span> <span class='s'>"Creates JS Proxy around Clojure vector to make it look like JS array,</span>
<span class='lineno'> 3</span> <span class='s'> without copying data"</span>
<span class='lineno'> 4</span> <span class='p'>[</span><span class='nv'>v</span><span class='p'>]</span>
<span class='lineno'> 5</span> <span class='p'>(</span><span class='nf'>.create</span> <span class='nv'>js/Proxy</span>
<span class='lineno'> 6</span> <span class='p'>(</span><span class='nf'>js-obj</span>
<span class='lineno'> 7</span> <span class='s'>"get"</span>
<span class='lineno'> 8</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>_</span> <span class='nv'>prop</span><span class='p'>]</span>
<span class='lineno'> 9</span> <span class='p'>(</span><span class='nf'>case</span> <span class='nv'>prop</span>
<span class='lineno'>10</span> <span class='s'>"length"</span> <span class='p'>(</span><span class='nb'>count </span><span class='nv'>v</span><span class='p'>)</span>
<span class='lineno'>11</span> <span class='p'>(</span><span class='nb'>get </span><span class='nv'>v</span> <span class='nv'>prop</span><span class='p'>)))</span>
<span class='lineno'>12</span> <span class='s'>"getPropertyDescriptor"</span>
<span class='lineno'>13</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>obj</span> <span class='nv'>prop</span><span class='p'>]</span>
<span class='lineno'>14</span> <span class='p'>(</span><span class='nf'>.getOwnPropertyDescriptor</span> <span class='nv'>js/Object</span> <span class='nv'>js/Array</span> <span class='nv'>prop</span><span class='p'>)))))</span>
</code></pre></div>
<p>The <code>get</code> function <em>(lines 7-11)</em> checks if accessed property name is <code>length</code> and returns <code>count</code> of <code>vector</code> <code>v</code> if it is <em>(line 10)</em>. Otherwise <em>(line 11)</em> it simply delegates to <code>vector</code>’s <code>get</code> function to fetch individual element.</p>
<p>The <code>getPropertyDescriptor</code> <em>(lines 12-14)</em> is required for when <a href='http://knockoutjs.com/'>KO</a> does “reflection” on our “array” (like checking if “length” property exists) and simply delegates calls to <code>js/Array</code>.</p>
<h3 id='the_finish_line'>The Finish Line</h3>
<p>The last change is to modify <code>observable-ref</code> once again, this time replacing <code>into-array</code> calls with our newly-created <code>vector-as-array</code> function:</p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='kd'>defn </span><span class='nv'>observable-ref</span> <span class='p'>[</span><span class='nv'>r</span><span class='p'>]</span>
<span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>state</span> <span class='p'>(</span><span class='nf'>ko/observable</span> <span class='p'>(</span><span class='nf'>vector-as-array</span> <span class='o'>@</span><span class='nv'>r</span><span class='p'>))]</span>
<span class='p'>(</span><span class='nf'>add-watch</span> <span class='nv'>r</span> <span class='nv'>state</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>obs</span> <span class='nv'>_</span> <span class='nv'>_</span> <span class='nv'>new</span><span class='p'>]</span> <span class='p'>(</span><span class='nf'>obs</span> <span class='p'>(</span><span class='nf'>vector-as-array</span> <span class='nv'>new</span><span class='p'>))))</span>
<span class='nv'>state</span><span class='p'>))</span>
</code></pre></div>
<p>And it’s hard to believe, but <strong>IT WORKS!</strong></p>
<h2 id='conclusion'>Conclusion</h2>
<p>Here is a quick summary of our accomplishments:</p>
<ul>
<li>We’ve marshaled Clojure data structure created on the Server all the way to the Client using <a href='https://github.com/edn-format/edn'>edn encoding</a> which is a natural textual representation for Clojure data.</li>
<li>We’ve received this data on the client using core.async to make our code look sequential</li>
<li>Then we’ve <a href='https://github.com/edn-format/edn'>edn-decoded</a> response and called regular Clojure functions on received data to extract and sort the headers</li>
<li>We’ve animated our HTML page by manipulating only Clojure data structures</li>
<li>Finally we’ve created <a href='http://knockoutjs.com/'>KO</a> bindings directly to Clojure data without EVER converting it to JavaScript.</li>
</ul>
<p>Granted, the final step required sacrifices (using experimental JavaScript features), and this is why I’ve mentioned above that it depends on reader’s point of view if final goal was achieved or not.</p>
<p><strong>I’ve certainly achieved my goal of learning new interesting technologies and I had lots of fun along the way!</strong></p>
<p><em>And if anyone ever reads this, I hope you did too :-).</em></p>
<h2 id='_source_code'><a name='src'> </a> Source code</h2>
<p>Full source code <a href='https://github.com/Dimagog/dimagog.github.io/tree/ClojureAllTheWay4'>can be found on GitHub</a>.</p>
<p>If you want to build and run it locally, execute:</p>
<pre><code>git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay4 --single-branch ClojureAllTheWay4
cd ClojureAllTheWay4
lein ring server</code></pre>Clojure All The Way - Knockout2013-07-22T00:00:00-07:00http://dimagog.github.io/blog/clojure/clojurescript/2013/07/22/clojure-all-the-way-knockout<p>This is a third article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).</p>
<p>This time we’ll continue building on <a href='/blog/clojure/clojurescript/2013/07/18/clojure-all-the-way-the-client'>previous post</a> and will integrate ClojureScript with <a href='http://knockoutjs.com/'>Knockout.js</a>.</p>
<h2 id='where_we_are'>Where we are</h2>
<p>We start where we left off in the <a href='/blog/clojure/clojurescript/2013/07/18/clojure-all-the-way-the-client'>previous post</a>: we have a Server that implements <code>echo</code> HTTP API and calling it returns <a href='https://github.com/edn-format/edn'>edn-encoded</a> data.</p>
<p>And we have a Client capable of calling this API, receiving data, and <a href='https://github.com/edn-format/edn'>edn-decoding</a> it. But it displays the data (headers of our HTTP request) in a very primitive way by simply setting text of DOM element.</p>
<h2 id='our_goal__clojurescript__knockout'>Our Goal = ClojureScript + Knockout</h2>
<p>We’d like to nicely format received headers as HTML table. We can of cause write ClojureScript code to build HTML for it using <a href='https://github.com/weavejester/hiccup'>Hiccup</a>-like library. But where is fun in that!</p>
<p>It would be more entertaining to use some JavaScript data-binding library. First, because I like <a href='http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller'>MVC</a>-based approach. But more importantly because it will allow us to explore ClojureScript integration with existing JavaScript framework.</p>
<p>We’ll use <a href='http://knockoutjs.com/'>Knockout.js</a> <em>(abbreviated <a href='http://knockoutjs.com/'>KO</a> for the rest of this post)</em> as it happens to be my favorite <a href='http://addyosmani.com/blog/understanding-mvvm-a-guide-for-javascript-developers/'>MVVM</a> framework for JavaScript. And also <a href='http://knockoutjs.com/'>KO</a> claims that it is very smart and highly optimized in minimizing DOM modifications, so I expect it to be efficient as well.</p>
<blockquote>
<p>There is an excellent 20-min <a href='http://channel9.msdn.com/Events/MIX/MIX11/FRM08'>Knockout.js demo video</a> on <a href='http://knockoutjs.com/'>KO</a> web site. If you completely new to <a href='http://knockoutjs.com/'>KO</a> I’d recommend to watch it first. It will make material below easier to understand, plus it’s a truly <a href='http://channel9.msdn.com/Events/MIX/MIX11/FRM08'>masterfully-done demo</a>.</p>
</blockquote>
<h2 id='naive_knockout_integration'>Naive Knockout integration</h2>
<p>Let’s see what is the minimal effort required to integrate our code with <a href='http://knockoutjs.com/'>KO</a>, and then we’ll improve on it.</p>
<h3 id='preparing_input_data'>Preparing input data</h3>
<p>First we need to massage our data: headers are represented as a <code>map</code> but <a href='http://knockoutjs.com/'>KO</a> requires it to be array’ish collection if we want to use its <code>foreach</code> binding. Easy enough: <code>map</code> converted to <code>seq</code> is actually a sequence of <code>vectors</code>, each <code>vector</code> being a key-value tuple. While we at it let’s sort it by header name as well:</p>
<div class='highlight'><pre><code class='clojure'> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>headers</span> <span class='p'>(</span><span class='nf'>->></span> <span class='s'>"/api/echo"</span> <span class='nv'>get-edn</span> <span class='nv'><!</span> <span class='ss'>:headers</span> <span class='p'>(</span><span class='nb'>sort-by </span><span class='nv'>first</span><span class='p'>))]</span>
</code></pre></div>
<p>Calling <code>sort-by</code> is the only addition to our previous code. It converts <code>map</code> to <code>seq</code> and also sorts it by <code>first</code> element of each tuple, i.e. by header name.</p>
<p>The last step is converting Clojure data to plain JavaScript data that <a href='http://knockoutjs.com/'>KO</a> understands. We do it by using <code>clj->js</code> function that will recursively convert our Clojure <code>vector</code> of <code>vectors</code> into JavaScript <code>Array</code> of <code>Arrays</code>:</p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='nf'>clj->js</span> <span class='nv'>headers</span><span class='p'>)</span>
</code></pre></div>
<p>I know the whole premise of the series is to <strong>avoid</strong> such explicit conversion. Please bear with me, we’ll deal with it in following sections.</p>
<h3 id='adding_knockout_bindings_to_html'>Adding Knockout bindings to HTML</h3>
<p><em>Note: Code samples include only interesting parts, <a href='#src'>full source code is available below</a>.</em></p>
<p>We need to change our <code>default.html</code> as follows (only relevant parts are shown):</p>
<div class='highlight'><pre><code class='html'><span class='lineno'> 1</span> <span class='nt'><head></span>
<span class='lineno'> 2</span> <span class='nt'><script </span><span class='na'>src=</span><span class='s'>"http://ajax.aspnetcdn.com/ajax/knockout/knockout-2.2.1.js"</span><span class='nt'>></script></span>
<span class='lineno'> 3</span> <span class='nt'><script </span><span class='na'>defer</span> <span class='na'>src=</span><span class='s'>"app.js"</span><span class='nt'>></script></span>
<span class='lineno'> 4</span> <span class='nt'></head></span>
<span class='lineno'> 5</span> <span class='nt'><body></span>
<span class='lineno'> 6</span> <span class='nt'><div</span> <span class='na'>id=</span><span class='s'>"content"</span><span class='nt'>></span>
<span class='lineno'> 7</span> <span class='nt'><table></span>
<span class='lineno'> 8</span> <span class='nt'><thead><tr><th></span>Header<span class='nt'></th><th></span>Value<span class='nt'></th></tr></thead></span>
<span class='lineno'> 9</span> <span class='nt'><tbody</span> <span class='na'>data-bind=</span><span class='s'>"foreach: $root"</span><span class='nt'>></span>
<span class='lineno'>10</span> <span class='nt'><tr></span>
<span class='lineno'>11</span> <span class='nt'><td</span> <span class='na'>data-bind=</span><span class='s'>"text: $data[0]"</span><span class='nt'>></td></span>
<span class='lineno'>12</span> <span class='nt'><td</span> <span class='na'>data-bind=</span><span class='s'>"text: $data[1]"</span><span class='nt'>></td></span>
<span class='lineno'>13</span> <span class='nt'></tr></span>
<span class='lineno'>14</span> <span class='nt'></tbody></span>
<span class='lineno'>15</span> ...
</code></pre></div>
<p>First we add reference to <a href='http://knockoutjs.com/'>KO</a> itself in <em>line 2</em>. Then we create a table <em>(line 7)</em> with static headers <em>(line 8)</em> and <a href='http://knockoutjs.com/'>KO</a>-bound body <em>(line 9)</em>. The <code>foreach</code> in <em>line 9</em> iterates over <code>$root</code> binding <em>(not explained yet)</em> expecting it to be an array and replicates its HTML body for each element while binding <code>$data</code> to it.</p>
<p>In our case each element is a key-value tuple for each header represented as a 2-element array. So <code>td</code>s in <em>lines 11-12</em> simply extract first element for the header name and second for the value.</p>
<h3 id='applying_root_knockout_binding'>Applying root Knockout binding</h3>
<p>Now to connect our ClojureScript data to <a href='http://knockoutjs.com/'>KO</a> bindings in HTML we just need to call <code>ko/applyBindings</code>. And our final code for this step looks like this:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='nf'>go</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>headers</span> <span class='p'>(</span><span class='nf'>->></span> <span class='s'>"/api/echo"</span> <span class='nv'>get-edn</span> <span class='nv'><!</span> <span class='ss'>:headers</span> <span class='p'>(</span><span class='nb'>sort-by </span><span class='nv'>first</span><span class='p'>))]</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nf'>ko/applyBindings</span> <span class='p'>(</span><span class='nf'>clj->js</span> <span class='nv'>headers</span><span class='p'>))))</span>
</code></pre></div>
<p>We obtain the data and massage it in <em>line 2</em>. Then convert Clojure data to JavaScript using <code>clj->js</code> and pass it as a root binding to <a href='http://knockoutjs.com/'>KO</a> using <code>ko/applyBindings</code> in <em>line 3</em>.</p>
<p><strong>And viola!</strong> Our headers are shown in a nice-looking table (with little CSS help):</p>
<p><img alt='table1' src='/assets/table1.png' /></p>
<p>Now readers familiar with <a href='http://knockoutjs.com/'>KO</a> are probably shaking their head in disgust: no observables, and <code>applyBindings</code> should be called only once (not from some potentially reusable <code>go</code>-routine).</p>
<p>And let’s not forget about explicit <code>clj->js</code> call.</p>
<p><strong>Fear not! we’ll take care of it right away!</strong></p>
<h2 id='using_computed_observables'>Using computed observables</h2>
<p>My first (but not final) improvement attempt is to use <a href='http://knockoutjs.com/'>KO</a>’s <a href='http://knockoutjs.com/documentation/computedObservables.html'>computed observable</a> to do automatic <code>clj->js</code> translation.</p>
<blockquote>
<p><a href='http://knockoutjs.com/'>KO</a> documentation says that <a href='http://knockoutjs.com/documentation/extenders.html'>extenders</a> could be a better fit, but I’m more familiar with computed observables.</p>
</blockquote>
<p>First we create a helper function to build our <a href='http://knockoutjs.com/documentation/computedObservables.html'>computed observable</a>:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='kd'>defn </span><span class='nv'>observable</span> <span class='p'>[</span><span class='nv'>val</span><span class='p'>]</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='nf'>ko/computed</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>state</span> <span class='p'>(</span><span class='nf'>ko/observable</span> <span class='p'>(</span><span class='nf'>clj->js</span> <span class='nv'>val</span><span class='p'>))]</span>
<span class='lineno'>4</span> <span class='p'>(</span><span class='nf'>js-obj</span>
<span class='lineno'>5</span> <span class='s'>"read"</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[]</span> <span class='p'>(</span><span class='nf'>state</span><span class='p'>))</span>
<span class='lineno'>6</span> <span class='s'>"write"</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>new</span><span class='p'>]</span> <span class='p'>(</span><span class='nf'>state</span> <span class='p'>(</span><span class='nf'>clj->js</span> <span class='nv'>new</span><span class='p'>)))))))</span>
</code></pre></div>
<p><code>ko/computed</code> observable <em>(line 2)</em> takes a JavaScript object created in <em>line 4</em> that specifies <code>read</code> and <code>write</code> functions. The actual state is kept in regular <code>ko/observable</code> created in <em>line 3</em>.</p>
<p><a href='http://knockoutjs.com/'>KO</a> observables are functions. Calling it with no arguments returns current value and calling it with one arguments sets its value. So <code>(state)</code> in <em>line 5</em> gets the current <code>state</code> and <code>(state (clj->js new))</code> in <em>line 6</em> sets it (after converting <code>new</code> to JavaScript representation).</p>
<p>Now our main code looks like this:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='k'>def </span><span class='nv'>view-model</span> <span class='p'>(</span><span class='nf'>observable</span> <span class='p'>[]))</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='nf'>ko/applyBindings</span> <span class='nv'>view-model</span><span class='p'>)</span>
<span class='lineno'>3</span>
<span class='lineno'>4</span> <span class='p'>(</span><span class='nf'>go</span>
<span class='lineno'>5</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>headers</span> <span class='p'>(</span><span class='nf'>->></span> <span class='s'>"/api/echo"</span> <span class='nv'>get-edn</span> <span class='nv'><!</span> <span class='ss'>:headers</span> <span class='p'>(</span><span class='nb'>sort-by </span><span class='nv'>first</span><span class='p'>))]</span>
<span class='lineno'>6</span> <span class='p'>(</span><span class='nf'>view-model</span> <span class='nv'>headers</span><span class='p'>)))</span>
</code></pre></div>
<p>We create our <code>view-model</code> in <em>line 1</em> using our <code>observable</code> helper, and set it to empty vector first. Then we bind it in <em>line 2</em>, this needs to be done only once. And when we obtain our <code>headers</code> in <em>line 5</em> we set <code>view-model</code> observalble in <em>line 6</em> (remember - observables are functions). This in turn triggers DOM update.</p>
<p><em>Note: using global <code>vars</code> like <code>view-model</code> is not recommended in good Clojure code, but will do for our sample code.</em></p>
<p><strong>Great!</strong> <code>clj->js</code> conversion is nowhere to be seen in our main code and we adhere to good <a href='http://knockoutjs.com/'>KO</a> practices, <strong>but …</strong></p>
<h3 id='whats_wrong_with_it'>What’s wrong with it?</h3>
<p>First, getting/setting <code>view-model</code> by calling it as a function is not very <em>Clojury</em>.</p>
<p>Second, and more importantly, if we read <code>(view-model)</code> we’ll get JavaScript-converted objects, not the original. The original is “lost in translation”. Which means that if we want to modify our view model in ClojureScript we need to keep it separately somewhere.</p>
<p>And where do we usually keep changing data in Clojure? <strong>In <code>refs</code> of cause!</strong> At this point a light bulb should begin hovering above your head: the <code>refs</code> in Clojure have watchers, so they can notify whoever is interested when they change.</p>
<p><code>refs</code> sound suspiciously familiar to <code>ko/observables</code>: both track changing state and both send notifications when change happens. Can we somehow bridge the two? We certainly can!</p>
<h2 id='using_observable_refs'>Using Observable refs</h2>
<p>Let’s create a helper function to create a <code>ko/observable</code> tracking Clojure <code>ref</code>:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='kd'>defn </span><span class='nv'>observable-ref</span> <span class='p'>[</span><span class='nv'>r</span><span class='p'>]</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>state</span> <span class='p'>(</span><span class='nf'>ko/observable</span> <span class='p'>(</span><span class='nf'>clj->js</span> <span class='o'>@</span><span class='nv'>r</span><span class='p'>))]</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nf'>add-watch</span> <span class='nv'>r</span> <span class='nv'>state</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>obs</span> <span class='nv'>_</span> <span class='nv'>_</span> <span class='nv'>new</span><span class='p'>]</span> <span class='p'>(</span><span class='nf'>obs</span> <span class='p'>(</span><span class='nf'>clj->js</span> <span class='nv'>new</span><span class='p'>))))</span>
<span class='lineno'>4</span> <span class='nv'>state</span><span class='p'>))</span>
</code></pre></div>
<p><code>observable-ref</code> takes a <code>ref</code> as parameter <code>r</code> (the <code>ref</code> itself, not its value!). We create a <code>ko\observable</code> named <code>state</code> in <em>line 2</em> with initial value of <code>r</code> converted to JavaScript. Then in <em>line 3</em> we add a watcher for the <code>r</code>.</p>
<p>The watcher <code>fn</code> takes 4 parameters, but we only care about 2: <code>obs</code> is our observable <code>state</code> and <code>new</code> is the new value of the <code>ref</code> <code>r</code>. Then we simply convert <code>new</code> value to JavaScript representation and put it into <code>obs</code>.</p>
<p>And finally we return <code>state</code> observable from the function so that it can be passed to <a href='http://knockoutjs.com/'>KO</a>.</p>
<p>Our main code now changes to this:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='k'>def </span><span class='nv'>view-model</span> <span class='p'>(</span><span class='nf'>atom</span> <span class='p'>[]))</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='nf'>ko/applyBindings</span> <span class='p'>(</span><span class='nf'>observable-ref</span> <span class='nv'>view-model</span><span class='p'>))</span>
<span class='lineno'>3</span>
<span class='lineno'>4</span> <span class='p'>(</span><span class='nf'>go</span>
<span class='lineno'>5</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>headers</span> <span class='p'>(</span><span class='nf'>->></span> <span class='s'>"/api/echo"</span> <span class='nv'>get-edn</span> <span class='nv'><!</span> <span class='ss'>:headers</span> <span class='p'>(</span><span class='nb'>sort-by </span><span class='nv'>first</span><span class='p'>))]</span>
<span class='lineno'>6</span> <span class='p'>(</span><span class='nf'>reset!</span> <span class='nv'>view-model</span> <span class='nv'>headers</span><span class='p'>)))</span>
</code></pre></div>
<p>Notice that <code>view-model</code> created in <em>line 1</em> is now a Clojure <code>ref</code> (in this case an <code>atom</code>). And we don’t even save the observable returned by our <code>observable-ref</code> helper, we just pass it directly to <code>ko/applyBindings</code> in <em>line 2</em>.</p>
<p>And modifying our <code>view-model</code> <em>(line 6)</em> is as simple as modifying any other Clojure <code>atom</code>, yet DOM immediately reflects the changes!</p>
<p><strong>Isn’t it nice!</strong> Let’s have some fun with it …</p>
<h2 id='animating_our_table'>Animating our table</h2>
<p>Just for fun, let’s pretend that our Internet connection is very slow and headers are received one by one, slooowly. We can emulate this by <code>conj</code>ing headers to <code>view-model</code> one at a time with delay:</p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='nf'>go</span>
<span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>headers</span> <span class='p'>(</span><span class='nf'>->></span> <span class='s'>"/api/echo"</span> <span class='nv'>get-edn</span> <span class='nv'><!</span> <span class='ss'>:headers</span> <span class='p'>(</span><span class='nb'>sort-by </span><span class='nv'>first</span><span class='p'>))]</span>
<span class='p'>(</span><span class='nb'>doseq </span><span class='p'>[</span><span class='nv'>h</span> <span class='nv'>headers</span><span class='p'>]</span>
<span class='p'>(</span><span class='nf'><!</span> <span class='p'>(</span><span class='nf'>timeout</span> <span class='mi'>500</span><span class='p'>))</span>
<span class='p'>(</span><span class='nf'>swap!</span> <span class='nv'>view-model</span> <span class='nb'>conj </span><span class='nv'>h</span><span class='p'>)))</span>
</code></pre></div>
<p>Then we’ll delay for a second and pretend that terrible computer virus eats our headers one at a time:</p>
<div class='highlight'><pre><code class='clojure'> <span class='p'>(</span><span class='nf'><!</span> <span class='p'>(</span><span class='nf'>timeout</span> <span class='mi'>1000</span><span class='p'>))</span>
<span class='p'>(</span><span class='nf'>while</span> <span class='p'>(</span><span class='nb'>seq </span><span class='o'>@</span><span class='nv'>view-model</span><span class='p'>)</span>
<span class='p'>(</span><span class='nf'>swap!</span> <span class='nv'>view-model</span> <span class='nv'>rest</span><span class='p'>)</span>
<span class='p'>(</span><span class='nf'><!</span> <span class='p'>(</span><span class='nf'>timeout</span> <span class='mi'>500</span><span class='p'>)))</span>
</code></pre></div>
<p>Now delay for another second to keep the suspense, and finally restore our headers table to calm down the user:</p>
<div class='highlight'><pre><code class='clojure'> <span class='p'>(</span><span class='nf'><!</span> <span class='p'>(</span><span class='nf'>timeout</span> <span class='mi'>1000</span><span class='p'>))</span>
<span class='p'>(</span><span class='nf'>reset!</span> <span class='nv'>view-model</span> <span class='nv'>headers</span><span class='p'>)</span>
</code></pre></div>
<p><strong>Pure Clojure data manipulation code, yet <a href='http://knockoutjs.com/'>KO</a> faithfully reflects all our changes in the DOM!</strong></p>
<p><strong>Mission accomplished!</strong></p>
<p>Or is it? <em>I have tentative plans for another post in the series, but it requires a little more research.</em> I’ll keep you posted.</p>
<p><strong>UPDATE:</strong> Keeping you posted, here is the <a href='/blog/clojure/clojurescript/2013/07/31/clojure-all-the-way-deep-knockout'>next post</a>.</p>
<h2 id='one_last_thing_optimized_builds'>One last thing: optimized builds</h2>
<p>Unlike Steve Job’s famous “one last things” this one is boring, <strong>but important</strong>. If we try to build our ClojureScript code with <code>:advanced</code> optimization it will not work. This is because Google Closure compiler minifies all names in produced .js file including names like <code>ko</code> and <code>observable</code>. Certainly not what we want.</p>
<p>The solution is to add so-called <a href='https://developers.google.com/closure/compiler/docs/api-tutorial3#externs'>externs file</a> and feed it to Google Closure compiler, so it would know which names it is not supposed to touch.</p>
<p>Conceptually the <a href='https://developers.google.com/closure/compiler/docs/api-tutorial3#externs'>externs file</a> is similar to C/C++ .h headers files, or .asmmeta files for C#. Ours is called <code>ko.externs.js</code> and looks like this:</p>
<div class='highlight'><pre><code class='javascript'><span class='kd'>var</span> <span class='nx'>ko</span> <span class='o'>=</span> <span class='p'>{};</span>
<span class='nx'>ko</span><span class='p'>.</span><span class='nx'>applyBindings</span> <span class='o'>=</span> <span class='kd'>function</span><span class='p'>()</span> <span class='p'>{};</span>
<span class='nx'>ko</span><span class='p'>.</span><span class='nx'>observable</span> <span class='o'>=</span> <span class='kd'>function</span><span class='p'>()</span> <span class='p'>{};</span>
<span class='nx'>ko</span><span class='p'>.</span><span class='nx'>computed</span> <span class='o'>=</span> <span class='kd'>function</span><span class='p'>()</span> <span class='p'>{};</span>
</code></pre></div>
<p>Just declarations, no code. And after adding it to <code>project.clj</code>:</p>
<div class='highlight'><pre><code class='clojure'><span class='ss'>:externs</span> <span class='p'>[</span><span class='s'>"cljs/ko.externs.js"</span><span class='p'>]</span>
</code></pre></div>
<p>… we can build our optimized code, and it will work correctly:</p>
<pre><code>lein cljsbuild once opt</code></pre>
<p>If you are wondering how big is produced optimized <code>app.js</code> file (I sure was), it is <code>118'871 bytes</code>.</p>
<h2 id='_source_code'><a name='src'> </a> Source code</h2>
<p>Full source code <a href='https://github.com/Dimagog/dimagog.github.io/tree/ClojureAllTheWay3'>can be found on GitHub</a>.</p>
<p>If you want to build and run it locally, execute:</p>
<pre><code>git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay3 --single-branch ClojureAllTheWay3
cd ClojureAllTheWay3
lein ring server</code></pre>Clojure All The Way - The Client2013-07-18T00:00:00-07:00http://dimagog.github.io/blog/clojure/clojurescript/2013/07/18/clojure-all-the-way-the-client<p>This is a second article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).</p>
<p>The <a href='/blog/clojure/2013/07/14/clojure-all-the-way-the-server'>previous post</a> dealt with the Server part, now let’s look at the Client side.</p>
<h2 id='starting_point'>Starting Point</h2>
<p>We start where we left off in the <a href='/blog/clojure/2013/07/14/clojure-all-the-way-the-server'>previous post</a>: we have a Server that implements <code>echo</code> HTTP API and calling it returns <a href='https://github.com/edn-format/edn'>edn-encoded</a> data.</p>
<p>And we have a Client capable of calling this API and receiving data, but it still treats it as text, and not as structured Clojure data. The client was actually explained not in the last post, but in <a href='/blog/clojure/clojurescript/2013/07/12/making-http-requests-from-clojurescript-with-core.async'>post before that</a>.</p>
<h2 id='first_small_improvement_to_get_function'>First, small improvement to GET function</h2>
<p><em>Note: Code samples include only interesting parts, <a href='#src'>full source code is available below</a>.</em></p>
<p>The <code>GET</code> function in previous <a href='/blog/clojure/clojurescript/2013/07/12/making-http-requests-from-clojurescript-with-core.async'>client post</a> was implemented using <code>(>! ch ...)</code> inside <code>go</code>-routine <em>(line 6)</em>:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='kd'>defn </span><span class='nv'>GET</span> <span class='p'>[</span><span class='nv'>url</span><span class='p'>]</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>ch</span> <span class='p'>(</span><span class='nf'>chan</span> <span class='mi'>1</span><span class='p'>)]</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nf'>xhr/send</span> <span class='nv'>url</span>
<span class='lineno'>4</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>event</span><span class='p'>]</span>
<span class='lineno'>5</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>res</span> <span class='p'>(</span><span class='nb'>-> </span><span class='nv'>event</span> <span class='nv'>.-target</span> <span class='nv'>.getResponseText</span><span class='p'>)]</span>
<span class='lineno'>6</span> <span class='p'>(</span><span class='nf'>go</span> <span class='p'>(</span><span class='nf'>>!</span> <span class='nv'>ch</span> <span class='nv'>res</span><span class='p'>)</span>
<span class='lineno'>7</span> <span class='p'>(</span><span class='nf'>close!</span> <span class='nv'>ch</span><span class='p'>)))))</span>
<span class='lineno'>8</span> <span class='nv'>ch</span><span class='p'>))</span>
</code></pre></div>
<p>In comments to <a href='/blog/clojure/clojurescript/2013/07/12/making-http-requests-from-clojurescript-with-core.async'>that post</a> Alexander Solovyov suggested using <code>put!</code> instead. It’s a good idea (thanks Alexander!) since we don’t care when channel write completes. Here is the change:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='kd'>defn </span><span class='nv'>GET</span> <span class='p'>[</span><span class='nv'>url</span><span class='p'>]</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>ch</span> <span class='p'>(</span><span class='nf'>chan</span> <span class='mi'>1</span><span class='p'>)]</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nf'>xhr/send</span> <span class='nv'>url</span>
<span class='lineno'>4</span> <span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>event</span><span class='p'>]</span>
<span class='lineno'>5</span> <span class='p'>(</span><span class='nf'>put!</span> <span class='nv'>ch</span> <span class='p'>(</span><span class='nb'>-> </span><span class='nv'>event</span> <span class='nv'>.-target</span> <span class='nv'>.getResponseText</span><span class='p'>))</span>
<span class='lineno'>6</span> <span class='p'>(</span><span class='nf'>close!</span> <span class='nv'>ch</span><span class='p'>)))</span>
<span class='lineno'>7</span> <span class='nv'>ch</span><span class='p'>))</span>
</code></pre></div>
<p>the inner <code>let</code> and <code>go</code> blocks are gone and we use <code>put!</code> in <em>line 5</em> instead.</p>
<p>Now back to our main topic …</p>
<h2 id='getting_clojure_data_from_server_response'>Getting Clojure data from Server response</h2>
<p>Let’s add a new function <code>get-edn</code> that uses <code>GET</code> function above to get data from the Server, and then converts it to Clojure data structures, i.e. <a href='https://github.com/edn-format/edn'>edn-decodes</a> it. It is actually very simple, all we need to do is call <code>read-string</code> function from <code>cljs.reader</code> namespace.</p>
<p>And in addition to being useful for our purposes it will also show how async functions compose.</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='kd'>defn </span><span class='nv'>get-edn</span> <span class='p'>[</span><span class='nv'>url</span><span class='p'>]</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='nf'>go</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nb'>-> </span><span class='p'>(</span><span class='nf'>GET</span> <span class='nv'>url</span><span class='p'>)</span>
<span class='lineno'>4</span> <span class='nv'><!</span>
<span class='lineno'>5</span> <span class='nv'>read-string</span><span class='p'>)))</span>
</code></pre></div>
<p>Let’s look at it from inside out. In <em>line 3</em> we call our <code>GET</code> that returns a <code>channel</code> that will eventually contain result (text). In <em>line 4</em> we read the value from this <code>channel</code> potentially “parking” this activity, but not blocking. This is why we need this code wrapped in <code>go</code>-routine. And then in <em>line 5</em> we pass returned text to <code>read-string</code> that <a href='https://github.com/edn-format/edn'>edn-decodes</a> it and returns Clojure data structure (let’s call this value <code>result</code>).</p>
<p>But what happens next? How is <code>result</code> propagated to the caller of <code>get-edn</code>? To answer that we need to understand what <code>get-edn</code> returns. Following normal Clojure rules <code>get-edn</code> returns the value returned by <code>go</code> form. This is a <code>channel</code>, now who writes to this <code>channel</code>? The <code>go</code> form does. Here is how <code>go</code> works:</p>
<ul>
<li>first it creates a <code>channel</code> and promptly returns it to the caller</li>
<li><em>eventually</em> it evaluates all statement inside, potentially “parking” this activity when <code><!</code> and <code>>!</code> forms are encountered</li>
<li>it writes the value of the last statement to the <code>channel</code>, <code>close!</code>es the <code>channel</code> and completes.</li>
</ul>
<p>In our case the last and only statement inside <code>go</code> is our “conveyor” producing <code>result</code> defined two paragraphs ago, i.e. decoded response. The <code>go</code> form writes <code>result</code> to the <code>channel</code> it has created and this is how it gets to <code>get-edn</code> caller.</p>
<p><strong>And this is how async functions compose!</strong> Notice that we didn’t even have to create a <code>channel</code> explicitly in <code>get-edn</code>!</p>
<p>The reason we had to create <code>channel</code> in <code>GET</code> function is because we were converting callback-based API to asynchronous one. The caller of our <code>fn</code> in there was not async-aware, and actually ignored the return value. So we had to propagate it to <code>GET</code> caller thru manually-created <code>channel</code> <code>ch</code>.</p>
<h2 id='calling_getedn_function'>Calling get-edn function</h2>
<p>To prove to ourselves that <code>get-edn</code> returns Clojure map let’s call some function on it before displaying it, for example let’s extract only <code>:headers</code>:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='nf'>go</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='nf'>dom/set-text!</span> <span class='p'>(</span><span class='nf'>sel1</span> <span class='err'>:</span><span class='o'>#</span><span class='nv'>log</span><span class='p'>)</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nb'>-> </span><span class='p'>(</span><span class='nf'>get-edn</span> <span class='s'>"/api/echo"</span><span class='p'>)</span>
<span class='lineno'>4</span> <span class='nv'><!</span>
<span class='lineno'>5</span> <span class='ss'>:headers</span><span class='p'>))</span>
</code></pre></div>
<p>The whole block is wrapped in <code>go</code> routine because we use <code><!</code> (read channel) form. We call <code>get-edn</code> in <em>line 3</em> that returns a channel, we read from it in <em>line 4</em>, potentially “parking” (but again, not blocking) this activity. Then we call <code>:headers</code> in <em>line 5</em> to get only headers portion of respose proving that result is actually a Clojure map.</p>
<p>And if you run the sample code, you’ll see the headers displayed.</p>
<p>Now this is nice, but the way they are displayed is kind of lame. We’ll do something about it in the <a href='/blog/clojure/clojurescript/2013/07/22/clojure-all-the-way-knockout'>next post</a>, while staying on topic of using Clojure data structures as much as possible.</p>
<h2 id='_source_code'><a name='src'> </a> Source code</h2>
<p>Full source code <a href='https://github.com/Dimagog/dimagog.github.io/tree/ClojureAllTheWay2'>can be found on GitHub</a>.</p>
<p>If you want to build and run it locally, execute:</p>
<pre><code>git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay2 --single-branch ClojureAllTheWay2
cd ClojureAllTheWay2
lein ring server</code></pre>Clojure All The Way - The Server2013-07-14T00:00:00-07:00http://dimagog.github.io/blog/clojure/2013/07/14/clojure-all-the-way-the-server<p>This is a first article in mini-series. The goal of mini-series is to create a Client-Server environment where Clojure data structures are pervasive and we don’t have to deal with JSON and JavaScript objects in Clojure/ClojureScript land (as much as possible).</p>
<p>In this post we’ll develop a Web server in Clojure (using <a href='https://github.com/ring-clojure/ring'>Ring</a> and <a href='https://github.com/weavejester/compojure'>Compojure</a>) that exposes simple API to serve Clojure data structures in their natural text representation (fancy term for this is <a href='https://github.com/edn-format/edn'>edn-encoded</a>).</p>
<h2 id='the_basics_are_not_covered'>The Basics are not covered</h2>
<p>The basics of creating <a href='https://github.com/ring-clojure/ring'>Ring</a>+<a href='https://github.com/weavejester/compojure'>Compojure</a> Web server are covered very well on Interwebs. In this post I’ll only focus on creating API returning <a href='https://github.com/edn-format/edn'>edn-encoded</a> Clojure data.</p>
<h2 id='serving_the_site_itself_static_files'>Serving the site itself (static files)</h2>
<p><em>Note: Code samples include only interesting parts, <a href='#src'>full source code is available below</a>.</em></p>
<p>Let’s start with vanilla web server that simply serves static files:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='nf'>defroutes</span> <span class='nv'>app</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='nf'>GET</span> <span class='s'>"/"</span> <span class='p'>[]</span> <span class='p'>(</span><span class='nf'>file-response</span> <span class='s'>"default.html"</span> <span class='p'>{</span><span class='ss'>:root</span> <span class='s'>"html"</span><span class='p'>}))</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nf'>files</span> <span class='s'>""</span> <span class='p'>{</span><span class='ss'>:root</span> <span class='s'>"html"</span><span class='p'>})</span>
<span class='lineno'>4</span> <span class='p'>(</span><span class='nf'>not-found</span> <span class='s'>"<h1>404 Page not found</h1>"</span><span class='p'>)</span>
<span class='lineno'>5</span> <span class='p'>)</span>
</code></pre></div>
<ul>
<li>line 2 returns (not redirects to) <code>default.html</code> when server root is accessed</li>
<li>line 3 serves static files from “html” directory</li>
<li>line 4 is only hit if static file is not found and returns 404</li>
</ul>
<h2 id='adding_echo_api'>Adding Echo API</h2>
<p>Now let’s add an <code>echo</code> API that simply echoes back the client’s HTTP request.</p>
<p><em>To be precise:</em> upon receiving HTTP <code>GET /api/echo</code> request the server will respond with <code>200 OK</code> and body containing <a href='https://github.com/edn-format/edn'>edn-encoded</a> Clojure representation of request (a map of maps, strings and numbers).</p>
<p>Of cause we are not going to do heavy-lifting ourselves - that’s the whole point of <a href='https://github.com/ring-clojure/ring'>Ring</a>. To paraphrase Apple’s commercials “there is a <em>middleware</em> for that”. And it’s called <code>ring.middleware.format-response</code>. Let’s use it:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='nf'>defroutes</span> <span class='nv'>handler</span>
<span class='lineno'>2</span> <span class='p'>(</span><span class='nf'>GET</span> <span class='s'>"/"</span> <span class='nv'>...</span><span class='p'>)</span>
<span class='lineno'>3</span> <span class='p'>(</span><span class='nf'>GET</span> <span class='s'>"/api/echo"</span> <span class='nv'>request</span>
<span class='lineno'>4</span> <span class='p'>{</span><span class='ss'>:status</span> <span class='mi'>200</span>
<span class='lineno'>5</span> <span class='ss'>:body</span> <span class='nv'>request</span><span class='p'>})</span>
<span class='lineno'>6</span> <span class='nv'>...</span><span class='p'>)</span>
<span class='lineno'>7</span>
<span class='lineno'>8</span> <span class='p'>(</span><span class='k'>def </span><span class='nv'>app</span> <span class='p'>(</span><span class='nb'>-> </span><span class='nv'>handler</span>
<span class='lineno'>9</span> <span class='nv'>wrap-clojure-response</span><span class='p'>))</span>
</code></pre></div>
<p>The <code>app</code> was renamed to <code>handler</code> <em>(line 1)</em> and new <code>app</code> wraps this <code>handler</code> into <code>wrap-clojure-response</code> <em>(lines 8-9)</em>.</p>
<p>The new <a href='https://github.com/weavejester/compojure'>Compojure</a> route <em>(lines 3-5)</em> simply takes a <code>request</code> (a Clojure map) and builds a 200-response from it setting <code>:body</code> to <code>request</code> (still a Clojure map).</p>
<p><strong>And the <code>wrap-clojure-response</code> <em>middleware</em> is the one converting Clojure map <code>:body</code> returned by <code>handler</code> to <a href='https://github.com/edn-format/edn'>edn-encoded</a> textual representation.</strong></p>
<p>There is one problem with this code: <code>:body</code> of <code>request</code> that we are sending back is not a Clojure data structure. It’s a Java object of <code>HttpInput org.eclipse.jetty.server.HttpInput</code> type. That’s not good: it cannot be <a href='https://github.com/edn-format/edn'>edn-encoded</a> and Client won’t be able to decode it anyway. A small tweak removes it from the response we are sending <em>(call to <code>dissoc</code> in line 3)</em>:</p>
<div class='highlight'><pre><code class='clojure'><span class='lineno'>1</span> <span class='p'>(</span><span class='nf'>GET</span> <span class='s'>"/api/echo"</span> <span class='nv'>request</span>
<span class='lineno'>2</span> <span class='p'>{</span><span class='ss'>:status</span> <span class='mi'>200</span>
<span class='lineno'>3</span> <span class='ss'>:body</span> <span class='p'>(</span><span class='nb'>dissoc </span><span class='nv'>request</span> <span class='ss'>:body</span><span class='p'>)})</span>
</code></pre></div>
<h2 id='testing'>Testing</h2>
<p>To test our Server we can use a slightly modified client from my <a href='/blog/clojure/clojurescript/2013/07/12/making-http-requests-from-clojurescript-with-core.async'>earlier post</a>. And sure enough it shows client’s request returned back to us.</p>
<p>But ClojureScript Client still treats response as text (not structured Clojure data). We’ll deal with it in the <a href='/blog/clojure/clojurescript/2013/07/18/clojure-all-the-way-the-client'>next post</a>.</p>
<h2 id='_source_code'><a name='src'> </a> Source code</h2>
<p>Full source code <a href='https://github.com/Dimagog/dimagog.github.io/tree/ClojureAllTheWay1'>can be found on GitHub</a>.</p>
<p>If you want to build and run it locally, execute:</p>
<pre><code>git clone https://github.com/Dimagog/dimagog.github.io.git -b ClojureAllTheWay1 --single-branch ClojureAllTheWay1
cd ClojureAllTheWay1
lein ring server</code></pre>Making HTTP requests from ClojureScript with core.async2013-07-12T23:40:00-07:00http://dimagog.github.io/blog/clojure/clojurescript/2013/07/12/making-http-requests-from-clojurescript-with-core.async<p>I’ve always been a big fan of <code>async/await</code> feature in C# (long before it was publicly available). So, naturally, I was very excited to read the <a href='http://clojure.com/blog/2013/06/28/clojure-core-async-channels.html'>announcement</a> about Clojure <a href='https://github.com/clojure/core.async'>core.async</a> library. Even more so after I’ve learned that it works for ClojureScript as well.</p>
<p>Here is an example of using <a href='https://github.com/clojure/core.async'>core.async</a> to convert callback-based <code>goog.net.XhrIo</code> <code>send</code> API to sequential-looking one.</p>
<h2 id='leningen_config_boring_stuff'>Leningen config (boring stuff)</h2>
<p><em>Note: Code samples include only interesting parts, <a href='#src'>full source code is available below</a>.</em></p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='kd'>defproject </span><span class='nv'>...</span>
<span class='ss'>:repositories</span> <span class='p'>{</span>
<span class='s'>"sonatype-oss-public"</span> <span class='s'>"https://oss.sonatype.org/content/groups/public/"</span>
<span class='p'>}</span>
<span class='ss'>:dependencies</span> <span class='p'>[</span>
<span class='p'>[</span><span class='nv'>org.clojure/clojure</span> <span class='s'>"1.5.1"</span><span class='p'>]</span>
<span class='p'>[</span><span class='nv'>org.clojure/clojurescript</span> <span class='s'>"0.0-1820"</span><span class='p'>]</span>
<span class='p'>[</span><span class='nv'>org.clojure/core.async</span> <span class='s'>"0.1.0-SNAPSHOT"</span><span class='p'>]</span>
<span class='nv'>...</span>
</code></pre></div>
<p>We need Sonatype because it is hosting SNAPSHOTs, and there is no release version of <a href='https://github.com/clojure/core.async'>core.async</a> available yet. Also I’m not using the latest version of ClojureScript because <a href='http://www.lighttable.com/'>Light Table</a> does not like it.</p>
<h2 id='defining_get_function_cool_stuff'>Defining GET function (cool stuff)</h2>
<p>First the <code>ns</code> header:</p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='kd'>ns </span><span class='nv'>app</span>
<span class='p'>(</span><span class='ss'>:require</span>
<span class='p'>[</span><span class='nv'>goog.net.XhrIo</span> <span class='ss'>:as</span> <span class='nv'>xhr</span><span class='p'>]</span>
<span class='p'>[</span><span class='nv'>cljs.core.async</span> <span class='ss'>:as</span> <span class='nv'>async</span> <span class='ss'>:refer</span> <span class='p'>[</span><span class='nv'>chan</span> <span class='nv'>close!</span><span class='p'>]])</span>
<span class='p'>(</span><span class='ss'>:require-macros</span>
<span class='p'>[</span><span class='nv'>cljs.core.async.macros</span> <span class='ss'>:refer</span> <span class='p'>[</span><span class='nv'>go</span> <span class='nv'>alt!</span><span class='p'>]]))</span>
</code></pre></div>
<p>It’s not idiomatic to have all uppercase function names in Clojure, but since this function mimics http GET method we’ll make an exception for it.</p>
<p>The idea is to return a channel from GET function that will have a result of <code>send</code> call when it completes. The callback simply extracts the result and writes it to the channel:</p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='kd'>defn </span><span class='nv'>GET</span> <span class='p'>[</span><span class='nv'>url</span><span class='p'>]</span>
<span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>ch</span> <span class='p'>(</span><span class='nf'>chan</span> <span class='mi'>1</span><span class='p'>)]</span>
<span class='p'>(</span><span class='nf'>xhr/send</span> <span class='nv'>url</span>
<span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>event</span><span class='p'>]</span>
<span class='p'>(</span><span class='k'>let </span><span class='p'>[</span><span class='nv'>res</span> <span class='p'>(</span><span class='nb'>-> </span><span class='nv'>event</span> <span class='nv'>.-target</span> <span class='nv'>.getResponseText</span><span class='p'>)]</span>
<span class='p'>(</span><span class='nf'>go</span> <span class='p'>(</span><span class='nf'>>!</span> <span class='nv'>ch</span> <span class='nv'>res</span><span class='p'>)</span>
<span class='p'>(</span><span class='nf'>close!</span> <span class='nv'>ch</span><span class='p'>)))))</span>
<span class='nv'>ch</span><span class='p'>))</span>
</code></pre></div>
<p>We create channel <code>ch</code> of size 1 (default is 0) because there is no reason to block callback until reader arrives. Instead it writes the response once it becomes available and disappears.</p>
<p>You may think that callback could be simplified as:</p>
<div class='highlight'><pre><code class='clojure'><span class='c1'>; WARNING: Broken code</span>
<span class='p'>(</span><span class='k'>fn </span><span class='p'>[</span><span class='nv'>event</span><span class='p'>]</span>
<span class='p'>(</span><span class='nf'>go</span> <span class='p'>(</span><span class='nf'>>!</span> <span class='nv'>ch</span> <span class='p'>(</span><span class='nb'>-> </span><span class='nv'>event</span> <span class='nv'>.-target</span> <span class='nv'>.getResponseText</span><span class='p'>))</span>
<span class='p'>(</span><span class='nf'>close!</span> <span class='nv'>ch</span><span class='p'>)))))</span>
</code></pre></div>
<p><strong>but that does not work</strong>. This version tries to extract result from <code>event</code> inside <code>go</code> routine which will execute <em>eventually</em>. By the time it runs <code>event</code> does not have the result anymore.</p>
<p>Also it’s a good idea to <code>close!</code> the channel <code>ch</code> after we write the result <code>res</code>, so if anyone tries to read from <code>ch</code> again by mistake, they will get <code>nil</code> returned immediately.</p>
<h2 id='using_get_function_even_cooler_stuff'>Using GET function (even cooler stuff)</h2>
<p>Let’s define a helper <code>log</code> function first:</p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='kd'>defn </span><span class='nv'>log</span> <span class='p'>[</span><span class='nv'>s</span><span class='p'>]</span>
<span class='p'>(</span><span class='nf'>.log</span> <span class='nv'>js/console</span> <span class='p'>(</span><span class='nb'>str </span><span class='nv'>s</span><span class='p'>))))</span>
</code></pre></div>
<p>And now we can call our <code>GET</code> function and print the result:</p>
<div class='highlight'><pre><code class='clojure'><span class='p'>(</span><span class='nf'>go</span>
<span class='p'>(</span><span class='nf'>log</span> <span class='p'>(</span><span class='nf'><!</span> <span class='p'>(</span><span class='nf'>GET</span> <span class='s'>"http://dimagog.github.io/sitemap.txt"</span><span class='p'>))))</span>
</code></pre></div>
<p>We have sequential-looking code that looks like it’s blocking, but it’s not. It is fully asyncronous!</p>
<h2 id='_source_code'><a name='src'> </a> Source code</h2>
<p>Full source code <a href='https://github.com/Dimagog/AsyncGET'>can be found on GitHub</a>.</p>
<p>If you want to build and run it locally, execute:</p>
<pre><code>git clone https://github.com/Dimagog/AsyncGET.git
cd AsyncGET
lein ring server</code></pre>About Me2013-07-12T22:40:00-07:00http://dimagog.github.io/blog/personal/2013/07/12/about-me<p>I work for <a href='http://microsoft.com/jobs'>Microsoft</a> on <a href='http://www.windowsazure.com'>Windows Azure</a> team where I (mostly) program in C#. And in my spare time I love to play with new and interesting languages and technologies inside and outside of “Microsoft stack” such as Clojure, Haskell, Erlang, F#, Scala, Riak, Datomic, Storm, Akka, Hierarchical State Machines, ANTLR, etc.</p>
<p>I tend to forget a lot of what I learn, so I’ve decided to start this Blog where I can keep and share my notes as well as practice technical writing.</p>
<p>Hopefully others will find something useful here as well. Comments and suggestions are certainly welcome!</p>
<p>I share links that I find interesting on <a href='https://plus.google.com/109391987177502206351/posts'>Google+</a> and on <a href='https://www.facebook.com/dmitry.kakurin.9'>Facebook</a>. <br /> My Twitter is <a href='https://twitter.com/DimaPhone'>@DimaPhone</a> (not dimagog), and here is <a href='http://www.linkedin.com/pub/dmitry-kakurin/1/272/a2a'>my LinkedIn page</a>.</p>
<p>- Dmitry Kakurin <em>(dimagog)</em></p>