Jekyll2023-11-25T04:50:49+00:00https://thenitinprakash.com/feed.xmlNitinPersonal blogNitin Prakashme@thenitinprakash.comUnderstanding Postgres Storage System2023-02-07T00:00:00+00:002023-02-07T00:00:00+00:00https://thenitinprakash.com/2023-02-07/Understading-Postgres-Storage-System<p>I’ve been trying to figure out the internals of Postgres storage
system just to understand what has kept it around for so long. In the
process, I stumbled upon <a href="https://dsf.berkeley.edu/papers/ERL-M87-06.pdf">this
paper</a> which outlines
the architectural ideas involved in the early development of postgres.
Note that this paper was published in 1987, so the current version of
Postgres does not adhere completely to what the paper suggests. But
some of the ideas presented are remarkable regardless.</p>
<p>In a nutshell, the Postgres storage engine can be understood as a
collection of 3 design principles:</p>
<ul>
<li>Provide transaction management without the necessity of writing a
large amount of specialized crash recovery code. Why? Because it’s
hard to write safe code this way and hard to debug consequently.</li>
<li>Accomodate the historical state of the data based on a
write-once-read-many (WORM) optical disk (or other archival medium) in
addition to the current state on an ordinary magnetic disk. This was
the birth of <strong>Vacuum cleaner</strong>.</li>
<li>Hardware is a <em>resource</em>. The storage system can take advantage
of existence of non-volatile main memory in some reasonable quantity.</li>
</ul>
<p>Before diving into the above principles in detail, lets see how the
transaction system works in postgres.</p>
<h3 id="transactions">Transactions</h3>
<ul>
<li>Transactions can be thought of as unique identifiers for postgres
operations.</li>
<li>Each transaction is a 40 bit unsigned integer that are
sequentially assigned starting at 1 and increments for each new
transaction. These integers are called <strong>Transaction identifier
(XID)</strong>. However, modern Postgres uses 32 bit unsigned integers.</li>
<li>Simplifying further, these XIDs can be used to determine the order
of transactions: Eg: If transaction A has an <code class="language-plaintext highlighter-rouge">XID = 100</code> and
transaction B has an <code class="language-plaintext highlighter-rouge">XID = 101</code>, then transaction B occurred after
transaction A.</li>
<li>These XIDs were what Postgres used for crash recovery instead of
Write Ahead Logs (WAL). But this is no longer true. Modern Postgres
system makes use of WAL logs.</li>
<li>XIDs are used for multi-version concurrency control (MVCC). When a
row in a table is updated or deleted, a new version of the row is
created with a new XID, while the old version remains in the table
with its original XID. This allows multiple transactions to read and
write to the same table at the same time without interfering with
each other.</li>
</ul>
<h3 id="relational-storage">Relational storage</h3>
<p>Per row metadata:</p>
<table>
<thead>
<tr>
<th>Field</th>
<th style="text-align: right">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>OID</td>
<td style="text-align: right">a system-assigned unique record identifier</td>
</tr>
<tr>
<td>X<sub>min</sub></td>
<td style="text-align: right">transaction identifier of the interaction inserting the record</td>
</tr>
<tr>
<td>T<sub>min</sub></td>
<td style="text-align: right">commit time of X<sub>Min</sub> (when the record became valid)</td>
</tr>
<tr>
<td>C<sub>min</sub></td>
<td style="text-align: right">command identifier of the interaction inserting the record</td>
</tr>
<tr>
<td>X<sub>max</sub></td>
<td style="text-align: right">transaction identifier of the interaction deleting the record</td>
</tr>
<tr>
<td>T<sub>max</sub></td>
<td style="text-align: right">commit time of X<sub>max</sub> (when the record stopped being valid)</td>
</tr>
<tr>
<td>C<sub>max</sub></td>
<td style="text-align: right">command identifier of the interaction deleting the record</td>
</tr>
<tr>
<td>PTR</td>
<td style="text-align: right">a forward pointer</td>
</tr>
</tbody>
</table>
<p><br /></p>
<p>The current row per metadata can be found
<a href="https://www.postgresql.org/docs/16/storage-page-layout.html#STORAGE-TUPLE-LAYOUT">here</a></p>
<ol>
<li><code class="language-plaintext highlighter-rouge">INSERT</code>: Fields OID, X<sub>min</sub> and C<sub>min</sub> were
set. Rest of the fields were left blank.</li>
<li><code class="language-plaintext highlighter-rouge">UPDATE</code>: 2 operations happen at this point.
<ul>
<li>X<sub>Max</sub> and C<sub>max</sub> are set to the identity of
the current interaction in the record being replaced to indicate that
it has now become invalid.</li>
<li>A new record is inserted into the data base with the proposed
replacement value for the data fields. OID is set to the OID of the
record being replaced, and X<sub>min</sub> and C<sub>min</sub> are set
to the identity of the current interaction. PTR of the new interation
now pointed to the older interaction.</li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">DELETE</code>: X<sub>max</sub> and C<sub>max</sub> were set. Tmin was
also set to indicate the transaction has been committed.</li>
</ol>
<p>In case of <code class="language-plaintext highlighter-rouge">UPDATE</code> transaction, only a few fields would differ as
OID, X<sub>min</sub> and C<sub>min</sub> etc was being
reused. Therefore, in order to optimize space complexity, following
steps were taken:</p>
<ul>
<li>Initial record is stored uncompressed (known as <em>Anchor point</em>).</li>
<li>We find the difference of the record being updated to that of Anchor
point and only the changes were stored.</li>
<li>PTR is altered on the anchor point to point to the updated record
(known as <em>Delta record</em>).</li>
<li>Consecutive updates would generate a singly linked list of delta
records where the head of the list is the anchor point.</li>
</ul>
<blockquote>
<p>NOTE: It looks like the above above mechanism does not exist
anymore. It currently has <strong><a href="https://github.com/postgres/postgres/blob/master/src/backend/access/heap/README.HOT">Heap Only Tuples
(HOT)</a></strong>. From
the README, long story short:</p>
<p>HOT eliminates redundant index entries and allows the re-use of
space taken by DELETEd or obsoleted UPDATEd tuples without performing
a table-wide vacuum. It does this by allowing single-page vacuuming,
also called “defragmentation”.</p>
</blockquote>
<h3 id="record-access">Record access</h3>
<p>Each page:</p>
<ul>
<li>there is a line table containing pointers to the starting byte of
each anchor point record on that page.</li>
<li>a pointer to the next and the previous logical page</li>
</ul>
<p>Hence, Postgres can scan a relation by following the forward linked
list. Although it can execute query plans backward as well, therefore,
needing a pointer to the previous page.</p>
<p>Secondary indexes can be constructed for any base relation, and each
index is maintained by an access method that provides procedures for
access to the index, such as get-record-by-key, insert-record, and
delete-record. The term <code class="language-plaintext highlighter-rouge">secondary index</code> is used vaguely in the
modern times. I recommend getting an idea from
<a href="https://stackoverflow.com/a/51087864/6244324">here</a>.</p>
<p>When a record is inserted - an anchor point is constructed for the
record along with index entries for each secondary index. Each index
record contains a key or a collection of keys along with a pointer to
an entry in the line table on the page where the indexed record
resides.</p>
<p>When an existing record is updated - a delta record is constructed and
chained onto the appropriate anchor record. If an indexed field has
been modified, an entry is added to the appropriate index containing
the new key(s) and a pointer to the anchor record.</p>
<p>An important thing to note for modern postgres system, <em>it does
secondary index maintenance for all secondary indexes unless no
indexed columns have changed</em>. So if there 3 secondary indexes and an
<code class="language-plaintext highlighter-rouge">UPDATE</code> operation changes a column used by 1 of them then maintenance
is done for all of them – unless HOT is used.</p>
<p>There’s a lot of other noticeable things around secondary indexes but
I won’t go into depth. Feel free to read the paper.</p>
<h3 id="vacuuming-the-disk">Vacuuming the disk</h3>
<p>An asynchronous process (called <strong>vacuum cleaner</strong>) was responsible for
sweeping records which are no longer valid to the archive. The syntax
to invoke this cleaner was:</p>
<p><code class="language-plaintext highlighter-rouge">vacuum rel-name after T</code></p>
<p>where T is time, eg: ‘30 days’</p>
<p>Conditions for vacuum cleaner to find valid candidates for archival:</p>
<ul>
<li>X<sub>max</sub> is non empty and is a committed transaction and <code>now - T<sub>max</sub> >= T</code>.</li>
<li>X<sub>max</sub> is non empty and is an aborted transaction.</li>
<li>X<sub>min</sub> is non empty and is an aborted transaction.</li>
</ul>
<p>There were certain <em>status</em> assigned to those records so that cleaner
can ensure no data is irrecoverably lost. In the first case, a record
wascopied to the archive unless <strong>no-archive</strong> status is set for this
relation. Whereas, in the second and third case, cleaner would simply
reclaim the space.</p>
<h3 id="vacuum-process">Vacuum process</h3>
<p>Vacuuming was done in three phases, namely:</p>
<ol>
<li>Write an archive record and its associated index records</li>
<li>Write a new anchor point in the current database</li>
<li>Reclaim the space occupied by the old anchor point and its delta records</li>
</ol>
<p>What postgres was doing at the time wasn’t crash safe. But it managed
to handle it correctly. Several scenarios:</p>
<ul>
<li>
<p>If a crash occured while the vacuum cleaner is writing the
historical record in phase 1, then the data still existed in the
magnetic disk database and could be revacuumed some later time.</p>
</li>
<li>
<p>If a crash occured after some index records was written, then it was
possible for the same record to be accessed in a magnetic disk
relation and in an archive relation.</p>
</li>
</ul>
<p>In either case, the duplicate record consumed system resources but it
wasn’t really convcerning because of postgres is a relational system
and removed duplicate records during processing.</p>
<h3 id="vacuuming-cost">Vacuuming cost</h3>
<p>The paper discusses two scenarios:</p>
<ul>
<li>A record is inserted, updated K times and then deleted. The whole
chain of records from insertion to deletion is <em>vacuumed at once</em>.</li>
<li>The vacuum is run after K updates, and a new anchor record must be
inserted.</li>
</ul>
<p>The above is accompanied by several constant parameters:</p>
<ul>
<li>There are Z secondary indexes for both the archive and magnetic disk
relation</li>
<li>No key changes are made during these K updates</li>
<li>Anchor point and all its delta records reside on the same page</li>
</ul>
<table>
<thead>
<tr>
<th> </th>
<th>whole chain</th>
<th>K updates</th>
</tr>
</thead>
<tbody>
<tr>
<td>archive writes</td>
<td>1 + Z</td>
<td>1 + Z</td>
</tr>
<tr>
<td>disk reads</td>
<td>1</td>
<td>1</td>
</tr>
<tr>
<td>disk writes</td>
<td>1 + Z</td>
<td>1 + Z</td>
</tr>
</tbody>
</table>
<p><br />
The above table reflects the IO count for vacuuming. The cost PER
RECORD of the vacuum cleaner is inversely proportional to the length
of the chain. As long as an anchor point and several delta records are
vacuumed together, the cost claimed was marginal.</p>
<p>There are other things that paper discusses like the archival medium,
magnetic disk indexes etc but I’m refraining from writing those just
yet because I don’t understand them completely.</p>Nitin PrakashI’ve been trying to figure out the internals of Postgres storage system just to understand what has kept it around for so long. In the process, I stumbled upon this paper which outlines the architectural ideas involved in the early development of postgres. Note that this paper was published in 1987, so the current version of Postgres does not adhere completely to what the paper suggests. But some of the ideas presented are remarkable regardless.Using SQLite with Haskell2021-10-20T00:00:00+00:002021-10-20T00:00:00+00:00https://thenitinprakash.com/2021-10-20/Using-Sqlite-with-Haskell<p>As a Haskell programmer, it’s quite natural to think about <em>type safety</em> almost always. Even when it comes to doing simplest of the things. There’s a whole bunch of debate on why dead simple database packages like <a href="https://github.com/haskellari/postgresql-simple">postgres-simple</a> or <a href="https://github.com/nurpax/sqlite-simple">sqlite-simple</a> should be avoided generally. The inlcination is towards libraries which offer more type safety (not to mention complexity comes along with it) like <a href="https://github.com/yesodweb/persistent">Persistent</a>. It offers a lot of features which guarantees type safety with SQL queries. Under the hood, it makes a ton of template haskell usage to produce a large amount of types and type classes to simplify serialization to and from the database. But the reality is, <code class="language-plaintext highlighter-rouge">persistent</code> is not as easy to use as <code class="language-plaintext highlighter-rouge">*-simple</code> libraries mentioned earlier. And it’s not always worth it to make things complex for a bit of type safety. If our use case is simple enough (and even in most of the code bases with heavy database usage, it really is) we can probably get away with writing plain SQL queries with enough type safety for things to not break at runtime.</p>
<p>All that being said, it’s a nice idea to have type safe queries floating around in your codebase. Or you can do what I do, start with something simple and then iterate to make things better.</p>
<p>The idea of this blog is the present a nice and concise way of writing SQL queries using Haskell without having to fight the type system unnecessarily. And these queries do not deviate a lot from writing standard SQL either.</p>
<p>Let’s first create a new experimental table in an existing database schema. We’ll define a few data types based on top of this and carry on from there.</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{-# LANGUAGE OverloadedStrings #-}</span>
<span class="cp">{-# LANGUAGE QuasiQuotes #-}</span>
<span class="cp">{-# LANGUAGE TemplateHaskell #-}</span>
<span class="kr">import</span> <span class="nn">Database.SQLite.Simple</span> <span class="p">(</span><span class="kt">Connection</span> <span class="p">(</span><span class="o">..</span><span class="p">),</span> <span class="nf">execute_</span><span class="p">)</span>
<span class="kr">import</span> <span class="nn">Database.SQLite.Simple.QQ</span> <span class="p">(</span><span class="nf">sql</span><span class="p">)</span>
<span class="kr">import</span> <span class="k">qualified</span> <span class="nn">Data.Text</span> <span class="k">as</span> <span class="n">T</span>
<span class="n">createPersonTable</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">createPersonTable</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">execute_</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
CREATE TABLE IF NOT EXISTS person
( id INTEGER PRIMARY KEY AUTOINCREMENT
, name TEXT NOT NULL
, email TEXT UNIQUE NOT NULL
, phone_number TEXT UNIQUE NOT NULL
, role TEXT NOT NULL
-- ^ This could even be an enum type at the DB layer
)
<span class="o">|]</span>
</code></pre></div></div>
<p>This essentially captures some basic information of a person. The catch here would be to define <code class="language-plaintext highlighter-rouge">role</code> in our haskell code base as a sum type. The reasoning behind this is we don’t want random user roles to corrupt our database. We need to set a bound on the the values allowed on it. Once we have it, the real trouble is inserting and fetching the data itself. That is, navigation of data to and from the database.</p>
<p>If we were to define data type <code class="language-plaintext highlighter-rouge">Person</code> which corresponds to the person table, it would look like:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">import</span> <span class="k">qualified</span> <span class="nn">Data.Text</span> <span class="k">as</span> <span class="n">T</span>
<span class="kr">data</span> <span class="kt">Person</span> <span class="o">=</span> <span class="kt">Person</span>
<span class="p">{</span> <span class="n">pUUID</span> <span class="o">::</span> <span class="o">!</span><span class="kt">Integer</span>
<span class="p">,</span> <span class="n">pName</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="n">pEmail</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="n">pRole</span> <span class="o">::</span> <span class="o">!</span><span class="kt">Role</span>
<span class="p">,</span> <span class="n">pNumber</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">}</span> <span class="kr">deriving</span> <span class="n">stock</span> <span class="p">(</span><span class="kt">Eq</span><span class="p">,</span> <span class="kt">Show</span><span class="p">)</span>
</code></pre></div></div>
<p>Note that the above is not going to compile just yet because we still need to go ahead and implement <code class="language-plaintext highlighter-rouge">Role</code> type. So let’s do that first.</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">{-# LANGUAGE LambdaCase #-}</span>
<span class="c1">-- We are going to limit to only 3 user roles</span>
<span class="kr">data</span> <span class="kt">Role</span>
<span class="o">=</span> <span class="kt">Hashira</span>
<span class="o">|</span> <span class="kt">Kakushi</span>
<span class="o">|</span> <span class="kt">DemonLord</span>
<span class="kr">deriving</span> <span class="p">(</span><span class="kt">Eq</span><span class="p">,</span> <span class="kt">Show</span><span class="p">,</span> <span class="kt">Read</span><span class="p">,</span> <span class="kt">Enum</span><span class="p">,</span> <span class="kt">Bounded</span><span class="p">)</span>
<span class="n">roleToText</span> <span class="o">::</span> <span class="kt">Role</span> <span class="o">-></span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="n">roleToText</span> <span class="o">=</span> <span class="nf">\</span><span class="kr">case</span>
<span class="kt">Hashira</span> <span class="o">-></span> <span class="s">"hashira"</span>
<span class="kt">Kakushi</span> <span class="o">-></span> <span class="s">"kakushi"</span>
<span class="kt">DemondLord</span> <span class="o">-></span> <span class="s">"demon lord
</span></code></pre></div></div>
<p align="center">
<img src="../assets/images/6_post/rengoku.jpeg" style="width:350px;height:300px;" />
</p>
<p>Cool! What now? Well, we’ve only setup the required data types that represent a person. We haven’t implemented anything that can let <code class="language-plaintext highlighter-rouge">person</code> communicate with the database itself. We need something (function, instance, miracle etc) in order to convert a <strong>sequence</strong> of fields (that we will fetch from the DB) to our haskell data type. Similary, we’ll need something to convert our haskell data type to a <strong>sequnce</strong> of fields that sqlite can understand while inserting data into the <code class="language-plaintext highlighter-rouge">person</code> table. Fortunately, <code class="language-plaintext highlighter-rouge">sqlite-simple</code> offers <code class="language-plaintext highlighter-rouge">FromRow</code> and <code class="language-plaintext highlighter-rouge">ToRow</code> typeclasses which is build exactly for this purpose!</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">import</span> <span class="nn">Control.Applicative</span>
<span class="kr">import</span> <span class="nn">Database.SQLite.Simple.FromRow</span> <span class="p">(</span><span class="kt">FromRow</span> <span class="p">(</span><span class="o">..</span><span class="p">))</span>
<span class="kr">import</span> <span class="nn">Database.SQLite.Simple.Internal</span> <span class="p">(</span><span class="kt">Field</span> <span class="p">(</span><span class="o">..</span><span class="p">))</span>
<span class="kr">import</span> <span class="nn">Database.SQLite.Simple.ToRow</span> <span class="p">(</span><span class="kt">ToRow</span> <span class="p">(</span><span class="o">..</span><span class="p">))</span>
<span class="c1">-- Why so many @field@ though?</span>
<span class="c1">-- According to the docs,</span>
<span class="c1">-- The number of calls to @field@ must match the number of fields returned</span>
<span class="c1">-- in a single row of the query result. Otherwise, a @ConversionFailed@</span>
<span class="c1">-- exception will be thrown.</span>
<span class="kr">instance</span> <span class="kt">FromRow</span> <span class="kt">Person</span> <span class="kr">where</span>
<span class="n">fromRow</span> <span class="o">=</span> <span class="kt">Person</span> <span class="o"><$></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span>
<span class="c1">-- We ignore pUUID as it is set to auto increment at the DB layer and is already</span>
<span class="c1">-- managed by SQLite. Ideally, you should let UUIDs be handled at the DB layer</span>
<span class="c1">-- itself unless you have a very strong reason not to</span>
<span class="kr">instance</span> <span class="kt">ToRow</span> <span class="kt">Person</span> <span class="kr">where</span>
<span class="n">toRow</span> <span class="p">(</span><span class="kt">Person</span> <span class="n">_pUUID</span> <span class="n">pName</span> <span class="n">pEmail</span> <span class="n">pRole</span> <span class="n">pNumber</span><span class="p">)</span> <span class="o">=</span>
<span class="n">toRow</span> <span class="p">(</span><span class="n">pName</span><span class="p">,</span> <span class="n">pEmail</span><span class="p">,</span> <span class="n">pRole</span><span class="p">,</span> <span class="n">pNumber</span><span class="p">)</span>
</code></pre></div></div>
<p>At this point, we should have an understanding of how data conversions are taking place between the code and database. Let’s write a function real quick to see if we are able insert and fetch a person or not.</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">import</span> <span class="nn">Database.SQLite.Simple</span> <span class="p">(</span><span class="kt">Connection</span> <span class="p">(</span><span class="o">..</span><span class="p">),</span> <span class="kt">NamedParam</span> <span class="p">(</span><span class="o">..</span><span class="p">),</span> <span class="nf">executeNamed_</span><span class="p">,</span> <span class="nf">query_</span><span class="p">)</span>
<span class="n">createNewPerson</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">createNewPerson</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">executeNamed</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
INSERT INTO person
(name, email, phone_number, role)
VALUES
(:name, :email, :phoneNumber, :userRole)
<span class="o">|]</span> <span class="p">[</span> <span class="s">":name"</span> <span class="o">:=</span> <span class="p">(</span><span class="s">"Rengoku"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span><span class="p">)</span>
<span class="p">,</span> <span class="s">":phoneNumber"</span> <span class="o">:=</span> <span class="p">(</span><span class="s">"+91234567899"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span><span class="p">)</span>
<span class="p">,</span> <span class="s">":email"</span> <span class="o">:=</span> <span class="p">(</span><span class="s">"rengoku@demonslayer.com"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span><span class="p">)</span>
<span class="p">,</span> <span class="s">":userRole"</span> <span class="o">:=</span> <span class="kt">Hashira</span>
<span class="p">]</span>
<span class="n">getAllPerson</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="p">[</span><span class="kt">Person</span><span class="p">]</span>
<span class="n">getAllPerson</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">query_</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
SELECT id
, name
, email
, role
, phone_number
FROM person
<span class="o">|]</span>
</code></pre></div></div>
<p align="center">
<img src="../assets/images/6_post/wtf-confused.gif" style="width:350px;height:300px;" />
</p>
<p><code class="language-plaintext highlighter-rouge">getAllPerson</code> is very straight forward. But what in the name of sweet type classes is going on inside <code class="language-plaintext highlighter-rouge">createNewPerson</code> !?</p>
<p>Let’s go step by step. There are 3 main components to the function:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">Connection</code> type in the type signature</li>
<li>Named Parameters ( Weird looking equalities)</li>
<li>Structure of the query itself</li>
</ul>
<p>And this is going to be the case for most of the queries you write which needs some sort of parameter substitution. At least, I’d recommend writing queries using named parameters in order to keep things clean and much more readable. We could’ve written <code class="language-plaintext highlighter-rouge">createNewPerson</code> without named parameters like so:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">import</span> <span class="nn">Database.SQLite.Simple</span> <span class="p">(</span><span class="nf">execute</span><span class="p">)</span>
<span class="n">createNewPerson</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">createNewPerson</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">execute</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
INSERT INTO person
(name, email, phone_number, role)
VALUES
(?, ?, ?, ?)
<span class="o">|]</span> <span class="p">(</span> <span class="s">"Rengoku"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="s">"+91234567899"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="s">"rengoku@demonslayer.com"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="kt">Hashira</span>
<span class="p">)</span>
</code></pre></div></div>
<p>But the readability reduces as soon as the query grows and there are bunch of substitution happening. Also, there is a slight chance of messing up the order of the arguments (eg: type safety won’t save you if two consecute <code class="language-plaintext highlighter-rouge">Text</code> type substitutions are happening) leading to a corrupt DB state. Using named parameters remove these shortcomings.</p>
<p>Alright enough talk, let’s actually see if this works or not. We’ll write a <code class="language-plaintext highlighter-rouge">main</code> method and fire up the repl to test and spam the code.</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">import</span> <span class="nn">Database.SQLite.Simple</span> <span class="p">(</span><span class="nf">close</span><span class="p">,</span> <span class="nf">open</span><span class="p">)</span>
<span class="n">flushPersonTable</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">flushPersonTable</span> <span class="n">conn</span> <span class="o">=</span>
<span class="n">execute_</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
DROP TABLE IF EXISTS person;
<span class="o">|]</span>
<span class="c1">-- Drops the person table and creates a fresh new one</span>
<span class="c1">-- This is done in order to play with fresh data every time we</span>
<span class="c1">-- run @main@ in the repl. Not that it's a rule.</span>
<span class="n">resetDb</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">resetDb</span> <span class="n">conn</span> <span class="o">=</span> <span class="kr">do</span>
<span class="n">flushPersonTable</span> <span class="n">conn</span>
<span class="n">createPersonTable</span> <span class="n">conn</span>
<span class="n">main</span> <span class="o">::</span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">main</span> <span class="o">=</span> <span class="kr">do</span>
<span class="c1">-- Open up a connection to the database and this will be passed to each query we run.</span>
<span class="c1">-- In practice, we should maintain a pool of connections so that we can run SQl</span>
<span class="c1">-- queries in parallel. Also, this practice enables you to use queries inside a</span>
<span class="c1">-- DB transaction.</span>
<span class="n">conn</span> <span class="o"><-</span> <span class="n">open</span> <span class="s">"test.db"</span>
<span class="n">resetDb</span> <span class="n">conn</span>
<span class="n">createNewPerson</span> <span class="n">conn</span>
<span class="n">rows</span> <span class="o"><-</span> <span class="n">getAllPerson</span> <span class="n">conn</span>
<span class="n">mapM_</span> <span class="n">print</span> <span class="n">rows</span>
<span class="n">close</span> <span class="n">conn</span>
</code></pre></div></div>
<p>Aw snap! We’ve run into errors. Out of all the things GHC is complaining, we focus ourselves on the following two first</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">src</span><span class="o">/</span><span class="kt">SqliteSimple</span><span class="o">.</span><span class="n">hs</span><span class="o">:</span><span class="mi">54</span><span class="o">:</span><span class="mi">56</span><span class="o">:</span> <span class="n">error</span><span class="o">:</span>
<span class="err">•</span> <span class="kt">No</span> <span class="kr">instance</span> <span class="n">for</span> <span class="p">(</span><span class="kt">FromField</span> <span class="kt">Role</span><span class="p">)</span> <span class="n">arising</span> <span class="n">from</span> <span class="n">a</span> <span class="n">use</span> <span class="kr">of</span> <span class="err">‘</span><span class="n">field</span><span class="err">’</span>
<span class="err">•</span> <span class="kt">In</span> <span class="n">the</span> <span class="n">second</span> <span class="n">argument</span> <span class="kr">of</span> <span class="err">‘</span><span class="p">(</span><span class="o"><*></span><span class="p">)</span><span class="err">’</span><span class="p">,</span> <span class="n">namely</span> <span class="err">‘</span><span class="n">field</span><span class="err">’</span>
<span class="kt">In</span> <span class="n">the</span> <span class="n">first</span> <span class="n">argument</span> <span class="kr">of</span> <span class="err">‘</span><span class="p">(</span><span class="o"><*></span><span class="p">)</span><span class="err">’</span><span class="p">,</span> <span class="n">namely</span>
<span class="err">‘</span><span class="kt">Person</span> <span class="o"><$></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span><span class="err">’</span>
<span class="kt">In</span> <span class="n">the</span> <span class="n">expression</span><span class="o">:</span>
<span class="kt">Person</span> <span class="o"><$></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span>
<span class="o">|</span>
<span class="mi">54</span> <span class="o">|</span> <span class="n">fromRow</span> <span class="o">=</span> <span class="kt">Person</span> <span class="o"><$></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span>
<span class="o">|</span> <span class="o">^^^^^</span>
</code></pre></div></div>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">src</span><span class="o">/</span><span class="kt">SqliteSimple</span><span class="o">.</span><span class="n">hs</span><span class="o">:</span><span class="mi">58</span><span class="o">:</span><span class="mi">56</span><span class="o">:</span> <span class="n">error</span><span class="o">:</span>
<span class="err">•</span> <span class="kt">No</span> <span class="kr">instance</span> <span class="n">for</span> <span class="p">(</span><span class="kt">ToField</span> <span class="kt">Role</span><span class="p">)</span> <span class="n">arising</span> <span class="n">from</span> <span class="n">a</span> <span class="n">use</span> <span class="kr">of</span> <span class="err">‘</span><span class="n">toRow</span><span class="err">’</span>
<span class="err">•</span> <span class="kt">In</span> <span class="n">the</span> <span class="n">expression</span><span class="o">:</span> <span class="n">toRow</span> <span class="p">(</span><span class="n">pName</span><span class="p">,</span> <span class="n">pEmail</span><span class="p">,</span> <span class="n">pRole</span><span class="p">,</span> <span class="n">pNumber</span><span class="p">)</span>
<span class="kt">In</span> <span class="n">an</span> <span class="n">equation</span> <span class="n">for</span> <span class="err">‘</span><span class="n">toRow</span><span class="err">’</span><span class="o">:</span>
<span class="n">toRow</span> <span class="p">(</span><span class="kt">Person</span> <span class="n">_pUUID</span> <span class="n">pName</span> <span class="n">pEmail</span> <span class="n">pRole</span> <span class="n">pNumber</span><span class="p">)</span>
<span class="o">=</span> <span class="n">toRow</span> <span class="p">(</span><span class="n">pName</span><span class="p">,</span> <span class="n">pEmail</span><span class="p">,</span> <span class="n">pRole</span><span class="p">,</span> <span class="n">pNumber</span><span class="p">)</span>
<span class="kt">In</span> <span class="n">the</span> <span class="kr">instance</span> <span class="n">declaration</span> <span class="n">for</span> <span class="err">‘</span><span class="kt">ToRow</span> <span class="kt">Person</span><span class="err">’</span>
<span class="o">|</span>
<span class="mi">58</span> <span class="o">|</span> <span class="n">toRow</span> <span class="p">(</span><span class="kt">Person</span> <span class="n">_pUUID</span> <span class="n">pName</span> <span class="n">pEmail</span> <span class="n">pRole</span> <span class="n">pNumber</span><span class="p">)</span> <span class="o">=</span> <span class="n">toRow</span> <span class="p">(</span><span class="n">pName</span><span class="p">,</span> <span class="n">pEmail</span><span class="p">,</span> <span class="n">pRole</span><span class="p">,</span> <span class="n">pNumber</span><span class="p">)</span>
<span class="o">|</span> <span class="o">^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span>
</code></pre></div></div>
<p>Previously, we defined a <code class="language-plaintext highlighter-rouge">FromRow</code> and <code class="language-plaintext highlighter-rouge">ToRow</code> instance when we wanted to convert a <strong>collection</strong> of Haskell types to and from a <strong>collection</strong> of SQL types. But what happens when we introduce a new data type which is not understood by GHC natively. In our case, when we defined <code class="language-plaintext highlighter-rouge">Role</code>, we were good as GHC knew how to interpret sum types but it doesn’t know the intentions behind it’s usage. In other words, how we want <code class="language-plaintext highlighter-rouge">Role</code> to intereact with other data types (internally and from other packages) is left entirely to us. We <em>tell</em> GHC our intentions by impementing different instances for it. Therefore, we’ll have to define <code class="language-plaintext highlighter-rouge">ToField</code> and <code class="language-plaintext highlighter-rouge">FromField</code> instances so that the compiler knows how to convert <code class="language-plaintext highlighter-rouge">Role</code> to an SQL data type. Also, these implementations are used by <code class="language-plaintext highlighter-rouge">ToRow</code> and <code class="language-plaintext highlighter-rouge">FromRow</code>. What we had earlier was just a step in the right direction but it was incomplete.</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">import</span> <span class="nn">Database.SQLite.Simple.FromField</span> <span class="p">(</span><span class="kt">Field</span> <span class="p">(</span><span class="o">..</span><span class="p">),</span> <span class="kt">FromField</span> <span class="p">(</span><span class="o">..</span><span class="p">),</span> <span class="nf">returnError</span><span class="p">)</span>
<span class="kr">import</span> <span class="nn">Database.SQLite.Simple.Ok</span> <span class="p">(</span><span class="kt">Ok</span> <span class="p">(</span><span class="o">..</span><span class="p">))</span>
<span class="kr">import</span> <span class="nn">Database.SQLite.Simple.ToField</span> <span class="p">(</span><span class="kt">ToField</span> <span class="p">(</span><span class="o">..</span><span class="p">))</span>
<span class="c1">-- This prepares a value for substitution into a query string.</span>
<span class="kr">instance</span> <span class="kt">ToField</span> <span class="kt">Role</span> <span class="kr">where</span>
<span class="n">toField</span> <span class="o">=</span> <span class="kt">SQLText</span> <span class="o">.</span> <span class="n">roleToText</span>
<span class="kr">instance</span> <span class="kt">FromField</span> <span class="kt">Role</span> <span class="kr">where</span>
<span class="n">fromField</span> <span class="p">(</span><span class="kt">Field</span> <span class="p">(</span><span class="kt">SQLText</span> <span class="s">"hashira"</span><span class="p">)</span> <span class="kr">_</span><span class="p">)</span> <span class="o">=</span> <span class="kt">Ok</span> <span class="kt">Hahsira</span>
<span class="n">fromField</span> <span class="p">(</span><span class="kt">Field</span> <span class="p">(</span><span class="kt">SQLText</span> <span class="s">"kakushi"</span><span class="p">)</span> <span class="kr">_</span><span class="p">)</span> <span class="o">=</span> <span class="kt">Ok</span> <span class="kt">Kakushi</span>
<span class="n">fromField</span> <span class="p">(</span><span class="kt">Field</span> <span class="p">(</span><span class="kt">SQLText</span> <span class="s">"demon lord"</span><span class="p">)</span> <span class="kr">_</span><span class="p">)</span> <span class="o">=</span> <span class="kt">Ok</span> <span class="kt">DemonLord</span>
<span class="n">fromField</span> <span class="n">f</span> <span class="o">=</span> <span class="n">returnError</span> <span class="kt">ConversionFailed</span> <span class="n">f</span> <span class="s">"role does not exist"</span>
</code></pre></div></div>
<p>One of the question from above can be regarding the definition of <code class="language-plaintext highlighter-rouge">ToField</code> and <code class="language-plaintext highlighter-rouge">FromField</code> instances of <code class="language-plaintext highlighter-rouge">Role</code>. How did we come up with that? Well, the secret ingredient here is reading the docs. According to the <a href="https://hackage.haskell.org/package/sqlite-simple-0.4.18.0/docs/src/Database.SQLite.Simple.ToField.html#local-6989586621679115476">docs</a>, The way <code class="language-plaintext highlighter-rouge">ToField</code> is defined for <code class="language-plaintext highlighter-rouge">Text</code> types are:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">instance</span> <span class="kt">ToField</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span> <span class="kr">where</span>
<span class="n">toField</span> <span class="o">=</span> <span class="kt">SQLText</span>
<span class="cp">{-# INLINE toField #-}</span>
<span class="kr">instance</span> <span class="kt">FromField</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span> <span class="kr">where</span>
<span class="n">fromField</span> <span class="p">(</span><span class="kt">Field</span> <span class="p">(</span><span class="kt">SQLText</span> <span class="n">txt</span><span class="p">)</span> <span class="kr">_</span><span class="p">)</span> <span class="o">=</span> <span class="kt">Ok</span> <span class="n">txt</span>
<span class="n">fromField</span> <span class="n">f</span> <span class="o">=</span> <span class="n">returnError</span> <span class="kt">ConversionFailed</span> <span class="n">f</span> <span class="s">"need a text"</span>
</code></pre></div></div>
<p>It is quite clear from the above on how we can define our own version of <code class="language-plaintext highlighter-rouge">instance ToField Role</code> given that we want a <code class="language-plaintext highlighter-rouge">text</code> type in the database. If we try reloading our REPL now and run <code class="language-plaintext highlighter-rouge">main</code>, we can see that we have a successful transaction between Haskell layer and DB layer:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">*</span><span class="kt">SqliteSimple</span><span class="o">></span> <span class="n">main</span>
<span class="kt">Person</span> <span class="p">{</span><span class="n">pUUID</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="n">pName</span> <span class="o">=</span> <span class="s">"Rengoku"</span><span class="p">,</span> <span class="n">pEmail</span> <span class="o">=</span> <span class="s">"rengoku@demonslayer.com"</span><span class="p">,</span> <span class="n">pRole</span> <span class="o">=</span> <span class="kt">Hashira</span><span class="p">,</span> <span class="n">pNumber</span> <span class="o">=</span> <span class="s">"+91234567899"</span><span class="p">}</span>
<span class="o">*</span><span class="kt">SqliteSimple</span><span class="o">></span>
</code></pre></div></div>
<p>This was a high level tutorial on how to use <code class="language-plaintext highlighter-rouge">sqlite-simple</code> (or any database like <code class="language-plaintext highlighter-rouge">postgres-simple</code> and <code class="language-plaintext highlighter-rouge">mysql-simple</code>) for that matter. Note that <code class="language-plaintext highlighter-rouge">postgres-simple</code> does not ship named parameters natively. <a href="https://github.com/Holmusk/postgresql-simple-named">postgres-simple-named</a> would be a suitable chouce for that as it is built on top of <code class="language-plaintext highlighter-rouge">postgres-simple</code>. Anyways, we’ve only seen how to play with very basic data types. Let’s get a bit more creative. We can play with –</p>
<ul>
<li>timestamps</li>
<li>Joins</li>
<li>JSON structures</li>
</ul>
<p>to see the usefulness and ease of writing SQL queries with haskell.</p>
<p>Let’s introduce a <code class="language-plaintext highlighter-rouge">created_at</code> value for a person. Let’s make the following changes,</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">import</span> <span class="nn">Data.Time</span> <span class="p">(</span><span class="kt">UTCTime</span> <span class="p">(</span><span class="o">..</span><span class="p">))</span>
<span class="n">createPersonTable</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">createPersonTable</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">execute_</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
CREATE TABLE IF NOT EXISTS person
( id INTEGER PRIMARY KEY AUTOINCREMENT
, name TEXT NOT NULL
, email TEXT UNIQUE NOT NULL
, phone_number TEXT UNIQUE NOT NULL
, role TEXT NOT NULL -- This could even be an enum type at the DB layer
-- Changed --
, created_at IMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
------------
)
<span class="o">|]</span>
<span class="kr">data</span> <span class="kt">Person</span> <span class="o">=</span> <span class="kt">Person</span>
<span class="p">{</span> <span class="n">pUUID</span> <span class="o">::</span> <span class="o">!</span><span class="kt">Integer</span>
<span class="p">,</span> <span class="n">pName</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="n">pEmail</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="n">pRole</span> <span class="o">::</span> <span class="o">!</span><span class="kt">Role</span>
<span class="p">,</span> <span class="n">pNumber</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="c1">-- Changed --</span>
<span class="p">,</span> <span class="n">pCreatedAt</span> <span class="o">::</span> <span class="o">!</span><span class="kt">UTCTime</span>
<span class="c1">-------------</span>
<span class="p">}</span> <span class="kr">deriving</span> <span class="n">stock</span> <span class="p">(</span><span class="kt">Eq</span><span class="p">,</span> <span class="kt">Show</span><span class="p">)</span>
<span class="c1">-- Now we need to handle the added field in our instances</span>
<span class="c1">-- Compiler will scream at you that certain fields are missing</span>
<span class="kr">instance</span> <span class="kt">FromRow</span> <span class="kt">Person</span> <span class="kr">where</span>
<span class="n">fromRow</span> <span class="o">=</span> <span class="kt">Person</span> <span class="o"><$></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span>
<span class="c1">-- when inserting a new person, ignore UUID and created_at (Handled by SQLite).</span>
<span class="kr">instance</span> <span class="kt">ToRow</span> <span class="kt">Person</span> <span class="kr">where</span>
<span class="n">toRow</span> <span class="p">(</span><span class="kt">Person</span> <span class="n">_pUUID</span> <span class="n">pName</span> <span class="n">pEmail</span> <span class="n">pRole</span> <span class="n">pNumber</span> <span class="n">_pJoinedAt</span><span class="p">)</span> <span class="o">=</span>
<span class="n">toRow</span> <span class="p">(</span><span class="n">pName</span><span class="p">,</span> <span class="n">pEmail</span><span class="p">,</span> <span class="n">pRole</span><span class="p">,</span> <span class="n">pNumber</span><span class="p">)</span>
</code></pre></div></div>
<p>Now if we reload the REPL and run <code class="language-plaintext highlighter-rouge">main</code>, we get:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">*</span><span class="kt">SqliteSimple</span><span class="o">></span> <span class="o">:</span><span class="n">r</span>
<span class="p">[</span><span class="mi">1</span> <span class="kr">of</span> <span class="mi">1</span><span class="p">]</span> <span class="kt">Compiling</span> <span class="kt">SqliteSimple</span> <span class="p">(</span> <span class="n">src</span><span class="o">/</span><span class="kt">SqliteSimple</span><span class="o">.</span><span class="n">hs</span><span class="p">,</span> <span class="n">interpreted</span> <span class="p">)</span>
<span class="kt">Ok</span><span class="p">,</span> <span class="n">one</span> <span class="kr">module</span> <span class="err">loaded.</span>
<span class="err">*</span><span class="nn">SqliteSimple</span><span class="o">></span> <span class="n">main</span>
<span class="kt">Person</span> <span class="p">{</span><span class="n">pUUID</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="n">pName</span> <span class="o">=</span> <span class="s">"Rengoku"</span><span class="p">,</span> <span class="n">pEmail</span> <span class="o">=</span> <span class="s">"rengoku@demonslayer.com"</span><span class="p">,</span> <span class="n">pRole</span> <span class="o">=</span> <span class="kt">Hashira</span><span class="p">,</span> <span class="n">pNumber</span> <span class="o">=</span> <span class="s">"+91234567899"</span><span class="p">,</span> <span class="n">pJoinedAt</span> <span class="o">=</span> <span class="mi">2021</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="mi">25</span> <span class="mi">07</span><span class="o">:</span><span class="mi">21</span><span class="o">:</span><span class="mi">54</span> <span class="kt">UTC</span><span class="p">}</span>
<span class="o">*</span><span class="kt">SqliteSimple</span><span class="o">></span>
</code></pre></div></div>
<p>Notice the <code class="language-plaintext highlighter-rouge">pJoinedAt</code> at the end and the fact that we didn’t have to write a custom <code class="language-plaintext highlighter-rouge">FromField</code> instance for a timestamp value. That’s because <code class="language-plaintext highlighter-rouge">sqlite-simple</code> ships <code class="language-plaintext highlighter-rouge">instance FromField UTCTime</code> which gets imported while importing <code class="language-plaintext highlighter-rouge">FromField (..)</code>. This reasoning holds true even for other <code class="language-plaintext highlighter-rouge">Text</code> and <code class="language-plaintext highlighter-rouge">Int</code> types.</p>
<p>Now let’s create another entity so we can experiment with SQL joins.</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">-- A data type that represent the favourite anime and character of a user</span>
<span class="kr">data</span> <span class="kt">FavouriteAnime</span> <span class="o">=</span> <span class="kt">FavouriteAnime</span>
<span class="p">{</span> <span class="n">faUUID</span> <span class="o">::</span> <span class="o">!</span><span class="kt">Integer</span>
<span class="p">,</span> <span class="n">faAnimeName</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="n">faFavouriteCharacter</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">,</span> <span class="n">faUserName</span> <span class="o">::</span> <span class="o">!</span><span class="kt">T</span><span class="o">.</span><span class="kt">Text</span>
<span class="p">}</span> <span class="kr">deriving</span> <span class="p">(</span><span class="kt">Eq</span><span class="p">,</span> <span class="kt">Show</span><span class="p">)</span>
<span class="kr">instance</span> <span class="kt">FromRow</span> <span class="kt">FavouriteAnime</span> <span class="kr">where</span>
<span class="n">fromRow</span> <span class="o">=</span> <span class="kt">FavouriteAnime</span> <span class="o"><$></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span> <span class="o"><*></span> <span class="n">field</span>
<span class="n">createAnimeTable</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">createAnimeTable</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">execute_</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
CREATE TABLE IF NOT EXISTS favourite_anime
( id INTEGER PRIMARY KEY AUTOINCREMENT
, anime_name TEXT NOT NULL
, favourite_character TEXT UNIQUE
, person_id INTEGER NOT NULL
-- We reference the person_id but we need to fetch the
-- person name as per our Haskell data type
, FOREIGN KEY(person_id) REFERENCES person(id)
)
<span class="o">|]</span>
<span class="n">insertFavouriteAnime</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">insertFavouriteAnime</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">executeNamed</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
INSERT INTO favourite_anime
(anime_name, favourite_character, person_id)
VALUES
(:animeName, :favouriteCharacter, :personId)
<span class="o">|]</span> <span class="p">[</span> <span class="s">":animeName"</span> <span class="o">:=</span> <span class="p">(</span><span class="s">"Attack on titans"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span><span class="p">)</span>
<span class="p">,</span> <span class="s">":favouriteCharacter"</span> <span class="o">:=</span> <span class="p">(</span><span class="s">"Levi Ackerman"</span> <span class="o">::</span> <span class="kt">T</span><span class="o">.</span><span class="kt">Text</span><span class="p">)</span>
<span class="p">,</span> <span class="s">":personId"</span> <span class="o">:=</span> <span class="p">(</span><span class="mi">1</span> <span class="o">::</span> <span class="kt">Int</span><span class="p">)</span>
<span class="p">]</span>
<span class="c1">-- This example demonstrates just one LEFT JOIN for simplicity purposes.</span>
<span class="c1">-- But, if you can write a plain SQL query in SQLite, you can write it here.</span>
<span class="c1">-- So it can be as complicated as you want it to be.</span>
<span class="c1">-- RECOMMENDED: Keep things simple.</span>
<span class="n">getAllFavAnime</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="p">[</span><span class="kt">FavouriteAnime</span><span class="p">]</span>
<span class="n">getAllFavAnime</span> <span class="n">conn</span> <span class="o">=</span> <span class="n">query_</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
SELECT fa.id
, fa.anime_name
, fa.favourite_character
, p.name
FROM favourite_anime fa
LEFT JOIN person p ON p.id = fa.person_id
<span class="o">|]</span>
<span class="n">flushAnimeTable</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">flushAnimeTable</span> <span class="n">conn</span> <span class="o">=</span>
<span class="n">execute_</span> <span class="n">conn</span> <span class="o">[</span><span class="n">sql</span><span class="o">|</span>
DROP TABLE IF EXISTS favourite_anime;
<span class="o">|]</span>
<span class="c1">-- And the corresponding changes to @resetDb@ and @main@</span>
<span class="n">resetDb</span> <span class="o">::</span> <span class="kt">Connection</span> <span class="o">-></span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">resetDb</span> <span class="n">conn</span> <span class="o">=</span> <span class="kr">do</span>
<span class="c1">-- drop all the tables</span>
<span class="n">flushPersonTable</span> <span class="n">conn</span>
<span class="n">flushAnimeTable</span> <span class="n">conn</span>
<span class="c1">-- create fresh ones</span>
<span class="n">createPersonTable</span> <span class="n">conn</span>
<span class="n">createAnimeTable</span> <span class="n">conn</span>
<span class="n">main</span> <span class="o">::</span> <span class="kt">IO</span> <span class="nb">()</span>
<span class="n">main</span> <span class="o">=</span> <span class="kr">do</span>
<span class="n">conn</span> <span class="o"><-</span> <span class="n">open</span> <span class="s">"test.db"</span>
<span class="n">resetDb</span> <span class="n">conn</span>
<span class="n">createNewPerson</span> <span class="n">conn</span>
<span class="n">insertFavouriteAnime</span> <span class="n">conn</span>
<span class="n">pRows</span> <span class="o"><-</span> <span class="n">getAllPerson</span> <span class="n">conn</span>
<span class="n">faRows</span> <span class="o"><-</span> <span class="n">getAllFavAnime</span> <span class="n">conn</span>
<span class="n">mapM_</span> <span class="n">print</span> <span class="n">pRows</span>
<span class="n">mapM_</span> <span class="n">print</span> <span class="n">faRows</span>
<span class="n">close</span> <span class="n">conn</span>
</code></pre></div></div>
<p align="center">
<img src="../assets/images/6_post/anime-nerd.png" style="width:350px;height:300px;" />
</p>
<p>Let’s reload the REPL and run the <code class="language-plaintext highlighter-rouge">main</code> function,</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">*</span><span class="kt">SqliteSimple</span><span class="o">></span> <span class="o">:</span><span class="n">r</span>
<span class="p">[</span><span class="mi">1</span> <span class="kr">of</span> <span class="mi">1</span><span class="p">]</span> <span class="kt">Compiling</span> <span class="kt">SqliteSimple</span> <span class="p">(</span> <span class="n">src</span><span class="o">/</span><span class="kt">SqliteSimple</span><span class="o">.</span><span class="n">hs</span><span class="p">,</span> <span class="n">interpreted</span> <span class="p">)</span>
<span class="kt">Ok</span><span class="p">,</span> <span class="n">one</span> <span class="kr">module</span> <span class="err">loaded.</span>
<span class="err">*</span><span class="nn">SqliteSimple</span><span class="o">></span> <span class="n">main</span>
<span class="kt">Person</span> <span class="p">{</span><span class="n">pUUID</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="n">pName</span> <span class="o">=</span> <span class="s">"Rengoku"</span><span class="p">,</span> <span class="n">pEmail</span> <span class="o">=</span> <span class="s">"rengoku@demonslayer.com"</span><span class="p">,</span> <span class="n">pRole</span> <span class="o">=</span> <span class="kt">Hashira</span><span class="p">,</span> <span class="n">pNumber</span> <span class="o">=</span> <span class="s">"+91234567899"</span><span class="p">,</span> <span class="n">pJoinedAt</span> <span class="o">=</span> <span class="mi">2021</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="mi">25</span> <span class="mi">10</span><span class="o">:</span><span class="mi">46</span><span class="o">:</span><span class="mi">47</span> <span class="kt">UTC</span><span class="p">}</span>
<span class="kt">FavouriteAnime</span> <span class="p">{</span><span class="n">faUUID</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="n">faAnimeName</span> <span class="o">=</span> <span class="s">"Attack on titans"</span><span class="p">,</span> <span class="n">faFavouriteCharacter</span> <span class="o">=</span> <span class="s">"Levi Ackerman"</span><span class="p">,</span> <span class="n">faUserName</span> <span class="o">=</span> <span class="s">"Rengoku"</span><span class="p">}</span>
</code></pre></div></div>
<p>Just as expected!! These are just some basic stuff I wanted to write about. These can surely be extended to do a lot of complicated stuff as well
if it fits the relational data model. So feel free to experiment as much as you want.</p>
<p>One concern now is that these queries are unsafe in nature. These will result into runtime errors and we want to avoid that. One of the ways of doing that is to wrap
the queries provided by <code class="language-plaintext highlighter-rouge">sqlite-simple</code> into some <code class="language-plaintext highlighter-rouge">Maybe</code> or <code class="language-plaintext highlighter-rouge">Either</code> types to catch these kind of errors when running tests.</p>
<p>p.s. Always write tests.</p>Nitin PrakashAs a Haskell programmer, it’s quite natural to think about type safety almost always. Even when it comes to doing simplest of the things. There’s a whole bunch of debate on why dead simple database packages like postgres-simple or sqlite-simple should be avoided generally. The inlcination is towards libraries which offer more type safety (not to mention complexity comes along with it) like Persistent. It offers a lot of features which guarantees type safety with SQL queries. Under the hood, it makes a ton of template haskell usage to produce a large amount of types and type classes to simplify serialization to and from the database. But the reality is, persistent is not as easy to use as *-simple libraries mentioned earlier. And it’s not always worth it to make things complex for a bit of type safety. If our use case is simple enough (and even in most of the code bases with heavy database usage, it really is) we can probably get away with writing plain SQL queries with enough type safety for things to not break at runtime.Clojure. An introduction2020-04-25T00:00:00+00:002020-04-25T00:00:00+00:00https://thenitinprakash.com/2020-04-25/Into-Clojure<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Whatever your background, Clojure is almost guaranteed to affect
the way you think about programming.
- Eli Bendersky
</code></pre></div></div>
<p>I’ve been meaning to start a series on Clojure for quite a while now. I started exploring functional programming around June, 2019 and stumbled upon this language. I sure had my doubts on how impactful it might be. However, I was quite shocked with the pleasant development experience. The elegant integration with emacs and the joy of using REPL are all something that keeps me enganged in exploring clojure more and more everyday. I can keep going but first, let’s dig into the background of Clojure so that we don’t get lost right at the beginning.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A language that doesn't affect the way you think about programming,
is not worth knowing.
- Alan Perlis
</code></pre></div></div>
<h3 id="what-is-clojure">What is clojure?</h3>
<ul>
<li>A functional programming language.</li>
<li>A complied language. Yet dynamic in nature i.e., clojure features are also supported at runtime.</li>
<li>Values built-in data structure philosphy. That is, it provides built-in support for <em>map</em>, <em>vector</em>, <em>hash-map</em>, <em>list</em> etc.</li>
<li>Provides persitent data structures that essentially focuses on immutability and multi-threading.</li>
<li>Shares <em>code as data</em> philosphy with a macro system.</li>
</ul>
<p>For those of you who know, it’s a LISP! Don’t get confused now. Being a lisp does not mean it can’t be used to build production ready softwares. It’s fairly important to understand that clojure comes with a lot of features alongside being a lisp.</p>
<p align="center">
<img src="../assets/images/2020-04-25-clojure_intro.jpg" />
</p>
<p>Having said that, clojure provides a very powerful support for interfaces. These interfaces are called <em>protocols</em> and discourages thinking in terms of classes with inheritance. It supports <em>multiple dispatch</em> functions as well. Clojure also provides a great support for concurrency and parallelism in multiple ways.</p>
<h3 id="code-as-data">Code as data</h3>
<p>Let’s consider the follwing block of code. By the looks of it, anyone can say that it takes two arguments, <em>x</em> and <em>y</em> and returns the addition of the two.</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="k">defn</span><span class="w"> </span><span class="n">add</span><span class="w">
</span><span class="p">[</span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="p">]</span><span class="w">
</span><span class="p">(</span><span class="nb">+</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="p">))</span><span class="w">
</span></code></pre></div></div>
<p>Hold your horses. Let’s take a close look and try to understand the way it can be interpreted. In the above code,</p>
<ul>
<li><em>x</em> and <em>y</em> are <strong>symbols</strong>. A symbol is basically an object representing a name. These are first class names in Clojure.</li>
<li>[x y] is a vector containing the symbols.</li>
<li>(+ x y) is a linked list data structure containing two symbols <em>x</em> and <em>y</em>.</li>
</ul>
<p>From the above, I want to infer that when we talk about any piece of clojure code,</p>
<ul>
<li>We talk about a code that will execute.</li>
<li>We think of it as a data structure consisting of <em>vectors</em>, <em>maps</em>, <em>intergers</em>, <em>strings</em> etc.</li>
</ul>
<p>In computer programming, this is called <a href="https://en.wikipedia.org/wiki/Homoiconicity">Homoiconicity</a>. Clojure is said to be homoiconic. In simple terms, it is said to share <em>code as data</em> philosphy. Code forms are data structures and data structures can be thought of as forms and executed as code.</p>
<p align="center">
<img src="../assets/images/2020-04-25-code_as_data.jpg" style="width:350px;height:300px;" />
</p>
<p>We’ll look at some basic terms before diving deep into web programming.</p>
<h3 id="functions">Functions</h3>
<p>Creating and using functions, and creating functions that use functions, is what Clojure programers do. Clojure has a number of ways to create a function. I’ll give an overview on some of them.</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="k">def</span><span class="w"> </span><span class="n">print-string</span><span class="w">
</span><span class="p">(</span><span class="k">fn</span><span class="w"> </span><span class="p">[]</span><span class="w"> </span><span class="s">"That's cute. What else clojure is capable of?"</span><span class="p">))</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/print-string</span><span class="w">
</span><span class="p">(</span><span class="nf">print-string</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="s">"That's cute. What else clojure is capable of?"</span><span class="w">
</span></code></pre></div></div>
<p><em>def</em> is used declare a variable. <em>fn</em> creates a simple function object. It can be stored in a var, passed to functions etc.
<em>defn</em> is a macro that makes defining functions a little bit simpler. The above can be written as:</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="k">defn</span><span class="w"> </span><span class="n">print-string</span><span class="w">
</span><span class="p">[]</span><span class="w">
</span><span class="s">"That's cute. What else clojure is capable of?"</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/print-string</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nf">print-string</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="s">"That's cute. What else clojure is capable of?"</span><span class="w">
</span></code></pre></div></div>
<p>We can also overload arity in a single function, self-reference it.</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="k">defn</span><span class="w"> </span><span class="n">sum</span><span class="w">
</span><span class="p">([</span><span class="n">x</span><span class="p">]</span><span class="w"> </span><span class="n">x</span><span class="p">)</span><span class="w">
</span><span class="p">([</span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="p">]</span><span class="w"> </span><span class="p">(</span><span class="nb">+</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="p">))</span><span class="w">
</span><span class="p">([</span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="n">z</span><span class="p">]</span><span class="w"> </span><span class="p">(</span><span class="nb">+</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="w"> </span><span class="n">z</span><span class="p">)))</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/sum</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nf">sum</span><span class="w"> </span><span class="mi">1</span><span class="p">)</span><span class="w">
</span><span class="mi">1</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nf">sum</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="mi">2</span><span class="p">)</span><span class="w">
</span><span class="mi">3</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nf">sum</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="mi">4</span><span class="p">)</span><span class="w">
</span><span class="mi">7</span><span class="w">
</span></code></pre></div></div>
<h3 id="repl">REPL</h3>
<p>This stands for <em>Read-Eval-Print-Loop</em>. The REPL reads in text through the reader and further transforms it into a Clojure data structure. We can imagine REPL as our playground where we can test running code or even try out new ideas.</p>
<p>The reason it is such a fantastic feature is because of the feedback loop which tremendously enhances development cycle. This kind of feature is not available for most languages. We’ll see the power of REPL in further sections. You can also read more about REPL <a href="https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop">here</a>.</p>
<h3 id="pure-functions">Pure functions</h3>
<p>These are functions that will always produce the same result for a given input. These cannot have any observable side effects and don’t depend on any kind of outside state, other than that which was given as arguments to the function. The result of such functions don’t change during the execution of the program or between executions of the program, as the dependency on outside state can lead to changes in the result of the function.</p>
<blockquote>
<p>Side effects are observed application state change outside a function</p>
</blockquote>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="k">defn</span><span class="w"> </span><span class="n">product</span><span class="w">
</span><span class="p">[</span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="p">]</span><span class="w"> </span><span class="p">(</span><span class="nb">*</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="n">y</span><span class="p">))</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/sum</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nf">product</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="mi">2</span><span class="p">)</span><span class="w">
</span><span class="mi">2</span><span class="w">
</span></code></pre></div></div>
<p>The above function <em>product</em> will always result the same for a fixed <em>x</em> and <em>y</em>. Even the examples above where we discussed multi arity were pure functions.</p>
<p>But,</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="k">def</span><span class="w"> </span><span class="n">principal</span><span class="w"> </span><span class="mi">1000</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/amount</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="k">defn</span><span class="w"> </span><span class="n">interest</span><span class="w">
</span><span class="p">[</span><span class="nb">time</span><span class="w"> </span><span class="n">rate</span><span class="p">]</span><span class="w"> </span><span class="p">(</span><span class="nb">/</span><span class="w"> </span><span class="p">(</span><span class="nb">*</span><span class="w"> </span><span class="n">principal</span><span class="w"> </span><span class="nb">time</span><span class="w"> </span><span class="n">rate</span><span class="p">)</span><span class="w"> </span><span class="mi">100</span><span class="p">))</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/interest</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nf">interest</span><span class="w"> </span><span class="mi">10</span><span class="w"> </span><span class="mi">5</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="mi">50000</span><span class="w">
</span></code></pre></div></div>
<p>The above function <em>interest</em> is not a pure function as it depends on a variable <em>principal</em>. If it changed, <em>interest</em> would produce a different result for the same <em>time</em> and <em>rate</em>. These are called <strong>impure functions</strong>.</p>
<h3 id="keywords">Keywords</h3>
<p>According to Clojure documentation, <em>keywords</em> can be described as symbols that evaluate to themeselves.</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">user></span><span class="w"> </span><span class="no">:foobar</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="no">:foobar</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nb">keyword?</span><span class="w"> </span><span class="no">:foorbar</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="n">true</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="nb">keyword</span><span class="w"> </span><span class="mi">100</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="n">nil</span><span class="w">
</span></code></pre></div></div>
<p>Don’t worry about the functioning of <code class="language-plaintext highlighter-rouge">keyword?</code> right now. You can perhaps go through <a href="https://clojure.org/api/cheatsheet">cheatsheet</a> that has various built-in functions listed categorically. What I would like you to understand is that keywords provide a very fast equality test. And also these are the keys of a hashmap in clojure.</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="k">def</span><span class="w"> </span><span class="n">sample-map</span><span class="w"> </span><span class="p">{</span><span class="no">:a</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="no">:b</span><span class="w"> </span><span class="mi">2</span><span class="p">})</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/sample-map</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="n">sample-map</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="p">{</span><span class="no">:a</span><span class="w"> </span><span class="mi">1</span><span class="n">,</span><span class="w"> </span><span class="no">:b</span><span class="w"> </span><span class="mi">2</span><span class="p">}</span><span class="w">
</span><span class="n">user></span><span class="w"> </span><span class="p">(</span><span class="no">:a</span><span class="w"> </span><span class="n">sample-map</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="mi">1</span><span class="w">
</span></code></pre></div></div>
<h3 id="collections-and-sequences">Collections and Sequences</h3>
<p>There a lot of operations over collections and sequences provided by clojure. But before diving into those operation, we need to understand the difference between a <em>sequence</em> abstraction and a <em>collection</em> abstraction.</p>
<blockquote>
<p>Every sequence is a collection, but not every collection is a sequence.</p>
</blockquote>
<p align="center">
<img src="../assets/images/2020-04-25-seq_vs_coll.jpg" style="width:350px;height:300px;" />
</p>
<p>The collection abstraction is closely related to the sequence abstraction. All of Clojure’s core data structures — vectors, maps, lists and sets — take part in both abstractions. The main difference is that <em>sequence</em> is about operating on the <em>individual element</em> that is expected to be accessed in linear fashion whereas <em>collection</em> is more about the overall data structure.</p>
<div class="language-clojure highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">user.core></span><span class="w"> </span><span class="p">(</span><span class="k">def</span><span class="w"> </span><span class="n">foo</span><span class="w"> </span><span class="o">'</span><span class="p">[</span><span class="mi">1</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="mi">4</span><span class="p">])</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="o">#</span><span class="ss">'user/foo</span><span class="w">
</span><span class="n">user.core></span><span class="w"> </span><span class="n">foo</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="p">[</span><span class="mi">1</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="mi">3</span><span class="w"> </span><span class="mi">4</span><span class="p">]</span><span class="w">
</span><span class="c1">;; An individual element can't be empty.</span><span class="w">
</span><span class="c1">;; Only the whole data structure can be emtpty.</span><span class="w">
</span><span class="c1">;; Therefore, when we talk about collections we</span><span class="w">
</span><span class="c1">;; are more so referring to the data structure as whole</span><span class="w">
</span><span class="n">user.core></span><span class="w"> </span><span class="p">(</span><span class="nf">empty?</span><span class="w"> </span><span class="n">foo</span><span class="p">)</span><span class="w">
</span><span class="n">=></span><span class="w"> </span><span class="n">false</span><span class="w">
</span></code></pre></div></div>
<p><a href="https://stackoverflow.com/a/22439707/6244324">This stackoverflow answer</a> can be helpful in getting the overview easily.</p>
<h3 id="starting-a-simple-project">Starting a simple project</h3>
<p>This section focuses on generating an outline for a simple clojure project (not for web programming). We will use <a href="https://leiningen.org/">Leiningen</a> as our tool to automate any sort of clojure project throughout the series. There are others such as <em>boot</em> but we will focus on <em>lein</em>.</p>
<p>To generate a project skeleton, a simple <em>lein</em> command can do the trick:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>lein new app clojure-first
</code></pre></div></div>
<p>This should genetate a structure as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>├── CHANGELOG.md
├── LICENSE
├── README.md
├── doc
│ └── intro.md
├── project.clj
├── resources
├── src
│ └── clojure_first
│ └── core.clj
├── target
└── test
└── clojure_first
└── core_test.clj
</code></pre></div></div>
<p>To run the project,</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>lein run
</code></pre></div></div>
<p>This command successfully runs because there is a <code class="language-plaintext highlighter-rouge">-main</code> function in <code class="language-plaintext highlighter-rouge">src/clojure_first/core.clj</code> which is the entry point for the app. Remove that and you might end up setting your hair on fire. Well, to ensure that doesn’t happend we depend on lein.</p>
<p>To try out repl, you can run</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>lein repl
</code></pre></div></div>
<h3 id="conclusion">Conclusion</h3>
<p>In this post we figured out:</p>
<ol>
<li>What is clojure?</li>
<li>Clojure philosphy.</li>
<li>Basic terms related to clojure.</li>
<li>Starting a simple clojure project.</li>
</ol>
<p>I’m not going to debate on what editor you should and should not use. But I highly recommend one of the follwing:</p>
<ol>
<li>Emacs + cider</li>
<li>IntelliJ + Cursive</li>
</ol>
<p>Both of the above are the most used development environment for clojure. I personally use emacs for clojure development.</p>
<p>In the next post we’ll setup a web project and try to understand it’s working.</p>Nitin PrakashWhatever your background, Clojure is almost guaranteed to affect the way you think about programming. - Eli BenderskyDynamic Programming2019-03-17T00:00:00+00:002019-03-17T00:00:00+00:00https://thenitinprakash.com/2019-03-17/Dynamic-Programming-in-Reinforcement-Learning<p>Dynamic programming is a technique for solving a complex problem by breaking it down into a collection of simpler subproblems. Smaller subproblems are solved only once and stored in a data structure like array, map etc. The fundamental idea being to optimize a given recursive problem and improve upon the complexity.</p>
<p>Stating the same as an RL paradigm:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Dymanic programming refers to solving a given problem using a collection
of algorithms that computes optimal policies given a perfect model of the
environment as a finite Markov decision process.
</code></pre></div></div>
<p>From the above reference, it should be clear that when we talk about DP we are just formulating a structure for the search of an optimal policy. In any paradigm of solving problems, DP requires two conditions to be satisfied:</p>
<ul>
<li>Optimal substructure</li>
<li>Ovarlapping subproblems</li>
</ul>
<p>Fortunately, MDPs satisfy both the properties.</p>Nitin PrakashDynamic programming is a technique for solving a complex problem by breaking it down into a collection of simpler subproblems. Smaller subproblems are solved only once and stored in a data structure like array, map etc. The fundamental idea being to optimize a given recursive problem and improve upon the complexity.Markov Decision Process2018-09-09T00:00:00+00:002018-09-09T00:00:00+00:00https://thenitinprakash.com/2018-09-09/Markov-Decision-Process<p>In the previous post we saw the what kind of technicalities Reinforcement learning present to us. We know what an agent-environment interface is. We also looked up the reward-punishment process. But there’s a lot of questions that still need to be answered. For example,</p>
<ul>
<li>Where’s the boundary in agent-environment interface?</li>
<li>Do we rely on immediate rewards and overlook that reward generated in long term?</li>
<li>What are the characteristics of a state an agent is present in?</li>
</ul>
<p>I’ll try to answer such questions while explaining MDPs.</p>
<p>MDPs are pretty much the whole process of an agent making decision and trying to maximize rewards. But things are much more complex that just that statement. To understand MDPs, we’ll first need to know what a Markov process is.</p>
<h2 id="markov-processes">Markov Processes</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
A Markov process is a sequence of states over
disctrete time steps in such a way that the
past decisions do not affect the future decision making process.
</code></pre></div></div>
<p>The latter phrase is also know as the <strong>Markov Proporty</strong>. Formally, Markov property states that <em>The future is independent of the past given the present</em>. We can represent a markov process mathematically using a tuple <strong>(S, T)</strong>. Here <strong>S</strong> represents a finite set of states and <strong>T</strong> is the state tranisition probabilty matrix. So basically if an agent makes a transition from a state <strong>S</strong> to <strong>$S’$</strong>, we can represent the markov process as:</p>
\[\begin{align}
P_{SS'} = p\left[S_{t + 1} = S' | S_{t} = S\right]
\end{align}\]
<p align="center">
<img src="../assets/images/2018-09-09-state.png" />
<p style="font-size: 70%" align="center">© Sutton & Barto</p>
</p>
<p>Let’s break down the above equation into much simpler statements. Imagine an agent interacting with an environmnet over a series of time steps <em>t = 0, 1, 2…</em>. For each time step, the agent recieves a representation of the environment’s state which further results into the action taken. And when the agent recieves a numerical reward <strong>$R_{t + 1}$</strong> for the action taken in the corresponding state <strong>$S_t$</strong>, it finds itself in a new state <strong>$S_{t + 1}$</strong>. So the series will look like:</p>
\[\begin{align}
S_0, A_0, R_1, S_1, A_1, R_2, S_2, A_2, R_3,...
\end{align}\]
<p>Note that we are considering discrete time steps only because it’s a lot more mathematically convenient although the idea can be extended to constinuous time steps.</p>
<p>In a finite MDP, the sets of states, actions, and rewards <em>(S, A, and R)</em> all have a finite number of
elements. In this case, the random variables <em>$R_t$</em> and <em>$S_t$</em> have well defined discrete probability distributions dependent only on the preceding state and action. This means that for particular values of these random variables, $s \in S$ and $r \in R$, there is a probabilty of those values occuring at time step <em>t</em>, given a particular value of peceding state and action. Mathematically, it can be stated as:</p>
\[\begin{align}
p\left(s', r | s, a\right) = Pr\left\{S_{t} = s', R_{t} = r | S_{t − 1} = s, A_{t − 1} = a\right\}
\end{align}\]
<p>The above definition is nothing but markov process with values. There are a lot of other notations that are conventionally used. In order to make the Markov process representation easier for us, let’s simplify it a little bit. Earlier, we had a tuple <strong>(S, T)</strong> that defined a markov process. Now we add two more elements to it, <strong>R</strong> and <strong>$\gamma$</strong>. Therefore, Markov reward process can be stated as <strong>(S, P, R, $\gamma$)</strong>, where:</p>
<ul>
<li>
<p>Reward function <strong>R</strong>,
\(\begin{align}
R_s = E\left[R_{t+1} | S_t = s\right]
\end{align}\)</p>
</li>
<li>
<p>State transition probabity matrix <strong>P</strong>,
\(\begin{align}
P_{SS'} = p\left[S_{t + 1} = S' | S_{t} = S\right]
\end{align}\)</p>
</li>
<li>
<p>Discount factor,
\(\begin{align}
\gamma \in [0, 1]
\end{align}\)</p>
</li>
</ul>
<p>Even after all such definitions, MDP framework is flexible and the idea can be extended to much more complex problems. One of them being considering continuous time duration instead of discrete time steps. Even the idea of actions can be extended. For example, some actions might control what an agent chooses to think about, or where it focuses its attention. There is no physical boundary between an agent and an environment. The general rule we follow is that anything that cannot be changed arbitrarily by the agent is considered to be outside of it and therefore part of the environment. We do not assume that everything in the environment is unknown to the agent. Also, the boundary can be located at different places for different purposes. The boundary for making high-level decisions can vary from the boundary for low-level decisions.</p>
<p>Now that we know what a Markov process is, we can generalize that any learning task that is based on actions, states and rewards can fit into this particular segment. Although, it is to be kept in mind that <em>all the decision based learning problem might not be a markov process</em>.</p>
<h3 id="examples">Examples</h3>
<p>Let’s go ahead and frame some examples that we can think of as a Markov process.</p>
<p>One such example could be of a robot trying to escape from a maze. We can define the possible states as the position of the robot at a particular time step. The actions can be said to be the path it decides to take. For example, to move forward, backwards, or sideways. Rewards can be negative if the robot collides with a wall for his action and positive for every correct action. Therefore, this particular example can be stated as a tuple (S, P, R, $\gamma$) which is the definition of a Markov reward process.</p>
<p><br /></p>
<p align="center">
<img src="../assets/images/2018-09-09-maze.jpg" />
</p>
<p><br />
Another example can be learning how to drive a car around the streets. The state could be a vector representing the distance to each of the lateral sides, the directional heading, the velocity and acceleration of the car. The actions could be to change any of the state vectors i.e. to accelerate or decelerate or to change heading. The rewards could be zero if the car is proceeding comfortably along and minus one if the car collides with an obstable(person, wall etc).</p>
<p align="center">
<img src="../assets/images/2018-09-09-cars.png" />
<p style="font-size: 70%" align="center">© xkcd</p>
</p>
<h2 id="returns-and-value-function">Returns and Value function</h2>
<p>Until now we’ve looked into the kind of reinforcement learning tasks and the concept of states. Previously, we saw that the agent’s goal is to maximize the cumulative reward it receives in the long run. How might this be defined formally? I’ll be answering such questions in this section.</p>
<p>Let’s suppose that sequence of rewards received after time step <em>t</em> is denoted $R_{t+1}$, $R_{t+2}$ , $R_{t+3}$. What we need to do is maximize the <strong>expected return</strong>, denoted by <strong>$G_t$</strong>, is defined as some specific function of the reward sequence. We say some specific function because there are multiple forms to it. The simplest being:</p>
\[\begin{align}
G_t = R_{t+1} + R_{t+2} + R_{t+3} + \cdots + R_T
\end{align}\]
<p>where <strong>T</strong> is the final time step in the sequence. But what if <strong>T = $\infty$</strong> i.e., the task goes on to continue without any limit. Then the return value as per the stated formula becomes problematic because it could easily tend to infinty. Therefore, we introduce the concept of <em>discounting</em>. According to this approach, the agent tries to select actions so that the sum of the discounted rewards it receives over the future is maximized. In particular, it chooses $A_t$ to maximize the expected <strong>discounted return</strong>.</p>
\[\begin{align}
G_t & = R_{t+1} + \gamma R_{t+2} + \gamma^2R_{t+3} + \cdots \\
& = \sum_{k=0}^\infty \gamma^k R_{t + k + 1}
\end{align}\]
<ul>
<li>The discount $\gamma \in [0, 1]$ is the present value of future rewards.</li>
<li>The value of receiving reward R after k + 1 time-steps is $\gamma^k R$.</li>
<li>If $\gamma$ = 0, the agent is <strong>myopic</strong> in being concerned only with maximizing immediate rewards. The objective being learning how to choose $A_t$ in order to maximize only $R_{t+1}$.</li>
<li>If $\gamma$ = 1, the return objective takes future rewards into account more strongly. We can say that the agent becomes more <strong>farsighted</strong>.</li>
</ul>
<p>Here, the discount rate determines the present value of future rewards: a reward received k time steps in the future is worth only $\gamma^{k − 1}$ times what it would be worth if it were received immediately. But other than being mathematically convenient, what may be the possible advantages of discounted return? One of the factor is that discounted return avoid infinte returns in a cyclic markov process as the math doesn’t work out well for cyclic cases. Another reason is the uncertainity about the future as it may not be fully represented.</p>
<h2 id="policies-and-value-functions">Policies and Value Functions</h2>
<p>There is a notion of <em>effective state</em> that measures the future rewards in terms of expected return. These functions of states that measure it’s effectiveness i.e., how good it is for an agent to be in that particular state are called <strong>Value functions</strong>. But value functions are defined with respect to particular ways of acting, called <strong>policies</strong>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Formally, a policy can be defined as a mapping from states
to probabilities of selecting each possible action.
</code></pre></div></div>
<p>A policy in reinforcement learning is denoted by <strong>$\pi$</strong>. So, if an agent is following a policy $\pi$ at a given time step t, then $\pi(a|s)$ is the probability that $A_t$ = a if $S_t$ = s. The value of a state <em>s</em> under a policy <em>$\pi$</em>, denoted $v_\pi (s)$, is the expected return when starting in <em>$s$</em> and following <em>$\pi$</em> thereafter. Basically, a value function of a state tells us how <em>good</em> it is for our agent to be in that particular state under some policy $\pi$. Therefore, if we state the value mathematically for MDPs</p>
\[\begin{align}
v_\pi (s) = E_\pi\left[G_t | S_t = s\right]
\end{align}\]
<p>where $E_π[]$ denotes the expected value of a random variable given that the agent follows policy <em>$\pi$</em>, and <em>t</em> is any time step. Similarly, we define the value of taking action <em>a</em> in state <em>s</em> under a policy $\pi$ as <em>Action-value function for policy $\pi$</em> and it can be stated as</p>
\[\begin{align}
q_\pi (s, a) = E_\pi\left[G_t | S_t = s, A_t = a\right]
\end{align}\]
<p>The value functions $v_{\pi}$ and $q_{\pi}$ are estimated from experience. In general, we solve a reinforcement learning task by finding a policy that maximizes the output of the value function over the long run. Therefore, out of existant policies, we chose one that generates the maximum reward. A policy $\pi$ will be preferred over a policy $\pi’$ if its expected return is greater than or equal to that of $\pi’$ for all states. This is called an optimal policy. Note that if there exists more than one optimal policy then all of them must share the same state-value function and action-value function.</p>
<h2 id="bellman-equation">Bellman Equation</h2>
<p>The Bellman equation expresses the relationship between a value of a state and the value of a successor state and they are crucial to understand how Reinforcement Learning algorithms work. To derive Bellman equation, we go back to the value function,</p>
\[\begin{align}
v_\pi (s) = E_\pi\left[G_t | S_t = s\right]
\end{align}\]
<p>This value function can be decomposed into two parts:</p>
<ul>
<li>Immediate reward $R_{t+1}$</li>
<li>And discount values of succeding states</li>
</ul>
\[\begin{align}
v_\pi (s) & = E\left[G_t | S_t = s\right]\\
& = E\left[R_{t+1} + \gamma R_{t+2} + \gamma^2R_{t+3} + \cdots | S_t = s\right] \\
& = E\left[R_{t+1} + \gamma (R_{t+2} + \gamma R_{t+3} + \cdots) | S_t = s\right] \\
& = E\left[R_{t+1} + \gamma G_{t+1} | S_t = s\right]
\end{align}\]
<p>In the above equation $G_{t+1}$ is the discounted expected return of successor states and can be replaced by $v(S_{t+1})$, giving us:</p>
\[\begin{align}
v_\pi (s) & = E\left[G_t | S_t = s\right]\\
& = E\left[R_{t+1} + \gamma v(S_{t+1}) | S_t = s\right]
\end{align}\]
<p>This equation is called the Bellman’s equation for a Markov process. Keep in mind that Bellman equation exists for both value function and action-value function.</p>
\[\begin{align}
q_\pi (s, a) & = E_\pi\left[G_t | S_t = s, A_t = a\right]\\
& = E\left[R_{t+1} + \gamma q_\pi(S_{t+1}, A_{t+1}) | S_t = s, A_t = a\right]
\end{align}\]
<p>We can see that Bellman equation is a linear equation. So if we know the value of $s_{t+1}$, we can easily calculate the value of $s_t$. This opens a lot of doors for iterative approaches for calculating the value for each state, since if we know the value of the next state, we can know the value of the current state.</p>
<p>In the next post, we’ll be utilising the concepts of policies, states and actions to code our first RL agent. We’ll also see how Bellman equation is helpful in RL scenarios.</p>Nitin PrakashIn the previous post we saw the what kind of technicalities Reinforcement learning present to us. We know what an agent-environment interface is. We also looked up the reward-punishment process. But there’s a lot of questions that still need to be answered. For example, Where’s the boundary in agent-environment interface? Do we rely on immediate rewards and overlook that reward generated in long term? What are the characteristics of a state an agent is present in?Diving deep into Reinforcement Learning2018-06-26T00:00:00+00:002018-06-26T00:00:00+00:00https://thenitinprakash.com/2018-06-26/Diving-deep-into-Reinforcement-learning<p>In the <a href="http://www.thenitinprakash.com/blog/2018/06/19/Reinforcement-Learning-What-And-Why/">previous post</a> I gave an introduction to RL and exlpained very briefly how things are and why is it any different from other machine learning methods.
In this post we will go through some examples and try to formalize the idea of RL.</p>
<p>The following digram depucts the working of a reinforcement learning agent precisely.</p>
<p align="center">
<img src="../assets/images/2_post/rl.jpeg" />
</p>
<p>We know that RL is based on rewards and punishements. A lot of complexities could arise when an <em>agent</em> tries to maiximize the reward in a certain situation. To obtain a lot of reward, a reinforcement learning agent must prefer actions that it has tried in the past and found to be effective in producing reward. but in order to have knowledge about such actions, it has to explore it’s options. Some of them can be the first time our agent will be applying those options and other might be repeated. So basically, an agent has to <strong>explore</strong> and <strong>exploit</strong> the already explored actions in order to find a solution. The dilemma is that none of the two can be pursued exclusively without failing at a particular task. The agent must try a variety of actions and progressively favor those that appear to be best.</p>
<p>Unlike other paradigms of Machine Learning, RL explicitly takes the <em>whole</em> problem into account. An RL agent should be able to sense the state of the enviornment and also be able to influence it. The whole point being that the interaction between the agent and environment should be maximum. There can be cases when RL is used in conjunction with supervised learning. Those cases mostly arise when we want eliminate non-critical capabilities of an agent.</p>
<h2 id="examples">Examples</h2>
<p>Now that we know a little more about RL, where can we use it? Let’s consider some examples to understand The above theory:</p>
<ul>
<li>
<p>A chess player decides to make a move. His decision is addressed by anticipating possible counterattacks and by intuitive judgment of the desirability of particular move.</p>
</li>
<li>
<p>OpenAI’s Dota 2 bot is a very recent and excellent example. Success in Dota requires players to develop intuitions about their opponents and plan accordingly. Bot developed by OpenAI has learned — entirely via self-play — to predict where other players will move, to improvise in response to unfamiliar situations. Also, how to influence the other player’s allied units to help it succeed.</p>
</li>
<li>
<p>An Finanacial broker sells his stocks only when he/she can see the possibilty of profit. He/she analyzes the previous data and considers possible shift in the prices of the stocks. Then try to optimize the financial objective overall.</p>
</li>
<li>
<p>A mobile robot decides whether it should enter a new room in search of more trash to collect or start trying to find its way back to its battery recharging station. It makes its decision based on the current charge level of its battery and how quickly and easily it has been able to find the recharger in the past.</p>
</li>
</ul>
<p>All the above examples signify one thing i.e., interaction between the agent and it’s environment despite uncertainity. The agent has to monitor it’s environment frequently in order to keep the interaction maximum. This <em>interaction</em> helps in gaining experience which can improve performance over the course of time.</p>
<h2 id="elements-of-reinforcement-learning">Elements of Reinforcement Learning</h2>
<p>In the <a href="http://www.thenitinprakash.com/blog/2018/06/19/Reinforcement-Learning-What-And-Why/">previous post</a> we saw that there are certain elements to RL:</p>
<ul>
<li>Policy</li>
<li>Reward</li>
<li>Value function</li>
<li>Model of the environment</li>
</ul>
<p>One by one, we’ll take a look what the above elements mean.</p>
<ol>
<li>
<p><strong>Policy</strong>
<br /><br />
A <strong>policy</strong> defines the agent’s behaviour at a given time in a certain environment. In simple terms, we can think of <em>policy</em> as response by the agent to a certain situation. Usually, a <em>policy</em> is represented by a function. We will look at different policy functions and it’s stochastic nature while learning <strong>value iterations</strong>.</p>
</li>
<li>
<p><strong>Reward Signals</strong>
<br /><br />
The name speaks for itself. Every time the agent takes an action, the environment responds back with a numerical digit called <strong>reward</strong> and it defines the track towards goal. A reward at a particular step would mean correct action and a punishment would mean wrong step. Therefore, <strong><em>rewards can be considered as the primary basis for altering policy</em></strong>. If a policy results in low rewards for actions then it can be changed in order to generate more reward for the same kind of action progressilvely.
<br /><br />
Just like policies, reward signals can also be stochastic.</p>
</li>
<li>
<p><strong>Value Function</strong>
<br /><br />
We saw that reward signal specifies whether an action is good or bad for a time step. But what if we want to see if it’s the long run? That’s where <strong>value function</strong> comes into play. A <strong><em>value of a state can be considered as the total amount of reward an agent can expect to accumulate over the future</em></strong>. The sole purpose of estimating values is to maximize reward. Without any dependency on rewards, it doesn’t really make much sense.</p>
</li>
<li>
<p><strong>Model</strong>
<br /><br />
A <strong>model</strong> depicts the behaviour of the environment. For example, given a state and action, the model might predict the resultant next state and next reward. In simpler terms, model can used for <em>planning</em> the course of action after considering possible future situations. Note that <strong><em>Models are optional</em></strong>. There can be a <em>model-based</em> RL or a <em>model-free</em> RL.</p>
</li>
</ol>
<h2 id="tic-tac-toe">Tic-Tac-Toe</h2>
<p>Let’s get our focus on a single example and contrast RL with other approaches. For the sake of convinience, I’ll take an easy game of tic-tac-toe.</p>
<p>A skilled player plays so as never to lose. If not, then what’s the point? Let us assume that we are playing against an imperfect player, one whose play is sometimes incorrect and allows us to win. For simplicity, we will consider draws and losses equally bad.</p>
<p align="center">
<img src="../assets/images/3_post/ttt.jpg" />
</p>
<p>This seems not much of a complicated problem but for once, what if we applied approaches other than RL?</p>
<ul>
<li>Let’s say we apply <strong><em>minimax</em></strong> method from <em>game theory</em>. It would be difficult to apply that in this case because it assumes a particular way of playing by the opponent. Minimax player would never reach a game state from which it could lose. In fact it will always win from that state because of incorrect play by the opponent.</li>
</ul>
<p align="center">
<img src="../assets/images/3_post/tt-lost.jpg" style="width:500px;height:400px;" />
</p>
<ul>
<li>One can also apply <strong><em>dynamic programming</em></strong> to optimize the results. But the problem posed here is that it requires as input a complete specification of that opponent, including the probabilities with which the opponent makes each move.</li>
</ul>
<p>Even though the first point is totally out of the picture now, we’ll see that the second method is not really any different from some of the RL methods.</p>
<h4 id="an-approach-to-play-tic-tac-toe-using-rl">An approach to play tic-tac-toe using RL</h4>
<ul>
<li>
<p>Let’s say we have a table of numbers, one for each state of the game. Each number will be the latest estimate of the probability of our winning from that state. We call this estimate as <em>value</em>. A state will be considered better than the other only if the probablity of winning is higher in that state.</p>
</li>
<li>
<p>Initially, we set values of all states to 0.5, representing a guess that we have a 50% chance of winning.</p>
</li>
<li>
<p>To select our moves we examine the states that would result from each of our possible moves and watch out the values in the table.</p>
</li>
<li>
<p>We will follow a <strong><em>greedy</em></strong> approach now. What I mean to say is that we will select the moves that will lead us to a higher value state or we can say the state with higher probability of winning.</p>
</li>
<li>
<p>Sometimes we make a <strong><em>exploratory</em></strong> random move because they cause us to experience states that we might otherwise never see.</p>
</li>
</ul>
<p align="center">
<img src="../assets/images/3_post/tictacttoe.png" />
<p style="font-size: 70%" align="center">© Sutton & Barto</p>
</p>
<p>In the above diagram, the solid lines represent the moves made during a game. The dashed lines represent moves that the <em>agent</em> considered but did not make. Second move was an exploratory move. It was taken even though another move, the one leading to <strong><em>e<sup>∗</sup></em></strong>, was ranked higher. One thing to note is that <strong><em>exploratory moves do not contribute towards learning of an agent</em></strong>. The curved arrows that lead a state backwards are the <strong>backup</strong> arrows. While our agent is playing a game, we need more accurate estimates of a state. In order to do this, we <em>backup</em> the value of the state after each greedy move to the state before the move. By doing so, we update the value of the previous state to be closer to the next state.</p>
<p>Let <strong>s</strong> denote the state before the greedy move, and <strong>s`</strong> the state after the move, then the update to the estimated value of <strong>s</strong>, denoted <strong><em>V(s)</em></strong>, can be written as</p>
<p align="center">
V(s) ← V(s) + α[V(s`) − V(s)]
</p>
<p><em>α</em> is a small positive fraction called the <strong>step-size parameter</strong>. If this parameter is manipulated correctly over time to converge to an optimal policy then our agent could perform well against opponents who change the style of playing in between the games.</p>
<p>Everything’s fine but what’s the significance of this simple example? This example illustrates the differences between evolutionary methods and methods that learn value functions. There is a clear emphasis on learning while interacting with an environment, in this case with an opponent player. Also, there is a clear goal, and correct behavior requires planning and anticipation that takes into account delayed effects of opponent’s choices.</p>
<p>I’ll be implementing a minimal source code to of the above example once we learn few RL algorithms.</p>
<p>The thing about RL is that it isn’t just limited to low level system. We can apply it to a high level architecture and make it do a lot more. There have been numerous development over the years in this segment</p>Nitin PrakashIn the previous post I gave an introduction to RL and exlpained very briefly how things are and why is it any different from other machine learning methods. In this post we will go through some examples and try to formalize the idea of RL.Reinforcement Learning. What and Why?2018-06-19T00:00:00+00:002018-06-19T00:00:00+00:00https://thenitinprakash.com/2018-06-19/Reinforcement-Learning-What-And-Why<p>Back in 2015, <em>AlphaGo</em>, an AI-powered system played the <a href="https://en.wikipedia.org/wiki/Go">Game of Go</a> against Mr.Fan Hui and Mr.Lee Sedol, the best players of the game, beating them both. This marked a historical accomplishment in the AI-world as it was the first time a machine was able to beat the best of humans in their own game.</p>
<p>Even in Chess, AI-powered systems have been able to gain wins over human players. This is all part of Reinforcement learning. These results are nothing short of amazing. RL is not only used to beat humans at games, it is used in different areas of Computer Science and Technology. Eg:-</p>
<ul>
<li><strong>Power systems</strong> – to predict and minimize transmission losses and enhance Microgrid performance..</li>
<li><strong>Medical image segmentation</strong> - optimally finding the appropriate local thresholding and structuring element values and segment the prostate in ultrasound images.</li>
<li><strong>Robotics</strong> - navigation in indoor environments.</li>
</ul>
<h2 id="what-is-reinforcement-learning">What is Reinforcement Learning?</h2>
<p align="center">
<img src="../assets/images/2_post/goodboy.png" />
<p style="font-size: 70%" align="center">© facebook.com/brevitycomic</p>
</p>
<p>Before we jump onto the definition, let’s go through an analogy that’ll help us build an intuition.</p>
<p>Suppose that you own a dog, how would you train your good boy? Most likely by telling him what to do and if he does it you’ll reward him by giving a cookie. Otherwise wait until he performs the action. If you tell him to roll and he does it, then a cookie. But for some action like tearing up the newspaper, you might give him a time out. So basically, reward for correct action and punishment for wrong ones.</p>
<p>We follow same strategy in RL. In RL, our <em>software agent</em> is concerned with it’s <em>actions</em> in order to maximize the <em>reward</em>. In much simpler terms, RL is learning what we do and how to map situations to actions in order to maximize a numerical reward. The catch here is that the model is never told what to do but left to discover itself by trying out different options and figure out which action yield better reward. This follows nearly the mechanism of a biological Neural network. Why do I say so? Because we tend to learn and perform better when given reward for our actions.</p>
<p>There are some terms that we need to know before actually diving deeper. That’ll give us a complete overview of how things are in RL.</p>
<ul>
<li><strong>Agent</strong>: A system that takes actions to change the state of the environment and gain rewards.</li>
<li><strong>Environment</strong>: The external system that an agent is embedded in, and can perceive and act on.</li>
<li><strong>Reward</strong>: A scalar value which represents the degree to which a state or action is desirable.</li>
<li><strong>State</strong>: This can be viewed as a summary of the past history of the system, that determines its future.</li>
</ul>
<p align="center">
<img src="../assets/images/2_post/rl.jpeg" />
</p>
<p>There are other subelements like <em>policy</em>, <em>model</em> and <em>reward signal</em> but we’ll take those into account later.</p>
<h2 id="how-is-it-any-different-from-supervised-and-unsupervised-learning">How is it any different from Supervised and unsupervised learning?</h2>
<p>Supervised learning is the area of machine learning where learning is done from a training set of labeled examples provided by a knowledgeble external supervisor. Each label is described as a situation with a specification of the correct action the system should take. The sole objective of this is that the system should generalize the responses to certain situations that are not present in the training set. In interactive problems, this approach is not desirable where examples are both correct and represents all the situations an agent has to act upon.</p>
<p align="center">
<img src="https://imgs.xkcd.com/comics/machine_learning.png" />
<p style="font-size: 70%" align="center">© xkcd.com</p>
</p>
<p>It might be possible to mistake RL as a kind of unsupervised learning. Unsupervised learning is the mechanism of inferring a function that describes the structure of <strong>unlabeled</strong>(neither classified nor categorized) data. So bascially, we try to find some kind of structure and pattern hidden in our data. What makes Reinforcement learning different is it’s focus on trying to maximize a reward signal instead of trying to find hidden structure. Yes, finding the structure for an agent can be useful but it will not address the problem of maximizing the numerical reward.</p>
<h2 id="why-reinforcement-learning">Why Reinforcement Learning?</h2>
<p>At the very beginning, I pointed out the power of RL based models and some use cases. But are these all that we can do? <strong>NO</strong>. There’s a lot more to it.</p>
<p>Being influenced by deep learning techniques, it has it’s way to optimize results in supervised problems. It has also been used to implement <em>attention</em> techniques in Image processing. Incorporating deep learning techniques and Reinforcement leraning methods work really well and has proved to be efficient in a lot of different problems. One cannot deny the power of Reinforcement Learning.</p>
<p align="center">
<img src="../assets/images/2_post/rl_xk.png" />
<p style="font-size: 70%" align="center">© xkcd.com</p>
</p>
<p>This blog is a very basic overview of what is RL and why is it important.
I’ve been following Richard Sutton’s and Andrew Barto’s <a href="https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf">Reinforcement Learning: An Introduction</a> and as I proceed, I’ll write the gist of every topic explaining the concepts.
Perhaps try to solve the exercises and write codes for the algorithms described in the book.</p>Nitin PrakashBack in 2015, AlphaGo, an AI-powered system played the Game of Go against Mr.Fan Hui and Mr.Lee Sedol, the best players of the game, beating them both. This marked a historical accomplishment in the AI-world as it was the first time a machine was able to beat the best of humans in their own game.