Immutable Stack 2

I spent my time this past weekend at Kitchen Table Coders' Immutable Stack 2, a two day interactive workshop focusing on Datomic and Om-Next. The workshop was enjoyably taught by David Nolen and Kovas Boguta, who were excellent complements to one another, and who managed to create a relaxed environment where people were free to ask questions throughout. Kovas' section brought out his dry, understated humor, and in Davids' section we witnessed his exuberance for the subject. Largely because of these two, the attitude of the two days was very relaxed: despite being in a room filled with technically competent people, the questions and answers were entirely positive and well respected, resulting in a fast-paced, knowledge-dense two days.

The workshop was premised on the idea of the “immutable stack” with Saturday devoted to Datomic (taught by Kovas) and Sunday devoted to Om-next (David). Although we can glimpse a future of a truly vertical stack, the workshop was taught as two independent units that drew upon one another and culminated in an interactive demo of an app David is writing with Datomic as the datastore and Om-Next on the front end. The workshop was structured as lectures followed by interactive breakout sessions, allowing you to be exposed to a concept and then immediately to practice it in code. David and Kovas have made the repo available.

Datomic#

To me, the most potent and useable ideas of the weekend were related to Datomic. Theres a lot of good stuff packed into Datomic thats hard to appreciate until you start poking around and this is where the workshop format really shined. Kovas introduced the data model, the schemas, and the pull syntax and then we played around with it. At some point during that first morning I had a bit of an ““AHA” moment. The simplicity of the Datomic datom model has a lot of parallels to Hadley Wickam’s tidy data with triplet columns being essentially datoms: the smallest unit of relationship. In the case of R’s (and pandas) dataframes one can then reshape the data using stack/unstack, melt/cast, pivot etc. How does that work with Datomic? Quite well, actually. But quite differently. Whereas in R/Python, you are getting the same join logic as in SQL, the Datomic/datalog model gives a lot more flexibility in how to return the data. Understanding that you can obtain data in essentially any format you desire is part of the beauty of the datomic data model, and part of its natural fit in a JSON world. You still get joins and relationships but its all implicit.

Conceptually the Datomic model might seem a bit confusing but once you learn how to read the query clauses it becomes a very powerful way to query your datasource by succinctly stating the logic of the relationships you are interested in. Furthermore, the “pull syntax” allows you to return data in essentially any way that you could conceivably want (within the constraints of the schema of course). It is this flexibility that gives the Datomic data model a lot if its power (Although at this point I am really only talking about datalog). Some of the immutable, time-based features seemed to ignite interest among the DB admins and backend engineers among us. For example, when Kovas introduced the transaction model and the benefits that can come when, for example, you query only the transactions committed to the database within a given time window, there was some noticeable excitement from some of those guys. Although I don’t need this feature, its clear to see the benefit - for audits, for “git blame”, for whatever. Everything is data; its “turtles all the way down”: the schemas and tranactions are also entities/attribute/values within your database just as your actual data is. And these can be queried just as your data is queried. As a last point, the simple model EASILY accommodates ragged data since there is nothing equivalent to table-level enforcement of values. If data is heterogenous and a particular entity doesn’t have a value - then it simply doesn’t have a value. Thats it. I can see cases where thats not ideal, but then again theres nothing stopping you from validating prior to transacting it in.

I walked out on Saturday night, like most others, with a brain hurting from sustained attention-span and insufficient caffeination (fixed on day 2). A very educational day full of material that would have been difficult to have picked up solo.

Om-Next#

Sunday was Om-next, David’s next iteration of Om, that aims not only to provide a Clojurescript wrapper to Facebook’s React as it had in the past, but also to provide a messaging solution that would allow clients to specify the data needed by their components. In this case his inspiration was Netflix’s Falcor and Facebook’s Relay. These were designed to solve problems about communication between clients and servers by allowing clients to specify the exact information they need. As a bit of a disclaimer, a lot of the problems this technology was designed to solve are foreign to me. I have generated a grand total of zero REST endpoints so the pain of alleviating the problems of hand-coding thousands of endpoints is a problem I can only appreciate in the abstract and, as a corollary, I probably under-appreciated the elegance of the solution David has created.

David’s section was a fair bit of history and then an introduction to Om-Next that tried to stress the main idea: fully stateless components built on top of REACT, and a messaging framework for parsing/mutating appstate and communicating those changes remotely. While I think the main ideas did come through, it was a heavier lift to get up to speed with Om-Next. David introduced the defui macro and the static protocols you attach to these components that allow you to define, in a totally stateless way, the data they use. We then spent a bit of time describing how OM-next parses app-state and handles mutation of data. And we discussed how local appstate can be passed along to the server and did a bit of interactive work that allows us to track and manipulate the data backing a UI without ever going to the browser. Theres a lot of concepts here:

the REACT lifecycle,
the defui Macro and the various protocols attached to it,
the manner in which data is passed to the components,
how this data gets processed by the components and how OM keeps track of it,
and how OM can reconcile new incoming data from the server.

So although I think the logic of what David is achieving was comprehensible, I was unable to grasp in a meaningful way how the data was being passed around the system. Its was difficult for me to assess how much of this is due to my lack of familiarity with front end dev and how much of this is due to the technology being in flux. I can imagine, for example, a much smaller number of concepts being exposed to an end user/developer to get up and running rather quickly. But we were drinking from the firehose and I got soaked.

To Om-Next’s credit the many participants who were front end guys were on the edge of their seats when David walked us through his Om-Next To-Do App (WIP) which was indeed a fully “Immutable stack”. One guy in particular seemed absolutely delighted by the magic many of us were struggling to comprehend. I get the feeling he had written thousands of endpoints.

Recap#

I can heartily recommend a KTC production. The teachers were knowledgeable and accommodating and the material is fresh. I thought the speed of the workshop was about right but if I had a criticism it is the volume of new ideas, and where we left off for each technology. For example, I could imagine a follow up day on Datomic that builds on the basics and ends by running an app instance on AWS backed by Dynamo DB. That would be really fantastic. Similarly, I could imagine a day-2 of Om-Next where we build a front end app and make it talk to a bunch of backends. In short, the ideas we learned were great but, in trying to fit the entire stack into 2 days, some material had to be cut. I think a two day course that takes you from zero to building-your-own thing might be the way to go down the line. However, I only offer that as a suggestion since I feel that I received a great education this weekend, both in terms of ideas and in practical technical knowledge.

Thank You David. Thank You Kovas.

A few other small notes:#

“value proposition” seems to be a thing lots of Clojure guys talk about
Kovas may experience a spiritual awakening and change his name to “World Peace”