In part 1 I've talked about the db-view approach that has different trade-offs in comparison to a REST API. In essence you decide to develop your backend API endpoint with a single frontend app in mind. Thereby your API endpoint and frontend tends to stay simple.
This part is about the example app that I've developed for this blog post series. It implements the well-known TodoMVC example, an app to manage todo items. In comparison to many other TodoMVC implementations out there, it implements the frontend and the backend. It is a core principle of the db-view approach that you design your API exclusively for one frontend app, instead of providing a more generic API. The latter one introduces various complexities which needs to be solved, despite the fact that they are not strictly required for your app.
The source code for the example app can be found here:
https://github.com/maxweber/todomvc-db-view
It uses Clojure for the backend and ClojureScript for the frontend. The Leiningen build tool is used for the Clojure part, while shadow-cljs is used for the frontend. The example README describes how to start the app and the required tools.
The API endpoint /db-view/get is used to receive the :db-view/output map from the server. This is the corresponding frontend code:
(ns todomvc-db-view.db-view.get
(:require [cljs.core.async :as a]
[cljs-http.client :as http]
[todomvc-db-view.state.core :as state])
(:require-macros [cljs.core.async.macros :refer [go]]))
;; Concept:
;;
;; Gets and refreshes the db-view from the server. Provides the server
;; API endpoint '/db-view/get' with the `:db-view/input` map from the
;; app state. Receives the `:db-view/output` map in the API response
;; and stores it in the app state. Reagent will trigger a rerender of
;; all UI components which depend on changed parts of the
;; `:db-view/output`.
(defn get-view
"Provides the server API endpoint '/db-view/get' with the
`:db-view/input` map from the app state and returns the API response
that contains the `:db-view/output` map."
[state-value]
(go
(let [response (<! (http/request
{:request-method :post
:url "/db-view/get"
;; NOTE: for a production app add
;; authorization here and prefer the
;; Transit format:
:edn-params (:db-view/input state-value)}))]
(:body response))))
(defn refresh!
"Provides the server API endpoint '/db-view/get' with the
`:db-view/input` map from the app state, receives the
`:db-view/output` map from the API response and stores it in the app
state."
[]
(go
(:db-view/output
(swap! state/state
assoc
:db-view/output
(<! (get-view @state/state))))))
In the recent years I tend to use a light form of Literate programming, where you provide more explanation of the logic in a natural language. Therefore you find the explanation of the source code in the 'Concept' header, the comments and the doc-strings of the functions above.
The app state is hold in a Reagent atom:
(ns todomvc-db-view.state.core
(:require [reagent.core :as r]))
;; Concept:
;;
;; Namespace with the Reagent atom `state` that holds the global app
;; state of this ClojureScript app.
(def initial-state-value
{:db-view/input {:todo/list {}}})
(defonce state
(r/atom initial-state-value))
The counterpart for the /db-view/get API endpoint on the server-side looks like this:
(ns todomvc-db-view.db-view.get
(:require [datomic.api :as d]
[todomvc-db-view.util.edn :as edn]
[todomvc-db-view.db-view.todo-list :as todo-list]
[todomvc-db-view.db-view.todo-edit :as todo-edit]
[todomvc-db-view.db-view.todo-new :as todo-new]))
;; Concept:
;;
;; Provides the API endpoint to get the db-view. The request body
;; contains the parameters to query the database to assemble the
;; `:db-view/output` map, that contains the required data for the
;; current active UI parts. This value is returned in the response
;; body and the client stores it in the Reagent app state atom, where
;; the UI components can access it.
(defn get-view
"Main entry point to gather the `:db-view/output` map for the
client. Based on the given Datomic database value `db` and the
`:db-view/input` map from the client."
[db db-view-input]
(merge
(todo-list/get-view db
db-view-input)
(todo-edit/get-view db
db-view-input)
(todo-new/get-view db
db-view-input)
;; NOTE: add other db-view parts here.
))
(defn ring-handler
"Ring handler to get the `:db-view/output` map for the given
`:db-view/input` map in the `request` body."
[db request]
(when (and (= (:request-method request) :post)
(= (:uri request) "/db-view/get"))
;; NOTE: for a production app rather use
;; [Transit](https://github.com/cognitect/transit-format)
;; here instead of EDN:
(let [db-view-input (edn/read-string (slurp (:body request)))]
;; NOTE: for a production app do the appropriate authorization
;; checks:
(edn/response
(get-view db
db-view-input)))))
I already included the commands into the example app, which are used to perform side-effects like adding a new todo-item. Commands will be explained in detail in the next blog post. Now let's have a look at the namespace that contains the db-view implementation for the todo list UI:
(ns todomvc-db-view.db-view.todo-list
(:require [datomic.api :as d]))
;; Concept:
;;
;; The db-view part for the todo list UI.
(defn q-all
"Queries all todo-item entities and pulls the attributes which are
required for todo list UI."
[db]
(d/q
'[:find
[(pull ?e
[:db/id :todo/title :todo/done]) ...]
:where
[?e :todo/title]]
db))
(defn get-view
"Returns the db-view for the todo list UI."
[db db-view-input]
(when-let [params (:todo/list db-view-input)]
(let [all (q-all db)
{:keys [active completed]} (group-by (fn [todo-item]
(if (:todo/done todo-item)
:completed
:active))
all)
todo-items (case (:todo/filter params)
:active
active
:completed
completed
all)]
{:todo/list {:todo/list-items todo-items
:todo/active-count (count active)
:todo/completed-count (count completed)}})))
In the code above I removed the command-related parts for the purpose of this blog post, but you can already find it in the Github repo. Altogether the code is really straightforward:
:todo/filter parameter as :todo/list-itemsThe notify part of the app enables the real-time or rather auto-update behaviour of the example app. To try it out, just open the example app in two browser windows, then you see that changes in one window also get replicated in the other window. On the client side the code looks like this:
(ns todomvc-db-view.db-view.notify
(:require [cljs-http.client :as http]
[todomvc-db-view.db-view.get :as get]
[cljs.core.async :as a])
(:require-macros [cljs.core.async.macros :refer [go-loop]]))
;; Concept:
;;
;; Listens for changes in the Datomic transaction log, which affect
;; the current logged-in user. Uses HTTP long polling to allow the API
;; endpoint on the server to push a message. The server will notify
;; the client everytime, when a transaction contains something
;; relevant for the current logged-in user.
;;
;; Due to this mechanism the client can show updates, which were issued
;; by the server (a finished background job for example or a change by
;; another user).
(defn start-listening
"Starts a go-loop that opens a long-polling request to the
'/db-view/notify' API endpoint. Refreshes the `:db-view/output` map
in the app state, when it receives a HTTP 200 response. Sleeps for a
short moment, when it receives an error response to not DDoS the
server in the case of a server issue."
[]
(go-loop []
(let [response (<! (http/request
{:request-method :post
:url "/db-view/notify"}))]
(if (= (:status response)
200)
(<! (get/refresh!))
(<! (a/timeout 2000)))
(recur))))
The server counterpart looks like this:
(ns todomvc-db-view.db-view.notify
(:require [org.httpkit.server :as httpkit]
[datomic.api :as d]
[todomvc-db-view.util.edn :as edn]))
;; Concept:
;;
;; Provides an API endpoint that allows clients to listen for changes
;; in the Datomic transaction log, which affect their current
;; logged-in user.
;;
;; Thereby the client can refresh the `:db-view/output` map as soon as
;; it is notified by this API endpoint. HTTP long polling is used here
;; to allow the server to push a message to the client. It is less
;; complex to implement in comparison to WebSockets. Furthermore the
;; low latency and reduced payload size of WebSockets is not required
;; for this use case.
(defonce client-listeners-state
;; holds the httpkit channels of the clients, which are waiting for
;; a db-view notify:
(atom {}))
(defn ring-handler
"Ring-handler for the '/db-view/notify' API endpoint."
[request]
(when (and (= (:request-method request) :post)
(= (:uri request) "/db-view/notify"))
;; NOTE: for a production app add an authentication check here:
(httpkit/with-channel request channel
;; register the user's browser session in the
;; `client-listeners-state`:
(swap! client-listeners-state
assoc
channel
channel)
(httpkit/on-close channel
(fn [status]
;; remove the user's browser session from
;; the `client-listeners-state` as soon as
;; the channel is closed, meaning the user
;; has closed the browser tab or the network
;; connection was interrupted:
(swap! client-listeners-state
dissoc
channel))))))
(defn notify
"A Datomic transaction listener that notifies all user browser
sessions, where the user was affected by the transaction of the
`tx-report`."
[tx-report]
(let [basis-t (d/basis-t (:db-after tx-report))
response (edn/response
{:db/basis-t basis-t})]
;; NOTE: for a production app only send notifications to the users
;; which are affected by this `tx-report`:
(doseq [channel (vals @client-listeners-state)]
(httpkit/send! channel
response))))
As already mentioned in the note above you would normally only notify the connected users (client-listeners) which are affected by the current Datomic transaction report. For example if your app supports teams and one user makes a change on the team's todo list, only the team mates which are currently online would be notified.
Datomic has this great feature that you can access the transaction log and even receive push updates of new transactions via the datomic.api/tx-report-queue. Most likely you will have multiple web servers, which response to db-view requests and executes commands (which transacts Datomic transactions). With the help of Datomic's transaction queue it does not matter to which server the client has opened the long-polling HTTP request, since all servers will be informed of the new transaction and initiate the necessary notification for an affected user.
The db-view approach also works for other databases than Datomic. But most of them do not provide access to a transaction queue. A do-it-yourself alternative to push the notifications is an effort that should not be underestimated. Therefore I would recommend services like Pusher or Pubnub, where you can push messages to the user's web browser, without building the required infrastructure yourself. The server would then send the notifications to the affected users right after it has transacted the corresponding Datomic transaction.
As already mentioned, the next blog post will cover the concept of commands in the context of a db-view application.