Build More Reliable Web Apps with Offline-First Principles
Customers are increasingly purchasing goods and services using smartphones, either through native or web applications. And, they want to make these purchases wherever they may be, whether it’s inside trains, in the countryside or roaming in foreign countries.
However, mobile networks are still known to be unreliable. Even in highly covered areas and with modern networks, you still encounter high latency and network failure rates. But it’s not just the network at fault here: the customer experience for failure rates also depends on how a given application deals with being offline.
In one extreme end of the quality spectrum, you have an application that, when a network error occurs, the application presents the user with a crude error message and no clue of how to recover.
On the opposite end of the spectrum are applications that automatically deal with network failures — this is the good end of the spectrum. Perhaps they create unique transaction identifiers that allow the back-end to detect duplicated transactions, allowing the application to automatically try reposting the transactions after a failure. Perhaps this application back-end supports a polling mechanism that allows the client application to inquire about the state of the transaction. Perhaps the application is responsive enough that it allows the customer to be informed of the intermediate states of the transaction, such as when each of the legs of the train trip has been reserved, when the payment has been authorized, when the reservations have all been confirmed, etc.
These top-of-breed resilient applications are probably also resilient to some client-side failures: The transaction continues correctly if the user closes the tab, the browser crashes or the device shuts down. It’s able to resume from where the customer left off, updating itself with the most recent transaction status.
Building an application on the better side of the spectrum is not only necessary to beat the competition, but it typically involves a huge investment on back-end and front-end development. Or, at least that’s what most people think.
But, this isn’t the way it has to be. Say hello to offline first.
What is Offline First?
Offline first is a way of building applications where having network connectivity is an enhancement, not a necessity. Instead of building applications with the always-on desktop mindset, when you build applications in which the default mode is offline, you’re prone to deliver a better overall customer experience.
Offline-first techniques and technologies exist to prepare an application to deliver a good experience to customers while it’s offline. For instance, the recent addition of Service Workers into most browsers allows a web application to intercept HTTP requests and either populate a cache or use it while it’s offline. But when we’re dealing with e-commerce transactions, what techniques and technologies can help us?
Client has Local Data
The only way that an application can access data while it’s offline is, of course, to store that data locally. Some database technologies exist that work on the client side. SQLite is the reference of embeddable databases and it’s often used on native applications. In the web realm, PouchDB offers a document store on top of the storage the browser offers. PouchDB also has some nice features — it can sync with a back-end CouchDB server, Cloudant, a PouchDB Node.js server or any other database that implements the CouchDB replication protocol.
In this architecture, each client has its own dedicated database, which is then replicated to a dedicated database on the back-end. Each database may then contain the customer documents. (A document is a JSON object that may contain any arbitrary data).
One database per customer may sound strange to people who are used to relational databases, but it’s a usual pattern when using CouchDB and variants. It’s also a way to clearly and naturally separate and enforce which data a user has access to.
The Sync Protocol
When a change happens on the client or on the server, the sync protocol kicks in and tries to replicate that change to the other side. If no network connection is possible at that time, the client will try to reconnect. Once a connection is possible, both databases will be able to talk to each other and synchronize.
In this architecture, both the client and the server can make changes to the data concurrently. If a conflict arises for a given document, the replication protocol makes sure that both databases converge to the same version of that document. When a conflict happens, Pouch and CouchDB keep all the conflict data around. If the programmer so wishes, they can solve that conflict with any strategy they deem correct and that minimizes data loss.
Back-end Integration with Node.js
On top of being able to sync, PouchDB (and CouchDB) can keep the entire revision history for any document, and provide an asynchronous changes feed that can stream all the versions of all the documents. This transforms PouchDB (or CouchDB) into an event queue to be processed by a clerk process. This clerk can then process document changes and interact with back-end services at will.
In turn, this clerk worker can perform changes in any given document, which will then get propagated into the client application by the sync protocol.
The clerk can be implemented in any language runtime, however, Node.js has an open-source pouch-clerk library, so you don’t have to do your own building here and can simply use the library and easily roll out on your own.
In this architecture, each transaction can then be described by one document on the customer database. This document should contain all the data that is necessary to perform the next step of the transaction.
If you’re purchasing goods, it could contain all the products with respective quantities and the payment method identifier. If you’re purchasing a plane trip, it should contain all the flights, passenger data and payment methods. And so on.
This document contains one important attribute: the state. Since any transaction has at least two states (started and finished), the clerk can react to transitions and you can model these transitions to your particular flow. For instance, if you’re building a taxi application, you can model the states (requested, searching driver, driver assigned, driver en route, driver arrived, picked up, in transit, etc). You could easily model long-running transactions this way, where the transaction document slowly transitions state.
The Client UI
Building a client UI for this architecture should be relatively easy: it must sync the UI state with the local transaction state. (If you’re using React and Redux you can use the pouch-redux-middleware package, which keeps in sync the Redux state with a local PouchDB database).
Now, you can simply build the UI using a reactive technology like React with no concern for handling networking errors: the architecture handles it for you.
Building the data layer of an application using offline-first principles requires you to re-architecture the front-end database. However, by exploiting some of the PouchDB or CouchDB database capabilities and using Node.js, it allows you to plug into existing back-end services to finally deliver an improved transaction experience.