Building low-boilerplate backend services in Node.js

As a backend developer, I have repeatedly found myself in the position of having to build backend services that share a lot (or even all) of plumbing with existing services. In this article I talk about my approach of using "stack-as-a-library" packages to avoid having to repeatedly write boilerplate code, to reduce the time taken to spin up new services and to improve maintainability of a set of services.

For the purpose of the following narrative let's consider a scenario where several microservices provide CRUD API access to separate collections in a MongoDB database. The exact technology stack is not important here, only that we are creating a set of similar services that differ only in the business logic that they contain.

Typically such an application will contain 3 types of code:

• Configuration boilerplate (e.g Express server, middleware and Mongo client)
• Operational code (e.g logging, metrics, monitoring APIs etc.)
• Business logic (including tests)

We can visualise this as the following diagram:

From looking at this diagram it is immediately obvious that there is duplication. The only thing that differs between the different services is the business logic. Duplicating the boilerplate means that any changes (e.g. improvements, extra tests) would need to be copied to between the service codebases. It also adds clutter to the codebases - a slight but constant irritation to the developers working with it.

A solution to this seems quite obvious: isolate the configuration into a separate package and reuse that across similar services:

Despite seeming obvious, this is not how services are typically built. Instead, every time the boilerplate and tests are re-written, likely in a somewhat different way. We use frameworks but only as a way of structuring the code. At best, the framework may include some built-in features but these are invariably quite generic and requiring further assembly and customisation.

As an alternative example, consider the following snippet of code, which creates and Express API to return some data from a Mongo collection:

import { load } from "@101-ways/core-mongo-express";

load().then((sr) => {
  const collection = sr.mongo.db().collection("widgets");

  sr.express.app.get("/api/widgets", async (req, res) => {
    const results = await collection.find({}).toArray();
    res.json({ results });
  });
});

The sole interesting property of this toy example is that it is almost entirely business logic. And yet, it also includes out of the box:

• Structured logging (JSON with ECS schema)
• Logging for Express requests and responses, including tracing data and response time
• Debug logging for Mongo queries, including tracing data and response time

A more complete example can be seen here. Again, the most interesting thing about it is that it is almost entirely business logic and tests.

The exact set of features is not important and will differ between organisations and teams. What is important is that it is standardised and encapsulated for reusability, providing a single place where any improvements can be made. The developer is freed to write the code that actually matters to the business. Routine maintenance and updates also become easier with the single package containing the stack code.

For an example of how such a package can be put together see this repository. It is a proof-of-concept that I built for the consultancy company I work at, intentionally making it quite granular so as to demonstrate the possibility of mixing and matching technologies for different customer requirements. When I implemented a similar pattern in my previous teams it was sufficient to have a single package with all the stack pieces bundled together as there was no need for variation.

I hope this can serve as inspiration to build your own base packages to encapsulate your set of technologies and operational experience and help you do more with less effort and drudgery. If you have any questions just reach out and will be happy to help out.