Sentinel Stand
TwitterRSS

Using Docker with NodeJS in development and production

April 30, 2020 (updated July 18, 2020)

In this article we'll create a production Docker image for a Node/Express app. We'll also add Docker to the development process using Docker Compose so we can easily spin up our services, including the Node app itself, on our local machine in an isolated and reproducable manner.

The app will be written using newer JavaScript syntax to demonstrate how Babel can be included in the build process. Your current Node version may not support certain modern JavaScript features, like ECMAScript modules (import and export), so Babel will be used to convert the code into a backwards compatible version.

Don't want to use Babel? That's fine. You can skip the Babel steps and jump straight to the Docker sections.

Goals

  • Add Babel to the production build
  • Make the Docker production image as small as possible
  • Use Nodemon and babel-node in development to recompile the code automatically whenever a file is changed
  • Only create one Dockerfile (using multi-stage builds)
  • Use volumes to separate local node modules from the container's node modules
  • (Deployment won't be covered in this article)

(This article has an companion Git repository github.com/nicoqh/node-docker-boilerplate)

Set up a Node project

Before we start using Docker, let's quickly set up a simple Express-powered Node app.

We'll need two directories: one for the raw source code (src) and one for the Babel-compiled code (dist).

mkdir src
mkdir dist

As with most Node-based projects, we need a package.json that lists our dependencies. Simply add an empty object for now:

{}

We also need a .gitignore file which instructs Git to ignore the node_modules and the dist directories. Everything inside dist will be compiled from the source files, so we don't want to add it to version control.

node_modules
dist

Then, install Express:

npm install express

Finally, create the entry point for the app, src/server.js:

// src/server.js
import express from "express";

const app = express();

app.use('/', (req, res) => {
  res.send('Hello, world!')
});

// Instead of hardcoding the port number, we can pass
// it at runtime by setting an environment variable.
const port = process.env.PORT || 3000;
app.listen(port, () => {
  console.log(`Express is listening on port ${port}`);
})

export default app;

This app won't be able to run if your version of Node doesn't support ECMAScript modules (import and export). Even if it does, you may want to use other JavaScript features that aren't currently supported. This brings us to Babel.

Compiling with Babel for production

According to its website, "Babel is a toolchain that is mainly used to convert ECMAScript 2015+ code into a backwards compatible version of JavaScript in current and older browsers or environments [like Node]."

In other words, we need Babel to transform our code into something Node understands. Newer syntax which isn't supported by our current Node version will be transformed to backwards compatible JavaScript. This includes support for ECMAScript Modules (import, export).

To use Babel we need the compiler core and the Babel command line:

npm install --save-dev @babel/core @babel/cli

With Babel installed, we can run it on our source code:

./node_modules/.bin/babel src --out-dir dist

The compiled code is emitted to the dist directory. But without providing any configuration options, Babel simply parses our code and leaves it untouched.

For Babel to do anything useful we need to enable some plugins. Plugins are responsible for transforming the code and understanding the syntax. Each plugin provides support for a specific language feature, like arrow functions, object rest/spread and ECMAScript modules. Instead of installing a bunch of individual plugins—one for each language transformation—Babel offers collections of plugins which are called "presets".

One particularly useful preset is called @babel/preset-env. This is a "smart" preset that allows us to use new JavaScript syntax without needing to manage which specific syntax transformations are needed by your target environment. (A "target environment" is an environment on which we want our code to run, e.g. Node 12.) If we specify a target, the preset will generate a list of required plugins (features missing from our target environment) and feed it to Babel.

Let's install this preset:

npm install --save-dev @babel/preset-env

Then create a .babelrc.json config file and tell Babel to use the preset:

{
  "presets": [
    ["@babel/preset-env", {
      "debug": true,
      "targets": {
        "node": "current"
      }
    }]
  ]
}

The preset is configured to target the current version of Node (the version you have installed). We've also added the debug option which will instruct Babel to output the targets and plugins it uses during compilation.

Let's run Babel again:

./node_modules/.bin/babel src --out-dir dist

The output will list the target (the current version of Node, e.g. "node": "12.16.2") and the plugins it decided to use, and the compiled output will be stored in dist.

To run the compiled code:

node dist/server.js

Next, let's add some npm scripts to package.json so we don't have to type out the commands in full:

{
  "scripts": {
    "build": "npm run clean && babel ./src --out-dir dist",
    "start": "node ./dist/server.js",
    "clean": "rm -rf ./dist && mkdir dist"
  },
  // ... dependencies removed for brevity
}

These three npm scripts allow us to create a production build and start the Node server:

  • npm run clean: Delete the dist directory to remove old code before a new build
  • npm run build: Run the clean command, then run Babel on the source code and output the result to dist
  • npm run start: Start the app, i.e. run the compiled code

Using Babel in development

During development we want Babel to automatically compile the source code whenever a file is modified. For this we need two packages: Nodemon and babel-node.

Nodemon is a utility that monitors your files and automatically restarts the Node server whenever a file is changed. The usage is simple: Instead of running node ./dist/server.js, we run nodemon ./dist/server.js. Nodemon will run node and restart the process whenever a file change is detected.

babel-node is a command line interface that works exactly like node, except it compiles the code with Babel before running it. This means we can run babel-node directly on our source files without an explicit compilation step (should not be used in production!).

Nodemon can execute and monitor other programs than node using the --exec option. In our case we want Nodemon to monitor babel-node instead of node.

Let's install both packages:

npm install --save-dev @babel/node nodemon

And add a new npm script:

{
  "scripts": {
    // ... production scripts omitted for brevity
    "dev": "nodemon --exec babel-node ./src/server.js"
  }
}

Running npm run dev will start Nodemon, which executes babel-node. babel-node compiles our source code on the fly and runs it, and Nodemon monitors the source files to restart babel-node if anything changes.

Now that our npm scripts are ready and Babel has been configured, let's proceed to Docker.

Docker in production using multi-stage builds

A Docker container is an isolated environment for your app. The container packages up all the necessary code, dependencies and system tools into one unit of software so that your application can run independently and reliably on any environment.

Although they function differently, containers can be thought of as a light-weight virtual machines.

To create a Docker container we need a Docker image (a Docker container is essentially a running instance of an image). The "recipe" for a Docker image is stored in a file called the Dockerfile.

A Dockerfile contains a set of instructions, i.e. commands that are needed to assemble a Docker image. A valid Dockerfile must start with the FROM instruction. This sets the base image for subsequent instructions. Valid base images include ubuntu, node or node:12-alpine.

It's not uncommon to create more than one Dockerfile. You can create a Dockerfile for development, one for testing and one for production. The reason for this is that each image has different requirements: the testing image would need testing tools and linters (development dependencies), while the production image should only contain the bare minimum production dependencies and no superfluous artifacts.

However, Docker supports "multi-stage" builds which lets us achieve the same goals using only one Dockerfile.

By using multiple FROM instructions in the same Dockerfile, we can create multiple intermediary images which use artifacts (files, compiled code etc.) from each other. Each FROM instruction initializes a new "build stage" which can be referenced by other build stages. The last FROM instruction denotes the final Docker image.

For example, we can define a "builder" stage which compiles our source code with Babel, and a final "release" stage which copies only the compiled code from the builder stage. The release stage would thus be free from any development tools like Babel. It wouldn't even contain the source code, only the compiled build. This ensures a smaller final image. It's also a good security measure, as potential security vulnerabilities in the development tools won't be included in the final release.

With this in mind, let's create a Dockerfile (each instruction will be explained below):

# ---------- Base ----------
FROM node:12-alpine AS base

WORKDIR /app

# ---------- Builder ----------
# Creates:
# - node_modules: production dependencies (no dev dependencies)
# - dist: A production build compiled with Babel
FROM base AS builder

COPY package*.json .babelrc.json ./

RUN npm install

COPY ./src ./src

RUN npm run build

RUN npm prune --production # Remove dev dependencies

# ---------- Release ----------
FROM base AS release

COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist

USER node

CMD ["node", "./dist/server.js"]

Let's take a look at each build stage.

# ---------- Base ----------
FROM node:12-alpine AS base
WORKDIR /app

FROM initializes the first "build stage" by setting a base image for subsequent instructions. This image is based on the official Docker image for Node, specifically the Alpine version (a tiny Linux distribution). We give this build stage the name "base" (denoted by AS base). The image built in this stage can later be referred to by its name.

(Side note: You may want to use a more specific version of the base Node image to ensure that each build is as similar as possible.)

WORKDIR sets the working directory for subsequent instructions. This directory is created by Docker if it doesn't already exist.

The next stage is the "builder" stage.

# ---------- Builder ----------
FROM base AS builder
COPY package*.json .babelrc.json ./
RUN npm install
COPY ./src ./src
RUN npm run build
RUN npm prune --production # Remove dev dependencies

FROM base AS builder initializes a new build stage based on the previous "base" stage, and names it "builder".

The COPY instruction copies package.json, package-lock.json and .babelrc.json into the working directory. This lets us install our dependencies with RUN npm install.

After the dependencies have been installed, we COPY the entire source directory and compile it with npm run build. The build is emitted to the dist directory, as specified by our npm script.

Lastly we remove the development dependencies (packages listed under devDependencies in package.json) from node_modules. The node_modules directory now contains only production dependencies, which is what we need for our next stage: the final release image.

# ---------- Release ----------
FROM base AS release
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
USER node
CMD ["node", "./dist/server.js"]

The final stage produces our final release image. Note that this stage is based on the first "base" stage (not the "builder" stage!). This means that the artifacts from the "builder" stage are not included. This is good because we want to pick and choose what to include from the builder stage. Specifically, we want two things: the compiled app (dist) and the production dependencies (node_modules).

We copy these artifacts using COPY --from=builder. The --from flag is used to set the source location to a previous build stage that will be used instead of a build context set by the user.

By default Docker runs the container as the root user. For security purposes we change this to the less privileged node user, which is included in the official Node Docker image.

Lastly, the CMD instruction provides a default executable for the container: node ./dist/server.js.

The image is ready to be built using the docker build command:

docker build -t node-docker-boilerplate .

This command produces a Docker image from the Dockerfile which can be run in a container. The -t flag sets a name and optionally a "tag" for the image. (You may want to tag your image with a version number, but how to properly name and tag your Docker images is outside the scope of this article.)

The dot (.) is the path to the build context, which is where Docker looks for files, like the Dockerfile (more on this below).

When the image is built we can create and run a container:

docker run -p 3000:3000 --env PORT=3000 --rm node-docker-boilerplate

This command creates a container and runs it, binding the port 3000 of the container to port 3000 of the host machine (your machine). The --env flag sets an environment variable that is used by our Node app. The --rm flag tells Docker to remove the container when it exits. Lastly, the image's name is specified.

Docker context and .dockerignore

When building an image with docker build, Docker needs to know where it can find the files that are referenced during the build process (e.g. in the Dockerfile). The files that Docker has access to make up the build context. For example, in order for COPY source.txt destination.txt to work, the file source.txt must be present in Docker's build context.

The build context is passed as an argument to docker build. Often the build context is the local working directory, denoted by a dot (.), like this: docker build -t my-image .

The entire build context is sent to Docker (the Docker server) when a build is initiated. However, Docker doesn't need every file in our local working directory. If we exclude the files that aren't needed, we can speed up the build process and potentially reduce the size of the final image. To exclude files and directories from the build context, add them to a file named .dockerignore (view commit):

.git
.gitignore
.dockerignore
Dockerfile
dist
node_modules
npm-debug.log
README.md

Our build step doesn't reference any of these files, so they can safely be ignored by Docker.

Docker in development using Docker Compose

Docker Compose is a tool used to define and run multi-container Docker applications. We define all the services we need in a YAML configuration file (like an Express service and database service), and a simple command creates and starts the services according to this configuration. Docker Compose is great for creating local development environments.

Let's define some requirements for our setup:

  • Developers should not be required to have Node installed on their development machine.
  • The development environment should be similar to the production environment
  • We don't need a build step. Our npm script npm run dev uses Nodemon to detect file changes and babel-node to compile the app on the fly.
  • Since the source files are stored on a local computer, we don't want to copy them into a container every time they change.

To get started, create the Compose file docker-compose.yml:

version: "3.4"
services:
  express:
    image: node:12-alpine
    volumes:
      - type: bind
        source: ./
        target: /app
    working_dir: /app
    command: npm run dev
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=development
      - PORT=3000

This configuration defines a service called "express" which uses the same Node Alpine base image used by our production image. The version option at the top specifies which version of the Compose file format we're using.

Using the volumes option, the current working directory ./ on your local machine is mounted into the container at /app:

    volumes:
      - type: bind
        source: ./
        target: /app

This is called a "bind mount". Bind mounts let us mount a file or directory on the host (local) machine into a container. Any changes you make to the files on the host machine is mirrored in the container, and vice versa.

The default command to run when we start the container is npm run dev:

    command: npm run dev

This is the npm script we created earlier that runs babel-node to both compile and run the code.

Before we can spin up our services we need to install the app's dependencies. If we install the dependencies by running npm install on our local machine, the node_modules directory will be mirrored in the container, which is a consequence of the bind mount we created above. But this approach is problematic. Some libraries need to compile during the installation process, and the resulting artifacts―if compiled on your local machine―won't necessarily run properly inside the Docker container. The container environment is different from your local machine. They probably don't even run the same operating system.

On the other hand, if we run npm install inside the container, the resulting node_modules directory won't necessarily run properly on our local machine (in case you want to run your Node app locally without Docker).

In other words, the node_modules directory on your host machine should be separate from the node_modules directory inside the Docker container.

To achieve this we can use a Docker concept called volumes. While volumes are similar to bind mounts in some respects, they work differently and are often used for different purposes. A bind mount is a file or directory that is mounted into the container from the host machine. Since these files and directories are managed by the host machine, processes running on the host machine (i.e. outside the container) can access and modify them. For example, when mounting the current directory ./ into the container like we did above, you can easily modify the source files using your favorite text editor, and the changes will be reflected inside the container. Bind mounts are, by their nature, dependent on the host and thus not very portable.

Volumes, on the other hand, are fully managed by Docker. When creating volumes, you don't reference the host's filesystem at all. Volumes can be created and managed independently from, and without starting, a container. By using different volume drivers they can even be stored on remote hosts or in the cloud. Volumes are also easy to share between containers. And since Docker manages the volume's content, they can't easily be accessed from the host machine.

If we create a volume for node_modules, it can live isolated from the host/local machine and be mounted into the container when the container starts. Whatever the container writes to the volume is persisted for later use.

To create the volume we need to define it by name in a top-level volumes key in docker-compose.yml. We also need to add it to the definition of the express service. The updated docker-compose.yml looks like this:

version: "3.4"
services:
  express:
    image: node:12-alpine
    volumes:
      - type: bind
        source: ./
        target: /app
      - type: volume
        source: nodemodules # name of the volume, see below
        target: /app/node_modules
        volume:
          nocopy: true
    working_dir: /app
    command: npm run dev
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=development
      - PORT=3000
volumes:
    nodemodules:

We've kept the previously defined bind mount and added a volume called nodemodules which is mounted inside the container at /app/node_modules. This volume will store the node modules used by the container. Its content is not mirrored on the host machine, as is the case with bind mounts (though, technically, the contents is stored somewhere on the host machine).

The nocopy: true option disables copying of data from the container when the volume is created; if the target directory (/app/node_modules) already exists inside the container, this directory's content is copied into the volume and shared with any other service that uses the volume. (However, this only applies if the volume is created by starting the container, as above; if the volume is created beforehand, e.g. with docker volume create, nothing is copied.) Our container doesn't already have the directory /app/node_modules (we're using a clean base Node image, as specified by the image option), so this option isn't strictly necessary.

Finally we're ready to install the dependencies.

Docker Compose has a subcommand called run which is used to run one-time commands in a service. We can use this to run npm install inside the "express" service.

docker-compose run --rm express npm install

This command will start the "express" service, including its volumes, and run npm install against it. The dependencies will be installed into the container's /app/node_modules, which is the target directory of the volume we created and named "nodemodules".

When the installation is complete, spin up all the services (there's currently only one):

docker-compose up

The docker-compose command will read docker-compose.yml and start the "express" service.

To access the app, open http://localhost:3000 with your browser. When you're done, stop the services with CTRL/CMD + C.

You can also run the services in detached mode (in the background) with the -d option. If so, start and stop the services with the following commands:

docker-compose up -d
docker-compose down

If you delete the node_modules directory on your local machine, you'll see that the containerized Node app still runs perfectly because it has access to its own node modules!

(Side note: To install Node dependencies in a container that is already running you can use the exec subcommand instead of run. exec allows you to run commands in a running container: docker-compose exec express npm install.)

Alternative setups and adding tests

There are multiple ways to use Docker, both in production and during development. Our production build contains three stages: a base stage, a builder stage which compiles the app, and a final release stage which copies the build files from the builder stage.

If you wanted to, say, add another step that runs your tests, you could split up the stages in a slightly different way. An intermediary stage could install the dependencies and create two different node_modules directories: one for production and one for development. Then, both the test and release stages could copy the artifacts they needed from the "dependencies" stage.

Here's an example Dockerfile:

# ---------- Base ----------
FROM node:12-alpine AS base

WORKDIR /app

# ---------- Builder ----------
FROM base AS builder

COPY package*.json .babelrc.json ./

# Install the production dependencies
RUN npm install --only=production

# Copy the production dependencies to a new directory
RUN cp -R node_modules node_modules_production

# Install all dependencies, both production and development
RUN npm install

# Copy the source files
COPY ./src ./src

# Build the app
RUN npm run build

# ---------- Tests ----------
FROM builder AS tests

RUN npm run test

# ---------- Release ----------
FROM base AS release

# Copy the production dependencies
COPY --from=builder /app/node_modules_production ./node_modules

# Copy the compiled app
COPY --from=builder /app/dist ./dist

USER node

CMD ["node", "./dist/server.js"]

Closing remarks

The Docker ecosystem is vast and there's a lot concepts to wrap your head around. But if you decide to put in the effort, Docker can definitely bring immense benefits and make your life easier.

The the official docs is a good place to start if you want to learn more. You can also check out some "best practices" from the official Node Docker repo.

And if you've found any errors in this article, or have suggestions for improvements, please leave a comment or reach out on Twitter!

Back to homepage•Twitter