The Docker Handbook
Search…
Containerizing a Multi-Container JavaScript Application
Now that you've learned enough about networks in Docker, in this sub-section you'll learn to containerize a full-fledged multi-container project. The project you'll be working with is a simple
notes-api powered by Express.js and PostgreSQL.In this project there are two containers in total that you'll have to connect using a network. Apart from this, you'll also learn about concepts like environment variables and named volumes. So without further ado, lets jump right in.
Setting Up The Custom Bridge Network
As you've learned in the previous section, the containers have to be attached to a user-defined bridge network in order to communicate with each other using container names. To do so, create a network named notes-api-network in your system:
1
docker network create notes-api-network
Copied!
Running the Database Server
The database server in this project is a simple PostgreSQL server and uses the official postgres image. According to the official docs, in order to run a container with this image, you must provide the
POSTGRES_PASSWORD environment variable. Apart from this one, I'll also provide a name for the default database using the POSTGRES_DB environment variable. PostgreSQL by default listens on 5432 port, so you need to publish that as well.To run the database server you can execute the following command:
1
docker container run \
2
--detach \
3
--name=notes-db \
4
--env POSTGRES_DB=notesdb \
5
--env POSTGRES_PASSWORD=secret \
6
--network=notes-api-network \
7
postgres:12
8
9
# a7b287d34d96c8e81a63949c57b83d7c1d71b5660c87f5172f074bd1606196dc
10
11
docker container ls
12
13
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
14
# a7b287d34d96 postgres:12 "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp notes-db
Copied!
The
--env option for the container run and container create commands can be used for providing environment variables to a container. As you can see, the database container has been created successfully and is running now.Although the container is running, there is a small problem. Databases like PostgreSQL, MongoDB, MySQL persist their data in a directory. PostgreSQL uses the
/var/lib/postgresql/data directory inside the container to persist data. Now what if the container gets destroyed for some reason? You'll lose all your data. To solve this problem, a named volume can be used.Working With Named Volumes
Previously you've worked with bind mounts and anonymous volumes. A named volume is very similar to an anonymous volume except the fact that you can refer to a named volume using its name. Volumes are also logical objects in Docker and can be manipulated using the CLI. The
volume create command can be used for creating a named volume.The generic syntax for the command is as follows:
1
docker volume create <volume name>
Copied!
To create a volume named
notes-db-data you can execute the following command:1
docker volume create notes-db-data
2
3
# notes-db-data
4
5
docker volume ls
6
7
# DRIVER VOLUME NAME
8
# local notes-db-data
Copied!
This volume can now be mounted to
/var/lib/postgresql/data inside the notes-db container. To do so, stop and remove the notes-db container:1
docker container stop notes-db
2
3
# notes-db
4
5
docker container rm notes-db
6
7
# notes-db
Copied!
Now run a new container and assign the volume using the
--volume or -v option.1
docker container run \
2
--detach \
3
--volume notes-db-data:/var/lib/postgresql/data \
4
--name=notes-db \
5
--env POSTGRES_DB=notesdb \
6
--env POSTGRES_PASSWORD=secret \
7
--network=notes-api-network \
8
postgres:12
9
10
# 37755e86d62794ed3e67c19d0cd1eba431e26ab56099b92a3456908c1d346791
Copied!
Now inspect the
notes-db container to make sure that the mounting was successful:1
docker container inspect --format='{{range .Mounts}} {{ .Name }} {{end}}' notes-db
2
3
# notes-db-data
Copied!
Now the data will be safely stored inside the
notes-db-data volume and can be reused in the future. A bind mount can also be used instead of a named volume here, but I prefer a named volume in such scenarios.Accessing Logs From a Container
In order to see the logs from a container, you can use the
container logs command. The generic syntax for the command is as follows:1
docker container logs <container identifier>
Copied!
To access the logs from the
notes-db container, you can execute the following command:1
docker container logs notes-db
2
3
# The files belonging to this database system will be owned by user "postgres".
4
# This user must also own the server process.
5
6
# The database cluster will be initialized with locale "en_US.utf8".
7
# The default database encoding has accordingly been set to "UTF8".
8
# The default text search configuration will be set to "english".
9
#
10
# Data page checksums are disabled.
11
#
12
# fixing permissions on existing directory /var/lib/postgresql/data ... ok
13
# creating subdirectories ... ok
14
# selecting dynamic shared memory implementation ... posix
15
# selecting default max_connections ... 100
16
# selecting default shared_buffers ... 128MB
17
# selecting default time zone ... Etc/UTC
18
# creating configuration files ... ok
19
# running bootstrap script ... ok
20
# performing post-bootstrap initialization ... ok
21
# syncing data to disk ... ok
22
#
23
#
24
# Success. You can now start the database server using:
25
#
26
# pg_ctl -D /var/lib/postgresql/data -l logfile start
27
#
28
# initdb: warning: enabling "trust" authentication for local connections
29
# You can change this by editing pg_hba.conf or using the option -A, or
30
# --auth-local and --auth-host, the next time you run initdb.
31
# waiting for server to start....2021-01-25 13:39:21.613 UTC [47] LOG: starting PostgreSQL 12.5 (Debian 12.5-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
32
# 2021-01-25 13:39:21.621 UTC [47] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
33
# 2021-01-25 13:39:21.675 UTC [48] LOG: database system was shut down at 2021-01-25 13:39:21 UTC
34
# 2021-01-25 13:39:21.685 UTC [47] LOG: database system is ready to accept connections
35
# done
36
# server started
37
# CREATE DATABASE
38
#
39
#
40
# /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
41
#
42
# 2021-01-25 13:39:22.008 UTC [47] LOG: received fast shutdown request
43
# waiting for server to shut down....2021-01-25 13:39:22.015 UTC [47] LOG: aborting any active transactions
44
# 2021-01-25 13:39:22.017 UTC [47] LOG: background worker "logical replication launcher" (PID 54) exited with exit code 1
45
# 2021-01-25 13:39:22.017 UTC [49] LOG: shutting down
46
# 2021-01-25 13:39:22.056 UTC [47] LOG: database system is shut down
47
# done
48
# server stopped
49
#
50
# PostgreSQL init process complete; ready for start up.
51
#
52
# 2021-01-25 13:39:22.135 UTC [1] LOG: starting PostgreSQL 12.5 (Debian 12.5-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
53
# 2021-01-25 13:39:22.136 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
54
# 2021-01-25 13:39:22.136 UTC [1] LOG: listening on IPv6 address "::", port 5432
55
# 2021-01-25 13:39:22.147 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
56
# 2021-01-25 13:39:22.177 UTC [75] LOG: database system was shut down at 2021-01-25 13:39:22 UTC
57
# 2021-01-25 13:39:22.190 UTC [1] LOG: database system is ready to accept connections
Copied!
Evident by the text in line 57, the database is up and ready for accepting connections from the outside. There is also the
--follow or -f option for the command which lets you attach the console to the logs output and get a continuous stream of text.Attaching The Database Server (in case you missed it earlier)
Your container should be connected to the network "notes-api-network" now. Recall that you can use
docker network inspect --format='{{range .Containers}}{{.Name}}{{end}}' notes-api-network to verify, and it should look like this:1
docker network inspect --format='{{range .Containers}}{{.Name}}{{end}}' notes-api-network
2
3
# notes-db
Copied!
If that command instead returns a blank line, you can attach the
notes-db container to this network by executing the following command:1
docker network connect notes-api-network notes-db
Copied!
Writing The Dockerfile
Go to the directory where you've cloned the project codes. Inside there, go inside the
notes-api/api directory, and create a new Dockerfile in there. Put following code in the file:1
# stage one
2
FROM node:lts-alpine as builder
3
4
# install dependencies for node-gyp
5
RUN apk add --no-cache python make g++
6
7
WORKDIR /app
8
9
COPY ./package.json .
10
RUN npm install --only=prod
11
12
# stage two
13
FROM node:lts-alpine
14
15
EXPOSE 3000
16
ENV NODE_ENV=production
17
18
USER node
19
RUN mkdir -p /home/node/app
20
WORKDIR /home/node/app
21
22
COPY . .
23
COPY --from=builder /app/node_modules /home/node/app/node_modules
24
25
CMD [ "node", "bin/www" ]
Copied!
This is a multi-staged build. The first stage is used for building and installing the dependencies using
node-gyp and the second stage is for running the application. I'll go through the steps briefly:- Stage 1 uses
node:lts-alpineas its base and usesbuilderas the stage name. - On line 5, we install
python,makeandg++. Thenode-gyptool requires these three packages to run. - On line 7, we set
/appdirectory as theWORKDIR. - On line 9 and 10, we copy the
package.jsonfile to theWORKDIRand installs all the dependencies. - Stage 2 also uses
node-lts:alpineas the base. - On line 16, we set the
NODE_ENVenvironment variable toproduction. This is important for the API to run properly. - From line 18 to line 20, we set the default user to
node, create the/home/node/appdirectory and set that as theWORKDIR. - On line 22, we copy all the project files and on line 23 we copy the
node_modulesdirectory from thebuilderstage. This directory contains all the built dependencies necessary for running the application. - On line 25, we set the default command.
To build an image from this
Dockerfile, you can execute the following command:1
docker image build --tag notes-api .
2
3
# Sending build context to Docker daemon 37.38kB
4
# Step 1/14 : FROM node:lts-alpine as builder
5
# ---> 471e8b4eb0b2
6
# Step 2/14 : RUN apk add --no-cache python make g++
7
# ---> Running in 5f20a0ecc04b
8
# fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
9
# fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
10
# (1/21) Installing binutils (2.33.1-r0)
11
# (2/21) Installing gmp (6.1.2-r1)
12
# (3/21) Installing isl (0.18-r0)
13
# (4/21) Installing libgomp (9.3.0-r0)
14
# (5/21) Installing libatomic (9.3.0-r0)
15
# (6/21) Installing mpfr4 (4.0.2-r1)
16
# (7/21) Installing mpc1 (1.1.0-r1)
17
# (8/21) Installing gcc (9.3.0-r0)
18
# (9/21) Installing musl-dev (1.1.24-r3)
19
# (10/21) Installing libc-dev (0.7.2-r0)
20
# (11/21) Installing g++ (9.3.0-r0)
21
# (12/21) Installing make (4.2.1-r2)
22
# (13/21) Installing libbz2 (1.0.8-r1)
23
# (14/21) Installing expat (2.2.9-r1)
24
# (15/21) Installing libffi (3.2.1-r6)
25
# (16/21) Installing gdbm (1.13-r1)
26
# (17/21) Installing ncurses-terminfo-base (6.1_p20200118-r4)
27
# (18/21) Installing ncurses-libs (6.1_p20200118-r4)
28
# (19/21) Installing readline (8.0.1-r0)
29
# (20/21) Installing sqlite-libs (3.30.1-r2)
30
# (21/21) Installing python2 (2.7.18-r0)
31
# Executing busybox-1.31.1-r9.trigger
32
# OK: 212 MiB in 37 packages
33
# Removing intermediate container 5f20a0ecc04b
34
# ---> 637ca797d709
35
# Step 3/14 : WORKDIR /app
36
# ---> Running in 846361b57599
37
# Removing intermediate container 846361b57599
38
# ---> 3d58a482896e
39
# Step 4/14 : COPY ./package.json .
40
# ---> 11b387794039
41
# Step 5/14 : RUN npm install --only=prod
42
# ---> Running in 2e27e33f935d
43
# added 269 packages from 220 contributors and audited 1137 packages in 140.322s
44
#
45
# 4 packages are looking for funding
46
# run `npm fund` for details
47
#
48
# found 0 vulnerabilities
49
#
50
# Removing intermediate container 2e27e33f935d
51
# ---> eb7cb2cb0b20
52
# Step 6/14 : FROM node:lts-alpine
53
# ---> 471e8b4eb0b2
54
# Step 7/14 : EXPOSE 3000
55
# ---> Running in 4ea24f871747
56
# Removing intermediate container 4ea24f871747
57
# ---> 1f0206f2f050
58
# Step 8/14 : ENV NODE_ENV=production
59
# ---> Running in 5d40d6ac3b7e
60
# Removing intermediate container 5d40d6ac3b7e
61
# ---> 31f62da17929
62
# Step 9/14 : USER node
63
# ---> Running in 0963e1fb19a0
64
# Removing intermediate container 0963e1fb19a0
65
# ---> 0f4045152b1c
66
# Step 10/14 : RUN mkdir -p /home/node/app
67
# ---> Running in 0ac591b3adbd
68
# Removing intermediate container 0ac591b3adbd
69
# ---> 5908373dfc75
70
# Step 11/14 : WORKDIR /home/node/app
71
# ---> Running in 55253b62ff57
72
# Removing intermediate container 55253b62ff57
73
# ---> 2883cdb7c77a
74
# Step 12/14 : COPY . .
75
# ---> 8e60893a7142
76
# Step 13/14 : COPY --from=builder /app/node_modules /home/node/app/node_modules
77
# ---> 27a85faa4342
78
# Step 14/14 : CMD [ "node", "bin/www" ]
79
# ---> Running in 349c8ca6dd3e
80
# Removing intermediate container 349c8ca6dd3e
81
# ---> 9ea100571585
82
# Successfully built 9ea100571585
83
# Successfully tagged notes-api:latest
Copied!
Before you run a container using this image, make sure the database container is running, and is attached to the
notes-api-network.1
docker container inspect notes-db
2
3
# [
4
# {
5
# ...
6
# "State": {
7
# "Status": "running",
8
# "Running": true,
9
# "Paused": false,
10
# "Restarting": false,
11
# "OOMKilled": false,
12
# "Dead": false,
13
# "Pid": 11521,
14
# "ExitCode": 0,
15
# "Error": "",
16
# "StartedAt": "2021-01-26T06:55:44.928510218Z",
17
# "FinishedAt": "2021-01-25T14:19:31.316854657Z"
18
# },
19
# ...
20
# "Mounts": [
21
# {
22
# "Type": "volume",
23
# "Name": "notes-db-data",
24
# "Source": "/var/lib/docker/volumes/notes-db-data/_data",
25
# "Destination": "/var/lib/postgresql/data",
26
# "Driver": "local",
27
# "Mode": "z",
28
# "RW": true,
29
# "Propagation": ""
30
# }
31
# ],
32
# ...
33
# "NetworkSettings": {
34
# ...
35
# "Networks": {
36
# "bridge": {
37
# "IPAMConfig": null,
38
# "Links": null,
39
# "Aliases": null,
40
# "NetworkID": "e4c7ce50a5a2a49672155ff498597db336ecc2e3bbb6ee8baeebcf9fcfa0e1ab",
41
# "EndpointID": "2a2587f8285fa020878dd38bdc630cdfca0d769f76fc143d1b554237ce907371",
42
# "Gateway": "172.17.0.1",
43
# "IPAddress": "172.17.0.2",
44
# "IPPrefixLen": 16,
45
# "IPv6Gateway": "",
46
# "GlobalIPv6Address": "",
47
# "GlobalIPv6PrefixLen": 0,
48
# "MacAddress": "02:42:ac:11:00:02",
49
# "DriverOpts": null
50
# },
51
# "notes-api-network": {
52
# "IPAMConfig": {},
53
# "Links": null,
54
# "Aliases": [
55
# "37755e86d627"
56
# ],
57
# "NetworkID": "06579ad9f93d59fc3866ac628ed258dfac2ed7bc1a9cd6fe6e67220b15d203ea",
58
# "EndpointID": "5b8f8718ec9a5ec53e7a13cce3cb540fdf3556fb34242362a8da4cc08d37223c",
59
# "Gateway": "172.18.0.1",
60
# "IPAddress": "172.18.0.2",
61
# "IPPrefixLen": 16,
62
# "IPv6Gateway": "",
63
# "GlobalIPv6Address": "",
64
# "GlobalIPv6PrefixLen": 0,
65
# "MacAddress": "02:42:ac:12:00:02",
66
# "DriverOpts": {}
67
# }
68
# }
69
# }
70
# }
71
# ]
Copied!
I've shortened the output for easy viewing here. On my system, the
notes-db container is running, uses the notes-db-data volume and is attached to the notes-api-network bridge.Once you're assured that everything is in place, you can run a new container by executing the following command:
1
docker container run \
2
--detach \
3
--name=notes-api \
4
--env DB_HOST=notes-db \
5
--env DB_DATABASE=notesdb \
6
--env DB_PASSWORD=secret \
7
--publish=3000:3000 \
8
--network=notes-api-network \
9
notes-api
10
11
# f9ece420872de99a060b954e3c236cbb1e23d468feffa7fed1e06985d99fb919
Copied!
You should be able to understand this long command by yourself, I'll go through the environment variables briefly. The
notes-api application requires three environment variables to be set. They are as follows:DB_HOST- This is the host of the database server. Given both the database server and the API is attached to the same user-defined bridge network, the database server can be refereed to using its container name which isnotes-dbin this case.DB_DATABASE- The database that this API will use. On Running the Database Server we set the default database name tonotesdbusing thePOSTGRES_DBenvironment variable. We'll use that here.DB_PASSWORD- Password for connecting to the database. This was also set on Running the Database Server sub-section using thePOSTGRES_PASSWORDenvironment variable.
To check if the container is running properly or not, you can use the
container ls command:1
docker container ls
2
3
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4
# f9ece420872d notes-api "docker-entrypoint.s…" 12 minutes ago Up 12 minutes 0.0.0.0:3000->3000/tcp notes-api
5
# 37755e86d627 postgres:12 "docker-entrypoint.s…" 17 hours ago Up 14 minutes 5432/tcp notes-db
Copied!
The container is running now. You can visit
http://127.0.0.1:3000/ to see the API in action.
The API has five routes in total that you can see inside the
/notes-api/api/api/routes/notes.js file. It was bootstrapped with one of my open-source projects:https://github.com/fhsinchy/create-node-rocket-api
github.com
spare a ⭐ to keep me motivated
Although the container is running, there is one last thing that you'll have to do before you can start using it. You'll have to run the database migration necessary for setting up the database tables, and you can do that by executing
npm run db:migrate command inside the container.Executing Commands in a Running Container
You've already learned about executing commands in a stopped container. Another scenario is executing a command inside a running container. For this, you'll have to use the
exec command to execute a custom command inside a running container.The generic syntax for the
exec command is as follows:1
docker container exec <container identifier> <command>
Copied!
To execute
npm run db:migrate inside the notes-api container, you can execute the following command:1
docker container exec notes-api npm run db:migrate
2
3
# > [email protected] db:migrate /home/node/app
4
# > knex migrate:latest
5
#
6
# Using environment: production
7
# Batch 1 run: 1 migrations
Copied!
In cases where you want to run an interactive command inside a running container, you'll have to use the
-it flag. As an example, if you want to access the shell running inside the notes-api container, you can execute the following command:1
docker container exec -it notes-api sh
2
3
# / # uname -a
4
# Linux b5b1367d6b31 5.10.9-201.fc33.x86_64 #1 SMP Wed Jan 20 16:56:23 UTC 2021 x86_64 Linux
Copied!
Writing Management Scripts
Managing a multi-container project along with the network and volumes and stuff means writing a lot of commands. To simplify the process, I usually take help of simple shell scripts and a Makefile. You'll find four shell scripts in the
notes-api directory. They are as follows:boot.sh- Used for starting the containers if they already exist.build.sh- Creates and runs the containers. It also creates the images, volumes and networks if necessary.destroy.sh- Removes all containers, volumes and networks associated with this project.stop.sh- Stops all running containers.
There is also a
Makefile that contains four targets named start, stop, build and destroy each invoking the previously mentioned shell scripts.If the container is in running state in your system, executing
make stop should stop all the containers. executing make destroy should stop the containers and remove everything. Make sure you're running the scripts inside the notes-api directory (if you're still in notes-api\api, you can use the command cd .. to back up one level to the correct directory before running the following):1
make destroy
2
3
# ./shutdown.sh
4
# stopping api container --->
5
# notes-api
6
# api container stopped --->
7
8
# stopping db container --->
9
# notes-db
10
# db container stopped --->
11
12
# shutdown script finished
13
14
# ./destroy.sh
15
# removing api container --->
16
# notes-api
17
# api container removed --->
18
19
# removing db container --->
20
# notes-db
21
# db container removed --->
22
23
# removing db data volume --->
24
# notes-db-data
25
# db data volume removed --->
26
27
# removing network --->
28
# notes-api-network
29
# network removed --->
30
31
# destroy script finished
Copied!
If you're getting permission denied error than execute
chmod +x on the scripts:1
chmod +x boot.sh build.sh destroy.sh shutdown.sh
Copied!
I'm not going to explain these scripts because they're simple
if-else statement along with some Docker commands that you've already seen many times. If you have some understanding of the Linux shell, you should be able to understand the scripts as well.Last modified 11mo ago