How to Use Docker Volume and Docker Network for DevOps | Day 19 of 90 Days Of DevOps

In this blog, I’ll show you how to use Docker Volume an important feature of Docker that enables persistent data storage.

Docker Volume and Docker Network are essential for developing, testing, and deploying complex applications that have multiple components or microservices.

I’ll just give an introduction to Docker Network and I’ll not cover the practical example of it. But, we will cover the example for Docker Volume.

Docker Volume

Docker Volume is a storage mechanism that is used for persisting data generated by and used by Docker containers. Docker Volumes are managed by Docker itself and are independent of the container lifecycle. This means that the data in a volume is not lost when a container is stopped or deleted, and can be easily backed up or migrated.

Docker Volumes have several advantages over other methods of storing data in containers, such as:

• Volumes are easier to use and more flexible than bind mounts, which depend on the directory structure and OS of the host machine.
• Volumes can be shared and reused among multiple containers, even across different hosts.
• Volumes can use different drivers to store data on remote hosts, and cloud providers, or to encrypt or compress the data.
• Volumes can be created with default content from an existing container or image.

Docker Network

Docker Network is a virtual network created by Docker to enable communication between Docker containers. Docker Network allows containers to connect to each other and to non-Docker workloads, such as external services or databases. Docker Network also provides network isolation and security for containers.

Docker Network supports different types of networks, such as:

• Bridge: The default network type that connects containers to a private network on the host machine. Containers on the same bridge network can communicate with each other by name or alias, while containers on different bridge networks need to use port mapping or links to communicate.
• Host: A network type that connects containers directly to the host’s network stack, without any isolation. Containers on the host network can communicate with each other and with any other process on the host, but they cannot use port mapping or links.
• Overlay: A network type that connects containers across multiple hosts or clusters, using a distributed key-value store such as Consul or etcd. Containers on the same overlay network can communicate with each other by name or alias, regardless of their physical location.
• Macvlan: A network type that assigns a MAC address to each container and connects them to the host’s physical network interface, without any isolation. Containers on the same macvlan network can communicate with each other and with any other device on the same subnet, but they cannot communicate with the host machine.
• None: A network type that disables networking for a container, providing only a loopback interface. Containers on the none network cannot communicate with any other container or process.

Create a multi-container docker-compose file

You will create a multi-container docker-compose file that will bring up and bring down containers in a single shot. You will create an application that consists of a web server, an API server, and a MongoDB database.

To create a docker-compose file, you need to use a text editor of your choice and save the file as docker-compose.yml in your project directory. Here is an example of how your docker-compose file might look like:

version: "3.9" # specify the version of the compose file format
services: # define the services or containers that make up your application
  web: # name of the first service
    image: nginx # name of the image to use for this service
    ports: # list of ports to expose on the host machine
      - "80:80"
    volumes: # list of volumes to mount on this service
      - ./web:/usr/share/nginx/html # mount the ./web directory on the host as /usr/share/nginx/html in the container
    depends_on: # list of services that this service depends on
      - api # depend on the api service
  api: # name of the second service
    build: ./api # path to the directory containing the Dockerfile for this service
    environment: # list of environment variables to pass to this service
      - MONGO_URL=mongodb://db:27017/test # specify the URL for connecting to MongoDB
    ports: # list of ports to expose on the host machine
      - "3000:3000"
    depends_on: # list of services that this service depends on
      - db # depend on the db service
  db: # name of the third service
    image: mongo # name of the image to use for this service
    volumes: # list of volumes to mount on this service
      - db_data:/data/db # mount a named volume called db_data as /data/db in the container
volumes: # define any named volumes used by the services
  db_data: # name of the volume for persisting database data

This file defines an application that consists of three services:

• web: A web server that uses the nginx image and exposes port 80 on the host machine. It mounts the ./web directory on the host as /usr/share/nginx/html in the container, which contains the static files for the web page. It depends on the api service, which provides the backend functionality.
• api: An API server that uses a custom image built from the Dockerfile in the ./api directory. It passes an environment variable that specifies the URL for connecting to MongoDB. It exposes port 3000 on the host machine, which is the port that the API listens on. It depends on the db service, which provides the database for storing data.
• db: A database server that uses the mongo image and mounts a named volume called db_data as /data/db in the container, which persists the database data. The named volume is also defined at the end of the file.

To run this application using Docker Compose, you need to navigate to the directory where your docker-compose.yml file is located, and then run this command:

docker compose up -d

This will create and start all the services defined in your file in detached mode, which means they run in the background. You can also use the docker compose scale command to increase or decrease the number of replicas for a specific service. For example, if you want to scale up the api service to three replicas, you can use this command:

docker compose scale api=3

This will create two more containers for the api service and distribute the load among them.

To view the status of all containers, you can use the docker compose ps command, which shows information such as name, command, state, and ports for each container. For example, you might see something like this:

NAME                COMMAND                  SERVICE             STATUS              PORTS
test_api_1          "docker-entrypoint.s…"   api                 running             0.0.0.0:3000->3000/tcp
test_api_2          "docker-entrypoint.s…"   api                 running             0.0.0.0:3001->3000/tcp
test_api_3          "docker-entrypoint.s…"   api                 running             0.0.0.0:3002->3000/tcp
test_db_1           "docker-entrypoint.s…"   db                  running             27017/tcp
test_web_1          "/docker-entrypoint.…"   web                 running             0.0.0.0:80->80/tcp

To view the logs of a specific service, you can use the docker compose logs command with the name of the service. For example, if you want to view the logs of the web service, you can use this command:

docker compose logs web

This will display the standard output and standard error of the web service.

To stop and remove all containers, networks, and volumes associated with the application, you can run this command:

docker compose down

This will stop and remove everything that was created by docker compose up.

Learn how to use Docker Volumes and Named Volumes

Here you will learn how to use Docker Volumes and Named Volumes to share files and directories between multiple containers. You will create two or more containers that read and write data to the same volume using the docker run — mount command. You will verify that the data is the same in all containers by using the docker exec command to run commands inside each container. You will use the docker volume ls command to list all volumes and docker volume rm command to remove the volume when you’re done.

To create a named volume, you need to use the docker volume create command with a name for your volume. For example, if you want to create a volume named my-volume, you can use this command:

docker volume create my-volume

This will create a volume named my-volume on your host machine.

To mount a named volume on a container, you need to use the docker run — mount command with a type of volume and a source and target for your mount point. For example, if you want to mount my-volume as /data in an alpine container, you can use this command:

docker run -d --name alpine1 --mount type=volume,source=my-volume,target=/data alpine tail -f /dev/null

This will create and start a container named alpine1 from the alpine image and mount my-volume as /data in it.

To create another container that mounts the same volume, you can use a similar command with a different name for your container. For example, if you want to create another alpine container named alpine2 that mounts my-volume as /data, you can use this command:

docker run -d --name alpine2 --mount type=volume,source=my-volume,target=/data alpine tail -f /dev/null

This will create and start a container named alpine2 from the alpine image and mount my-volume as /data in it.

To verify that the data is the same in both containers, you can use the docker exec command to run commands inside each container. For example, if you want to create a file named hello.txt with some content in /data in alpine1, you can use this command:

docker exec alpine1 sh -c "echo 'Hello from alpine1' > /data/hello.txt"

This will run a shell command inside alpine1 that writes “Hello from alpine1” to /data/hello.txt.

To check if the file is also present in /data in alpine2, you can use this command:

docker exec alpine2 cat /data/hello.txt

This will run a cat command inside alpine2 that displays the content of /data/hello.txt. You should see something like this:

Hello from alpine1

This means that the data is shared between both containers through the same volume.

To list all volumes on your host machine, you can use the docker volume ls command, which shows the name and driver of each volume. For example, you might see something like this:

DRIVER    VOLUME NAME
local     my-volume

You can see that your volume is using the local driver and has the name my-volume.

To remove a volume from your host machine, you need to use the docker volume rm command with the name of the volume. For example, if you want to remove my-volume, you can use this command:

docker volume rm my-volume

This will remove the volume and delete its data. You can also use flags such as -f or — force to remove a volume even if it is being used by a container.

Conclusion

In this blog, I have shown you how to use Docker Volume an important feature of Docker that enables persistent data storage. I have also shown you how to create a multi-container docker-compose file that will bring up and bring down containers in a single shot. I have also shown you how to create and mount volumes on containers, verify that the data is shared between containers, and list and remove volumes using Docker commands.

Thank you for reading and happy learning! 😊