Docker hands on guide

This hands on session is more focus on how to setup our project locally using docker to run and test in our local machine and how we can improve DX(Developer Experience) when hand overing the project.

I will do another blog regarding what is DX and how we can use it to improve productivity.

Common Problem We Have

In software development developers are using different machines, different dependencies and lots of different setups and yet develop the same system. We are frequently hearing one of the following questions.

This was working on my machine? How to run this project? How to set up the database? If you need to solve one of these questions then you are in the correct place.

How to solve the above

To solve all the above we need to have the same environment or need to have instructions to run the project or need to provide commands to setup the database or we need to Dockerize the project.

Yeah we are going to learn how we are going to Dockerize the project. Tighten your seat belts.

High level project that we are going to Dockerize

Following is the high level diagram of the Project that we are using. Please note that you do not need to know the internals of the project and this is a simple project which shows set of books which are stored in the database.

Sections that we are going to cover

In this session we learn about the followings,

1. How to run a docker image
2. Docker volumes
3. How to integrate backend with MySQL container
4. Docker networking
5. How to run a full stack setup using Docker compose

Photo by Ian Taylor on Unsplash

Before we go forward I recommend the following blogs as prerequisites.

1. What is Docker
2. Why we need Docker
3. Docker basic commands
4. How to create a custom Docker image
5. Docker image build process
6. Docker build and run commands
7. Docker Volumes
8. Docker multiple startup commands

Its good to have the above basic knowledge on Docker. So let’s dive in.

1. How to run a Docker container

In this section we learn about how to start a container using an image. In our project we need a MySQL server where either we have to install in our machine or we can create a MySQL server using a docker image.

You can simply create a container using the following command and for more information you can refer this.

docker run \
  --name <any-name-to-identify-conatiner> \
  -e ENV_VARIABLE_NAME=ENV_VARIABLE_VALUE \
  -p SOURCE_PORT:TARTGET_PORT \
  <image_name:image_tag>

When we run a docker container using an image we need to provide the following,

1. A name to identify the container (if not docker will provide a name)
2. Provide environment variables if needed. In the above we have provided one.
3. Port binding to local machine to container. This will help to communicate to the container.
4. Image name which we use to create the container.

Please note that there are many configs we can use in docker such as volumes, networks etc.

When we create a MySQL container we need to provide the following.

mysql

Here we go,

docker run -d \
  --name mysql-server \
  -v db-data:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD=rootpassword \
  -e MYSQL_DATABASE=book_store \
  -e MYSQL_USER=user \
  -e MYSQL_PASSWORD=password \
  -p 3308:3306 \
  mysql/mysql-server:5.7

• -v used to mentioned the volume mounts, Here we mount volume to persists data even if stop the container and start it again or create a new container.
• -e used to add environment variables for set username, password, database (Optional) and there are many more which is in their documentation which can be used if needed.
• -p used to expose the port to the local machine.
• Finally the image that we use to create the container.

Once you run it you can access it using the following command.

mysql -h 127.0.0.1 -P 3308 -u user -ppassword

Isn’t it hard to provide it via the command line? What if we have 10 or more environment variables? Is there another way of doing it?

Yes. Next we are going to learn about docker compose where we can configure one or more containers and run it using a config file. What you need to do is run docker-compose up . Let’s dive into a sample file.

version: '3' # Docker compose version
services:
  db:
    image: mysql/mysql-server:5.7  # Image
    volumes:
      - db-data:/var/lib/mysql  # Location to persist data
    environment: # Environment variables
      MYSQL_ROOT_PASSWORD: rootpassword
      MYSQL_DATABASE: book_store
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    ports: # Exposed port
      - "3308:3306"
    restart: always
volumes: # Volumes which used in services section
  db-data:

Above will start a container same as the docker command.

2. Docker Volumes

You might have a question on volumes in above section. Let’s clear it out. In containers when you start a container and remove it all the data will be wiped out. But there are scenarios that you need to persists some data for future use. Most classic example is the database. You might create a database container and remove it and in the next day you need to use the same data which was in that container.

How to solve this? You might think what if we save the data in our host machine? Yes it is. Volumes mount one of your local machine location to container and all the data will be saved in that location.

In the above example we have a volume called db-data and it is mounted to the container location /var/lib/mysql where database save the data. Next time we start the container and we have all the data inside our container.

You can find detailed blog here and repo link can be found here.

3. How to integrate backend with MySQL container

In this section we learn multi step image building for our backend. I have added all the code here and for this section checkout the section_3 branch and run it.

Important — When we build images we need to make sure that we pack only the programs that are needed to run since we need to concern about the size of the image.

You can learn how to create your custom image here.

In this session we focus on docker multistage image building process. In our go program first we build the executable and then copy the executable to another image to make sure that we have only the minimal set of files and programs in the built image.

Following is the docker file to build the image.

# Start from the official Go image to create a build artifact.
FROM golang:1.18 as builder

# Set the Current Working Directory inside the container
WORKDIR /app

# Copy go mod and sum files
COPY go.mod go.sum ./

# Download all dependencies. Dependencies will be cached if the go.mod and go.sum files are not changed
RUN go mod download

# Copy the source from the current directory to the Working Directory inside the container
COPY . .

# Build the Go app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .

######## Start a new stage from scratch #######
FROM alpine:latest  

RUN apk --no-cache add ca-certificates

WORKDIR /root/

# Copy the Pre-built binary file from the previous stage
COPY --from=builder /app/main .

# Expose port 8080 to the outside world
EXPOSE 8080

# Command to run the executable
CMD ["./main"]

As you can see using COPY command we can copy from the previous container builder to this image. Wait what from container to image.

In image building process we create containers and at the end we create the snapshot. For more information you can read this.

You can simply go to backend folder and run docker build -t backend . to build the image and to run it you can run docker run -p 8080:8080 backend . (Please make sure that your MySQL container is up and running. You can find the full code here.

Let’s open the url in your browser http://localhost:8080/books .Is it working? No it is giving an error. Why is that? It is all about networking.

4. Docker networking

In this section we are learning about how to call a container within docker host. We are in progress of writing a separate blog on docker networking which include much details.

In Docker compose we can easily label different conatiner names as services and If both containers are shared the same Docker host (within same network) we can call the container using <service_name>:<port> as follows.

Let’s update the Docker compose to support this.

version: '3'
services:
  backend:
    build: ./backend  
    ports:
      - "8080:8080"
    depends_on:
      - db
  db:
    image: mysql/mysql-server:5.7 
    volumes:
      - db-data:/var/lib/mysql
      - ./mysql/init.sql:/docker-entrypoint-initdb.d/init.sql
    environment:
      MYSQL_ROOT_PASSWORD: rootpassword
      MYSQL_DATABASE: book_store
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    ports:
      - "3308:3306"
    restart: always
volumes:
  db-data:

I have updated the backend MySQL connection string as follows.

db, err := sql.Open("mysql", "user:password@tcp(db:3306)/book_store?parseTime=true")

Let’s start the containers using docker-compose up --build . Build command will build the image and start the containers. Updated code can be found here.

5. How to run a full stack setup using Docker compose

We are at the final stage where we need to add the frontend using docker and start the whole system using Docker compose. (Please note for the frontend we are using react dev server for now)

Following is the docker-compose.

version: '3'
services:
  frontend:
    build: ./frontend
    ports:
      - "80:80"
    depends_on:
      - backend
  backend:
    build: ./backend  
    ports:
      - "8080:8080"
    depends_on:
      - db
  db:
    image: mysql/mysql-server:5.7 
    volumes:
      - db-data:/var/lib/mysql
      - ./mysql/init.sql:/docker-entrypoint-initdb.d/init.sql
    environment:
      MYSQL_ROOT_PASSWORD: rootpassword
      MYSQL_DATABASE: book_store
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    ports:
      - "3308:3306"
    restart: always
volumes:
  db-data:

You can find the full code here and enjoy.

If you have found this helpful please hit that 👏 and share it on social media :).