How to set environment variables using Google Cloud Build

Let's talk about something that might seem straightforward but can get quite tricky :

handling environment variables when building Docker images with Cloud Build.

I'll share some insights from my experience and break down how to do it properly.

The Basic Setup

First, let's look at what we're trying to achieve. We want to:

• Build a Docker image using Cloud Build
• Pass environment variables securely
• Make sure these variables are available during build time and runtime

Understanding the Moving Parts

We're dealing with three main components:

• Cloud Build configuration (cloudbuild.yaml)
• Dockerfile
• Environment variables

The Cloud Build Configuration

Let's break down our Cloud Build configuration:

steps:
  - name: 'alpine'
    entrypoint: 'sh'
    args:
      - '-c'
      - |
        echo $_DB_HOST
        # ... other echo statements

This first step is actually a clever debug step! It helps us verify that our substitution variables are being passed correctly.

Pretty handy when things go wrong 🤓

The Tricky Part: Variable Passing

Here's where it gets interesting (and potentially confusing).

We have three levels of variable passing:

• Cloud Build substitutions ($_VARIABLE)
• Docker build arguments (ARG)
• Container environment variables (ENV)

  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/my-image', '.']
    env:
      - 'DB_HOST=$_DB_HOST'
      - 'DB_NAME=$_DB_NAME'
      # ... other variables

What are the substitutions variables?

Substitution variables are defined in the Cloud Build configuration file and are replaced with their values during the build process.

substitutions:
  _DB_HOST: your-host
  _DB_NAME: your-db
  _DB_PASSWORD: your-password
  _DB_PORT: your-port
  _DB_USER: your-user

You can define these variables in the cloudbuild.yaml file or in the Cloud Build UI.

Common Gotchas and Limitations 😅

Substitution Variable Naming: Cloud Build substitution variables MUST start with '_' (underscore). This is a common source of confusion!

Variable Scope: Variables defined in Cloud Build don't automatically transfer to your Docker build. You need to explicitly pass them through.

Build vs Runtime: Remember that ARG variables in Dockerfile are only available during build time. That's why we need to convert them to ENV variables if we want them at runtime.

Best Practices

Debug First

Always include a debug step (like our alpine step) to verify variables are being passed correctly.

Use ARG and ENV Together

In your Dockerfile, use this pattern:

ARG DB_HOST
ENV DB_HOST=$DB_HOST

Security Considerations

Remember that environment variables in your final image can be viewed by anyone with access to the image. Consider using secrets management for sensitive data.

Example: Node.js App

Here's an example of how you can set up environment variables for a Node.js app:

FROM node:18-alpine

WORKDIR /app

# Pass build-time variables
ARG DB_HOST
ARG DB_NAME
ARG DB_PASSWORD
ARG DB_PORT
ARG DB_USER

ENV DB_HOST=$DB_HOST
ENV DB_NAME=$DB_NAME
ENV DB_PASSWORD=$DB_PASSWORD
ENV DB_PORT=$DB_PORT
ENV DB_USER=$DB_USER

COPY package*.json ./

RUN npm install

COPY . .

RUN npm run build

EXPOSE 3000

CMD ["npm", "start"]

Then we need to have a .env file with our environment variables:

DB_HOST=your-host
DB_NAME=your-db
DB_PASSWORD=your-password
DB_PORT=your-port
DB_USER=your-user

for cloud build, we need to have a cloudbuild.yaml file:

steps:
  # Debug step to verify variables are being passed correctly
  - name: 'alpine'
    entrypoint: 'sh'
    args:
      - '-c'
      - |
        echo $_DB_HOST
        echo $_DB_NAME
        echo $_DB_PASSWORD
        echo $_DB_PORT
        echo $_DB_USER

  # Build the Docker image
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/my-image', '.']
    env:
      - 'DB_HOST=$_DB_HOST'
      - 'DB_NAME=$_DB_NAME'
      - 'DB_PASSWORD=$_DB_PASSWORD'
      - 'DB_PORT=$_DB_PORT'
      - 'DB_USER=$_DB_USER'

  # Push the Docker image to Google Container Registry
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'gcr.io/$PROJECT_ID/my-image']

images:
  - 'gcr.io/$PROJECT_ID/my-image'

options:
  logging: 'CLOUD_LOGGING_ONLY'  # Log build output to Cloud Logging only

Building locally w/o gcloud

To build your setup locally, you can use the docker CLI:

docker build \
  --build-arg DB_HOST=$(grep DB_HOST .env | cut -d '=' -f2) \
  --build-arg DB_NAME=$(grep DB_NAME .env | cut -d '=' -f2) \
  --build-arg DB_PASSWORD=$(grep DB_PASSWORD .env | cut -d '=' -f2) \
  --build-arg DB_PORT=$(grep DB_PORT .env | cut -d '=' -f2) \
  --build-arg DB_USER=$(grep DB_USER .env | cut -d '=' -f2) \
  -t node-app .

This command reads the environment variables from a .env file and passes them to the Docker build process.

Conclusion

While setting up environment variables in Cloud Build might seem complicated at first, it follows a logical pattern once you understand the flow.