header image

Code Snippets

Travis-CI Auto-Deploy to Kubernetes

| Last updated: 
#!/bin/bash -ev
# TravisCI Autodeploy to Kubernetes Script

# Ensure all required environment variables are present
if [ -z "$KUBERNETES_CLUSTER_CERTIFICATE" ] || \
    [ -z "$KUBERNETES_SERVER" ] || \
    [ -z "$KUBERNETES_SERVICE_ACC_TOKEN" ]; then
    >&2 echo 'Required variable unset, docker build and deploy failed'
    exit 1
fi

# Generate latest docker image name
container_version=$(git rev-parse --short=8 HEAD)

echo : "
Travis-CI autodeploy to kubernetes script
Repo: $TRAVIS_REPO_SLUG
Image Version: $container_version
"

# Update kube deployment files with new container version
find ./kube/ -type f -print0 | xargs sed -i "s/CONTAINER_VERSION/$container_version/g"

# Install kubernetes cli and add to path
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl

# Create cerifiticate file
echo "$KUBERNETES_CLUSTER_CERTIFICATE" | base64 --decode > cert.crt

# Apply updated kubernetes application config
kubectl --kubeconfig=/dev/null \
  --certificate-authority=cert.crt \
  --server="$KUBERNETES_SERVER" \
  --token="$KUBERNETES_SERVICE_ACC_TOKEN" \
  apply -f ./kube/

Automatically apply updated Kubernetes config to control plane using Travis-CI

Add script to project in suitable scripts directory:

./scripts/travis-ci/travis-ci-autodeploy-kube.sh

Add the travis-ci-autodeploy-kube.sh script as a script to the travis-ci deploy hook in travis.yml:

deploy:
  provider: script
  script: bash ./scripts/travis-ci/travis-ci-autodeploy-kube.sh
  on:
    branch: master

In the Kubernetes deployment create and apply a ServiceAccount object and bind a role to it, allowing limited deploy access to the cluster. For example:

Account: cicd-service-account.yml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cicd
  namespace: default

Role: cicd-role.yml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cicd
rules:
  - apiGroups: ["", "apps", "batch", "extensions", "storage.k8s.io"]
    resources: ["deployments", "services", "replicasets", "pods", "jobs", "cronjobs", "storageclasses", "persistentvolumeclaims"]
    verbs: ["*"]

Binding: cicd-role-binding.yml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cicd
subjects:
  - kind: ServiceAccount
    name: cicd
    namespace: default
roleRef:
  kind: ClusterRole
  name: cicd
  apiGroup: rbac.authorization.k8s.io

Ensure the required environment variables are set in the build. For secret credentials, add to build directly on travis-ci.com, or by using Travis secure values (see notes on max length of secure credentials).

  • KUBERNETES_CLUSTER_CERTIFICATE - Can be found (base64 encoded) in the .kube/config file on the Kubernetes server, key: cluster.certificate-authority-data.
  • KUBERNETES_SERVER - Can also be found in the .kube/config file on the Kubernetes server. Note the IP address may need substituting for your domain name.
  • KUBERNETES_SERVICE_ACC_TOKEN - After the above service account has been created, run kubectl get secret $(kubectl get secrets --namespace default | grep cicd-token | awk '{print $1}') -o jsonpath='{.data.token}' --namespace default | base64 --decode on kubernetes server to get the service account access token.

Before deploying, the script will also search and replace and references of 'CONTAINER_VERSION' in config files contained in ./kube/ with the first characters of the git commit. This provides specific container deployment and identification, allowing for quick rollbacks if required.

Notes

  • The CI/CD role is based in the default Kubernetes namespace, you can change this to suit your app setup.

  • The auto-deploy script assumes you have tagged your build image name with the first 8 characters of the git commit hash it was built from.

  • Max length of Travis-CI encrypted values seems to be around 128 bytes, therefore the KUBERNETES_CLUSTER_CERTIFICATE and KUBERNETES_SERVICE_ACC_TOKEN will have to be added directly to Travis-CI using either their web app or travis set env.

    • When adding environment variables as Travis-CI secure values they will only be available in pull requests created by trusted users. Therefore, if an outside user makes a pull request to your repository, they will not have access to your access token and the deployment will fail.

  • Depending on when your travis-ci account was made, secret may require encrypting with the --com flag. See this stackoverflow for more details.

Back to all snippets