Kubernetes with GitLab CI/CD: Integration Testing and Binding Custom Domain

| October 22, 2020

In the previous article, we’ve described how to integrate Kubernetes cluster with GitLab server in order to automate CI/CD pipelines while running your applications in containers. In today’s article, we’ll describe how you can test this integration of Kubernetes with GitLab, as well as how to bind custom domains to the applications running within this setup in order to make them production-ready.

GitLab and Kubernetes Integration Testing

We have created a GitLab CI/CD example repository specifically for this article, named "gitlab-k8s-cicd-demo". It builds and deploys a "Hello World" Payara-based Java application. Therefore, you can fork the mentioned repository to your GitHub account for the further importing.

Java Project Import

1. Import the gitlab-k8s-cicd-demo repository from GitHub into your GitLab server. 

GitLab Java Project

More detailed instruction on importing projects is described in the tutorial “Private GitLab Server Automatic Installation with Jelastic PaaS”.

2. Then, go to the Your projects tab once importing is finished.

Kubernetes Integration Projects

3. And choose the project you’ve just imported.

Kubernetes Integration Demo

Project Variables

The next step is to set up the project’s variables used in the pipeline template file gitlab-k8s-cicd-demo/.gitlab-ci.yml

Open the project Settings tab, click on the CI/CD item, expand the Variables tab, and add the variables.

Kubernetes GitLab Variables

As for our project, these are the following variables:

  • K8S_SERVER - contains a Kubernetes control plane node URL. To get this value, log in the control plane node of K8s cluster via SSH and issue command:

kubectl cluster-info | grep 'Kubernetes control plane' | awk '/http/ {print $NF}'

Kubernetes nodes
  • K8S_CA_B64 - this variable must contain a certificate required by the Kubernetes authorization module. To obtain a certificate, issue command at the same control plane node and take the value from the keypair:

kubectl config view --raw -o json | grep certificate-authority-data

Kubernetes master node
Kubernetes access token
  • CI_REGISTRY_USER and CI_REGISTRY_PASSWORD - Docker Hub registry user credentials

Make sure that you have five variables defined for your project similar to the picture below:

GitLab Kubernetes Variables list

Pipeline Activation

Now it’s time to check whether your GitLab and Kubernetes pipeline is ready for production.  
1. Click on CI/CD > Pipelines > Run Pipeline.

GitLab Kubernetes Integration Run Pipeline

2. Confirm the action by clicking on the Run Pipeline button.

GitLab Kubernetes Integration Pipeline

3. Wait a few minutes and enjoy watching how your project automatically goes through the required steps of build and deploy.

GitLab Kubernetes Integration Pipeline Flow

4. When all the stages succeed, you can open your application. To get an application URL, click on the last stage deploy-topology.

GitLab Kubernetes Integration Pipeline Deploy

5. In the open output window find a record that specifies a context at which your application can be accessed, for example, /helloworld.

GitLab Kubernetes Integration Context

6. Add this context to the Kubernetes environment domain and press Enter to access the Hello World java application like: 

https://kubernetes.vip.jelastic.cloud/helloworld/

GitLab Kubernetes Integration Pipeline Flow
Note: Within the proposed example we use NGINX as an Ingress controller, therefore in case of other proxies such as HAProxy or Traefik you should apply respective changes to the topology/ingress.yml file.

So, you have passed testing of Kubernetes with GitLab integration within Jelastic PaaS, building and deploying an application. Now it is time to think about a production part of its lifecycle.

Custom Domain Name Binding

With respect to production mode, you should bind a custom domain name to serve your application. You can do it in two ways:

1. Add a dedicated Load Balancer node to the cluster topology and attach a public IP address to it. Register a custom domain name that points to this IP. Bind the custom domain to the load balancer node and issue a valid SSL certificate with help of Let's Encrypt SSL Add-On.

2. Add public IP to the worker node and register a custom domain name that points to this IP. In this case, domain binding and certificate issuing will be done automatically.

We will cover both options, so you can decide which one fits your case better.

Bind Custom Domain Name via Load Balancer

1. Add a Load Balancer node to the cluster topology and attach a public IP address to it. 

GitLab Kubernetes Public IP

2. Create an A record for the domain (e.g. kubernetes.jele.website) that points to the provided IP address at your domain registrar.

GitLab Kubernetes Load Balancer A record

3. Install Let’s Encrypt SSL Add-On on the load balancer node.

Jelastic Let's Encrypt Free SSL

4. Bind the custom domain name through External Domain(s) field.

GitLab Kubernetes Integration Custom Domain

5. Now you can run the pipeline and use a new domain name for accessing your applications.

GitLab Kubernetes Application Deploy

Bind Custom Domain Name to Worker

1. Attach public IP to the worker node.

GitLab Kubernetes Integration Worker

2. Create an A record for the domain (e.g. myservice.jele.website) that points to the provided IP address at your domain registrar.

Kubernetes Worker A record

3. Then, go to the add-ons of the Control Plane node and install Certificate Manager(Cert-Manager for short).

Kubernetes Certificate Manager

Cert-Manager is a Kubernetes service that provides TLS certificates from Let’s Encrypt or other CAs and manages their life cycles. Certificates can be requested and configured through the Ingress resource annotation using the cert-manager.io/cluster-issuer annotation, adding a tls section to the Ingress specification and configuring one or more Issuer or ClusterIssuer to determine the preferred CA. More information about Issuer and ClusterIssuer objects can be found in the official cert-manager documentation for Issuer items.

Enter a registered custom domain and apply to proceed to the installation and automatic certificate issuance.

Kubernetes Custom Domain

After successful notification a non-trusted certificate will be issued for the custom domain. This is due to the letsencrypt-staging request defined in the root ingress resource (helloworld-cert) which was applied upon Cert-Manager installation. The Let’s Encrypt’s staging environment will not issue trusted certificates but is used to ensure that the verification process is working properly before moving to production. 

So in order to get your application traffic to be encrypted via trusted SSL certificate, switch from Let’s Encrypt staging issuer to production one. 

To do this open cert-manager root ingress resource in the WebSSH terminal with command:

kubectl edit ing helloworld-cert

And replace the line:

cert-manager.io/cluster-issuer: letsencrypt-staging 

with

cert-manager.io/cluster-issuer: letsencrypt-prod 

Save changes and exit in the same way as it does in vi text editor.

Kubernetes Ingress

If instead of helloworld-cert, another ingress is used that requests and uses certificate for custom domain, you need to make sure that a trusted certificate is requested and issued. If necessary, edit the required ingress in the same way as helloworld-cert above.

Keep in mind that in case there is an appropriate certificate the new one won’t be requested. If you require to force the new certificate issuance you should delete existing one.

4. Finally, to make an application available via custom domain you should create an ingress resource for it. You can use prepared config from the topology directory of the project: topology/ingress-custom-domain.yml. Just replace <custom domain> with the real custom domain you used at the previous step.

For our example the modified ingress config looks like:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gitlab-hwservice-app
  namespace: master
  annotations:
    kubernetes.io/ingress.class: nginx-cert
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    kubernetes.io/tls-acme: "true"
spec:
  tls:
  - hosts:
    - myservice.jele.website
    secretName: external-domain-tls
  rules:
  - host: myservice.jele.website
    http:
      paths:
      - path: /helloworld
        pathType: Prefix
        backend:
          service: 
            name: hwservice
            port: 
              number: 8080

5. Once the ingress is created and applied the application will be available by new URL: 
https://myservice.jele.website/helloworld/

Kubernetes Worker Application

Congratulations! Now you know how easy it is to automate CI/CD pipelines for your applications using GitLab and Kubernetes integration in Jelastic PaaS. Explore the benefits while running your projects at one of Jelastic cloud hosting providers.

Related Articles

Kubernetes Integration with GitLab CI/CD Pipeline