How to Federate Resources across Kubernetes Clusters for Unified Deployment

| June 23, 2021
Kubernetes Federated Resources

In the previous article, we have outlined the steps required to create a Kubernetes Federation within Jelastic PaaS. This tutorial demonstrates how to federate the resources and perform a unified deployment of an application to all members of the Federation.

So to move on, we consider that you’ve already followed the mentioned article and configured Kubernetes cluster federation with two members, in our case these are:

  • Federation Host Cluster: fedhost.vip.jelastic.cloud
  • Federation Member Cluster: member1.demo.jelastic.com

In order to properly federate resources across the clusters, log in via SSH to the master node of the Host Cluster and go through the steps below.

Federated Namespaces

Federated resources are namespace limited. First, we have to create a FederatedNamespace object in order to distribute custom namespace as federated.

1. Let the namespaces be federated:

fedhost~$ kubefedctl enable namespaces --kubefed-namespace kube-federation-system

2. Then create some namespace for example myfedns:

fedhost~$ kubectl create ns myfedns

3. The next step is to federate the namespace. Keep in mind that doing namespace federated all of the already existing resources within this namespace will be federated as well in case they are allowed to.

You can federate the namespace in two ways:

  • Via command with a common set of parameters:

fedhost~$ kubefedctl federate ns myfedns

  • Via config file specifying an extended set of parameters, for example federated_ns.yaml:
apiVersion: types.kubefed.io/v1beta1
kind: FederatedNamespace
metadata:
 name: myfedns
 namespace: myfedns
spec:
 placement:
   clusters:
   - name: fedhost
   - name: member1

This config indicates that custom namespace myfedns is federated into two instances: fedhost and member1. With such a config, you can specify the members will participate in the federation.

4. Apply config and check whether namespace was federated or not:

fedhost~$ kubectl apply -f federated_ns.yaml

The second approach allows us to provide more parameters, but in a nutshell they are pretty similar.

5. Check whether the namespace was federated successfully from the Host Cluster:

fedhost~$ kubectl -n myfedns get federatednamespaces

fedhost~$ kubectl get namespaces --context member1

Federated Namespaces

Federated Deployments

1. Enable deployments to be federated:

fedhost~$ kubefedctl enable deployments.apps --kubefed-namespace kube-federation-system

2. In case you already have a deployment (for example, deploy1), you may do it federated with a command:

fedhost~$ kubefedctl -n myfedns federate deployment deploy1

But if you need a new federated deployment, just create and apply the federated deployment configuration file for example federated_deployment.yaml. As an application to be deployed we’re going to use the NGINX web server defined as image: nginxdemos/nginx-hello.

apiVersion: types.kubefed.io/v1beta1
kind: FederatedDeployment
metadata:
 name: federated-deploy1
 namespace: myfedns
spec:
 template:
   metadata:
     name: deploy1
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: hello1
         version: v1
     template:
       metadata:
         labels:
           app: hello1
           version: v1
       spec:
         containers:
         - image: nginxdemos/nginx-hello
           imagePullPolicy: IfNotPresent
           name: hello1
           ports:
           - containerPort: 8080
 placement:
   clusters:
   - name: fedhost
   - name: member1
 overrides:
   - clusterName: master
     clusterOverrides:
     - path: "/spec/replicas"
       value: 2

3. Apply federated deployment on the Host Cluster:

fedhost~$ kubectl apply -f federated_deployment.yaml

4. Thus new deployment should be pushed to all members of Federation within the namespace myfedns.

fedhost~$ kubectl get deployment --context fedhost -n myfedns

fedhost~$ kubectl get deployment --context member1 -n myfedns

Federated Deployments

Federated Services

1. Let the services to be federated:

fedhost~$ kubefedctl enable services --kubefed-namespace kube-federation-system

2. You can federate existing service with basic parameters, for example svc1.

fedhost~$ kubefedctl -n myfedns federate service svc1

To create a new federated service prepare and apply a service configuration file, for example federated_service.yaml.

apiVersion: types.kubefed.io/v1beta1
kind: FederatedService
metadata:
 name: federated-svc1
 namespace: myfedns
spec:
 template:
   metadata:
     name: service1
     labels:
       app: hello1
   spec:
     ports:
     - name: http
       port: 80
       targetPort: 8080
     selector:
       app: hello1
 placement:
   clusters:
   - name: fedhost
   - name: member1

3. Apply service to the KubeFed on the Host Cluster and check its availability on all members:

fedhost~$ kubectl apply -f federated_service.yaml

fedhost~$ kubectl get svc --context fedhost -n myfedns

fedhost~$ kubectl get svc --context member1 -n myfedns

Federated Services

Federated Ingress

It’s time to create an ingress for the service federated-svc1. Let’s enable an ingress federation on the Host Cluster.

1. Currently the default active ingress type is ingresses.extensions. This type was declared as deprecated commencing from K8s version 1.19 and it will be removed from version 1.22+. They recommend using the new type ingresses.networking.k8s.io. That’s why disable deprecated ingress type and then enable new one:

fedhost~$ kubefedctl disable ingresses.extensions --kubefed-namespace kube-federation-system

fedhost~$ kubefedctl enable ingresses.networking.k8s.io --kubefed-namespace kube-federation-system

Kubernetes Federated Ingress

2. On the next step you can federate an existing ingress, for example ing1:

fedhost~$ kubefedctl -n myfedns federate ingresses.networking.k8s.io ing1

Note: For existing ingress you can’t specify new or override existing parameters.

Also, you can create and federate a new ingress, for example with this configuration:

apiVersion: types.kubefed.io/v1beta1
kind: FederatedIngress
metadata:
 name: federated-ing1
 namespace: myfedns
spec:
 template:
   annotations:
     kubernetes.io/ingress.class: nginx
     nginx.ingress.kubernetes.io/ssl-redirect: "false"
     nginx.ingress.kubernetes.io/rewrite-target: /$2
   spec:
     rules:
     - http:
         paths:
         - path: /test(/|$)(.*)
           pathType: Prefix
           backend:
             service:
               name: federated-svc1
               port:
                 number: 8080
 placement:
   clusters:
   - name: fedhost
   - name: member1

Save it as federated_ingress.yaml and apply:

fedhost~$ kubectl apply -f federated_ingress.yaml

3. Check ingresses availability across federation in the same way as you did for services or just try to open deployed NGINX application in a browser.

NGINX service should be running and exposed to outside by K8s URL + context path "/test" according to rewrite-target annotation at all KubeFed members:

  • Host Cluster
deployed NGINX application
  • Member Cluster 1
NGINX application

Congratulations! Now your federated applications can be managed as a unified deployment across multiple Kubernetes clusters located in different regions. Try it yourself using preferred Jelastic PaaS cloud providers.

Related Articles

Multi-Region Kubernetes Cluster Federation in Jelastic PaaS

Jelastic Released Kubernetes Package with Integrated Auto-Clustering and Pay-per-Use Pricing Model

Kubernetes Cluster Setup with Automated Scaling and Pay-per-Use Pricing