K8s - Binding IPv4 services externally as IPv6

As we know that there is a scarcity of global IPv4 address these days. Users are moving towards IPv6 addresses for their network. Hence, demand for services to be available over IPv6 addresses is increasing as well. It is not trivial to move your entire workload to Ipv6, which has been hosted as IPv4. In Kubernetes terminology, a seamless way to expose existing Ipv4 services as Ipv6 is needed for such use-cases and a functionality like NAT64 comes into play.

This blog is going to explain how we can provide IPv4 Kubernetes environment available to external IPv6 world using LoxiLB's NAT64 and load-balancer class functionality. We will follow the same topology from the previous blog with some changes at the external network side. Before we start this blog, we would suggest you to read the previous blog for better understanding as we may be skipping a few details.

Prepare the setup

We will install the setup using same steps explained in previous blog. You can download all the configuration files used for this blog from here. There is one difference though. We are going to use kube-loxilb instead of loxi-ccm.

Install kube-loxilb

"kube-loxilb" is LoxiLB's implementation of Kubernetes service load-balancer spec which includes support for load-balancer class, advanced IPAM etc. If you want to know more about kube-loxilb, please read this.

After downloading kube-loxilb.yaml file, find loxiURL and make changes as per your network, replace the IP addresses with LoxiLB docker IPs(facing towards Kubernetes network).

    args:
       - --loxiURL=http://192.168.59.101:11111,http://192.168.59.111:11111
       - --externalCIDR=3ffe::1/96
       - --setBGP=true
       - --setLBMode=2

Now, simply apply it:

$ sudo kubectl apply -f kube-loxilb.yaml 
$ sudo kubectl get pods -n kube-system | grep loxi
kube-loxilb-5f586c8497-mjxfs            1/1     Running   0            25s

External Router configuration

We are using bird bgp on an external router. Its configuration can be found here. Save the configuration as /etc/bird/bird6.conf and restart the bird service. One can run any bgp router in its place.

sudo apt-get install bird2 –yes
## Change bird.conf as needed
sudo systemctl restart bird6

Verify BGP connections

LoxiLB Node(s):

$ sudo docker exec -it loxilb gobgp neigh
Peer              AS  Up/Down State       |#Received  Accepted
192.168.59.211 64512 01:58:21 Establ      |        1         1
192.168.59.212 64512 01:58:22 Establ      |        1         1
192.168.59.213 64512 01:58:21 Establ      |        1         1
192.168.59.214 64512 01:58:14 Establ      |        1         1
2001::2        65001 01:58:20 Establ      |        0         0

External Router:

$ sudo birdc6
BIRD 1.6.8 ready.
bird> show protocols
name     proto    table    state  since       info
kernel1  Kernel   master   up     14:31:20    
device1  Device   master   up     14:31:20    
llb1     BGP      master   up     14:43:10    Established   
llb2     BGP      master   up     14:49:45    Established   
bird> 

Create a nginx Service

Let's create some worker services in k8s. We have used this nginx.yaml to create the service. Apply it in K8s node:

$ vagrant@node1:~$ sudo kubectl apply -f nginx.yaml
service/nginx-service created
deployment.apps/nginx-deployment created

Check external LB service is created :

$ sudo kubectl get svc
NAME             TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)           AGE
nginx-service   LoadBalancer   10.233.38.172   3ffe::1       8765:31469/TCP   9s

This output confirms that the service of type LoadBalancer - “3ffe::1 8765" has been created for an Ipv4 cluster-IP 10.233.38.172. We can verify the service created in any loxilb node:

$ sudo docker exec -it loxilb loxicmd get lb -o wide
| EXTERNAL IP | PORT  | PROTOCOL | BLOCK | SELECT |  MODE   |  ENDPOINT IP   | TARGET PORT | WEIGHT | STATE  |
|-------------|-------|----------|-------|--------|---------|----------------|-------------|--------|--------|
| 3ffe::1     | 8765  | tcp      |     0 | rr     | fullnat | 192.168.59.213 |       31478 |      5 | active |
|             |       |          |       |        |         | 192.168.59.214 |       31478 |      5 | active |

Let's revisit the BGP connections arrangement again:

The service IPv6 address will be advertised to external router as per the policy defined in the gobgp_loxilb.yaml used for this setup. We can verify the routes in external router:

$ sudo ip -6 route
::1 dev lo proto kernel metric 256 pref medium
2001::/64 dev enp0s8 proto kernel metric 256 pref medium
3ffe::1 via 2001::150 dev enp0s8 proto bird metric 1024 pref medium
fe80::/64 dev enp0s3 proto kernel metric 100 pref medium
fe80::/64 dev enp0s8 proto kernel metric 101 pref medium

Let's see how it works

We can try to access created service in the browser using the service IP as:

We hope you liked our blog. Please visit out github and website for more information.