Jump toUpdate content

Exposing services in Scaleway Kubernetes

Reviewed on 12 August 2021Published on 12 August 2021
Requirements:

Exposing the Ingress Controller through a Load Balancer service

Note:

You need a Kubernetes cluster deployed with an ingress controller. To deploy your cluster with a preinstalled ingress controller, enter the Advanced Options during cluster creation, set Deploy an ingress controller to Yes, and select the ingress controller from the dropdown list:

An Ingress Controller is an entry point that ingests your HTTP/HTTPS traffic and dispatches it to your services. More closely, it is a reverse proxy that will dynamically configure itself and will forward HTTP/HTTPS traffic to your services.

Scaleway Kubernetes has implemented the following ingress controllers: nginx, traefik, and traefik2.

Exposing Nginx

  1. Create and open the file nginx-loadbalancer.yml in your favorite text editor and copy the following content into it:

    apiVersion: v1
    kind: Service
    metadata:
    name: nginx-ingress
    namespace: kube-system
    labels:
    k8s.scw.cloud/ingress: nginx
    spec:
    type: LoadBalancer
    ports:
    - port: 80
    name: http
    targetPort: 8000
    - port: 443
    name: https
    targetPort: 8443
    selector:
    app.kubernetes.io/name: nginx
  2. Use kubectl to deploy the configuration:

    $ kubectl create -f nginx-loadbalancer.yml
    service/nginx-ingress created
  3. Verify that your Load Balancer has been deployed correctly:

    # kubectl get svc -n kube-system
    nginx-ingress Load Balancer 10.32.18.72 51.159.24.212 80:30122/TCP,443:31362/TCP 90s

    You can see here that the IP address of your Load Balancer is 51.159.24.212. If you ‘curl’ it you can reach the default backend(displaying “404 page not found”) as no ingress objects are created and you are reaching it through the IP address:

    $ curl 51.159.24.212
    404 page not found
    Note:

    It may take a few minutes until your cluster is fully deployed and Nginx becomes available.

Exposing Traefik2

To expose Traefik2 with a Scaleway Load Balancer, deploy the following yaml file on your cluster:

  1. Create and open the file traefik-loadbalancer.yml in a text editor and copy the following content into it:

    apiVersion: v1
    kind: Service
    metadata:
    name: traefik-ingress
    namespace: kube-system
    labels:
    k8s.scw.cloud/ingress: traefik2
    spec:
    type: LoadBalancer
    ports:
    - port: 80
    name: http
    targetPort: 8000
    - port: 443
    name: https
    targetPort: 8443
    selector:
    app.kubernetes.io/name: traefik
  2. Use kubectl to deploy the configuration:

    $ kubectl create -f traefik-loadbalancer.yml
    service/traefik-ingress created
  3. Verify that your Load Balancer has been deployed correctly:

    $ kubectl get svc -n kube-system
    traefik-ingress Load Balancer 10.37.89.202 195.154.68.108 80:30509/TCP,443:32138/TCP 43s

    You can see here that the IP address of your Load Balancer is 195.154.68.108. If you ‘curl’ it you can reach the default backend(displaying “404 page not found”) as no ingress objects are created and you are reaching it through the IP address:

    $ curl 195.154.68.108
    404 page not found
    Note:

    It may take a few minutes until your cluster is fully deployed and Traefik becomes available.

Creating a Load Balancer service

  1. Create a YAML manifest called lb.yaml and paste the following content into it. Then save the file and quit your text editor:

    apiVersion: v1
    kind: Service
    metadata:
    name: traefik-ingress
    namespace: kube-system
    labels:
    k8s-app: traefik-ingress-lb
    spec:
    type: LoadBalancer
    ports:
    - port: 80
    name: http
    targetPort: 80
    - port: 443
    name: https
    targetPort: 443
    selector:
    k8s-app: traefik-ingress-lb
  2. Run the following command to

    # kubectl create -f lb.yaml
    service/traefik-ingress created

    This manifest creates a service of the Load Balancer type. With it, you will get a public IP address. As this Load Balancer is created on Scaleway, you can also see it in your console.

    Tip:

    The Load Balancer is managed automatically for you by the cloud controller manager. If you want to know more about our cloud controller manager, check out our dedicated documentation.

  3. Get the IP address of your newly created Load Balancer:

    # kubectl get svc -n kube-system
    nginx-ingress Load Balancer 10.32.18.72 51.159.24.212 80:30122/TCP,443:31362/TCP 90s

Using the Kubernetes Kapsule Wildcard DNS

By default, on Kubernetes Kapsule, a wildcard round-robin DNS record is created, pointing to all your cluster nodes. This means that every time you add or delete a node in your cluster, the DNS record is updated to reflect the state of your nodes.

  1. Test the DNS entry of your Kubernetes Kapsule cluster (you can get the FQDN of your cluster in the Scaleway console):

    # host test.c39a0d71-f66c-4657-8fe1-c3280012311c.nodes.k8s.fr-par.scw.cloud
    test.49087273-8296-46cc-a82c-f08cb9623ce2.nodes.k8s.fr-par.scw.cloud is an alias for 49087273-8296-46cc-a82c-f08cb9623ce2.nodes.k8s.fr-par.scw.cloud
    49087273-8296-46cc-a82c-f08cb9623ce2.nodes.k8s.fr-par.scw.cloud has address 51.15.207.3
  2. Use a test application called cafe-ingress, to test the way it works. The application serves different web pages depending on the URL you type. Deploy it with the following command:

    $ kubectl create -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/master/examples/complete-example/cafe.yaml
  3. Create the ingress object with this YAML manifest, called cafe-ingress.yml. Please note that we use our DNS wildcard on the host stanza of this YAML file.

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
    name: cafe-ingress
    spec:
    rules:
    - host: test.49087273-8296-46cc-a82c-f08cb9623ce2.nodes.k8s.fr-par.scw.cloud
    http:
    paths:
    - path: /tea
    pathType: Prefix
    backend:
    service:
    name: tea-svc
    port:
    number: 80
    - path: /coffee
    pathType: Prefix
    backend:
    service:
    name: coffee-svc
    port:
    number: 80
  4. Deploy the application and check its status:

    # kubectl create -f cafe-ingress.yaml
    # kubectl get ing
    NAME CLASS HOSTS ADDRESS PORTS AGE
    cafe-ingress <none> test.49087273-8296-46cc-a82c-f08cb9623ce2.nodes.k8s.fr-par.scw.cloud 80 4m11s

    You can test that this ingress is configured correctly by accessing your test application:

    # curl http://test.49087273-8296-46cc-a82c-f08cb9623ce2.nodes.k8s.fr-par.scw.cloud/coffee
    Server address: 100.64.0.181:8080
    Server name: coffee-5f56ff9788-68xs2
    Date: 28/Apr/2020:13:34:26 +0000
    URI: /coffee
    Request ID: 9d2ee64655b936384a64cf89e7a975b0

Creating a wildcard DNS record and point your domain name to the IP address

  1. Create a wildcard DNS record, using the Scaleway DNS product, pointing to this IP address (the domain used in this tutorial will be “mytest.com”). A wildcard record (*.mydomain.com) allows you to point any sub-domain of your domain to the configured IP address.

  2. Verify that the domain is pointed to the IP address of your Load Balancer:

    $ host foobar.mytest.com
    foobar.mytest.com has address 195.154.68.108

    Your domain is now pointing to your Load Balancer IP, you can resolve any of your subdomain with that IP.

Deploying Cert Manager

Cert-manager is in charge of creating Let’s Encrypt TLS certificates to secure your website, to sum-up:

  • Create an ingress object for a specific subdomain (for instance foobar.mytest.com)
  • Let’s Encrypt must be sure that the domain belongs to you. For this reason, Let’s Encrypt requests a “challenge”, in our case, an HTTP challenge. Meaning here that Let’s Encrypt will try to reach foobar.mytest.com, and is able to see a specific hash on this page.
  • Cert-manager is serving this page for you by creating an ingress object and using an HTTP server.
  • When the challenge is ok, the certificate is created and added in a certificate object.
  • You can then use this certificate object to serve your website securely (HTTPS).
Note:

Any modification to the Traefik2 deployed by Kapsule may be overwritten by the reconciliation process, consider installing it yourself for a production usage.

  1. Modify the default Traefik 2 daemonset running on Kapsule to do that, add --providers.kubernetesIngress.ingressClass=traefik-cert-manager in the cmd stanza.

    $ kubectl edit ds traefik -n kube-system
    daemonset.apps/traefik edited
    []
    - --global.checknewversion
    - --global.sendanonymoususage
    - --entryPoints.traefik.address=:9000
    - --entryPoints.web.address=:8000
    - --entryPoints.websecure.address=:8443
    - --providers.kubernetesIngress.ingressClass=traefik-cert-manager
    - --api.dashboard=true
    - --ping=true
    - --providers.kubernetescrd
    - --providers.kubernetesingress
    []
  2. Delete the existing Traefik pods in order to get the new arguments.

    $ kubectl -n kube-system delete pod -l app.kubernetes.io/name=traefik
  3. Use the command below to install cert-manager and its needed CRD (Custom Resource Definitions):

    $ kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.1.1/cert-manager.yaml
    customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
    customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
    customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
    customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
    customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
    customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
    namespace/cert-manager created
    serviceaccount/cert-manager-cainjector created
    serviceaccount/cert-manager created
    serviceaccount/cert-manager-webhook created
    clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
    clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created
    clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
    clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created
    clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created
    clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created
    clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
    clusterrole.rbac.authorization.k8s.io/cert-manager-view created
    clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
    clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created
    clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created
    clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
    clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created
    clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created
    clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created
    clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
    role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
    role.rbac.authorization.k8s.io/cert-manager:leaderelection created
    role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
    rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
    rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created
    rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
    service/cert-manager created
    service/cert-manager-webhook created
    deployment.apps/cert-manager-cainjector created
    deployment.apps/cert-manager created
    deployment.apps/cert-manager-webhook created
    mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
    validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
See Also