Using the Proxy protocol v2 with Load Balancer
- compute
- Proxy-Protocol
- IP
- Load-Balancer
Proxy Protocol V2 Overview
The PROXY Protocol allows an application, like a web server such as Apache or Nginx, to retrieve client information of a user passing via a load balanced infrastructure.
The protocol transports connection information including the originating IP address, the proxy server IP address, and both ports.
This information can be useful for automatic delivery of localized contents, blacklisting of abusive users or for logging purposes.
You may need certain IAM permissions to carry out some actions described on this page. This means:
- you are the Owner of the Scaleway Organization in which the actions will be carried out, or
- you are an IAM user of the Organization, with a policy granting you the necessary permission sets
- You have an account and are logged into the Scaleway console
- You have configured your SSH key
- You have created and configured a Load Balancer in front of your servers
- You have sudo privileges or access to the root user.
When using a Load Balancer in front of your web servers, the load balancer’s IP address is passed by default to your web servers. This result is visible in the access log files of the web server:
51.159.26.16 - - [28/Jun/2019:13:42:25 +0000] "GET /favicon.ico HTTP/1.1" 200 26066 "http://example.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"51.159.26.16 - - [28/Jun/2019:13:42:25 +0000] "GET / HTTP/1.1" 200 26099 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"51.159.26.16 - - [28/Jun/2019:13:42:26 +0000] "GET /favicon.ico HTTP/1.1" 200 26068 "http://example.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"51.159.26.16 - - [28/Jun/2019:13:42:49 +0000] "GET / HTTP/1.1" 200 26022 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.1 Safari/605.1.15"
Only the IP 51.159.26.16
is visible in the log file, which is the Load Balancer’s IP address and not the IP of the different users.
The originating IP is also visible in the X-Forwarded-for:
HTTP header.
Enabling PROXY Protocol
You can enable proxy for your Load Balancer in two ways through the Scaleway console:
- Two options:
- Option 1: Configure backend rules upon creating your Load Balancer.
- Option 2: Click «Edit Icon», in the Backend tab of an existing Load Balancer.
- Select a proxy option from the drop-down menu.
Configuring Proxy Protocol in Nginx Web Server
The proxy_protocol
parameter must be set within the http {}
block of the listen directive of a server block to configure NGINX to accept PROXY protocol headers.
-
Make sure that Nginx is installed with the
http_realip_module
. This is the case in the precompiled version that is delivered with Ubuntu Bionic Beaver (18.04).nginx -V 2>&1 | grep -- 'http_realip_module' -
Open the configuration file of nginx, i.e.
/etc/nginx/nginx.conf
in a text editor, for example,nano
:nano /etc/nginx/nginx.conf -
Enable the
proxy_protocol
by adding/modifying the following lines in theserver {}
block:server {...listen 80 proxy_protocol;listen 443 ssl proxy_protocol;...} -
Set the IP address of the Load Balancer with the
set_real_ip_from
directive in theserver {}
block:set_real_ip_from 51.159.26.16; -
To change the IP address of the Load Balancer to the clients IP address, received from the PROXY protocol header, specify in the
server {}
block theproxy_protocol
parameter to the real_ip_header directive:real_ip_header proxy_protocol; -
As the clients IP address is now known to Nginx, configure the correct logging of it. Set the
proxy_set_header
directive with the$proxy_protocol_addr
variable in thehttp {}
block:http {proxy_set_header X-Real-IP $proxy_protocol_addr;proxy_set_header X-Forwarded-For $proxy_protocol_addr;} -
Configure the
$proxy_protocol_addr
variable to thelog_format
directive in the http block:http {#...log_format logs '$proxy_protocol_addr - $remote_user [$time_local] ''"$request" $status $body_bytes_sent ''"$http_referer" "$http_user_agent"';} -
The configuration file should look like the following example:
http {proxy_set_header X-Real-IP $proxy_protocol_addr;proxy_set_header X-Forwarded-For $proxy_protocol_addr;log_format logs '$proxy_protocol_addr - $remote_user [$time_local] ''"$request" $status $body_bytes_sent ''"$http_referer" "$http_user_agent"';#...server {server_name example.com;listen 80 proxy_protocol;set_real_ip_from 51.159.26.16;real_ip_header proxy_protocol;#...}} -
Save the configuration file, exit the editor, and test the syntax of the file:
nginx -t -
Restart Nginx:
systemctl restart nginx.service195.154.228.158 - - [28/Jun/2019:15:44:23 +0000] "GET /favicon.ico HTTP/1.1" 200 26062 "http://51.159.26.16/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"51.159.26.16 - - [28/Jun/2019:15:44:31 +0000] "GET /index.php HTTP/1.1" 200 26100 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"195.154.228.158 - - [28/Jun/2019:15:44:32 +0000] "GET /favicon.ico HTTP/1.1" 200 26065 "http://51.159.26.16/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"51.159.26.16 - - [28/Jun/2019:15:44:50 +0000] "GET /index.php HTTP/1.1" 200 26097 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"Important:Once ProxyProtocol is enabled the Nginx virtual host will no longer accept direct connections.
Configuring Proxy Protocol in Apache Web Server
The Proxy Protocol support in Apache is still in an early stage. It is currently not natively supported with the Apache version provided by Ubuntu. An unofficial plugin is available for Apache versions prior to 2.4.30. To use it, follow these steps:
-
Download and install the required software components:
apt install git apache2-dev -
Download the sources of the plugin and enter their directory:
git clone https://github.com/roadrunner2/mod-proxy-protocol.gitcd mod-proxy-protocol -
Build the plugin:
apxs -i -a -c mod_proxy_protocol.c -
Activate the module in the configuration of the virtual host:
<VirtualHost *:80>...ProxyProtocol On...</VirtualHost> -
Replace all
%h
by%a
of theLogFormat
directives in the Apache configuration file (/etc/apache2/apache2.conf
):LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combinedLogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combinedLogFormat "%a %l %u %t \"%r\" %>s %O" commonLogFormat "%{Referer}i -> %U" refererLogFormat "%{User-agent}i" agent -
Restart Apache to activate the new configuration:
apache2ctl restartImportant:Once ProxyProtocol is enabled the Apache virtual host will no longer accept direct connections.
Configuring Proxy Protocol in Apache with mod_remoteip
An Apache version >= 2.4.30 is required to use mod_remoteip. This version is currently not yet available in Ubuntu’s repositories, and you have to install it by yourself. Use it only if you know what you are doing.
- Enable the module
remoteip
:a2enmod remoteip - Configure the Apache virtual host configuration:
<VirtualHost *:80>...RemoteIPProxyProtocol On...</VirtualHost>
- Restart the Apache web server:
apache2ctl restart
Once ProxyProtocol is enabled the Apache virtual host will no longer accept direct connections.
Configuring Proxy protocol for ingress-nginx
Deployed through kubectl
-
Add the two annotations below to your ingress controller
service
:service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2: "true"
service.beta.kubernetes.io/scw-loadbalancer-use-hostname: "true"
kubectl annotate -n ingress-nginx services ingress-nginx-controller "service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2=true"kubectl annotate -n ingress-nginx services ingress-nginx-controller "service.beta.kubernetes.io/scw-loadbalancer-use-hostname=true" -
Add the configuration below to the ingress controller
configmap
:kubectl patch -n ingress-nginx configmaps ingress-nginx-controller \-p '{"data":{"use-forwarded-headers": "true", "compute-full-forwarded-for": "true", "use-proxy-protocol": "true"}}'
Deployed through helm
If you’re using the ingress-nginx
Helm chart, find below a custom values file that will automatically create a load balancer resource and configure it to use proxy-protocol-v2
, and add the required ingress controller configuration to support this new communication protocol:
-
Create the file below (e.g.
ingress-nginx-scw.yaml
) using a text editor.controller:# Do not use Deployment if using `externalTrafficPolicy: "Local"`kind: DaemonSetservice:# Your LB IP here if you want (optional)#loadBalancerIP: "X.X.X.X"# This means that each node will deliver traffic only to the node-local endpoints of the service,# without masquerading the client source IP. (Traffic mistakenly sent to a node with no endpoints will be dropped.)externalTrafficPolicy: "Local"annotations:# Configure the Scaleway LB to use Proxy-Protocolservice.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2: "true"service.beta.kubernetes.io/scw-loadbalancer-use-hostname: "true"config:# Configure the ingress controller to use Proxy-Protocoluse-forwarded-headers: "true"compute-full-forwarded-for: "true"use-proxy-protocol: "true" -
Apply the above, on a new or an existing
ingress-nginx
installation:# Once the above file is created, you can run the below# https://kubernetes.github.io/ingress-nginx/deploy/#quick-starthelm upgrade --install ingress-nginx ingress-nginx \--repo https://kubernetes.github.io/ingress-nginx \--namespace ingress-nginx --create-namespace \-f ingress-nginx-scw.yaml