Remote desktop with hardware acceleration on GPU servers using Linux

Introduction : Use cases and difficulties

Why do some users want a remote desktop service with hardware acceleration on a Linux server in the cloud ?

When it comes to cloud Linux servers, people often use them with command in line instructions over an SSH access. But regularly, some users need a graphical environment.

Of course, there are users who are not yet familiar with CLI system administration, and looking for a graphical user interface : a desktop environment like the one on their computer (Ubuntu Desktop / Unity, Mate, KDE Plasma, etc.). But there are also experienced Linux users who need a graphical user interface for a totally different reason : They use hardware acceleration.

Hardware acceleration allows the server's GPU to be used to speed up graphics tasks and reduce CPU load. This results in a smoother and more responsive user experience.

• There are plenty of users running software with graphical features coupled to GPU, for example :
• A video game designer who edits files for Unreal Engine or Unity
• A mechanical engineer who draws object for 3D printers with FreeCAD
• An animation designer who wants to render his movie with Blender
• A data scientist who does physical simulations (computer vision, fluid mechanics simulation) with CUDA
• A designer of a real estate company using a virtual tour software.

What’s the challenge of setting this up in the cloud ?

In the cloud, having a screen like you would have if you're plugged on the HDMI/VGA port of the server is a little bit more tricky. The screen needs to be streamed on the network (between the cloud server and the user’s laptop). It’s where you need a remote desktop solution :

• A remote desktop server application installed on the Linux server
• A remote desktop client application installed on the user laptop (depending on the solution, it can be just a web browser)

When it comes to remote desktop using hardware acceleration, instead of using Linux, a lot of users go to Microsoft Windows or Apple MacOS environments, where remote desktop servers are easier to use.
For example, at Scaleway :

• They can use Apple Mac Mini as a dedicated server, which has GPUs and ARD (Apple Remote Desktop handling hardware acceleration in a few clicks. Whether you are using the native MacOS client, or a client like RealVNC for Linux or Windows. How to access to the remote desktop of a Mac mini | Scaleway Documentation
• They can set up their own Windows OS on GPU Elastic Metal and GPU Dedibox dedicated servers, and use Microsoft Remote Desktop, which works with native graphics acceleration too.
• Be careful with the Windows and Nvidia licensing. Especially if you plan to do virtualisation specific drivers may be needed.

In the Linux world, you need to set up your own remote desktop server application. But almost all open source remote access solutions based on VNC or XRDP do not have a simple graphical acceleration feature yet.
It’s not easy at all to find an open source solution, especially if you are not an expert on this topic !

How to use Scaleway’s GPU acceleration on Linux remote desktop ?

On our GPU Instances, GPU Elastic Metal and GPU Dedibox, you can use our preconfigured Linux images.

Until now, to have a remote desktop and a graphical environment on Linux, we provided tutorials such as this one. But nothing with GPU acceleration on Linux.

Recently, we managed to find an open source solution to provide a full desktop streaming (this is also sometimes called a “webtop”) including the audio streaming of the server.

This solution is named Selkies, and it’s an open-source low-latency high-performance Linux-native GPU/CPU-accelerated WebRTC HTML5 remote desktop streaming platform. It streams content from the server to a web browser by using WebRTC, so that you can access any available GPUs for hardware encoding or decoding of real-time streams.

It runs inside this Docker container, provides a KDE Plasma graphical environment, and does not require the installation of a remote desktop client application on the user’s computer. It is accessible from the internet browser of the end user, and allows him to control the application.

You just have to run the container. And with only one docker run command, you have a remote desktop server fully operational. Optionally, if you want to encrypt the connection, you can use an ssh tunnel to do port forwarding.

Example:

# Login into the GPU server and forward a port to encrypt the connectionssh -L 8080:localhost:8080 root@<PUBLIC_IP># Clone the repository and go inside itgit clone https://github.com/selkies-project/docker-nvidia-glx-desktop.gitcd docker-nvidia-glx-desktop# Run the Docker containerdocker run --name xgl -it -d --gpus all --tmpfs /dev/shm:rw \  -e TZ=UTC \  -e DISPLAY_SIZEW=1920 \  -e DISPLAY_SIZEH=1080 \  -e DISPLAY_REFRESH=60 \  -e DISPLAY_DPI=96 \  -e DISPLAY_CDEPTH=24 \  -e PASSWD=mypasswd \  -e SELKIES_ENCODER=nvh264enc \  -e SELKIES_VIDEO_BITRATE=8000 \  -e SELKIES_FRAMERATE=60 \  -e SELKIES_AUDIO_BITRATE=128000 \  -e SELKIES_BASIC_AUTH_PASSWORD=<YOUR PASSWORD HERE> \  -e SELKIES_BASIC_AUTH_USER=<YOUR LOGIN HERE> \  -p 8080:8080 \  -e SELKIES_TURN_PROTOCOL=udp \  -e SELKIES_TURN_PORT=3478 \  -e TURN_MIN_PORT=65534 \  -e TURN_MAX_PORT=65535 \  -p 3478:3478 \  -p 3478:3478/udp \  -p 65534-65535:65534-65535 \  -p 65534-65535:65534-65535/udp \  --network=host \  --restart always \  ghcr.io/selkies-project/nvidia-glx-desktop:latest# On your laptop, open a browser at localhost:8080Copy code

As an example, I did a Terraform project containing a Cloud-init script to automate the deployment of Selkies remote desktop container at the boot of a GPU powered Scaleway Instance.

The Selkies website, the readme.md files of the Terraform example and the Selkies container will give you all the details about the infrastructure and software architecture.

Selkies remote desktop works on GPU powered Instances (Render, 3070, L4, L40S, H100 PCIe). Using the Ubuntu GPU OS image provided by Scaleway, the needed drivers and Docker engine are already installed. My Terraform project example works with all these instances.

It can also work on GPU powered Bare Metal servers (Elastic Metal, Dedibox). Using one of the Linux OS images provided by Scaleway. This way, you have access to the power of a full physical server. But it comes with a few more steps to set up by hand : Docker engine installation, and Nvidia toolkit /CUDA driver installation.

As an example, leveraging this kind of technology over Linux has a significant impact in this kind of businesses :
Product configurators with rendered 3D graphics for retail and automotive industries
Remote interactive visualization of large datasets for data science industries
Lightweight, ephemeral, low-cost virtual workstations for 3D design teams of industrial companies

Video demonstration

Here is a demonstration of remote desktop with hardware acceleration on 3 different environments :
An Apple Silicon MacMini M4 running MacOS
A GPU powered virtual machine (GPU L4 Instance) running Ubuntu
A GPU powered dedicated server (GPU L40S ElasticMetal) running Ubuntu