Enabling GPU acceleration on Ubuntu on WSL2 with the NVIDIA CUDA Platform

local-optimum · February 9, 2022, 10:11am

Key	Value
Summary	Enabling GPU acceleration on Ubuntu on WSL2 with the NVIDIA CUDA Platform
Categories	wsl2
Difficulty	2
Author	carlos.santanadeoliveira@canonical.com

Note: This doc is not being maintained anymore. Please see instead https://canonical-ubuntu-wsl.readthedocs-hosted.com/ .

Overview

Duration: 02:30

While WSL’s default setup allows you to develop cross-platform applications without leaving Windows, enabling GPU acceleration inside WSL provides users with direct access to the hardware. This provides support for GPU-accelerated AI/ML training and the ability to develop and test applications built on top of technologies, such as OpenVINO, OpenGL, and CUDA that target Ubuntu while staying on Windows.

What you will learn:

How to install a Windows graphical device driver compatible with WSL2
How to install the NVIDIA CUDA toolkit for WSL 2 on Ubuntu
How to compile and run a sample CUDA application on Ubuntu on WSL2

What you will need:

A Windows 10 version 21H2 or newer physical machine equipped with an NVIDIA graphics card and administrative permission to be able to install device drivers
Ubuntu on WSL2 previously installed
Familiarity with Linux command line utilities and interacting with Ubuntu on WSL2

ⓘ Note: If you need more introductory topics, such as how to install Ubuntu on WSL, refer to previous tutorials that can be found here for Windows 11 and here for Windows 10.

Prerequisites

The following steps assume a specific hardware configuration. Although the concepts are essentially the same for other architectures, different hardware configurations will require the appropriate graphics drivers and CUDA toolkit.

Make sure the following prerequisites are met before moving forward:

A physical machine with Windows 10 version 21H2 or higher
NVIDIA’s graphic card
Ubuntu 20.04 or higher installed on WSL 2
Broadband internet connection able to download a few GB of data

Install the appropriate Windows vGPU driver for WSL

Duration: 05:00

ⓘ Specific drivers are needed to enable use of a virtual GPU, which is how Ubuntu applications are able to access your GPU hardware, so you’ll need to follow this step even if your system drivers are up-to-date.

Please refer to the official WSL documentation for up-to-date links matching your specific GPU vendor. You can find these in Install support for Linux GUI apps > Prerequisites . For this example, we will download the NVIDIA GPU Driver for WSL.

ⓘ Note: This is the only device driver you’ll need to install. Do not install any display driver on Ubuntu.

Once downloaded, double-click on the executable file and click Yes to allow the program to make changes to your computer.

Confirm the default directory and allow the self-extraction process to proceed.

A splash screen appears with the driver version number and quickly turns into the main installer window. Read and accept the license terms to continue.

Confirm the wizard defaults by clicking Next and wait until the end of the installation. You might be prompted to restart your computer.

This step ends with a screen similar to the image below.

Install NVIDIA CUDA on Ubuntu

Duration: 08:00

ⓘ Normally, CUDA toolkit for Linux will have the device driver for the GPU packaged with it. On WSL 2, the CUDA driver used is part of the Windows driver installed on the system, and, therefore, care must be taken not to install this Linux driver as previously mentioned.

The following commands will install the WSL-specific CUDA toolkit version 11.6 on Ubuntu 22.04 AMD64 architecture. Be aware that older versions of CUDA (<=10) don’t support WSL 2. Also notice that attempting to install the CUDA toolkit packages straight from the Ubuntu repository (“cuda”, “cuda-11-0”, or “cuda-drivers”) will attempt to install the Linux NVIDIA graphics driver, which is not what you want on WSL 2. So, first remove the old GPG key:

sudo apt-key del 7fa2af80

Then setup the appropriate package for Ubuntu WSL:

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin

sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600

sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/3bf863cc.pub

sudo add-apt-repository 'deb https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/ /'

sudo apt-get update

sudo apt-get -y install cuda

Once complete, you should see a series of outputs that end in done.:

Congratulations! You should have a working installation of CUDA by now. Let’s test it in the next step.

Compile a sample application

Duration: 01:00

NVidia provides an open source repository on GitHub with samples for CUDA Developers to explore the features available in the CUDA Toolkit. Building one of these is a great way to test your CUDA installation. Let’s choose the simplest one just to validate that our installation works.

Let’s say you have a ~/Dev/ directory where you usually put your working projects. Navigate inside the directory and git clone the cuda-samples repository:

cd ~/Dev

git clone https://github.com/nvidia/cuda-samples

To build the application, go to the cloned repository directory and type make:

cd ~/Dev/cuda-samples/Samples/1_Utilities/deviceQuery

make

A successful build will look like the screenshot below. Once complete, run the application:

./deviceQuery

You should see a similar output to the following detailing the functionality of your CUDA setup (the exact results depend on your hardware setup):

Enjoy Ubuntu on WSL!

Duration: 1:00

That’s all folks! In this tutorial, we’ve shown you how to enable GPU acceleration on Ubuntu on WSL 2 and demonstrated its functionality with the NVIDIA CUDA toolkit, from installation through to compiling and running a sample application.

We hope you enjoy using Ubuntu inside WSL for your Data Science projects. Don’t forget to check out our blog for the latest news on all things Ubuntu.

Hello @local-optimum , thanks for your work, this tutorial is very useful!
After going through this tutorial, I think there is a minor issue that maybe worths some notice. I think after installing nvidia gpu driver on windows, the ubuntu subsystem should be restarted, like using powershell to execute wsl --shutdown and then start ubuntu again, if ubuntu is kept running during the gpu driver installation process.
Otherwise, the built cuda sample application in the last tutorial step will surprise the user with a segmentation fault message.
It’s so in my case, do not know if it generalizes to others. Hope this helps!

maxloo · December 22, 2023, 4:31pm

@local-optimum, I’ve once tried this: https://github.com/facebookresearch/llama/issues/936. I already have this in my .wslconfig:

[wsl2]
memory=24GB
processors=24

In order to attempt what you’ve suggested, I first remove the CUDA Runtime Library from the Linux side:
sudo apt-get --purge remove libcudart11.0:amd64

This was done because I thought that “2. Install the appropriate Windows vGPU driver for WSL” was meant for Windows, where I’ve already previously installed the studio version of the vGPU driver. Hence, I proceeded straight to “3. Install NVIDIA CUDA on Ubuntu” and removed the previous CUDA runtime library before installing your version.

I made it to the last step, which was repeated after everything failed, and I rebooted WSL2:

maxloo@maxloolen:~/src/Dev/cuda-samples/Samples/1_Utilities/deviceQuery$ sudo apt-get -y install cuda
[sudo] password for maxloo:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
cuda is already the newest version (12.3.1-1).
0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded.

Here’s the indication of failure:

maxloo@maxloolen:~/src/Dev/cuda-samples/Samples/1_Utilities/deviceQuery$ ./deviceQuery
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 100
-> no CUDA-capable device is detected
Result = FAIL

Due to the failure, I reinstalled vGPU on my Windows side as well, but this time, I tried installing the gaming version, not the studio version. This did not work. I also had this setting in my ~/.bashrc file due to my previous installation, which was also a bit of a hack because I either found it on the Internet or with the help of ChatGPT:

export PATH=/usr/local/cuda-12.3/bin${PATH:+:${PATH}}
if [ -z "${LD_LIBRARY_PATH}" ]; then
    export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64
else
    export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64:$LD_LIBRARY_PATH
fi

I tried removing it, but that did not worked too. Can anyone reading this help?