How to do it...

1. We start by downloading NVIDIA with the following command in the terminal (adjust the download link accordingly if needed; make sure you use CUDA 8 and not CUDA 9 for now):

curl -O http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb

1. Next, we unpack the file and update all all packages in the package lists. Afterwards, we remove the downloaded file:

sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
rm cuda-repo-ubuntu1604_8.0.61-1_amd64.deb

1. Now, we're ready to install CUDA with the following command:

sudo apt-get install cuda-8-0

1. Next, we need to set the environment variables and add them to the shell script .bashrc:

echo 'export CUDA_HOME=/usr/local/cuda' >> ~/.bashrc
echo 'export PATH=$PATH:$CUDA_HOME/bin' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CUDA_HOME/lib64' >> ~/.bashrc

1. Make sure to reload the shell script afterwards with the following command:

source ~/.bashrc

1. You can check whether the CUDA 8.0 driver and toolkit are correctly installed using the following commands in your terminal:

nvcc --version
nvidia-smi

The output of the last command should look something like this:

1. Here, we can see that an NVIDIA P100 GPU with 16 GB of memory is correctly connected and ready to use. 

1. We are now ready to install cuDNN. Make sure the NVIDIA cuDNN file is available on the machine, for example, by copying from your local machine to the server if needed. For Google cloud compute engine (make sure you've set up gcloud and the project and zone are set up correctly), you can use the following command (replace local-directory and instance-name with your own settings):

gcloud compute scp local-directory/cudnn-8.0-linux-x64-v6.0.tgz instance-name

1. First we unpack the file before copying to the right directory as root:

cd
tar xzvf cudnn-8.0-linux-x64-v6.0.tgz
sudo cp cuda/lib64/* /usr/local/cuda/lib64/
sudo cp cuda/include/cudnn.h /usr/local/cuda/include/

1. To clean up our space, we can remove the files we've used for installation, as follows:

rm -rf ~/cuda
rm cudnn-8.0-linux-x64-v5.1.tgz