> ## Documentation Index
> Fetch the complete documentation index at: https://vastai-80aa3a82-examples-openclaw-openai-serverless.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# CUDA

# CUDA Programming on Vast.ai

## Introduction

This guide walks you through setting up and running CUDA applications on Vast.ai's cloud platform. You'll learn how to set up a CUDA development environment, connect to your instance, and develop CUDA applications efficiently using NVIDIA's development tools.

## Prerequisites

* A Vast.ai account
* Basic familiarity with CUDA programming concepts
* Basic knowledge of Linux command line
* [(Optional) Install TLS Certificate for Jupyter](/guides/instances/jupyter)
* [(Optional) SSH client installed on your local machine and SSH public key added the Keys section at cloud.vast.ai](/guides/instances/sshscp)
* [(Optional) Vast-cli installed on your local machine for command-line management](/cli/hello-world)
* [(Optional) Docker knowledge for customizing development environments](https://docs.docker.com/get-started/)

## Setup

### 1. Selecting the Right Template

Navigate to the [Templates tab](https://cloud.vast.ai/templates/) to view recommended templates.

Search for [NVIDIA CUDA](https://cloud.vast.ai?ref_id=62897\&template_id=61e14a0dd1f97aa0aa6719d20bc9b02e) template if:

* You need a standard CUDA development environment
* You want pre-configured security features (TLS, authentication)
* You require Jupyter notebook integration
* You need additional development tools like Tensorboard

[Make a custom CUDA template](/guides/templates/creating-templates) if:

* You need a specific CUDA or Python version
* You have special library requirements
* You want to minimize image size for faster instance startup

### 2. Edit the Template and Select Template

You can edit the template to use Jupyter launch mode if: 

* You're behind a corporate firewall that blocks SSH
* You prefer browser-based development
* You want persistent terminal sessions that survive browser disconnects
* You need quick access without SSH client setup
* You want to combine CUDA development with notebook documentation
* You plan to switch between multiple terminal sessions in the browser

You can edit the template to use SSH launch mode if: 

* You're using [VSCode Remote-SSH](https://code.visualstudio.com/docs/remote/ssh) or other IDE integrations
* You need lowest possible terminal latency
* You prefer using your local terminal emulator
* You want to use advanced terminal features like tmux
* You're doing extensive command-line development
* You need to transfer files frequently using scp or rsync

### 2. Create Your Instance

Select your desired GPU configuration based on your computational needs from the [Search tab](https://cloud.vast.ai/create/). For CUDA development, consider: 

* System Requirements: 
  * RAM: Minimum 16GB for development tools
  * Storage: 10GB is usually sufficient
    * CUDA Toolkit core: \~2GB
    * Development files and builds: \~3-4GB
    * Room for source code and dependencies: \~4GB
  * CPU: 4+ cores recommended for compilation
  * Network: 100+ Mbps for remote development

 Rent the GPU of your choice.

### 3. Connecting to Your Instance

Go to [Instances tab](https://cloud.vast.ai/instances/) to see your instance being created. There are multiple ways to connect to your instance:

* If Jupyter launch mode is selected in your template:
  * Click the "OPEN" button or "Jupyter" button on your instance card 
  * Access a full development environment with notebook support
* If you selected SSH launch mode:
  * Click Open Terminal Access button 
  * Copy Direct ssh connect string contents that looks like this "ssh -p 12345 root\@123.456.789.10 -L 8080:localhost:8080"
  * You take the ssh command and execute in your terminal in your [Mac or Linux-based computer or in Powershell](/guides/instances/sshscp)
  * You can use [Powershell or Windows Putty tools](/guides/instances/sshscp) if you have a Windows computer

## Installation

### Setting Up Your Development Environment

1. The base environment includes:
   * CUDA toolkit and development tools
   * Python with common ML libraries
   * Development utilities (gcc, make, etc.)
2. Install additional CUDA dependencies:

```bash theme={null}
apt-get update
apt-get install -y cuda-samples
```

### Configuring Your Workspace

1. Navigate to your workspace:

```bash theme={null}
cd ${WORKSPACE}
```

1. Set up CUDA environment variables:

```bash theme={null}
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc
```

## Troubleshooting

### Common Issues and Solutions

CUDA not found:

* Check if GPU is detectable: `nvidia-smi`

```linux theme={null}
nvidia-smi

```

If output like "No devices were found" shows up,  report the machine after clicking on the wrench icon and rent a different machine.

## Best Practices

### Development Workflow

* Code Organization
  * Keep source files in `${WORKSPACE}`
  * Use version control for code management
  * Maintain separate directories for builds and source
* Performance Optimization
  * Use proper CUDA stream management
  * Optimize memory transfers
  * Profile code using NVIDIA tools

## Advanced Topics

### Custom Environment Setup

Create a provisioning script for custom environment setup:

```bash theme={null}
#!/bin/bash
. /venv/main/bin/activate
pip install additional-packages
wget custom-tools.tar.gz
```

### Remote Development Setup

Configure VS Code or other IDEs for [remote development](https://code.visualstudio.com/docs/remote/ssh):

* Use SSH port forwarding for secure connections
* Configure development tools to use remote CUDA compiler
* Set up source synchronization using Syncthing

## Conclusion

You now have a fully configured CUDA development environment on Vast.ai. This setup provides the flexibility of cloud GPU resources with the convenience of local development.

## Additional Resources

* [NVIDIA CUDA Documentation](https://docs.nvidia.com/cuda/)
* [Vast.ai Documentation](https://vast.ai/docs/)
* [CUDA Sample Projects](https://github.com/NVIDIA/cuda-samples)