Quickstart

Deploy your first Serverless endpoint in 5 minutes using a ready-to-use template.

This quickstart gets you running a Serverless endpoint on Runpod in minutes, using a ready-to-use template to deploy a language model and send a test request.

Requirements

• A Runpod account with available credits.
• A Runpod API key.

Step 1: Set up your environment

Choose your preferred method for interacting with Runpod. If using the CLI or REST API, you’ll need to configure your API key.

Runpod CLI

Install and configure the Runpod CLI.

macOS/Linux:

# Install runpodctl
curl -fsSL https://install.runpod.io | bash

# Configure with your API key
runpodctl doctor

Windows:

# Install using PowerShell
iwr -useb https://install.runpod.io/windows | iex

# Configure with your API key
runpodctl doctor

Verify the installation:

runpodctl version

REST API

Export your Runpod API key as an environment variable:

export RUNPOD_API_KEY="your_api_key_here"

Web

No setup required. Log in to the Runpod console to get started.

Step 2: Deploy an endpoint

Deploy a vLLM worker with a small, fast language model.

Runpod CLI

First, create a Serverless template with the vLLM worker image:

runpodctl template create \
  --name "vllm-qwen" \
  --image "runpod/worker-v1-vllm:stable-cuda12.1.0" \
  --env '{"MODEL_NAME": "Qwen/Qwen2.5-0.5B-Instruct"}' \
  --serverless

Note the template ID from the output. Then create an endpoint using that template:

runpodctl serverless create \
  --name "my-first-endpoint" \
  --template-id YOUR_TEMPLATE_ID \
  --gpu-id "NVIDIA GeForce RTX 4090" \
  --workers-min 0 \
  --workers-max 3

The output includes your endpoint ID:

Endpoint created successfully
ID: abc123xyz
Name: my-first-endpoint

REST API

First, create a template using the vLLM worker image:

curl --request POST \
  --url https://rest.runpod.io/v1/templates \
  --header "Authorization: Bearer $RUNPOD_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "name": "vllm-qwen",
    "imageName": "runpod/worker-v1-vllm:stable-cuda12.1.0",
    "isServerless": true,
    "env": {
      "MODEL_NAME": "Qwen/Qwen2.5-0.5B-Instruct"
    }
  }'

Note the id from the response. Then create an endpoint using that template:

curl --request POST \
  --url https://rest.runpod.io/v1/endpoints \
  --header "Authorization: Bearer $RUNPOD_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "name": "my-first-endpoint",
    "templateId": "YOUR_TEMPLATE_ID",
    "gpuTypeIds": ["NVIDIA GeForce RTX 4090", "NVIDIA L4", "NVIDIA RTX A4000"],
    "workersMin": 0,
    "workersMax": 3,
    "idleTimeout": 5
  }'

The response includes your endpoint ID:

{
  "id": "abc123xyz",
  "name": "my-first-endpoint",
  ...
}

Web

1. Go to the Serverless section and click New Endpoint.
2. Under The Hub, click vLLM.
3. Click Deploy vX.X.X.
4. In the Model field, enter: Qwen/Qwen2.5-0.5B-Instruct
5. Click Next then Create Endpoint.
6. Once deployed, note your Endpoint ID from the endpoint details page—you’ll need it for API requests.

Your endpoint will begin initializing. This takes 1-2 minutes while Runpod provisions resources and loads the model.

Step 3: Send a request

Once your endpoint shows Ready status, send a test request. If you haven’t already, export your API key in your terminal:

export RUNPOD_API_KEY="your_api_key_here"

cURL

Run this command in your terminal, replacing YOUR_ENDPOINT_ID with your actual endpoint ID:

curl --request POST \
  --url "https://api.runpod.ai/v2/YOUR_ENDPOINT_ID/runsync" \
  --header "Authorization: Bearer $RUNPOD_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "input": {
      "prompt": "What is the capital of France?",
      "max_tokens": 100
    }
  }'

Python

Create a file called test_endpoint.py and paste the following code:

test_endpoint.py

import requests
import os

ENDPOINT_ID = "YOUR_ENDPOINT_ID"  # Replace with your endpoint ID
API_KEY = os.environ.get("RUNPOD_API_KEY")

if not API_KEY:
    raise ValueError("RUNPOD_API_KEY environment variable not set")

response = requests.post(
    f"https://api.runpod.ai/v2/{ENDPOINT_ID}/runsync",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "input": {
            "prompt": "What is the capital of France?",
            "max_tokens": 100
        }
    }
)

print(response.json())

Install dependencies and run the script:

pip install requests
python test_endpoint.py

You should receive a response like this:

{
  "id": "sync-abc123-xyz",
  "status": "COMPLETED",
  "output": {
    "text": "The capital of France is Paris.",
    ...
  }
}

The first request may take 30-60 seconds as the worker loads the model into GPU memory. Subsequent requests will complete in just a few seconds until the worker scales down due to inactivity.

Step 4: Clean up

To avoid ongoing charges, delete your endpoint when you’re done testing.

Runpod CLI

List your endpoints to find the ID:

runpodctl serverless list

Delete the endpoint:

runpodctl serverless delete YOUR_ENDPOINT_ID

Optionally, delete the template you created:

runpodctl template delete YOUR_TEMPLATE_ID

REST API

curl --request DELETE \
  --url "https://rest.runpod.io/v1/endpoints/YOUR_ENDPOINT_ID" \
  --header "Authorization: Bearer $RUNPOD_API_KEY"

Web

1. Go to the Serverless section.
2. Click the three dots on your endpoint and select Delete Endpoint.
3. Type the endpoint name to confirm.

You’ve successfully deployed and tested your first Serverless endpoint.

Next steps

Build a custom worker

Create your own handler function and Docker image.

Send requests

Learn about sync, async, and streaming requests.

Endpoint settings

Configure scaling, timeouts, and GPU selection.

Configure vLLM

Customize your vLLM deployment for different models.