Sam3 Video Rle API Documentation

Sam3 Video Rle API Documentation

Playground

SAM 3 Video RLE is a unified foundation model for prompt-based segmentation in video. Track and segment objects across frames using text, points, or boxes, returning RLE encoded masks for efficient processing. Ready-to-use REST inference API, best performance, no coldstarts, affordable pricing.

Features

SAM3 Video Segmentation RLE segments and tracks target objects across video frames, then returns both a preview video URL and RLE mask data in JSON format. Use text prompts, point prompts, box prompts, or a combination of them to generate frame-level masks for programmatic video workflows.

Why Choose This?

• Video object segmentation
  Segment people, objects, animals, clothing, vehicles, or other visible targets across video frames.

• Frame-level object tracking
  Track selected subjects consistently through the video.

• RLE mask output
  Receive compact Run-Length Encoded mask data for downstream processing.

• Multiple prompt types
  Use text prompts, point prompts, box prompts, or combined prompt guidance for more accurate targeting.

• Multi-object targeting
  Track multiple objects by listing them in the prompt, such as person, car, dog.

• Optional mask visualization
  Use apply_mask to generate a visual preview of the segmentation result.

Parameters

How to Use

1. Upload a video — Provide the source video you want to segment.
2. Write a target prompt — Describe the object or objects to track, such as the man, red car, or person, cloth.
3. Add point or box prompts optional — Use coordinates or boxes when the target is ambiguous.
4. Configure mask visualization optional — Enable apply_mask when you want a preview video with the segmentation mask applied.
5. Submit — Generate the video preview and RLE mask JSON.

Output

The response returns both:

• A video URL — A preview or mask-applied video output.
• RLE JSON data — Frame-level segmentation masks encoded in RLE format.

RLE means Run-Length Encoding. Instead of storing every mask pixel one by one, it stores continuous runs of pixels, making mask data smaller and easier to pass into downstream computer-vision or editing pipelines.

The RLE JSON can be decoded with standard mask-processing tools when you need binary masks for each frame.

Pricing

Pricing is based on input video duration.

Billed duration is rounded up to the next whole second, with a minimum billed duration of 3 seconds and a maximum billed duration of 600 seconds. Pricing is charged at $0.05 per 5 seconds.

prompt, point_prompts, box_prompts, and apply_mask do not add separate charges.

Best Use Cases

• Video annotation — Generate frame-level segmentation masks for labeling and dataset workflows.
• Object tracking pipelines — Track subjects or objects across frames for automated processing.
• Video editing automation — Use RLE masks for programmatic object isolation, removal, or replacement.
• Computer vision workflows — Feed compact mask data into downstream CV systems.
• VFX and compositing — Generate rotoscoping-style masks for post-production pipelines.
• Mask preview workflows — Use the returned video URL to quickly inspect segmentation quality.

Pro Tips

• Use short, concrete prompts such as person, dog, car, or shirt.
• Use comma-separated prompts for multiple targets, such as person, backpack, bicycle.
• Use point prompts when text alone is ambiguous.
• Use box prompts when you need to constrain segmentation to a specific region.
• Enable apply_mask when you want a visual preview of the segmentation result.
• Use the RLE JSON when you need programmatic access to frame-by-frame masks.
• Use stable footage with clear subject separation for better tracking consistency.

Notes

• video and prompt are required.
• Output includes both a video URL and RLE mask JSON.
• RLE output is structured mask data, not a standalone video file.
• Use standard RLE-compatible tooling to decode masks for downstream processing.

Related Models

• SAM3 Video — Segment video targets and return direct video output.
• SAM3 Image RLE — Segment image targets and return RLE mask data.
• SAM3 Image — Segment image targets and return image output.

Authentication

For authentication details, please refer to the Authentication Guide.

API Endpoints

Submit Task & Query Result

set -euo pipefail

export WAVESPEED_API_KEY="your-api-key"

REQUEST_BODY=$(cat <<'JSON'
{
  "prompt": "A cinematic ocean wave at sunrise, highly detailed",
  "video": "https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4",
  "apply_mask": true
}
JSON
)

# 1. Submit the prediction once. Never retry this request automatically:
# a retry after an unclear failure can create a second, billed task.
CHECK='The task may already exist. Check History or the predictions list before submitting again.'
if ! SUBMIT_RESPONSE=$(curl --silent --show-error --fail-with-body \
  --connect-timeout 10 --max-time 60 \
  -X POST "https://api.wavespeed.ai/api/v3/wavespeed-ai/sam3-video-rle" \
  -H "Authorization: Bearer ${WAVESPEED_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "${REQUEST_BODY}"); then
  printf 'Submit failed. %s\n' "${CHECK}" >&2
  exit 1
fi

TASK=$(printf '%s' "${SUBMIT_RESPONSE}" | jq 'if type == "object" and has("data") then .data else . end')
PREDICTION_ID=$(printf '%s' "${TASK}" | jq -r '.id // empty')
if [ -z "${PREDICTION_ID}" ]; then
  printf 'Submission response did not contain a prediction id. %s\n' "${CHECK}" >&2
  exit 1
fi
RESULT_URL="https://api.wavespeed.ai/api/v3/predictions/${PREDICTION_ID}/result"

# 2. Poll until the prediction reaches a final status.
BODY_FILE=$(mktemp)
trap 'rm -f "${BODY_FILE}"' EXIT
while true; do
  # Transient errors (429, 5xx) are retried, honoring Retry-After.
  HTTP_CODE=$(curl --silent --show-error --fail-with-body \
    --connect-timeout 10 --max-time 60 --retry 5 --retry-delay 2 \
    --output "${BODY_FILE}" --write-out '%{http_code}' \
    "${RESULT_URL}" \
    -H "Authorization: Bearer ${WAVESPEED_API_KEY}" || true)
  case "${HTTP_CODE}" in
    2??) ;;
    000|429|5??)  # dropped connection or temporary error: keep polling
      printf 'Task %s: HTTP %s, polling again in 10s\n' "${PREDICTION_ID}" "${HTTP_CODE}" >&2
      sleep 10; continue ;;
    *) printf 'Task %s: HTTP %s\n' "${PREDICTION_ID}" "${HTTP_CODE}" >&2
      cat "${BODY_FILE}" >&2; exit 1 ;;
  esac
  RESULT=$(jq 'if type == "object" and has("data") then .data else . end' "${BODY_FILE}")
  STATUS=$(printf '%s' "${RESULT}" | jq -r '.status // empty')

  case "${STATUS}" in
    completed) printf '%s\n' "${RESULT}" | jq '.outputs'; break ;;
    failed|cancelled|timeout|deleted)
      printf 'Task %s ended with status %s\n' "${PREDICTION_ID}" "${STATUS}" >&2
      printf '%s\n' "${RESULT}" | jq . >&2; exit 1 ;;
    *) sleep 2 ;;
  esac
done

Parameters

Task Submission Parameters

Request Parameters

Response Parameters

Result Request Parameters

Result Response Parameters