> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aireserve.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Video input (understanding)

> Send a video to a model and ask questions about it — inline bytes up to 20 MB, or upload once and reference by URI up to 1 GB.

Send a video **to** a model and ask questions about it — summarize a
recording, extract action items from a meeting, describe what happens in a clip.
Bedrock-style `video` content blocks are supported in user messages, mirroring
the `bytes` | `s3Location` pattern of Bedrock Converse's
video source. This is the input-side counterpart of
[Video generation](/capabilities/video-generation), which *creates* video from a
text prompt — don't confuse the two.

## Model support

Video input is available on the **Gemini family only**. Sending a video
block to any other model returns an HTTP **400** before the request reaches
the provider (see [Limits & errors](#limits-&-errors)).

| Model                    | Video input   |
| ------------------------ | ------------- |
| `gemini-2.5-flash`       | yes           |
| `gemini-2.5-pro`         | yes           |
| `gemini-3-flash-preview` | yes           |
| Every other model        | no — HTTP 400 |

## Inline video (≤ 20 MB)

For small files, hand the SDK **raw bytes** — it base64-encodes them into
the request for you. Base64 inflates bytes by \~33% against the gateway's 30 MB
request cap, so inline video is capped at **20 MB raw** per request
(larger inline payloads are rejected with HTTP 413 — use the upload flow below instead).
All five SDKs support video input as of **SDK v0.3.0**.

<CodeGroup>
  ```python Python theme={"dark"}
  with open("demo.mp4", "rb") as f:
      video_bytes = f.read()

  response = client.converse(
      modelId="gemini-2.5-flash",
      messages=[{
          "role": "user",
          "content": [
              {"text": "Summarize what happens in this video."},
              {"video": {"format": "mp4", "source": {"bytes": video_bytes}}},
          ],
      }],
  )
  print(response["output"]["message"]["content"][0]["text"])
  ```

  ```typescript TypeScript theme={"dark"}
  import { readFile } from "node:fs/promises";

  const videoBytes = new Uint8Array(await readFile("demo.mp4"));

  const response = await client.send(
    new ConverseCommand({
      modelId: "gemini-2.5-flash",
      messages: [
        {
          role: "user",
          content: [
            { text: "Summarize what happens in this video." },
            { video: { format: "mp4", source: { bytes: videoBytes } } },
          ],
        },
      ],
    })
  );
  ```

  ```go Go theme={"dark"}
  videoBytes, err := os.ReadFile("demo.mp4")
  if err != nil {
      log.Fatal(err)
  }

  resp, err := client.Converse(ctx, &audacityruntime.ConverseInput{
      ModelId: audacity.String("gemini-2.5-flash"),
      Messages: []types.Message{{
          Role: types.ConversationRoleUser,
          Content: []types.ContentBlock{
              &types.ContentBlockMemberText{Value: "What happens in this video?"},
              &types.ContentBlockMemberVideo{Value: types.VideoBlock{
                  Format: types.VideoFormatMp4,
                  Source: &types.VideoSourceMemberBytes{Value: videoBytes},
              }},
          },
      }},
  })
  ```

  ```java Java theme={"dark"}
  byte[] videoBytes = java.nio.file.Files.readAllBytes(java.nio.file.Path.of("demo.mp4"));

  ConverseResponse resp = client.converse(request -> request
      .modelId("gemini-2.5-flash")
      .messages(Message.builder()
          .role(ConversationRole.USER)
          .content(
              ContentBlock.fromText("What happens in this video?"),
              ContentBlock.fromVideo(VideoBlock.builder()
                  .format(VideoFormat.MP4)
                  .source(VideoSource.fromBytes(videoBytes))
                  .build()))
          .build()));
  ```

  ```rust Rust theme={"dark"}
  use audacity_sdk::{ContentBlock, VideoBlock, VideoFormat, VideoSource};

  let video_bytes = std::fs::read("demo.mp4")?;

  let response = client.converse()
      .model_id("gemini-2.5-flash")
      .messages(
          Message::builder()
              .role(ConversationRole::User)
              .content(ContentBlock::Text("What happens in this clip?".into()))
              .content(ContentBlock::Video(VideoBlock {
                  format: VideoFormat::Mp4,
                  source: VideoSource::Bytes(video_bytes),
              }))
              .build()?
      )
      .send()
      .await?;
  ```
</CodeGroup>

## Large videos: upload, then reference by URI

For files over 20 MB (up to **1 GB**), upload once and reference
the returned `audacity://files/…` URI in as many requests as you like. The
flow has three steps:

1. `POST /v1/files` with `content_type` and `size_bytes` — returns a `file_id`, a signed `upload_url` (valid \~15 minutes), the `uri` to reference in chat requests, and `expires_at`.
2. Upload the bytes to `upload_url` over a **resumable session** (Google Cloud Storage protocol: one POST opens the session, then PUT the bytes).
3. Reference the video with `{"video": {"format": …, "source": {"uri": …}}}` in a `Converse` call.

The file-create step from curl:

```bash theme={"dark"}
# 1) Create an upload slot
curl -s https://api.aireserve.com/v1/files \
  -H "Authorization: Bearer $AUDACITY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"content_type": "video/mp4", "size_bytes": 52428800}'
```

```json theme={"dark"}
{
  "file_id": "3f8a1e2c-9d4b-4c6a-b7e1-2a5c8d9f0e13",
  "upload_url": "https://storage.googleapis.com/…&X-Goog-Signature=…",
  "uri": "audacity://files/3f8a1e2c-9d4b-4c6a-b7e1-2a5c8d9f0e13",
  "expires_at": "2026-07-09T05:20:00.000Z"
}
```

```bash theme={"dark"}
# 2) Open the resumable session (the session URI comes back in the Location header)
SESSION_URI=$(curl -si -X POST "$UPLOAD_URL" \
  -H "x-goog-resumable: start" \
  -H "Content-Type: video/mp4" \
  | awk 'tolower($1)=="location:" {print $2}' | tr -d "\r")

# 3) Upload the bytes (a single PUT works; the SDK helpers chunk + auto-resume)
curl -X PUT "$SESSION_URI" --data-binary @keynote.mp4
```

`GET /v1/files/{file_id}` reports upload status — `"pending"`
before the bytes land, then `"uploaded"` with `size_bytes` and
`content_type`.

The SDK helpers (`client.files.upload` in Python/TypeScript,
`UploadFile`/`uploadFile` in Go/Java, `upload_file()` in
Rust) run this whole flow for you and stream the file in 8 MB chunks, automatically
resuming from the last confirmed byte after a network drop. The full loop in Python:

```python theme={"dark"}
from audacity import Audacity

client = Audacity()   # AUDACITY_API_KEY from the environment

# Upload once — accepts raw bytes or a file path; resumable under the hood
upload = client.files.upload("keynote.mp4", content_type="video/mp4")
# {"file_id": …, "upload_url": …, "uri": "audacity://files/…", "expires_at": …}

# Reference the uploaded video by URI — reusable across requests for ~24 h
response = client.converse(
    modelId="gemini-2.5-pro",
    mediaResolution="low",   # optional cost knob — see below
    messages=[{
        "role": "user",
        "content": [
            {"text": "Summarize this keynote and list the action items."},
            {"video": {"format": "mp4", "source": {"uri": upload["uri"]}}},
        ],
    }],
)
print(response["output"]["message"]["content"][0]["text"])
```

Uploaded files are **transient inference inputs**: they expire after
**\~24 hours** and are scoped to your workspace's API keys — a URI leaked to
another workspace resolves against that workspace's namespace and simply does not exist.
Re-referencing the same URI across conversation turns is free; the gateway caches the
provider-side staging, so a large video is not re-transferred on every request.

<Info>
  **Calling without an SDK?** On the raw
  [OpenAI-protocol endpoint](/connect/openai-sdk), video rides the `file`
  content part — inline as a data URL in `file_data`, or by reference in
  `file_id` — with the MIME type in `format`:

  ```json theme={"dark"}
  {"type": "file", "file": {"file_data": "data:video/mp4;base64,…", "format": "video/mp4"}}
  {"type": "file", "file": {"file_id": "audacity://files/3f8a1e2c-…", "format": "video/mp4"}}
  ```

  A per-part `"detail"` field and a request-level
  `"media_resolution"` field control media resolution (next section).
</Info>

## Media resolution (video token cost)

Video is tokenized frame by frame on Gemini, so long clips get expensive fast. The
request-level `mediaResolution` option (wire name:
`media_resolution`) controls how densely video is sampled —
`"low"` processes video at roughly **4× fewer tokens**. Non-Gemini
models ignore the field; when unset, the model's default applies.

| Value        | Guidance                                                                                                  |
| ------------ | --------------------------------------------------------------------------------------------------------- |
| `low`        | \~4× fewer video tokens. The right default for summarization, transcription-style tasks, and long footage |
| `medium`     | Balanced sampling for general question-answering about a clip                                             |
| `high`       | Finer sampling — reading small on-screen text, UI walkthroughs, fine visual detail                        |
| `ultra_high` | Maximum fidelity at maximum token cost — dense charts or frame-critical analysis                          |

Raw-protocol callers can also set `detail` per content part; an explicit
per-part value wins over the request-level field. On **Gemini 2.5** models
the highest resolution across all parts applies to the whole request; true per-part
granularity starts with **Gemini 3**.

## Supported formats

| SDK `format` | MIME type (`content_type` for uploads) |
| ------------ | -------------------------------------- |
| `mp4`        | `video/mp4`                            |
| `mov`        | `video/mov`                            |
| `mkv`        | `video/x-matroska`                     |
| `webm`       | `video/webm`                           |
| `flv`        | `video/x-flv`                          |
| `mpeg`       | `video/mpeg`                           |
| `mpg`        | `video/mpg`                            |
| `wmv`        | `video/wmv`                            |
| `three_gp`   | `video/3gpp`                           |

## Limits & errors

* **Inline cap** — 20 MB of raw video per request (all video parts combined). Exceeding it returns **413** with a message pointing to the upload flow.
* **Upload cap** — 1 GB per file. `POST /v1/files` rejects a larger `size_bytes`, or an unsupported `content_type`, with **400**.
* **Upload URL expiry** — the signed `upload_url` is valid \~15 minutes; create a fresh slot if it lapses.
* **File expiry** — uploaded files auto-delete after \~24 hours. Referencing an expired or unknown URI fails with **400**: `video file "audacity://files/…" not found or expired; upload via POST /v1/files and retry`.
* **Non-video model** — sending video to anything outside the Gemini family returns **400** before the provider is contacted. The SDKs raise `ValidationException`; the raw response looks like:

```json theme={"dark"}
{
  "success": false,
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "model \"gpt-5.4-mini\" does not support video input; video is supported on: gemini-2.5-flash, gemini-2.5-pro, gemini-3-flash-preview",
    "request_id": "…"
  }
}
```

* Video blocks are valid in **user** messages only, matching image input.

<Note>
  **boto3 / AWS SDK path.** The [Bedrock-compatible
  endpoint](/connect/aws-sdk) does not accept `video` content blocks — it returns a
  `ValidationException` listing its supported block types. Use one of the
  AI Reserve SDKs or the raw OpenAI-protocol wire format above for video.
</Note>