Google Launches Agentic Video Understanding: How to Enable It in Gemini API
To enable agentic video understanding in the Gemini API, use a supported Flash model and set "processing": "agentic" on the video input. The model will then navigate the timeline dynamically—requesting relevant frames, audio, or transcript segments—instead of processing the whole video at a fixed frame rate.
Google launched the capability on September 1, 2026 for video uploads and public YouTube URLs. It is designed mainly for long videos and questions that require finding specific moments, counting actions, detecting anomalies, or connecting information spread across a recording.
What Google announced
Traditional Gemini video analysis uses static processing by default. It extracts frames at a fixed rate of one frame per second and places them into the model’s context in a single pass. Agentic video understanding changes that workflow: Gemini decides which sections to inspect, whether it needs visual frames, audio, or the transcript, and when to revisit a moment at a different sampling rate.
In Google’s published benchmarks, the agentic mode used up to 88% fewer tokens, reduced analysis costs by up to 66%, and improved quality by up to 7%. These are maximum benchmark results, not a promise for every video or prompt. Actual token use depends on the question, video content, and how deeply the model needs to inspect the timeline.
The launch details are available in Google’s official announcement, while the current configuration and limits appear in the Gemini video-understanding documentation.
Supported Gemini models
Google initially announced agentic video understanding for Gemini 3.7 Flash, Gemini 3.6 Flash, and Gemini 3.5 Flash-Lite. The current developer guide also lists Gemini 3.8 Flash, which became generally available on September 2, 2026.
- Gemini 3.8 Flash: the newest supported option and the sensible starting point for new applications.
- Gemini 3.7 Flash: the model used for Google’s headline launch comparisons.
- Gemini 3.6 Flash: a supported production model for existing integrations.
- Gemini 3.5 Flash-Lite: suited to cost-sensitive, high-volume workloads where lower latency matters.
Unsupported models fall back to static processing in some Google Cloud interfaces, so do not assume that adding the property guarantees agentic behavior. Verify the response steps as described below.
When agentic mode is better than static processing
Google recommends starting with agentic mode when response quality or token efficiency matters, particularly for long-form video. It is a strong fit for:
- Finding the exact moment a speaker makes a claim in a lecture or keynote.
- Searching a multi-hour recording for a rare object or event.
- Counting repeated actions that occur too quickly for one-frame-per-second sampling.
- Checking brief visual anomalies, state changes, or editing boundaries.
- Combining spoken information with slides, diagrams, captions, and on-screen activity.
- Creating chapter summaries grounded in both the transcript and visuals.
Static mode can still be the better choice for short clips under five minutes, latency-sensitive requests, or tasks that require consistent frame-level coverage across the entire clip. “Agentic” does not automatically mean faster: a complex investigation may involve several internal inspection steps.
How to enable agentic video understanding in Python
1. Install the Gemini SDK and set your key
Install Google’s current Python SDK and store your API key in the GEMINI_API_KEY environment variable. Do not hard-code a production key in source code or commit it to a repository.
pip install -U google-genai
export GEMINI_API_KEY="your-api-key"
2. Upload a long video through the Files API
Google recommends the Files API for videos larger than 100 MB, recordings longer than about 10 minutes, or files you plan to query more than once. Upload the file and wait until its processing state becomes active.
import time
from google import genai
client = genai.Client()
video = client.files.upload(file="lecture.mp4")
while video.state.name == "PROCESSING":
time.sleep(2)
video = client.files.get(name=video.name)
if video.state.name == "FAILED":
raise RuntimeError("Video processing failed")
3. Set processing to agentic
Add the processing property to the video object in the interaction. This is the key change from a normal video request.
interaction = client.interactions.create(
model="gemini-3.8-flash",
input=[
{
"type": "video",
"uri": video.uri,
"mime_type": video.mime_type,
"processing": "agentic"
},
{
"type": "text",
"text": (
"Identify the three most important claims in this lecture. "
"For each claim, give the timestamp and describe the slide "
"or visual evidence shown at that moment."
)
}
]
)
print(interaction.output_text)
For a long or difficult request, Google recommends streaming or background execution to reduce the risk of connection and authentication timeouts.
How to analyze a public YouTube video
You can replace the uploaded file URI with a public YouTube URL. The URL feature is currently in preview, and only public videos are supported—not private or unlisted uploads.
from google import genai
client = genai.Client()
interaction = client.interactions.create(
model="gemini-3.8-flash",
input=[
{
"type": "video",
"uri": "https://www.youtube.com/watch?v=VIDEO_ID",
"processing": "agentic"
},
{
"type": "text",
"text": (
"Find every product announcement in this video. "
"Return a table with the timestamp, product name, "
"announced feature, and the visual shown on screen."
)
}
]
)
print(interaction.output_text)
Google’s current limits allow up to eight hours of YouTube video per day on the free tier. The paid tier has no daily limit based specifically on YouTube video length, although normal billing, quotas, model limits, and abuse protections still apply. Preview pricing and limits can change.
How to verify that agentic mode actually ran
Do not rely only on the final answer. Inspect interaction.steps. A successful agentic run contains processing_call and processing_result entries, showing that Gemini requested and loaded targeted portions of the video.
for step in interaction.steps:
print(step.type)
You should see a sequence that can include:
thought— a summary of the current analysis step, when enabled.processing_call— a request to inspect part of the video or transcript.processing_result— the content returned for that internal request.model_output— the final response.
If those processing steps are absent, confirm the model ID, SDK version, video object, and exact placement of "processing": "agentic".
Write prompts that make dynamic video search useful
Ask for evidence and timestamps
“Summarize this video” is broad and may produce a reasonable overview, but it does not force precise inspection. Ask for timestamps, visible evidence, speaker names, quoted ideas in paraphrase, or specific state changes.
A stronger prompt is:
Find the section where the presenter compares the two battery designs. Give the start and end timestamps, summarize the conclusion, and describe the chart shown on screen. If the chart and spoken claim disagree, flag the difference.
Define what counts
Counting is more reliable when the unit is explicit. Instead of “count the repetitions,” specify whether partial movements count, how to treat cuts or replays, and the time range to inspect.
Separate retrieval from judgment
For high-stakes review, first ask Gemini to locate relevant moments and describe the evidence. Then evaluate those moments with a second prompt or human reviewer. This reduces the risk of mixing retrieval errors with interpretation errors.
Our guide on fact-checking AI answers and citations provides a practical verification checklist that also applies to video-derived claims.
File sizes and input methods
- Files API: up to 20 GB per file on paid access and 2 GB on free access; recommended for large, long, or reusable videos.
- Cloud Storage registration: up to 2 GB per file and useful for persistent Google Cloud assets.
- Inline data: best for one-off clips under 100 MB and shorter than about one minute.
- YouTube URL: convenient for public videos, but currently a preview feature.
The Files API reference is generally the safest choice for production workflows because you can process a file once and reuse its URI for multiple questions.
Cost and performance considerations
Agentic video understanding uses standard Gemini API token pricing with no separate feature fee. Token usage is variable because the model loads only the material it decides is relevant. A narrow question about one moment may be substantially cheaper than static ingestion of a long recording, while a broad forensic request may trigger deeper sampling.
Track input tokens, output tokens, latency, and answer quality for both modes on a representative test set. Do not choose a mode from Google’s aggregate benchmark alone. The official Gemini API pricing page, updated September 4, 2026, notes that dynamic sampling depth can exceed one frame per second for visually detailed segments.
Privacy and accuracy checklist
- Confirm that you have permission to upload and process the recording.
- Remove confidential segments that the model does not need.
- Keep API keys on the server and restrict their scope.
- Ask for timestamps and inspect the original footage before acting on a claim.
- Test fast action, small text, multiple speakers, and noisy audio separately.
- Log the model ID, processing mode, prompt, and response steps for reproducibility.
For account-level privacy choices across major assistants, see our guide to stopping AI services from using chats for model improvement. API and enterprise data terms may differ from consumer-chat settings, so review the agreement that applies to your project.
Video understanding is not video generation
This feature analyzes existing footage; it does not create or extend a cinematic clip. If your goal is to continue a generated scene, use the workflow in our guide to extending AI videos in Google Flow with Gemini Omni 1.1 Flash.
Bottom line
Agentic video understanding is a meaningful Gemini API update for long recordings and targeted visual questions. Enable it by adding "processing": "agentic" to a video input on a supported Flash model, then confirm the response contains processing calls and results. Start with Gemini 3.8 Flash for a new integration, benchmark it against static mode on your own footage, and always verify important answers against the cited timestamps.
