Guides
Vision input
Two ways to send images — URL or data URL (base64), plus size and format constraints.
Multimodal models accept image input. In the OpenAI protocol an image is an image_url content block.
Public URL
resp = client.chat.completions.create(
model="gpt-5.4",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "What is in this image?"},
{"type": "image_url", "image_url": {"url": "https://example.com/cat.jpg"}},
],
}],
)base64 (data URL)
import base64, pathlib
data = base64.b64encode(pathlib.Path("cat.jpg").read_bytes()).decode()
resp = client.chat.completions.create(
model="gpt-5.4",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Describe this image"},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{data}"}},
],
}],
)Things to watch
- Format:
image/jpeg,image/pngandimage/webpare safest; SVG is usually rejected - Size: base64 inflates the body by roughly a third — downscale so the long edge is 1024–2048px
- Reachability: the upstream vendor must be able to fetch your URL; private addresses and expired presigned links fail
- Capability: not every model accepts images; unsupported models return an error instead of ignoring the image
Anthropic and Gemini shapes
With the Anthropic protocol an image is {"type":"image","source":{"type":"base64",...}}; with Gemini
it is inline_data / file_data. The gateway interprets the inbound protocol and maps it — see
Protocol bridging.