feat: file attachment support in chat (images + text/code files)

Text files (.md, .py, .js, .json, etc.): read client-side and injected
into the message body as a fenced code block — works with all backends
with zero model capability requirements.

Images (PNG/JPG/WebP/GIF, max 5 MB): encoded as base64 data URL on the
client and sent as a separate attachment field. Backend formats them as
OpenAI multimodal content (text + image_url) for local_openai backends.
Claude CLI and Gemini CLI see the text message with a "📎 filename.png"
note; image data is never written to session history.

- index.html: 📎 button + hidden file input in mode-select row;
  attachment-row preview area with thumbnail (images) or filename chip
- app.js: _resolveAttachment(), file reader, clearAttachment();
  sendMessage/sendOrchestrate updated to allow no-text sends when a
  file is pending; attachment spread into chat payload for images
- chat.py: Attachment model; attachment field on ChatRequest;
  llm_attachment extracted in _stream_chat and passed to complete()
- llm_client.py: attachment param through complete()/_dispatch()/_local();
  _local() builds multimodal content array for vision calls
- style.css: #attach-btn, #attachment-row, #attachment-preview, thumb

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Scott Idem
2026-05-12 21:46:50 -04:00
parent 50c1997e91
commit 96b3c796c5
5 changed files with 215 additions and 15 deletions

View File

@@ -42,6 +42,12 @@ def _role_model_label(username: str, role: str, actual_backend: str) -> str:
return _backend_label(actual_backend, username, role)
class Attachment(BaseModel):
filename: str
mime_type: str
data: str # base64 data URL for images (e.g. "data:image/png;base64,...")
class ChatRequest(BaseModel):
message: str
session_id: str | None = None
@@ -55,6 +61,7 @@ class ChatRequest(BaseModel):
off_record: bool = False # skip session log (in-memory context preserved)
user: str = "scott"
persona: str = "inara"
attachment: Attachment | None = None # image attachment (text files injected client-side)
class BackendRequest(BaseModel):
@@ -103,6 +110,19 @@ async def _stream_chat(req: ChatRequest):
mode="otr" if req.off_record else "chat",
)
history = load_session(session_id)
# req.message already contains the full user text:
# - text files: client embedded content as a fenced code block
# - images: client added "📎 filename.png" note; image data is in req.attachment
# History always stores text only — base64 image data is never written to disk.
llm_attachment: dict | None = None
if req.attachment and req.attachment.mime_type.startswith("image/"):
llm_attachment = {
"filename": req.attachment.filename,
"mime_type": req.attachment.mime_type,
"data": req.attachment.data,
}
history.append({"role": "user", "content": req.message, "off_record": req.off_record})
task = asyncio.create_task(complete(
@@ -111,6 +131,7 @@ async def _stream_chat(req: ChatRequest):
model=req.model,
role=req.chat_role,
slot=req.slot,
attachment=llm_attachment,
))
try: