Gopher is the cross-app concierge. It knows what is in your inbox, what was discussed in meetings, what tasks are pending. What it could not answer was the simplest kind of question: what were you actually doing on a given day? Email and meeting summaries tell you what you were working on. Photos tell you where you were and what it looked like. The gap was obvious once the rest of the context was in place.
Immich is the self-hosted photo library running on the NAS. It has CLIP-based semantic search, which means you can query for concepts rather than filenames. "Beach", "birthday", "snow" all return relevant results without any tagging or manual organisation. The integration connects Gopher to Immich through four tools: semantic search with optional date bounds and media type filter, on-this-day memories, album listing, and album asset retrieval.
The Proxy Problem
The browser cannot reach Immich directly. The NAS is on the internal network; the browser is not. Every image and video the LLM references in its response has to travel through Gopher before it reaches the client. That means two proxy endpoints: GET /api/immich/thumb/{asset_id} for thumbnails, and GET /api/immich/video/{asset_id} for video playback. Both are auth-required and validate the asset ID format before touching the upstream service.
The video endpoint is more involved. Browsers request video in chunks using HTTP Range headers so they can seek without buffering the entire file first. The proxy forwards the Range header to Immich, streams the response back in 64KB chunks rather than loading it into memory, and passes through Content-Range, Content-Length, and Accept-Ranges so the browser's native video player works correctly. The CSP header also needed media-src 'self' added -- without it, the browser blocks the video element from loading at all.
Telling the Frontend What It Is Receiving
Photos and videos both render as thumbnails in the chat. Clicking a thumbnail should open the full image for a photo or start playback for a video. The frontend needs to know which is which before it opens the lightbox.
The obvious approach is to have the LLM signal the type in its markdown output. Use a prefix in the alt text:  for videos,  for images. This works until it does not. Local LLMs follow instructions inconsistently. The prefix would be present on some responses and absent on others, and debugging which model behaviour caused a broken lightbox is not a tractable problem.
The more reliable approach encodes the type in the URL itself. The _summarise_asset function returns thumb_url as /api/immich/thumb/{id}?t=VIDEO or /api/immich/thumb/{id}?t=IMAGE. The LLM is not parsing or generating the type at all -- it is copying the URL verbatim from the tool result. The renderMarkdown function reads [?&]t=VIDEO from the URL to set data-asset-type="video" on the thumbnail element. The proxy ignores the t parameter and only passes size to Immich. The type information travels with the asset reference through every transformation without requiring the LLM to do anything special.
The Hallucination Problem
The first version worked well for a single query. Ask for photos from last summer, get photos from last summer, click to open in the lightbox. Then comes the follow-up: "show me another one." The server returned a 400. Immich said the asset ID was not a UUID.
The cause: tool results are excluded from conversation history when rebuilding context for subsequent requests. The reasoning behind this is sound -- old email summaries resurfacing in unrelated queries is actively harmful. But it means the LLM cannot see the list of asset IDs it returned in the previous turn when answering a follow-up. Rather than calling the search tool again, it tried to derive a plausible-looking UUID from what it remembered of the previous response. The result was an ID that passed the local format check but did not exist in the Immich database.
The fix has two parts. The system prompt now explicitly instructs the LLM to call the search tool again for any "another" or "more" request rather than referencing previous results. And the thumbnail element gets onerror="this.style.display='none'", so a hallucinated ID produces a silent gap in the response rather than a broken image icon. The underlying tension -- tool results excluded for privacy, but needed for continuity -- is not fully resolved. It is a known limitation.
Per-User Settings
The initial implementation used a global env var: IMMICH_URL and IMMICH_API_KEY set at deploy time, shared across all users. This works when there is one user. It does not work when invited users might have their own Immich instances, and it is the wrong abstraction in any case. An integration credential is a per-user setting, not a server configuration item.
The credentials now live in a user_settings table in PostgreSQL, Fernet-encrypted alongside the other sensitive fields. A settings modal accessible from the header lets users enter their Immich URL and API key, test the connection before saving, and disconnect when needed. The global singleton is gone. Every request that touches Immich fetches the requesting user's credentials from the database. The Immich tools do not appear in the LLM's tool list for users who have not configured the integration.
The Settings UI also meant thinking about what "settings" means in this context. Right now it is one integration. Eventually it will be the surface where users manage all of their integrations -- Immich today, other self-hosted services as they are added. The modal is deliberately minimal for now.
Chat UX: Cancel and Rerun
Two small additions to the chat interface that make a noticeable difference. The send button becomes a stop button while a request is in flight, using AbortController to cancel the fetch. The server-side Ollama call keeps running -- Python cannot cancel it mid-flight -- but the UI unblocks immediately. A rerun button appears after each response and resends the last message without starting a new conversation, which is useful when the LLM decides to respond with text when you wanted a tool call, or when you just want a different take on the same query.