Commit Graph
8 Commits
Author SHA1 Message Date
stumpylog 179307d57a Fix: stream AI chat natively async so a disconnect cancels the LLM request
The chat answer was a sync generator wrapped for ASGI, pulling each chunk
through sync_to_async(next). A client disconnect cancelled the response
task, but the worker thread waiting on the LLM could not be interrupted,
so the upstream request kept generating the whole answer for nobody.

Make stream_chat_with_documents an async generator. All the blocking setup
(ORM queries, the index read lock and its thread-bound sqlite connection,
the query embedding, building the LLM client) moves into one sync
_prepare_chat called once through sync_to_async, which also hands the DB
connection back before the answer streams. The answer then streams through
llama-index's asynthesize over the pinned async httpx clients, so a
disconnect cancels the LLM request directly. The view returns the async
generator as is, replacing the sync-to-async wrapper.

The async pinned transport resolved hostnames with a blocking getaddrinfo
on the event loop, which would stall the whole worker on slow DNS now that
chat uses it in the web server. Resolve in a thread instead, and add the
async counterparts of the existing transport tests.
2026-11-17 10:04:56 -08:00
stumpylog a690885a8a Fix: stream AI chat answers under ASGI 2026-11-17 07:42:51 -08:00
Trenton H 6f99ad9469 Chore: Improve idle memory via more lazy imports (#14599)
* Chore: Import dateparser lazily so processes that never parse dates skip it

* Chore: Import turbohtml lazily in the mail parser

* Chore: Import lxml lazily in the SVG validator

* Chore: Import paperless_ai.indexing lazily in document tasks

* Chore: Import paperless_ai helpers lazily in views

* Chore: Read the OCRmyPDF sRGB profile on first use instead of at import

* Chore: Import pikepdf and img2pdf lazily so startup does not load them

* Chore: Reuse the initialized Django ASGI application for HTTP routing
2026-10-30 22:25:42 +00:00
Trenton H f502cd5e34 Chore: Add shared helpers for granting test permissions and use them (#14200) 2026-09-21 10:57:14 -07:00
Trenton H 99ce6b5db3 Chore: Share the API client fixtures and build test users cheaply (#14199) 2026-09-21 10:57:14 -07:00
shamoon d53c9070ba Fix: correct text/stream compression workaround (#14064) 2026-09-11 15:46:08 +00:00
Trenton Handshamoon 351892bbab Fix: skip vector store document id filter for unrestricted chat users (#13937)
* Fix: skip vector store document id filter for unrestricted chat users

ChatStreamingView built an IN filter from every permitted document id
for the "chat over all documents" case, which exceeds the vector
store's SQLite bound-parameter safety limit on installs with more
than ~32700 documents, silently returning no context. For a user who
can see every document (an active superuser), that filter never
narrows anything, so skip it and let the retriever search the whole
index instead.

* Minor improvements from a Claude review

* When a user is unrestricted chatting, still exclude trashed documents using a 'NOT IN' SQL statement.  Wire that up where we need it

* Update src/paperless_ai/chat.py

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-09-02 18:08:54 +00:00
Trenton HandClaude Sonnet 4.6 bb860a5834 Fix: Improvements for security around the AI (#12895)
* Fix: Validate and limit chat question input in ChatStreamingView

Add max_length=4000 to ChatStreamingSerializer.q and replace the bare
request.data["q"] read with proper serializer.is_valid(raise_exception=True)
so oversized or missing questions are rejected with HTTP 400 before
reaching the LLM.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: Add defensive prompt framing to mark document content as untrusted

* Also adds a system prompt which is treated higher that this is untrusted stuff

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 10:03:27 -07:00