Performance: load a cached Hugging Face embedding model without contacting the Hub

Every chat question and AI suggestion request builds the embedding model, and sentence-transformers
checks the Hub for a newer revision each time. With the model already cached that check cost about
2 seconds per request (0.2 seconds without it), and it fails outright when the Hub is unreachable.

Load from the local cache first and only download when the model is missing. A full rebuild still
checks the Hub, so a new model release is picked up together with re-embedding every document
instead of silently mixing revisions.
This commit is contained in:
stumpylog committed 2026-11-09 11:36:42 -08:00
1 parent 936b1b3bfa
commit fc5fc485de
5 files changed
+65 -6

No files matched your search

+2
View File
@@ -565,6 +565,8 @@ Specify `rebuild` to build the index from scratch from all documents in the data
this the first time you enable the feature, or after changing the embedding backend or
model.
When using the `huggingface` embedding backend, a model that is already downloaded is loaded from the local cache without contacting the Hugging Face Hub. Only `rebuild` checks the Hub for a newer revision of the model, so a model update is picked up together with re-embedding every document. If the model is not cached yet, it is downloaded on first use.
Specify `update` to incrementally index new and changed documents. This is what the
scheduled task runs.