amorfo77/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2026-09-09 03:07:59 +00:00
Code Issues Projects Releases Wiki Activity
11,553 Commits 41 Branches 231 Tags
739d5c589fbda90e155e0dc7e38e0039270b3d0d
Commit Graph
11553 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
stumpylog 739d5c589f Ensures an embed migration stops the migration chain, just in case 2026-06-15 08:28:26 -07:00
Trenton Holmes fa8f47165e Adds equality test and no covers some defensive error handling stuff 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 1ea49d5337 Chore(beta): deduplicate vector store internals via helper methods 
Extract three helpers to remove copy-paste between compact() and
_run_structural_migration():
- _meta_set_on(conn, key, value): static upsert into any connection's
  index_meta; _meta_set() now delegates to it
- _create_vec_table(conn, dim): CREATE VIRTUAL TABLE DDL (carries the
  nosemgrep annotation)
- _swap_in_compact(compact_path, db_path): close/replace/reconnect
  sequence used by both file-swap callers

Also normalises compact() error-path cleanup to unlink(missing_ok=True).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 f331b306e0 Enhancement(beta): add schema migration machinery to sqlite-vec vector store 
Adds versioned schema migration support modelled after PR #12968's LanceDB
approach, adapted for sqlite-vec's file-swap compaction pattern.

- SCHEMA_VERSION = 1 written to index_meta at table creation and preserved
  through compact()
- Migration dataclass with from_version, to_version, kind ("structural" or
  "re-embed"), description, and an optional apply(src, dst, dim) callable
- MIGRATIONS registry (empty at v1 baseline); add entries and bump
  SCHEMA_VERSION when the schema changes
- check_and_run_migrations(): structural migrations run via the same
  file-swap as compact() (no re-embed); re-embed migrations return True
  so the caller forces a full rebuild
- update_llm_index() calls check_and_run_migrations() under the write lock
  before any indexing work

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton Holmes fe7c2eca14 Try to resolve or silence some semgrep. But we're using SQL here, not an ORM and we control the inputs, not users 2026-06-15 08:28:26 -07:00
Trenton Holmes 5d2aacab6d Removes the benchmarking file 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 ebd590725c Chore(beta): ruff format pass on sqlite-vec AI files 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 c9c5549b14 Chore(beta): drop automatic LanceDB index cleanup on startup 
Leave legacy Lance directory removal to the user rather than deleting it
automatically on first run. Beta policy: user is expected to do a clean
re-embed anyway; no need for the system to silently delete their data.

Remove _cleanup_legacy_lance_index(), the forced-rebuild path that called
it, and the associated tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 c0f1325d4b Chore(beta): partial pyrefly cleanup on sqlite-vec vector store 
- Add MetadataFilter import and isinstance guard in _build_where()
- Add query_embedding None guard in query()
- Fix dict.get() type-checker ambiguity in get_configured_model_name()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 b7bb8cb833 Chore(beta): drop lancedb dependency 
Fixes #12970: the package whose wheels SIGILL on non-AVX2 CPUs is no
longer installed at all.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 25acf89d63 Test(beta): cover legacy LanceDB index cleanup and forced rebuild 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 2268ae4278 Enhancement(beta): move filename/storage path/ASN to node metadata 
Same treatment as title/tags/correspondent in #12944: excluded from
the embedded text, visible to the LLM via metadata prepend. Changes
embedded text for every document, so it ships inside the sqlite-vec
transition, whose forced rebuild re-embeds everything anyway.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 b26b1cc6ba Enhancement(beta): wire indexing pipeline to the sqlite-vec store 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 e8e994ca5b Enhancement(beta): switch AI vector store from LanceDB to sqlite-vec 
Fixes the non-AVX2 SIGILL class (#12970) at the root: lancedb is no
longer imported. sqlite-vec 0.1.9 wheels carry no baked SIMD, vec0
metadata columns give parameterized EQ/IN filtering, WAL preserves the
lock-free-reader model, and compact() rebuilds the table because vec0
DELETEs never reclaim space.

Implementation notes vs. the Task 3A draft:
- compact() uses a file-swap approach (new db file + Path.replace) rather
  than ALTER TABLE RENAME, which does not cascade to shadow tables in
  sqlite-vec 0.1.9 (upstream limitation).
- Bloat is tracked via a cumulative total_inserts counter in index_meta
  because the _rowids shadow table does not accumulate deleted rows in
  0.1.9 (contrary to the design doc assumption from #54).
- None distances from the zero-vector cosine edge case are mapped to
  similarity 0.0 rather than raising TypeError.
- Test suite updated accordingly: _bloat_ratio reads index_meta instead
  of _rowids; seed collision in force-compact test fixed (seed=100.0).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 2e1cde4382 Test(beta): port vector store tests to sqlite-vec backend 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Trenton HolmesandClaude Sonnet 4.6 a3dd8994af Chore(beta): add sqlite-vec 0.1.9 dependency 
Pinned exactly: the 0.1.9 wheels carry no baked SIMD flags (safe on
pre-AVX2 CPUs, the point of this migration); the 0.1.10 alphas bake
-mavx and would reintroduce the #12970 crash class.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 08:28:26 -07:00
Yuki MIZUNOandshamoon 11fb09e4f4 Fix (beta): don't send chat message on Enter while composing with IME (CJK) (#12999) 
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
 2026-06-13 13:48:19 +00:00
Trenton H 8ed4bf2011 Fix: Apply unicode normalization to all paths and path components (#12993) 2026-06-13 12:45:54 +00:00
Trenton H 92c016ce47 Fix: Handle the UTF 16 and BOM text files better (#12994) 2026-06-13 05:35:38 -07:00
shamoon fb3816486c Fix (beta): avoid DRF update calling save on all fields (#12992) 2026-06-12 11:14:26 -07:00
Trenton H 4394403beb Fix: release pooled DB connection during AI LLM/embedding calls (#12983) 2026-06-11 13:07:31 -07:00
Trenton H f188d308eb Fix: health-check pooled DB connections and close the pool on worker shutdown (#12977) 2026-06-11 05:49:10 -07:00
shamoon a5d6ff5f15 Fix: wrap long titles in delete confirm dialog (#12973) 2026-06-10 06:56:02 -07:00
shamoon 8405f66e38 Fix (beta): fix re-ordering in merge dialog (#12967) 2026-06-09 07:03:44 -07:00
shamoon c3459d8f62 Fix (beta): move task filtering to backend fully (#12956) 2026-06-07 22:45:15 +00:00
shamoon 6f8e39c2e0 Fix: avoid unnecessary creating new PDF with pw removal workflow (#12948) 2026-06-07 20:30:08 +00:00
Trenton HClaude Opus 4.8shamoon
 		eb292baa69 Enhancement (beta): Switch the AI vector store to LanceDB (#12944) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: shamoon <shamoon@users.noreply.github.com>
 2026-06-07 11:31:26 -07:00
shamoon 3d0b8343b9 Fixhancement (beta): tasks dismiss all (#12949) 2026-06-07 03:42:06 +00:00
shamoon a7cec673bb Fix (beta): correct chat message bg color (#12955) 2026-06-06 16:00:03 -07:00
shamoon 449fd97b1f Fix (beta): respect disable state for suggest endpoint, require change perms (#12942) 2026-06-05 14:16:53 +00:00
Trenton H fa0c4368d7 Fix: Ensure checksum comparison is using SHA256 in file handling (#12939) 2026-06-05 06:46:45 -07:00
shamoon 289d797837 Merge branch 'dev' into beta 2026-06-03 15:12:44 -07:00
dependabot[bot] f3eb8d4f58 docker-compose(deps): bump apache/tika in /docker/compose (#12912) 
Bumps apache/tika from 3.2.3.0 to 3.3.1.0.

---
updated-dependencies:
- dependency-name: apache/tika
  dependency-version: 3.3.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 13:13:14 -07:00
dependabot[bot] eab964124d docker-compose(deps): bump gotenberg/gotenberg in /docker/compose (#12910) 
Bumps gotenberg/gotenberg from 8.27 to 8.33.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.33'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 12:40:18 -07:00
Trenton H 7ef6ba69e6 Fix: Validate the AI backend settings earlier instead of crashing inside the AI module (#12903) 2026-06-03 12:16:09 -07:00
dependabot[bot] 2e9b07b77f docker-compose(deps): Bump nginx in /docker/compose (#12911) 
Bumps nginx from 1.29.5-alpine to 1.31.1-alpine.

---
updated-dependencies:
- dependency-name: nginx
  dependency-version: 1.31.1-alpine
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 11:41:13 -07:00
Trenton H abdcdccf08 Chore(deps): Silence a couple more vulnerabilities here (#12797) 2026-06-03 09:28:00 -07:00
shamoon 1663ed170c Enhancement (beta): add direct LLM language setting (#12906) 2026-06-03 15:53:22 +00:00
dependabot[bot]andshamoon 59f22a3d59 Chore(deps-dev): Bump @playwright/test from 1.59.1 to 1.60.0 in /src-ui (#12919) 
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
 2026-06-03 15:49:50 +00:00
shamoon 47a6fcfc39 Fix (beta): correctly apply i18n in suggestions dropdown (#12905) 2026-06-03 08:40:06 -07:00
dependabot[bot] edcc78d450 Chore(deps-dev): Bump @types/node from 25.6.0 to 25.9.1 in /src-ui (#12915) 
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.6.0 to 25.9.1.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 25.9.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 15:26:15 +00:00
dependabot[bot] 63d5b0f148 Chore(deps): Bump pdfjs-dist from 5.6.205 to 5.7.284 in /src-ui (#12918) 
Signed-off-by: dependabot[bot] <support@github.com>
 2026-06-03 15:16:59 +00:00
dependabot[bot] cd4122e438 Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates (#12913) 
Bumps the frontend-eslint-dependencies group with 4 updates in the /src-ui directory: [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin), [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser), [@typescript-eslint/utils](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/utils) and [eslint](https://github.com/eslint/eslint).


Updates `@typescript-eslint/eslint-plugin` from 8.59.1 to 8.60.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.60.0/packages/eslint-plugin)

Updates `@typescript-eslint/parser` from 8.59.1 to 8.60.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.60.0/packages/parser)

Updates `@typescript-eslint/utils` from 8.59.1 to 8.60.0
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/utils/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.60.0/packages/utils)

Updates `eslint` from 10.2.1 to 10.4.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/compare/v10.2.1...v10.4.0)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-version: 8.60.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-eslint-dependencies
- dependency-name: "@typescript-eslint/parser"
  dependency-version: 8.60.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-eslint-dependencies
- dependency-name: "@typescript-eslint/utils"
  dependency-version: 8.60.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-eslint-dependencies
- dependency-name: eslint
  dependency-version: 10.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-eslint-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 15:06:26 +00:00
dependabot[bot] bc883f5ade Chore(deps-dev): Bump webpack from 5.106.2 to 5.107.2 in /src-ui (#12917) 
Bumps [webpack](https://github.com/webpack/webpack) from 5.106.2 to 5.107.2.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Changelog](https://github.com/webpack/webpack/blob/main/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack/compare/v5.106.2...v5.107.2)

---
updated-dependencies:
- dependency-name: webpack
  dependency-version: 5.107.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 14:53:04 +00:00
GitHub Actions bafca06f5c Auto translate strings 2026-06-03 14:36:28 +00:00
dependabot[bot] b8bca9e836 Chore(deps): Bump zone.js from 0.16.1 to 0.16.2 in /src-ui (#12916) 
Bumps [zone.js](https://github.com/angular/angular/tree/HEAD/packages/zone.js) from 0.16.1 to 0.16.2.
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/packages/zone.js/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/zone.js-0.16.2/packages/zone.js)

---
updated-dependencies:
- dependency-name: zone.js
  dependency-version: 0.16.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 14:34:27 +00:00
dependabot[bot] 207085c687 Chore(deps-dev): Bump the frontend-jest-dependencies group (#12908) 
Bumps the frontend-jest-dependencies group in /src-ui with 3 updates: [jest](https://github.com/jestjs/jest/tree/HEAD/packages/jest), [jest-environment-jsdom](https://github.com/jestjs/jest/tree/HEAD/packages/jest-environment-jsdom) and [jest-preset-angular](https://github.com/thymikee/jest-preset-angular).


Updates `jest` from 30.3.0 to 30.4.2
- [Release notes](https://github.com/jestjs/jest/releases)
- [Changelog](https://github.com/jestjs/jest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/jestjs/jest/commits/v30.4.2/packages/jest)

Updates `jest-environment-jsdom` from 30.3.0 to 30.4.1
- [Release notes](https://github.com/jestjs/jest/releases)
- [Changelog](https://github.com/jestjs/jest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/jestjs/jest/commits/v30.4.1/packages/jest-environment-jsdom)

Updates `jest-preset-angular` from 16.1.4 to 16.1.5
- [Release notes](https://github.com/thymikee/jest-preset-angular/releases)
- [Changelog](https://github.com/thymikee/jest-preset-angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/thymikee/jest-preset-angular/compare/v16.1.4...v16.1.5)

---
updated-dependencies:
- dependency-name: jest
  dependency-version: 30.4.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-jest-dependencies
- dependency-name: jest-environment-jsdom
  dependency-version: 30.4.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-jest-dependencies
- dependency-name: jest-preset-angular
  dependency-version: 16.1.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-jest-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 14:16:06 +00:00
GitHub Actions 8b1de8711b Auto translate strings 2026-06-03 13:55:01 +00:00
dependabot[bot] e2f728f5d4 Chore(deps): Bump the frontend-angular-dependencies group (#12907) 
Bumps the frontend-angular-dependencies group in /src-ui with 20 updates:

| Package | From | To |
| --- | --- | --- |
| [@angular/cdk](https://github.com/angular/components) | `21.2.8` | `21.2.12` |
| [@angular/common](https://github.com/angular/angular/tree/HEAD/packages/common) | `21.2.10` | `21.2.14` |
| [@angular/compiler](https://github.com/angular/angular/tree/HEAD/packages/compiler) | `21.2.10` | `21.2.14` |
| [@angular/core](https://github.com/angular/angular/tree/HEAD/packages/core) | `21.2.10` | `21.2.14` |
| [@angular/forms](https://github.com/angular/angular/tree/HEAD/packages/forms) | `21.2.10` | `21.2.14` |
| [@angular/localize](https://github.com/angular/angular) | `21.2.10` | `21.2.14` |
| [@angular/platform-browser](https://github.com/angular/angular/tree/HEAD/packages/platform-browser) | `21.2.10` | `21.2.14` |
| [@angular/platform-browser-dynamic](https://github.com/angular/angular/tree/HEAD/packages/platform-browser-dynamic) | `21.2.10` | `21.2.14` |
| [@angular/router](https://github.com/angular/angular/tree/HEAD/packages/router) | `21.2.10` | `21.2.14` |
| [@ng-select/ng-select](https://github.com/ng-select/ng-select) | `21.8.0` | `21.8.2` |
| [@angular-devkit/core](https://github.com/angular/angular-cli) | `21.2.8` | `21.2.12` |
| [@angular-devkit/schematics](https://github.com/angular/angular-cli) | `21.2.8` | `21.2.12` |
| [@angular-eslint/builder](https://github.com/angular-eslint/angular-eslint/tree/HEAD/packages/builder) | `21.3.1` | `21.4.0` |
| [@angular-eslint/eslint-plugin](https://github.com/angular-eslint/angular-eslint/tree/HEAD/packages/eslint-plugin) | `21.3.1` | `21.4.0` |
| [@angular-eslint/eslint-plugin-template](https://github.com/angular-eslint/angular-eslint/tree/HEAD/packages/eslint-plugin-template) | `21.3.1` | `21.4.0` |
| [@angular-eslint/schematics](https://github.com/angular-eslint/angular-eslint/tree/HEAD/packages/schematics) | `21.3.1` | `21.4.0` |
| [@angular-eslint/template-parser](https://github.com/angular-eslint/angular-eslint/tree/HEAD/packages/template-parser) | `21.3.1` | `21.4.0` |
| [@angular/build](https://github.com/angular/angular-cli) | `21.2.8` | `21.2.12` |
| [@angular/cli](https://github.com/angular/angular-cli) | `21.2.8` | `21.2.12` |
| [@angular/compiler-cli](https://github.com/angular/angular/tree/HEAD/packages/compiler-cli) | `21.2.10` | `21.2.14` |


Updates `@angular/cdk` from 21.2.8 to 21.2.12
- [Release notes](https://github.com/angular/components/releases)
- [Changelog](https://github.com/angular/components/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/components/compare/v21.2.8...v21.2.12)

Updates `@angular/common` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/common)

Updates `@angular/compiler` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/compiler)

Updates `@angular/core` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/core)

Updates `@angular/forms` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/forms)

Updates `@angular/localize` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/compare/v21.2.10...v21.2.14)

Updates `@angular/platform-browser` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/platform-browser)

Updates `@angular/platform-browser-dynamic` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/platform-browser-dynamic)

Updates `@angular/router` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/router)

Updates `@ng-select/ng-select` from 21.8.0 to 21.8.2
- [Release notes](https://github.com/ng-select/ng-select/releases)
- [Changelog](https://github.com/ng-select/ng-select/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ng-select/ng-select/compare/v21.8.0...v21.8.2)

Updates `@angular-devkit/core` from 21.2.8 to 21.2.12
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.2.8...v21.2.12)

Updates `@angular-devkit/schematics` from 21.2.8 to 21.2.12
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.2.8...v21.2.12)

Updates `@angular-eslint/builder` from 21.3.1 to 21.4.0
- [Release notes](https://github.com/angular-eslint/angular-eslint/releases)
- [Changelog](https://github.com/angular-eslint/angular-eslint/blob/main/packages/builder/CHANGELOG.md)
- [Commits](https://github.com/angular-eslint/angular-eslint/commits/v21.4.0/packages/builder)

Updates `@angular-eslint/eslint-plugin` from 21.3.1 to 21.4.0
- [Release notes](https://github.com/angular-eslint/angular-eslint/releases)
- [Changelog](https://github.com/angular-eslint/angular-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/angular-eslint/angular-eslint/commits/v21.4.0/packages/eslint-plugin)

Updates `@angular-eslint/eslint-plugin-template` from 21.3.1 to 21.4.0
- [Release notes](https://github.com/angular-eslint/angular-eslint/releases)
- [Changelog](https://github.com/angular-eslint/angular-eslint/blob/main/packages/eslint-plugin-template/CHANGELOG.md)
- [Commits](https://github.com/angular-eslint/angular-eslint/commits/v21.4.0/packages/eslint-plugin-template)

Updates `@angular-eslint/schematics` from 21.3.1 to 21.4.0
- [Release notes](https://github.com/angular-eslint/angular-eslint/releases)
- [Changelog](https://github.com/angular-eslint/angular-eslint/blob/main/packages/schematics/CHANGELOG.md)
- [Commits](https://github.com/angular-eslint/angular-eslint/commits/v21.4.0/packages/schematics)

Updates `@angular-eslint/template-parser` from 21.3.1 to 21.4.0
- [Release notes](https://github.com/angular-eslint/angular-eslint/releases)
- [Changelog](https://github.com/angular-eslint/angular-eslint/blob/main/packages/template-parser/CHANGELOG.md)
- [Commits](https://github.com/angular-eslint/angular-eslint/commits/v21.4.0/packages/template-parser)

Updates `@angular/build` from 21.2.8 to 21.2.12
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.2.8...v21.2.12)

Updates `@angular/cli` from 21.2.8 to 21.2.12
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.2.8...v21.2.12)

Updates `@angular/compiler-cli` from 21.2.10 to 21.2.14
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.14/packages/compiler-cli)

---
updated-dependencies:
- dependency-name: "@angular/cdk"
  dependency-version: 21.2.12
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/common"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/compiler"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/core"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/forms"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/localize"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/platform-browser"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/platform-browser-dynamic"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/router"
  dependency-version: 21.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@ng-select/ng-select"
  dependency-version: 21.8.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-devkit/core"
  dependency-version: 21.2.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-devkit/schematics"
  dependency-version: 21.2.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-eslint/builder"
  dependency-version: 21.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-eslint/eslint-plugin"
  dependency-version: 21.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-eslint/eslint-plugin-template"
  dependency-version: 21.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-eslint/schematics"
  dependency-version: 21.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-eslint/template-parser"
  dependency-version: 21.4.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/build"
  dependency-version: 21.2.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/cli"
  dependency-version: 21.2.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/compiler-cli"
  dependency-version: 21.2.14
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-angular-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-06-03 06:53:21 -07:00
Trenton HClaude Sonnet 4.6shamoon
 		98dc191194 Fix: Lock AI index during reading and don't index documents many times during a bulk update (#12899) 
* Fix: Move LLM index lock outside index dir and skip per-doc tasks on bulk update

Two concurrency bugs from #12893:

[P1] Lock file lived inside LLM_INDEX_DIR. A rebuild calls
shutil.rmtree(LLM_INDEX_DIR), deleting the lock while a worker still
held it. A second worker then acquired a fresh lock on the new path and
ran concurrently, defeating serialisation. Move the lock to
DATA_DIR/locks/llm_index.lock (a new settings constant LLM_INDEX_LOCK)
so rmtree cannot touch it. The locks/ dir is created at settings load
time, matching the existing pattern for LOGGING_DIR.

[P2] document_updated was connected to add_or_update_document_in_llm_index
in apps.py. bulk_update_documents() emits document_updated for every
document in the batch, queuing N per-document LLM tasks, and then also
calls update_llm_index(rebuild=False) once at the end. Pass
skip_ai_index=True when sending document_updated from the bulk path so
the handler skips the per-document enqueue; the existing batch call at
the end of bulk_update_documents is the only LLM update for that path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: ghost vectors leave KeyError-prone nodes_dict entries after deletion

docstore.delete_document() removes a node from the docstore but leaves its
entry in index_struct.nodes_dict (the FAISS positional-id to node-UUID map).
A subsequent similarity query resolves the ghost position to the deleted UUID,
finds nothing in fetched_nodes_by_id, and raises KeyError inside
_insert_fetched_nodes_into_query_result.

Purge stale nodes_dict entries after each docstore deletion and re-sync the
mutated index_struct into the kvstore so persist() writes the updated mapping.
Dead FAISS vectors remain in the flat index until the next full rebuild
(IndexFlatL2 is append-only); add a try/except KeyError around
retriever.retrieve() as a defensive fallback for any residual ghost positions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: acquire index lock in query_similar_documents

query_similar_documents() loaded the index and ran the FAISS retriever
without holding the file lock. All write paths (update_llm_index,
llm_index_add_or_update_document, llm_index_remove_document) hold
FileLock(_index_lock_path()), so a concurrent rebuild calling
shutil.rmtree(LLM_INDEX_DIR) while a read is mid-load produces an IOError
or corrupt partial state.

Wrap the load_or_build_index() call and all subsequent retriever work inside
FileLock. The early-return guards (vector_store_file_exists check, empty
allowed_document_ids) remain outside the lock; the DB query for the final
result set also stays outside.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: skip LLM index enqueue on document_updated during version addition

When a document is consumed as a new version of an existing document, the
consumer fires document_consumption_finished (which triggers
add_or_update_document_in_llm_index) and then document_updated for the root
document. Both signals are connected to the same handler, so the root document
was enqueued for LLM indexing twice per version-addition event.

Pass skip_ai_index=True on the consumer's version-addition document_updated
send so the handler's existing guard suppresses the duplicate enqueue.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Test: bulk_update_documents must not enqueue per-doc LLM tasks

With AI enabled, bulk_update_documents() sends document_updated for every
document in the batch. The skip_ai_index=True kwarg (added in the P2 fix)
prevents add_or_update_document_in_llm_index from enqueuing a per-document
task for each one. Only the single update_llm_index call at the end should run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Debug level log sure

* Update src/paperless_ai/indexing.py

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>

* Apply suggestion from @shamoon

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
 2026-06-02 10:46:29 -07:00