Compare commits

..
Author SHA1 Message Date
stumpylogandClaude Opus 5 296ddff37e Collapse near-duplicate outbound guard tests into parametrized cases
Several tests in test_network.py and test_network_integration.py had
grown into near-copies of each other, differing only in the URL or
host they fed the guard while asserting the same thing. That made the
suite noisier to read and to extend than the behaviour it covers
warrants.

Merge the invalid-host rejection tests into one parametrized test over
all seven unusual host forms, and likewise for their allow_internal
counterparts that assert nothing is rejected. Merge the
resolve_public_addresses tests that only check the host reaches the
resolver unchanged, and the ones that check a private answer blocks
regardless of how the host was spelled, keeping the stricter resolver
call assertion on both merged cases. Add localhost as a fourth case to
the numeric-host-forms test in the transport integration tests, so the
name and the non-canonical numeric spellings of loopback are covered
by one test.

No behaviour or assertion strength changes; this only reduces
duplicated test bodies while keeping every original case addressable
by its own parametrize id.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 13:49:35 -07:00
stumpylogandClaude Opus 5 de8edc31c5 Fold the outbound guard's public-address patch into a named fixture
Six tests in test_network_integration.py and one in test_workflows.py each
patched paperless.network.is_public_ip inline to make every resolved
address count as public, so the guard's own address policy would not get
in the way of whatever the test was actually checking. The line was
repeated verbatim at every call site and understated what it did: read
literally it makes every address public, while each docstring around it
already explained the intent as loopback being treated as public.

Add an every_address_is_public fixture to conftest.py, following the
existing pattern for the other outbound test fixtures: a thin wrapper
that imports a new allow_all_addresses helper from
paperless_testing.outbound and calls it with mocker. Apply it at each
call site with pytest.mark.usefixtures, dropping the mocker parameter
from tests that no longer need it directly. The test that patches
is_public_ip with a selective side effect to allow only one address is
left untouched, since that selectivity is load-bearing for what it
verifies.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:56:55 -07:00
stumpylogandClaude Opus 5 46c150eb67 Deduplicate the outbound connect guard's sync and async paths
The two connect_tcp implementations each repeated the per-attempt budget
arithmetic verbatim, and only the sync one delegated resolution and error
mapping to a helper, so a drift between the copies could silently give one
stack a different timeout policy from the other. In the IMAP client,
_connect_pinned re-asserted a fact its only caller had already established,
which reads as a runtime invariant check on a security-relevant path when it
is only a narrowing aid.

Move the budget arithmetic into one helper called from both loops, add an
async twin of the resolve helper so the two loop bodies differ only by await,
and pass the narrowed address tuple into _connect_pinned instead of asserting
it. The PinnedIMAP4 docstring now spells out that no pinning and pinning that
yielded nothing are different things, and the monotonic clock seam says why it
exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:06 -07:00
stumpylogandClaude Opus 5 58777076c5 Cover webhooks to internal hosts when they are allowed
Every webhook test that reaches a real socket ran with internal requests
disallowed, and the rest intercept above the transport. Wiring the
transport to always disallow internal addresses would therefore have
passed the suite while breaking webhooks to internal hosts on every
default install.

A new test sends a webhook to localhost with internal requests allowed
and checks that the payload arrives and that the guard resolved nothing.
The Host header test's docstring is also corrected: the header now comes
from the URL, not from a resolved hostname.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:06 -07:00
stumpylogandClaude Opus 5 d414297c05 Fix: Validate IP literals from the resolver's answer only
Outbound host checks parsed IP literals themselves and skipped the
resolver for them. That second parser is what let a host such as
8.8.8.8%2eexample pass as the public address 8.8.8.8, and even with
zone ids limited to IPv6 it remains one more place where the checked
host can be read differently from the connected one.

The separate literal parsing is removed. Every host now goes to
getaddrinfo, which answers numeric literals itself without a lookup, and
only the addresses it returns are classified. Zone ids are still dropped
from the resolver's answers, where a scoped IPv6 literal comes back as
fe80::1%1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:06 -07:00
stumpylogandClaude Opus 5 d53a930aed Fix: Stop treating a dotted quad followed by "%" as an IP literal
The literal check stripped everything after the first "%" from any host,
so a name such as 8.8.8.8%2e169-254-169-254.sslip.io was accepted as the
public address 8.8.8.8 without a DNS lookup. requests percent-decodes the
host before connecting, so Remote OCR would then resolve
8.8.8.8.169-254-169-254.sslip.io and reach an internal address even with
internal endpoints disallowed.

A zone id is now stripped only when the part before "%" is an IPv6
address; any other host containing "%" is looked up as a name. URL
validation with internal addresses disallowed also rejects a host that
contains "%" at all, since the name checked there could otherwise differ
from the one an HTTP client decodes and dials.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:05 -07:00
stumpylogandClaude Opus 5 78ca920771 Document outbound connection policy for internal-address settings
The internal-address settings now describe that a hostname is blocked if any resolved address is non-public. Guarded requests connect directly without proxy variables, and webhook requests never follow redirects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:05 -07:00
stumpylogandClaude Opus 5 4c5255a8dd Remove the URL-rewriting pinned transport
Every consumer now uses the guarded transports, so the request-rewriting
transport and its helpers are no longer needed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:05 -07:00
stumpylogandClaude Opus 5 283e49b1ee Match get_mailbox annotations to what callers actually pass
get_mailbox declared port and security as always-present types, but
MailAccount.imap_port is nullable and imap_security is stored as a plain
integer, so type checking flagged every call site as passing the wrong
type.

Widen the annotations to port: int | None and security: int, matching the
model fields; IntegerChoices members still compare equal to plain ints, so
the existing branching in get_mailbox is unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:05 -07:00
stumpylogandClaude Opus 5 3130fc3a7c Use shared outbound resolution for IMAP host pinning
get_mailbox validates the IMAP host with resolve_public_addresses and keeps
its existing error messages; the pinned client dials typed addresses.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:49:05 -07:00
stumpylogandClaude Opus 5 8f56c6167b Use the guarded transport for workflow webhooks
A blocked webhook now raises OutboundRequestBlockedError, which the task
treats as an expected failure and does not retry. The webhook security
tests run against a real local server instead of a patched resolver.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:48:44 -07:00
stumpylogandClaude Opus 5 eb644f3fac Report outbound policy blocks from AI requests as 502
AIClient raises LLMBlockedError when a request was refused by the outbound
connection policy, including when the openai SDK wraps the block in
APIConnectionError. ai_suggestions answers 502 instead of a 500.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:48:44 -07:00
stumpylogandClaude Opus 5 6ad00ca55a Use guarded transports for AI LLM and embedding clients
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:48:21 -07:00
stumpylogandClaude Opus 5 526adbad1a Fix two weak assertions in the outbound guard integration tests
The environment proxy test could not fail for any regression in the guard:
httpx ignores environment proxy variables whenever a transport is passed
explicitly, and the address the test pointed the proxy at was itself
internal, so a blocked direct connection was indistinguishable from a
blocked proxied one. It now builds the client through the production
factory, which does not pass a transport, points the proxy variables at a
second recording server, and asserts the real origin server receives the
request while the proxy server sees no connection at all.

Several tests asserting a blocked request never reached the target server
checked only a connection counter that is incremented after accept() in
the server thread, which does not rule out a guard that connects and then
fails validation afterward. Each of those tests now also asserts the dial
recorder saw no address dialled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:48:21 -07:00
stumpylogandClaude Opus 5 d53453fb71 Add real-socket tests for the outbound connection guard
A local HTTP server, a per-hostname resolver fake and dial spies exercise
the guarded transports end to end: address fallback, blocking before any
connection, the Host header, TLS server name, per-host connection pooling,
numeric host spellings, environment proxies and redirects to blocked hosts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:48:20 -07:00
stumpylogandClaude Opus 5 4eaf8032ad Add guarded httpx transports and client factories
The transports install the outbound guard on httpcore's connection pool
after checking its exact layout, and accept no proxy, uds or retries
options. httpx, httpcore and anyio become declared dependencies, pinned
narrowly where private attributes are relied on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:47:40 -07:00
stumpylogandClaude Opus 5 9e1f938d56 Guard outbound connections in the httpcore network backend
With internal addresses disallowed, the backend resolves the origin host,
rejects it if any address is non-public, and dials the validated literals
in family-interleaved order under the caller's connect timeout. Resolver
failures surface as connect errors; unix sockets are always refused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:47:40 -07:00
stumpylogandClaude Opus 5 0da50ad348 Reject outbound URLs that HTTP clients may split differently
urllib3 treats a backslash as the end of the URL authority, while urlparse
and httpx do not. For a URL such as http://127.0.0.1\@evil.example/ the
check resolved evil.example while urllib3 would connect to 127.0.0.1, so a
redirect to such a URL could reach an internal host.

When internal addresses are disallowed, validate_outbound_http_url now
rejects any URL containing a backslash, an ASCII control character or
whitespace before resolving it. URLs validated with internal addresses
allowed are unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:47:40 -07:00
stumpylogandClaude Opus 5 5e971bc0ce Resolve outbound hosts to validated public addresses
resolve_public_addresses and its async twin return every resolved address
in resolver order, de-duplicated and zone-stripped, and reject the whole
name if any address is non-public. validate_outbound_http_url uses them and
keeps its existing messages.

validate_outbound_http_url now resolves the hostname as httpx and urllib3
encode it (IDNA 2008). It previously let getaddrinfo apply the stdlib IDNA
2003 codec, which encodes characters such as "ß" differently, so a URL
could pass the check under one DNS name and be connected to under another.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:47:40 -07:00
stumpylogandClaude Opus 5 17a91c385d Type outbound block errors and classify addresses with is_global
is_public_ip now takes an ipaddress object and relies on is_global, keeping
multicast and the NAT64 well-known prefix as explicit extra exclusions.
Adds OutboundRequestBlockedError and HostResolutionError, both picklable so
Celery keeps them intact on task failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 12:47:40 -07:00
222 changed files with 55051 additions and 54164 deletions

No files matched your search

-53
View File
@@ -1,53 +0,0 @@
name: Install apt packages
description: >
Installs apt packages with a per-attempt timeout and retries. The hosted runner's Azure apt mirror intermittently serves archives at a crawl, which apt's own timeout does not catch. A stuck attempt is killed and retried, which usually lands on a healthy mirror node.
inputs:
packages:
description: Space separated list of packages to install
required: true
attempts:
description: Maximum number of attempts
default: "3"
attempt-timeout:
description: Seconds before a single attempt is killed
default: "180"
runs:
using: composite
steps:
- name: Install ${{ inputs.packages }}
shell: bash
env:
PACKAGES: ${{ inputs.packages }}
ATTEMPTS: ${{ inputs.attempts }}
ATTEMPT_TIMEOUT: ${{ inputs.attempt-timeout }}
run: |
SUDO=""
if [ "$(id -u)" -ne 0 ]; then
SUDO="sudo"
fi
attempt_install() {
$SUDO apt-get update -q
# Intentionally unquoted, this is a list of package names
$SUDO apt-get install -q --yes --no-install-recommends $PACKAGES
}
export -f attempt_install
export SUDO PACKAGES
for attempt in $(seq 1 "${ATTEMPTS}"); do
echo "::group::apt install attempt ${attempt}/${ATTEMPTS}"
start=$(date +%s)
if timeout --kill-after=10 "${ATTEMPT_TIMEOUT}" bash -c attempt_install; then
echo "::endgroup::"
echo "Installed in $(( $(date +%s) - start ))s"
exit 0
fi
echo "::endgroup::"
echo "::warning::apt install attempt ${attempt}/${ATTEMPTS} failed or exceeded ${ATTEMPT_TIMEOUT}s"
# A killed attempt can leave dpkg half configured
$SUDO dpkg --configure -a || true
done
echo "::error::apt install failed after ${ATTEMPTS} attempts"
exit 1
+1
View File
@@ -59,6 +59,7 @@ updates:
- "drf-*" - "drf-*"
- "djangorestframework" - "djangorestframework"
- "whitenoise" - "whitenoise"
- "bleach"
- "jinja2" - "jinja2"
# Async, Task Queuing & Caching # Async, Task Queuing & Caching
async-tasks: async-tasks:
+12 -11
View File
@@ -80,7 +80,7 @@ jobs:
needs: changes needs: changes
if: needs.changes.outputs.backend_changed == 'true' if: needs.changes.outputs.backend_changed == 'true'
name: "Python ${{ matrix.python-version }}" name: "Python ${{ matrix.python-version }}"
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
strategy: strategy:
@@ -102,19 +102,20 @@ jobs:
with: with:
python-version: "${{ matrix.python-version }}" python-version: "${{ matrix.python-version }}"
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with: with:
version: ${{ env.DEFAULT_UV_VERSION }} version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true enable-cache: true
python-version: ${{ steps.setup-python.outputs.python-version }} python-version: ${{ steps.setup-python.outputs.python-version }}
- name: Install system dependencies - name: Install system dependencies
timeout-minutes: 12 timeout-minutes: 10
uses: $/.github/actions/apt-install run: |
with: sudo apt-get update -qq
packages: unpaper tesseract-ocr imagemagick ghostscript poppler-utils sudo apt-get install -qq --no-install-recommends \
unpaper tesseract-ocr imagemagick ghostscript poppler-utils
- name: Configure ImageMagick - name: Configure ImageMagick
run: | run: |
sudo cp docker/rootfs/etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-7/policy.xml sudo cp docker/rootfs/etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml
- name: Install Python dependencies - name: Install Python dependencies
env: env:
PYTHON_VERSION: ${{ steps.setup-python.outputs.python-version }} PYTHON_VERSION: ${{ steps.setup-python.outputs.python-version }}
@@ -138,13 +139,13 @@ jobs:
pytest pytest
- name: Upload test results to Codecov - name: Upload test results to Codecov
if: always() if: always()
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1 uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with: with:
flags: backend-python-${{ matrix.python-version }} flags: backend-python-${{ matrix.python-version }}
files: junit.xml files: junit.xml
report_type: test_results report_type: test_results
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1 uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with: with:
flags: backend-python-${{ matrix.python-version }} flags: backend-python-${{ matrix.python-version }}
files: coverage.xml files: coverage.xml
@@ -158,7 +159,7 @@ jobs:
needs: changes needs: changes
if: needs.changes.outputs.backend_changed == 'true' if: needs.changes.outputs.backend_changed == 'true'
name: Check project typing name: Check project typing
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
env: env:
@@ -175,7 +176,7 @@ jobs:
with: with:
python-version: "${{ env.DEFAULT_PYTHON }}" python-version: "${{ env.DEFAULT_PYTHON }}"
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with: with:
version: ${{ env.DEFAULT_UV_VERSION }} version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true enable-cache: true
+6 -6
View File
@@ -24,10 +24,10 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
include: include:
- runner: ubuntu-26.04 - runner: ubuntu-24.04
arch: amd64 arch: amd64
platform: linux/amd64 platform: linux/amd64
- runner: ubuntu-26.04-arm - runner: ubuntu-24.04-arm
arch: arm64 arch: arm64
platform: linux/arm64 platform: linux/arm64
runs-on: ${{ matrix.runner }} runs-on: ${{ matrix.runner }}
@@ -106,7 +106,7 @@ jobs:
echo "repository=${repo_name}" echo "repository=${repo_name}"
echo "name=${repo_name}" >> $GITHUB_OUTPUT echo "name=${repo_name}" >> $GITHUB_OUTPUT
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with: with:
@@ -132,7 +132,7 @@ jobs:
type=semver,pattern={{major}}.{{minor}} type=semver,pattern={{major}}.{{minor}}
- name: Build and push by digest - name: Build and push by digest
id: build id: build
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with: with:
context: . context: .
file: ./Dockerfile file: ./Dockerfile
@@ -163,7 +163,7 @@ jobs:
archive: false archive: false
merge-and-push: merge-and-push:
name: Merge and Push Manifest name: Merge and Push Manifest
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
needs: build-arch needs: build-arch
if: needs.build-arch.outputs.should-push == 'true' if: needs.build-arch.outputs.should-push == 'true'
environment: image-publishing environment: image-publishing
@@ -182,7 +182,7 @@ jobs:
echo "Downloaded digests:" echo "Downloaded digests:"
ls -la /tmp/digests/ ls -la /tmp/digests/
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4.3.0
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
with: with:
+3 -3
View File
@@ -65,7 +65,7 @@ jobs:
needs: changes needs: changes
if: needs.changes.outputs.docs_changed == 'true' if: needs.changes.outputs.docs_changed == 'true'
name: Build Documentation name: Build Documentation
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
steps: steps:
- uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0 - uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
- name: Checkout - name: Checkout
@@ -78,7 +78,7 @@ jobs:
with: with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }} python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with: with:
version: ${{ env.DEFAULT_UV_VERSION }} version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true enable-cache: true
@@ -102,7 +102,7 @@ jobs:
name: Deploy Documentation name: Deploy Documentation
needs: [changes, build] needs: [changes, build]
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && needs.changes.outputs.docs_changed == 'true' if: github.event_name == 'push' && github.ref == 'refs/heads/main' && needs.changes.outputs.docs_changed == 'true'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
pages: write pages: write
id-token: write id-token: write
+12 -13
View File
@@ -72,7 +72,7 @@ jobs:
needs: changes needs: changes
if: needs.changes.outputs.frontend_changed == 'true' if: needs.changes.outputs.frontend_changed == 'true'
name: Install Dependencies name: Install Dependencies
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
steps: steps:
@@ -104,7 +104,7 @@ jobs:
name: Lint name: Lint
needs: [changes, install-dependencies] needs: [changes, install-dependencies]
if: needs.changes.outputs.frontend_changed == 'true' if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
steps: steps:
@@ -137,7 +137,7 @@ jobs:
name: "Unit Tests (${{ matrix.shard-index }}/${{ matrix.shard-count }})" name: "Unit Tests (${{ matrix.shard-index }}/${{ matrix.shard-count }})"
needs: [changes, install-dependencies] needs: [changes, install-dependencies]
if: needs.changes.outputs.frontend_changed == 'true' if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
strategy: strategy:
@@ -174,13 +174,13 @@ jobs:
run: cd src-ui && pnpm run test --max-workers=2 --shard=${{ matrix.shard-index }}/${{ matrix.shard-count }} run: cd src-ui && pnpm run test --max-workers=2 --shard=${{ matrix.shard-index }}/${{ matrix.shard-count }}
- name: Upload test results to Codecov - name: Upload test results to Codecov
if: always() if: always()
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1 uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with: with:
flags: frontend-node-${{ matrix.node-version }} flags: frontend-node-${{ matrix.node-version }}
directory: src-ui/ directory: src-ui/
report_type: test_results report_type: test_results
- name: Upload coverage to Codecov - name: Upload coverage to Codecov
uses: codecov/codecov-action@303a32d7a59b442fa8d48b6a1cc6825c09c847a5 # v7.1.1 uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with: with:
flags: frontend-node-${{ matrix.node-version }} flags: frontend-node-${{ matrix.node-version }}
directory: src-ui/coverage/ directory: src-ui/coverage/
@@ -188,10 +188,10 @@ jobs:
name: E2E Tests name: E2E Tests
needs: [changes, install-dependencies] needs: [changes, install-dependencies]
if: needs.changes.outputs.frontend_changed == 'true' if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
container: mcr.microsoft.com/playwright:v1.62.1-resolute container: mcr.microsoft.com/playwright:v1.62.1-noble
env: env:
PLAYWRIGHT_BROWSERS_PATH: /ms-playwright PLAYWRIGHT_BROWSERS_PATH: /ms-playwright
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1 PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
@@ -216,7 +216,7 @@ jobs:
with: with:
python-version: '3.12' python-version: '3.12'
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with: with:
version: '0.12.x' version: '0.12.x'
enable-cache: false enable-cache: false
@@ -231,10 +231,9 @@ jobs:
- name: Install dependencies - name: Install dependencies
run: cd src-ui && pnpm install --frozen-lockfile run: cd src-ui && pnpm install --frozen-lockfile
- name: Install backend system dependencies - name: Install backend system dependencies
timeout-minutes: 12 run: |
uses: $/.github/actions/apt-install apt-get update
with: apt-get install --yes --quiet --no-install-recommends libmagic1
packages: libmagic1
- name: Install backend dependencies - name: Install backend dependencies
env: env:
PYTHON_VERSION: ${{ steps.setup-python.outputs.python-version }} PYTHON_VERSION: ${{ steps.setup-python.outputs.python-version }}
@@ -246,7 +245,7 @@ jobs:
name: Frontend Build name: Frontend Build
needs: [changes, unit-tests, e2e-tests] needs: [changes, unit-tests, e2e-tests]
if: needs.changes.outputs.frontend_changed == 'true' if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
steps: steps:
+9 -10
View File
@@ -14,7 +14,7 @@ permissions: {}
jobs: jobs:
wait-for-docker: wait-for-docker:
name: Wait for Docker Build name: Wait for Docker Build
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
checks: read checks: read
statuses: read statuses: read
@@ -30,7 +30,7 @@ jobs:
build-release: build-release:
name: Build Release name: Build Release
needs: wait-for-docker needs: wait-for-docker
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
steps: steps:
@@ -59,7 +59,7 @@ jobs:
with: with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }} python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with: with:
version: ${{ env.DEFAULT_UV_VERSION }} version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: false enable-cache: false
@@ -70,10 +70,9 @@ jobs:
run: | run: |
uv sync --python "${PYTHON_VERSION}" --dev --frozen uv sync --python "${PYTHON_VERSION}" --dev --frozen
- name: Install system dependencies - name: Install system dependencies
timeout-minutes: 12 run: |
uses: $/.github/actions/apt-install sudo apt-get update -qq
with: sudo apt-get install -qq --no-install-recommends gettext liblept5
packages: gettext libleptonica6
# ---- Build Documentation ---- # ---- Build Documentation ----
- name: Build documentation - name: Build documentation
env: env:
@@ -145,7 +144,7 @@ jobs:
publish-release: publish-release:
name: Publish Release name: Publish Release
needs: build-release needs: build-release
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: write contents: write
pull-requests: write pull-requests: write
@@ -197,7 +196,7 @@ jobs:
name: Append Changelog name: Append Changelog
needs: publish-release needs: publish-release
if: needs.publish-release.outputs.prerelease == 'false' if: needs.publish-release.outputs.prerelease == 'false'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: write contents: write
pull-requests: write pull-requests: write
@@ -213,7 +212,7 @@ jobs:
with: with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }} python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with: with:
version: ${{ env.DEFAULT_UV_VERSION }} version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: false enable-cache: false
+3 -3
View File
@@ -15,7 +15,7 @@ permissions:
jobs: jobs:
zizmor: zizmor:
name: Run zizmor name: Run zizmor
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
contents: read contents: read
actions: read actions: read
@@ -29,7 +29,7 @@ jobs:
uses: zizmorcore/zizmor-action@cc914d7f3750a2d13d75c7f184a1060aa0e9d482 # v0.6.4 uses: zizmorcore/zizmor-action@cc914d7f3750a2d13d75c7f184a1060aa0e9d482 # v0.6.4
semgrep: semgrep:
name: Semgrep CE name: Semgrep CE
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
container: container:
image: semgrep/semgrep:1.155.0@sha256:cc869c685dcc0fe497c86258da9f205397d8108e56d21a86082ea4886e52784d image: semgrep/semgrep:1.155.0@sha256:cc869c685dcc0fe497c86258da9f205397d8108e56d21a86082ea4886e52784d
if: github.actor != 'dependabot[bot]' if: github.actor != 'dependabot[bot]'
@@ -44,7 +44,7 @@ jobs:
- name: Run Semgrep - name: Run Semgrep
run: semgrep scan --config auto --sarif-output results.sarif run: semgrep scan --config auto --sarif-output results.sarif
- name: Upload results to GitHub code scanning - name: Upload results to GitHub code scanning
uses: github/codeql-action/upload-sarif@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2 uses: github/codeql-action/upload-sarif@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
if: always() if: always()
with: with:
sarif_file: results.sarif sarif_file: results.sarif
+4 -4
View File
@@ -17,7 +17,7 @@ jobs:
cleanup-images: cleanup-images:
name: Cleanup Image Tags for ${{ matrix.primary-name }} name: Cleanup Image Tags for ${{ matrix.primary-name }}
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
environment: registry-maintenance environment: registry-maintenance
strategy: strategy:
fail-fast: false fail-fast: false
@@ -29,7 +29,7 @@ jobs:
steps: steps:
- name: Clean temporary images - name: Clean temporary images
if: "${{ env.TOKEN != '' }}" if: "${{ env.TOKEN != '' }}"
uses: stumpylog/image-cleaner-action/ephemeral@21f875bab2376314e0525c614e923c669434f0e4 # v0.13.0 uses: stumpylog/image-cleaner-action/ephemeral@4fe057d991d63b8f6d5d22c40f17c1bca2226537 # v0.12.0
with: with:
token: "${{ env.TOKEN }}" token: "${{ env.TOKEN }}"
owner: "${{ github.repository_owner }}" owner: "${{ github.repository_owner }}"
@@ -42,7 +42,7 @@ jobs:
cleanup-untagged-images: cleanup-untagged-images:
name: Cleanup Untagged Images Tags for ${{ matrix.primary-name }} name: Cleanup Untagged Images Tags for ${{ matrix.primary-name }}
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
needs: needs:
- cleanup-images - cleanup-images
environment: registry-maintenance environment: registry-maintenance
@@ -56,7 +56,7 @@ jobs:
steps: steps:
- name: Clean untagged images - name: Clean untagged images
if: "${{ env.TOKEN != '' }}" if: "${{ env.TOKEN != '' }}"
uses: stumpylog/image-cleaner-action/untagged@21f875bab2376314e0525c614e923c669434f0e4 # v0.13.0 uses: stumpylog/image-cleaner-action/untagged@4fe057d991d63b8f6d5d22c40f17c1bca2226537 # v0.12.0
with: with:
token: "${{ env.TOKEN }}" token: "${{ env.TOKEN }}"
owner: "${{ github.repository_owner }}" owner: "${{ github.repository_owner }}"
+3 -3
View File
@@ -21,7 +21,7 @@ on:
jobs: jobs:
analyze: analyze:
name: Analyze name: Analyze
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
actions: read actions: read
contents: read contents: read
@@ -39,7 +39,7 @@ jobs:
persist-credentials: false persist-credentials: false
# Initializes the CodeQL tools for scanning. # Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2 uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
with: with:
languages: ${{ matrix.language }} languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file. # If you wish to specify custom queries, you can do so here or in a config file.
@@ -47,4 +47,4 @@ jobs:
# Prefix the list here with "+" to use these queries and those in the config file. # Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main # queries: ./path/to/local/query, your-org/your-repo/queries@main
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2 uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
+2 -2
View File
@@ -13,7 +13,7 @@ jobs:
synchronize-with-crowdin: synchronize-with-crowdin:
name: Crowdin Sync name: Crowdin Sync
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
environment: translation-sync environment: translation-sync
steps: steps:
- name: Checkout - name: Checkout
@@ -22,7 +22,7 @@ jobs:
token: ${{ secrets.PNGX_BOT_PAT }} token: ${{ secrets.PNGX_BOT_PAT }}
persist-credentials: false persist-credentials: false
- name: crowdin action - name: crowdin action
uses: crowdin/github-action@9c23991700c0ec5256fd41089b9d9d7d540e424e # v3.3.0 uses: crowdin/github-action@0d5670f539973aea2f01abce61a8989934df0025 # v3.0.2
with: with:
upload_translations: false upload_translations: false
download_translations: true download_translations: true
+1 -1
View File
@@ -7,7 +7,7 @@ jobs:
# Note: peakoss/anti-slop does not support the `issues` event yet (all of its # Note: peakoss/anti-slop does not support the `issues` event yet (all of its
# issue inputs are still commented out upstream), so the checks that the PR Bot # issue inputs are still commented out upstream), so the checks that the PR Bot
# workflow gets from the action are implemented manually here. # workflow gets from the action are implemented manually here.
runs-on: ubuntu-slim runs-on: ubuntu-latest
permissions: permissions:
issues: write issues: write
steps: steps:
+2 -2
View File
@@ -4,7 +4,7 @@ on:
types: [opened] types: [opened]
jobs: jobs:
Anti-slop: Anti-slop:
runs-on: ubuntu-slim runs-on: ubuntu-latest
permissions: permissions:
contents: read contents: read
issues: read issues: read
@@ -24,7 +24,7 @@ jobs:
ASLOP-PR-VERIFY ASLOP-PR-VERIFY
pr-bot: pr-bot:
name: Automated PR Bot name: Automated PR Bot
runs-on: ubuntu-slim runs-on: ubuntu-latest
# Runs after Anti-slop so the welcome comment can see whether the PR was closed # Runs after Anti-slop so the welcome comment can see whether the PR was closed
# instead of racing it. Still runs if that job fails, so labeling is not lost. # instead of racing it. Still runs if that job fails, so labeling is not lost.
needs: Anti-slop needs: Anti-slop
+1 -1
View File
@@ -12,7 +12,7 @@ permissions:
jobs: jobs:
pr_opened_or_reopened: pr_opened_or_reopened:
name: pr_opened_or_reopened name: pr_opened_or_reopened
runs-on: ubuntu-slim runs-on: ubuntu-24.04
permissions: permissions:
# write permission is required for autolabeler # write permission is required for autolabeler
pull-requests: write pull-requests: write
+5 -5
View File
@@ -9,7 +9,7 @@ jobs:
stale: stale:
name: 'Stale' name: 'Stale'
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
issues: write issues: write
pull-requests: write pull-requests: write
@@ -34,7 +34,7 @@ jobs:
lock-threads: lock-threads:
name: 'Lock Old Threads' name: 'Lock Old Threads'
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-26.04 runs-on: ubuntu-24.04
permissions: permissions:
issues: write issues: write
pull-requests: write pull-requests: write
@@ -58,7 +58,7 @@ jobs:
close-answered-discussions: close-answered-discussions:
name: 'Close Answered Discussions' name: 'Close Answered Discussions'
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-slim runs-on: ubuntu-24.04
permissions: permissions:
discussions: write discussions: write
steps: steps:
@@ -117,7 +117,7 @@ jobs:
close-outdated-discussions: close-outdated-discussions:
name: 'Close Outdated Discussions' name: 'Close Outdated Discussions'
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-slim runs-on: ubuntu-24.04
permissions: permissions:
discussions: write discussions: write
steps: steps:
@@ -211,7 +211,7 @@ jobs:
close-unsupported-feature-requests: close-unsupported-feature-requests:
name: 'Close Unsupported Feature Requests' name: 'Close Unsupported Feature Requests'
if: github.repository_owner == 'paperless-ngx' if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-slim runs-on: ubuntu-24.04
permissions: permissions:
discussions: write discussions: write
steps: steps:
+5 -6
View File
@@ -8,7 +8,7 @@ env:
jobs: jobs:
generate-translate-strings: generate-translate-strings:
name: Generate Translation Strings name: Generate Translation Strings
runs-on: ubuntu-26.04 runs-on: ubuntu-latest
environment: translation-sync environment: translation-sync
permissions: permissions:
contents: write contents: write
@@ -25,12 +25,11 @@ jobs:
id: setup-python id: setup-python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
- name: Install system dependencies - name: Install system dependencies
timeout-minutes: 12 run: |
uses: $/.github/actions/apt-install sudo apt-get update -qq
with: sudo apt-get install -qq --no-install-recommends gettext
packages: gettext
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0 uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with: with:
version: ${{ env.DEFAULT_UV_VERSION }} version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true enable-cache: true
+457 -706
View File
File diff suppressed because it is too large. Load diff
+3 -3
View File
@@ -38,7 +38,7 @@ repos:
- json - json
# See https://github.com/prettier/prettier/issues/15742 for the fork reason # See https://github.com/prettier/prettier/issues/15742 for the fork reason
- repo: https://github.com/rbubley/mirrors-prettier - repo: https://github.com/rbubley/mirrors-prettier
rev: 'v3.9.9' rev: 'v3.9.6'
hooks: hooks:
- id: prettier - id: prettier
types_or: types_or:
@@ -46,11 +46,11 @@ repos:
- ts - ts
- markdown - markdown
additional_dependencies: additional_dependencies:
- prettier@3.9.9 - prettier@3.9.6
- 'prettier-plugin-organize-imports@4.3.0' - 'prettier-plugin-organize-imports@4.3.0'
# Python hooks # Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit - repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.16.9 rev: v0.16.7
hooks: hooks:
- id: ruff-check - id: ruff-check
- id: ruff-format - id: ruff-format
+14244 -6100
View File
File diff suppressed because it is too large. Load diff
+2 -4
View File
@@ -30,7 +30,7 @@ RUN set -eux \
# Purpose: Installs s6-overlay and rootfs # Purpose: Installs s6-overlay and rootfs
# Comments: # Comments:
# - Don't leave anything extra in here either # - Don't leave anything extra in here either
FROM ghcr.io/astral-sh/uv:0.12.23-python3.14-trixie-slim AS s6-overlay-base FROM ghcr.io/astral-sh/uv:0.12.16-python3.14-trixie-slim AS s6-overlay-base
WORKDIR /usr/src/s6 WORKDIR /usr/src/s6
@@ -171,9 +171,7 @@ RUN set -eux \
&& cp /etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml \ && cp /etc/ImageMagick-6/paperless-policy.xml /etc/ImageMagick-6/policy.xml \
&& echo "Cleaning up image layer" \ && echo "Cleaning up image layer" \
&& rm --force --verbose *.deb \ && rm --force --verbose *.deb \
&& rm --recursive --force --verbose /var/lib/apt/lists/* \ && rm --recursive --force --verbose /var/lib/apt/lists/*
&& echo "Configuring interactive shells to source the s6 container environment" \
&& echo '. /etc/profile.d/contenv.sh' >> /etc/bash.bashrc
WORKDIR /usr/src/paperless/src/ WORKDIR /usr/src/paperless/src/
+1 -1
View File
@@ -24,7 +24,7 @@ services:
network_mode: host network_mode: host
restart: unless-stopped restart: unless-stopped
greenmail: greenmail:
image: docker.io/greenmail/standalone:2.1.14 image: docker.io/greenmail/standalone:2.1.13
hostname: greenmail hostname: greenmail
container_name: greenmail container_name: greenmail
environment: environment:
-18
View File
@@ -1,18 +0,0 @@
#!/bin/sh
# Source s6 container environment for interactive shells.
# Ensures variables resolved from *_FILE secret injection are visible
# when using 'docker exec bash'. Does not affect s6 services (those
# use with-contenv directly). Has no effect in non-container contexts
# because the directory will not exist.
# Note: sh/dash shells opened via 'docker exec sh' are not covered;
# only bash-based sessions benefit from this file.
_pngx_contenv="/run/s6/container_environment"
if [ -d "${_pngx_contenv}" ]; then
for _pngx_f in "${_pngx_contenv}"/*; do
[ -f "${_pngx_f}" ] || continue
_pngx_name=$(basename "${_pngx_f}")
_pngx_val=$(cat "${_pngx_f}")
export "${_pngx_name}=${_pngx_val}"
done
fi
unset _pngx_contenv _pngx_f _pngx_name _pngx_val
+9 -27
View File
@@ -136,15 +136,13 @@ for suggested generation and embedding models.
### AI-assisted suggestions ### AI-assisted suggestions
With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type, With AI enabled, Paperless-ngx can suggest a title, tags, correspondent, document type,
storage path and dates by sending the document to the LLM using "Suggest" button on the document storage path and dates by sending the document to the LLM. This is **opt-in per request**
detail page. You can choose which type of suggestions are requested by default under Settings > and surfaces through the "Suggest" control on the document detail page, alongside the
Documents, either ML (classifier-based) suggestions, AI suggestions, or both. When both are requested classic classifier-based suggestions — it does not disable them. Suggestions are requested
the results are combined. automatically when you open a document that carries an inbox tag unless "Automatically request
suggestions for inbox documents" under Settings > Documents is disabled. Suggestion output
Suggestions are requested automatically when you open a document that carries an inbox tag language can be steered with
unless "Automatically request suggestions for inbox documents" under Settings > Documents is disabled. [`PAPERLESS_AI_LLM_OUTPUT_LANGUAGE`](configuration.md#PAPERLESS_AI_LLM_OUTPUT_LANGUAGE)
Suggestion output language can be steered with [`PAPERLESS_AI_LLM_OUTPUT_LANGUAGE`](configuration.md#PAPERLESS_AI_LLM_OUTPUT_LANGUAGE)
(otherwise it follows the user's UI language). (otherwise it follows the user's UI language).
### The LLM index (RAG) and similar documents ### The LLM index (RAG) and similar documents
@@ -155,11 +153,8 @@ in similar existing documents, and the document chat can retrieve relevant conte
Enable it by setting Enable it by setting
[`PAPERLESS_AI_LLM_EMBEDDING_BACKEND`](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_BACKEND) [`PAPERLESS_AI_LLM_EMBEDDING_BACKEND`](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_BACKEND)
(`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). By default, the main (`huggingface` for fully-local embeddings, or `ollama` / `openai-like`). The index is only
LLM API key and endpoint are used, but an optional embedding-specific[API key](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_API_KEY) built when AI is enabled **and** an embedding backend is set.
and [endpoint](configuration.md#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) can be configured.
The index is only built when AI is enabled **and** an embedding backend is set.
The index is updated automatically on a schedule controlled by The index is updated automatically on a schedule controlled by
[`PAPERLESS_LLM_INDEX_TASK_CRON`](configuration.md#PAPERLESS_LLM_INDEX_TASK_CRON) (daily by [`PAPERLESS_LLM_INDEX_TASK_CRON`](configuration.md#PAPERLESS_LLM_INDEX_TASK_CRON) (daily by
@@ -1010,19 +1005,6 @@ documents to both separate and categorize them in a single operation.
**Example:** A 6-page scan with TAG:invoice on page 3 and TAG:receipt on page 5 will create **Example:** A 6-page scan with TAG:invoice on page 3 and TAG:receipt on page 5 will create
three documents: pages 1-2 (no tags), pages 3-4 (tagged "invoice"), and pages 5-6 (tagged "receipt"). three documents: pages 1-2 (no tags), pages 3-4 (tagged "invoice"), and pages 5-6 (tagged "receipt").
### Barcode Contents {#barcode-contents}
By default, Paperless only uses barcodes for splitting, ASNs and tags. With
[`PAPERLESS_CONSUMER_STORE_BARCODE_VALUES`](configuration.md#PAPERLESS_CONSUMER_STORE_BARCODE_VALUES)
enabled, it stores the content of every barcode with the document, e.g. payment codes or QR codes.
- Barcodes are listed on the **Metadata** tab with page, type and content, and can be copied.
- The API returns them in the `barcodes` field of `/api/documents/{id}/metadata/`.
- They can be [searched](usage.md#searching-barcodes), e.g. `barcodes:DE89370400440532013000`.
- Only the first [`PAPERLESS_CONSUMER_BARCODE_MAX_PAGES`](configuration.md#PAPERLESS_CONSUMER_BARCODE_MAX_PAGES)
pages are scanned. Reprocessing reads the barcodes of existing documents.
- Each version keeps its own barcodes, and the newest version's are shown and searched.
## Automatic collation of double-sided documents {#collate} ## Automatic collation of double-sided documents {#collate}
!!! note !!! note
+6 -28
View File
@@ -1576,6 +1576,9 @@ ports.
#### [`PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS=<bool>`](#PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS) {#PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS} #### [`PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS=<bool>`](#PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS) {#PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS}
: If set to false, webhooks cannot be sent to internal URLs (e.g., localhost). : If set to false, webhooks cannot be sent to internal URLs (e.g., localhost).
A hostname is blocked if any of the addresses it resolves to is non-public.
Webhook requests connect directly, without using the `HTTP_PROXY` or
`HTTPS_PROXY` environment variables, and never follow redirects.
Defaults to true, which allows internal requests. Defaults to true, which allows internal requests.
@@ -1584,7 +1587,7 @@ ports.
#### [`PAPERLESS_EMAIL_ALLOW_INTERNAL_HOSTS=<bool>`](#PAPERLESS_EMAIL_ALLOW_INTERNAL_HOSTS) {#PAPERLESS_EMAIL_ALLOW_INTERNAL_HOSTS} #### [`PAPERLESS_EMAIL_ALLOW_INTERNAL_HOSTS=<bool>`](#PAPERLESS_EMAIL_ALLOW_INTERNAL_HOSTS) {#PAPERLESS_EMAIL_ALLOW_INTERNAL_HOSTS}
: If set to false, incoming mail account connections are blocked when the : If set to false, incoming mail account connections are blocked when the
configured IMAP hostname resolves to a non-public address (for example, configured IMAP hostname resolves to any non-public address (for example,
localhost, link-local, or RFC1918 private ranges). localhost, link-local, or RFC1918 private ranges).
Defaults to true, which allows internal hosts. Defaults to true, which allows internal hosts.
@@ -1796,13 +1799,6 @@ assigns or creates tags if a properly formatted barcode is detected.
Defaults to false. Defaults to false.
#### [`PAPERLESS_CONSUMER_STORE_BARCODE_VALUES=<bool>`](#PAPERLESS_CONSUMER_STORE_BARCODE_VALUES) {#PAPERLESS_CONSUMER_STORE_BARCODE_VALUES}
: Stores the content of every barcode found during consumption, see
[Barcode Contents](advanced_usage.md#barcode-contents).
Defaults to false.
## Audit Trail ## Audit Trail
#### [`PAPERLESS_AUDIT_LOG_ENABLED=<bool>`](#PAPERLESS_AUDIT_LOG_ENABLED) {#PAPERLESS_AUDIT_LOG_ENABLED} #### [`PAPERLESS_AUDIT_LOG_ENABLED=<bool>`](#PAPERLESS_AUDIT_LOG_ENABLED) {#PAPERLESS_AUDIT_LOG_ENABLED}
@@ -2140,13 +2136,6 @@ for language and resource considerations.
Defaults to None. Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_API_KEY=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_API_KEY) {#PAPERLESS_AI_LLM_EMBEDDING_API_KEY}
: The API key to use for the embedding backend. If not supplied, embeddings use
`PAPERLESS_AI_LLM_API_KEY`.
Defaults to None.
#### [`PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) {#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT} #### [`PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT=<str>`](#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT) {#PAPERLESS_AI_LLM_EMBEDDING_ENDPOINT}
: The endpoint / url to use for the embedding backend. If not supplied, embeddings use : The endpoint / url to use for the embedding backend. If not supplied, embeddings use
@@ -2228,22 +2217,11 @@ used with the OpenAI-compatible backend to target a custom provider or local gat
#### [`PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS=<bool>`](#PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS) {#PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS} #### [`PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS=<bool>`](#PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS) {#PAPERLESS_AI_LLM_ALLOW_INTERNAL_ENDPOINTS}
: If set to false, Paperless blocks AI endpoint URLs that resolve to non-public addresses (e.g., localhost, etc). : If set to false, Paperless blocks AI endpoint URLs that resolve to non-public addresses (e.g., localhost, etc).
A hostname is blocked if any of the addresses it resolves to is non-public, and redirects are checked the same way.
Requests to a configured AI endpoint connect directly, without using the `HTTP_PROXY` or `HTTPS_PROXY` environment variables.
Defaults to true, which allows internal endpoints. Defaults to true, which allows internal endpoints.
#### [`PAPERLESS_AI_LLM_EXTRA_PARAMS=<json>`](#PAPERLESS_AI_LLM_EXTRA_PARAMS) {#PAPERLESS_AI_LLM_EXTRA_PARAMS}
: A JSON object of extra parameters sent with every LLM request, for providers that require a parameter Paperless does not
set itself. Values here override Paperless' own, and no validation is performed. Whatever you put here is passed to the
backend as-is, so an invalid parameter will simply be rejected by your provider. For example, current OpenAI reasoning
models refuse tool calls on the chat completions API unless reasoning is off:
```
PAPERLESS_AI_LLM_EXTRA_PARAMS={"reasoning_effort": "none"}
```
Defaults to empty, which adds nothing to requests.
#### [`PAPERLESS_LLM_INDEX_TASK_CRON=<cron expression>`](#PAPERLESS_LLM_INDEX_TASK_CRON) {#PAPERLESS_LLM_INDEX_TASK_CRON} #### [`PAPERLESS_LLM_INDEX_TASK_CRON=<cron expression>`](#PAPERLESS_LLM_INDEX_TASK_CRON) {#PAPERLESS_LLM_INDEX_TASK_CRON}
: Configures the schedule to update the AI embeddings of text content and metadata for all documents. Only performed if : Configures the schedule to update the AI embeddings of text content and metadata for all documents. Only performed if
+1 -15
View File
@@ -613,7 +613,7 @@ The following workflow action types are available:
- The request headers as key-value pairs - The request headers as key-value pairs
For security reasons, webhooks can be limited to specific ports and disallowed from connecting to local URLs. See the relevant For security reasons, webhooks can be limited to specific ports and disallowed from connecting to local URLs. See the relevant
[configuration settings](configuration.md#workflow-webhooks) to change this behavior. If you are allowing non-admins to create workflows, [configuration settings](configuration.md#workflow-webhooks) to change this behavior. Webhook requests connect directly (proxy environment variables are not used) and do not follow redirects. If you are allowing non-admins to create workflows,
you may want to adjust these settings to prevent abuse. you may want to adjust these settings to prevent abuse.
##### Move to Trash {#workflow-action-move-to-trash} ##### Move to Trash {#workflow-action-move-to-trash}
@@ -1061,20 +1061,6 @@ notes.user:alice notes.note:insurance
The bare `notes:` prefix is shorthand for `notes.note:`. The bare `notes:` prefix is shorthand for `notes.note:`.
#### Searching barcodes
If [barcode contents are stored](advanced_usage.md#barcode-contents), they can be searched by
content or type, but only with a field name:
```
barcodes.value:DE89370400440532013000
barcodes.format:qrcode
barcodes:wifi barcodes:guest
```
`barcodes:` is shorthand for `barcodes.value:`. Separators are stripped, so each part of e.g.
`WIFI:S:Guest;P:secret;;` can be searched on its own.
All of these can be combined. Syntax not described here may not work as expected, and an unknown field name is searched as ordinary text. All of these can be combined. Syntax not described here may not work as expected, and an unknown field name is searched as ordinary text.
!!! note !!! note
+11 -8
View File
@@ -17,8 +17,10 @@ classifiers = [
# TODO: Move certain things to groups and then utilize that further # TODO: Move certain things to groups and then utilize that further
# This will allow testing to not install a webserver, mysql, etc # This will allow testing to not install a webserver, mysql, etc
dependencies = [ dependencies = [
"anyio>=4.12",
"azure-ai-documentintelligence>=1.0.2", "azure-ai-documentintelligence>=1.0.2",
"babel>=2.17", "babel>=2.17",
"bleach~=6.4.0",
"celery[redis]~=5.6.2", "celery[redis]~=5.6.2",
"channels~=4.2", "channels~=4.2",
"channels-redis~=4.2", "channels-redis~=4.2",
@@ -33,8 +35,8 @@ dependencies = [
"django-compression-middleware~=0.5.0", "django-compression-middleware~=0.5.0",
"django-cors-headers~=4.9.0", "django-cors-headers~=4.9.0",
"django-extensions~=4.1", "django-extensions~=4.1",
"django-filter>=25.1,<27", "django-filter~=25.1",
"django-guardian>=3.3.3,<3.6", "django-guardian>=3.3.3,<3.5",
"django-multiselectfield~=1.0.1", "django-multiselectfield~=1.0.1",
"django-rich~=2.2.0", "django-rich~=2.2.0",
"django-soft-delete~=1.0.18", "django-soft-delete~=1.0.18",
@@ -44,8 +46,10 @@ dependencies = [
"drf-spectacular-sidecar>=2026.7.1,<2026.10", "drf-spectacular-sidecar>=2026.7.1,<2026.10",
"drf-writable-nested~=0.7.1", "drf-writable-nested~=0.7.1",
"filelock~=3.32.0", "filelock~=3.32.0",
"flower>=2.0.1,<2.3", "flower>=2.0.1,<2.2",
"gotenberg-client[httpx]~=1.0", "gotenberg-client[httpx]~=1.0",
"httpcore~=1.0.9",
"httpx~=0.28.1",
"httpx-oauth~=0.17", "httpx-oauth~=0.17",
"ijson>=3.5.1", "ijson>=3.5.1",
"imap-tools>=1.14,<1.16", "imap-tools>=1.14,<1.16",
@@ -76,7 +80,6 @@ dependencies = [
"tantivy~=0.26.0", "tantivy~=0.26.0",
"tika-client[httpx]~=1.0", "tika-client[httpx]~=1.0",
"torch>=2.13,<2.15", "torch>=2.13,<2.15",
"turbohtml~=1.10.0",
"watchfiles>=1.2", "watchfiles>=1.2",
"whitenoise~=6.11", "whitenoise~=6.11",
"whoosh-compat[tantivy]==0.3", "whoosh-compat[tantivy]==0.3",
@@ -84,13 +87,13 @@ dependencies = [
] ]
[project.optional-dependencies] [project.optional-dependencies]
mariadb = [ mariadb = [
"mysqlclient>=2.2.7,<2.4", "mysqlclient~=2.2.7",
] ]
postgres = [ postgres = [
"psycopg[c,pool]==3.3.4", "psycopg[c,pool]==3.3.4",
# Direct dependency for proper resolution of the pre-built wheels # Direct dependency for proper resolution of the pre-built wheels
"psycopg-c==3.3.4", "psycopg-c==3.3.4",
"psycopg-pool==3.3.3", "psycopg-pool==3.3.1",
] ]
webserver = [ webserver = [
"granian[uvloop]>=2.7,<2.9", "granian[uvloop]>=2.7,<2.9",
@@ -112,7 +115,7 @@ lint = [
testing = [ testing = [
"daphne", "daphne",
"factory-boy~=3.3.1", "factory-boy~=3.3.1",
"faker>=40.36,<40.40", "faker>=40.36,<40.39",
"imagehash", "imagehash",
"pytest~=9.1.1", "pytest~=9.1.1",
"pytest-cov~=7.1.0", "pytest-cov~=7.1.0",
@@ -136,6 +139,7 @@ typing = [
"mypy", "mypy",
"mypy-baseline", "mypy-baseline",
"pyrefly", "pyrefly",
"types-bleach",
"types-channels", "types-channels",
"types-colorama", "types-colorama",
"types-dateparser", "types-dateparser",
@@ -150,7 +154,6 @@ typing = [
[tool.uv] [tool.uv]
required-version = ">=0.9.0" required-version = ">=0.9.0"
prerelease = "disallow"
environments = [ environments = [
"sys_platform == 'darwin'", "sys_platform == 'darwin'",
"sys_platform == 'linux'", "sys_platform == 'linux'",
-5
View File
@@ -91,11 +91,6 @@
"glob": "**/*", "glob": "**/*",
"input": "node_modules/pdfjs-dist/iccs/", "input": "node_modules/pdfjs-dist/iccs/",
"output": "/assets/iccs/" "output": "/assets/iccs/"
},
{
"glob": "**/*",
"input": "node_modules/pdfjs-dist/cmaps/",
"output": "/assets/cmaps/"
} }
], ],
"styles": [ "styles": [
+250 -364
View File
File diff suppressed because it is too large. Load diff
+22 -22
View File
@@ -15,16 +15,16 @@
}, },
"private": true, "private": true,
"dependencies": { "dependencies": {
"@angular/cdk": "^22.2.0", "@angular/cdk": "^22.1.6",
"@angular/common": "~22.2.0", "@angular/common": "~22.1.6",
"@angular/compiler": "~22.2.0", "@angular/compiler": "~22.1.6",
"@angular/core": "~22.2.0", "@angular/core": "~22.1.6",
"@angular/forms": "~22.2.0", "@angular/forms": "~22.1.6",
"@angular/localize": "~22.2.0", "@angular/localize": "~22.1.6",
"@angular/platform-browser": "~22.2.0", "@angular/platform-browser": "~22.1.6",
"@angular/router": "~22.2.0", "@angular/router": "~22.1.6",
"@ng-bootstrap/ng-bootstrap": "^21.0.0", "@ng-bootstrap/ng-bootstrap": "^21.0.0",
"@ng-select/ng-select": "~24.1.2", "@ng-select/ng-select": "~24.1.1",
"@ngneat/dirty-check-forms": "^3.0.3", "@ngneat/dirty-check-forms": "^3.0.3",
"@popperjs/core": "^2.11.8", "@popperjs/core": "^2.11.8",
"bootstrap": "^5.3.8", "bootstrap": "^5.3.8",
@@ -45,29 +45,29 @@
}, },
"devDependencies": { "devDependencies": {
"@angular-builders/jest": "^22.0.1", "@angular-builders/jest": "^22.0.1",
"@angular-devkit/core": "^22.2.0", "@angular-devkit/core": "^22.1.8",
"@angular-devkit/schematics": "^22.2.0", "@angular-devkit/schematics": "^22.1.8",
"@angular-eslint/builder": "22.5.0", "@angular-eslint/builder": "22.5.0",
"@angular-eslint/eslint-plugin": "22.5.0", "@angular-eslint/eslint-plugin": "22.5.0",
"@angular-eslint/eslint-plugin-template": "22.5.0", "@angular-eslint/eslint-plugin-template": "22.5.0",
"@angular-eslint/schematics": "22.5.0", "@angular-eslint/schematics": "22.5.0",
"@angular-eslint/template-parser": "22.5.0", "@angular-eslint/template-parser": "22.5.0",
"@angular/build": "22.2.0", "@angular/build": "22.1.8",
"@angular/cli": "22.2.0", "@angular/cli": "22.1.8",
"@angular/compiler-cli": "~22.2.0", "@angular/compiler-cli": "~22.1.6",
"@playwright/test": "^1.62.1", "@playwright/test": "^1.62.1",
"@types/jest": "^30.0.0", "@types/jest": "^30.0.0",
"@types/node": "^26.6.2", "@types/node": "^26.5.0",
"@typescript-eslint/eslint-plugin": "^8.70.1", "@typescript-eslint/eslint-plugin": "^8.70.0",
"@typescript-eslint/parser": "^8.70.1", "@typescript-eslint/parser": "^8.70.0",
"@typescript-eslint/utils": "^8.70.1", "@typescript-eslint/utils": "^8.70.0",
"eslint": "^10.11.0", "eslint": "^10.10.0",
"jest": "30.5.2", "jest": "30.5.1",
"jest-environment-jsdom": "^30.5.2", "jest-environment-jsdom": "^30.5.1",
"jest-junit": "^17.0.0", "jest-junit": "^17.0.0",
"jest-preset-angular": "^17.0.0", "jest-preset-angular": "^17.0.0",
"jest-websocket-mock": "^2.5.0", "jest-websocket-mock": "^2.5.0",
"prettier": "^3.9.9", "prettier": "^3.9.6",
"prettier-plugin-organize-imports": "^4.3.0", "prettier-plugin-organize-imports": "^4.3.0",
"ts-node": "~10.9.2", "ts-node": "~10.9.2",
"typescript": "^6.0.3" "typescript": "^6.0.3"
+1959 -1311
View File
File diff suppressed because it is too large. Load diff
@@ -253,24 +253,6 @@
</div> </div>
</div> </div>
@if (aiEnabled) {
<div class="row mb-3">
<div class="col-md-3 col-form-label pt-0">
<span i18n>Suggestions default to</span>
</div>
<div class="col">
<fieldset class="btn-group btn-group-sm">
<input type="radio" class="btn-check" id="suggestionSourceBoth" [value]="SuggestionSource.Both" formControlName="documentEditingSuggestionSource">
<label class="btn btn-outline-primary" for="suggestionSourceBoth"><ng-container i18n>Both</ng-container></label>
<input type="radio" class="btn-check" id="suggestionSourceML" [value]="SuggestionSource.ML" formControlName="documentEditingSuggestionSource">
<label class="btn btn-outline-primary" for="suggestionSourceML"><i-bs class="me-1" name="cpu"></i-bs><ng-container i18n>ML only</ng-container></label>
<input type="radio" class="btn-check" id="suggestionSourceAI" [value]="SuggestionSource.AI" formControlName="documentEditingSuggestionSource">
<label class="btn btn-outline-primary" for="suggestionSourceAI"><i-bs class="me-1" name="stars"></i-bs><ng-container i18n>AI only</ng-container></label>
</fieldset>
</div>
</div>
}
<div class="row"> <div class="row">
<div class="col"> <div class="col">
<pngx-input-check i18n-title title="Automatically request suggestions for inbox documents" i18n-hint hint="If un-checked, suggestions must be requested via the Suggest button." formControlName="documentEditingAutoSuggest"></pngx-input-check> <pngx-input-check i18n-title title="Automatically request suggestions for inbox documents" i18n-hint hint="If un-checked, suggestions must be requested via the Suggest button." formControlName="documentEditingAutoSuggest"></pngx-input-check>
@@ -307,7 +307,7 @@ describe('SettingsComponent', () => {
expect(toastErrorSpy).toHaveBeenCalled() expect(toastErrorSpy).toHaveBeenCalled()
expect(storeSpy).toHaveBeenCalled() expect(storeSpy).toHaveBeenCalled()
expect(appearanceSettingsSpy).not.toHaveBeenCalled() expect(appearanceSettingsSpy).not.toHaveBeenCalled()
expect(setSpy).toHaveBeenCalledTimes(35) expect(setSpy).toHaveBeenCalledTimes(34)
expect(setSpy).toHaveBeenCalledWith(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [ expect(setSpy).toHaveBeenCalledWith(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [
HideableSidebarItemID.Workflows, HideableSidebarItemID.Workflows,
]) ])
@@ -44,7 +44,6 @@ import {
HIDEABLE_SIDEBAR_ITEM_IDS, HIDEABLE_SIDEBAR_ITEM_IDS,
HideableSidebarItemID, HideableSidebarItemID,
SETTINGS_KEYS, SETTINGS_KEYS,
SuggestionSource,
} from 'src/app/data/ui-settings' } from 'src/app/data/ui-settings'
import { User } from 'src/app/data/user' import { User } from 'src/app/data/user'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive' import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
@@ -185,7 +184,6 @@ export class SettingsComponent
documentEditingRemoveInboxTags: new FormControl(null), documentEditingRemoveInboxTags: new FormControl(null),
documentEditingOverlayThumbnail: new FormControl(null), documentEditingOverlayThumbnail: new FormControl(null),
documentEditingAutoSuggest: new FormControl(null), documentEditingAutoSuggest: new FormControl(null),
documentEditingSuggestionSource: new FormControl(null),
documentDetailsHiddenFields: new FormControl([]), documentDetailsHiddenFields: new FormControl([]),
searchDbOnly: new FormControl(null), searchDbOnly: new FormControl(null),
searchLink: new FormControl(null), searchLink: new FormControl(null),
@@ -219,11 +217,6 @@ export class SettingsComponent
public readonly PdfZoomScale = PdfZoomScale public readonly PdfZoomScale = PdfZoomScale
public readonly PdfEditorEditMode = PdfEditorEditMode public readonly PdfEditorEditMode = PdfEditorEditMode
public readonly SuggestionSource = SuggestionSource
get aiEnabled(): boolean {
return this.settings.get(SETTINGS_KEYS.AI_ENABLED)
}
public readonly documentDetailFieldOptions = documentDetailFieldOptions public readonly documentDetailFieldOptions = documentDetailFieldOptions
public readonly sidebarItemOptions = HIDEABLE_SIDEBAR_ITEM_IDS.map((id) => ({ public readonly sidebarItemOptions = HIDEABLE_SIDEBAR_ITEM_IDS.map((id) => ({
@@ -411,9 +404,6 @@ export class SettingsComponent
documentEditingAutoSuggest: this.settings.get( documentEditingAutoSuggest: this.settings.get(
SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST
), ),
documentEditingSuggestionSource: this.settings.get(
SETTINGS_KEYS.DOCUMENT_EDITING_SUGGESTION_SOURCE
),
documentDetailsHiddenFields: this.settings.get( documentDetailsHiddenFields: this.settings.get(
SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS
), ),
@@ -635,10 +625,6 @@ export class SettingsComponent
SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST, SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST,
this.settingsForm.value.documentEditingAutoSuggest this.settingsForm.value.documentEditingAutoSuggest
) )
this.settings.set(
SETTINGS_KEYS.DOCUMENT_EDITING_SUGGESTION_SOURCE,
this.settingsForm.value.documentEditingSuggestionSource
)
this.settings.set( this.settings.set(
SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS, SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS,
this.settingsForm.value.documentDetailsHiddenFields this.settingsForm.value.documentDetailsHiddenFields
@@ -154,29 +154,11 @@
& section { & section {
position: absolute; position: absolute;
text-align: initial; text-align: initial;
pointer-events: auto;
box-sizing: border-box; box-sizing: border-box;
transform-origin: 0 0; transform-origin: 0 0;
} }
& :is(.linkAnnotation, .buttonWidgetAnnotation.pushButton) > a {
position: absolute;
inset: 0;
font-size: 1em;
transition: none;
}
& :is(.linkAnnotation, .buttonWidgetAnnotation.pushButton):not(.hasBorder)
> a:hover {
opacity: 0.2;
background-color: rgb(255 255 0);
}
& .annotationTextContent { & .annotationTextContent {
opacity: 0; opacity: 0;
} }
} }
:host ::ng-deep .textLayer.selecting ~ .annotationLayer section {
pointer-events: none;
}
@@ -1,11 +1,7 @@
import { SimpleChange } from '@angular/core' import { SimpleChange } from '@angular/core'
import { ComponentFixture, TestBed } from '@angular/core/testing' import { ComponentFixture, TestBed } from '@angular/core/testing'
import * as pdfjs from 'pdfjs-dist/legacy/build/pdf.mjs' import * as pdfjs from 'pdfjs-dist/legacy/build/pdf.mjs'
import { import { PDFSinglePageViewer, PDFViewer } from 'pdfjs-dist/web/pdf_viewer.mjs'
LinkTarget,
PDFSinglePageViewer,
PDFViewer,
} from 'pdfjs-dist/web/pdf_viewer.mjs'
import { PngxPdfViewerComponent } from './pdf-viewer.component' import { PngxPdfViewerComponent } from './pdf-viewer.component'
import { PdfRenderMode, PdfZoomLevel, PdfZoomScale } from './pdf-viewer.types' import { PdfRenderMode, PdfZoomLevel, PdfZoomScale } from './pdf-viewer.types'
@@ -62,16 +58,6 @@ describe('PngxPdfViewerComponent', () => {
expect((component as any).pdfViewer).toBeInstanceOf(PDFViewer) expect((component as any).pdfViewer).toBeInstanceOf(PDFViewer)
}) })
it('opens external links in a new tab', () => {
const linkService = (component as any).linkService
expect(linkService.options).toEqual(
expect.objectContaining({
externalLinkTarget: LinkTarget.BLANK,
externalLinkRel: 'noopener noreferrer nofollow',
})
)
})
it('resolves the worker source relative to the document base URI', async () => { it('resolves the worker source relative to the document base URI', async () => {
setBaseHref('/paperless/') setBaseHref('/paperless/')
const getDocumentSpy = jest.spyOn(pdfjs, 'getDocument') const getDocumentSpy = jest.spyOn(pdfjs, 'getDocument')
@@ -90,7 +76,6 @@ describe('PngxPdfViewerComponent', () => {
withCredentials: true, withCredentials: true,
wasmUrl: expect.stringContaining('/paperless/assets/wasm/'), wasmUrl: expect.stringContaining('/paperless/assets/wasm/'),
iccUrl: expect.stringContaining('/paperless/assets/iccs/'), iccUrl: expect.stringContaining('/paperless/assets/iccs/'),
cMapUrl: expect.stringContaining('/paperless/assets/cmaps/'),
}) })
}) })
@@ -21,7 +21,6 @@ import {
} from 'pdfjs-dist/legacy/build/pdf.mjs' } from 'pdfjs-dist/legacy/build/pdf.mjs'
import { import {
EventBus, EventBus,
LinkTarget,
PDFFindController, PDFFindController,
PDFLinkService, PDFLinkService,
PDFSinglePageViewer, PDFSinglePageViewer,
@@ -76,11 +75,7 @@ export class PngxPdfViewerComponent
private lastViewerPage?: number private lastViewerPage?: number
private readonly eventBus = new EventBus() private readonly eventBus = new EventBus()
private readonly linkService = new PDFLinkService({ private readonly linkService = new PDFLinkService({ eventBus: this.eventBus })
eventBus: this.eventBus,
externalLinkTarget: LinkTarget.BLANK,
externalLinkRel: 'noopener noreferrer nofollow',
})
private readonly findController = new PDFFindController({ private readonly findController = new PDFFindController({
eventBus: this.eventBus, eventBus: this.eventBus,
linkService: this.linkService, linkService: this.linkService,
@@ -190,7 +185,6 @@ export class PngxPdfViewerComponent
withCredentials: true, withCredentials: true,
wasmUrl: new URL('assets/wasm/', this.document.baseURI).toString(), wasmUrl: new URL('assets/wasm/', this.document.baseURI).toString(),
iccUrl: new URL('assets/iccs/', this.document.baseURI).toString(), iccUrl: new URL('assets/iccs/', this.document.baseURI).toString(),
cMapUrl: new URL('assets/cmaps/', this.document.baseURI).toString(),
} }
this.loadingTask = getDocument(initOptions) this.loadingTask = getDocument(initOptions)
try { try {
@@ -1,84 +1,58 @@
<div class="d-flex align-items-center"> <div class="btn-group">
<div class="btn-group"> <button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())" [aria-label]="noSuggestions ? 'No suggestions' : 'Suggest'" i18n-aria-label>
<button type="button" class="btn btn-sm btn-outline-primary" (click)="clickSuggest()" [disabled]="disabled() || loading() || (suggestions() && !aiEnabled())" [aria-label]="noSuggestions ? 'No suggestions' : 'Suggest'" i18n-aria-label> @if (loading()) {
@if (loading()) { <div class="spinner-border spinner-border-sm" role="status"></div>
<div class="spinner-border spinner-border-sm" role="status"></div> } @else if (noSuggestions) {
} @else if (noSuggestions) { <i-bs width="1.2em" height="1.2em" name="check-circle"></i-bs>
<i-bs width="1.2em" height="1.2em" name="check-circle"></i-bs> } @else {
} @else { <i-bs width="1.2em" height="1.2em" name="stars"></i-bs>
<i-bs width="1.2em" height="1.2em" name="lightbulb"></i-bs>
}
@if (noSuggestions) {
<span class="d-none d-lg-inline ps-1" i18n>No suggestions</span>
} @else {
<span class="d-none d-lg-inline ps-1" i18n>Suggest</span>
}
@if (totalSuggestions > 0) {
<span class="badge bg-primary ms-2">{{ totalSuggestions }}</span>
}
</button>
@if (aiEnabled()) {
<div class="btn-group" ngbDropdown #dropdown="ngbDropdown" [popperOptions]="popperOptions">
<button type="button" class="btn btn-sm btn-outline-primary" ngbDropdownToggle [disabled]="disabled() || loading() || !suggestions()" aria-expanded="false" aria-controls="suggestionsDropdown" aria-label="Suggestions dropdown">
<span class="visually-hidden" i18n>Show suggestions</span>
</button>
<div ngbDropdownMenu aria-labelledby="suggestionsDropdown" class="shadow suggestions-dropdown">
<div class="list-group list-group-flush small pb-0">
@if (novelSuggestions === 0 && fieldSuggestions === 0) {
<div class="list-group-item text-muted fst-italic">
<small class="text-muted small fst-italic" i18n>No novel suggestions</small>
</div>
}
@if (suggestions()?.suggested_tags?.length > 0) {
<small class="list-group-item text-uppercase text-muted small"><i-bs class="me-2" name="tags"></i-bs><ng-container i18n>Tags</ng-container></small>
@for (tag of suggestions().suggested_tags; track tag) {
<button type="button" class="list-group-item list-group-item-action bg-light" (click)="addTag.emit(tag)">{{ tag }}</button>
}
}
@if (suggestions()?.suggested_document_types?.length > 0) {
<div class="list-group-item text-uppercase text-muted small"><i-bs class="me-2" name="hash"></i-bs><ng-container i18n>Document Types</ng-container></div>
@for (type of suggestions().suggested_document_types; track type) {
<button type="button" class="list-group-item list-group-item-action bg-light" (click)="addDocumentType.emit(type)">{{ type }}</button>
}
}
@if (suggestions()?.suggested_correspondents?.length > 0) {
<div class="list-group-item text-uppercase text-muted small"><i-bs class="me-2" name="person"></i-bs><ng-container i18n>Correspondents</ng-container></div>
@for (correspondent of suggestions().suggested_correspondents; track correspondent) {
<button type="button" class="list-group-item list-group-item-action bg-light" (click)="addCorrespondent.emit(correspondent)">{{ correspondent }}</button>
}
}
@if (fieldSuggestions > 0) {
<div class="list-group-item text-muted fst-italic">
<small class="text-muted small fst-italic" i18n>{fieldSuggestions, plural, =1 {1 suggestion available below} other {{{fieldSuggestions}} suggestions available below}}</small>
</div>
}
</div>
</div>
</div>
} }
</div> @if (noSuggestions) {
<span class="d-none d-lg-inline ps-1" i18n>No suggestions</span>
} @else {
<span class="d-none d-lg-inline ps-1" i18n>Suggest</span>
}
@if (totalSuggestions > 0) {
<span class="badge bg-primary ms-2">{{ totalSuggestions }}</span>
}
</button>
@if (aiEnabled()) { @if (aiEnabled()) {
<div ngbDropdown autoClose="outside" placement="bottom-end" [popperOptions]="popperOptions"> <div class="btn-group" ngbDropdown #dropdown="ngbDropdown" [popperOptions]="popperOptions">
<button type="button" class="btn btn-sm btn-link position-relative" ngbDropdownToggle [disabled]="disabled() || loading()" i18n-title title="Suggestion options"> <button type="button" class="btn btn-sm btn-outline-primary" ngbDropdownToggle [disabled]="disabled() || loading() || !suggestions()" aria-expanded="false" aria-controls="suggestionsDropdown" aria-label="Suggestions dropdown">
<i-bs name="three-dots"></i-bs> <span class="visually-hidden" i18n>Show suggestions</span>
@if (source() !== defaultSource()) {
<span class="position-absolute top-0 start-100 translate-middle p-1 bg-primary border border-light rounded-circle">
<span class="visually-hidden" i18n>Not using default</span>
</span>
}
</button> </button>
<div ngbDropdownMenu class="shadow p-3">
<div class="small text-muted mb-2" i18n>Suggest using:</div> <div ngbDropdownMenu aria-labelledby="suggestionsDropdown" class="shadow suggestions-dropdown">
<div class="form-check small"> <div class="list-group list-group-flush small pb-0">
<input class="form-check-input" type="checkbox" id="suggestionSourceML" [checked]="useML" [disabled]="useML && !useAI" (change)="setSources($event.target.checked, useAI)"> @if (novelSuggestions === 0 && reusableSuggestions === 0) {
<label class="form-check-label d-inline-flex align-items-center gap-1" for="suggestionSourceML"><i-bs name="cpu"></i-bs><ng-container i18n>ML</ng-container></label> <div class="list-group-item text-muted fst-italic">
</div> <small class="text-muted small fst-italic" i18n>No novel suggestions</small>
<div class="form-check small"> </div>
<input class="form-check-input" type="checkbox" id="suggestionSourceAI" [checked]="useAI" [disabled]="useAI && !useML" (change)="setSources(useML, $event.target.checked)"> }
<label class="form-check-label d-inline-flex align-items-center gap-1" for="suggestionSourceAI"><i-bs name="stars"></i-bs><ng-container i18n>AI</ng-container></label> @if (suggestions()?.suggested_tags?.length > 0) {
<small class="list-group-item text-uppercase text-muted small"><i-bs class="me-2" name="tags"></i-bs><ng-container i18n>Tags</ng-container></small>
@for (tag of suggestions().suggested_tags; track tag) {
<button type="button" class="list-group-item list-group-item-action bg-light" (click)="addTag.emit(tag)">{{ tag }}</button>
}
}
@if (suggestions()?.suggested_document_types?.length > 0) {
<div class="list-group-item text-uppercase text-muted small"><i-bs class="me-2" name="hash"></i-bs><ng-container i18n>Document Types</ng-container></div>
@for (type of suggestions().suggested_document_types; track type) {
<button type="button" class="list-group-item list-group-item-action bg-light" (click)="addDocumentType.emit(type)">{{ type }}</button>
}
}
@if (suggestions()?.suggested_correspondents?.length > 0) {
<div class="list-group-item text-uppercase text-muted small"><i-bs class="me-2" name="person"></i-bs><ng-container i18n>Correspondents</ng-container></div>
@for (correspondent of suggestions().suggested_correspondents; track correspondent) {
<button type="button" class="list-group-item list-group-item-action bg-light" (click)="addCorrespondent.emit(correspondent)">{{ correspondent }}</button>
}
}
@if (reusableSuggestions > 0) {
<div class="list-group-item text-muted fst-italic">
<small class="text-muted small fst-italic" i18n>{reusableSuggestions, plural, =1 {1 existing value suggested below} other {{{reusableSuggestions}} existing values suggested below}}</small>
</div>
}
</div> </div>
</div> </div>
</div> </div>
@@ -1,7 +1,3 @@
.suggestions-dropdown { .suggestions-dropdown {
min-width: 250px; min-width: 250px;
} }
.btn-link.dropdown-toggle::after {
display: none;
}
@@ -1,7 +1,6 @@
import { ComponentFixture, TestBed } from '@angular/core/testing' import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap' import { NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { SuggestionSource } from 'src/app/data/ui-settings'
import { SuggestionsDropdownComponent } from './suggestions-dropdown.component' import { SuggestionsDropdownComponent } from './suggestions-dropdown.component'
describe('SuggestionsDropdownComponent', () => { describe('SuggestionsDropdownComponent', () => {
@@ -84,7 +83,7 @@ describe('SuggestionsDropdownComponent', () => {
fixture.detectChanges() fixture.detectChanges()
expect(fixture.nativeElement.textContent).toContain( expect(fixture.nativeElement.textContent).toContain(
'2 suggestions available below' '2 existing values suggested below'
) )
expect(fixture.nativeElement.textContent).not.toContain( expect(fixture.nativeElement.textContent).not.toContain(
'No novel suggestions' 'No novel suggestions'
@@ -108,7 +107,7 @@ describe('SuggestionsDropdownComponent', () => {
expect(component.totalSuggestions).toBe(4) expect(component.totalSuggestions).toBe(4)
expect(fixture.nativeElement.textContent).toContain('Arbitration') expect(fixture.nativeElement.textContent).toContain('Arbitration')
expect(fixture.nativeElement.textContent).toContain( expect(fixture.nativeElement.textContent).toContain(
'2 suggestions available below' '2 existing values suggested below'
) )
}) })
@@ -125,46 +124,10 @@ describe('SuggestionsDropdownComponent', () => {
}) })
expect(component.novelSuggestions).toBe(0) expect(component.novelSuggestions).toBe(0)
expect(component.totalSuggestions).toBe(6) expect(component.totalSuggestions).toBe(5)
fixture.componentRef.setInput('appliedStoragePath', 7) fixture.componentRef.setInput('appliedStoragePath', 7)
expect(component.totalSuggestions).toBe(5) expect(component.totalSuggestions).toBe(4)
})
it('should count title and dates as field suggestions without calling them existing values', () => {
fixture.componentRef.setInput('aiEnabled', true)
fixture.componentRef.setInput('fetchedSources', [SuggestionSource.ML])
fixture.componentRef.setInput('suggestions', {
title: 'Suggested title',
dates: ['2026-01-04', '2026-02-01', '2026-03-01'],
correspondents: [1, 2, 3, 4],
document_types: [1, 2, 3, 4],
tags: [1, 2, 3, 4, 5, 6, 7, 8],
})
fixture.detectChanges()
component.clickSuggest()
fixture.detectChanges()
expect(component.reusableSuggestions).toBe(16)
expect(component.fieldSuggestions).toBe(20)
expect(component.totalSuggestions).toBe(20)
expect(fixture.nativeElement.textContent).toContain(
'20 suggestions available below'
)
expect(fixture.nativeElement.textContent).not.toContain('existing value')
})
it('should not count title or date suggestions matching the current values', () => {
fixture.componentRef.setInput('suggestions', {
title: 'Current title',
dates: ['2026-01-04', '2026-02-01'],
})
expect(component.fieldSuggestions).toBe(3)
fixture.componentRef.setInput('appliedTitle', 'Current title')
fixture.componentRef.setInput('appliedCreated', '2026-01-04')
expect(component.fieldSuggestions).toBe(1)
expect(component.totalSuggestions).toBe(1)
}) })
it('should show when a completed request returned no suggestions', () => { it('should show when a completed request returned no suggestions', () => {
@@ -181,28 +144,6 @@ describe('SuggestionsDropdownComponent', () => {
expect(fixture.nativeElement.textContent).toContain('No suggestions') expect(fixture.nativeElement.textContent).toContain('No suggestions')
}) })
it('should wait for all pending responses before showing the empty state', () => {
fixture.componentRef.setInput('aiEnabled', true)
fixture.componentRef.setInput('source', SuggestionSource.Both)
fixture.componentRef.setInput('fetchedSources', [SuggestionSource.ML])
fixture.componentRef.setInput('suggestions', { tags: [] })
fixture.componentRef.setInput('loading', true)
fixture.detectChanges()
expect(component.noSuggestions).toBeFalsy()
expect(fixture.nativeElement.textContent).not.toContain('No suggestions')
expect(
fixture.nativeElement.querySelector('[role="status"]')
).not.toBeNull()
fixture.componentRef.setInput('loading', false)
fixture.detectChanges()
expect(component.noSuggestions).toBeTruthy()
expect(fixture.nativeElement.textContent).toContain('No suggestions')
expect(fixture.nativeElement.querySelector('[role="status"]')).toBeNull()
})
it('should not show the empty state before a request or with suggestions', () => { it('should not show the empty state before a request or with suggestions', () => {
expect(component.noSuggestions).toBeFalsy() expect(component.noSuggestions).toBeFalsy()
@@ -238,71 +179,14 @@ describe('SuggestionsDropdownComponent', () => {
it('should toggle dropdown when clickSuggest is called and suggestions are not null', () => { it('should toggle dropdown when clickSuggest is called and suggestions are not null', () => {
fixture.componentRef.setInput('aiEnabled', true) fixture.componentRef.setInput('aiEnabled', true)
fixture.componentRef.setInput('fetchedSources', [SuggestionSource.ML])
fixture.detectChanges() fixture.detectChanges()
fixture.componentRef.setInput('suggestions', { fixture.componentRef.setInput('suggestions', {
suggested_correspondents: [], suggested_correspondents: [],
suggested_tags: [], suggested_tags: [],
suggested_document_types: [], suggested_document_types: [],
}) })
fixture.detectChanges()
component.clickSuggest() component.clickSuggest()
expect(component.dropdown.isOpen()).toBeTruthy() expect(component.dropdown.open).toBeTruthy()
expect(fixture.nativeElement.textContent).toContain('No novel suggestions') expect(fixture.nativeElement.textContent).toContain('No novel suggestions')
}) })
it('should fetch unfetched sources and show existing suggestions', () => {
jest.spyOn(component.getSuggestions, 'emit')
fixture.componentRef.setInput('aiEnabled', true)
fixture.componentRef.setInput('source', SuggestionSource.Both)
fixture.componentRef.setInput('fetchedSources', [SuggestionSource.ML])
fixture.componentRef.setInput('suggestions', { tags: [1] })
fixture.detectChanges()
component.clickSuggest()
expect(component.getSuggestions.emit).toHaveBeenCalledWith(
SuggestionSource.Both
)
expect(component.dropdown.isOpen()).toBeTruthy()
})
it('should only show source options when AI is enabled', () => {
expect(
fixture.nativeElement.querySelector('#suggestionSourceML')
).toBeNull()
fixture.componentRef.setInput('aiEnabled', true)
fixture.detectChanges()
fixture.nativeElement
.querySelector('button[title="Suggestion options"]')
.click()
fixture.detectChanges()
expect(
fixture.nativeElement.querySelector('#suggestionSourceML')
).not.toBeNull()
})
it('should emit source changes and never allow no source', () => {
const emitSpy = jest.spyOn(component.sourceChange, 'emit')
component.setSources(true, true)
expect(emitSpy).toHaveBeenCalledWith(SuggestionSource.Both)
component.setSources(true, false)
expect(emitSpy).toHaveBeenCalledWith(SuggestionSource.ML)
component.setSources(false, true)
expect(emitSpy).toHaveBeenCalledWith(SuggestionSource.AI)
emitSpy.mockClear()
component.setSources(false, false)
expect(emitSpy).not.toHaveBeenCalled()
})
it('should indicate a non-default source', () => {
fixture.componentRef.setInput('aiEnabled', true)
fixture.componentRef.setInput('source', SuggestionSource.AI)
fixture.componentRef.setInput('defaultSource', SuggestionSource.AI)
fixture.detectChanges()
expect(fixture.nativeElement.textContent).not.toContain('Not using default')
fixture.componentRef.setInput('source', SuggestionSource.Both)
fixture.detectChanges()
expect(fixture.nativeElement.textContent).toContain('Not using default')
})
}) })
@@ -8,7 +8,6 @@ import {
import { NgbDropdown, NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap' import { NgbDropdown, NgbDropdownModule } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { DocumentSuggestions } from 'src/app/data/document-suggestions' import { DocumentSuggestions } from 'src/app/data/document-suggestions'
import { SuggestionSource } from 'src/app/data/ui-settings'
import { pngxPopperOptions } from 'src/app/utils/popper-options' import { pngxPopperOptions } from 'src/app/utils/popper-options'
@Component({ @Component({
@@ -19,29 +18,21 @@ import { pngxPopperOptions } from 'src/app/utils/popper-options'
}) })
export class SuggestionsDropdownComponent { export class SuggestionsDropdownComponent {
public popperOptions = pngxPopperOptions public popperOptions = pngxPopperOptions
public readonly SuggestionSource = SuggestionSource
@ViewChild('dropdown') dropdown: NgbDropdown @ViewChild('dropdown') dropdown: NgbDropdown
readonly suggestions = input<DocumentSuggestions>(null) readonly suggestions = input<DocumentSuggestions>(null)
readonly aiEnabled = input(false) readonly aiEnabled = input(false)
readonly loading = input(false) readonly loading = input(false)
readonly disabled = input(false) readonly disabled = input(false)
readonly source = input<SuggestionSource>(SuggestionSource.ML)
readonly defaultSource = input<SuggestionSource>(SuggestionSource.ML)
readonly fetchedSources = input<SuggestionSource[]>([])
readonly appliedTags = input<number[]>([]) readonly appliedTags = input<number[]>([])
readonly appliedCorrespondent = input<number>(null) readonly appliedCorrespondent = input<number>(null)
readonly appliedDocumentType = input<number>(null) readonly appliedDocumentType = input<number>(null)
readonly appliedStoragePath = input<number>(null) readonly appliedStoragePath = input<number>(null)
readonly appliedTitle = input<string>(null)
readonly appliedCreated = input<string>(null)
@Output() @Output()
getSuggestions: EventEmitter<SuggestionSource> = new EventEmitter() getSuggestions: EventEmitter<SuggestionsDropdownComponent> =
new EventEmitter()
@Output()
sourceChange: EventEmitter<SuggestionSource> = new EventEmitter()
@Output() @Output()
addTag: EventEmitter<string> = new EventEmitter() addTag: EventEmitter<string> = new EventEmitter()
@@ -62,42 +53,12 @@ export class SuggestionsDropdownComponent {
} }
if (!this.suggestions()) { if (!this.suggestions()) {
this.getSuggestions.emit(this.source()) this.getSuggestions.emit(this)
} else if (this.hasUnfetchedSources) {
// sources changed, fetch the rest and show what we have meanwhile
this.getSuggestions.emit(this.source())
this.dropdown?.open()
} else { } else {
this.dropdown?.toggle() this.dropdown?.toggle()
} }
} }
get useML(): boolean {
return this.source() !== SuggestionSource.AI
}
get useAI(): boolean {
return this.source() !== SuggestionSource.ML
}
get hasUnfetchedSources(): boolean {
const fetched = this.fetchedSources()
return (
(this.useML && !fetched.includes(SuggestionSource.ML)) ||
(this.useAI && !fetched.includes(SuggestionSource.AI))
)
}
public setSources(ml: boolean, ai: boolean) {
if (ml && ai) {
this.sourceChange.emit(SuggestionSource.Both)
} else if (ml) {
this.sourceChange.emit(SuggestionSource.ML)
} else if (ai) {
this.sourceChange.emit(SuggestionSource.AI)
}
}
get novelSuggestions(): number { get novelSuggestions(): number {
return ( return (
(this.suggestions()?.suggested_correspondents?.length ?? 0) + (this.suggestions()?.suggested_correspondents?.length ?? 0) +
@@ -130,28 +91,7 @@ export class SuggestionsDropdownComponent {
} }
get totalSuggestions(): number { get totalSuggestions(): number {
return this.novelSuggestions + this.fieldSuggestions return this.novelSuggestions + this.reusableSuggestions
}
get fieldSuggestions(): number {
return (
this.reusableSuggestions +
this.unappliedTitleSuggestions +
this.unappliedDateSuggestions
)
}
// hide a title or date suggestion equal to the current value
private get unappliedTitleSuggestions(): number {
const title = this.suggestions()?.title
return title && title !== this.appliedTitle() ? 1 : 0
}
private get unappliedDateSuggestions(): number {
const created = this.appliedCreated()
return (this.suggestions()?.dates ?? []).filter(
(date) => !created || date !== created
).length
} }
private countUnapplied(suggested: number[], applied: number[]): number { private countUnapplied(suggested: number[], applied: number[]): number {
@@ -162,7 +102,6 @@ export class SuggestionsDropdownComponent {
get noSuggestions(): boolean { get noSuggestions(): boolean {
const suggestions = this.suggestions() const suggestions = this.suggestions()
return ( return (
!this.loading() &&
suggestions != null && suggestions != null &&
!suggestions.title && !suggestions.title &&
!suggestions.tags?.length && !suggestions.tags?.length &&
@@ -1,46 +0,0 @@
<table class="table table-borderless align-baseline">
<thead>
<tr>
<th i18n>Page</th>
<th i18n>Type</th>
<th i18n>Content</th>
<th></th>
</tr>
</thead>
<tbody>
@for (barcode of barcodes(); track $index) {
<tr>
<td>{{ barcode.page }}</td>
<td class="text-nowrap">{{ barcode.format }}</td>
<td class="text-break">
@if (isLink(barcode.value)) {
<a
[href]="barcode.value"
target="_blank"
rel="noopener noreferrer nofollow"
>{{ barcode.value }}</a
>
} @else {
{{ barcode.value }}
}
</td>
<td class="text-end">
<button
type="button"
class="btn btn-sm btn-outline-primary"
(click)="copy($index)"
title="Copy content"
i18n-title
>
@if (copiedIndex() === $index) {
<i-bs name="clipboard-check"></i-bs>
} @else {
<i-bs name="clipboard"></i-bs>
}
<span class="visually-hidden" i18n>Copy content</span>
</button>
</td>
</tr>
}
</tbody>
</table>
@@ -1,73 +0,0 @@
import { Clipboard } from '@angular/cdk/clipboard'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { DocumentBarcodesComponent } from './document-barcodes.component'
const barcodes = [
{ page: 1, value: 'ASN00123', format: 'Code128' },
{ page: 2, value: 'https://example.com/invoice/4711', format: 'QRCode' },
{ page: 2, value: 'javascript:alert(1)', format: 'QRCode' },
]
describe('DocumentBarcodesComponent', () => {
let component: DocumentBarcodesComponent
let fixture: ComponentFixture<DocumentBarcodesComponent>
let clipboard: Clipboard
beforeEach(async () => {
TestBed.configureTestingModule({
imports: [
DocumentBarcodesComponent,
NgxBootstrapIconsModule.pick(allIcons),
],
}).compileComponents()
fixture = TestBed.createComponent(DocumentBarcodesComponent)
component = fixture.componentInstance
clipboard = TestBed.inject(Clipboard)
fixture.componentRef.setInput('barcodes', barcodes)
fixture.detectChanges()
})
it('should display all barcodes', () => {
const rows = fixture.nativeElement.querySelectorAll('tbody tr')
expect(rows).toHaveLength(3)
expect(rows[0].textContent).toContain('ASN00123')
expect(rows[0].textContent).toContain('Code128')
})
it('should only link http(s) values', () => {
const links = fixture.nativeElement.querySelectorAll('tbody a')
expect(links).toHaveLength(1)
expect(links[0].getAttribute('href')).toEqual(
'https://example.com/invoice/4711'
)
expect(links[0].getAttribute('target')).toEqual('_blank')
})
it('should copy a value and show feedback', () => {
jest.useFakeTimers()
const copySpy = jest.spyOn(clipboard, 'copy').mockReturnValue(true)
const buttons = fixture.nativeElement.querySelectorAll('tbody button')
buttons[0].click()
fixture.detectChanges()
expect(copySpy).toHaveBeenCalledWith('ASN00123')
expect(component.copiedIndex()).toEqual(0)
expect(buttons[0].querySelector('i-bs').getAttribute('name')).toEqual(
'clipboard-check'
)
jest.advanceTimersByTime(3000)
fixture.detectChanges()
expect(component.copiedIndex()).toBeNull()
expect(buttons[0].querySelector('i-bs').getAttribute('name')).toEqual(
'clipboard'
)
jest.useRealTimers()
})
it('should not show feedback if copying failed', () => {
jest.spyOn(clipboard, 'copy').mockReturnValue(false)
component.copy(1)
expect(component.copiedIndex()).toBeNull()
})
})
@@ -1,38 +0,0 @@
import { Clipboard } from '@angular/cdk/clipboard'
import { Component, inject, input, OnDestroy, signal } from '@angular/core'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { DocumentBarcode } from 'src/app/data/document-barcode'
@Component({
selector: 'pngx-document-barcodes',
templateUrl: './document-barcodes.component.html',
imports: [NgxBootstrapIconsModule],
})
export class DocumentBarcodesComponent implements OnDestroy {
private readonly clipboard = inject(Clipboard)
readonly barcodes = input<DocumentBarcode[]>([])
readonly copiedIndex = signal<number>(null)
private copyTimeout: ReturnType<typeof setTimeout>
public isLink(value: string): boolean {
try {
const url = new URL(value.trim())
return ['http:', 'https:'].includes(url.protocol) && !!url.host
} catch {
return false
}
}
public copy(index: number) {
if (!this.clipboard.copy(this.barcodes()[index].value)) return
this.copiedIndex.set(index)
clearTimeout(this.copyTimeout)
this.copyTimeout = setTimeout(() => this.copiedIndex.set(null), 3000)
}
ngOnDestroy(): void {
clearTimeout(this.copyTimeout)
}
}
@@ -134,17 +134,11 @@
[loading]="suggestionsLoading()" [loading]="suggestionsLoading()"
[suggestions]="suggestions()" [suggestions]="suggestions()"
[aiEnabled]="aiEnabled" [aiEnabled]="aiEnabled"
[source]="suggestionSource"
[defaultSource]="defaultSuggestionSource"
[fetchedSources]="fetchedSuggestionSources()"
[appliedTags]="documentForm.value.tags" [appliedTags]="documentForm.value.tags"
[appliedCorrespondent]="documentForm.value.correspondent" [appliedCorrespondent]="documentForm.value.correspondent"
[appliedDocumentType]="documentForm.value.document_type" [appliedDocumentType]="documentForm.value.document_type"
[appliedStoragePath]="documentForm.value.storage_path" [appliedStoragePath]="documentForm.value.storage_path"
[appliedTitle]="documentForm.value.title" (getSuggestions)="getSuggestions()"
[appliedCreated]="documentForm.value.created"
(getSuggestions)="getSuggestions($event)"
(sourceChange)="suggestionSourceOverride.set($event)"
(addTag)="createTag($event)" (addTag)="createTag($event)"
(addDocumentType)="createDocumentType($event)" (addDocumentType)="createDocumentType($event)"
(addCorrespondent)="createCorrespondent($event)"> (addCorrespondent)="createCorrespondent($event)">
@@ -356,10 +350,6 @@
</table> </table>
} }
@if (metadata()?.barcodes?.length > 0) {
<h6 i18n>Barcodes</h6>
<pngx-document-barcodes [barcodes]="metadata().barcodes"></pngx-document-barcodes>
}
@if (metadata()?.original_metadata?.length > 0) { @if (metadata()?.original_metadata?.length > 0) {
<pngx-metadata-collapse i18n-title title="Original document metadata" [metadata]="metadata()?.original_metadata"></pngx-metadata-collapse> <pngx-metadata-collapse i18n-title title="Original document metadata" [metadata]="metadata()?.original_metadata"></pngx-metadata-collapse>
} }
@@ -43,7 +43,7 @@ import {
} from 'src/app/data/filter-rule-type' } from 'src/app/data/filter-rule-type'
import { StoragePath } from 'src/app/data/storage-path' import { StoragePath } from 'src/app/data/storage-path'
import { Tag } from 'src/app/data/tag' import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS, SuggestionSource } from 'src/app/data/ui-settings' import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { PermissionsGuard } from 'src/app/guards/permissions.guard' import { PermissionsGuard } from 'src/app/guards/permissions.guard'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe' import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe' import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -662,45 +662,6 @@ describe('DocumentDetailComponent', () => {
) )
}) })
it.each([
['tag', 'createTag', 'tags', 'suggested_tags'],
[
'document type',
'createDocumentType',
'document_type',
'suggested_document_types',
],
[
'correspondent',
'createCorrespondent',
'correspondent',
'suggested_correspondents',
],
])(
'should create a %s after ML-only suggestions',
(_, method, field, suggestedField) => {
initNormally()
component.suggestions.set({ tags: [1] })
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
component[method]('New value')
openModal.componentInstance.succeeded.next({
id: 12,
name: 'New value',
is_inbox_tag: false,
color: '#ff0000',
text_color: '#000000',
})
if (field === 'tags') {
expect(component.tagsInput.value).toContain(12)
} else {
expect(component.documentForm.get(field).value).toBe(12)
}
expect(component.suggestions()[suggestedField]).toEqual([])
}
)
it('should support creating storage path', () => { it('should support creating storage path', () => {
initNormally() initNormally()
let openModal: NgbModalRef let openModal: NgbModalRef
@@ -1567,113 +1528,6 @@ describe('DocumentDetailComponent', () => {
expect(component.suggestionsLoading()).toBeFalsy() expect(component.suggestionsLoading()).toBeFalsy()
}) })
it('should get and merge ML and AI suggestions when source is both', () => {
settingsService.set(
SETTINGS_KEYS.DOCUMENT_EDITING_SUGGESTION_SOURCE,
SuggestionSource.Both
)
const getSetting = settingsService.get.bind(settingsService)
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) =>
key === SETTINGS_KEYS.AI_ENABLED ? true : getSetting(key)
)
const suggestionsSpy = jest
.spyOn(documentService, 'getSuggestions')
.mockReturnValue(of({ tags: [42], dates: ['2024-01-01'] }))
const aiSuggestionsSpy = jest
.spyOn(documentService, 'getAiSuggestions')
.mockReturnValue(
of({ title: 'AI title', tags: [42, 43], suggested_tags: ['New'] })
)
initNormally()
expect(suggestionsSpy).toHaveBeenCalled()
expect(aiSuggestionsSpy).toHaveBeenCalled()
expect(component.suggestions().title).toEqual('AI title')
expect(component.suggestions().tags).toEqual([42, 43])
expect(component.suggestions().suggested_tags).toEqual(['New'])
expect(component.suggestions().dates).toEqual(['2024-01-01'])
})
it('should only fetch sources not yet fetched for the document', () => {
settingsService.set(SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST, false)
settingsService.set(
SETTINGS_KEYS.DOCUMENT_EDITING_SUGGESTION_SOURCE,
SuggestionSource.ML
)
const getSetting = settingsService.get.bind(settingsService)
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) =>
key === SETTINGS_KEYS.AI_ENABLED ? true : getSetting(key)
)
const suggestionsSpy = jest
.spyOn(documentService, 'getSuggestions')
.mockReturnValue(of({ tags: [42] }))
const aiSuggestionsSpy = jest
.spyOn(documentService, 'getAiSuggestions')
.mockReturnValue(of({ tags: [43] }))
initNormally()
component.getSuggestions()
expect(suggestionsSpy).toHaveBeenCalledTimes(1)
expect(aiSuggestionsSpy).not.toHaveBeenCalled()
component.getSuggestions(SuggestionSource.Both)
expect(suggestionsSpy).toHaveBeenCalledTimes(1)
expect(aiSuggestionsSpy).toHaveBeenCalledTimes(1)
expect(component.suggestions().tags).toEqual([42, 43])
component.getSuggestions(SuggestionSource.Both)
expect(suggestionsSpy).toHaveBeenCalledTimes(1)
expect(aiSuggestionsSpy).toHaveBeenCalledTimes(1)
})
it('should use the per-document source override and reset it on document change', () => {
settingsService.set(SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST, false)
const getSetting = settingsService.get.bind(settingsService)
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) =>
key === SETTINGS_KEYS.AI_ENABLED ? true : getSetting(key)
)
initNormally()
expect(component.suggestionSource).toEqual(SuggestionSource.AI)
component.suggestionSourceOverride.set(SuggestionSource.ML)
expect(component.suggestionSource).toEqual(SuggestionSource.ML)
jest
.spyOn(documentService, 'get')
.mockReturnValueOnce(of(Object.assign({}, doc)))
;(component as any).loadDocument(doc.id, true)
expect(component.suggestionSourceOverride()).toBeNull()
expect(component.fetchedSuggestionSources()).toEqual([])
})
it('should keep suggestions from one source if the other fails', () => {
settingsService.set(
SETTINGS_KEYS.DOCUMENT_EDITING_SUGGESTION_SOURCE,
SuggestionSource.Both
)
const getSetting = settingsService.get.bind(settingsService)
jest
.spyOn(settingsService, 'get')
.mockImplementation((key) =>
key === SETTINGS_KEYS.AI_ENABLED ? true : getSetting(key)
)
const errorSpy = jest.spyOn(toastService, 'showError')
jest
.spyOn(documentService, 'getSuggestions')
.mockReturnValue(of({ tags: [42] }))
jest
.spyOn(documentService, 'getAiSuggestions')
.mockReturnValue(throwError(() => new Error('failed')))
initNormally()
expect(errorSpy).toHaveBeenCalled()
expect(component.suggestions().tags).toEqual([42])
expect(component.fetchedSuggestionSources()).toEqual([SuggestionSource.ML])
})
it('should show error if needed for get suggestions', () => { it('should show error if needed for get suggestions', () => {
const suggestionsSpy = jest.spyOn(documentService, 'getSuggestions') const suggestionsSpy = jest.spyOn(documentService, 'getSuggestions')
const errorSpy = jest.spyOn(toastService, 'showError') const errorSpy = jest.spyOn(toastService, 'showError')
@@ -28,7 +28,7 @@ import {
import { dirtyCheck, DirtyComponent } from '@ngneat/dirty-check-forms' import { dirtyCheck, DirtyComponent } from '@ngneat/dirty-check-forms'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { DeviceDetectorService } from 'ngx-device-detector' import { DeviceDetectorService } from 'ngx-device-detector'
import { BehaviorSubject, merge, Observable, of, Subject, timer } from 'rxjs' import { BehaviorSubject, Observable, of, Subject, timer } from 'rxjs'
import { import {
catchError, catchError,
debounceTime, debounceTime,
@@ -48,10 +48,7 @@ import { DataType } from 'src/app/data/datatype'
import { Document, DocumentVersionInfo } from 'src/app/data/document' import { Document, DocumentVersionInfo } from 'src/app/data/document'
import { DocumentMetadata } from 'src/app/data/document-metadata' import { DocumentMetadata } from 'src/app/data/document-metadata'
import { DocumentNote } from 'src/app/data/document-note' import { DocumentNote } from 'src/app/data/document-note'
import { import { DocumentSuggestions } from 'src/app/data/document-suggestions'
DocumentSuggestions,
mergeSuggestions,
} from 'src/app/data/document-suggestions'
import { DocumentType } from 'src/app/data/document-type' import { DocumentType } from 'src/app/data/document-type'
import { FilterRule } from 'src/app/data/filter-rule' import { FilterRule } from 'src/app/data/filter-rule'
import { import {
@@ -66,7 +63,7 @@ import {
import { ObjectWithId } from 'src/app/data/object-with-id' import { ObjectWithId } from 'src/app/data/object-with-id'
import { StoragePath } from 'src/app/data/storage-path' import { StoragePath } from 'src/app/data/storage-path'
import { Tag } from 'src/app/data/tag' import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS, SuggestionSource } from 'src/app/data/ui-settings' import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { User } from 'src/app/data/user' import { User } from 'src/app/data/user'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive' import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe' import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
@@ -135,7 +132,6 @@ import { ShareLinksDialogComponent } from '../common/share-links-dialog/share-li
import { SuggestionsDropdownComponent } from '../common/suggestions-dropdown/suggestions-dropdown.component' import { SuggestionsDropdownComponent } from '../common/suggestions-dropdown/suggestions-dropdown.component'
import { DocumentNotesComponent } from '../document-notes/document-notes.component' import { DocumentNotesComponent } from '../document-notes/document-notes.component'
import { ComponentWithPermissions } from '../with-permissions/with-permissions.component' import { ComponentWithPermissions } from '../with-permissions/with-permissions.component'
import { DocumentBarcodesComponent } from './document-barcodes/document-barcodes.component'
import { DocumentHistoryComponent } from './document-history/document-history.component' import { DocumentHistoryComponent } from './document-history/document-history.component'
import { DocumentVersionDropdownComponent } from './document-version-dropdown/document-version-dropdown.component' import { DocumentVersionDropdownComponent } from './document-version-dropdown/document-version-dropdown.component'
import { MetadataCollapseComponent } from './metadata-collapse/metadata-collapse.component' import { MetadataCollapseComponent } from './metadata-collapse/metadata-collapse.component'
@@ -178,7 +174,6 @@ interface IncomingDocumentUpdate {
DateComponent, DateComponent,
DocumentLinkComponent, DocumentLinkComponent,
MetadataCollapseComponent, MetadataCollapseComponent,
DocumentBarcodesComponent,
PermissionsFormComponent, PermissionsFormComponent,
SelectComponent, SelectComponent,
TagsComponent, TagsComponent,
@@ -245,10 +240,6 @@ export class DocumentDetailComponent
private readonly autoSuggestSetting = this.settings.getSignal<boolean>( private readonly autoSuggestSetting = this.settings.getSignal<boolean>(
SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST SETTINGS_KEYS.DOCUMENT_EDITING_AUTO_SUGGEST
) )
private readonly suggestionSourceSetting =
this.settings.getSignal<SuggestionSource>(
SETTINGS_KEYS.DOCUMENT_EDITING_SUGGESTION_SOURCE
)
private readonly hiddenFieldsSetting = this.settings.getSignal< private readonly hiddenFieldsSetting = this.settings.getSignal<
DocumentDetailFieldID[] DocumentDetailFieldID[]
>(SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS) >(SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS)
@@ -270,9 +261,6 @@ export class DocumentDetailComponent
readonly metadata = signal<DocumentMetadata>(undefined) readonly metadata = signal<DocumentMetadata>(undefined)
readonly suggestions = signal<DocumentSuggestions>(undefined) readonly suggestions = signal<DocumentSuggestions>(undefined)
readonly suggestionsLoading = signal(false) readonly suggestionsLoading = signal(false)
// per-document, resets on navigation
readonly suggestionSourceOverride = signal<SuggestionSource>(null)
readonly fetchedSuggestionSources = signal<SuggestionSource[]>([])
readonly users = signal<User[]>(undefined) readonly users = signal<User[]>(undefined)
readonly title = signal<string>(undefined) readonly title = signal<string>(undefined)
@@ -377,15 +365,6 @@ export class DocumentDetailComponent
return this.autoSuggestSetting() return this.autoSuggestSetting()
} }
get defaultSuggestionSource(): SuggestionSource {
return this.aiEnabled ? this.suggestionSourceSetting() : SuggestionSource.ML
}
get suggestionSource(): SuggestionSource {
if (!this.aiEnabled) return SuggestionSource.ML
return this.suggestionSourceOverride() ?? this.defaultSuggestionSource
}
get archiveContentRenderType(): ContentRenderType { get archiveContentRenderType(): ContentRenderType {
const hasArchiveVersion = const hasArchiveVersion =
this.metadata()?.has_archive_version ?? this.metadata()?.has_archive_version ??
@@ -611,8 +590,6 @@ export class DocumentDetailComponent
} }
this.documentId.set(doc.id) this.documentId.set(doc.id)
this.suggestions.set(null) this.suggestions.set(null)
this.suggestionSourceOverride.set(null)
this.fetchedSuggestionSources.set([])
const openDocument = this.openDocumentService.getOpenDocument( const openDocument = this.openDocumentService.getOpenDocument(
this.documentId() this.documentId()
) )
@@ -1100,44 +1077,29 @@ export class DocumentDetailComponent
return this.documentForm.get('custom_fields') as FormArray return this.documentForm.get('custom_fields') as FormArray
} }
getSuggestions(source: SuggestionSource = this.suggestionSource) { getSuggestions() {
const sources = (
source === SuggestionSource.Both
? [SuggestionSource.ML, SuggestionSource.AI]
: [source]
).filter((s) => !this.fetchedSuggestionSources().includes(s))
if (!sources.length) return
this.suggestionsLoading.set(true) this.suggestionsLoading.set(true)
merge( const suggestionsObservable = this.aiEnabled
...sources.map((s) => ? this.documentsService.getAiSuggestions(this.documentId())
(s === SuggestionSource.AI : this.documentsService.getSuggestions(this.documentId())
? this.documentsService.getAiSuggestions(this.documentId()) suggestionsObservable
: this.documentsService.getSuggestions(this.documentId())
).pipe(
first(),
map((result) => ({ source: s, result })),
catchError((error) => {
this.toastService.showError(
$localize`Error retrieving suggestions.`,
error
)
return of(null)
})
)
)
)
.pipe( .pipe(
first(),
takeUntil(this.unsubscribeNotifier), takeUntil(this.unsubscribeNotifier),
takeUntil(this.docChangeNotifier), takeUntil(this.docChangeNotifier),
finalize(() => this.suggestionsLoading.set(false)) finalize(() => this.suggestionsLoading.set(false))
) )
.subscribe((response) => { .subscribe({
if (!response) return next: (result) => {
this.fetchedSuggestionSources.update((f) => [...f, response.source]) this.suggestions.set(result)
this.suggestions.set( },
mergeSuggestions(this.suggestions(), response.result) error: (error) => {
) this.suggestions.set(null)
this.toastService.showError(
$localize`Error retrieving suggestions.`,
error
)
},
}) })
} }
@@ -1154,7 +1116,7 @@ export class DocumentDetailComponent
if (this.suggestions()) { if (this.suggestions()) {
this.suggestions.set({ this.suggestions.set({
...this.suggestions(), ...this.suggestions(),
suggested_tags: (this.suggestions().suggested_tags ?? []).filter( suggested_tags: this.suggestions().suggested_tags.filter(
(tag) => tag !== newTag.name (tag) => tag !== newTag.name
), ),
}) })
@@ -1193,12 +1155,10 @@ export class DocumentDetailComponent
this.documentForm.get('document_type').setValue(newDocumentType.id) this.documentForm.get('document_type').setValue(newDocumentType.id)
this.documentForm.get('document_type').markAsDirty() this.documentForm.get('document_type').markAsDirty()
if (this.suggestions()) { if (this.suggestions()) {
this.suggestions.set({ this.suggestions().suggested_document_types =
...this.suggestions(), this.suggestions().suggested_document_types.filter(
suggested_document_types: ( (dt) => dt !== newName
this.suggestions().suggested_document_types ?? [] )
).filter((dt) => dt !== newName),
})
} }
}) })
} }
@@ -1225,12 +1185,10 @@ export class DocumentDetailComponent
this.documentForm.get('correspondent').setValue(newCorrespondent.id) this.documentForm.get('correspondent').setValue(newCorrespondent.id)
this.documentForm.get('correspondent').markAsDirty() this.documentForm.get('correspondent').markAsDirty()
if (this.suggestions()) { if (this.suggestions()) {
this.suggestions.set({ this.suggestions().suggested_correspondents =
...this.suggestions(), this.suggestions().suggested_correspondents.filter(
suggested_correspondents: ( (c) => c !== newName
this.suggestions().suggested_correspondents ?? [] )
).filter((c) => c !== newName),
})
} }
}) })
} }
@@ -146,19 +146,6 @@ describe('DocumentListComponent', () => {
expect(reloadSpy).toHaveBeenCalled() expect(reloadSpy).toHaveBeenCalled()
}) })
it('should stop reloading on document deleted after destroy', () => {
const reloadSpy = jest.spyOn(documentListService, 'reload')
const documentDeletedSubject = new Subject<boolean>()
jest
.spyOn(websocketStatusService, 'onDocumentDeleted')
.mockReturnValue(documentDeletedSubject)
fixture.detectChanges()
fixture.destroy()
reloadSpy.mockClear()
documentDeletedSubject.next(true)
expect(reloadSpy).not.toHaveBeenCalled()
})
it('should show score sort fields on fulltext queries', () => { it('should show score sort fields on fulltext queries', () => {
documentListService.setFilterRules([ documentListService.setFilterRules([
{ {
@@ -270,12 +270,9 @@ export class DocumentListComponent
this.list.reload() this.list.reload()
}) })
this.websocketStatusService this.websocketStatusService.onDocumentDeleted().subscribe(() => {
.onDocumentDeleted() this.list.reload()
.pipe(takeUntil(this.unsubscribeNotifier)) })
.subscribe(() => {
this.list.reload()
})
this.route.paramMap this.route.paramMap
.pipe( .pipe(
-7
View File
@@ -1,7 +0,0 @@
export interface DocumentBarcode {
page: number
value: string
format: string
}
-4
View File
@@ -1,5 +1,3 @@
import { DocumentBarcode } from './document-barcode'
export interface DocumentMetadata { export interface DocumentMetadata {
original_checksum?: string original_checksum?: string
@@ -14,6 +12,4 @@ export interface DocumentMetadata {
has_archive_version?: boolean has_archive_version?: boolean
lang?: string lang?: string
barcodes?: DocumentBarcode[]
} }
@@ -15,33 +15,3 @@ export interface DocumentSuggestions {
dates?: string[] // ISO-formatted date string e.g. 2022-11-03 dates?: string[] // ISO-formatted date string e.g. 2022-11-03
} }
const union = <T>(a: T[] = [], b: T[] = []): T[] => [...new Set([...a, ...b])]
export function mergeSuggestions(
a: DocumentSuggestions,
b: DocumentSuggestions
): DocumentSuggestions {
if (!a) return b
return {
title: a.title || b.title,
tags: union(a.tags, b.tags),
suggested_tags: union(a.suggested_tags, b.suggested_tags),
correspondents: union(a.correspondents, b.correspondents),
suggested_correspondents: union(
a.suggested_correspondents,
b.suggested_correspondents
),
document_types: union(a.document_types, b.document_types),
suggested_document_types: union(
a.suggested_document_types,
b.suggested_document_types
),
storage_paths: union(a.storage_paths, b.storage_paths),
suggested_storage_paths: union(
a.suggested_storage_paths,
b.suggested_storage_paths
),
dates: union(a.dates, b.dates),
}
}
-17
View File
@@ -330,13 +330,6 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_CONSUMER_TAG_BARCODE_SPLIT', config_key: 'PAPERLESS_CONSUMER_TAG_BARCODE_SPLIT',
category: ConfigCategory.Barcode, category: ConfigCategory.Barcode,
}, },
{
key: 'barcode_store_values',
title: $localize`Store Barcode Contents`,
type: ConfigOptionType.Boolean,
config_key: 'PAPERLESS_CONSUMER_STORE_BARCODE_VALUES',
category: ConfigCategory.Barcode,
},
{ {
key: 'ai_enabled', key: 'ai_enabled',
title: $localize`AI Enabled`, title: $localize`AI Enabled`,
@@ -360,14 +353,6 @@ export const PaperlessConfigOptions: ConfigOption[] = [
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_MODEL', config_key: 'PAPERLESS_AI_LLM_EMBEDDING_MODEL',
category: ConfigCategory.AI, category: ConfigCategory.AI,
}, },
{
key: 'llm_embedding_api_key',
title: $localize`LLM Embedding API Key`,
type: ConfigOptionType.Password,
note: $localize`Used for embeddings when set, otherwise LLM API key is used.`,
config_key: 'PAPERLESS_AI_LLM_EMBEDDING_API_KEY',
category: ConfigCategory.AI,
},
{ {
key: 'llm_embedding_endpoint', key: 'llm_embedding_endpoint',
title: $localize`LLM Embedding Endpoint`, title: $localize`LLM Embedding Endpoint`,
@@ -465,7 +450,6 @@ export interface PaperlessConfig extends ObjectWithId {
barcode_enable_tag: boolean barcode_enable_tag: boolean
barcode_tag_mapping: object barcode_tag_mapping: object
barcode_tag_split: boolean barcode_tag_split: boolean
barcode_store_values: boolean
remote_ocr_engine: string remote_ocr_engine: string
remote_ocr_api_key: string remote_ocr_api_key: string
remote_ocr_endpoint: string remote_ocr_endpoint: string
@@ -473,7 +457,6 @@ export interface PaperlessConfig extends ObjectWithId {
ai_enabled: boolean ai_enabled: boolean
llm_embedding_backend: string llm_embedding_backend: string
llm_embedding_model: string llm_embedding_model: string
llm_embedding_api_key: string
llm_embedding_endpoint: string llm_embedding_endpoint: string
llm_embedding_chunk_size: number llm_embedding_chunk_size: number
llm_context_size: number llm_context_size: number
-13
View File
@@ -20,12 +20,6 @@ export enum GlobalSearchType {
TITLE_CONTENT = 'title-content', TITLE_CONTENT = 'title-content',
} }
export enum SuggestionSource {
ML = 'ml',
AI = 'ai',
Both = 'both',
}
export enum CollapsibleSection { export enum CollapsibleSection {
ATTRIBUTES = 'attributes', ATTRIBUTES = 'attributes',
} }
@@ -104,8 +98,6 @@ export const SETTINGS_KEYS = {
'general-settings:document-editing:overlay-thumbnail', 'general-settings:document-editing:overlay-thumbnail',
DOCUMENT_EDITING_AUTO_SUGGEST: DOCUMENT_EDITING_AUTO_SUGGEST:
'general-settings:document-editing:auto-suggest', 'general-settings:document-editing:auto-suggest',
DOCUMENT_EDITING_SUGGESTION_SOURCE:
'general-settings:document-editing:suggestion-source',
DOCUMENT_DETAILS_HIDDEN_FIELDS: DOCUMENT_DETAILS_HIDDEN_FIELDS:
'general-settings:document-details:hidden-fields', 'general-settings:document-details:hidden-fields',
SEARCH_DB_ONLY: 'general-settings:search:db-only', SEARCH_DB_ONLY: 'general-settings:search:db-only',
@@ -334,11 +326,6 @@ export const SETTINGS: UiSetting[] = [
type: 'boolean', type: 'boolean',
default: true, default: true,
}, },
{
key: SETTINGS_KEYS.DOCUMENT_EDITING_SUGGESTION_SOURCE,
type: 'string',
default: SuggestionSource.AI,
},
{ {
key: SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS, key: SETTINGS_KEYS.DOCUMENT_DETAILS_HIDDEN_FIELDS,
type: 'array', type: 'array',
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
Loaded 100 of 222 files, more files were not shown because too many files have changed in this diff. Show more