Compare commits

..
13 Commits
Author SHA1 Message Date
GitHub Actions bfe2d92619 Auto translate strings 2026-07-25 07:13:46 +00:00
shamoonandGitHub 4f0845b094 Fixhancement: PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT (#13281) 2026-07-25 00:12:12 -07:00
shamoonandGitHub dee568e2a6 Tweak: adjust top navbar wrapping when AI chat button visible (#13280) 2026-07-24 23:50:10 -07:00
shamoon f60a2af841 Merge branch 'main' into dev 2026-07-24 19:08:23 -07:00
Trenton HandGitHub 33cd48a17f Fix: preserve document fields during Gotenberg conversion (#13271)
Gotenberg's LibreOffice route enables updateIndexes by default, which refreshes dynamic fields (e.g. auto-dates) to the current date. Disable it so field values are preserved as authored.
2026-07-24 13:59:51 -07:00
Trenton HandGitHub d76dbf5366 Documentation: Add the NumPy CPU baseline increase to the migration guide (#13269) 2026-07-24 09:18:07 -07:00
Trenton HandGitHub b66182cd7b Fix: Makes the email date aware as soon as possible during parsing (#13266) 2026-07-24 15:39:25 +00:00
Trenton HandGitHub 3fe6562c57 Fix: Handle a plain string as Celery sometimes provides for the traceback (#13267) 2026-07-24 15:27:57 +00:00
Trenton HandGitHub 737d568f80 Fix: Emit the torch index into the requirements.txt for people still using it (#13265) 2026-07-24 08:07:52 -07:00
Matthias MastandGitHub b992f9fc05 Fix: handle notes without a user when building the search index (#13260) 2026-07-24 06:39:11 -07:00
github-actions[bot]GitHubgithub-actions <41898282+github-actions[bot]@users.noreply.github.com>
ed714b8a5a Changelog v3.0.2 - GHA (#13244)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-23 18:56:06 -07:00
shamoon e0ab6a2e75 Merge branch 'dev' 2026-07-23 18:13:49 -07:00
github-actions[bot]GitHubgithub-actions <41898282+github-actions[bot]@users.noreply.github.com>
90ea4f27c4 Changelog v3.0.1 - GHA (#13240)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-23 17:59:34 -07:00
15 changed files with 312 additions and 60 deletions
+1 -1
View File
@@ -86,7 +86,7 @@ jobs:
# ---- Prepare Release ---- # ---- Prepare Release ----
- name: Generate requirements file - name: Generate requirements file
run: | run: |
uv export --quiet --no-dev --all-extras --format requirements-txt --output-file requirements.txt uv export --quiet --no-dev --all-extras --emit-index-url --format requirements-txt --output-file requirements.txt
- name: Compile messages - name: Compile messages
env: env:
PAPERLESS_SECRET_KEY: "ci-release-not-a-real-secret" PAPERLESS_SECRET_KEY: "ci-release-not-a-real-secret"
+59
View File
@@ -1,5 +1,64 @@
# Changelog # Changelog
## paperless-ngx 3.0.2
### Bug Fixes
- Fix: fix broken migration in 3.0.1 [@shamoon](https://github.com/shamoon) ([#13242](https://github.com/paperless-ngx/paperless-ngx/pull/13242))
### All App Changes
- Fix: fix broken migration in 3.0.1 [@shamoon](https://github.com/shamoon) ([#13242](https://github.com/paperless-ngx/paperless-ngx/pull/13242))
## paperless-ngx 3.0.1
### Bug Fixes
- Fix: selection\_data re-derives the filtered document set 5 times over [@stumpylog](https://github.com/stumpylog) ([#13229](https://github.com/paperless-ngx/paperless-ngx/pull/13229))
- Fix: exclude source document from its own RAG similarity results [@stumpylog](https://github.com/stumpylog) ([#13233](https://github.com/paperless-ngx/paperless-ngx/pull/13233))
- Fix: split mailrule maximum\_age clamp into its own migration [@stumpylog](https://github.com/stumpylog) ([#13231](https://github.com/paperless-ngx/paperless-ngx/pull/13231))
- Chore: mark yes in confirm button for translation [@shamoon](https://github.com/shamoon) ([#13225](https://github.com/paperless-ngx/paperless-ngx/pull/13225))
- Fix: correct URL for W001 check [@shamoon](https://github.com/shamoon) ([#13220](https://github.com/paperless-ngx/paperless-ngx/pull/13220))
- Fix: ensure create dialog uses correct attribute type [@shamoon](https://github.com/shamoon) ([#13221](https://github.com/paperless-ngx/paperless-ngx/pull/13221))
- Fix: correct database engine from postgres to postgresql [@OberstVonGatow](https://github.com/OberstVonGatow) ([#13213](https://github.com/paperless-ngx/paperless-ngx/pull/13213))
- Fix: remove unnecessary .distinct() dominating document list queries [@stumpylog](https://github.com/stumpylog) ([#13205](https://github.com/paperless-ngx/paperless-ngx/pull/13205))
- Fix: also wrap non-breaking words in chat messages, handle whitespace [@shamoon](https://github.com/shamoon) ([#13211](https://github.com/paperless-ngx/paperless-ngx/pull/13211))
- Fix: tag/custom-field document\_count scales badly with tag count [@stumpylog](https://github.com/stumpylog) ([#13203](https://github.com/paperless-ngx/paperless-ngx/pull/13203))
- Fix: batch document user\_can\_change checks to avoid per-row N+1 [@stumpylog](https://github.com/stumpylog) ([#13204](https://github.com/paperless-ngx/paperless-ngx/pull/13204))
- Fix: fix app title restoration [@shamoon](https://github.com/shamoon) ([#13208](https://github.com/paperless-ngx/paperless-ngx/pull/13208))
- Fix: handle long wrapping titles in AI chat document list [@shamoon](https://github.com/shamoon) ([#13206](https://github.com/paperless-ngx/paperless-ngx/pull/13206))
### Dependencies
- Chore(deps-dev): Bump postcss from 8.5.6 to 8.5.22 in /src/paperless\_mail/templates in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13236](https://github.com/paperless-ngx/paperless-ngx/pull/13236))
- Chore(deps): Bump pyasn1 from 0.6.3 to 0.6.4 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13237](https://github.com/paperless-ngx/paperless-ngx/pull/13237))
- Chore(deps): Bump the uv group across 1 directory with 8 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13189](https://github.com/paperless-ngx/paperless-ngx/pull/13189))
### All App Changes
<details>
<summary>17 changes</summary>
- Chore(deps-dev): Bump postcss from 8.5.6 to 8.5.22 in /src/paperless\_mail/templates in the npm\_and\_yarn group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13236](https://github.com/paperless-ngx/paperless-ngx/pull/13236))
- Fix: selection\_data re-derives the filtered document set 5 times over [@stumpylog](https://github.com/stumpylog) ([#13229](https://github.com/paperless-ngx/paperless-ngx/pull/13229))
- Chore: reword "Settings > Default permissions" description for clarity [@NotaInutilis](https://github.com/NotaInutilis) ([#13232](https://github.com/paperless-ngx/paperless-ngx/pull/13232))
- Chore(deps): Bump pyasn1 from 0.6.3 to 0.6.4 in the uv group across 1 directory @[dependabot[bot]](https://github.com/apps/dependabot) ([#13237](https://github.com/paperless-ngx/paperless-ngx/pull/13237))
- Chore(deps): Bump the uv group across 1 directory with 8 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#13189](https://github.com/paperless-ngx/paperless-ngx/pull/13189))
- Fix: exclude source document from its own RAG similarity results [@stumpylog](https://github.com/stumpylog) ([#13233](https://github.com/paperless-ngx/paperless-ngx/pull/13233))
- Fix: split mailrule maximum\_age clamp into its own migration [@stumpylog](https://github.com/stumpylog) ([#13231](https://github.com/paperless-ngx/paperless-ngx/pull/13231))
- Chore: mark yes in confirm button for translation [@shamoon](https://github.com/shamoon) ([#13225](https://github.com/paperless-ngx/paperless-ngx/pull/13225))
- Fix: correct URL for W001 check [@shamoon](https://github.com/shamoon) ([#13220](https://github.com/paperless-ngx/paperless-ngx/pull/13220))
- Fix: ensure create dialog uses correct attribute type [@shamoon](https://github.com/shamoon) ([#13221](https://github.com/paperless-ngx/paperless-ngx/pull/13221))
- Fix: correct database engine from postgres to postgresql [@OberstVonGatow](https://github.com/OberstVonGatow) ([#13213](https://github.com/paperless-ngx/paperless-ngx/pull/13213))
- Fix: remove unnecessary .distinct() dominating document list queries [@stumpylog](https://github.com/stumpylog) ([#13205](https://github.com/paperless-ngx/paperless-ngx/pull/13205))
- Fix: also wrap non-breaking words in chat messages, handle whitespace [@shamoon](https://github.com/shamoon) ([#13211](https://github.com/paperless-ngx/paperless-ngx/pull/13211))
- Fix: tag/custom-field document\_count scales badly with tag count [@stumpylog](https://github.com/stumpylog) ([#13203](https://github.com/paperless-ngx/paperless-ngx/pull/13203))
- Fix: batch document user\_can\_change checks to avoid per-row N+1 [@stumpylog](https://github.com/stumpylog) ([#13204](https://github.com/paperless-ngx/paperless-ngx/pull/13204))
- Fix: fix app title restoration [@shamoon](https://github.com/shamoon) ([#13208](https://github.com/paperless-ngx/paperless-ngx/pull/13208))
- Fix: handle long wrapping titles in AI chat document list [@shamoon](https://github.com/shamoon) ([#13206](https://github.com/paperless-ngx/paperless-ngx/pull/13206))
</details>
## paperless-ngx 3.0.0 ## paperless-ngx 3.0.0
### Breaking Changes ### Breaking Changes
+21 -7
View File
@@ -522,22 +522,36 @@ do CORS calls. Set this to your public domain name.
fail2ban with log entries for failed authorization attempts. Value should be fail2ban with log entries for failed authorization attempts. Value should be
IP address(es). IP address(es).
This setting also controls allauth's By default, this setting also controls allauth's trusted proxy count,
[`ALLAUTH_TRUSTED_PROXY_COUNT`](https://docs.allauth.org/en/latest/account/configuration.html), which is set to the number of proxies listed here. Override that default
which is set to the number of proxies listed here. Without this, with [`PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT`](#PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT)
allauth cannot determine the client IP address for rate limiting when when the list length does not match the number of proxy hops.
running behind a reverse proxy, resulting in a `403 Forbidden` on login.
Defaults to empty string. Defaults to empty string.
#### [`PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT=<integer>`](#PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT) {#PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT}
: Sets allauth's
[`ALLAUTH_TRUSTED_PROXY_COUNT`](https://docs.allauth.org/en/latest/common/rate_limits.html#configuration).
This is the number of trusted proxy **hops** represented in each
`X-Forwarded-For` header, not the number of IP addresses through which those
proxies may be reached. For example, a single dual-stack proxy is one hop even
when its IPv4 and IPv6 addresses are both listed in
[`PAPERLESS_TRUSTED_PROXIES`](#PAPERLESS_TRUSTED_PROXIES).
Only trust `X-Forwarded-For` when untrusted clients cannot connect directly
to Paperless-ngx.
Defaults to the number of entries in `PAPERLESS_TRUSTED_PROXIES`.
#### [`PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER=<header-name>`](#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER) {#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER} #### [`PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER=<header-name>`](#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER) {#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER}
: Sets allauth's : Sets allauth's
[`ALLAUTH_TRUSTED_CLIENT_IP_HEADER`](https://docs.allauth.org/en/latest/account/configuration.html). [`ALLAUTH_TRUSTED_CLIENT_IP_HEADER`](https://docs.allauth.org/en/latest/common/rate_limits.html#configuration).
Use this when your reverse proxy sets a dedicated header for the real Use this when your reverse proxy sets a dedicated header for the real
client IP instead of `X-Forwarded-For`, for example `X-Real-IP` (nginx) client IP instead of `X-Forwarded-For`, for example `X-Real-IP` (nginx)
or `CF-Connecting-IP` (Cloudflare). When set, this takes precedence over or `CF-Connecting-IP` (Cloudflare). When set, this takes precedence over
[`PAPERLESS_TRUSTED_PROXIES`](#PAPERLESS_TRUSTED_PROXIES). `PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT`.
Defaults to none. Defaults to none.
+65 -4
View File
@@ -133,7 +133,7 @@ PAPERLESS_DB_OPTIONS="sslmode=require,sslrootcert=/certs/ca.pem,pool.max_size=10
## OCR and Archive File Generation Settings ## OCR and Archive File Generation Settings
The settings that control OCR behaviour and archive file generation have been redesigned. The old settings that coupled these two concerns together are **removed** old values are not silently honoured; a startup warning is logged if any removed variable is still set in your environment. The settings that control OCR behaviour and archive file generation have been redesigned. The old settings that coupled these two concerns together are **removed** - old values are not silently honoured; a startup warning is logged if any removed variable is still set in your environment.
### Removed settings ### Removed settings
@@ -166,7 +166,7 @@ Remove any `PAPERLESS_OCR_SKIP_ARCHIVE_FILE` variable from your environment. If
# v2: skip OCR when text present, always archive # v2: skip OCR when text present, always archive
PAPERLESS_OCR_MODE=skip PAPERLESS_OCR_MODE=skip
# v3: equivalent (auto is the new default) # v3: equivalent (auto is the new default)
# No change needed auto is the default # No change needed - auto is the default
# v2: skip OCR when text present, skip archive too # v2: skip OCR when text present, skip archive too
PAPERLESS_OCR_MODE=skip_noarchive PAPERLESS_OCR_MODE=skip_noarchive
@@ -189,7 +189,7 @@ PAPERLESS_ARCHIVE_FILE_GENERATION=auto
If you use the **remote OCR parser** (Azure AI), note that it always produces a If you use the **remote OCR parser** (Azure AI), note that it always produces a
searchable PDF and stores it as the archive copy. `ARCHIVE_FILE_GENERATION=never` searchable PDF and stores it as the archive copy. `ARCHIVE_FILE_GENERATION=never`
has no effect for documents handled by the remote parser the archive is produced has no effect for documents handled by the remote parser - the archive is produced
unconditionally by the remote engine. unconditionally by the remote engine.
## Search Index (Whoosh -> Tantivy) ## Search Index (Whoosh -> Tantivy)
@@ -324,8 +324,69 @@ option since v1.8.0.
Allauth changed how it determines the client IP address for login rate limiting. Users running Allauth changed how it determines the client IP address for login rate limiting. Users running
behind a reverse proxy may need to set behind a reverse proxy may need to set
[`PAPERLESS_TRUSTED_PROXIES`](configuration.md#PAPERLESS_TRUSTED_PROXIES), [`PAPERLESS_TRUSTED_PROXIES`](configuration.md#PAPERLESS_TRUSTED_PROXIES),
[`PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT`](configuration.md#PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT),
[`PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER`](configuration.md#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER), [`PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER`](configuration.md#PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER),
or both, to avoid `403 Forbidden` errors on login. or a combination of these settings to avoid `403 Forbidden` errors on login.
The proxy count is the number of proxy hops in `X-Forwarded-For`, which may
differ from the number of configured proxy IP addresses.
## Minimum CPU Requirements (NumPy Baseline)
Starting with NumPy 2.4.0, official `manylinux` x86_64 wheels are compiled with a minimum
CPU baseline of `x86-64-v2`, which requires SSE3, SSSE3, SSE4.1, SSE4.2, POPCNT, and
CMPXCHG16B. This is a [deliberate upstream change](https://github.com/numpy/numpy/issues/27851)
citing that these instructions have been present in over 99.7% of CPUs since 2008
([Intel](<https://en.wikipedia.org/wiki/Nehalem_(microarchitecture)>)) or 2011
([AMD](<https://en.wikipedia.org/wiki/Bulldozer_(microarchitecture)>)).
NumPy is a dependency of the document classifier (via scikit-learn), so any CPU that
predates SSE4.2 support will crash with `SIGILL` (illegal instruction) when the classifier
is loaded or trained, regardless of whether AI features are enabled.
This differs from NumPy's optional SIMD dispatch (e.g. AVX2, AVX512), which is detected and
selected safely at runtime - the `x86-64-v2` requirement above is a hard floor baked into the
wheel, with no runtime fallback.
### Affected hardware
CPUs older than roughly 2008 (Intel) or 2011 (AMD) that lack SSE4.2 support. This is more
likely to affect low-power or embedded hardware - e.g. early Atom, Celeron, or pre-Bulldozer
AMD chips - than typical desktop or server hardware from the last decade.
Check for SSE4.2 support with:
```bash
grep -o -m1 sse4_2 /proc/cpuinfo
```
If this prints nothing, your CPU is affected.
### Symptoms
The Celery worker (and potentially the web server) repeatedly crashes and restarts with a
`SIGILL` error, typically visible in `dmesg`/`journalctl` as a `trap invalid opcode` inside
`_multiarray_umath...so`. Because the classifier is trained on a periodic schedule
(hourly, by default), affected instances see intermittent, hard-to-reproduce document
consumption failures whenever that scheduled task runs and takes down the worker process
mid-task.
### Action Required (for affected hardware only)
There is no way to make the classifier itself work on such CPUs - it requires an unofficial
NumPy build with `cpu-baseline=none`, which is not something we can ship. The practical
path forward is to stop the classifier from ever loading or training, which avoids
importing NumPy at all:
```bash
PAPERLESS_TRAIN_TASK_CRON=disable
```
This disables the periodic classifier training task (see
[`PAPERLESS_TRAIN_TASK_CRON`](configuration.md#PAPERLESS_TRAIN_TASK_CRON)). Automatic
matching based on the classifier (suggested correspondents, document types, tags, and
storage paths from trained rules) will no longer be available, but rule-based matching is
unaffected, and document consumption itself will no longer be at risk of crashing the
worker.
## Database Migrations ## Database Migrations
@@ -281,7 +281,7 @@ main {
} }
@media screen and (min-width: 376px) and (max-width: 768px) { @media screen and (min-width: 376px) and (max-width: 768px) {
.navbar-toggler { .navbar:not(:has(pngx-chat)) .navbar-toggler {
// compensate for 2 buttons on the right // compensate for 2 buttons on the right
margin-right: 45px; margin-right: 45px;
} }
+7 -1
View File
@@ -475,7 +475,13 @@ class TantivyBackend:
note_texts: list[str] = [] note_texts: list[str] = []
for note in document.notes.all(): for note in document.notes.all():
num_notes += 1 num_notes += 1
doc.add_json("notes", {"note": note.note, "user": note.user.username}) doc.add_json(
"notes",
{
"note": note.note,
"user": note.user.username if note.user else None,
},
)
note_texts.append(note.note) note_texts.append(note.note)
if note_texts: if note_texts:
doc.add_text("notes_text", " ".join(note_texts)) doc.add_text("notes_text", " ".join(note_texts))
+11 -1
View File
@@ -1264,7 +1264,17 @@ def task_failure_handler(
"error_message": str(exception) if exception else "Unknown error", "error_message": str(exception) if exception else "Unknown error",
} }
if traceback: if traceback:
tb_str = "".join(_tb.format_tb(traceback)) # billiard/celery pass a pre-formatted string instead of a real
# traceback object when the worker process itself died (e.g.
# WorkerLostError from a SIGILL) since there's no live traceback
# to walk in that case.
tb_str = (
traceback
if isinstance(traceback, str)
else "".join(
_tb.format_tb(traceback),
)
)
result_data["traceback"] = tb_str[:5000] result_data["traceback"] = tb_str[:5000]
now = timezone.now() now = timezone.now()
@@ -844,6 +844,23 @@ class TestFieldHandling:
f"Expected 1, got {len(ids)}. Note content should be searchable via notes.note: prefix." f"Expected 1, got {len(ids)}. Note content should be searchable via notes.note: prefix."
) )
def test_notes_without_user_are_indexed(self, backend: TantivyBackend) -> None:
"""Notes whose user was deleted (SET_NULL) must not break indexing."""
doc = Document.objects.create(
title="Doc with orphaned note",
content="test",
checksum="NT2",
pk=81,
)
Note.objects.create(document=doc, note="Orphaned note", user=None)
backend.add_or_update(doc)
ids = backend.search_ids("notes.note:orphaned", user=None)
assert len(ids) == 1, (
f"Expected 1, got {len(ids)}. Notes without a user should still be indexed."
)
class TestHighlightHits: class TestHighlightHits:
"""Test highlight_hits returns proper HTML strings, not raw Snippet objects.""" """Test highlight_hits returns proper HTML strings, not raw Snippet objects."""
+38 -38
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr "" msgstr ""
"Project-Id-Version: paperless-ngx\n" "Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n" "Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-07-23 19:21+0000\n" "POT-Creation-Date: 2026-07-25 07:12+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n" "PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n" "Last-Translator: \n"
"Language-Team: English\n" "Language-Team: English\n"
@@ -1946,151 +1946,151 @@ msgstr ""
msgid "paperless application settings" msgid "paperless application settings"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:545 #: paperless/settings/__init__.py:556
msgid "English (US)" msgid "English (US)"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:546 #: paperless/settings/__init__.py:557
msgid "Arabic" msgid "Arabic"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:547 #: paperless/settings/__init__.py:558
msgid "Afrikaans" msgid "Afrikaans"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:548 #: paperless/settings/__init__.py:559
msgid "Belarusian" msgid "Belarusian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:549 #: paperless/settings/__init__.py:560
msgid "Bulgarian" msgid "Bulgarian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:550 #: paperless/settings/__init__.py:561
msgid "Catalan" msgid "Catalan"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:551 #: paperless/settings/__init__.py:562
msgid "Czech" msgid "Czech"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:552 #: paperless/settings/__init__.py:563
msgid "Danish" msgid "Danish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:553 #: paperless/settings/__init__.py:564
msgid "German" msgid "German"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:554 #: paperless/settings/__init__.py:565
msgid "Greek" msgid "Greek"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:555 #: paperless/settings/__init__.py:566
msgid "English (GB)" msgid "English (GB)"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:556 #: paperless/settings/__init__.py:567
msgid "Spanish" msgid "Spanish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:557 #: paperless/settings/__init__.py:568
msgid "Persian" msgid "Persian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:558 #: paperless/settings/__init__.py:569
msgid "Finnish" msgid "Finnish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:559 #: paperless/settings/__init__.py:570
msgid "French" msgid "French"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:560 #: paperless/settings/__init__.py:571
msgid "Hungarian" msgid "Hungarian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:561 #: paperless/settings/__init__.py:572
msgid "Indonesian" msgid "Indonesian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:562 #: paperless/settings/__init__.py:573
msgid "Italian" msgid "Italian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:563 #: paperless/settings/__init__.py:574
msgid "Japanese" msgid "Japanese"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:564 #: paperless/settings/__init__.py:575
msgid "Korean" msgid "Korean"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:565 #: paperless/settings/__init__.py:576
msgid "Luxembourgish" msgid "Luxembourgish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:566 #: paperless/settings/__init__.py:577
msgid "Norwegian" msgid "Norwegian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:567 #: paperless/settings/__init__.py:578
msgid "Dutch" msgid "Dutch"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:568 #: paperless/settings/__init__.py:579
msgid "Polish" msgid "Polish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:569 #: paperless/settings/__init__.py:580
msgid "Portuguese (Brazil)" msgid "Portuguese (Brazil)"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:570 #: paperless/settings/__init__.py:581
msgid "Portuguese" msgid "Portuguese"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:571 #: paperless/settings/__init__.py:582
msgid "Romanian" msgid "Romanian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:572 #: paperless/settings/__init__.py:583
msgid "Russian" msgid "Russian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:573 #: paperless/settings/__init__.py:584
msgid "Slovak" msgid "Slovak"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:574 #: paperless/settings/__init__.py:585
msgid "Slovenian" msgid "Slovenian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:575 #: paperless/settings/__init__.py:586
msgid "Serbian" msgid "Serbian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:576 #: paperless/settings/__init__.py:587
msgid "Swedish" msgid "Swedish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:577 #: paperless/settings/__init__.py:588
msgid "Turkish" msgid "Turkish"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:578 #: paperless/settings/__init__.py:589
msgid "Ukrainian" msgid "Ukrainian"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:579 #: paperless/settings/__init__.py:590
msgid "Vietnamese" msgid "Vietnamese"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:580 #: paperless/settings/__init__.py:591
msgid "Chinese Simplified" msgid "Chinese Simplified"
msgstr "" msgstr ""
#: paperless/settings/__init__.py:581 #: paperless/settings/__init__.py:592
msgid "Chinese Traditional" msgid "Chinese Traditional"
msgstr "" msgstr ""
+4 -4
View File
@@ -272,10 +272,7 @@ class MailDocumentParser:
logger.debug("Building formatted text from email") logger.debug("Building formatted text from email")
self._text = build_formatted_text(mail) self._text = build_formatted_text(mail)
if is_naive(mail.date): self._date = mail.date
self._date = make_aware(mail.date)
else:
self._date = mail.date
logger.debug("Creating a PDF from the email") logger.debug("Creating a PDF from the email")
if self._mailrule_id: if self._mailrule_id:
@@ -502,6 +499,9 @@ class MailDocumentParser:
f"Could not parse {filepath}: {err}", f"Could not parse {filepath}: {err}",
) from err ) from err
if is_naive(parsed.date):
parsed.date = make_aware(parsed.date)
return parsed return parsed
def tika_parse(self, html: str) -> str: def tika_parse(self, html: str) -> str:
+5
View File
@@ -421,6 +421,11 @@ class TikaDocumentParser:
logger.info("Converting %s to PDF as %s", document_path, pdf_path) logger.info("Converting %s to PDF as %s", document_path, pdf_path)
with self._gotenberg_client.libre_office.to_pdf() as route: with self._gotenberg_client.libre_office.to_pdf() as route:
# Preserve document fields as authored. updateIndexes (Gotenberg's
# default) triggers a refresh() that rewrites dynamic fields like
# auto-dates to the current date.
route.update_indexes(update_indexes=False)
# Set the output format of the resulting PDF. # Set the output format of the resulting PDF.
# OutputTypeConfig reads the database-stored ApplicationConfiguration # OutputTypeConfig reads the database-stored ApplicationConfiguration
# first, then falls back to the PAPERLESS_OCR_OUTPUT_TYPE env var. # first, then falls back to the PAPERLESS_OCR_OUTPUT_TYPE env var.
+14 -3
View File
@@ -460,11 +460,22 @@ def _parse_paperless_url():
PAPERLESS_URL = _parse_paperless_url() PAPERLESS_URL = _parse_paperless_url()
def _get_allauth_trusted_proxy_count(trusted_proxies: list[str]) -> int:
count = get_int_from_env(
"PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT",
len(trusted_proxies),
)
if count < 0:
raise ImproperlyConfigured(
"PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT must be zero or greater",
)
return count
# For use with trusted proxies # For use with trusted proxies
TRUSTED_PROXIES = get_list_from_env("PAPERLESS_TRUSTED_PROXIES") TRUSTED_PROXIES = get_list_from_env("PAPERLESS_TRUSTED_PROXIES")
# Derive allauth's proxy count from the same list so X-Forwarded-For is trusted ALLAUTH_TRUSTED_PROXY_COUNT = _get_allauth_trusted_proxy_count(TRUSTED_PROXIES)
# correctly when users have configured PAPERLESS_TRUSTED_PROXIES.
ALLAUTH_TRUSTED_PROXY_COUNT = len(TRUSTED_PROXIES)
ALLAUTH_TRUSTED_CLIENT_IP_HEADER = os.getenv( ALLAUTH_TRUSTED_CLIENT_IP_HEADER = os.getenv(
"PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER", "PAPERLESS_ALLAUTH_TRUSTED_CLIENT_IP_HEADER",
) )
@@ -145,6 +145,41 @@ class TestEmailFileParsing:
assert parsed_msg.text == "This is just a simple Text Mail.\n" assert parsed_msg.text == "This is just a simple Text Mail.\n"
assert parsed_msg.to == ("some@one.de",) assert parsed_msg.to == ("some@one.de",)
def test_parse_file_to_message_makes_naive_date_aware(
self,
mocker: MockerFixture,
mail_parser: MailDocumentParser,
simple_txt_email_file: Path,
) -> None:
"""
GIVEN:
- An email whose parsed date is naive (no tzinfo)
WHEN:
- The .eml file is parsed into a MailMessage
THEN:
- The resulting message date is made timezone-aware
"""
mock_message = mocker.Mock(
from_values="mail@someserver.de",
date=datetime.datetime(2022, 10, 12, 21, 40, 43),
)
mocker.patch(
"paperless.parsers.mail.MailMessage.from_bytes",
return_value=mock_message,
)
parsed_msg = mail_parser.parse_file_to_message(simple_txt_email_file)
assert timezone.is_aware(parsed_msg.date)
assert parsed_msg.date.replace(tzinfo=None) == datetime.datetime(
2022,
10,
12,
21,
40,
43,
)
class TestEmailMetadataExtraction: class TestEmailMetadataExtraction:
""" """
@@ -223,4 +223,11 @@ class TestTikaParser:
assert form_field_found assert form_field_found
# Field updates must be disabled so LibreOffice preserves document
# fields (e.g. auto-date fields) as authored rather than refreshing
# them to the current date.
assert any(
b'name="updateIndexes"' in part and b"false" in part for part in parts
)
httpx_mock.reset() httpx_mock.reset()
@@ -3,7 +3,9 @@ from unittest import TestCase
from unittest import mock from unittest import mock
import pytest import pytest
from django.core.exceptions import ImproperlyConfigured
from paperless.settings import _get_allauth_trusted_proxy_count
from paperless.settings import _get_search_language_setting from paperless.settings import _get_search_language_setting
from paperless.settings import _parse_paperless_url from paperless.settings import _parse_paperless_url
from paperless.settings import default_threads_per_worker from paperless.settings import default_threads_per_worker
@@ -35,6 +37,31 @@ class TestThreadCalculation(TestCase):
self.assertLessEqual(default_workers * default_threads, i) self.assertLessEqual(default_workers * default_threads, i)
def test_allauth_trusted_proxy_count_defaults_to_trusted_proxies(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT", raising=False)
assert _get_allauth_trusted_proxy_count(["proxy-v4", "proxy-v6"]) == 2
def test_allauth_trusted_proxy_count_can_be_overridden(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT", "1")
assert _get_allauth_trusted_proxy_count(["proxy-v4", "proxy-v6"]) == 1
def test_allauth_trusted_proxy_count_rejects_negative_values(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("PAPERLESS_ALLAUTH_TRUSTED_PROXY_COUNT", "-1")
with pytest.raises(ImproperlyConfigured, match="must be zero or greater"):
_get_allauth_trusted_proxy_count([])
@pytest.mark.parametrize( @pytest.mark.parametrize(
("env_value", "expected"), ("env_value", "expected"),
[ [