mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-03-30 21:02:45 +00:00
Compare commits
3 Commits
chore/plug
...
feature-sh
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4c5a61750b | ||
|
|
fffb4461a7 | ||
|
|
20258ea86e |
1
.github/ISSUE_TEMPLATE/bug-report.yml
vendored
1
.github/ISSUE_TEMPLATE/bug-report.yml
vendored
@@ -21,7 +21,6 @@ body:
|
||||
- [The installation instructions](https://docs.paperless-ngx.com/setup/#installation).
|
||||
- [Existing issues and discussions](https://github.com/paperless-ngx/paperless-ngx/search?q=&type=issues).
|
||||
- Disable any custom container initialization scripts, if using
|
||||
- Remove any third-party parser plugins — issues caused by or requiring changes to a third-party plugin will be closed without investigation.
|
||||
|
||||
If you encounter issues while installing or configuring Paperless-ngx, please post in the ["Support" section of the discussions](https://github.com/paperless-ngx/paperless-ngx/discussions/new?category=support).
|
||||
- type: textarea
|
||||
|
||||
@@ -723,81 +723,6 @@ services:
|
||||
|
||||
1. Note the `:ro` tag means the folder will be mounted as read only. This is for extra security against changes
|
||||
|
||||
## Installing third-party parser plugins {#parser-plugins}
|
||||
|
||||
Third-party parser plugins extend Paperless-ngx to support additional file
|
||||
formats. A plugin is a Python package that advertises itself under the
|
||||
`paperless_ngx.parsers` entry point group. Refer to the
|
||||
[developer documentation](development.md#making-custom-parsers) for how to
|
||||
create one.
|
||||
|
||||
!!! warning "Third-party plugins are not officially supported"
|
||||
|
||||
The Paperless-ngx maintainers do not provide support for third-party
|
||||
plugins. Issues caused by or requiring changes to a third-party plugin
|
||||
will be closed without further investigation. Always reproduce problems
|
||||
with all plugins removed before filing a bug report.
|
||||
|
||||
### Docker
|
||||
|
||||
Use a [custom container initialization script](#custom-container-initialization)
|
||||
to install the package before the webserver starts. Create a shell script and
|
||||
mount it into `/custom-cont-init.d`:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# /path/to/my/scripts/install-parsers.sh
|
||||
|
||||
pip install my-paperless-parser-package
|
||||
```
|
||||
|
||||
Mount it in your `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
webserver:
|
||||
# ...
|
||||
volumes:
|
||||
- /path/to/my/scripts:/custom-cont-init.d:ro
|
||||
```
|
||||
|
||||
The script runs as `root` before the webserver starts, so the package will be
|
||||
available when Paperless-ngx discovers plugins at startup.
|
||||
|
||||
### Bare metal
|
||||
|
||||
Install the package into the same Python environment that runs Paperless-ngx.
|
||||
If you followed the standard bare-metal install guide, that is the `paperless`
|
||||
user's environment:
|
||||
|
||||
```bash
|
||||
sudo -Hu paperless pip3 install my-paperless-parser-package
|
||||
```
|
||||
|
||||
If you are using `uv` or a virtual environment, activate it first and then run:
|
||||
|
||||
```bash
|
||||
uv pip install my-paperless-parser-package
|
||||
# or
|
||||
pip install my-paperless-parser-package
|
||||
```
|
||||
|
||||
Restart all Paperless-ngx services after installation so the new plugin is
|
||||
discovered.
|
||||
|
||||
### Verifying installation
|
||||
|
||||
On the next startup, check the application logs for a line confirming
|
||||
discovery:
|
||||
|
||||
```
|
||||
Loaded third-party parser 'My Parser' v1.0.0 by Acme Corp (entrypoint: 'my_parser').
|
||||
```
|
||||
|
||||
If this line does not appear, verify that the package is installed in the
|
||||
correct environment and that its `pyproject.toml` declares the
|
||||
`paperless_ngx.parsers` entry point.
|
||||
|
||||
## MySQL Caveats {#mysql-caveats}
|
||||
|
||||
### Case Sensitivity
|
||||
|
||||
@@ -370,367 +370,121 @@ docker build --file Dockerfile --tag paperless:local .
|
||||
|
||||
## Extending Paperless-ngx
|
||||
|
||||
Paperless-ngx supports third-party document parsers via a Python entry point
|
||||
plugin system. Plugins are distributed as ordinary Python packages and
|
||||
discovered automatically at startup — no changes to the Paperless-ngx source
|
||||
are required.
|
||||
|
||||
!!! warning "Third-party plugins are not officially supported"
|
||||
|
||||
The Paperless-ngx maintainers do not provide support for third-party
|
||||
plugins. Issues that are caused by or require changes to a third-party
|
||||
plugin will be closed without further investigation. If you believe you
|
||||
have found a bug in Paperless-ngx itself (not in a plugin), please
|
||||
reproduce it with all third-party plugins removed before filing an issue.
|
||||
Paperless-ngx does not have any fancy plugin systems and will probably never
|
||||
have. However, some parts of the application have been designed to allow
|
||||
easy integration of additional features without any modification to the
|
||||
base code.
|
||||
|
||||
### Making custom parsers
|
||||
|
||||
Paperless-ngx uses parsers to add documents. A parser is responsible for:
|
||||
Paperless-ngx uses parsers to add documents. A parser is
|
||||
responsible for:
|
||||
|
||||
- Extracting plain-text content from the document
|
||||
- Generating a thumbnail image
|
||||
- _optional:_ Detecting the document's creation date
|
||||
- _optional:_ Producing a searchable PDF archive copy
|
||||
- Retrieving the content from the original
|
||||
- Creating a thumbnail
|
||||
- _optional:_ Retrieving a created date from the original
|
||||
- _optional:_ Creating an archived document from the original
|
||||
|
||||
Custom parsers are distributed as ordinary Python packages and registered
|
||||
via a [setuptools entry point](https://setuptools.pypa.io/en/latest/userguide/entry_point.html).
|
||||
No changes to the Paperless-ngx source are required.
|
||||
Custom parsers can be added to Paperless-ngx to support more file types. In
|
||||
order to do that, you need to write the parser itself and announce its
|
||||
existence to Paperless-ngx.
|
||||
|
||||
#### 1. Implementing the parser class
|
||||
|
||||
Your parser must satisfy the `ParserProtocol` structural interface defined in
|
||||
`paperless.parsers`. The simplest approach is to write a plain class — no base
|
||||
class is required, only the right attributes and methods.
|
||||
|
||||
**Class-level identity attributes**
|
||||
|
||||
The registry reads these before instantiating the parser, so they must be
|
||||
plain class attributes (not instance attributes or properties):
|
||||
The parser itself must extend `documents.parsers.DocumentParser` and
|
||||
must implement the methods `parse` and `get_thumbnail`. You can provide
|
||||
your own implementation to `get_date` if you don't want to rely on
|
||||
Paperless-ngx' default date guessing mechanisms.
|
||||
|
||||
```python
|
||||
class MyCustomParser:
|
||||
name = "My Format Parser" # human-readable name shown in logs
|
||||
version = "1.0.0" # semantic version string
|
||||
author = "Acme Corp" # author / organisation
|
||||
url = "https://example.com/my-parser" # docs or issue tracker
|
||||
class MyCustomParser(DocumentParser):
|
||||
|
||||
def parse(self, document_path, mime_type):
|
||||
# This method does not return anything. Rather, you should assign
|
||||
# whatever you got from the document to the following fields:
|
||||
|
||||
# The content of the document.
|
||||
self.text = "content"
|
||||
|
||||
# Optional: path to a PDF document that you created from the original.
|
||||
self.archive_path = os.path.join(self.tempdir, "archived.pdf")
|
||||
|
||||
# Optional: "created" date of the document.
|
||||
self.date = get_created_from_metadata(document_path)
|
||||
|
||||
def get_thumbnail(self, document_path, mime_type):
|
||||
# This should return the path to a thumbnail you created for this
|
||||
# document.
|
||||
return os.path.join(self.tempdir, "thumb.webp")
|
||||
```
|
||||
|
||||
**Declaring supported MIME types**
|
||||
If you encounter any issues during parsing, raise a
|
||||
`documents.parsers.ParseError`.
|
||||
|
||||
Return a `dict` mapping MIME type strings to preferred file extensions
|
||||
(including the leading dot). Paperless-ngx uses the extension when storing
|
||||
archive copies and serving files for download.
|
||||
The `self.tempdir` directory is a temporary directory that is guaranteed
|
||||
to be empty and removed after consumption finished. You can use that
|
||||
directory to store any intermediate files and also use it to store the
|
||||
thumbnail / archived document.
|
||||
|
||||
After that, you need to announce your parser to Paperless-ngx. You need to
|
||||
connect a handler to the `document_consumer_declaration` signal. Have a
|
||||
look in the file `src/paperless_tesseract/apps.py` on how that's done.
|
||||
The handler is a method that returns information about your parser:
|
||||
|
||||
```python
|
||||
@classmethod
|
||||
def supported_mime_types(cls) -> dict[str, str]:
|
||||
def myparser_consumer_declaration(sender, **kwargs):
|
||||
return {
|
||||
"application/x-my-format": ".myf",
|
||||
"application/x-my-format-alt": ".myf",
|
||||
"parser": MyCustomParser,
|
||||
"weight": 0,
|
||||
"mime_types": {
|
||||
"application/pdf": ".pdf",
|
||||
"image/jpeg": ".jpg",
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Scoring**
|
||||
|
||||
When more than one parser can handle a file, the registry calls `score()` on
|
||||
each candidate and picks the one with the highest result and equal scores favor third-party parsers over built-ins.. Return `None` to
|
||||
decline handling a file even though the MIME type is listed as supported (for
|
||||
example, when a required external service is not configured).
|
||||
|
||||
| Score | Meaning |
|
||||
| ------ | --------------------------------------------------------------------------------- |
|
||||
| `None` | Decline — do not handle this file |
|
||||
| `10` | Default priority used by all built-in parsers |
|
||||
| `20` | Priority used by the remote OCR built-in parser, allowing it to replace Tesseract |
|
||||
| `> 10` | Override a built-in parser for the same MIME type |
|
||||
|
||||
```python
|
||||
@classmethod
|
||||
def score(
|
||||
cls,
|
||||
mime_type: str,
|
||||
filename: str,
|
||||
path: "Path | None" = None,
|
||||
) -> int | None:
|
||||
# Inspect filename or file bytes here if needed.
|
||||
return 10
|
||||
```
|
||||
|
||||
**Archive and rendition flags**
|
||||
|
||||
```python
|
||||
@property
|
||||
def can_produce_archive(self) -> bool:
|
||||
"""True if parse() can produce a searchable PDF archive copy."""
|
||||
return True # or False if your parser doesn't produce PDFs
|
||||
|
||||
@property
|
||||
def requires_pdf_rendition(self) -> bool:
|
||||
"""True if the original format cannot be displayed by a browser
|
||||
(e.g. DOCX, ODT) and the PDF output must always be kept."""
|
||||
return False
|
||||
```
|
||||
|
||||
**Context manager — temp directory lifecycle**
|
||||
|
||||
Paperless-ngx always uses parsers as context managers. Create a temporary
|
||||
working directory in `__enter__` (or `__init__`) and remove it in `__exit__`
|
||||
regardless of whether an exception occurred. Store intermediate files,
|
||||
thumbnails, and archive PDFs inside this directory.
|
||||
|
||||
```python
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Self
|
||||
from types import TracebackType
|
||||
|
||||
from django.conf import settings
|
||||
|
||||
class MyCustomParser:
|
||||
...
|
||||
|
||||
def __init__(self, logging_group: object = None) -> None:
|
||||
settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
|
||||
self._tempdir = Path(
|
||||
tempfile.mkdtemp(prefix="paperless-", dir=settings.SCRATCH_DIR)
|
||||
)
|
||||
self._text: str | None = None
|
||||
self._archive_path: Path | None = None
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: TracebackType | None,
|
||||
) -> None:
|
||||
shutil.rmtree(self._tempdir, ignore_errors=True)
|
||||
```
|
||||
|
||||
**Optional context — `configure()`**
|
||||
|
||||
The consumer calls `configure()` with a `ParserContext` after instantiation
|
||||
and before `parse()`. If your parser doesn't need context, a no-op
|
||||
implementation is fine:
|
||||
|
||||
```python
|
||||
from paperless.parsers import ParserContext
|
||||
|
||||
def configure(self, context: ParserContext) -> None:
|
||||
pass # override if you need context.mailrule_id, etc.
|
||||
```
|
||||
|
||||
**Parsing**
|
||||
|
||||
`parse()` is the core method. It must not return a value; instead, store
|
||||
results in instance attributes and expose them via the accessor methods below.
|
||||
Raise `documents.parsers.ParseError` on any unrecoverable failure.
|
||||
|
||||
```python
|
||||
from documents.parsers import ParseError
|
||||
|
||||
def parse(
|
||||
self,
|
||||
document_path: Path,
|
||||
mime_type: str,
|
||||
*,
|
||||
produce_archive: bool = True,
|
||||
) -> None:
|
||||
try:
|
||||
self._text = extract_text_from_my_format(document_path)
|
||||
except Exception as e:
|
||||
raise ParseError(f"Failed to parse {document_path}: {e}") from e
|
||||
|
||||
if produce_archive and self.can_produce_archive:
|
||||
archive = self._tempdir / "archived.pdf"
|
||||
convert_to_pdf(document_path, archive)
|
||||
self._archive_path = archive
|
||||
```
|
||||
|
||||
**Result accessors**
|
||||
|
||||
```python
|
||||
def get_text(self) -> str | None:
|
||||
return self._text
|
||||
|
||||
def get_date(self) -> "datetime.datetime | None":
|
||||
# Return a datetime extracted from the document, or None to let
|
||||
# Paperless-ngx use its default date-guessing logic.
|
||||
return None
|
||||
|
||||
def get_archive_path(self) -> Path | None:
|
||||
return self._archive_path
|
||||
|
||||
def get_page_count(self, document_path: Path, mime_type: str) -> int | None:
|
||||
# If the format doesn't have the concept of pages, return None
|
||||
return count_pages(document_path)
|
||||
|
||||
```
|
||||
|
||||
**Thumbnail**
|
||||
|
||||
`get_thumbnail()` may be called independently of `parse()`. Return the path
|
||||
to a WebP image inside `self._tempdir`. The image should be roughly 500 × 700
|
||||
pixels.
|
||||
|
||||
```python
|
||||
def get_thumbnail(self, document_path: Path, mime_type: str) -> Path:
|
||||
thumb = self._tempdir / "thumb.webp"
|
||||
render_thumbnail(document_path, thumb)
|
||||
return thumb
|
||||
```
|
||||
|
||||
**Optional methods**
|
||||
|
||||
These are called by the API on demand, not during the consumption pipeline.
|
||||
Implement them if your format supports the information; otherwise return
|
||||
`None` / `[]`.
|
||||
|
||||
```python
|
||||
|
||||
def extract_metadata(
|
||||
self,
|
||||
document_path: Path,
|
||||
mime_type: str,
|
||||
) -> "list[MetadataEntry]":
|
||||
# Must never raise. Return [] if metadata cannot be read.
|
||||
from paperless.parsers import MetadataEntry
|
||||
return [
|
||||
MetadataEntry(
|
||||
namespace="https://example.com/ns/",
|
||||
prefix="ex",
|
||||
key="Author",
|
||||
value="Alice",
|
||||
)
|
||||
]
|
||||
```
|
||||
|
||||
#### 2. Registering via entry point
|
||||
|
||||
Add the following to your package's `pyproject.toml`. The key (left of `=`)
|
||||
is an arbitrary name used only in log output; the value is the
|
||||
`module:ClassName` import path.
|
||||
|
||||
```toml
|
||||
[project.entry-points."paperless_ngx.parsers"]
|
||||
my_parser = "my_package.parsers:MyCustomParser"
|
||||
```
|
||||
|
||||
Install your package into the same Python environment as Paperless-ngx (or
|
||||
add it to the Docker image), and the parser will be discovered automatically
|
||||
on the next startup. No configuration changes are needed.
|
||||
|
||||
To verify discovery, check the application logs at startup for a line like:
|
||||
|
||||
```
|
||||
Loaded third-party parser 'My Format Parser' v1.0.0 by Acme Corp (entrypoint: 'my_parser').
|
||||
```
|
||||
|
||||
#### 3. Utilities
|
||||
|
||||
`paperless.parsers.utils` provides helpers you can import directly:
|
||||
|
||||
| Function | Description |
|
||||
| --------------------------------------- | ---------------------------------------------------------------- |
|
||||
| `read_file_handle_unicode_errors(path)` | Read a file as UTF-8, replacing invalid bytes instead of raising |
|
||||
| `get_page_count_for_pdf(path)` | Count pages in a PDF using pikepdf |
|
||||
| `extract_pdf_metadata(path)` | Extract XMP metadata from a PDF as a `list[MetadataEntry]` |
|
||||
|
||||
#### Minimal example
|
||||
|
||||
A complete, working parser for a hypothetical plain-XML format:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Self
|
||||
from types import TracebackType
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from django.conf import settings
|
||||
|
||||
from documents.parsers import ParseError
|
||||
from paperless.parsers import ParserContext
|
||||
|
||||
|
||||
class XmlDocumentParser:
|
||||
name = "XML Parser"
|
||||
version = "1.0.0"
|
||||
author = "Acme Corp"
|
||||
url = "https://example.com/xml-parser"
|
||||
|
||||
@classmethod
|
||||
def supported_mime_types(cls) -> dict[str, str]:
|
||||
return {"application/xml": ".xml", "text/xml": ".xml"}
|
||||
|
||||
@classmethod
|
||||
def score(cls, mime_type: str, filename: str, path: Path | None = None) -> int | None:
|
||||
return 10
|
||||
|
||||
@property
|
||||
def can_produce_archive(self) -> bool:
|
||||
return False
|
||||
|
||||
@property
|
||||
def requires_pdf_rendition(self) -> bool:
|
||||
return False
|
||||
|
||||
def __init__(self, logging_group: object = None) -> None:
|
||||
settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
|
||||
self._tempdir = Path(tempfile.mkdtemp(prefix="paperless-", dir=settings.SCRATCH_DIR))
|
||||
self._text: str | None = None
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
|
||||
shutil.rmtree(self._tempdir, ignore_errors=True)
|
||||
|
||||
def configure(self, context: ParserContext) -> None:
|
||||
pass
|
||||
|
||||
def parse(self, document_path: Path, mime_type: str, *, produce_archive: bool = True) -> None:
|
||||
try:
|
||||
tree = ET.parse(document_path)
|
||||
self._text = " ".join(tree.getroot().itertext())
|
||||
except ET.ParseError as e:
|
||||
raise ParseError(f"XML parse error: {e}") from e
|
||||
|
||||
def get_text(self) -> str | None:
|
||||
return self._text
|
||||
|
||||
def get_date(self):
|
||||
return None
|
||||
|
||||
def get_archive_path(self) -> Path | None:
|
||||
return None
|
||||
|
||||
def get_thumbnail(self, document_path: Path, mime_type: str) -> Path:
|
||||
from PIL import Image, ImageDraw
|
||||
img = Image.new("RGB", (500, 700), color="white")
|
||||
ImageDraw.Draw(img).text((10, 10), "XML Document", fill="black")
|
||||
out = self._tempdir / "thumb.webp"
|
||||
img.save(out, format="WEBP")
|
||||
return out
|
||||
|
||||
def get_page_count(self, document_path: Path, mime_type: str) -> int | None:
|
||||
return None
|
||||
|
||||
def extract_metadata(self, document_path: Path, mime_type: str) -> list:
|
||||
return []
|
||||
```
|
||||
|
||||
### Developing date parser plugins
|
||||
- `parser` is a reference to a class that extends `DocumentParser`.
|
||||
- `weight` is used whenever two or more parsers are able to parse a
|
||||
file: The parser with the higher weight wins. This can be used to
|
||||
override the parsers provided by Paperless-ngx.
|
||||
- `mime_types` is a dictionary. The keys are the mime types your
|
||||
parser supports and the value is the default file extension that
|
||||
Paperless-ngx should use when storing files and serving them for
|
||||
download. We could guess that from the file extensions, but some
|
||||
mime types have many extensions associated with them and the Python
|
||||
methods responsible for guessing the extension do not always return
|
||||
the same value.
|
||||
|
||||
## Using Visual Studio Code devcontainer
|
||||
|
||||
Another easy way to get started with development is to use Visual Studio
|
||||
Code devcontainers. This approach will create a preconfigured development
|
||||
environment with all of the required tools and dependencies.
|
||||
[Learn more about devcontainers](https://code.visualstudio.com/docs/devcontainers/containers).
|
||||
The .devcontainer/vscode/tasks.json and .devcontainer/vscode/launch.json files
|
||||
contain more information about the specific tasks and launch configurations (see the
|
||||
non-standard "description" field).
|
||||
|
||||
To get started:
|
||||
|
||||
1. Clone the repository on your machine and open the Paperless-ngx folder in VS Code.
|
||||
|
||||
2. VS Code will prompt you with "Reopen in container". Do so and wait for the environment to start.
|
||||
|
||||
3. In case your host operating system is Windows:
|
||||
- The Source Control view in Visual Studio Code might show: "The detected Git repository is potentially unsafe as the folder is owned by someone other than the current user." Use "Manage Unsafe Repositories" to fix this.
|
||||
- Git might have detecteded modifications for all files, because Windows is using CRLF line endings. Run `git checkout .` in the containers terminal to fix this issue.
|
||||
|
||||
4. Initialize the project by running the task **Project Setup: Run all Init Tasks**. This
|
||||
will initialize the database tables and create a superuser. Then you can compile the front end
|
||||
for production or run the frontend in debug mode.
|
||||
|
||||
5. The project is ready for debugging, start either run the fullstack debug or individual debug
|
||||
processes. Yo spin up the project without debugging run the task **Project Start: Run all Services**
|
||||
|
||||
## Developing Date Parser Plugins
|
||||
|
||||
Paperless-ngx uses a plugin system for date parsing, allowing you to extend or replace the default date parsing behavior. Plugins are discovered using [Python entry points](https://setuptools.pypa.io/en/latest/userguide/entry_point.html).
|
||||
|
||||
#### Creating a Date Parser Plugin
|
||||
### Creating a Date Parser Plugin
|
||||
|
||||
To create a custom date parser plugin, you need to:
|
||||
|
||||
@@ -738,7 +492,7 @@ To create a custom date parser plugin, you need to:
|
||||
2. Implement the required abstract method
|
||||
3. Register your plugin via an entry point
|
||||
|
||||
##### 1. Implementing the Parser Class
|
||||
#### 1. Implementing the Parser Class
|
||||
|
||||
Your parser must extend `documents.plugins.date_parsing.DateParserPluginBase` and implement the `parse` method:
|
||||
|
||||
@@ -778,7 +532,7 @@ class MyDateParserPlugin(DateParserPluginBase):
|
||||
yield another_datetime
|
||||
```
|
||||
|
||||
##### 2. Configuration and Helper Methods
|
||||
#### 2. Configuration and Helper Methods
|
||||
|
||||
Your parser instance is initialized with a `DateParserConfig` object accessible via `self.config`. This provides:
|
||||
|
||||
@@ -811,11 +565,11 @@ def _filter_date(
|
||||
"""
|
||||
```
|
||||
|
||||
##### 3. Resource Management (Optional)
|
||||
#### 3. Resource Management (Optional)
|
||||
|
||||
If your plugin needs to acquire or release resources (database connections, API clients, etc.), override the context manager methods. Paperless-ngx will always use plugins as context managers, ensuring resources can be released even in the event of errors.
|
||||
|
||||
##### 4. Registering Your Plugin
|
||||
#### 4. Registering Your Plugin
|
||||
|
||||
Register your plugin using a setuptools entry point in your package's `pyproject.toml`:
|
||||
|
||||
@@ -826,7 +580,7 @@ my_parser = "my_package.parsers:MyDateParserPlugin"
|
||||
|
||||
The entry point name (e.g., `"my_parser"`) is used for sorting when multiple plugins are found. Paperless-ngx will use the first plugin alphabetically by name if multiple plugins are discovered.
|
||||
|
||||
#### Plugin Discovery
|
||||
### Plugin Discovery
|
||||
|
||||
Paperless-ngx automatically discovers and loads date parser plugins at runtime. The discovery process:
|
||||
|
||||
@@ -837,7 +591,7 @@ Paperless-ngx automatically discovers and loads date parser plugins at runtime.
|
||||
|
||||
If multiple plugins are installed, a warning is logged indicating which plugin was selected.
|
||||
|
||||
#### Example: Simple Date Parser
|
||||
### Example: Simple Date Parser
|
||||
|
||||
Here's a minimal example that only looks for ISO 8601 dates:
|
||||
|
||||
@@ -869,30 +623,3 @@ class ISODateParserPlugin(DateParserPluginBase):
|
||||
if filtered_date is not None:
|
||||
yield filtered_date
|
||||
```
|
||||
|
||||
## Using Visual Studio Code devcontainer
|
||||
|
||||
Another easy way to get started with development is to use Visual Studio
|
||||
Code devcontainers. This approach will create a preconfigured development
|
||||
environment with all of the required tools and dependencies.
|
||||
[Learn more about devcontainers](https://code.visualstudio.com/docs/devcontainers/containers).
|
||||
The .devcontainer/vscode/tasks.json and .devcontainer/vscode/launch.json files
|
||||
contain more information about the specific tasks and launch configurations (see the
|
||||
non-standard "description" field).
|
||||
|
||||
To get started:
|
||||
|
||||
1. Clone the repository on your machine and open the Paperless-ngx folder in VS Code.
|
||||
|
||||
2. VS Code will prompt you with "Reopen in container". Do so and wait for the environment to start.
|
||||
|
||||
3. In case your host operating system is Windows:
|
||||
- The Source Control view in Visual Studio Code might show: "The detected Git repository is potentially unsafe as the folder is owned by someone other than the current user." Use "Manage Unsafe Repositories" to fix this.
|
||||
- Git might have detecteded modifications for all files, because Windows is using CRLF line endings. Run `git checkout .` in the containers terminal to fix this issue.
|
||||
|
||||
4. Initialize the project by running the task **Project Setup: Run all Init Tasks**. This
|
||||
will initialize the database tables and create a superuser. Then you can compile the front end
|
||||
for production or run the frontend in debug mode.
|
||||
|
||||
5. The project is ready for debugging, start either run the fullstack debug or individual debug
|
||||
processes. Yo spin up the project without debugging run the task **Project Start: Run all Services**
|
||||
|
||||
@@ -468,7 +468,7 @@
|
||||
"time": 0.951,
|
||||
"request": {
|
||||
"method": "GET",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__in=9",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__in=9",
|
||||
"httpVersion": "HTTP/1.1",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -534,7 +534,7 @@
|
||||
"time": 0.653,
|
||||
"request": {
|
||||
"method": "GET",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=10&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=9",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=9",
|
||||
"httpVersion": "HTTP/1.1",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
|
||||
@@ -883,7 +883,7 @@
|
||||
"time": 0.93,
|
||||
"request": {
|
||||
"method": "GET",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=4",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=4",
|
||||
"httpVersion": "HTTP/1.1",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
@@ -961,7 +961,7 @@
|
||||
"time": -1,
|
||||
"request": {
|
||||
"method": "GET",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=4",
|
||||
"url": "http://localhost:8000/api/documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=4",
|
||||
"httpVersion": "HTTP/1.1",
|
||||
"cookies": [],
|
||||
"headers": [
|
||||
|
||||
@@ -16,7 +16,7 @@ test('basic filtering', async ({ page }) => {
|
||||
await expect(page).toHaveURL(/tags__id__all=9/)
|
||||
await expect(page.locator('pngx-document-list')).toHaveText(/8 documents/)
|
||||
await page.getByRole('button', { name: 'Document type' }).click()
|
||||
await page.getByRole('menuitem', { name: /^Invoice Test/ }).click()
|
||||
await page.getByRole('menuitem', { name: 'Invoice Test 3' }).click()
|
||||
await expect(page).toHaveURL(/document_type__id__in=1/)
|
||||
await expect(page.locator('pngx-document-list')).toHaveText(/3 documents/)
|
||||
await page.getByRole('button', { name: 'Reset filters' }).first().click()
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1081,7 +1081,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">205</context>
|
||||
<context context-type="linenumber">208</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8901931207592071833" datatype="html">
|
||||
@@ -3029,7 +3029,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">200</context>
|
||||
<context context-type="linenumber">203</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.ts</context>
|
||||
@@ -7504,7 +7504,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">192</context>
|
||||
<context context-type="linenumber">195</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/data/document.ts</context>
|
||||
@@ -8817,7 +8817,7 @@
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">197</context>
|
||||
<context context-type="linenumber">200</context>
|
||||
</context-group>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/data/document.ts</context>
|
||||
@@ -9020,56 +9020,56 @@
|
||||
<source>Title & content</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">195</context>
|
||||
<context context-type="linenumber">198</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7408932238599462499" datatype="html">
|
||||
<source>File type</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">202</context>
|
||||
<context context-type="linenumber">205</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="2649431021108393503" datatype="html">
|
||||
<source>More like</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">211</context>
|
||||
<context context-type="linenumber">214</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3697582909018473071" datatype="html">
|
||||
<source>equals</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">217</context>
|
||||
<context context-type="linenumber">220</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5325481293405718739" datatype="html">
|
||||
<source>is empty</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">221</context>
|
||||
<context context-type="linenumber">224</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6166785695326182482" datatype="html">
|
||||
<source>is not empty</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">225</context>
|
||||
<context context-type="linenumber">228</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="4686622206659266699" datatype="html">
|
||||
<source>greater than</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">229</context>
|
||||
<context context-type="linenumber">232</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8014012170270529279" datatype="html">
|
||||
<source>less than</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">233</context>
|
||||
<context context-type="linenumber">236</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="5195932016807797291" datatype="html">
|
||||
@@ -9078,14 +9078,14 @@
|
||||
)?.name"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">274,278</context>
|
||||
<context context-type="linenumber">277,281</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8170755470576301659" datatype="html">
|
||||
<source>Without correspondent</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">280</context>
|
||||
<context context-type="linenumber">283</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="317796810569008208" datatype="html">
|
||||
@@ -9094,14 +9094,14 @@
|
||||
)?.name"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">286,290</context>
|
||||
<context context-type="linenumber">289,293</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="4362173610367509215" datatype="html">
|
||||
<source>Without document type</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">292</context>
|
||||
<context context-type="linenumber">295</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="232202047340644471" datatype="html">
|
||||
@@ -9110,70 +9110,70 @@
|
||||
)?.name"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">298,302</context>
|
||||
<context context-type="linenumber">301,305</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1562820715074533164" datatype="html">
|
||||
<source>Without storage path</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">304</context>
|
||||
<context context-type="linenumber">307</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8180755793012580465" datatype="html">
|
||||
<source>Tag: <x id="PH" equiv-text="this.tagSelectionModel.items.find((t) => t.id == +rule.value)?.name"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">308,310</context>
|
||||
<context context-type="linenumber">311,313</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6494566478302448576" datatype="html">
|
||||
<source>Without any tag</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">314</context>
|
||||
<context context-type="linenumber">317</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="8644099678903817943" datatype="html">
|
||||
<source>Custom fields query</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">318</context>
|
||||
<context context-type="linenumber">321</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6523384805359286307" datatype="html">
|
||||
<source>Title: <x id="PH" equiv-text="rule.value"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">321</context>
|
||||
<context context-type="linenumber">324</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1872523635812236432" datatype="html">
|
||||
<source>ASN: <x id="PH" equiv-text="rule.value"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">324</context>
|
||||
<context context-type="linenumber">327</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="102674688969746976" datatype="html">
|
||||
<source>Owner: <x id="PH" equiv-text="rule.value"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">327</context>
|
||||
<context context-type="linenumber">330</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="3550877650686009106" datatype="html">
|
||||
<source>Owner not in: <x id="PH" equiv-text="rule.value"/></source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">330</context>
|
||||
<context context-type="linenumber">333</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="1082034558646673343" datatype="html">
|
||||
<source>Without an owner</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/document-list/filter-editor/filter-editor.component.ts</context>
|
||||
<context context-type="linenumber">333</context>
|
||||
<context context-type="linenumber">336</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7210076240260527720" datatype="html">
|
||||
|
||||
@@ -20,9 +20,9 @@ import { Subject, filter, takeUntil } from 'rxjs'
|
||||
import { NEGATIVE_NULL_FILTER_VALUE } from 'src/app/data/filter-rule-type'
|
||||
import { MatchingModel } from 'src/app/data/matching-model'
|
||||
import { ObjectWithPermissions } from 'src/app/data/object-with-permissions'
|
||||
import { SelectionDataItem } from 'src/app/data/results'
|
||||
import { FilterPipe } from 'src/app/pipes/filter.pipe'
|
||||
import { HotKeyService } from 'src/app/services/hot-key.service'
|
||||
import { SelectionDataItem } from 'src/app/services/rest/document.service'
|
||||
import { pngxPopperOptions } from 'src/app/utils/popper-options'
|
||||
import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
|
||||
import { ClearableBadgeComponent } from '../clearable-badge/clearable-badge.component'
|
||||
|
||||
@@ -300,7 +300,7 @@ describe('BulkEditorComponent', () => {
|
||||
parameters: { add_tags: [101], remove_tags: [] },
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -332,7 +332,7 @@ describe('BulkEditorComponent', () => {
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -423,7 +423,7 @@ describe('BulkEditorComponent', () => {
|
||||
parameters: { correspondent: 101 },
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -455,7 +455,7 @@ describe('BulkEditorComponent', () => {
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -521,7 +521,7 @@ describe('BulkEditorComponent', () => {
|
||||
parameters: { document_type: 101 },
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -553,7 +553,7 @@ describe('BulkEditorComponent', () => {
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -619,7 +619,7 @@ describe('BulkEditorComponent', () => {
|
||||
parameters: { storage_path: 101 },
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -651,7 +651,7 @@ describe('BulkEditorComponent', () => {
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -717,7 +717,7 @@ describe('BulkEditorComponent', () => {
|
||||
parameters: { add_custom_fields: [101], remove_custom_fields: [102] },
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -749,7 +749,7 @@ describe('BulkEditorComponent', () => {
|
||||
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
|
||||
.flush(true)
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -858,7 +858,7 @@ describe('BulkEditorComponent', () => {
|
||||
documents: [3, 4],
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -951,7 +951,7 @@ describe('BulkEditorComponent', () => {
|
||||
documents: [3, 4],
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -986,7 +986,7 @@ describe('BulkEditorComponent', () => {
|
||||
source_mode: 'latest_version',
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -1027,7 +1027,7 @@ describe('BulkEditorComponent', () => {
|
||||
metadata_document_id: 3,
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -1046,7 +1046,7 @@ describe('BulkEditorComponent', () => {
|
||||
delete_originals: true,
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -1067,7 +1067,7 @@ describe('BulkEditorComponent', () => {
|
||||
archive_fallback: true,
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -1153,7 +1153,7 @@ describe('BulkEditorComponent', () => {
|
||||
},
|
||||
})
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
@@ -1460,7 +1460,7 @@ describe('BulkEditorComponent', () => {
|
||||
expect(toastServiceShowInfoSpy).toHaveBeenCalled()
|
||||
expect(listReloadSpy).toHaveBeenCalled()
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
) // list reload
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
|
||||
@@ -16,7 +16,6 @@ import { first, map, Observable, Subject, switchMap, takeUntil } from 'rxjs'
|
||||
import { ConfirmDialogComponent } from 'src/app/components/common/confirm-dialog/confirm-dialog.component'
|
||||
import { CustomField } from 'src/app/data/custom-field'
|
||||
import { MatchingModel } from 'src/app/data/matching-model'
|
||||
import { SelectionDataItem } from 'src/app/data/results'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
|
||||
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
|
||||
@@ -33,6 +32,7 @@ import {
|
||||
DocumentBulkEditMethod,
|
||||
DocumentService,
|
||||
MergeDocumentsRequest,
|
||||
SelectionDataItem,
|
||||
} from 'src/app/services/rest/document.service'
|
||||
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
|
||||
import { ShareLinkBundleService } from 'src/app/services/rest/share-link-bundle.service'
|
||||
|
||||
@@ -76,7 +76,6 @@ import {
|
||||
FILTER_TITLE_CONTENT,
|
||||
NEGATIVE_NULL_FILTER_VALUE,
|
||||
} from 'src/app/data/filter-rule-type'
|
||||
import { SelectionData, SelectionDataItem } from 'src/app/data/results'
|
||||
import {
|
||||
PermissionAction,
|
||||
PermissionType,
|
||||
@@ -85,7 +84,11 @@ import {
|
||||
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
|
||||
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
|
||||
import { DocumentTypeService } from 'src/app/services/rest/document-type.service'
|
||||
import { DocumentService } from 'src/app/services/rest/document.service'
|
||||
import {
|
||||
DocumentService,
|
||||
SelectionData,
|
||||
SelectionDataItem,
|
||||
} from 'src/app/services/rest/document.service'
|
||||
import { SearchService } from 'src/app/services/rest/search.service'
|
||||
import { StoragePathService } from 'src/app/services/rest/storage-path.service'
|
||||
import { TagService } from 'src/app/services/rest/tag.service'
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import { Document } from './document'
|
||||
|
||||
export interface Results<T> {
|
||||
count: number
|
||||
|
||||
@@ -7,20 +5,3 @@ export interface Results<T> {
|
||||
|
||||
all: number[]
|
||||
}
|
||||
|
||||
export interface SelectionDataItem {
|
||||
id: number
|
||||
document_count: number
|
||||
}
|
||||
|
||||
export interface SelectionData {
|
||||
selected_storage_paths: SelectionDataItem[]
|
||||
selected_correspondents: SelectionDataItem[]
|
||||
selected_tags: SelectionDataItem[]
|
||||
selected_document_types: SelectionDataItem[]
|
||||
selected_custom_fields: SelectionDataItem[]
|
||||
}
|
||||
|
||||
export interface DocumentResults extends Results<Document> {
|
||||
selection_data?: SelectionData
|
||||
}
|
||||
|
||||
@@ -127,10 +127,13 @@ describe('DocumentListViewService', () => {
|
||||
expect(documentListViewService.currentPage).toEqual(1)
|
||||
documentListViewService.reload()
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush(full_results)
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/selection_data/`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
expect(documentListViewService.isReloading).toBeFalsy()
|
||||
expect(documentListViewService.activeSavedViewId).toBeNull()
|
||||
@@ -142,12 +145,12 @@ describe('DocumentListViewService', () => {
|
||||
it('should handle error on page request out of range', () => {
|
||||
documentListViewService.currentPage = 50
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=50&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=50&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush([], { status: 404, statusText: 'Unexpected error' })
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
expect(documentListViewService.currentPage).toEqual(1)
|
||||
@@ -164,7 +167,7 @@ describe('DocumentListViewService', () => {
|
||||
]
|
||||
documentListViewService.setFilterRules(filterRulesAny)
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__in=${tags__id__in}`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__in=${tags__id__in}`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush(
|
||||
@@ -172,13 +175,13 @@ describe('DocumentListViewService', () => {
|
||||
{ status: 404, statusText: 'Unexpected error' }
|
||||
)
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
// reset the list
|
||||
documentListViewService.setFilterRules([])
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
})
|
||||
|
||||
@@ -186,7 +189,7 @@ describe('DocumentListViewService', () => {
|
||||
documentListViewService.currentPage = 1
|
||||
documentListViewService.sortField = 'custom_field_999'
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-custom_field_999&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-custom_field_999&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush(
|
||||
@@ -195,7 +198,7 @@ describe('DocumentListViewService', () => {
|
||||
)
|
||||
// resets itself
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
})
|
||||
|
||||
@@ -210,7 +213,7 @@ describe('DocumentListViewService', () => {
|
||||
]
|
||||
documentListViewService.setFilterRules(filterRulesAny)
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__in=${tags__id__in}`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__in=${tags__id__in}`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush('Generic error', { status: 404, statusText: 'Unexpected error' })
|
||||
@@ -218,7 +221,7 @@ describe('DocumentListViewService', () => {
|
||||
// reset the list
|
||||
documentListViewService.setFilterRules([])
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
})
|
||||
|
||||
@@ -227,7 +230,7 @@ describe('DocumentListViewService', () => {
|
||||
expect(documentListViewService.sortReverse).toBeTruthy()
|
||||
documentListViewService.setSort('added', false)
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=added&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=added&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
expect(documentListViewService.sortField).toEqual('added')
|
||||
@@ -235,12 +238,12 @@ describe('DocumentListViewService', () => {
|
||||
|
||||
documentListViewService.sortField = 'created'
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=created&truncate_content=true`
|
||||
)
|
||||
expect(documentListViewService.sortField).toEqual('created')
|
||||
documentListViewService.sortReverse = true
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
expect(documentListViewService.sortReverse).toBeTruthy()
|
||||
@@ -283,7 +286,7 @@ describe('DocumentListViewService', () => {
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=${page}&page_size=${
|
||||
documentListViewService.pageSize
|
||||
}&ordering=${reverse ? '-' : ''}${sort}&truncate_content=true&include_selection_data=true`
|
||||
}&ordering=${reverse ? '-' : ''}${sort}&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
expect(documentListViewService.currentPage).toEqual(page)
|
||||
@@ -300,7 +303,7 @@ describe('DocumentListViewService', () => {
|
||||
}
|
||||
documentListViewService.loadFromQueryParams(convertToParamMap(params))
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
|
||||
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&tags__id__all=${tags__id__all}`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
expect(documentListViewService.filterRules).toEqual([
|
||||
@@ -310,12 +313,15 @@ describe('DocumentListViewService', () => {
|
||||
},
|
||||
])
|
||||
req.flush(full_results)
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/selection_data/`
|
||||
)
|
||||
})
|
||||
|
||||
it('should use filter rules to update query params', () => {
|
||||
documentListViewService.setFilterRules(filterRules)
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
|
||||
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&tags__id__all=${tags__id__all}`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
})
|
||||
@@ -324,26 +330,34 @@ describe('DocumentListViewService', () => {
|
||||
documentListViewService.currentPage = 2
|
||||
let req = httpTestingController.expectOne((request) =>
|
||||
request.urlWithParams.startsWith(
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush(full_results)
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/selection_data/`
|
||||
)
|
||||
req.flush([])
|
||||
|
||||
documentListViewService.setFilterRules(filterRules, true)
|
||||
|
||||
const filteredReqs = httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=${tags__id__all}`
|
||||
)
|
||||
expect(filteredReqs).toHaveLength(1)
|
||||
filteredReqs[0].flush(full_results)
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/selection_data/`
|
||||
)
|
||||
req.flush([])
|
||||
expect(documentListViewService.currentPage).toEqual(1)
|
||||
})
|
||||
|
||||
it('should support quick filter', () => {
|
||||
documentListViewService.quickFilter(filterRules)
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
|
||||
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&tags__id__all=${tags__id__all}`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
})
|
||||
@@ -366,21 +380,21 @@ describe('DocumentListViewService', () => {
|
||||
convertToParamMap(params)
|
||||
)
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=${page}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&include_selection_data=true&tags__id__all=${tags__id__all}`
|
||||
`${environment.apiBaseUrl}documents/?page=${page}&page_size=${documentListViewService.pageSize}&ordering=-added&truncate_content=true&tags__id__all=${tags__id__all}`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
// reset the list
|
||||
documentListViewService.currentPage = 1
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&include_selection_data=true&tags__id__all=9`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&tags__id__all=9`
|
||||
)
|
||||
documentListViewService.setFilterRules([])
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true`
|
||||
)
|
||||
documentListViewService.sortField = 'created'
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
documentListViewService.activateSavedView(null)
|
||||
})
|
||||
@@ -388,18 +402,21 @@ describe('DocumentListViewService', () => {
|
||||
it('should support navigating next / previous', () => {
|
||||
documentListViewService.setFilterRules([])
|
||||
let req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(documentListViewService.currentPage).toEqual(1)
|
||||
documentListViewService.pageSize = 3
|
||||
req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush({
|
||||
count: 3,
|
||||
results: documents.slice(0, 3),
|
||||
})
|
||||
httpTestingController
|
||||
.expectOne(`${environment.apiBaseUrl}documents/selection_data/`)
|
||||
.flush([])
|
||||
expect(documentListViewService.hasNext(documents[0].id)).toBeTruthy()
|
||||
expect(documentListViewService.hasPrevious(documents[0].id)).toBeFalsy()
|
||||
documentListViewService.getNext(documents[0].id).subscribe((docId) => {
|
||||
@@ -446,7 +463,7 @@ describe('DocumentListViewService', () => {
|
||||
expect(documentListViewService.currentPage).toEqual(1)
|
||||
documentListViewService.pageSize = 3
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true`
|
||||
)
|
||||
jest
|
||||
.spyOn(documentListViewService, 'getLastPage')
|
||||
@@ -461,7 +478,7 @@ describe('DocumentListViewService', () => {
|
||||
expect(reloadSpy).toHaveBeenCalled()
|
||||
expect(documentListViewService.currentPage).toEqual(2)
|
||||
const reqs = httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(reqs.length).toBeGreaterThan(0)
|
||||
})
|
||||
@@ -496,11 +513,11 @@ describe('DocumentListViewService', () => {
|
||||
.mockReturnValue(documents)
|
||||
documentListViewService.currentPage = 2
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
documentListViewService.pageSize = 3
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=2&page_size=3&ordering=-created&truncate_content=true`
|
||||
)
|
||||
const reloadSpy = jest.spyOn(documentListViewService, 'reload')
|
||||
documentListViewService.getPrevious(1).subscribe({
|
||||
@@ -510,7 +527,7 @@ describe('DocumentListViewService', () => {
|
||||
expect(reloadSpy).toHaveBeenCalled()
|
||||
expect(documentListViewService.currentPage).toEqual(1)
|
||||
const reqs = httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(reqs.length).toBeGreaterThan(0)
|
||||
})
|
||||
@@ -523,10 +540,13 @@ describe('DocumentListViewService', () => {
|
||||
it('should support select a document', () => {
|
||||
documentListViewService.reload()
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush(full_results)
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/selection_data/`
|
||||
)
|
||||
documentListViewService.toggleSelected(documents[0])
|
||||
expect(documentListViewService.isSelected(documents[0])).toBeTruthy()
|
||||
documentListViewService.toggleSelected(documents[0])
|
||||
@@ -548,13 +568,16 @@ describe('DocumentListViewService', () => {
|
||||
it('should support select page', () => {
|
||||
documentListViewService.pageSize = 3
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=3&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush({
|
||||
count: 3,
|
||||
results: documents.slice(0, 3),
|
||||
})
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/selection_data/`
|
||||
)
|
||||
documentListViewService.selectPage()
|
||||
expect(documentListViewService.selected.size).toEqual(3)
|
||||
expect(documentListViewService.isSelected(documents[5])).toBeFalsy()
|
||||
@@ -563,10 +586,13 @@ describe('DocumentListViewService', () => {
|
||||
it('should support select range', () => {
|
||||
documentListViewService.reload()
|
||||
const req = httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(req.request.method).toEqual('GET')
|
||||
req.flush(full_results)
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/selection_data/`
|
||||
)
|
||||
documentListViewService.toggleSelected(documents[0])
|
||||
expect(documentListViewService.isSelected(documents[0])).toBeTruthy()
|
||||
documentListViewService.selectRangeTo(documents[2])
|
||||
@@ -586,7 +612,7 @@ describe('DocumentListViewService', () => {
|
||||
|
||||
documentListViewService.setFilterRules(filterRules)
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=9`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=9`
|
||||
)
|
||||
const reqs = httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id&tags__id__all=9`
|
||||
@@ -602,7 +628,7 @@ describe('DocumentListViewService', () => {
|
||||
const cancelSpy = jest.spyOn(documentListViewService, 'cancelPending')
|
||||
documentListViewService.reload()
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true&tags__id__all=9`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=9`
|
||||
)
|
||||
expect(cancelSpy).toHaveBeenCalled()
|
||||
})
|
||||
@@ -621,7 +647,7 @@ describe('DocumentListViewService', () => {
|
||||
documentListViewService.setFilterRules([])
|
||||
expect(documentListViewService.sortField).toEqual('created')
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
})
|
||||
|
||||
@@ -648,11 +674,11 @@ describe('DocumentListViewService', () => {
|
||||
expect(localStorageSpy).toHaveBeenCalled()
|
||||
// reload triggered
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
documentListViewService.displayFields = null
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
expect(documentListViewService.displayFields).toEqual(
|
||||
DEFAULT_DISPLAY_FIELDS.filter((f) => f.id !== DisplayField.ADDED).map(
|
||||
@@ -692,7 +718,7 @@ describe('DocumentListViewService', () => {
|
||||
it('should generate quick filter URL preserving default state', () => {
|
||||
documentListViewService.reload()
|
||||
httpTestingController.expectOne(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
|
||||
)
|
||||
const urlTree = documentListViewService.getQuickFilterUrl(filterRules)
|
||||
expect(urlTree).toBeDefined()
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { Injectable, inject } from '@angular/core'
|
||||
import { ParamMap, Router, UrlTree } from '@angular/router'
|
||||
import { Observable, Subject, takeUntil } from 'rxjs'
|
||||
import { Observable, Subject, first, takeUntil } from 'rxjs'
|
||||
import {
|
||||
DEFAULT_DISPLAY_FIELDS,
|
||||
DisplayField,
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
Document,
|
||||
} from '../data/document'
|
||||
import { FilterRule } from '../data/filter-rule'
|
||||
import { DocumentResults, SelectionData } from '../data/results'
|
||||
import { SavedView } from '../data/saved-view'
|
||||
import { DOCUMENT_LIST_SERVICE } from '../data/storage-keys'
|
||||
import { SETTINGS_KEYS } from '../data/ui-settings'
|
||||
@@ -18,7 +17,7 @@ import {
|
||||
isFullTextFilterRule,
|
||||
} from '../utils/filter-rules'
|
||||
import { paramsFromViewState, paramsToViewState } from '../utils/query-params'
|
||||
import { DocumentService } from './rest/document.service'
|
||||
import { DocumentService, SelectionData } from './rest/document.service'
|
||||
import { SettingsService } from './settings.service'
|
||||
|
||||
const LIST_DEFAULT_DISPLAY_FIELDS: DisplayField[] = DEFAULT_DISPLAY_FIELDS.map(
|
||||
@@ -294,17 +293,27 @@ export class DocumentListViewService {
|
||||
activeListViewState.sortField,
|
||||
activeListViewState.sortReverse,
|
||||
activeListViewState.filterRules,
|
||||
{ truncate_content: true, include_selection_data: true }
|
||||
{ truncate_content: true }
|
||||
)
|
||||
.pipe(takeUntil(this.unsubscribeNotifier))
|
||||
.subscribe({
|
||||
next: (result) => {
|
||||
const resultWithSelectionData = result as DocumentResults
|
||||
this.initialized = true
|
||||
this.isReloading = false
|
||||
activeListViewState.collectionSize = result.count
|
||||
activeListViewState.documents = result.results
|
||||
this.selectionData = resultWithSelectionData.selection_data ?? null
|
||||
|
||||
this.documentService
|
||||
.getSelectionData(result.all)
|
||||
.pipe(first())
|
||||
.subscribe({
|
||||
next: (selectionData) => {
|
||||
this.selectionData = selectionData
|
||||
},
|
||||
error: () => {
|
||||
this.selectionData = null
|
||||
},
|
||||
})
|
||||
|
||||
if (updateQueryParams && !this._activeSavedViewId) {
|
||||
let base = ['/documents']
|
||||
|
||||
@@ -12,7 +12,7 @@ import {
|
||||
import { DocumentMetadata } from 'src/app/data/document-metadata'
|
||||
import { DocumentSuggestions } from 'src/app/data/document-suggestions'
|
||||
import { FilterRule } from 'src/app/data/filter-rule'
|
||||
import { Results, SelectionData } from 'src/app/data/results'
|
||||
import { Results } from 'src/app/data/results'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { queryParamsFromFilterRules } from '../../utils/query-params'
|
||||
import {
|
||||
@@ -24,6 +24,19 @@ import { SettingsService } from '../settings.service'
|
||||
import { AbstractPaperlessService } from './abstract-paperless-service'
|
||||
import { CustomFieldsService } from './custom-fields.service'
|
||||
|
||||
export interface SelectionDataItem {
|
||||
id: number
|
||||
document_count: number
|
||||
}
|
||||
|
||||
export interface SelectionData {
|
||||
selected_storage_paths: SelectionDataItem[]
|
||||
selected_correspondents: SelectionDataItem[]
|
||||
selected_tags: SelectionDataItem[]
|
||||
selected_document_types: SelectionDataItem[]
|
||||
selected_custom_fields: SelectionDataItem[]
|
||||
}
|
||||
|
||||
export enum BulkEditSourceMode {
|
||||
LATEST_VERSION = 'latest_version',
|
||||
EXPLICIT_SELECTION = 'explicit_selection',
|
||||
|
||||
@@ -45,6 +45,8 @@ from documents.models import DocumentType
|
||||
from documents.models import Note
|
||||
from documents.models import SavedView
|
||||
from documents.models import SavedViewFilterRule
|
||||
from documents.models import ShareLink
|
||||
from documents.models import ShareLinkBundle
|
||||
from documents.models import StoragePath
|
||||
from documents.models import Tag
|
||||
from documents.models import UiSettings
|
||||
@@ -55,6 +57,7 @@ from documents.models import WorkflowActionWebhook
|
||||
from documents.models import WorkflowTrigger
|
||||
from documents.settings import EXPORTER_ARCHIVE_NAME
|
||||
from documents.settings import EXPORTER_FILE_NAME
|
||||
from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME
|
||||
from documents.settings import EXPORTER_THUMBNAIL_NAME
|
||||
from documents.utils import compute_checksum
|
||||
from documents.utils import copy_file_with_basic_stats
|
||||
@@ -389,6 +392,8 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
"app_configs": ApplicationConfiguration.objects.all(),
|
||||
"notes": Note.global_objects.all(),
|
||||
"documents": Document.global_objects.order_by("id").all(),
|
||||
"share_links": ShareLink.global_objects.all(),
|
||||
"share_link_bundles": ShareLinkBundle.objects.order_by("id").all(),
|
||||
"social_accounts": SocialAccount.objects.all(),
|
||||
"social_apps": SocialApp.objects.all(),
|
||||
"social_tokens": SocialToken.objects.all(),
|
||||
@@ -409,6 +414,7 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
)
|
||||
|
||||
document_manifest: list[dict] = []
|
||||
share_link_bundle_manifest: list[dict] = []
|
||||
manifest_path = (self.target / "manifest.json").resolve()
|
||||
|
||||
with StreamingManifestWriter(
|
||||
@@ -427,6 +433,15 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
for record in batch:
|
||||
self._encrypt_record_inline(record)
|
||||
document_manifest.extend(batch)
|
||||
elif key == "share_link_bundles":
|
||||
# Accumulate for file-copy loop; written to manifest after
|
||||
for batch in serialize_queryset_batched(
|
||||
qs,
|
||||
batch_size=self.batch_size,
|
||||
):
|
||||
for record in batch:
|
||||
self._encrypt_record_inline(record)
|
||||
share_link_bundle_manifest.extend(batch)
|
||||
elif self.split_manifest and key in (
|
||||
"notes",
|
||||
"custom_field_instances",
|
||||
@@ -445,6 +460,12 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
document_map: dict[int, Document] = {
|
||||
d.pk: d for d in Document.global_objects.order_by("id")
|
||||
}
|
||||
share_link_bundle_map: dict[int, ShareLinkBundle] = {
|
||||
b.pk: b
|
||||
for b in ShareLinkBundle.objects.order_by("id").prefetch_related(
|
||||
"documents",
|
||||
)
|
||||
}
|
||||
|
||||
# 3. Export files from each document
|
||||
for index, document_dict in enumerate(
|
||||
@@ -478,6 +499,19 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
else:
|
||||
writer.write_record(document_dict)
|
||||
|
||||
for bundle_dict in share_link_bundle_manifest:
|
||||
bundle = share_link_bundle_map[bundle_dict["pk"]]
|
||||
|
||||
bundle_target = self.generate_share_link_bundle_target(
|
||||
bundle,
|
||||
bundle_dict,
|
||||
)
|
||||
|
||||
if not self.data_only and bundle_target is not None:
|
||||
self.copy_share_link_bundle_file(bundle, bundle_target)
|
||||
|
||||
writer.write_record(bundle_dict)
|
||||
|
||||
# 4.2 write version information to target folder
|
||||
extra_metadata_path = (self.target / "metadata.json").resolve()
|
||||
metadata: dict[str, str | int | dict[str, str | int]] = {
|
||||
@@ -598,6 +632,47 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
archive_target,
|
||||
)
|
||||
|
||||
def generate_share_link_bundle_target(
|
||||
self,
|
||||
bundle: ShareLinkBundle,
|
||||
bundle_dict: dict,
|
||||
) -> Path | None:
|
||||
"""
|
||||
Generates the export target for a share link bundle file, when present.
|
||||
"""
|
||||
if not bundle.file_path:
|
||||
return None
|
||||
|
||||
bundle_name = Path(bundle.file_path)
|
||||
if bundle_name.is_absolute():
|
||||
bundle_name = Path(bundle_name.name)
|
||||
|
||||
bundle_name = Path("share_link_bundles") / bundle_name
|
||||
bundle_target = (self.target / bundle_name).resolve()
|
||||
bundle_dict["fields"]["file_path"] = str(
|
||||
bundle_name.relative_to("share_link_bundles"),
|
||||
)
|
||||
bundle_dict[EXPORTER_SHARE_LINK_BUNDLE_NAME] = str(bundle_name)
|
||||
return bundle_target
|
||||
|
||||
def copy_share_link_bundle_file(
|
||||
self,
|
||||
bundle: ShareLinkBundle,
|
||||
bundle_target: Path,
|
||||
) -> None:
|
||||
"""
|
||||
Copies a share link bundle ZIP into the export directory.
|
||||
"""
|
||||
bundle_source_path = bundle.absolute_file_path
|
||||
if bundle_source_path is None:
|
||||
raise FileNotFoundError(f"Share link bundle {bundle.pk} has no file path")
|
||||
|
||||
self.check_and_copy(
|
||||
bundle_source_path,
|
||||
None,
|
||||
bundle_target,
|
||||
)
|
||||
|
||||
def _encrypt_record_inline(self, record: dict) -> None:
|
||||
"""Encrypt sensitive fields in a single record, if passphrase is set."""
|
||||
if not self.passphrase:
|
||||
|
||||
@@ -32,10 +32,12 @@ from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import DocumentType
|
||||
from documents.models import Note
|
||||
from documents.models import ShareLinkBundle
|
||||
from documents.models import Tag
|
||||
from documents.settings import EXPORTER_ARCHIVE_NAME
|
||||
from documents.settings import EXPORTER_CRYPTO_SETTINGS_NAME
|
||||
from documents.settings import EXPORTER_FILE_NAME
|
||||
from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME
|
||||
from documents.settings import EXPORTER_THUMBNAIL_NAME
|
||||
from documents.signals.handlers import check_paths_and_prune_custom_fields
|
||||
from documents.signals.handlers import update_filename_and_move_files
|
||||
@@ -348,18 +350,42 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
f"Failed to read from archive file {doc_archive_path}",
|
||||
) from e
|
||||
|
||||
def check_share_link_bundle_validity(bundle_record: dict) -> None:
|
||||
if EXPORTER_SHARE_LINK_BUNDLE_NAME not in bundle_record:
|
||||
return
|
||||
|
||||
bundle_file = bundle_record[EXPORTER_SHARE_LINK_BUNDLE_NAME]
|
||||
bundle_path: Path = self.source / bundle_file
|
||||
if not bundle_path.exists():
|
||||
raise CommandError(
|
||||
f'The manifest file refers to "{bundle_file}" which does not '
|
||||
"appear to be in the source directory.",
|
||||
)
|
||||
try:
|
||||
with bundle_path.open(mode="rb"):
|
||||
pass
|
||||
except Exception as e:
|
||||
raise CommandError(
|
||||
f"Failed to read from share link bundle file {bundle_path}",
|
||||
) from e
|
||||
|
||||
self.stdout.write("Checking the manifest")
|
||||
for manifest_path in self.manifest_paths:
|
||||
for record in iter_manifest_records(manifest_path):
|
||||
# Only check if the document files exist if this is not data only
|
||||
# We don't care about documents for a data only import
|
||||
if not self.data_only and record["model"] == "documents.document":
|
||||
if self.data_only:
|
||||
continue
|
||||
if record["model"] == "documents.document":
|
||||
check_document_validity(record)
|
||||
elif record["model"] == "documents.sharelinkbundle":
|
||||
check_share_link_bundle_validity(record)
|
||||
|
||||
def _import_files_from_manifest(self) -> None:
|
||||
settings.ORIGINALS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
settings.THUMBNAIL_DIR.mkdir(parents=True, exist_ok=True)
|
||||
settings.ARCHIVE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
settings.SHARE_LINK_BUNDLE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
self.stdout.write("Copy files into paperless...")
|
||||
|
||||
@@ -374,6 +400,18 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
for record in iter_manifest_records(manifest_path)
|
||||
if record["model"] == "documents.document"
|
||||
]
|
||||
share_link_bundle_records = [
|
||||
{
|
||||
"pk": record["pk"],
|
||||
EXPORTER_SHARE_LINK_BUNDLE_NAME: record.get(
|
||||
EXPORTER_SHARE_LINK_BUNDLE_NAME,
|
||||
),
|
||||
}
|
||||
for manifest_path in self.manifest_paths
|
||||
for record in iter_manifest_records(manifest_path)
|
||||
if record["model"] == "documents.sharelinkbundle"
|
||||
and record.get(EXPORTER_SHARE_LINK_BUNDLE_NAME)
|
||||
]
|
||||
|
||||
for record in self.track(document_records, description="Copying files..."):
|
||||
document = Document.global_objects.get(pk=record["pk"])
|
||||
@@ -416,6 +454,26 @@ class Command(CryptMixin, PaperlessCommand):
|
||||
|
||||
document.save()
|
||||
|
||||
for record in self.track(
|
||||
share_link_bundle_records,
|
||||
description="Copying share link bundles...",
|
||||
):
|
||||
bundle = ShareLinkBundle.objects.get(pk=record["pk"])
|
||||
bundle_file = record[EXPORTER_SHARE_LINK_BUNDLE_NAME]
|
||||
bundle_source_path = (self.source / bundle_file).resolve()
|
||||
bundle_target_path = bundle.absolute_file_path
|
||||
if bundle_target_path is None:
|
||||
raise CommandError(
|
||||
f"Share link bundle {bundle.pk} does not have a valid file path.",
|
||||
)
|
||||
|
||||
with FileLock(settings.MEDIA_LOCK):
|
||||
bundle_target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
copy_file_with_basic_stats(
|
||||
bundle_source_path,
|
||||
bundle_target_path,
|
||||
)
|
||||
|
||||
def _decrypt_record_if_needed(self, record: dict) -> dict:
|
||||
fields = self.CRYPT_FIELDS_BY_MODEL.get(record.get("model", ""))
|
||||
if fields:
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
EXPORTER_FILE_NAME = "__exported_file_name__"
|
||||
EXPORTER_THUMBNAIL_NAME = "__exported_thumbnail_name__"
|
||||
EXPORTER_ARCHIVE_NAME = "__exported_archive_name__"
|
||||
EXPORTER_SHARE_LINK_BUNDLE_NAME = "__exported_share_link_bundle_name__"
|
||||
|
||||
EXPORTER_CRYPTO_SETTINGS_NAME = "__crypto__"
|
||||
EXPORTER_CRYPTO_SALT_NAME = "__salt_hex__"
|
||||
|
||||
@@ -1144,56 +1144,6 @@ class TestDocumentApi(DirectoriesMixin, DocumentConsumeDelayMixin, APITestCase):
|
||||
self.assertEqual(len(response.data["all"]), 50)
|
||||
self.assertCountEqual(response.data["all"], [d.id for d in docs])
|
||||
|
||||
def test_list_with_include_selection_data(self) -> None:
|
||||
correspondent = Correspondent.objects.create(name="c1")
|
||||
doc_type = DocumentType.objects.create(name="dt1")
|
||||
storage_path = StoragePath.objects.create(name="sp1")
|
||||
tag = Tag.objects.create(name="tag")
|
||||
|
||||
matching_doc = Document.objects.create(
|
||||
checksum="A",
|
||||
correspondent=correspondent,
|
||||
document_type=doc_type,
|
||||
storage_path=storage_path,
|
||||
)
|
||||
matching_doc.tags.add(tag)
|
||||
|
||||
non_matching_doc = Document.objects.create(checksum="B")
|
||||
non_matching_doc.tags.add(Tag.objects.create(name="other"))
|
||||
|
||||
response = self.client.get(
|
||||
f"/api/documents/?tags__id__in={tag.id}&include_selection_data=true",
|
||||
)
|
||||
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
self.assertIn("selection_data", response.data)
|
||||
|
||||
selected_correspondent = next(
|
||||
item
|
||||
for item in response.data["selection_data"]["selected_correspondents"]
|
||||
if item["id"] == correspondent.id
|
||||
)
|
||||
selected_tag = next(
|
||||
item
|
||||
for item in response.data["selection_data"]["selected_tags"]
|
||||
if item["id"] == tag.id
|
||||
)
|
||||
selected_type = next(
|
||||
item
|
||||
for item in response.data["selection_data"]["selected_document_types"]
|
||||
if item["id"] == doc_type.id
|
||||
)
|
||||
selected_storage_path = next(
|
||||
item
|
||||
for item in response.data["selection_data"]["selected_storage_paths"]
|
||||
if item["id"] == storage_path.id
|
||||
)
|
||||
|
||||
self.assertEqual(selected_correspondent["document_count"], 1)
|
||||
self.assertEqual(selected_tag["document_count"], 1)
|
||||
self.assertEqual(selected_type["document_count"], 1)
|
||||
self.assertEqual(selected_storage_path["document_count"], 1)
|
||||
|
||||
def test_statistics(self) -> None:
|
||||
doc1 = Document.objects.create(
|
||||
title="none1",
|
||||
|
||||
@@ -89,46 +89,6 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
|
||||
self.assertEqual(len(results), 0)
|
||||
self.assertCountEqual(response.data["all"], [])
|
||||
|
||||
def test_search_with_include_selection_data(self) -> None:
|
||||
correspondent = Correspondent.objects.create(name="c1")
|
||||
doc_type = DocumentType.objects.create(name="dt1")
|
||||
storage_path = StoragePath.objects.create(name="sp1")
|
||||
tag = Tag.objects.create(name="tag")
|
||||
|
||||
matching_doc = Document.objects.create(
|
||||
title="bank statement",
|
||||
content="bank content",
|
||||
checksum="A",
|
||||
correspondent=correspondent,
|
||||
document_type=doc_type,
|
||||
storage_path=storage_path,
|
||||
)
|
||||
matching_doc.tags.add(tag)
|
||||
|
||||
with AsyncWriter(index.open_index()) as writer:
|
||||
index.update_document(writer, matching_doc)
|
||||
|
||||
response = self.client.get(
|
||||
"/api/documents/?query=bank&include_selection_data=true",
|
||||
)
|
||||
|
||||
self.assertEqual(response.status_code, status.HTTP_200_OK)
|
||||
self.assertIn("selection_data", response.data)
|
||||
|
||||
selected_correspondent = next(
|
||||
item
|
||||
for item in response.data["selection_data"]["selected_correspondents"]
|
||||
if item["id"] == correspondent.id
|
||||
)
|
||||
selected_tag = next(
|
||||
item
|
||||
for item in response.data["selection_data"]["selected_tags"]
|
||||
if item["id"] == tag.id
|
||||
)
|
||||
|
||||
self.assertEqual(selected_correspondent["document_count"], 1)
|
||||
self.assertEqual(selected_tag["document_count"], 1)
|
||||
|
||||
def test_search_custom_field_ordering(self) -> None:
|
||||
custom_field = CustomField.objects.create(
|
||||
name="Sortable field",
|
||||
|
||||
@@ -2,6 +2,7 @@ import hashlib
|
||||
import json
|
||||
import shutil
|
||||
import tempfile
|
||||
from datetime import timedelta
|
||||
from io import StringIO
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
@@ -11,6 +12,7 @@ import pytest
|
||||
from allauth.socialaccount.models import SocialAccount
|
||||
from allauth.socialaccount.models import SocialApp
|
||||
from allauth.socialaccount.models import SocialToken
|
||||
from django.conf import settings
|
||||
from django.contrib.auth.models import Group
|
||||
from django.contrib.auth.models import Permission
|
||||
from django.contrib.contenttypes.models import ContentType
|
||||
@@ -31,6 +33,8 @@ from documents.models import CustomFieldInstance
|
||||
from documents.models import Document
|
||||
from documents.models import DocumentType
|
||||
from documents.models import Note
|
||||
from documents.models import ShareLink
|
||||
from documents.models import ShareLinkBundle
|
||||
from documents.models import StoragePath
|
||||
from documents.models import Tag
|
||||
from documents.models import User
|
||||
@@ -39,6 +43,7 @@ from documents.models import WorkflowAction
|
||||
from documents.models import WorkflowTrigger
|
||||
from documents.sanity_checker import check_sanity
|
||||
from documents.settings import EXPORTER_FILE_NAME
|
||||
from documents.settings import EXPORTER_SHARE_LINK_BUNDLE_NAME
|
||||
from documents.tests.utils import DirectoriesMixin
|
||||
from documents.tests.utils import FileSystemAssertsMixin
|
||||
from documents.tests.utils import SampleDirMixin
|
||||
@@ -306,6 +311,108 @@ class TestExportImport(
|
||||
):
|
||||
self.test_exporter(use_filename_format=True)
|
||||
|
||||
def test_exporter_includes_share_links_and_bundles(self) -> None:
|
||||
shutil.rmtree(Path(self.dirs.media_dir) / "documents")
|
||||
shutil.copytree(
|
||||
Path(__file__).parent / "samples" / "documents",
|
||||
Path(self.dirs.media_dir) / "documents",
|
||||
)
|
||||
|
||||
share_link = ShareLink.objects.create(
|
||||
slug="share-link-slug",
|
||||
document=self.d1,
|
||||
owner=self.user,
|
||||
file_version=ShareLink.FileVersion.ORIGINAL,
|
||||
expiration=timezone.now() + timedelta(days=7),
|
||||
)
|
||||
|
||||
bundle_relative_path = Path("nested") / "share-bundle.zip"
|
||||
bundle_source_path = settings.SHARE_LINK_BUNDLE_DIR / bundle_relative_path
|
||||
bundle_source_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
bundle_source_path.write_bytes(b"share-bundle-contents")
|
||||
bundle = ShareLinkBundle.objects.create(
|
||||
slug="share-bundle-slug",
|
||||
owner=self.user,
|
||||
file_version=ShareLink.FileVersion.ARCHIVE,
|
||||
expiration=timezone.now() + timedelta(days=7),
|
||||
status=ShareLinkBundle.Status.READY,
|
||||
size_bytes=bundle_source_path.stat().st_size,
|
||||
file_path=str(bundle_relative_path),
|
||||
built_at=timezone.now(),
|
||||
)
|
||||
bundle.documents.set([self.d1, self.d2])
|
||||
|
||||
manifest = self._do_export()
|
||||
|
||||
share_link_records = [
|
||||
record for record in manifest if record["model"] == "documents.sharelink"
|
||||
]
|
||||
self.assertEqual(len(share_link_records), 1)
|
||||
self.assertEqual(share_link_records[0]["pk"], share_link.pk)
|
||||
self.assertEqual(share_link_records[0]["fields"]["document"], self.d1.pk)
|
||||
self.assertEqual(share_link_records[0]["fields"]["owner"], self.user.pk)
|
||||
|
||||
share_link_bundle_records = [
|
||||
record
|
||||
for record in manifest
|
||||
if record["model"] == "documents.sharelinkbundle"
|
||||
]
|
||||
self.assertEqual(len(share_link_bundle_records), 1)
|
||||
bundle_record = share_link_bundle_records[0]
|
||||
self.assertEqual(bundle_record["pk"], bundle.pk)
|
||||
self.assertEqual(
|
||||
bundle_record["fields"]["documents"],
|
||||
[self.d1.pk, self.d2.pk],
|
||||
)
|
||||
self.assertEqual(
|
||||
bundle_record[EXPORTER_SHARE_LINK_BUNDLE_NAME],
|
||||
"share_link_bundles/nested/share-bundle.zip",
|
||||
)
|
||||
self.assertEqual(
|
||||
bundle_record["fields"]["file_path"],
|
||||
"nested/share-bundle.zip",
|
||||
)
|
||||
self.assertIsFile(self.target / bundle_record[EXPORTER_SHARE_LINK_BUNDLE_NAME])
|
||||
|
||||
with paperless_environment():
|
||||
ShareLink.objects.all().delete()
|
||||
ShareLinkBundle.objects.all().delete()
|
||||
shutil.rmtree(settings.SHARE_LINK_BUNDLE_DIR, ignore_errors=True)
|
||||
|
||||
call_command(
|
||||
"document_importer",
|
||||
"--no-progress-bar",
|
||||
self.target,
|
||||
skip_checks=True,
|
||||
)
|
||||
|
||||
imported_share_link = ShareLink.objects.get(pk=share_link.pk)
|
||||
self.assertEqual(imported_share_link.document_id, self.d1.pk)
|
||||
self.assertEqual(imported_share_link.owner_id, self.user.pk)
|
||||
self.assertEqual(
|
||||
imported_share_link.file_version,
|
||||
ShareLink.FileVersion.ORIGINAL,
|
||||
)
|
||||
|
||||
imported_bundle = ShareLinkBundle.objects.get(pk=bundle.pk)
|
||||
imported_bundle_path = imported_bundle.absolute_file_path
|
||||
self.assertEqual(imported_bundle.owner_id, self.user.pk)
|
||||
self.assertEqual(
|
||||
list(
|
||||
imported_bundle.documents.order_by("pk").values_list(
|
||||
"pk",
|
||||
flat=True,
|
||||
),
|
||||
),
|
||||
[self.d1.pk, self.d2.pk],
|
||||
)
|
||||
self.assertEqual(imported_bundle.file_path, "nested/share-bundle.zip")
|
||||
self.assertIsNotNone(imported_bundle_path)
|
||||
self.assertEqual(
|
||||
imported_bundle_path.read_bytes(),
|
||||
b"share-bundle-contents",
|
||||
)
|
||||
|
||||
def test_update_export_changed_time(self) -> None:
|
||||
shutil.rmtree(Path(self.dirs.media_dir) / "documents")
|
||||
shutil.copytree(
|
||||
|
||||
@@ -836,61 +836,6 @@ class DocumentViewSet(
|
||||
"custom_field_",
|
||||
)
|
||||
|
||||
def _get_selection_data_for_queryset(self, queryset):
|
||||
correspondents = Correspondent.objects.annotate(
|
||||
document_count=Count(
|
||||
"documents",
|
||||
filter=Q(documents__in=queryset),
|
||||
distinct=True,
|
||||
),
|
||||
)
|
||||
tags = Tag.objects.annotate(
|
||||
document_count=Count(
|
||||
"documents",
|
||||
filter=Q(documents__in=queryset),
|
||||
distinct=True,
|
||||
),
|
||||
)
|
||||
document_types = DocumentType.objects.annotate(
|
||||
document_count=Count(
|
||||
"documents",
|
||||
filter=Q(documents__in=queryset),
|
||||
distinct=True,
|
||||
),
|
||||
)
|
||||
storage_paths = StoragePath.objects.annotate(
|
||||
document_count=Count(
|
||||
"documents",
|
||||
filter=Q(documents__in=queryset),
|
||||
distinct=True,
|
||||
),
|
||||
)
|
||||
custom_fields = CustomField.objects.annotate(
|
||||
document_count=Count(
|
||||
"fields__document",
|
||||
filter=Q(fields__document__in=queryset),
|
||||
distinct=True,
|
||||
),
|
||||
)
|
||||
|
||||
return {
|
||||
"selected_correspondents": [
|
||||
{"id": t.id, "document_count": t.document_count} for t in correspondents
|
||||
],
|
||||
"selected_tags": [
|
||||
{"id": t.id, "document_count": t.document_count} for t in tags
|
||||
],
|
||||
"selected_document_types": [
|
||||
{"id": t.id, "document_count": t.document_count} for t in document_types
|
||||
],
|
||||
"selected_storage_paths": [
|
||||
{"id": t.id, "document_count": t.document_count} for t in storage_paths
|
||||
],
|
||||
"selected_custom_fields": [
|
||||
{"id": t.id, "document_count": t.document_count} for t in custom_fields
|
||||
],
|
||||
}
|
||||
|
||||
def get_queryset(self):
|
||||
latest_version_content = Subquery(
|
||||
Document.objects.filter(root_document=OuterRef("pk"))
|
||||
@@ -1038,25 +983,6 @@ class DocumentViewSet(
|
||||
|
||||
return response
|
||||
|
||||
def list(self, request, *args, **kwargs):
|
||||
if not get_boolean(
|
||||
str(request.query_params.get("include_selection_data", "false")),
|
||||
):
|
||||
return super().list(request, *args, **kwargs)
|
||||
|
||||
queryset = self.filter_queryset(self.get_queryset())
|
||||
selection_data = self._get_selection_data_for_queryset(queryset)
|
||||
|
||||
page = self.paginate_queryset(queryset)
|
||||
if page is not None:
|
||||
serializer = self.get_serializer(page, many=True)
|
||||
response = self.get_paginated_response(serializer.data)
|
||||
response.data["selection_data"] = selection_data
|
||||
return response
|
||||
|
||||
serializer = self.get_serializer(queryset, many=True)
|
||||
return Response({"results": serializer.data, "selection_data": selection_data})
|
||||
|
||||
def destroy(self, request, *args, **kwargs):
|
||||
from documents import index
|
||||
|
||||
@@ -2097,21 +2023,6 @@ class UnifiedSearchViewSet(DocumentViewSet):
|
||||
else None
|
||||
)
|
||||
|
||||
if get_boolean(
|
||||
str(
|
||||
request.query_params.get(
|
||||
"include_selection_data",
|
||||
"false",
|
||||
),
|
||||
),
|
||||
):
|
||||
result_ids = response.data.get("all", [])
|
||||
response.data["selection_data"] = (
|
||||
self._get_selection_data_for_queryset(
|
||||
Document.objects.filter(pk__in=result_ids),
|
||||
)
|
||||
)
|
||||
|
||||
return response
|
||||
except NotFound:
|
||||
raise
|
||||
|
||||
@@ -2,7 +2,7 @@ msgid ""
|
||||
msgstr ""
|
||||
"Project-Id-Version: paperless-ngx\n"
|
||||
"Report-Msgid-Bugs-To: \n"
|
||||
"POT-Creation-Date: 2026-03-30 16:39+0000\n"
|
||||
"POT-Creation-Date: 2026-03-28 20:59+0000\n"
|
||||
"PO-Revision-Date: 2022-02-17 04:17\n"
|
||||
"Last-Translator: \n"
|
||||
"Language-Team: English\n"
|
||||
@@ -1300,8 +1300,8 @@ msgid "workflow runs"
|
||||
msgstr ""
|
||||
|
||||
#: documents/serialisers.py:463 documents/serialisers.py:815
|
||||
#: documents/serialisers.py:2501 documents/views.py:2064
|
||||
#: documents/views.py:2122 paperless_mail/serialisers.py:143
|
||||
#: documents/serialisers.py:2501 documents/views.py:1990
|
||||
#: documents/views.py:2033 paperless_mail/serialisers.py:143
|
||||
msgid "Insufficient permissions."
|
||||
msgstr ""
|
||||
|
||||
@@ -1341,7 +1341,7 @@ msgstr ""
|
||||
msgid "Duplicate document identifiers are not allowed."
|
||||
msgstr ""
|
||||
|
||||
#: documents/serialisers.py:2587 documents/views.py:3694
|
||||
#: documents/serialisers.py:2587 documents/views.py:3605
|
||||
#, python-format
|
||||
msgid "Documents not found: %(ids)s"
|
||||
msgstr ""
|
||||
@@ -1609,24 +1609,24 @@ msgstr ""
|
||||
msgid "Unable to parse URI {value}"
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:2057 documents/views.py:2119
|
||||
#: documents/views.py:1983 documents/views.py:2030
|
||||
msgid "Invalid more_like_id"
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:3706
|
||||
#: documents/views.py:3617
|
||||
#, python-format
|
||||
msgid "Insufficient permissions to share document %(id)s."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:3749
|
||||
#: documents/views.py:3660
|
||||
msgid "Bundle is already being processed."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:3806
|
||||
#: documents/views.py:3717
|
||||
msgid "The share link bundle is still being prepared. Please try again later."
|
||||
msgstr ""
|
||||
|
||||
#: documents/views.py:3816
|
||||
#: documents/views.py:3727
|
||||
msgid "The share link bundle is unavailable."
|
||||
msgstr ""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user