mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-06-21 04:44:17 +00:00
Chore(beta): Update suggested broker to Valkey + Redis agnostic documentation (#13032)
Co-authored-by: upmcplanetracker <219436948+upmcplanetracker@users.noreply.github.com>
This commit is contained in:
@@ -30,7 +30,7 @@
|
||||
# documentation.
|
||||
services:
|
||||
broker:
|
||||
image: docker.io/library/redis:8
|
||||
image: docker.io/valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
# documentation.
|
||||
services:
|
||||
broker:
|
||||
image: docker.io/library/redis:8
|
||||
image: docker.io/valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
|
||||
@@ -27,7 +27,7 @@
|
||||
# documentation.
|
||||
services:
|
||||
broker:
|
||||
image: docker.io/library/redis:8
|
||||
image: docker.io/valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
# documentation.
|
||||
services:
|
||||
broker:
|
||||
image: docker.io/library/redis:8
|
||||
image: docker.io/valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
# documentation.
|
||||
services:
|
||||
broker:
|
||||
image: docker.io/library/redis:8
|
||||
image: docker.io/valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
|
||||
@@ -30,7 +30,7 @@
|
||||
# documentation.
|
||||
services:
|
||||
broker:
|
||||
image: docker.io/library/redis:8
|
||||
image: docker.io/valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
# documentation.
|
||||
services:
|
||||
broker:
|
||||
image: docker.io/library/redis:8
|
||||
image: docker.io/valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- redisdata:/data
|
||||
|
||||
+16
-12
@@ -22,7 +22,11 @@ or applicable default will be utilized instead.
|
||||
|
||||
## Required services
|
||||
|
||||
### Redis Broker
|
||||
### Message Broker
|
||||
|
||||
Paperless-ngx uses a Redis-compatible message broker. Any broker that
|
||||
speaks the Redis protocol works here, including [Valkey](https://valkey.io/)
|
||||
(the default in the bundled Docker Compose files) and Redis itself.
|
||||
|
||||
#### [`PAPERLESS_REDIS=<url>`](#PAPERLESS_REDIS) {#PAPERLESS_REDIS}
|
||||
|
||||
@@ -30,21 +34,21 @@ or applicable default will be utilized instead.
|
||||
fetching, index optimization and for training the automatic document
|
||||
matcher.
|
||||
|
||||
- If your Redis server needs login credentials PAPERLESS_REDIS =
|
||||
- If your broker needs login credentials PAPERLESS_REDIS =
|
||||
`redis://<username>:<password>@<host>:<port>`
|
||||
- With the requirepass option PAPERLESS_REDIS =
|
||||
`redis://:<password>@<host>:<port>`
|
||||
- To include the redis database index PAPERLESS_REDIS =
|
||||
- To include the database index PAPERLESS_REDIS =
|
||||
`redis://<username>:<password>@<host>:<port>/<DBIndex>`
|
||||
|
||||
[More information on securing your Redis
|
||||
Instance](https://redis.io/docs/latest/operate/oss_and_stack/management/security).
|
||||
[More information on securing your broker
|
||||
instance](https://valkey.io/topics/security/).
|
||||
|
||||
Defaults to `redis://localhost:6379`.
|
||||
|
||||
#### [`PAPERLESS_REDIS_PREFIX=<prefix>`](#PAPERLESS_REDIS_PREFIX) {#PAPERLESS_REDIS_PREFIX}
|
||||
|
||||
: Prefix to be used in Redis for keys and channels. Useful for sharing one Redis server among multiple Paperless instances.
|
||||
: Prefix to be used in the broker for keys and channels. Useful for sharing one broker among multiple Paperless instances.
|
||||
|
||||
Defaults to no prefix.
|
||||
|
||||
@@ -238,7 +242,7 @@ dictionaries; for example, `pool.max_size=20` sets
|
||||
|
||||
#### [`PAPERLESS_DB_READ_CACHE_ENABLED=<bool>`](#PAPERLESS_DB_READ_CACHE_ENABLED) {#PAPERLESS_DB_READ_CACHE_ENABLED}
|
||||
|
||||
: Caches the database read query results into Redis. This can significantly improve application response times by caching database queries, at the cost of slightly increased memory usage.
|
||||
: Caches the database read query results into the broker. This can significantly improve application response times by caching database queries, at the cost of slightly increased memory usage.
|
||||
|
||||
Defaults to `false`.
|
||||
|
||||
@@ -258,18 +262,18 @@ dictionaries; for example, `pool.max_size=20` sets
|
||||
|
||||
A high TTL increases memory usage over time. Memory may be used until end of TTL, even if the cache is invalidated with the `invalidate_cachalot` command.
|
||||
|
||||
In case of an out-of-memory (OOM) situation, Redis may stop accepting new data — including cache entries, scheduled tasks, and documents to consume.
|
||||
If your system has limited RAM, consider configuring a dedicated Redis instance for the read cache, with a memory limit and the eviction policy set to `allkeys-lru`.
|
||||
For more details, refer to the [Redis eviction policy documentation](https://redis.io/docs/latest/develop/reference/eviction/), and see the `PAPERLESS_READ_CACHE_REDIS_URL` setting to specify a separate Redis broker.
|
||||
In case of an out-of-memory (OOM) situation, the broker may stop accepting new data — including cache entries, scheduled tasks, and documents to consume.
|
||||
If your system has limited RAM, consider configuring a dedicated broker instance for the read cache, with a memory limit and the eviction policy set to `allkeys-lru`.
|
||||
For more details, refer to the [Redis eviction policy documentation](https://redis.io/docs/latest/develop/reference/eviction/), and see the `PAPERLESS_READ_CACHE_REDIS_URL` setting to specify a separate broker.
|
||||
|
||||
#### [`PAPERLESS_READ_CACHE_REDIS_URL=<url>`](#PAPERLESS_READ_CACHE_REDIS_URL) {#PAPERLESS_READ_CACHE_REDIS_URL}
|
||||
|
||||
: Defines the Redis instance used for the read cache.
|
||||
: Defines the broker instance used for the read cache.
|
||||
|
||||
Defaults to `None`.
|
||||
|
||||
!!! Note
|
||||
If this value is not set, the same Redis instance used for scheduled tasks will be used for caching as well.
|
||||
If this value is not set, the same broker instance used for scheduled tasks will be used for caching as well.
|
||||
|
||||
## Optional Services
|
||||
|
||||
|
||||
+4
-4
@@ -94,16 +94,16 @@ first-time setup.
|
||||
```
|
||||
|
||||
7. You can now either ...
|
||||
- install Redis or
|
||||
- install a Redis-compatible broker (e.g. Valkey or Redis) or
|
||||
|
||||
- use the included `scripts/start_services.sh` to use Docker to fire
|
||||
up a Redis instance (and some other services such as Tika,
|
||||
up a broker instance (and some other services such as Tika,
|
||||
Gotenberg and a database server) or
|
||||
|
||||
- spin up a bare Redis container
|
||||
- spin up a bare broker container
|
||||
|
||||
```bash
|
||||
docker run -d -p 6379:6379 --restart unless-stopped redis:latest
|
||||
docker run -d -p 6379:6379 --restart unless-stopped docker.io/valkey/valkey:9-alpine
|
||||
```
|
||||
|
||||
8. Continue with either back-end or front-end development – or both :-).
|
||||
|
||||
+9
-5
@@ -118,10 +118,14 @@ able to run paperless, you're a bit on your own. If you can't run the
|
||||
docker image, the documentation has instructions for bare metal
|
||||
installs.
|
||||
|
||||
## _What about the Redis licensing change and using one of the open source forks_?
|
||||
## _Which message broker should I use_?
|
||||
|
||||
Currently (October 2024), forks of Redis such as Valkey or Redirect are not officially supported by our upstream
|
||||
libraries, so using one of these to replace Redis is not officially supported.
|
||||
Paperless-ngx talks to a Redis-compatible message broker, so any broker that
|
||||
implements the Redis protocol will work. The bundled Docker Compose files
|
||||
default to [Valkey](https://valkey.io/), the open-source fork created after
|
||||
Redis' licensing change, but Redis itself and other wire-compatible brokers
|
||||
(such as Microsoft's Garnet) are equally fine.
|
||||
|
||||
However, they do claim to be compatible with the Redis protocol and will likely work, but we will
|
||||
not be updating from using Redis as the broker officially just yet.
|
||||
Existing installs can switch broker implementations in place: point
|
||||
[`PAPERLESS_REDIS`](configuration.md#PAPERLESS_REDIS) at the new instance and
|
||||
reuse the same data volume.
|
||||
|
||||
+10
-9
@@ -226,7 +226,8 @@ to enable polling and disable inotify. See [here](configuration.md#polling).
|
||||
build-essential python3-setuptools python3-wheel
|
||||
```
|
||||
|
||||
2. Install `redis` >= 6.0 and configure it to start automatically.
|
||||
2. Install a Redis-compatible broker (a current release of Valkey or
|
||||
Redis) and configure it to start automatically.
|
||||
|
||||
3. Optional: Install `postgresql` and configure a database, user, and
|
||||
password for Paperless-ngx. If you do not wish to use PostgreSQL,
|
||||
@@ -268,7 +269,7 @@ to enable polling and disable inotify. See [here](configuration.md#polling).
|
||||
6. Configure Paperless-ngx. See [configuration](configuration.md) for details.
|
||||
Edit the included `paperless.conf` and adjust the settings to your
|
||||
needs. Required settings for getting Paperless-ngx running are:
|
||||
- [`PAPERLESS_REDIS`](configuration.md#PAPERLESS_REDIS) should point to your Redis server, such as
|
||||
- [`PAPERLESS_REDIS`](configuration.md#PAPERLESS_REDIS) should point to your broker, such as
|
||||
`redis://localhost:6379`.
|
||||
- [`PAPERLESS_DBENGINE`](configuration.md#PAPERLESS_DBENGINE) is optional, and should be one of `postgres`,
|
||||
`mariadb`, or `sqlite`
|
||||
@@ -297,7 +298,7 @@ to enable polling and disable inotify. See [here](configuration.md#polling).
|
||||
|
||||
!!! warning
|
||||
|
||||
Ensure your Redis instance [is secured](https://redis.io/docs/latest/operate/oss_and_stack/management/security/).
|
||||
Ensure your broker instance [is secured](https://valkey.io/topics/security/).
|
||||
|
||||
7. Create the following directories if they do not already exist:
|
||||
- `/opt/paperless/media`
|
||||
@@ -389,9 +390,9 @@ to enable polling and disable inotify. See [here](configuration.md#polling).
|
||||
`Require=paperless-webserver.socket` in the `webserver` script
|
||||
and configure `granian` to listen on port 80 (set `GRANIAN_PORT`).
|
||||
|
||||
These services rely on Redis and optionally the database server, but
|
||||
These services rely on the broker and optionally the database server, but
|
||||
don't need to be started in any particular order. The example files
|
||||
depend on Redis being started. If you use a database server, you
|
||||
depend on the broker being started. If you use a database server, you
|
||||
should add additional dependencies.
|
||||
|
||||
!!! note
|
||||
@@ -494,7 +495,7 @@ installation. Keep these points in mind:
|
||||
for other services, you might as well use it for Paperless as well.
|
||||
- The task scheduler of Paperless, which is used to execute periodic
|
||||
tasks such as email checking and maintenance, requires a
|
||||
[Redis](https://redis.io/) message broker instance. The
|
||||
Redis-compatible message broker instance (such as Valkey or Redis). The
|
||||
Docker Compose route takes care of that.
|
||||
- The layout of the folder structure for your documents and data
|
||||
remains the same, so you can plug your old Docker volumes into
|
||||
@@ -582,16 +583,16 @@ commands as well.
|
||||
|
||||
1. Stop and remove the Paperless container.
|
||||
2. If using an external database, stop that container.
|
||||
3. Update Redis configuration.
|
||||
3. Update broker configuration.
|
||||
1. If `REDIS_URL` is already set, change it to [`PAPERLESS_REDIS`](configuration.md#PAPERLESS_REDIS)
|
||||
and continue to step 4.
|
||||
|
||||
1. Otherwise, add a new Redis service in `docker-compose.yml`,
|
||||
1. Otherwise, add a new broker service in `docker-compose.yml`,
|
||||
following [the example compose
|
||||
files](https://github.com/paperless-ngx/paperless-ngx/tree/main/docker/compose)
|
||||
|
||||
1. Set the environment variable [`PAPERLESS_REDIS`](configuration.md#PAPERLESS_REDIS) so it points to
|
||||
the new Redis container.
|
||||
the new broker container.
|
||||
|
||||
4. Update user mapping.
|
||||
1. If set, change the environment variable `PUID` to `USERMAP_UID`.
|
||||
|
||||
@@ -10,9 +10,9 @@ Check for the following issues:
|
||||
`CONSUMPTION_DIR` setting. Don't adjust this setting if you're
|
||||
using docker.
|
||||
|
||||
- Ensure that redis is up and running. Paperless does its task
|
||||
- Ensure that the broker is up and running. Paperless does its task
|
||||
processing asynchronously, and for documents to arrive at the task
|
||||
processor, it needs redis to run.
|
||||
processor, it needs the broker to run.
|
||||
|
||||
- Ensure that the task processor is running. Docker does this
|
||||
automatically. Manually invoke the task processor by executing
|
||||
|
||||
+1
-1
@@ -1097,7 +1097,7 @@ Paperless-ngx consists of the following components:
|
||||
errors (i.e., wrong email credentials, errors during consuming a
|
||||
specific file, etc).
|
||||
|
||||
- A [redis](https://redis.io/) message broker: This is a really
|
||||
- A message broker (such as Valkey or Redis): This is a really
|
||||
lightweight service that is responsible for getting the tasks from
|
||||
the webserver and the consumer to the task scheduler. These run in a
|
||||
different process (maybe even on different machines!), and
|
||||
|
||||
Reference in New Issue
Block a user