replacing multiprocessing pool with pipe (#491)

* replacing multiprocessing pool with pipe

* code styling fix

* dropping obsolete chunk_size config parameter
This commit is contained in:
Szasza Palmer
2024-03-24 12:41:46 +11:00
committed by GitHub
parent 1e565d9eb2
commit a915385246
2 changed files with 47 additions and 40 deletions

View File

@@ -126,8 +126,6 @@ The full set of configuration options are:
- `log_file` - str: Write log messages to a file at this path
- `n_procs` - int: Number of process to run in parallel when
parsing in CLI mode (Default: `1`)
- `chunk_size` - int: Number of files to give to each process
when running in parallel.
:::{note}
Setting this to a number larger than one can improve