Archivist

Archivist automatically classifies and files your PDF documents. It watches the inbox/ prefix of a self-hosted S3 bucket, identifies each document from the text of its pages (emitting company, type, date, business fields), then moves it — renamed according to a configurable template — into an organized prefix structure. Documents that cannot be classified with certainty go to pending/ for review in a local web dashboard.

The application is 100% offline: no cloud APIs, no data ever leaves your server.

How it works

A PDF is dropped into inbox/ (via Nextcloud or any S3 client).
Archivist extracts the text page by page, applies the classification rules, and computes the new file name.
Valid result: the file is moved to files/, under the path produced by its naming template. Status processed.
Invalid result (missing mandatory field, failing rule, unknown template): the file goes to pending/, along with the recognized fields and the errors. Status pending.
From the dashboard, you correct the fields of a pending file: it is re-filed and takes the status validated. You can also directly validate a processed file.

Duplicates (same sha256) are ignored: the inbox/ copy is deleted, the original stays in place — no data is lost, the content is identical.

Prerequisites

An S3 bucket, with three prefixes: inbox/, pending/, files/. The prefixes are created on first startup if they do not exist. Versioning is not required, but it is strongly recommended: without it, a file deleted by the application or by a bug is lost for good; with it, deleted objects stay recoverable via delete markers. The application logs a warning at startup when versioning is disabled.
An S3 user with read/write access to the bucket.
Docker.

Installation

docker run -d --name archivist \
  --restart unless-stopped \
  -p 8090:8090 \
  -v archivist-data:/app/pb_data \
  -v /path/to/config.jsonc:/app/config.jsonc:ro \
  -e ARCHIVIST_S3_ENDPOINT="s3.mydomain.fr:9000" \
  -e ARCHIVIST_S3_ACCESS_KEY="..." \
  -e ARCHIVIST_S3_SECRET_KEY="..." \
  -e ARCHIVIST_S3_BUCKET="documents" \
  -e ARCHIVIST_S3_USE_SSL="true" \
  -e ARCHIVIST_ADMIN_EMAIL="me@mydomain.fr" \
  -e ARCHIVIST_ADMIN_PASSWORD="..." \
  napnap75/archivist:latest

The dashboard is available at http://<server>:8090/, the PocketBase admin at http://<server>:8090/_/.

Configuration

Two distinct configurations:

Technical (infrastructure): container environment variables. Holds the S3 server address, credentials, bucket name, admin account.
Functional (business): the config.jsonc file, mounted into the container. Holds the fields to extract, the classification rules, and the naming templates.

The functional configuration is reloaded only when the container restarts.

Environment variables

Variable	Required	Description
`ARCHIVIST_S3_ENDPOINT`	yes	S3 server address (host:port)
`ARCHIVIST_S3_ACCESS_KEY`	yes	S3 access key
`ARCHIVIST_S3_SECRET_KEY`	yes	S3 secret
`ARCHIVIST_S3_BUCKET`	yes	Bucket name
`ARCHIVIST_S3_USE_SSL`	no	`true` (default) for HTTPS, `false` for HTTP
`ARCHIVIST_POLL_INTERVAL`	no	Cron expression for polling `inbox/`, default `* * * * *` (every minute)
`ARCHIVIST_ADMIN_EMAIL`	yes	Admin account (dashboard and PocketBase)
`ARCHIVIST_ADMIN_PASSWORD`	yes	Admin account password
`ARCHIVIST_HTTP_PORT`	no	Internal listening port, default `8090`

The pb_data volume holds the PocketBase SQLite database (documents, history, full-text search index): it must persist across restarts.

The config.jsonc file

Three sections:

fields: fields to extract, with name, type (text, number, date, boolean, enum) and whether they are mandatory. An unidentified mandatory field sends the document to pending/.
rules: an ordered tree of rules. Each rule activates if its when expression is true, then assigns fields (set), removes some (delete), and evaluates its child rules.
templates: naming templates. A template produces the full path under files/, with {...} expressions evaluated in the document context.

A commented config.jsonc example file is provided in the repository. The dashboard offers a dry-run mode to test rules on an existing document.

Minimal example

{
  "fields": [
    { "name": "emittingCompany", "type": "text", "mandatory": true },
    { "name": "documentType",    "type": "enum", "mandatory": true },
    { "name": "emissionDate",    "type": "date", "mandatory": true },
    { "name": "template",        "type": "text", "mandatory": false }
  ],
  "rules": [
    {
      "when": "pages[0] contains `Relevé de compte`",
      "set": [
        { "field": "documentType", "value": "Relevé de compte" },
        { "field": "template",     "value": "bank_statement" }
      ]
    }
  ],
  "templates": {
    "bank_statement": "{emittingCompany}/{emissionDate.Format(`2006-01-02`)}_{documentType}.pdf",
    "default":        "Divers/{documentType}.pdf"
  }
}

Dashboard

Document list with status, filters, and combined search: full text (content of all pages) and fields.
Detail view with PDF preview (streamed from S3) and extracted fields. Debug mode shows the detail of every rule evaluation.
Correction: a form to edit the fields. Saving is only possible if the selected template can be evaluated without error. A corrected file is re-filed and becomes validated; a corrected pending file leaves the review queue.
Validation: direct transition from processed to validated.
History: every document has a complete, timestamped trace (import, correction, validation) readable from its detail view.

Nextcloud integration

Mount the S3 bucket as Nextcloud external storage. Users drop files into inbox/, browse and share from files/. Archivist and Nextcloud work on the same objects; neither is the exclusive interface of the other.

Troubleshooting

A file stays in inbox/: check the container logs (docker logs archivist). Frequent causes: unreadable PDF (goes to pending/), transient S3 error (retried on the next pass).
A file goes to pending/: open its detail view in the dashboard; the classification error(s) are displayed there.
The container does not start: invalid config.jsonc. The full error message is in the logs; the application refuses to start with a partially valid configuration.
After editing config.jsonc: docker restart archivist.

6.4 KiB Raw Blame History