archivist: firstMatch takes a pages expression, not page numbers

This commit is contained in:
2026-11-01 11:51:42 +01:00
parent 3b743345f1
commit bfc02c626f
+2 -2
View File
@@ -47,7 +47,7 @@ One row per processed PDF file:
- `sha256`: the sha256 is a primary key as the system will reject as duplicate any file with the same hash
- `original_filename`: original file name in `inbox/`
- `file_key`: current S3 key of the file
- `pages`: page-by-page extracted text as a JSON array (index = page number, `pages[0]` = page 1); the FTS5 full-text index is built from the concatenation of all pages
- `pages`: page-by-page extracted text as a JSON array (0-based, `pages[0]` = page 1); the FTS5 full-text index is built from the concatenation of all pages
- `size`: file size
- `status`: `pending` | `processed` | `validated`
+ all the fields declared by the user in the configuration file
@@ -75,7 +75,7 @@ A `config.jsonc` file kept alongside the binary, reloaded on restart. Three sect
- `delete`: list of field names to unset (they become undefined again for later rules and template rendering)
- `rules`: child rules, evaluated in order only if the parent `when` matched; nesting is recursive
The application registers two built-in helpers in the expression environment. `firstMatch(regex, pages?)` returns the first match of the regex in the document text (first capture group when present); the optional second argument selects the pages to search — a number searches that page (1-based), a two-element list `[a, b]` searches pages a to b inclusive, a longer list of numbers searches exactly those pages, and when omitted the whole document is searched (e.g. `firstMatch(`[0-9]{2}/[0-9]{2}/[0-9]{4}`, 1)`). `period(value, unit, shift, layout?)` renders a period instead of a raw date: `value` is a native date (e.g. `emissionDate`), `unit` is `month` | `quarter` | `half` | `year`, `shift` is `-1` (previous period), `0` (period containing the date) or `+1` (next period) — e.g. a rent receipt issued mid-month uses the containing month, a service-charge call issued at the end of a quarter uses the next quarter, a bank statement uses the previous month. It returns a French string: `Janvier 2026`, `T2 2026`, `S1 2026` or `2026`. The optional `layout` argument is a standard Go reference-date layout applied to the period's first day, with month names rendered in French by the application (default layouts: `January 2006` for `month`, `2006` for `year`); it is ignored for `quarter` and `half`, whose formats (`T<quarter> <year>`, `S<half> <year>`) are fixed. A missing or invalid date is a rule error and routes the document to `pending/`. Parsing strings into native dates uses expr's standard `date(value, layout)` builtin (Go date layout, e.g. `date(`15/01/2026`, `02/01/2006`)`) — the format is carried by the parsing rule, so it can differ per document type.
The application registers two built-in helpers in the expression environment. `firstMatch(regex, texts?)` returns the first match of the regex in the document text (first capture group when present); the optional second argument is an expression evaluated to the text(s) to search — a single page (e.g. `pages[0]`, the first page), a slice (e.g. `pages[0:3]`, pages 1 to 3, expr's standard slice syntax, end exclusive), or any list of strings — and when omitted the whole document is searched (e.g. `firstMatch(`[0-9]{2}/[0-9]{2}/[0-9]{4}`, pages[0])`). `period(value, unit, shift, layout?)` renders a period instead of a raw date: `value` is a native date (e.g. `emissionDate`), `unit` is `month` | `quarter` | `half` | `year`, `shift` is `-1` (previous period), `0` (period containing the date) or `+1` (next period) — e.g. a rent receipt issued mid-month uses the containing month, a service-charge call issued at the end of a quarter uses the next quarter, a bank statement uses the previous month. It returns a French string: `Janvier 2026`, `T2 2026`, `S1 2026` or `2026`. The optional `layout` argument is a standard Go reference-date layout applied to the period's first day, with month names rendered in French by the application (default layouts: `January 2006` for `month`, `2006` for `year`); it is ignored for `quarter` and `half`, whose formats (`T<quarter> <year>`, `S<half> <year>`) are fixed. A missing or invalid date is a rule error and routes the document to `pending/`. Parsing strings into native dates uses expr's standard `date(value, layout)` builtin (Go date layout, e.g. `date(`15/01/2026`, `02/01/2006`)`) — the format is carried by the parsing rule, so it can differ per document type.
3. `templates`: named naming templates (e.g. `bank_statement: {emittingCompany}/{emissionDate.Format(`2006-01-02`)}_{documentType}_{bankAccount}.pdf`). A template is a literal string containing `{...}` placeholders: the inside of each placeholder is an `expr-lang/expr` expression evaluated in the document context (fields, helpers, builtins — e.g. `{emissionDate.Format(`2006-01-02`)}` or `{upper(emittingCompany)}`) and replaced by its result; the static text between placeholders stays literal. Unset optional fields evaluate to an empty string; a missing mandatory field or an evaluation error renders the result invalid and routes the file to `pending/`. All placeholder expressions are compiled at config load; "/" must not appear in the result (it is the S3 key delimiter). The template is chosen by the rules themselves: `template` is a field declared in the `fields` section like any other and set with `set` (e.g. `set template = `bank_statement`` when `emittingCompany in [`BRED`, `Fortunéo`] and documentType == `Relevé de compte``). If no rule sets `template`, the special `default` template is used. A `template` value with no matching entry renders the result invalid and routes the file to `pending/`.