feat(config): add frDate helper for French natural-language dates

Go date layouts only know English month names, so date(value, layout)
cannot parse "le 19 janvier 2026". frDate(text) extracts and parses such
dates (day optional, abbreviated months, accents optional) into a native
date; no match is a rule error routing the document to pending/.

Documented in arch.md §5, covered by unit tests.
This commit is contained in:
2026-11-03 22:56:47 +01:00
parent 09d0c9b99b
commit 5d09e7a780
3 changed files with 105 additions and 0 deletions
+2
View File
@@ -175,6 +175,8 @@ A `config.jsonc` file kept alongside the binary, reloaded on restart. Three sect
The application registers two built-in helpers in the expression environment. `firstMatch(regex, texts?)` returns the first match of the regex in the document text (first capture group when present), or an empty string when no match is found (parsing an empty string with `date()` is then a rule error and routes the document to `pending/`); the optional second argument is an expression evaluated to the text(s) to search — a single page (e.g. `pages[0]`, the first page), a slice (e.g. `pages[0:3]`, pages 1 to 3, expr's standard slice syntax, end exclusive), or any list of strings — and when omitted the whole document is searched (e.g. `firstMatch(`[0-9]{2}/[0-9]{2}/[0-9]{4}`, pages[0])`). `period(value, unit, shift, layout?)` renders a period instead of a raw date: `value` is a native date (e.g. `emissionDate`), `unit` is `month` | `quarter` | `half` | `year`, `shift` is `-1` (previous period), `0` (period containing the date) or `+1` (next period) — e.g. a rent receipt issued mid-month uses the containing month, a service-charge call issued at the end of a quarter uses the next quarter, a bank statement uses the previous month. It returns a French string: `Janvier 2026`, `T2 2026`, `S1 2026` or `2026`. The optional `layout` argument is a standard Go reference-date layout applied to the period's first day, with month names rendered in French by the application (default layouts: `January 2006` for `month`, `2006` for `year`); it is ignored for `quarter` and `half`, whose formats (`T<quarter> <year>`, `S<half> <year>`) are fixed. A missing or invalid date is a rule error and routes the document to `pending/`. Parsing strings into native dates uses expr's standard `date(value, layout)` builtin (Go date layout, e.g. `date(`15/01/2026`, `02/01/2006`)`) — the format is carried by the parsing rule, so it can differ per document type. The application registers two built-in helpers in the expression environment. `firstMatch(regex, texts?)` returns the first match of the regex in the document text (first capture group when present), or an empty string when no match is found (parsing an empty string with `date()` is then a rule error and routes the document to `pending/`); the optional second argument is an expression evaluated to the text(s) to search — a single page (e.g. `pages[0]`, the first page), a slice (e.g. `pages[0:3]`, pages 1 to 3, expr's standard slice syntax, end exclusive), or any list of strings — and when omitted the whole document is searched (e.g. `firstMatch(`[0-9]{2}/[0-9]{2}/[0-9]{4}`, pages[0])`). `period(value, unit, shift, layout?)` renders a period instead of a raw date: `value` is a native date (e.g. `emissionDate`), `unit` is `month` | `quarter` | `half` | `year`, `shift` is `-1` (previous period), `0` (period containing the date) or `+1` (next period) — e.g. a rent receipt issued mid-month uses the containing month, a service-charge call issued at the end of a quarter uses the next quarter, a bank statement uses the previous month. It returns a French string: `Janvier 2026`, `T2 2026`, `S1 2026` or `2026`. The optional `layout` argument is a standard Go reference-date layout applied to the period's first day, with month names rendered in French by the application (default layouts: `January 2006` for `month`, `2006` for `year`); it is ignored for `quarter` and `half`, whose formats (`T<quarter> <year>`, `S<half> <year>`) are fixed. A missing or invalid date is a rule error and routes the document to `pending/`. Parsing strings into native dates uses expr's standard `date(value, layout)` builtin (Go date layout, e.g. `date(`15/01/2026`, `02/01/2006`)`) — the format is carried by the parsing rule, so it can differ per document type.
`frDate(text)` extracts and parses a French natural-language date — the layout Go cannot express, since `date(value, layout)` only knows English month names. It matches `le 19 janvier 2026` (day and leading `le` optional: `Janvier 2026` gives the first of the month; `1er` and abbreviated month names like `févr.` are accepted; accents are optional) and returns a native date — e.g. `frDate(firstMatch(`(?:le )?\d{1,2} \S+ \d{4}`, pages[0]))`, or `frDate(pages[0])` when the first French-looking date of the page is the right one. A text containing no French date, or an impossible date (31 février), is a rule error and routes the document to pending/.
3. `templates`: named naming templates (e.g. `bank_statement: {emittingCompany}/{emissionDate.Format(`2006-01-02`)}_{documentType}_{bankAccount}.pdf`). A template is a literal string containing `{...}` placeholders: the inside of each placeholder is an `expr-lang/expr` expression evaluated in the document context (fields, helpers, builtins — e.g. `{emissionDate.Format(`2006-01-02`)}` or `{upper(emittingCompany)}`) and replaced by its result; the static text between placeholders stays literal. Unset optional fields evaluate to an empty string; a missing mandatory field or an evaluation error renders the result invalid and routes the file to `pending/`. All placeholder expressions are compiled at config load; "/" must not appear in the result (it is the S3 key delimiter). The template is chosen by the rules themselves: `template` is a field declared in the `fields` section like any other and set with `set` (e.g. `set template = `bank_statement`` when `emittingCompany in [`BRED`, `Fortunéo`] and documentType == `Relevé de compte``). If no rule sets `template`, the special `default` template is used. A `template` value with no matching entry renders the result invalid and routes the file to `pending/`. 3. `templates`: named naming templates (e.g. `bank_statement: {emittingCompany}/{emissionDate.Format(`2006-01-02`)}_{documentType}_{bankAccount}.pdf`). A template is a literal string containing `{...}` placeholders: the inside of each placeholder is an `expr-lang/expr` expression evaluated in the document context (fields, helpers, builtins — e.g. `{emissionDate.Format(`2006-01-02`)}` or `{upper(emittingCompany)}`) and replaced by its result; the static text between placeholders stays literal. Unset optional fields evaluate to an empty string; a missing mandatory field or an evaluation error renders the result invalid and routes the file to `pending/`. All placeholder expressions are compiled at config load; "/" must not appear in the result (it is the S3 key delimiter). The template is chosen by the rules themselves: `template` is a field declared in the `fields` section like any other and set with `set` (e.g. `set template = `bank_statement`` when `emittingCompany in [`BRED`, `Fortunéo`] and documentType == `Relevé de compte``). If no rule sets `template`, the special `default` template is used. A `template` value with no matching entry renders the result invalid and routes the file to `pending/`.
Validation on load: JSON schema, regex compilation, `expr` expression compilation (`when`, `eval` and template placeholder expressions) against a typed context (`Document{Pages, Fields}`) with the `firstMatch` and `period` helpers, regex precompilation and caching. Full rejection if invalid, no partial processing. Validation on load: JSON schema, regex compilation, `expr` expression compilation (`when`, `eval` and template placeholder expressions) against a typed context (`Document{Pages, Fields}`) with the `firstMatch` and `period` helpers, regex precompilation and caching. Full rejection if invalid, no partial processing.
+73
View File
@@ -190,6 +190,7 @@ func exprOptions() []expr.Option {
return []expr.Option{ return []expr.Option{
expr.Function("firstMatch", FirstMatch), expr.Function("firstMatch", FirstMatch),
expr.Function("period", Period), expr.Function("period", Period),
expr.Function("frDate", FrenchDate),
} }
} }
@@ -396,6 +397,78 @@ func toString(v any) string {
} }
} }
// frDateRe matches a French natural-language date, with or without a day:
// "le 19 janvier 2026", "Janvier 2026", "1er févr. 2026". Abbreviated month
// names are accepted, accents are optional.
var frDateRe = regexp.MustCompile(`(?i)(?:le\s+)?(?:(1er|\d{1,2})\s+)?(janv\.?|janvier|f[eé]vr\.?|f[eé]vrier|mars|avr\.?|avril|mai|juin|juil\.?|juillet|ao[uû]t|sept\.?|septembre|oct\.?|octobre|nov\.?|novembre|d[eé]c\.?|d[eé]cembre)\s+(\d{4})`)
// frMonths maps the folded month names to their numbers. Keys are lowercase
// and unaccented; foldFrench normalizes what the regex captured.
var frMonths = map[string]time.Month{
"janv": time.January, "janvier": time.January,
"fevr": time.February, "fevrier": time.February,
"mars": time.March,
"avr": time.April, "avril": time.April,
"mai": time.May,
"juin": time.June,
"juil": time.July, "juillet": time.July,
"aout": time.August,
"sept": time.September, "septembre": time.September,
"oct": time.October, "octobre": time.October,
"nov": time.November, "novembre": time.November,
"dec": time.December, "decembre": time.December,
}
// foldFrench lowercases and strips the French diacritics used in month
// names plus the abbreviation dot, so the lookup tolerates "Février",
// "FEVRIER" and "févr." alike.
func foldFrench(s string) string {
r := strings.ToLower(s)
for _, a := range [][2]string{{"é", "e"}, {"è", "e"}, {"ê", "e"}, {"û", "u"}, {"â", "a"}} {
r = strings.ReplaceAll(r, a[0], a[1])
}
return strings.TrimSuffix(r, ".")
}
// FrenchDate is the frDate(text) helper (arch.md §5). It extracts and
// parses a French natural-language date — "le 19 janvier 2026", the day
// being optional ("Janvier 2026" gives the first of the month) and
// abbreviated month names accepted ("1er févr. 2026") — into a native
// date. Go layouts only know English month names, so date(value, layout)
// cannot parse these. A text containing no French date is a rule error
// and routes the document to pending/.
func FrenchDate(args ...any) (any, error) {
if len(args) == 0 {
return nil, fmt.Errorf("frDate: text required")
}
text, _ := args[0].(string)
m := frDateRe.FindStringSubmatch(text)
if m == nil {
return nil, fmt.Errorf("frDate: no French date found")
}
day := 1
if m[1] != "" && !strings.EqualFold(m[1], "1er") {
d, err := strconv.Atoi(m[1])
if err != nil {
return nil, fmt.Errorf("frDate: %w", err)
}
day = d
}
month, ok := frMonths[foldFrench(m[2])]
if !ok {
return nil, fmt.Errorf("frDate: unknown month %q", m[2])
}
year, err := strconv.Atoi(m[3])
if err != nil {
return nil, fmt.Errorf("frDate: %w", err)
}
d := time.Date(year, month, day, 0, 0, 0, 0, time.UTC)
if d.Month() != month || d.Day() != day {
return nil, fmt.Errorf("frDate: %d %s %d is not a valid date", day, m[2], year)
}
return d, nil
}
var frenchMonths = []string{"Janvier", "Février", "Mars", "Avril", "Mai", "Juin", var frenchMonths = []string{"Janvier", "Février", "Mars", "Avril", "Mai", "Juin",
"Juillet", "Août", "Septembre", "Octobre", "Novembre", "Décembre"} "Juillet", "Août", "Septembre", "Octobre", "Novembre", "Décembre"}
+30
View File
@@ -24,3 +24,33 @@ func TestDefaultConfigExpressionHelpers(t *testing.T) {
t.Fatal("default config rendered an empty file name") t.Fatal("default config rendered an empty file name")
} }
} }
func TestFrenchDate(t *testing.T) {
cases := []struct {
in string
want string
}{
{"Facture en date du 19 janvier 2026", "19/01/2026"},
{"le 1er févr. 2026", "01/02/2026"},
{"Janvier 2026", "01/01/2026"}, // no day: first of the month
{"Avis arrêté au 3 septembre 2026", "03/09/2026"},
{"Situation au 31 decembre 2026", "31/12/2026"}, // unaccented
}
for _, c := range cases {
v, err := FrenchDate(c.in)
if err != nil {
t.Fatalf("frDate(%q): %v", c.in, err)
}
got, ok := v.(time.Time)
if !ok || got.Format("02/01/2006") != c.want {
t.Fatalf("frDate(%q) = %v, want %s", c.in, v, c.want)
}
}
if _, err := FrenchDate("no date in this text 123"); err == nil {
t.Fatal("frDate: expected an error when no French date is found")
}
if _, err := FrenchDate("le 31 février 2026"); err == nil {
t.Fatal("frDate: expected an error on an invalid date")
}
}