mirror of
https://github.com/napnap75/multiarch-docker-images.git
synced 2026-11-12 00:52:11 +01:00
First run of the web UI
This commit is contained in:
+16
-17
@@ -31,7 +31,7 @@ Everything runs offline on the server infrastructure: no cloud APIs, no outbound
|
||||
|
||||
## 3. S3 Layout
|
||||
|
||||
A single bucket. Versioning is not required, but it is strongly recommended: without it, any deletion performed by the application (or by a bug) is final; with it, delete markers keep deleted objects rec
|
||||
A single bucket. Versioning is not required, but it is strongly recommended: without it, any deletion performed by the application (or by a bug) is final; with it, delete markers keep deleted objects rec
|
||||
overable.
|
||||
|
||||
```
|
||||
@@ -64,7 +64,7 @@ The payload is self-contained: each event carries its own type, its own timestam
|
||||
Common payload keys:
|
||||
|
||||
- `event_type`: `imported` | `corrected` | `validated`
|
||||
- `timestamp`: RFC 3339 date of the
|
||||
- `timestamp`: RFC 3339 date of the
|
||||
event
|
||||
- `before` / `after`: the event's diff, never full snapshots — a key appears on a side only when its value changes in the event: same value on both sides, it is omitted; absent from `before` and present in `after`, the field was added (every `after` key at import, since nothing existed before); present in `before` and absent from `after`, it was deleted. `file_key` (the S3 key, unified with the `documents` record field) follows the same rule and appears only when the object actually moves; at import the status does not exist before, so `before` carries no `status`. A missing key means the value is unknown, there is no `null`
|
||||
|
||||
@@ -107,7 +107,7 @@ On classification failure, the document is routed to `pending/` and the payload
|
||||
"timestamp": "2026-01-15T14:23:07Z",
|
||||
"errors": ["mandatory field missing: emittingCompany"],
|
||||
"before": {
|
||||
"file_key": "inbox/scan
|
||||
"file_key": "inbox/scan
|
||||
_2026-01-15.pdf"
|
||||
},
|
||||
"after": {
|
||||
@@ -166,7 +166,7 @@ The diff is derived by comparing the two snapshots: a field present in `before`
|
||||
A `config.jsonc` file kept alongside the binary, reloaded on restart. Three sections:
|
||||
|
||||
1. `fields`: declaration of fields to extract from the files:
|
||||
- `na
|
||||
- `na
|
||||
me`: name of the field
|
||||
- `type`: `text` | `number` | `date` | `boolean` | `enum`
|
||||
- `mandatory`: true | false
|
||||
@@ -186,7 +186,7 @@ me`: name of the field
|
||||
|
||||
The expression environment provides expr's standard `date(value, layout)` builtin plus three built-in helpers registered by the application:
|
||||
|
||||
- `date(value, layout)`: expr's standard builtin. Parses a string into a native date using a Go date layout (e.g. `date(`15/01/2026`, `02/01/2006`)`). The format is carried by the parsing rule, so it can differ per document type. It only knows English month names: see
|
||||
- `date(value, layout)`: expr's standard builtin. Parses a string into a native date using a Go date layout (e.g. `date(`15/01/2026`, `02/01/2006`)`). The format is carried by the parsing rule, so it can differ per document type. It only knows English month names: see
|
||||
`frDate` below for French text.
|
||||
- `firstMatch(regex, texts?)`: returns the first match of the regex in the document text (first capture group when present), or an empty string when no match is found — e.g. `firstMatch(`[0-9]{2}/[0-9]{2}/[0-9]{4}`, pages[0])`. The optional second argument is an expression evaluated to the text(s) to search:
|
||||
- a single page: `pages[0]` (the first page)
|
||||
@@ -202,7 +202,7 @@ me`: name of the field
|
||||
- `value`: a native date (e.g. `emissionDate`); a missing or invalid date is a rule error and routes the document to `pending/`.
|
||||
- `unit`: `month` | `quarter` | `half` | `year`.
|
||||
- `shift`: `-1` (previous period), `0` (period containing the date) or `+1` (next period) — e.g. a rent receipt issued mid-month uses the containing month, a service-charge call issued at the end of a quarter uses the next quarter, a bank statement uses the previous month.
|
||||
- Returns a French string: `Janvier 2026`, `T2 2026`, `S1 2026` or `202
|
||||
- Returns a French string: `Janvier 2026`, `T2 2026`, `S1 2026` or `202
|
||||
6`.
|
||||
- The optional `layout` argument is a standard Go reference-date layout applied to the period's first day, with month names rendered in French by the application (default layouts: `January 2006` for `month`, `2006` for `year`); it is ignored for `quarter` and `half`, whose formats (`T<quarter> <year>`, `S<half> <year>`) are fixed.
|
||||
|
||||
@@ -220,7 +220,7 @@ Validation on load: JSON schema, regex compilation, `expr` expression compilatio
|
||||
|
||||
## 6. Processing Pipeline
|
||||
|
||||
Executed by the PocketBase cron job (
|
||||
Executed by the PocketBase cron job (
|
||||
configurable interval, e.g. every minute):
|
||||
|
||||
1. **Poll**: list objects in `inbox/`; an object is a candidate only if no `documents` record references its key yet, and its size and ETag are stable across two consecutive polls (write-completion wait). The same pass also picks up interrupted filings: any `documents` record whose `file_key` is still under `inbox/` (a filing that was interrupted between record creation and the physical move) is completed as described below.
|
||||
@@ -228,7 +228,7 @@ configurable interval, e.g. every minute):
|
||||
3. **Extraction**: page-by-page text (one entry per page, for rule evaluation and the FTS index) via `ledongthuc/pdf`, including headers and footers
|
||||
4. **Rule evaluation**: rules evaluated in order against the document pages and the fields set so far; full evaluation log written to the PocketBase logger at Debug level (section 9; the same trace feeds the built-in dry-run mode for debugging from the dashboard).
|
||||
5. **Validation**: a result is valid if and only if: all rules have been processed with no error, all mandatory fields are identified, the selected template exists in the configuration, and the naming template evaluated with no error. Otherwise: route directly to `pending/`.
|
||||
6. **Record**: create the `documents` record (sha256, current inbox key, pages, fields, status) and add an `imported` event including the original key (`before.file_key`). The record exists before any physical move, so an interrupted filing leaves a state the poller recovers automatically on its ne
|
||||
6. **Record**: create the `documents` record (sha256, current inbox key, pages, fields, status) and add an `imported` event including the original key (`before.file_key`). The record exists before any physical move, so an interrupted filing leaves a state the poller recovers automatically on its ne
|
||||
xt pass.
|
||||
7. **Filing**:
|
||||
- valid → resolve the target name first: if a file with the same name already exists in the target prefix, append a number to the filename; then copy to the new key (`files/` + rendered template — the template renders the full object path, there is no separate prefix), verify the target object exists, and delete the `inbox/` key only after the copy is confirmed; finally update the `documents` record with the final, conflict-resolved filename (`file_key`, status `processed`); the whole import is recorded as the single `imported` event of section 4.
|
||||
@@ -243,7 +243,7 @@ The review dashboard is a static frontend — plain HTML/CSS/JS files, no build
|
||||
### 7.1 Document list (US-32 / US-33)
|
||||
|
||||
- Table: one row per document, paginated (50 by default), default columns: all mandatory fields plus `status` and `file_key`; a column chooser exposes every declared field plus `created`/`updated`, and the choice persists in the browser (`localStorage`).
|
||||
- Sorting: click on
|
||||
- Sorting: click on
|
||||
a column header, ascending/descending, executed server-side through the standard PocketBase `sort` parameter.
|
||||
- Per-column filters, typed from the field declaration: contains for `text`, dropdown for `enum` and `status` (vocabulary from the config metadata route), date range for `date`, tri-state for `boolean`; translated client-side into a PocketBase filter expression.
|
||||
- Quick tabs with the per-status counts pre-filter the review queue (`pending` first).
|
||||
@@ -255,7 +255,7 @@ a column header, ascending/descending, executed server-side through the standard
|
||||
- Layout: two columns — the PDF preview embedded via the proxy route of section 7.3, and the correction form; below them, the document's event history, read-only, sorted by timestamp, payloads rendered as before/after diffs.
|
||||
- Form: one typed control per declared field, pre-filled with the current values; mandatory fields marked; the classification errors of the `imported` event, if any, displayed at the top.
|
||||
- Pipeline proposals (US-35): under each field whose value the pipeline would extract differently today, a discreet banner offers the proposed value with one-click adoption. The proposals come from the classification dry-run (section 7.3), re-run against the stored pages.
|
||||
- Live name (US-35): as the form changes, the rendered target name under `files/` is recomputed by the render dry-run and displayed with its errors; the draft save remains available, but the full correction refuses to run while errors remai
|
||||
- Live name (US-35): as the form changes, the rendered target name under `files/` is recomputed by the render dry-run and displayed with its errors; the draft save remains available, but the full correction refuses to run while errors remai
|
||||
n.
|
||||
- Exits, three and distinct: draft save (standard PATCH, US-22: values persist, no re-filing, no status change); full correction (US-36: the `/correct` route, re-filing and validation in one confirmed step); simple validation of a `processed` document (standard PATCH, US-23). A `pending` document only leaves the review queue through the full correction.
|
||||
|
||||
@@ -266,7 +266,7 @@ Beyond the standard PocketBase API, authenticated only:
|
||||
- `GET /api/archivist/config` — the configuration metadata the UI needs: declared fields (name, type, mandatory), the enum vocabularies computed from the constants the rules assign, the template names. Read-only; the dashboard is entirely driven by it (columns, filters, controls).
|
||||
- `GET /api/archivist/documents/{id}/pdf` — proxy route streaming the document from S3; the browser never reaches the S3 cluster.
|
||||
- `POST /api/archivist/documents/{id}/dry-run` — two exclusive modes, one response shape (`fields`, `template`, `rendered`, `errors`), persisting nothing. Without a body: a classification re-run — the rule tree evaluated against the stored pages, returning the proposed fields (the source of the pipeline proposals). With field values: a template render — the values converted to their native types (per-field error on type mismatch), merged with the record values, rendered against the stored pages (placeholders may call `firstMatch`/`frDate` on the document text), returning the target name and its errors. The modes are never hybrid: user values never enter the rule-evaluation context, so the proposal always stays what the pipeline would do alone.
|
||||
- `POST /api/archivist/documents/{id}/correct` — the full correction of US-36: transactional orchestration (conflict-suffix resolution, S3 copy, verification, old key deletion, record update with `file_key` and status `validated`, `corrected` event carrying the full diff, S3 rollback on mid-failure with the save refused). It reuses the engine internals (`FieldsFromRecord`, `Va
|
||||
- `POST /api/archivist/documents/{id}/correct` — the full correction of US-36: transactional orchestration (conflict-suffix resolution, S3 copy, verification, old key deletion, record update with `file_key` and status `validated`, `corrected` event carrying the full diff, S3 rollback on mid-failure with the save refused). It reuses the engine internals (`FieldsFromRecord`, `Va
|
||||
lidateValue`, `Render`, `ResolveConflict`, `DiffSnapshots`, `AppendEvent`), no parallel business logic. A route rather than a standard PATCH because the intent (re-file + validate) is not expressible in record data, the standard PATCH must stay side-effect-free for every client and the bulk edit (US-22 / US-33), and the operation is a multi-step transaction rather than a record save — the same line drawn by `POST /api/archivist/poll`.
|
||||
|
||||
### 7.4 Standard collections UI and hooks
|
||||
@@ -275,15 +275,14 @@ lidateValue`, `Render`, `ResolveConflict`, `DiffSnapshots`, `AppendEvent`), no p
|
||||
- Validation: performed from the same interface by changing the status to `validated`; the hook appends the `validated` event, and any other status change is refused. A `pending` file can only leave the review queue through a full correction from the dashboard (US-36).
|
||||
- Deletion: deleting a `documents` record from the same interface first deletes the S3 object at `file_key` (US-24); on S3 failure the deletion is refused and the record stays. The cascade delete of the `document_events` relation removes the history with the record; with versioning the object stays recoverable via its delete marker.
|
||||
|
||||
The static pages themselves carry no data and are public by nature; every fetch is authenticated against the PocketBase API and an unauthenticated visitor is redirected to the admin UI (`/_/`). Authentication: single ac
|
||||
count via PocketBase's standard auth. The configuration is reloaded only on restart; there is no runtime reload.
|
||||
The static pages themselves carry no data and are public by nature; every fetch is authenticated against the PocketBase API. An unauthenticated visitor is prompted to sign in with the superuser account; the dashboard exchanges those credentials through PocketBase's standard auth endpoint, stores the returned token locally, and retries the blocked request. The API continues to enforce superuser access. The configuration is reloaded only on restart; there is no runtime reload.
|
||||
### 7.5 Design (Beszel reference)
|
||||
|
||||
The dashboard's visual design takes Beszel (https://github.com/henrygd/beszel) as its reference. Beszel is built on shadcn/ui + Tailwind with semantic HSL CSS variables and a light/dark pair; archivist reproduces its palette, token names and layout feel in a single hand-written stylesheet (`pb_public/styles.css`) — no Tailwind, no CSS framework, consistent with the no-build-toolchain constraint of section 7.
|
||||
|
||||
- Semantic tokens, same names and values as Beszel's own stylesheet. Light theme: `--background: hsl(30 8% 98%)`, `--foreground: hsl(30 0% 10%)`, `--card: hsl(30 0% 100%)`, `--card-foreground: hsl(240 6% 12%)`, `--primary: hsl(240 5.88% 10%)`, `--primary-foreground: hsl(30 0% 100%)`, `--secondary: hsl(240 4.76% 95.88%)`, `--muted: hsl(26 6% 94%)`, `--muted-foreground: hsl(24 2.79% 35.1%)`, `--accent: hsl(20 23.08% 94%)`, `--destructive: hsl(0 66% 53%)`, `--border: hsl(30 8.11% 85.49%)`, `--input: hsl(30 4.29% 72.55%)`, `--ring: hsl(30 3.97% 49.41%)`, `--table-header: hsl(225 6% 97%)`, `--radius: 0.8rem`.
|
||||
- Dark theme: `--background: hsl(220 5.5% 9%)`, `--foreground: hsl(220 2% 97%)`, `--card: hsl(220 5.5% 10.5%)`, `--card-foreground: hsl(220 2% 97%)`, `--primary: hsl(220 2% 96%)`, `--primary-foreground: hsl(220 4% 10%)`, `--secondary: hsl(220 4% 16%)`, `--muted: hsl(220 6% 16%)`, `--muted-foreground: hsl(220 4% 67%)`, `--accent: hsl(220 5% 15.5%)`, `--destructive: hsl(0 62% 46%)`, `--border: hsl(220 3% 17%)`, `--input: hsl(220 4% 22%)`, `--ring: hsl(220 4% 80%)`, `--table-header: hsl(220 6% 13%)`, plus `color-scheme: dark` so native controls follow.
|
||||
- Theme switching, same scheme as Beszel: a `dark` class on `<html>`, defaulted from the OS `prefers-color-scheme`, with a manual toggle persisted in `localStorage`; the class is set by an inline script in the page head to avoid a flash of the wrong theme on first
|
||||
- Theme switching, same scheme as Beszel: a `dark` class on `<html>`, defaulted from the OS `prefers-color-scheme`, with a manual toggle persisted in `localStorage`; the class is set by an inline script in the page head to avoid a flash of the wrong theme on first
|
||||
paint.
|
||||
- Component language: cards on `--card` over `--background`, hairline `--border`, `--radius` rounding, sticky table headers on `--table-header` (the document list), secondary actions on `--secondary`/`--muted`, `--destructive` reserved for destructive actions and errors, focus rings on `--ring`.
|
||||
- Status colors, the only domain additions to Beszel's tokens: `pending` maps to a warm badge on `--accent`, `validated` to a green from Beszel's chart scale (`hsl(140 50% 48%)` light / `hsl(140 49% 60%)` dark), `processed` to a neutral `--muted` badge.
|
||||
@@ -304,7 +303,7 @@ Two channels, split by lifecycle:
|
||||
|
||||
Levels used:
|
||||
|
||||
- `Erro
|
||||
- `Erro
|
||||
r`: a failed operation (S3 error, event append failure).
|
||||
- `Warn`: a degraded but non-fatal state (versioning disabled, schema column removed, stuck record whose inbox object disappeared).
|
||||
- `Info`: pipeline outcomes (document imported, duplicate removed, interrupted filing recovered).
|
||||
@@ -330,7 +329,7 @@ The scenarios form one narrative; each subtest names the US it proves:
|
||||
2. Import — a valid simulated statement is classified and filed to `files/` with its `imported` event (US-10 to US-17).
|
||||
3. Duplicate — the same content re-uploaded is deleted from `inbox/`, no record, no event (US-12).
|
||||
4. Pending — a missing mandatory date routes the document to `pending/` with `errors` in the imported event and the recognized fields kept (US-13 to US-15).
|
||||
5. Correction — a field PATCH through the API persists the values with no side effect (no re-fil
|
||||
5. Correction — a field PATCH through the API persists the values with no side effect (no re-fil
|
||||
ing, no status change), `corrected` event carrying the field diff (US-22).
|
||||
6. Validation — a status-only PATCH appends the `validated` event, `file_key` untouched (US-23).
|
||||
7. Append-only — PATCH and DELETE on `document_events` are refused (US-04).
|
||||
@@ -352,7 +351,7 @@ The dashboard application routes are covered by the gate: the config metadata ro
|
||||
`Dockerfile`, two-stage build:
|
||||
|
||||
- Builder: `golang:1.27-alpine` (matches the `go 1.27` directive of PocketBase v0.41.1), CGO disabled, static binary, `TARGETARCH` honored for multi-platform builds under buildx. `go mod tidy` runs there because `go.sum` is not committed; once it is committed, the line becomes `go mod download` for fully reproducible builds.
|
||||
- Runtime: `scratch` with the `/archivist` binary, the CA bundle (the S3 endpoint is HTTPS by default), the timezone database (the cron and date rendering honor `TZ`), the static review-dashboard frontend copied to `/app/pb_public/` (read-only at runtime), and `/app/pb_data` + `/tmp`, the only writable directories, owned by
|
||||
- Runtime: `scratch` with the `/archivist` binary, the CA bundle (the S3 endpoint is HTTPS by default), the timezone database (the cron and date rendering honor `TZ`), the static review-dashboard frontend copied to `/app/pb_public/` (read-only at runtime), and `/app/pb_data` + `/tmp`, the only writable directories, owned by
|
||||
the unprivileged runtime user 65532. `config.jsonc` is never baked in: it is mounted at runtime, `ARCHIVIST_CONFIG` defaulting to `config.jsonc` relative to `/app`.
|
||||
|
||||
The container runs as 65532:65532 (non-root), exposes 8090, declares `/app/pb_data` as a volume, and has no shell: `docker exec` is not available. `.dockerignore` keeps the build context to the Go sources and the dashboard frontend; the E2E harness is not part of the image.
|
||||
|
||||
+17
-30
@@ -82,7 +82,7 @@ func TestMain(m *testing.M) {
|
||||
}
|
||||
if os.Getenv("E2E_KEEP") == "1" {
|
||||
fmt.
|
||||
Println("E2E: E2E_KEEP=1, containers left running:", appC, garageC)
|
||||
Println("E2E: E2E_KEEP=1, containers left running:", appC, garageC)
|
||||
return
|
||||
}
|
||||
teardown()
|
||||
@@ -213,8 +213,7 @@ func startApp(image string) error {
|
||||
if _, err := run("docker", args...); err != nil {
|
||||
return fmt.Errorf("app container: %w", err)
|
||||
}
|
||||
if err := retry(60*time.Second, checkHealth);
|
||||
err != nil {
|
||||
if err := retry(60*time.Second, checkHealth); err != nil {
|
||||
if out, _ := run("docker", "logs", appC); out != "" {
|
||||
return fmt.Errorf("app never became healthy; logs:\n%s", out)
|
||||
}
|
||||
@@ -300,8 +299,7 @@ func firstHexToken(s string) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func lineValue(s, prefix string) strin
|
||||
g {
|
||||
func lineValue(s, prefix string) string {
|
||||
for _, line := range strings.Split(s, "\n") {
|
||||
t := strings.TrimSpace(line)
|
||||
if strings.HasPrefix(t, prefix) {
|
||||
@@ -378,8 +376,7 @@ func listRecords(t *testing.T, collection, filter, sort string) []map[string]any
|
||||
q.Set("perPage", "200")
|
||||
status, out := call(t, http.MethodGet, "/api/collections/"+collection+"/records?"+q.Encode(), nil)
|
||||
if status != 200 {
|
||||
t.Fatalf("list %s: status %d: %v
|
||||
", collection, status, out)
|
||||
t.Fatalf("list %s: status %d: %v", collection, status, out)
|
||||
}
|
||||
items, _ := out["items"].([]any)
|
||||
recs := make([]map[string]any, 0, len(items))
|
||||
@@ -456,8 +453,7 @@ func s3Exists(t *testing.T, key string) bool {
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// --- in-memory test PDFs (no fixtures, no ge
|
||||
nerator dependency) ----------
|
||||
// --- in-memory test PDFs (no fixtures, no generator dependency) ----------
|
||||
|
||||
// makePDF builds a minimal valid single-page PDF whose page text is the
|
||||
// given lines. The classification outcome is the proof that the
|
||||
@@ -507,8 +503,7 @@ func putPDF(t *testing.T, key string, lines ...string) string {
|
||||
t.Fatalf("PutObject %s: %v", key, err)
|
||||
}
|
||||
sum := sha256.Sum256(data)
|
||||
return hex.EncodeToString(su
|
||||
m[:])
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// --- the config.jsonc mounted into the image under test -----------------
|
||||
@@ -572,8 +567,7 @@ func TestE2E(t *testing.T) {
|
||||
}
|
||||
}
|
||||
// PocketBase writes log batches with a small delay.
|
||||
var fou
|
||||
nd bool
|
||||
var found bool
|
||||
deadline := time.Now().Add(15 * time.Second)
|
||||
for !found && time.Now().Before(deadline) {
|
||||
for _, l := range pbLogs(t) {
|
||||
@@ -631,8 +625,7 @@ nd bool
|
||||
t.Fatalf("file_key: %v, want %v", doc["file_key"], want)
|
||||
}
|
||||
if !s3Exists(t, want) {
|
||||
t.Fatalf("object %s missing from S3", wan
|
||||
t)
|
||||
t.Fatalf("object %s missing from S3", want)
|
||||
}
|
||||
if s3Exists(t, "inbox/statement-1.pdf") {
|
||||
t.Fatal("source object still in inbox/")
|
||||
@@ -687,8 +680,7 @@ t)
|
||||
p := eventPayloadOfType(t, eventsOf(t, doc["id"].(string)), "imported")
|
||||
errs, ok := p["errors"].([]any)
|
||||
if !ok || len(errs) == 0 {
|
||||
t.Fatalf("
|
||||
want errors in the imported event, got %v", p)
|
||||
t.Fatalf("want errors in the imported event, got %v", p)
|
||||
}
|
||||
after, _ := p["after"].(map[string]any)
|
||||
if after["documentType"] != "Releve de compte" {
|
||||
@@ -737,8 +729,7 @@ want errors in the imported event, got %v", p)
|
||||
t.Fatalf("%s must not appear in a field-edit diff: %v", s, before)
|
||||
}
|
||||
if _, has := after[s]; has {
|
||||
t.Fatalf("%s must not appear in a field-edi
|
||||
t diff: %v", s, after)
|
||||
t.Fatalf("%s must not appear in a field-edit diff: %v", s, after)
|
||||
}
|
||||
}
|
||||
})
|
||||
@@ -788,8 +779,7 @@ t diff: %v", s, after)
|
||||
map[string]any{"event_type": "corrected"}); status < 400 {
|
||||
t.Fatalf("PATCH on document_events must be refused, got %d", status)
|
||||
}
|
||||
if status, _ := call(t, http.MethodDelete, "/api/collections/docum
|
||||
ent_events/records/"+id, nil); status < 400 {
|
||||
if status, _ := call(t, http.MethodDelete, "/api/collections/document_events/records/"+id, nil); status < 400 {
|
||||
t.Fatalf("DELETE on document_events must be refused, got %d", status)
|
||||
}
|
||||
if n := len(listRecords(t, "document_events", "", "")); n != len(evs) {
|
||||
@@ -820,7 +810,6 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
// 9. Recovery (US-11): a stuck record - file_key still under inbox/,
|
||||
// created directly through the API - is completed by the next pass.
|
||||
t.Run("recovery: a stuck filing is completed by the next pass", func(t *testing.T) {
|
||||
@@ -840,8 +829,7 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
t.Fatalf("stuck record creation: %d %v", status, out)
|
||||
}
|
||||
poll(t) // the recovery runs at the start of the pass
|
||||
want := "files/BRED/202
|
||||
6-02_Releve de compte_Thomas.pdf"
|
||||
want := "files/BRED/2026-02_Releve de compte_Thomas.pdf"
|
||||
doc := oneRecord(t, "documents", "(sha256='"+hash+"')")
|
||||
if doc["file_key"] != want {
|
||||
t.Fatalf("file_key: %v, want %v", doc["file_key"], want)
|
||||
@@ -1010,7 +998,7 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
}
|
||||
|
||||
// Nothing persisted.
|
||||
doc2 := oneRecord(t, "documents", "(id='" + id + "')")
|
||||
doc2 := oneRecord(t, "documents", "(id='"+id+"')")
|
||||
if doc2["file_key"] != doc["file_key"] || doc2["status"] != doc["status"] {
|
||||
t.Fatal("dry-run must not touch the record")
|
||||
}
|
||||
@@ -1027,7 +1015,7 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
hash := putPDF(t, "inbox/undated-2.pdf", "BRED", "Releve de compte", "Compte Thomas")
|
||||
poll(t)
|
||||
poll(t)
|
||||
doc := oneRecord(t, "documents", "(sha256='" + hash + "')")
|
||||
doc := oneRecord(t, "documents", "(sha256='"+hash+"')")
|
||||
id := doc["id"].(string)
|
||||
if doc["status"] != "pending" {
|
||||
t.Fatalf("status: %v", doc["status"])
|
||||
@@ -1038,7 +1026,7 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
if status != 200 {
|
||||
t.Fatalf("correct refused: %d %v", status, out)
|
||||
}
|
||||
doc = oneRecord(t, "documents", "(id='" + id + "')")
|
||||
doc = oneRecord(t, "documents", "(id='"+id+"')")
|
||||
if doc["file_key"] != want {
|
||||
t.Fatalf("file_key: %v, want %v", doc["file_key"], want)
|
||||
}
|
||||
@@ -1081,7 +1069,7 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
if status != 400 {
|
||||
t.Fatalf("want 400, got %d %v", status, out)
|
||||
}
|
||||
doc2 := oneRecord(t, "documents", "(id='" + id + "')")
|
||||
doc2 := oneRecord(t, "documents", "(id='"+id+"')")
|
||||
if doc2["file_key"] != doc["file_key"] || doc2["status"] != doc["status"] {
|
||||
t.Fatal("a refused correction must leave the record unchanged")
|
||||
}
|
||||
@@ -1097,7 +1085,7 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
if status != 200 {
|
||||
t.Fatalf("correct refused: %d %v", status, out)
|
||||
}
|
||||
doc2 := oneRecord(t, "documents", "(id='" + id + "')")
|
||||
doc2 := oneRecord(t, "documents", "(id='"+id+"')")
|
||||
if doc2["file_key"] != doc["file_key"] {
|
||||
t.Fatalf("file_key must not change: %v -> %v", doc["file_key"], doc2["file_key"])
|
||||
}
|
||||
@@ -1150,4 +1138,3 @@ ent_events/records/"+id, nil); status < 400 {
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -8,11 +8,14 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"ganigeorgiev.com/pocketbase/pocketbase/apis"
|
||||
"ganigeorgiev.com/pocketbase/pocketbase/core"
|
||||
"github.com/napnap75/archivist/internal/config"
|
||||
"github.com/napnap75/archivist/internal/storage"
|
||||
@@ -23,6 +26,9 @@ import (
|
||||
// PocketBase auth, but these routes bypass the record API and its hooks.
|
||||
func BindRoutes(app core.App, cfg *config.Config, s3 *storage.Client) {
|
||||
app.OnServe().BindFunc(func(e *core.ServeEvent) error {
|
||||
if !e.Router.HasRoute(http.MethodGet, "/{path...}") {
|
||||
e.Router.GET("/{path...}", apis.Static(os.DirFS("pb_public"), true))
|
||||
}
|
||||
e.Router.GET("/api/archivist/config", func(re *core.RequestEvent) error {
|
||||
if !re.HasSuperuserAuth() {
|
||||
return re.ForbiddenError("superusers only", nil)
|
||||
@@ -139,10 +145,10 @@ func fileName(key string) string {
|
||||
// dryRunResponse is the single response shape of the two exclusive
|
||||
// dry-run modes (US-35).
|
||||
type dryRunResponse struct {
|
||||
Fields map[string]any "json:"fields""
|
||||
Template string "json:"template""
|
||||
Rendered string "json:"rendered""
|
||||
Errors []string "json:"errors""
|
||||
Fields map[string]any `json:"fields"`
|
||||
Template string `json:"template"`
|
||||
Rendered string `json:"rendered"`
|
||||
Errors []string `json:"errors"`
|
||||
}
|
||||
|
||||
func handleDryRun(re *core.RequestEvent, cfg *config.Config) error {
|
||||
@@ -270,7 +276,7 @@ func handleCorrect(re *core.RequestEvent, cfg *config.Config, s3 *storage.Client
|
||||
if err := tx.Save(record); err != nil {
|
||||
return err
|
||||
}
|
||||
return AppendEvent(tx, record.Id, "corrected", before, after)
|
||||
return AppendEvent(tx, record.Id, "corrected", before, after, nil)
|
||||
})
|
||||
if txErr != nil {
|
||||
if refiled {
|
||||
|
||||
+2
-4
@@ -65,8 +65,7 @@ func LoadEnvConfig() (*EnvConfig, error) {
|
||||
return c, nil
|
||||
}
|
||||
|
||||
func configPathFromArgs(args []strin
|
||||
g, defaultPath string) (string, error) {
|
||||
func configPathFromArgs(args []string, defaultPath string) (string, error) {
|
||||
var configPath string
|
||||
flags := pflag.NewFlagSet("archivist", pflag.ContinueOnError)
|
||||
flags.SetInterspersed(true)
|
||||
@@ -117,8 +116,7 @@ func main() {
|
||||
}
|
||||
|
||||
// US-01: versioning recommendation, via the PocketBase logger
|
||||
// (visible in the admin dashboard, Das
|
||||
hboard > Logs).
|
||||
// (visible in the admin dashboard, Dashboard > Logs).
|
||||
if s3.StartupWarning != "" {
|
||||
app.Logger().Warn(s3.StartupWarning, "bucket", envCfg.S3Bucket)
|
||||
}
|
||||
|
||||
+166
-18
@@ -1,14 +1,13 @@
|
||||
"use strict";
|
||||
|
||||
// Shared helpers of the review-dashboard pages (US-30 to US-36).
|
||||
// The pages carry no data; everything goes through authenticated API
|
||||
// calls, the admin session cookie riding along.
|
||||
// The pages carry no data; everything goes through authenticated API calls.
|
||||
|
||||
function redirectLogin() {
|
||||
window.location.href = "/_/";
|
||||
}
|
||||
var memoryToken = "";
|
||||
var loginPromise = null;
|
||||
|
||||
function storedToken() {
|
||||
if (memoryToken) return memoryToken;
|
||||
try {
|
||||
return localStorage.getItem("archivist.token") || "";
|
||||
} catch (e) {
|
||||
@@ -16,17 +15,110 @@ function storedToken() {
|
||||
}
|
||||
}
|
||||
|
||||
// api performs one authenticated call. A stale stored token is dropped
|
||||
// and the call retried once; an unauthenticated visitor is redirected to
|
||||
// the admin UI (US-30).
|
||||
function saveToken(token) {
|
||||
memoryToken = token;
|
||||
try {
|
||||
localStorage.setItem("archivist.token", token);
|
||||
} catch (e) {}
|
||||
}
|
||||
|
||||
function clearToken() {
|
||||
memoryToken = "";
|
||||
try {
|
||||
localStorage.removeItem("archivist.token");
|
||||
} catch (e) {}
|
||||
}
|
||||
|
||||
function requestLogin() {
|
||||
if (loginPromise) return loginPromise;
|
||||
|
||||
loginPromise = new Promise(function (resolve) {
|
||||
var dialog = document.createElement("dialog");
|
||||
dialog.className = "auth-dialog";
|
||||
dialog.setAttribute("aria-labelledby", "auth-title");
|
||||
dialog.innerHTML = '<div class="auth-card">' +
|
||||
'<h2 id="auth-title">Connexion requise</h2>' +
|
||||
'<p>Connectez-vous avec un compte superutilisateur.</p>' +
|
||||
'<form class="auth-form">' +
|
||||
'<label>Adresse e-mail<input name="identity" type="email" autocomplete="username" required autofocus></label>' +
|
||||
'<label>Mot de passe<input name="password" type="password" autocomplete="current-password" required></label>' +
|
||||
'<p class="auth-error" role="alert" aria-live="polite"></p>' +
|
||||
'<div class="auth-actions"><a href="/_/">Administration PocketBase</a>' +
|
||||
'<button class="btn primary" type="submit">Se connecter</button></div>' +
|
||||
'</form></div>';
|
||||
|
||||
var form = dialog.querySelector("form");
|
||||
var error = dialog.querySelector(".auth-error");
|
||||
var submit = form.querySelector("button[type=submit]");
|
||||
form.addEventListener("submit", async function (event) {
|
||||
event.preventDefault();
|
||||
error.textContent = "";
|
||||
submit.disabled = true;
|
||||
try {
|
||||
var result = await rawFetch("POST", "/api/collections/_superusers/auth-with-password", {
|
||||
identity: form.elements.identity.value,
|
||||
password: form.elements.password.value
|
||||
}, "");
|
||||
var token = result.data && result.data.token;
|
||||
if (result.status !== 200 || !token) {
|
||||
error.textContent = result.data && result.data.message || "Identifiants invalides.";
|
||||
submit.disabled = false;
|
||||
return;
|
||||
}
|
||||
saveToken(token);
|
||||
dialog.close();
|
||||
dialog.remove();
|
||||
resolve(token);
|
||||
} catch (err) {
|
||||
error.textContent = "Connexion impossible. Vérifiez le réseau puis réessayez.";
|
||||
submit.disabled = false;
|
||||
}
|
||||
});
|
||||
dialog.addEventListener("cancel", function (event) {
|
||||
event.preventDefault();
|
||||
});
|
||||
document.body.appendChild(dialog);
|
||||
dialog.showModal();
|
||||
});
|
||||
|
||||
var pending = loginPromise;
|
||||
return pending.finally(function () {
|
||||
loginPromise = null;
|
||||
});
|
||||
}
|
||||
|
||||
// api retries the blocked request after sign-in. A stale token is cleared
|
||||
// first; the server still enforces superuser access on every protected route.
|
||||
async function api(method, path, body) {
|
||||
var res = await rawFetch(method, path, body, storedToken());
|
||||
if (res.status === 401 && storedToken()) {
|
||||
try { localStorage.removeItem("archivist.token"); } catch (e) {}
|
||||
res = await rawFetch(method, path, body, "");
|
||||
var token = storedToken();
|
||||
var res = await rawFetch(method, path, body, token);
|
||||
if (res.status === 401 && token) {
|
||||
clearToken();
|
||||
token = "";
|
||||
res = await rawFetch(method, path, body, token);
|
||||
}
|
||||
if (res.status === 401 || (res.status === 403 && path.indexOf("/api/archivist/") === 0)) {
|
||||
redirectLogin();
|
||||
token = await requestLogin();
|
||||
res = await rawFetch(method, path, body, token);
|
||||
if (res.status === 401) clearToken();
|
||||
}
|
||||
return res;
|
||||
}
|
||||
|
||||
// An iframe cannot attach the dashboard's Authorization header, so protected
|
||||
// binary resources use fetch and are assigned to the frame as object URLs.
|
||||
async function apiBlob(path) {
|
||||
var token = storedToken();
|
||||
var res = await rawBlob(path, token);
|
||||
if (res.status === 401 && token) {
|
||||
clearToken();
|
||||
token = "";
|
||||
res = await rawBlob(path, token);
|
||||
}
|
||||
if (res.status === 401 || (res.status === 403 && path.indexOf("/api/archivist/") === 0)) {
|
||||
token = await requestLogin();
|
||||
res = await rawBlob(path, token);
|
||||
if (res.status === 401) clearToken();
|
||||
}
|
||||
return res;
|
||||
}
|
||||
@@ -46,6 +138,12 @@ async function rawFetch(method, path, body, token) {
|
||||
return { status: res.status, data: data };
|
||||
}
|
||||
|
||||
async function rawBlob(path, token) {
|
||||
var opts = { method: "GET", headers: {} };
|
||||
if (token) opts.headers["Authorization"] = token;
|
||||
return fetch(path, opts);
|
||||
}
|
||||
|
||||
function errMsg(res) {
|
||||
if (res && res.data && res.data.message) return res.data.message;
|
||||
if (res && res.data) {
|
||||
@@ -94,6 +192,60 @@ function addOption(sel, value, label) {
|
||||
sel.appendChild(o);
|
||||
}
|
||||
|
||||
function buildEnumControl(meta, vocab, value) {
|
||||
var current = value === null || value === undefined ? "" : String(value);
|
||||
var wrapper = document.createElement("div");
|
||||
wrapper.className = "enum-control";
|
||||
wrapper.dataset.field = meta.name;
|
||||
|
||||
var select = document.createElement("select");
|
||||
addOption(select, "", "—");
|
||||
(vocab || []).forEach(function (item) { addOption(select, item, item); });
|
||||
if (current && !(vocab || []).includes(current)) addOption(select, current, current);
|
||||
addOption(select, "__archivist_custom_value__", "Ajouter une valeur…");
|
||||
|
||||
var custom = document.createElement("input");
|
||||
custom.type = "text";
|
||||
custom.placeholder = "Saisir une valeur";
|
||||
custom.setAttribute("aria-label", "Nouvelle valeur pour " + meta.name);
|
||||
custom.hidden = true;
|
||||
|
||||
select.value = current;
|
||||
select.addEventListener("change", function () {
|
||||
var adding = select.value === "__archivist_custom_value__";
|
||||
custom.hidden = !adding;
|
||||
if (adding) {
|
||||
custom.value = "";
|
||||
custom.focus();
|
||||
} else {
|
||||
custom.value = "";
|
||||
}
|
||||
});
|
||||
|
||||
Object.defineProperty(wrapper, "value", {
|
||||
get: function () { return custom.hidden ? select.value : custom.value; },
|
||||
set: function (next) {
|
||||
var nextValue = next === null || next === undefined ? "" : String(next);
|
||||
var optionExists = Array.from(select.options).some(function (option) {
|
||||
return option.value === nextValue;
|
||||
});
|
||||
if (optionExists) {
|
||||
select.value = nextValue;
|
||||
custom.value = "";
|
||||
custom.hidden = true;
|
||||
} else {
|
||||
select.value = "__archivist_custom_value__";
|
||||
custom.value = nextValue;
|
||||
custom.hidden = false;
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
wrapper.appendChild(select);
|
||||
wrapper.appendChild(custom);
|
||||
return wrapper;
|
||||
}
|
||||
|
||||
// buildValueControl builds the typed control of one field (US-33 bulk
|
||||
// edit, US-34 form, US-32 filters).
|
||||
function buildValueControl(meta, vocab, value) {
|
||||
@@ -119,11 +271,7 @@ function buildValueControl(meta, vocab, value) {
|
||||
el.value = value === true ? "true" : (value === false ? "false" : "");
|
||||
break;
|
||||
case "enum":
|
||||
el = document.createElement("select");
|
||||
addOption(el, "", "—");
|
||||
(vocab || []).forEach(function (v) { addOption(el, v, v); });
|
||||
el.value = s;
|
||||
break;
|
||||
return buildEnumControl(meta, vocab, value);
|
||||
default:
|
||||
el = document.createElement("input");
|
||||
el.type = "text";
|
||||
|
||||
@@ -27,6 +27,7 @@ async function init() {
|
||||
|
||||
initTheme("themeBtn");
|
||||
renderHead();
|
||||
loadPdf();
|
||||
var evs = await loadEvents();
|
||||
renderImportErrors(evs);
|
||||
renderHistory(evs);
|
||||
@@ -38,7 +39,7 @@ async function init() {
|
||||
}
|
||||
|
||||
async function loadEvents() {
|
||||
var q = "/api/collections/document_events/records?perPage=200&sort=timestamp&filter=" +
|
||||
var q = "/api/collections/document_events/records?perPage=200&sort=-timestamp&filter=" +
|
||||
encodeURIComponent("(document='" + docId + "')");
|
||||
var r = await api("GET", q);
|
||||
return (r.status === 200 && r.data.items) ? r.data.items : [];
|
||||
@@ -50,7 +51,31 @@ function renderHead() {
|
||||
var st = $("docstatus");
|
||||
st.textContent = record.status || "";
|
||||
st.className = "badge st-" + record.status;
|
||||
$("pdf").src = "/api/archivist/documents/" + docId + "/pdf";
|
||||
}
|
||||
|
||||
async function loadPdf() {
|
||||
var iframe = $("pdf");
|
||||
var error = $("pdferror");
|
||||
try {
|
||||
var response = await apiBlob("/api/archivist/documents/" + docId + "/pdf");
|
||||
if (!response.ok) {
|
||||
var data = null;
|
||||
try { data = await response.json(); } catch (e) {}
|
||||
error.textContent = "Aperçu indisponible : " + (data && data.message || "HTTP " + response.status);
|
||||
error.hidden = false;
|
||||
iframe.hidden = true;
|
||||
return;
|
||||
}
|
||||
var objectUrl = URL.createObjectURL(await response.blob());
|
||||
iframe.src = objectUrl;
|
||||
window.addEventListener("pagehide", function () { URL.revokeObjectURL(objectUrl); }, { once: true });
|
||||
} catch (e) {
|
||||
var reason = e && e.message ? e.message : String(e);
|
||||
console.error("PDF preview failed:", e);
|
||||
error.textContent = "Impossible de charger l'aperçu PDF : " + reason;
|
||||
error.hidden = false;
|
||||
iframe.hidden = true;
|
||||
}
|
||||
}
|
||||
|
||||
function payloadOf(ev) {
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
<div class="detailgrid">
|
||||
<section class="card pdfpane">
|
||||
<iframe id="pdf" title="Aperçu du document"></iframe>
|
||||
<div id="pdferror" class="errbox" hidden></div>
|
||||
</section>
|
||||
<section>
|
||||
<div id="importerrors" class="errbox" hidden></div>
|
||||
|
||||
@@ -161,12 +161,15 @@ tbody tr:hover { background: var(--accent); }
|
||||
|
||||
.pdfpane { padding: 0; overflow: hidden; height: 75vh; }
|
||||
.pdfpane iframe { width: 100%; height: 100%; border: 0; display: block; background: var(--muted); }
|
||||
#pdf[hidden] { display: none; }
|
||||
#pdferror { margin: 1rem; }
|
||||
|
||||
.formgrid { display: grid; grid-template-columns: 1fr 1fr; gap: .75rem 1rem; }
|
||||
@media (max-width: 700px) { .formgrid { grid-template-columns: 1fr; } }
|
||||
.fieldrow { display: flex; flex-direction: column; gap: .25rem; }
|
||||
.fieldrow > label { font-size: 12px; color: var(--muted-foreground); }
|
||||
.fieldrow input, .fieldrow select { width: 100%; }
|
||||
.enum-control { display: grid; gap: .35rem; }
|
||||
.mandatory { color: var(--destructive); }
|
||||
.proposal { font-size: 12px; color: var(--muted-foreground); display: flex; gap: .5rem; align-items: center; }
|
||||
.proposal[hidden] { display: none; }
|
||||
@@ -182,6 +185,24 @@ tbody tr:hover { background: var(--accent); }
|
||||
.errbox[hidden] { display: none; }
|
||||
.errbox ul { margin: .25rem 0 0 1rem; padding: 0; }
|
||||
|
||||
.auth-dialog {
|
||||
width: min(28rem, calc(100vw - 2rem));
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: 0;
|
||||
background: var(--card);
|
||||
color: var(--card-foreground);
|
||||
}
|
||||
.auth-dialog::backdrop { background: hsl(220 8% 10% / .52); }
|
||||
.auth-card { padding: 1.25rem; }
|
||||
.auth-card h2 { margin: 0; font-size: 1.1rem; }
|
||||
.auth-card > p { margin: .35rem 0 1rem; color: var(--muted-foreground); }
|
||||
.auth-form { display: grid; gap: .75rem; }
|
||||
.auth-form label { display: grid; gap: .25rem; color: var(--muted-foreground); }
|
||||
.auth-form input { width: 100%; }
|
||||
.auth-error { min-height: 1.3em; margin: 0; color: var(--destructive); }
|
||||
.auth-actions { display: flex; justify-content: space-between; align-items: center; gap: .75rem; flex-wrap: wrap; }
|
||||
|
||||
.targetname { font-size: 13px; color: var(--muted-foreground); margin-bottom: .75rem; word-break: break-all; }
|
||||
.actions { display: flex; gap: .75rem; margin-top: 1rem; flex-wrap: wrap; }
|
||||
|
||||
|
||||
@@ -253,7 +253,7 @@ Acceptance criteria:
|
||||
|
||||
- Given the image is built and running, when I open `http://<server>:8090/`, then the dashboard pages are served from `pb_public/` — plain HTML/CSS/JS, no build toolchain, dependencies vendored locally, nothing loaded from a CDN.
|
||||
- Given a page is opened, when it renders, then it carries no document data by itself; everything is loaded through authenticated API calls.
|
||||
- Given an unauthenticated visitor, when a page requests data, then the API refuses it and the page redirects to the admin UI (`/_/`).
|
||||
- Given an unauthenticated visitor, when a page requests data, then the page prompts for superuser credentials, authenticates through PocketBase, and retries the request; the API refuses access until authentication succeeds.
|
||||
- Given the pages, when they are styled, then they follow the Beszel design reference (section 7.5): same semantic HSL tokens, light/dark pair switched by a class on `<html>`, cards, hairline borders, sticky table headers — reproduced in one hand-written stylesheet, no Tailwind or CSS framework.
|
||||
|
||||
### US-31 — Expose the configuration metadata
|
||||
|
||||
Reference in New Issue
Block a user