sciagent code + Gitea Actions CI/CD
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Thinh Lam
@@ -0,0 +1,101 @@
|
||||
# Audit Log Manager — Implementation notes (review & debug)
|
||||
|
||||
This document describes the **Audit Log Manager** delivered in this repo: database schema, backend recording, admin API, frontend layout, and how to verify end-to-end (Postgres, API, MinIO).
|
||||
|
||||
## 1. Database
|
||||
|
||||
- **Migration:** `be0/migrations/008_audit_events.sql`
|
||||
- Creates PostgreSQL enum **`audit_action`** and table **`audit_events`** (append-only by convention).
|
||||
- Indexes: actor+time, entity+time, action+time, GIN on `metadata`.
|
||||
|
||||
**Apply** (example against local Docker Postgres — adjust connection):
|
||||
|
||||
```bash
|
||||
psql "$INITIATIVE_DATABASE_URL" -f be0/migrations/008_audit_events.sql
|
||||
```
|
||||
|
||||
(`INITIATIVE_DATABASE_URL` is typically `postgresql://…` for `psql`; the app uses SQLAlchemy async URL like `postgresql+asyncpg://…`.)
|
||||
|
||||
Docker Compose mounts `008_audit_events.sql` into `docker-entrypoint-initdb.d` for **new** databases only. **Existing** `initiative_pg_data` volumes still need the `psql -f …/008_audit_events.sql` step once.
|
||||
|
||||
## 2. Backend model & helpers
|
||||
|
||||
| Piece | Path |
|
||||
|-------|------|
|
||||
| ORM model | `be0/src/initiative_db/models.py` — `AuditEvent` |
|
||||
| Helpers | `be0/src/audit.py` — `record_audit`, `persist_audit_standalone`, `resolve_actor_fields` |
|
||||
|
||||
- **`await record_audit(session, …)`** — insert via a **SAVEPOINT**: if `audit_events` is missing (migration not applied), logs a warning and **does not roll back** the parent transaction (so login/register still succeed).
|
||||
- **`await persist_audit_standalone(…)`** — own session + `commit`; same missing-table handling without raising.
|
||||
|
||||
## 3. Where events are written
|
||||
|
||||
| Area | Actions / entity_type | Notes |
|
||||
|------|----------------------|--------|
|
||||
| Auth | `register` → `create` / `user` | Same transaction as user + roles |
|
||||
| Auth | OK login → `login` / `auth` | |
|
||||
| Auth | Bad login → `login_failed` / `auth` | **Standalone** insert after failed credentials |
|
||||
| Auth | Valid Bearer logout → `logout` / `auth` | **Standalone**; skipped if JWT does not decode |
|
||||
| Auth | Profile patch → `update` / `user` | Before/after: `fullName`, `phone` |
|
||||
| Auth | Password change → `update` / `user` | Snapshots `{ password: "[redacted|changed]" }` only |
|
||||
| Drafts autosave | `update` / `application_draft` | When `owner_id` is known (authenticated saves) |
|
||||
| MinIO evidence | `create`/`update`/`delete` / `application_evidence` | After DB + object write/delete in `main.py` |
|
||||
| Staff evidence review | `update` / `application_evidence_review` | |
|
||||
|
||||
Admin adjudication (`application_admin_results.py`):
|
||||
|
||||
- `create` / `application_admin_result`
|
||||
- `update` / …
|
||||
- `upsert` → `create` or `update` depending on prior row
|
||||
- `delete` — requires **`actor_user_id`**; `delete_admin_result(..., actor_user_id=…)` called from `main.py`
|
||||
|
||||
Legacy table **`audit_log`** (draft telemetry) remains unchanged.
|
||||
|
||||
## 4. Admin HTTP API
|
||||
|
||||
- **JWT decode** for admin routes (`decode_bearer_token`, `decode_access_token_user_id`) lives in **`be0/src/auth_jwt.py`** so audit routes do not pull in Argon2 at import time.
|
||||
- **Router:** `be0/src/admin_audit_routes.py`, mounted in `main.py` as **`/api/v1/admin`**.
|
||||
- **`GET /api/v1/admin/audit`** — list (admin JWT only).
|
||||
- Default window: **now − 7 days** → **now** if `from` / `to` omitted.
|
||||
- Query params: `from`, `to`, `actor_user_id`, `actor_email` **(exact, lowercased)** `entity_type`, `entity_id`, `action` (comma-separated), `request_id`, `page`, `page_size` (≤ 100), `sort` (`occurred_at:asc|desc`).
|
||||
- **`GET /api/v1/admin/audit/{id}`** — full row incl. `before` / `after` JSON.
|
||||
|
||||
Non-admin receives **403**.
|
||||
|
||||
## 5. Frontend
|
||||
|
||||
| Layer | Path |
|
||||
|-------|------|
|
||||
| Shared types + API client | `fe0/src/audit/` |
|
||||
| Admin UI | `fe0/src/admin/audit/` — `AuditLogManagerPage`, filters, table, detail **Sheet** |
|
||||
| Applicant-side copy (reuse) | `fe0/src/applicant/audit/actionLabels.ts` — Vietnamese labels for actions |
|
||||
| Nav | **`fe0/src/components/admin/DashboardSidebar.tsx`** (+ duplicate `DashboardSidebar.tsx`) — **« Nhật ký Audit »** → `/dashboard/admin/audit` |
|
||||
| Routes | `fe0/src/App.tsx` — **`admin/audit` before `admin`** nested under dashboard |
|
||||
|
||||
Detail panel loads **`/api/v1/admin/audit/{id}`** and runs **`microdiff`** on the client only (see `audit/jsonMicrodiffLines.ts`), per architecture doc.
|
||||
|
||||
Dependency: **`microdiff`** (added to `fe0/package.json`).
|
||||
|
||||
## 6. Debugging checklist
|
||||
|
||||
1. **500 on `/auth/login` after adding audit** — Postgres log / SQLAlchemy showed `relation "audit_events" does not exist`: apply migration `008`. Until then, **`record_audit`** skips the insert and auth still works.
|
||||
2. **`ECONNREFUSED` from `fe0` to `:4402`** — API not listening yet (`be0` still on NLTK pip, or old entrypoint failed on `ollama: command not found`). Rebuild `be0`; entrypoint now **skips Ollama** when the binary is absent. Wait for `Uvicorn running on http://0.0.0.0:4402` before logging in.
|
||||
3. **Table missing / Admin audit 503** — run migration `008_audit_events.sql`.
|
||||
4. **Empty list but events expected** — check `from`/`to`; default is last **7 days** on the API.
|
||||
5. **403 on `/api/v1/admin/audit`** — JWT must include **`admin`** in `roles` (same as other admin APIs).
|
||||
6. **Failed-login rows missing** — must use **`persist_audit_standalone`** path; if Postgres URL wrong, standalone insert is skipped (`is_postgres_enabled()`).
|
||||
7. **MinIO uploads without audit rows** — evidence handler must reach `upsert_evidence_artifact` **after** successful `s3.upload`; failures before that do not audit.
|
||||
8. **Comment typo broke build once** — `formatAuditTime.ts` JSDoc used invalid `**/”` sequence; stick to **` */`** closing block comments.
|
||||
|
||||
## 7. Suggested verification flow
|
||||
|
||||
1. Apply migration → restart API.
|
||||
2. Register / login → see `create`/`login` rows (filter `action=login,create`).
|
||||
3. Save applicant draft tab → `application_draft`.
|
||||
4. Upload evidence with MinIO configured → `application_evidence` with `metadata.minioBucket`.
|
||||
5. Open **`/dashboard/admin/audit`** as admin → pagination + row click opens JSON + microdiff.
|
||||
|
||||
## 8. References
|
||||
|
||||
- Spec: `assets/docs/audit-log-implementation.md`
|
||||
- Product plan: `docs/audit-log-manager-plan.md`
|
||||
Reference in New Issue
Block a user