sciagent code + Gitea Actions CI/CD
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Thinh Lam
@@ -0,0 +1,175 @@
|
||||
# Radical fix plan: admin applications, admin results, and dashboard consistency
|
||||
|
||||
This document captures how the current stack is wired, what went wrong with “save decision → see it in **Kết quả đăng ký**”, and a **phased, radical** plan to make the behavior **reliable by design**—not only by patching one screen.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem statement (what users experience)
|
||||
|
||||
1. An admin uses **Mẫu hồ sơ và Minh chứng** → **Duyệt (xem trước)** → **Xác nhận** (`AdminStaffReadonlyReviewDialog` + `upsertAdminApplicationResult`).
|
||||
2. They expect the submission to appear under **Kết quả đăng ký** (`ConsideredInitiativesList` → `ApprovedApplicationsList` with `lifecycle=decided`).
|
||||
3. Sometimes nothing changes, or behaviour is confusing, because **several independent subsystems** must stay aligned: HTTP client semantics, API routes, PostgreSQL `initiatives.status`, optional file fallback, React Query keys, and optional MinIO (evidence—not the same as admin result).
|
||||
|
||||
---
|
||||
|
||||
## 2. Implementation map (how it fits together)
|
||||
|
||||
### 2.1 Frontend
|
||||
|
||||
| Piece | Role |
|
||||
|--------|------|
|
||||
| `ConsideredInitiativesList` | Renders `ApprovedApplicationsList` with `lifecycle="decided"` — only rows whose **list `status`** is `approved` or `rejected`. |
|
||||
| `ApprovedApplicationsList` | `useQuery(["applications", filters])` → `GET /api/applications` with filters including `lifecycle`. |
|
||||
| `AdminDocxTemplatePreview` | Opens `AdminStaffReadonlyReviewDialog`; confirm calls `upsertAdminApplicationResult`. |
|
||||
| `applicationAdminResultApi` | `fetch` + `create` / `update`; `upsert` = **GET then POST or PUT**. |
|
||||
| `shared/api/client.ts` | Axios `validateStatus: (s) => s < 500` — **4xx responses do not throw** unless callers pass an override. |
|
||||
|
||||
### 2.2 Backend
|
||||
|
||||
| Piece | Role |
|
||||
|--------|------|
|
||||
| `POST/PUT/GET/DELETE …/admin-result` | Persists `application_admin_results` and sets **`initiatives.status`** to `approved` / `rejected` (PostgreSQL). |
|
||||
| `GET /api/applications` | **Primary:** `list_submitted_applications` from Postgres. **Fallback:** `_load_submitted_items()` file index if Postgres fails or is disabled. |
|
||||
| `GET /api/applications/{id}` | Same pattern: Postgres first, then file index. |
|
||||
| MinIO (S3-compatible) | Evidence / attachments buckets; **not** where admin “decision rows” live. Decisions are **Postgres**; MinIO only matters for evidence flows. |
|
||||
|
||||
### 2.3 Data flow (intended)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant UI as Admin UI
|
||||
participant API as be0 FastAPI
|
||||
participant PG as Postgres
|
||||
participant List as GET /api/applications
|
||||
|
||||
UI->>API: upsert admin-result (POST or PUT)
|
||||
API->>PG: application_admin_results + initiatives.status
|
||||
UI->>API: invalidate + refetch applications
|
||||
List->>PG: list initiatives + drafts → status approved/rejected
|
||||
List-->>UI: decided list
|
||||
```
|
||||
|
||||
Breakage happens when **any** step returns “success” without real data, reads from a **different backend** than writes, or treats **HTTP 404** as a valid JSON body.
|
||||
|
||||
---
|
||||
|
||||
## 3. Root causes (systemic, not one-line bugs)
|
||||
|
||||
### A. Global Axios: 4xx treated as success (`validateStatus < 500`)
|
||||
|
||||
- For `GET /api/.../admin-result` with **no row**, the server returns **404** + `{ detail: "…" }`.
|
||||
- With the default client, that **resolves** instead of rejects.
|
||||
- Any code that checks `if (!data)` **fails** → truthy object `{ detail }` looks like “existing result”.
|
||||
- **`upsert`** then chose **PUT** instead of **POST** on first save → no row created, **silent wrong success** possible.
|
||||
|
||||
**Partial fix already applied:** pass `axiosSuccessStatusOnly` (2xx-only) on all `applicationAdminResultApi` calls.
|
||||
|
||||
**Radical fix:** see §5.1 — stop relying on opt-in overrides.
|
||||
|
||||
### B. Client upsert = GET + mutate (race + footguns)
|
||||
|
||||
- Two round trips; duplicated server rules; easy to get wrong if GET semantics change.
|
||||
- Prefer **idempotent server upsert** (`PUT` with “create or replace” semantics) **or** `POST`-only with clear 409 handling.
|
||||
|
||||
### C. Dual source of truth for application lists (Postgres vs file index)
|
||||
|
||||
- Listing can **silently fall back** to `_load_submitted_items()` when Postgres throws.
|
||||
- Admin-result writes **only** hit Postgres.
|
||||
- Result: UI can show **stale or empty** “decided” data while DB was actually updated (or the reverse in dev).
|
||||
|
||||
### D. React Query key `["applications", filters]`
|
||||
|
||||
- Invalidate `["applications"]` is correct for TanStack Query partial matching, but **any** cache/subscription edge case should be covered by tests.
|
||||
|
||||
### E. MinIO vs admin decision (scope confusion)
|
||||
|
||||
- **Fixing “Kết quả đăng ký”** does not require MinIO updates.
|
||||
- Evidence upload paths are separate; do not conflate in testing or plans.
|
||||
|
||||
---
|
||||
|
||||
## 4. Verification already performed (baseline)
|
||||
|
||||
- **Docker:** `postgres`, `minio`, `be0` healthy; `GET /api/v1/test` OK.
|
||||
- **Postgres + `create_admin_result`:** `initiatives.status` and `application_admin_results` stay in sync when using the Python layer directly.
|
||||
- **Integration test `test_applications_db_integration`:** one failing test (`get_application_by_id` fallback `sub-…` without `submissionRecord.id`) — suggests **ID-resolution** edge cases still risky; align with list/GET contract in the same plan.
|
||||
- **Host Python** may lack `boto3`; validate MinIO from **`be0` container** or install dev deps locally for S3 tests.
|
||||
|
||||
---
|
||||
|
||||
## 5. Radical fixes (options, from smaller to larger)
|
||||
|
||||
### 5.1 Frontend: default to real HTTP semantics (highest leverage)
|
||||
|
||||
**Goal:** No API call “succeeds” with a 404/422 body unless explicitly handled.
|
||||
|
||||
**Options:**
|
||||
|
||||
1. **Change default `validateStatus` to 2xx-only** in `ApiClient`, then **globally fix** call sites that depended on reading `{ detail }` from a resolved 4xx response (likely few; grep for patterns).
|
||||
2. **Two clients:** `apiClientStrict` (2xx-only) for CRUD and `apiClient` legacy only where needed—migrate modules incrementally.
|
||||
3. **Response interceptor:** if `status >= 400`, reject with unified `ApiError` (preserves current “no throw on 4xx” idea but **never** returns `res.data` as success to `.then()`).
|
||||
|
||||
**Acceptance:** ESLint rule or CI script: forbid `apiClient.get/post/put/delete` without explicit `validateStatus` or wrapper.
|
||||
|
||||
### 5.2 Backend: atomic admin-result upsert
|
||||
|
||||
**Goal:** Single request, no client-side GET-before-POST.
|
||||
|
||||
- Expose **`PUT /api/applications/{id}/admin-result`** as **idempotent upsert** (create or update in one transaction), or document **`POST`** as upsert with unique constraint handling.
|
||||
- Optionally return **updated application row** snippet (`status`, `applicationId`) so the client can patch cache without listing.
|
||||
|
||||
### 5.3 Backend: single listing source in production
|
||||
|
||||
**Goal:** No silent list fallback when `INITIATIVE_DATABASE_URL` is set.
|
||||
|
||||
- On Postgres enabled: **fail listing with 503** and a clear JSON error instead of falling back to files; or log + metrics + feature flag.
|
||||
- Deprecate file index for `/api/applications` in environments where submissions are always in PG.
|
||||
|
||||
### 5.4 Contract tests (API + FE)
|
||||
|
||||
- **pytest/httpx:** `POST admin-result` → `GET /api/applications?lifecycle=decided` contains that `id`.
|
||||
- **Playwright or MSW:** Admin flow confirm → list row appears (requires auth fixture).
|
||||
|
||||
### 5.5 Observability
|
||||
|
||||
- Structured logs: `application_id`, `initiative_id`, `decision`, `source=postgres|file_fallback`.
|
||||
- Metrics: `applications_list_fallback_total`, `admin_result_upsert_duration_ms`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Recommended phases (practical rollout)
|
||||
|
||||
| Phase | Scope | Outcome |
|
||||
|-------|--------|---------|
|
||||
| **P0** | Keep `axiosSuccessStatusOnly` on admin-result API; add **one** e2e/API test: upsert → decided list. | Regression guard. |
|
||||
| **P1** | Introduce **strict HTTP** default or interceptor (§5.1); fix broken call sites. | Class of bugs eliminated. |
|
||||
| **P2** | Backend **idempotent PUT** upsert; simplify client to single call. | Fewer races, simpler mental model. |
|
||||
| **P3** | Remove or gate **file fallback** for `/api/applications` when PG is configured. | Align list with admin writes. |
|
||||
| **P4** | Fix failing DB test for `submissionRecord.id` omission; document canonical `applicationId` rules. | Predictable IDs end-to-end. |
|
||||
|
||||
---
|
||||
|
||||
## 7. Out of scope / non-goals
|
||||
|
||||
- **MinIO** consistency for “admin approve” — wrong layer unless the feature explicitly writes objects on decision.
|
||||
- **Council** flow (`saveCouncilReviewOutcome` / local storage) — separate product path; only mention if merging with admin outcomes.
|
||||
|
||||
---
|
||||
|
||||
## 8. Decision log (fill in as you implement)
|
||||
|
||||
| Date | Decision | Rationale |
|
||||
|------|----------|-----------|
|
||||
| | | |
|
||||
|
||||
---
|
||||
|
||||
## References (code)
|
||||
|
||||
- `fe0/src/shared/api/client.ts` — global `validateStatus`.
|
||||
- `fe0/src/lib/applicationReviewApi.ts` — `axiosSuccessStatusOnly`.
|
||||
- `fe0/src/lib/applicationAdminResultApi.ts` — admin-result CRUD + upsert.
|
||||
- `fe0/src/components/admin/result/ConsideredInitiativesList.tsx` — decided list entry point.
|
||||
- `be0/src/initiative_db/application_admin_results.py` — DB writes + `initiative.status`.
|
||||
- `be0/main.py` — `list_applications` Postgres vs file fallback.
|
||||
- `be0/tests/test_applications_db_integration.py` — Postgres integration tests.
|
||||
Reference in New Issue
Block a user