tlam89/sciagent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sciagent code + Gitea Actions CI/CD
CI/CD / backend (push) Failing after 2m8s
Details
CI/CD / frontend (push) Failing after 1m40s
Details
CI/CD / deploy (push) Has been skipped
Details

Browse Source 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Thinh Lam
2026-06-30 09:38:30 +07:00
commit 688fac73e9
1167 changed files with 158244 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
+23
View File
@@ -0,0 +1,23 @@
# Build context for the frontend_user / frontend_admin images is the repo ROOT (the
# npm workspace). Keep only root manifests + shared/ + the two app dirs; exclude the
# rest so the context stays small. (be0 builds from ./be0 and is unaffected by this file.)
**/node_modules
**/dist
**/dist-ssr
.git
.gitignore
.dockerignore
.claude
docs
be0
fe0
assets
database
Posgresdb
scripts
deploy
*.md
.env
.env.*
.DS_Store
**/__pycache__
.env.example
+105
View File
@@ -0,0 +1,105 @@
# ============================================================
# Production / docker-compose.prod.yml
# -----------------------------------------------------------
# 1. Copy: cp .env.example .env
# 2. Fill every value below (never commit .env — it is gitignored).
# 3. Prefer strong random secrets:
# openssl rand -base64 32
#
# Before deploy: ./scripts/verify-prod-env.sh
# Full deploy: ./scripts/deploy-prod.sh
# Stack map (FE→BE→DB→MinIO): docs/deploy-stack-overview.md
# Postgres / volume quirks: docs/deploy-production-docker.md
#
# If .env was ever committed to git, rotate ALL secrets below.
# ============================================================
# Public hostname or IP that browsers use to reach this machine.
PUBLIC_HOST=your-public-hostname-or-ip.example.com
FE_PORT=8081
# Optional: admin/council SPA port. Bound to 127.0.0.1 only in docker-compose.prod.yml
# (reach it via SSH tunnel or an authenticated reverse-proxy vhost). Defaults to 8082.
# FE_ADMIN_PORT=8082
# Optional: principal-investigator SPA port (research proposals + project cockpit). Defaults to 8083.
# FE_INV_PORT=8083
# Optional: publisher SPA port (research-result publication). Defaults to 8084.
# FE_PUB_PORT=8084
# Optional: extra CORS Allowed-Origins for be0 (comma-separated, no spaces). Production compose sets
# CORS_ORIGINS to http://${PUBLIC_HOST}:${FE_PORT} plus these extras automatically.
# CORS_ORIGINS_EXTRA=https://app.example.com,http://internal:8081
MINIO_API_PORT=19000
MINIO_CONSOLE_PORT=19001
MINIO_ROOT_USER=minio_root_change_me
MINIO_ROOT_PASSWORD=replace_with_long_random_secret
# --- HTTPS for MinIO presigned URLs (required if the SPA is https://…) ------------
# Mixed content blocks http://PUBLIC_HOST:19000 embedded from an HTTPS UI. Options:
# A) Proxied viewer only (already in-app) — no change needed for preview.
# B) HTTPS for direct MinIO links (iframe / “open presigned URL”) — put TLS in front
# of the S3 API port and align these with that public URL. See docs/minio-behind-https.md .
# Example subdomain (recommended):
# S3_PUBLIC_ENDPOINT_URL=https://minio-api.your-domain.com
# MINIO_SERVER_URL=https://minio-api.your-domain.com
# Optionally point the console at HTTPS too:
# MINIO_BROWSER_REDIRECT_URL=https://minio-console.your-domain.com
# If omitted, Compose keeps using http://${PUBLIC_HOST}:${MINIO_API_PORT} for both.
# Username + password are fixed the first time the Postgres volume is created (see comment below).
# Identifier only (letters, digits, underscore) — avoids URL / healthcheck pitfalls.
POSTGRES_USER=postgres_app_user
POSTGRES_PASSWORD=replace_with_long_random_secret
# Optional: only for scripts/sync-postgres-app-password.sh when the app role is not superuser
# or you must connect as a different DB superuser (e.g. postgres) to run ALTER ROLE.
# POSTGRES_SUPERUSER=postgres
# Database name created on first init (normally keep "initiatives").
POSTGRES_DB=initiatives
# --- Auth (required for production) ------------------------------------------------
# Generate: openssl rand -base64 48
JWT_SECRET=replace_with_openssl_rand_base64_48
# MinIO browser CORS — your public SPA origin (scheme + host, no trailing slash).
MINIO_API_CORS_ALLOW_ORIGIN=https://www.example.com
# Postgres + password caveat:
# Changing POSTGRES_USER/POSTGRES_PASSWORD here later does NOT change an existing Docker volume —
# Postgres only reads them when /var/lib/postgresql/data is empty. If login fails after editing .env:
# • Use the same password as first boot (e.g. dev stack used initiative / initiative_secret), or
# • With docker-compose.prod.yml stopped: docker volume rm …_initiative_pg_data then up again (drops DB), or
# • Run ./scripts/sync-postgres-app-password.sh to set the DB role password from this file (no wipe), or
# ---------------------------------------------------------------------------
# SMTP — outbound mail from be0 (registration OTP, password reset)
# ---------------------------------------------------------------------------
# docker-compose / docker-compose.prod passes these into the be0 container.
# Compose substitutes ${SMTP_*} from THIS file (repo-root `.env`), not from be0/.env alone.
# Omit AUTH_MAIL_LOG_ONLY (or set 0/false) when using real SMTP.
#
# SMTP_HOST=smtp.your-mail-provider.com
# SMTP_PORT=587
# SMTP_USER=your_smtp_username
# SMTP_PASSWORD=your_smtp_password
# AUTH_MAIL_FROM=noreply@your-institution.edu.vn
# SMTP_USE_TLS=1
#
# Public URL of the web app (password-reset / verify links in email). Production example:
# AUTH_PUBLIC_WEB_ORIGIN=https://your-app.example.com
#
# Dev-only: print OTP in be0 logs instead of sending mail
# AUTH_MAIL_LOG_ONLY=1
#
# Microsoft 365 / Outlook (smtp.office365.com), log shows 535 Authentication unsuccessful:
# • SMTP_USER = full mailbox address; SMTP_PASSWORD = correct app password if MFA is enabled
# (not your normal web-login password unless basic auth is allowed — many tenants require app passwords).
# • Exchange admin: enable "Authenticated SMTP" for the mailbox; security defaults may block SMTP AUTH.
# • After editing .env: docker compose up -d be0 (so the container reloads env).
.gitea/workflows/ci-cd.yml
+99
View File
@@ -0,0 +1,99 @@
name: CI/CD
# Gitea Actions pipeline for the UMP / ImageHub monorepo.
# backend — be0 (FastAPI, Python 3.11) pytest against a throwaway Postgres
# frontend — npm workspaces (shared + 4 Vite/React SPAs): typecheck, build, unit tests
# deploy — on push to main only: build + `docker compose up -d` on the host runner
#
# Runner labels expected (act_runner registered on 103.149.170.102):
# ci -> docker mode (clean, ephemeral) used by backend + frontend
# deploy -> host mode (drives host docker) used by deploy
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
backend:
runs-on: ci
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: initiative
POSTGRES_PASSWORD: initiative_secret
POSTGRES_DB: initiatives
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U initiative -d initiatives"
--health-interval 5s --health-timeout 5s --health-retries 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install backend deps (+ test deps)
working-directory: be0
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Unit tests — pytest PER FILE (isolates asyncpg event loop per module)
working-directory: be0
env:
INITIATIVE_DATABASE_URL: postgresql+asyncpg://initiative:initiative_secret@postgres:5432/initiatives
run: |
set -e
fail=0
for f in tests/test_*.py; do
echo "::group::$f"
python -m pytest "$f" -q || fail=1
echo "::endgroup::"
done
exit $fail
frontend:
runs-on: ci
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
- name: Install (workspaces)
run: npm ci
- name: Typecheck (all workspaces)
run: npm run typecheck
- name: Build (all workspaces)
run: npm run build
- name: Unit tests (workspaces w/ vitest — shared, investigator, publisher)
run: npm test --workspaces --if-present
# Deploy runs in HOST mode from a PERSISTENT dir (NOT the ephemeral runner
# workspace): docker-compose.prod.yml bind-mounts ./assets/minio-data and
# ./be0, so MinIO data + submitted files must live on a stable host path or
# they would be wiped on every deploy.
deploy:
needs: [backend, frontend]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: deploy
steps:
- name: Sync code to persistent deploy dir
run: |
set -euo pipefail
DEPLOY_DIR=/srv/sciagent
if [ ! -d "$DEPLOY_DIR/.git" ]; then
git clone http://localhost:3000/tlam89/sciagent.git "$DEPLOY_DIR"
fi
cd "$DEPLOY_DIR"
git fetch origin main
git reset --hard origin/main
- name: Materialize prod .env from secret
run: |
set -euo pipefail
printf '%s' "${{ secrets.PROD_ENV }}" > /srv/sciagent/.env
chmod 600 /srv/sciagent/.env
- name: Deploy stack (build locally, no registry pull)
run: cd /srv/sciagent && bash scripts/deploy-prod.sh --no-pull
- name: Stack health check
run: cd /srv/sciagent && bash scripts/check-prod-stack.sh
.gitignore
+41
View File
@@ -0,0 +1,41 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
# Secrets — commit only `.env.example`, never `.env`.
.env
.env.local
.env.*.local
# Keep the example/template
!.env.example
assets/minio-data/*
be0/.venv/
# HMW-mode marker — session-local toggle (/ultra-on … /ultra-off). Never commit;
# committing it would leave a fresh `git clone` stuck in token-burn mode.
.claude/hmw-mode.on
LICENSE
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Posgresdb/crud_examples.sql
+137
View File
@@ -0,0 +1,137 @@
-- =============================================================================
-- CRUD PATTERNS — Sáng kiến application system
-- =============================================================================
-- =============================================================================
-- CREATE: Submit a new application with multiple authors (atomic)
-- =============================================================================
BEGIN;
-- Set audit context
SELECT set_config('my.user_id', '42', true);
-- 1. Main record
INSERT INTO applications(code, title, registration_year, status, purpose,
is_technical_solution, primary_unit_id, created_by)
VALUES ('SK-2025-007',
'Hệ thống tự động điền hồ sơ sáng kiến',
2025, 'DRAFT',
'Tự động hoá việc điền các mẫu số 01–04',
TRUE, 2, 42)
RETURNING application_id \gset
-- 2. Authors (defer contribution-sum check until COMMIT)
SET CONSTRAINTS trg_contribution_total DEFERRED;
INSERT INTO application_authors(application_id, user_id, contribution_pct, role, display_order) VALUES
(:application_id, 42, 60.00, 'PRIMARY', 1),
(:application_id, 13, 25.00, 'CO_AUTHOR', 2),
(:application_id, 27, 15.00, 'CO_AUTHOR', 3);
-- 3. Orgs that tested it
INSERT INTO application_adopters(application_id, org_name, address, field) VALUES
(:application_id, 'Phòng KHCN', '217 Hồng Bàng, Q.5', 'Cải cách hành chính');
COMMIT;
-- =============================================================================
-- READ: Dashboard — paginated list with filters
-- =============================================================================
SELECT * FROM v_application_summary
WHERE registration_year = 2025
AND status = ANY(ARRAY['UNDER_REVIEW','EVALUATED']::text[])
AND title ILIKE '%động vật%' -- uses trigram index
ORDER BY avg_score DESC NULLS LAST, submitted_at DESC
LIMIT 20 OFFSET 0;
-- Read: full application with nested data (app layer usually does this as N queries
-- or one JSON aggregate — here's the aggregate version)
SELECT jsonb_build_object(
'application', to_jsonb(a.*),
'authors', (SELECT jsonb_agg(jsonb_build_object(
'user_id', u.user_id,
'name', u.full_name,
'pct', aa.contribution_pct,
'role', aa.role
) ORDER BY aa.display_order)
FROM application_authors aa
JOIN users u USING (user_id)
WHERE aa.application_id = a.application_id),
'evaluations',(SELECT jsonb_agg(to_jsonb(e.*))
FROM evaluations e WHERE e.application_id = a.application_id),
'attachments',(SELECT jsonb_agg(to_jsonb(att.*))
FROM attachments att WHERE att.application_id = a.application_id)
) AS document
FROM applications a
WHERE a.application_id = 1 AND a.deleted_at IS NULL;
-- Full-text search (Vietnamese-friendly; combine with unaccent for better recall)
SELECT application_id, code, title
FROM applications
WHERE to_tsvector('simple', title || ' ' || coalesce(introduction,''))
@@ plainto_tsquery('simple', 'đạo đức động vật')
ORDER BY registration_year DESC
LIMIT 10;
-- =============================================================================
-- UPDATE: Progress an application through the workflow
-- =============================================================================
-- Submit (DRAFT → SUBMITTED). Triggers populate submitted_at automatically.
UPDATE applications SET status = 'SUBMITTED' WHERE application_id = 7;
-- Assign to review panel
UPDATE applications SET status = 'UNDER_REVIEW' WHERE application_id = 7;
-- Upsert an evaluation (same evaluator re-scores)
INSERT INTO evaluations (application_id, evaluator_id, novelty_score, effectiveness_score, conclusion)
VALUES (7, 99, 32, 48, 'Đề nghị công nhận')
ON CONFLICT (application_id, evaluator_id)
DO UPDATE SET
novelty_score = EXCLUDED.novelty_score,
effectiveness_score = EXCLUDED.effectiveness_score,
conclusion = EXCLUDED.conclusion,
evaluated_at = NOW();
-- Update JSONB field: patch a single effectiveness sub-field
UPDATE applications
SET effectiveness = effectiveness || jsonb_build_object(
'economic',
'Tiết kiệm ~30% thời gian xét duyệt'
)
WHERE application_id = 7;
-- Partial update (PATCH-style) — only update provided fields. The app layer
-- generates SET clauses from the non-null fields in the request body.
UPDATE applications
SET title = COALESCE($1, title),
purpose = COALESCE($2, purpose),
updated_at = NOW()
WHERE application_id = $3 AND deleted_at IS NULL
RETURNING *;
-- =============================================================================
-- DELETE: Soft delete + restore
-- =============================================================================
-- Soft delete
UPDATE applications SET deleted_at = NOW() WHERE application_id = 7;
-- Restore
UPDATE applications SET deleted_at = NULL WHERE application_id = 7;
-- Hard delete (only for drafts, cascades to authors/evaluations/etc.)
DELETE FROM applications
WHERE application_id = 7
AND status = 'DRAFT';
-- =============================================================================
-- ANALYTICS: Materialized-view refresh (run nightly via cron/pgAgent)
-- =============================================================================
REFRESH MATERIALIZED VIEW CONCURRENTLY mv_annual_stats;
-- Leaderboard: top-scoring approved innovations
SELECT code, title, avg_score
FROM v_application_summary
WHERE status = 'APPROVED'
ORDER BY avg_score DESC
LIMIT 10;
Posgresdb/schema.sql
+422
View File
@@ -0,0 +1,422 @@
-- =============================================================================
-- SÁNG KIẾN (INNOVATION APPLICATION) DATABASE SCHEMA
-- PostgreSQL 14+
--
-- Domain: Manage innovation applications at ĐHYD TP.HCM (Vietnamese medical
-- university). Supports the full lifecycle: draft → submit → evaluate → approve.
--
-- Design principles:
-- - 3NF for entities, JSONB for semi-structured/optional narrative
-- - Soft delete (deleted_at) — legal/audit requires historical retention
-- - State machine on applications.status enforced by trigger
-- - Full audit_log via trigger on all CUD operations
-- - Contribution % sums to 100 enforced by DEFERRABLE trigger
-- =============================================================================
CREATE EXTENSION IF NOT EXISTS pg_trgm; -- fuzzy matching
CREATE EXTENSION IF NOT EXISTS unaccent; -- Vietnamese diacritics in search
-- Convenience: updated_at auto-maintenance
CREATE OR REPLACE FUNCTION touch_updated_at() RETURNS TRIGGER AS $$
BEGIN NEW.updated_at := NOW(); RETURN NEW; END;
$$ LANGUAGE plpgsql;
-- =============================================================================
-- REFERENCE: UNITS (departments, faculties, centers)
-- =============================================================================
CREATE TABLE units (
unit_id SERIAL PRIMARY KEY,
code VARCHAR(32) UNIQUE NOT NULL,
name VARCHAR(255) NOT NULL, -- full Vietnamese name
parent_unit_id INT REFERENCES units(unit_id) ON DELETE SET NULL,
type VARCHAR(32) NOT NULL
CHECK (type IN ('TRUONG','KHOA','PHONG','BO_MON','TRUNG_TAM','KHAC')),
is_active BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TRIGGER trg_units_touch BEFORE UPDATE ON units
FOR EACH ROW EXECUTE FUNCTION touch_updated_at();
-- =============================================================================
-- USERS (unified: authors, evaluators, admins — a user can wear many hats)
-- =============================================================================
CREATE TABLE users (
user_id SERIAL PRIMARY KEY,
full_name VARCHAR(255) NOT NULL,
title VARCHAR(64), -- PGS.TS, TS., GS., CN., ThS.
date_of_birth DATE,
email VARCHAR(255) UNIQUE,
phone VARCHAR(32),
id_number VARCHAR(32) UNIQUE, -- CCCD / hộ chiếu
unit_id INT REFERENCES units(unit_id) ON DELETE SET NULL,
position VARCHAR(255), -- chức danh: Trưởng phòng, GV cao cấp
qualification VARCHAR(64), -- trình độ: Tiến sĩ, Thạc sĩ, Cử nhân
user_type VARCHAR(32) NOT NULL DEFAULT 'AUTHOR'
CHECK (user_type IN ('AUTHOR','COUNCIL','ADMIN','STUDENT','EXTERNAL')),
is_active BOOLEAN NOT NULL DEFAULT TRUE,
deleted_at TIMESTAMPTZ, -- soft delete
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_users_unit ON users(unit_id);
CREATE INDEX idx_users_active ON users(is_active) WHERE deleted_at IS NULL;
CREATE INDEX idx_users_name_trgm ON users USING GIN (full_name gin_trgm_ops);
CREATE TRIGGER trg_users_touch BEFORE UPDATE ON users
FOR EACH ROW EXECUTE FUNCTION touch_updated_at();
-- =============================================================================
-- APPLICATIONS (sáng kiến) — the core entity
-- =============================================================================
CREATE TABLE applications (
application_id SERIAL PRIMARY KEY,
code VARCHAR(32) UNIQUE NOT NULL, -- e.g., 'SK-2025-001'
title TEXT NOT NULL,
title_en TEXT,
registration_year INT NOT NULL CHECK (registration_year BETWEEN 2000 AND 2100),
field_of_application TEXT, -- lĩnh vực áp dụng
-- Workflow state (enforced by trigger below)
status VARCHAR(32) NOT NULL DEFAULT 'DRAFT'
CHECK (status IN (
'DRAFT','SUBMITTED','UNDER_REVIEW',
'EVALUATED','APPROVED','REJECTED','WITHDRAWN'
)),
-- Mẫu 01 narrative (long text)
introduction TEXT, -- 1. Mở đầu
current_state TEXT, -- 4.1 Tình trạng đã biết
purpose TEXT, -- Mục đích
implementation_steps TEXT, -- Các bước thực hiện
required_conditions TEXT, -- Điều kiện cần thiết
results_achieved TEXT, -- Kết quả thu được
novelty_description TEXT, -- Tính mới
confidential_info TEXT, -- Thông tin cần bảo mật
-- 10 effectiveness sub-fields (all optional narrative) → JSONB
effectiveness JSONB NOT NULL DEFAULT '{}'::jsonb,
-- Shape: { "economic":"...", "teaching":"...", "productivity":"...",
-- "work_efficiency":"...", "quality":"...", "cost_reduction":"...",
-- "environment":"...", "health":"...", "safety":"...", "awareness":"..." }
-- Mẫu 02 fields
owner_org VARCHAR(255), -- chủ đầu tư
first_applied_date DATE, -- ngày áp dụng lần đầu
content_summary TEXT, -- nội dung sáng kiến (short)
author_assessment TEXT, -- đánh giá theo tác giả
org_assessment TEXT, -- đánh giá theo tổ chức
-- Mẫu 02 classification (mutually exclusive in form, but stored as flags)
is_technical_solution BOOLEAN NOT NULL DEFAULT FALSE,
is_from_research_article BOOLEAN NOT NULL DEFAULT FALSE,
is_from_book_material BOOLEAN NOT NULL DEFAULT FALSE,
CONSTRAINT chk_exactly_one_classification CHECK (
status = 'DRAFT' OR
(is_technical_solution::int + is_from_research_article::int + is_from_book_material::int) = 1
),
-- Workflow timestamps
submitted_at TIMESTAMPTZ,
decided_at TIMESTAMPTZ,
primary_unit_id INT REFERENCES units(unit_id),
created_by INT REFERENCES users(user_id),
deleted_at TIMESTAMPTZ, -- soft delete
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_apps_status ON applications(status) WHERE deleted_at IS NULL;
CREATE INDEX idx_apps_year ON applications(registration_year);
CREATE INDEX idx_apps_unit ON applications(primary_unit_id);
CREATE INDEX idx_apps_title_trgm ON applications USING GIN (title gin_trgm_ops);
CREATE INDEX idx_apps_fts ON applications USING GIN (
to_tsvector('simple',
coalesce(title,'') || ' ' ||
coalesce(introduction,'') || ' ' ||
coalesce(novelty_description,'')
)
);
CREATE INDEX idx_apps_effectiveness ON applications USING GIN (effectiveness);
CREATE TRIGGER trg_apps_touch BEFORE UPDATE ON applications
FOR EACH ROW EXECUTE FUNCTION touch_updated_at();
-- =============================================================================
-- APPLICATION_AUTHORS (M:N with contribution %)
-- =============================================================================
CREATE TABLE application_authors (
application_id INT NOT NULL REFERENCES applications(application_id) ON DELETE CASCADE,
user_id INT NOT NULL REFERENCES users(user_id),
contribution_pct NUMERIC(5,2) NOT NULL CHECK (contribution_pct > 0 AND contribution_pct <= 100),
role VARCHAR(32) NOT NULL DEFAULT 'CO_AUTHOR'
CHECK (role IN ('PRIMARY','CO_AUTHOR')),
display_order INT NOT NULL DEFAULT 0,
PRIMARY KEY (application_id, user_id)
);
CREATE INDEX idx_app_authors_user ON application_authors(user_id);
-- At most one PRIMARY author per application
CREATE UNIQUE INDEX uq_primary_per_app
ON application_authors(application_id) WHERE role = 'PRIMARY';
-- Deferrable check: contribution % must total 100 per application
CREATE OR REPLACE FUNCTION check_contribution_total() RETURNS TRIGGER AS $$
DECLARE v_total NUMERIC; v_app INT;
BEGIN
v_app := COALESCE(NEW.application_id, OLD.application_id);
SELECT COALESCE(SUM(contribution_pct),0) INTO v_total
FROM application_authors WHERE application_id = v_app;
-- Only enforce when application has left DRAFT
IF (SELECT status FROM applications WHERE application_id = v_app) <> 'DRAFT'
AND v_total <> 100 THEN
RAISE EXCEPTION 'Contribution % for application % must sum to 100 (got %)',
'%', v_app, v_total;
END IF;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE CONSTRAINT TRIGGER trg_contribution_total
AFTER INSERT OR UPDATE OR DELETE ON application_authors
DEFERRABLE INITIALLY DEFERRED
FOR EACH ROW EXECUTE FUNCTION check_contribution_total();
-- =============================================================================
-- ORGS that tested / adopted the innovation (Mẫu 01 inner table)
-- =============================================================================
CREATE TABLE application_adopters (
adopter_id SERIAL PRIMARY KEY,
application_id INT NOT NULL REFERENCES applications(application_id) ON DELETE CASCADE,
display_order INT NOT NULL DEFAULT 0,
org_name VARCHAR(255) NOT NULL,
address TEXT,
field TEXT
);
CREATE INDEX idx_adopters_app ON application_adopters(application_id);
-- =============================================================================
-- PARTICIPANTS in first application (Mẫu 02 inner table)
-- =============================================================================
CREATE TABLE application_participants (
participant_id SERIAL PRIMARY KEY,
application_id INT NOT NULL REFERENCES applications(application_id) ON DELETE CASCADE,
user_id INT REFERENCES users(user_id), -- optional link
display_order INT NOT NULL DEFAULT 0,
full_name VARCHAR(255) NOT NULL,
date_of_birth DATE,
work_unit VARCHAR(255),
position VARCHAR(255),
qualification VARCHAR(64),
support_content TEXT
);
CREATE INDEX idx_participants_app ON application_participants(application_id);
-- =============================================================================
-- EVALUATIONS (Mẫu 04) — council members score applications
-- =============================================================================
CREATE TABLE evaluations (
evaluation_id SERIAL PRIMARY KEY,
application_id INT NOT NULL REFERENCES applications(application_id) ON DELETE CASCADE,
evaluator_id INT NOT NULL REFERENCES users(user_id),
novelty_comments TEXT,
novelty_score INT NOT NULL DEFAULT 0
CHECK (novelty_score BETWEEN 0 AND 40),
effectiveness_comments TEXT,
effectiveness_score INT NOT NULL DEFAULT 0
CHECK (effectiveness_score BETWEEN 0 AND 60),
total_score INT GENERATED ALWAYS AS (novelty_score + effectiveness_score) STORED,
conclusion TEXT,
evaluated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (application_id, evaluator_id)
);
CREATE INDEX idx_eval_app ON evaluations(application_id);
CREATE INDEX idx_eval_evaluator ON evaluations(evaluator_id);
-- =============================================================================
-- COMMITMENTS (Bản cam kết) — for paper-based innovations
-- =============================================================================
CREATE TABLE commitments (
commitment_id SERIAL PRIMARY KEY,
application_id INT NOT NULL REFERENCES applications(application_id) ON DELETE CASCADE,
user_id INT NOT NULL REFERENCES users(user_id),
paper_title TEXT,
role_type VARCHAR(32) NOT NULL
CHECK (role_type IN ('PRIMARY_AUTHOR','CO_AUTHOR')),
-- 5 commitment checkboxes
is_legal_owner BOOLEAN NOT NULL DEFAULT FALSE,
is_authorized_by_owner BOOLEAN NOT NULL DEFAULT FALSE,
has_coauthor_consent BOOLEAN NOT NULL DEFAULT FALSE,
not_predatory_journal BOOLEAN NOT NULL DEFAULT FALSE,
complies_with_ip_law BOOLEAN NOT NULL DEFAULT FALSE,
signed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE (application_id, user_id)
);
CREATE INDEX idx_commit_app ON commitments(application_id);
-- =============================================================================
-- ATTACHMENTS (uploaded files — figures, flowcharts, annexes)
-- =============================================================================
CREATE TABLE attachments (
attachment_id SERIAL PRIMARY KEY,
application_id INT NOT NULL REFERENCES applications(application_id) ON DELETE CASCADE,
file_name VARCHAR(255) NOT NULL,
file_path TEXT NOT NULL, -- S3/MinIO key
file_size BIGINT,
mime_type VARCHAR(128),
kind VARCHAR(32) -- 'LUU_DO', 'PHU_LUC', 'KY_SO', 'KHAC'
CHECK (kind IS NULL OR kind IN ('LUU_DO','PHU_LUC','KY_SO','KHAC')),
uploaded_by INT REFERENCES users(user_id),
uploaded_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_attach_app ON attachments(application_id);
-- =============================================================================
-- AUDIT LOG — single table, populated by triggers on all CUD operations
-- =============================================================================
CREATE TABLE audit_log (
log_id BIGSERIAL PRIMARY KEY,
table_name VARCHAR(64) NOT NULL,
record_id TEXT NOT NULL,
action VARCHAR(16) NOT NULL CHECK (action IN ('INSERT','UPDATE','DELETE')),
changed_by INT, -- set from app via SET LOCAL my.user_id
changed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
old_data JSONB,
new_data JSONB
);
CREATE INDEX idx_audit_table_record ON audit_log(table_name, record_id);
CREATE INDEX idx_audit_user_time ON audit_log(changed_by, changed_at DESC);
-- Generic audit trigger function
CREATE OR REPLACE FUNCTION audit_trigger() RETURNS TRIGGER AS $$
DECLARE
v_user INT;
v_pk TEXT;
BEGIN
-- Get user_id from session var if app sets it; else NULL
BEGIN v_user := current_setting('my.user_id')::INT;
EXCEPTION WHEN OTHERS THEN v_user := NULL; END;
v_pk := COALESCE(
(row_to_json(NEW)::jsonb->>TG_ARGV[0]),
(row_to_json(OLD)::jsonb->>TG_ARGV[0])
);
INSERT INTO audit_log(table_name, record_id, action, changed_by, old_data, new_data)
VALUES (
TG_TABLE_NAME,
v_pk,
TG_OP,
v_user,
CASE WHEN TG_OP IN ('UPDATE','DELETE') THEN to_jsonb(OLD) END,
CASE WHEN TG_OP IN ('INSERT','UPDATE') THEN to_jsonb(NEW) END
);
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
-- Attach audit trigger to the important tables (pass PK column name as arg)
CREATE TRIGGER trg_audit_applications AFTER INSERT OR UPDATE OR DELETE ON applications
FOR EACH ROW EXECUTE FUNCTION audit_trigger('application_id');
CREATE TRIGGER trg_audit_authors AFTER INSERT OR UPDATE OR DELETE ON application_authors
FOR EACH ROW EXECUTE FUNCTION audit_trigger('application_id');
CREATE TRIGGER trg_audit_evaluations AFTER INSERT OR UPDATE OR DELETE ON evaluations
FOR EACH ROW EXECUTE FUNCTION audit_trigger('evaluation_id');
CREATE TRIGGER trg_audit_commitments AFTER INSERT OR UPDATE OR DELETE ON commitments
FOR EACH ROW EXECUTE FUNCTION audit_trigger('commitment_id');
-- =============================================================================
-- WORKFLOW STATE MACHINE ENFORCEMENT
-- =============================================================================
CREATE OR REPLACE FUNCTION enforce_application_transitions() RETURNS TRIGGER AS $$
DECLARE
allowed BOOLEAN := FALSE;
BEGIN
IF OLD.status = NEW.status THEN RETURN NEW; END IF;
-- Allowed transitions
allowed := CASE
WHEN OLD.status = 'DRAFT' AND NEW.status IN ('SUBMITTED','WITHDRAWN') THEN TRUE
WHEN OLD.status = 'SUBMITTED' AND NEW.status IN ('UNDER_REVIEW','WITHDRAWN','DRAFT') THEN TRUE
WHEN OLD.status = 'UNDER_REVIEW' AND NEW.status IN ('EVALUATED','WITHDRAWN') THEN TRUE
WHEN OLD.status = 'EVALUATED' AND NEW.status IN ('APPROVED','REJECTED') THEN TRUE
ELSE FALSE
END;
IF NOT allowed THEN
RAISE EXCEPTION 'Invalid status transition: % → %', OLD.status, NEW.status;
END IF;
-- Auto-set timestamps
IF NEW.status = 'SUBMITTED' AND OLD.status = 'DRAFT' THEN
NEW.submitted_at := NOW();
END IF;
IF NEW.status IN ('APPROVED','REJECTED') THEN
NEW.decided_at := NOW();
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER trg_app_state_machine
BEFORE UPDATE OF status ON applications
FOR EACH ROW EXECUTE FUNCTION enforce_application_transitions();
-- =============================================================================
-- CONVENIENCE VIEWS
-- =============================================================================
-- Dashboard: applications with author names and current evaluation average
CREATE VIEW v_application_summary AS
SELECT
a.application_id,
a.code,
a.title,
a.status,
a.registration_year,
u.name AS primary_unit_name,
(SELECT string_agg(usr.full_name, ', ' ORDER BY aa.display_order)
FROM application_authors aa
JOIN users usr ON usr.user_id = aa.user_id
WHERE aa.application_id = a.application_id) AS author_names,
(SELECT ROUND(AVG(total_score),2)
FROM evaluations WHERE application_id = a.application_id) AS avg_score,
(SELECT COUNT(*) FROM evaluations WHERE application_id = a.application_id) AS num_evaluations,
a.submitted_at,
a.decided_at
FROM applications a
LEFT JOIN units u ON u.unit_id = a.primary_unit_id
WHERE a.deleted_at IS NULL;
-- Materialized view: annual approval statistics (refresh nightly)
CREATE MATERIALIZED VIEW mv_annual_stats AS
SELECT
registration_year,
COUNT(*) FILTER (WHERE status = 'APPROVED') AS approved,
COUNT(*) FILTER (WHERE status = 'REJECTED') AS rejected,
COUNT(*) FILTER (WHERE status NOT IN ('APPROVED','REJECTED')) AS pending,
COUNT(*) AS total
FROM applications
WHERE deleted_at IS NULL
GROUP BY registration_year;
CREATE UNIQUE INDEX ON mv_annual_stats(registration_year);
Posgresdb/test_schema.sql
+83
View File
@@ -0,0 +1,83 @@
-- Validation tests: run in a single transaction per block
-- ===========================================================
-- 1. SEED: units + users
INSERT INTO units(code, name, type) VALUES
('DHYD', 'Đại học Y Dược TP.HCM', 'TRUONG'),
('KHCN', 'Phòng Khoa học Công nghệ', 'PHONG');
INSERT INTO users(full_name, title, email, id_number, unit_id, qualification, user_type) VALUES
('Trần Hùng', 'PGS.TS', 'tranhung@ump.edu.vn', '001001', 1, 'Tiến sĩ', 'AUTHOR'),
('Đỗ Quốc Vũ', 'CN.', 'doquocvu@ump.edu.vn', '001002', 2, 'Cử nhân', 'AUTHOR'),
('Nguyễn Hội đồng A', 'PGS.TS', 'hdA@ump.edu.vn', '002001', 1, 'Tiến sĩ', 'COUNCIL');
-- 2. CREATE an application in DRAFT state
INSERT INTO applications(code, title, registration_year, status, purpose, primary_unit_id, created_by)
VALUES ('SK-2025-001',
'Quy trình xét duyệt Đạo đức trong nghiên cứu trên động vật',
2025, 'DRAFT',
'Chuẩn hoá quy trình xét duyệt hồ sơ',
2, 2);
-- 3. ADD authors with DEFERRED constraint (sums to 100 at COMMIT)
BEGIN;
INSERT INTO application_authors(application_id, user_id, contribution_pct, role) VALUES
(1, 1, 50, 'CO_AUTHOR'),
(1, 2, 50, 'PRIMARY');
-- At this point sum=100, but app is DRAFT so constraint doesn't even care yet
COMMIT;
-- Verify
SELECT 'Authors inserted:' AS step, count(*) FROM application_authors;
-- 4. TRY to submit the application (DRAFT → SUBMITTED): needs classification
-- This should FAIL the check constraint because no classification flag is set
\echo 'Test 4: should FAIL (missing classification)'
UPDATE applications SET status='SUBMITTED' WHERE application_id=1;
\echo ''
-- Fix and retry
UPDATE applications
SET is_technical_solution = TRUE,
status = 'SUBMITTED'
WHERE application_id = 1;
SELECT 'After submit:' AS step, status, submitted_at FROM applications WHERE application_id=1;
-- 5. TRY invalid transition SUBMITTED → APPROVED (should FAIL)
\echo 'Test 5: should FAIL (illegal transition)'
UPDATE applications SET status='APPROVED' WHERE application_id=1;
\echo ''
-- Valid transitions
UPDATE applications SET status='UNDER_REVIEW' WHERE application_id=1;
-- 6. EVALUATOR scores the application
INSERT INTO evaluations(application_id, evaluator_id, novelty_score, effectiveness_score, conclusion)
VALUES (1, 3, 35, 50, 'Đề xuất công nhận');
SELECT 'Evaluation:' AS step, novelty_score, effectiveness_score, total_score FROM evaluations;
-- 7. Move to EVALUATED → APPROVED
UPDATE applications SET status='EVALUATED' WHERE application_id=1;
UPDATE applications SET status='APPROVED' WHERE application_id=1;
SELECT 'Final status:' AS step, status, decided_at IS NOT NULL AS has_decision_time
FROM applications WHERE application_id=1;
-- 8. READ: summary view
SELECT code, title, status, author_names, avg_score, num_evaluations
FROM v_application_summary;
-- 9. AUDIT trail: who changed what?
SELECT table_name, action, changed_at,
(new_data->>'status') AS new_status
FROM audit_log
WHERE table_name = 'applications'
ORDER BY log_id;
-- 10. Bad contribution sum should fail at COMMIT
\echo 'Test 10: should FAIL (sum != 100 on submitted app)'
BEGIN;
UPDATE application_authors SET contribution_pct = 30 WHERE application_id=1 AND user_id=1;
-- sum is now 30+50=80, but app is APPROVED so trigger will reject at commit
COMMIT;
README.md
+254
View File
@@ -0,0 +1,254 @@
Initiative Management System
The platform consists of two main services:
- **Frontend**: React-based web application with TypeScript and Vite
- **Backend**: FastAPI-based REST API with Python 3.11
- **AI Integration**: Ollama-powered document analysis and compliance checking
## Project Structure
```
poc/
├── fe0/ # Frontend service
│ ├── src/ # React application source
│ ├── public/ # Static assets
│ ├── package.json # Node.js dependencies
│ └── Dockerfile # Frontend container
├── be0/ # Backend service
│ ├── src/ # Python application source
│ ├── main.py # FastAPI application entry point
│ ├── requirements.txt # Python dependencies
│ └── Dockerfile # Backend container
├── assets/ # Shared resources and data
└── docker-compose.yml # Service orchestration
```
## Prerequisites
- Docker 20.10+
- Docker Compose 2.0+
- Git
## Quick Start
1. **Clone and setup**
```bash
git clone <repository-url>
cd poc
```
2. **Start all services**
```bash
docker-compose up --build
```
3. **Access the application**
- **Frontend**: http://localhost:8081
- **Backend API**: http://localhost:4402
- **API Documentation**: http://localhost:4402/docs
## Development Setup
### Frontend Development
```bash
cd fe0
npm install
npm run dev
```
**Available Scripts:**
- `npm run dev` - Start development server
- `npm run build` - Build for production
- `npm run preview` - Preview production build
- `npm run lint` - Run ESLint
**Technology Stack:**
- React 18 with TypeScript
- Vite for build tooling
- Tailwind CSS for styling
- shadcn/ui component library
- React Router for navigation
- TanStack Query for state management
### Backend Development
```bash
cd be0
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 4402 --reload
```
**Technology Stack:**
- FastAPI framework
- Python 3.11
- Pydantic for data validation
- LangChain for AI workflows
- Ollama for local AI models
- PDF processing with PyPDF and Docling
## API Documentation
### Core Endpoints
#### Workflow Management
- `POST /workflows` - Initialize new compliance workflow
- `GET /workflows/{workflow_id}` - Retrieve workflow status
- `PUT /workflows/{workflow_id}/items` - Update workflow items
- `POST /workflows/{workflow_id}/approvals` - Submit approvals
- `GET /workflows/{workflow_id}/report` - Generate status reports
- `POST /workflows/{workflow_id}/advance` - Progress to next phase
#### Document Processing
- `POST /upload_document` - Upload and parse documents
- `POST /get_page` - Retrieve specific document pages
- `POST /test_ollama` - Test AI model connectivity
#### System Health
- `GET /health` - Service health check
- `GET /` - API information and available endpoints
### Request/Response Examples
**Create Workflow:**
```json
POST /workflows
{
"project_name": "ISO 27001 Implementation",
"project_description": "Implement ISO 27001 controls",
"records_officer_email": "officer@company.com"
}
```
**Update Workflow Item:**
```json
PUT /workflows/{workflow_id}/items
{
"item_id": 1,
"status": "completed",
"comment": "Implementation completed",
"updated_by": "john.doe@company.com"
}
```
## Configuration
### Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `GENERIC_TIMEZONE`` | Application timezone | `UTC` |
| `NVIDIA_VISIBLE_DEVICES` | GPU access for AI models | `all` |
| `NVIDIA_DRIVER_CAPABILITIES` | GPU capabilities | `compute,utility` |
### Docker Network Configuration
Services communicate via a custom Docker network (`profyt-net`) with static IP addressing:
- Frontend: `192.168.42.20`
- Backend: `192.168.42.22`
## Features
### Compliance Management
- **ISO 27001** compliance tracking and reporting
- **Records Management** integration workflows
- **Risk Assessment** tools and dashboards
- **Document Processing** with AI-powered analysis
### Workflow Engine
- Multi-phase compliance workflows
- Approval management system
- Progress tracking and reporting
- Integration with external systems
### AI-Powered Analysis
- Document parsing and content extraction
- Compliance gap analysis
- Automated report generation
- Natural language processing for policy analysis
## Deployment
### Production Deployment
On the **application host** (SSH), from the repository root:
1. **Secrets & config**
```bash
cp .env.example .env
# Edit .env: PUBLIC_HOST, ports, MinIO and Postgres credentials (openssl rand -base64 32).
# Never commit `.env`. Postgres user/password apply only on FIRST empty DB volume — see `.env.example`.
./scripts/verify-prod-env.sh
```
2. **Deploy (pull, build, recreate containers)**
```bash
./scripts/deploy-prod.sh
# Air-gapped / no registry pull:
# ./scripts/deploy-prod.sh --no-pull
```
Or manually (must pass `/.env` explicitly if it is not named `.env` next to the compose file):
```bash
docker compose --env-file .env -f docker-compose.prod.yml pull
docker compose --env-file .env -f docker-compose.prod.yml up -d --build --remove-orphans
```
3. **Smoke checks** (`FE_PORT` and API port come from `.env` / compose; API is `127.0.0.1:4402` in prod compose)
```bash
# Replace 8081 with the FE_PORT value in .env when different.
curl -sf http://127.0.0.1:8081/
curl -sf http://127.0.0.1:4402/health
```
### Scaling Considerations
- **Frontend**: Stateless, horizontally scalable
- **Backend**: Consider database persistence for production
- **AI Models**: GPU requirements for optimal performance
- **Storage**: Implement proper file storage for documents
## Monitoring and Logging
### Application Logs
- Frontend logs: Available via Docker logs
- Backend logs: Stored in `be0/logs/` directory
- System logs: `docker-compose logs [service-name]`
### Health Monitoring
- Health check endpoints available
- Docker health checks configured
- Log aggregation recommended for production
## Security Considerations
### Current Implementation
- CORS enabled for cross-origin requests
- Input validation via Pydantic models
- File upload restrictions
### Production Recommendations
- Implement authentication/authorization
- Add rate limiting
- Enable HTTPS/TLS
- Implement proper secret management
- Add audit logging
## Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
### Development Guidelines
- Follow TypeScript best practices
- Write comprehensive tests
- Update documentation for new features
- Follow conventional commit messages
## License
This project is licensed under the terms specified in the LICENSE file.
be0/.env.example
+29
View File
@@ -0,0 +1,29 @@
# Copy to .env and adjust. docker-compose sets these for the be0 service when using the repo stack.
INITIATIVE_DATABASE_URL=postgresql+asyncpg://initiative:initiative_secret@localhost:15432/initiatives
# S3 / MinIO — server-to-server (API → object store)
S3_ENDPOINT_URL=http://localhost:19000
S3_ACCESS_KEY=minio_user
S3_SECRET_KEY=minio_password
S3_BUCKET_ATTACHMENTS=initiative-attachments
S3_BUCKET_EXPORTS=initiative-exports
S3_BUCKET_QUARANTINE=initiative-quarantine
# Optional: HTTPS base for presigned URLs (must match public MinIO TLS host; see docs/minio-behind-https.md)
# S3_PUBLIC_ENDPOINT_URL=https://minio-api.example.com
# Optional: comma-separated extra browser origins for CORS (merged with localhost defaults in main.py).
# In Docker dev stack, docker-compose.yml can set this; production compose adds your public UI URL automatically.
# CORS_ORIGINS=http://YOUR_LAN_IP:8081
# Local Python runs may load this file; Docker Compose uses the repo-root `.env` for ${SMTP_*} → be0.
# Password reset email (same SMTP block as `.env.example` beside docker-compose for dev stack.)
# OTP + reset use src/auth_mail.py: set SMTP_* for Option A or AUTH_MAIL_LOG_ONLY=1 locally.
# AUTH_MAIL_LOG_ONLY=1
# AUTH_PUBLIC_WEB_ORIGIN=http://localhost:8081
# SMTP_HOST=smtp.example.com
# SMTP_PORT=587
# SMTP_USER=
# SMTP_PASSWORD=
# AUTH_MAIL_FROM=noreply@example.com
# SMTP_USE_TLS=1
be0/CHAT_ASSISTANT_README.md
+223
View File
@@ -0,0 +1,223 @@
# Chat Assistant Module
## Overview
The Chat Assistant module provides a conversational AI interface for answering policy and compliance questions using Ollama.
## Architecture
### Backend (`be0/src/chat_assistant.py`)
The `ChatAssistant` class provides:
- **Chat functionality**: Conversational AI for policy questions
- **Content verification**: Verify content against compliance requirements
- **Policy Q&A**: Answer questions about policies and compliance
### Frontend (`fe0/src/features/chat/`)
The frontend chat feature includes:
- **Service layer**: API communication with backend
- **React hooks**: Easy-to-use hooks for chat functionality
- **Type definitions**: TypeScript types for type safety
## API Endpoints
### 1. Chat Endpoint
```
POST /api/v1/chat
```
**Request Body:**
```json
{
"message": "What are ISO 27001 requirements?",
"conversation_history": [
{
"role": "user",
"content": "Previous message"
},
{
"role": "assistant",
"content": "Previous response"
}
],
"context": "Optional context about policies"
}
```
**Response:**
```json
{
"message": "ISO 27001 is an information security management system...",
"model": "gemma3:27b",
"tokens_used": 150
}
```
### 2. Verify Content Endpoint
```
POST /api/v1/chat/verify
```
**Form Data:**
- `field_name`: Name of the field being verified
- `content`: Content to verify
- `verification_criteria`: (Optional) Specific criteria to check
**Response:**
```json
{
"message": "The content meets compliance requirements...",
"model": "gemma3:27b",
"tokens_used": 200
}
```
### 3. Policy Question Endpoint
```
POST /api/v1/chat/question
```
**Form Data:**
- `question`: The user's question
- `policy_context`: (Optional) Context about specific policies
**Response:**
```json
{
"message": "Answer to the policy question...",
"model": "gemma3:27b",
"tokens_used": 180
}
```
## Features
### 1. Conversational Context
- Maintains conversation history for context-aware responses
- Keeps last 10 messages for context
- System prompt guides the assistant's behavior
### 2. Policy Expertise
- Specialized in IT governance and compliance
- Knowledgeable about ISO 27001, NIST, GDPR, etc.
- Provides accurate, actionable advice
### 3. Content Verification
- Analyzes content against compliance requirements
- Provides detailed feedback
- Suggests improvements
## Usage
### Backend
```python
from src.chat_assistant import get_chat_assistant
# Get chat assistant instance
assistant = get_chat_assistant()
# Chat
request = ChatRequest(
message="What is ISO 27001?",
context="IT governance"
)
response = await assistant.chat(request)
# Verify content
response = await assistant.verify_content(
field_name="Project Description",
content="Our project implements security controls..."
)
```
### Frontend
```typescript
import { useChat } from '@/features/chat/hooks/useChat';
const { sendMessage, verifyContent, isLoading } = useChat();
// Send a message
const response = await sendMessage(
"What are compliance requirements?",
conversationHistory, // Optional
"ISO 27001 context" // Optional
);
// Verify content
const verification = await verifyContent(
"Project Name",
"Project content to verify"
);
```
## Configuration
### Model Selection
The default model is `gemma3:27b`. To change it:
```python
# In chat_assistant.py
assistant = ChatAssistant(model_name="your-model-name")
```
### System Prompt
The system prompt can be customized in the `ChatAssistant.__init__` method to change the assistant's behavior and expertise.
## Logging
All chat interactions are logged to:
- `be0/logs/ChatAssistant.log`
This helps with debugging and monitoring.
## Error Handling
The module includes comprehensive error handling:
- Catches and logs all exceptions
- Returns user-friendly error messages
- Raises HTTPException for API errors
## Testing
To test the chat assistant:
1. **Start the backend:**
```bash
cd be0
docker-compose up be0
```
2. **Test via API:**
```bash
curl -X POST http://localhost:4402/api/v1/chat \
-H "Content-Type: application/json" \
-d '{"message": "What is ISO 27001?"}'
```
3. **Test via Frontend:**
- Open the Dashboard
- Use the ChatAssistant component
- Ask questions or verify content
## Integration
The ChatAssistant is integrated with:
- **ChatAssistant.tsx**: React component in the Dashboard
- **useChat hook**: React hook for chat functionality
- **chatService**: API service layer
## Future Enhancements
Potential improvements:
1. Streaming responses for real-time text generation
2. Multi-turn conversation management
3. Document context injection
4. Voice input/output
5. Response rating and feedback
6. Conversation export
7. Custom model fine-tuning
be0/Dockerfile
+34
View File
@@ -0,0 +1,34 @@
FROM python:3.11
# Set the working directory
WORKDIR /app
# Copy the requirements file
COPY ./requirements.txt /app/
# Install dependencies and set up Python environment
RUN apt-get update && apt-get install -y --no-install-recommends \
zstd \
curl \
git \
build-essential \
python3-pip \
libreoffice-writer-nogui \
&& rm -rf /var/lib/apt/lists/*
# RUN curl -fsSL https://ollama.com/install.sh | sh
RUN pip install --upgrade pip
WORKDIR /app
RUN pip install --no-cache-dir -r requirements.txt
RUN pip install nltk
# Avoid runtime GitHub downloads (slow/hanging in some networks) before Uvicorn starts.
RUN python3 -m nltk.downloader punkt punkt_tab stopwords averaged_perceptron_tagger_eng wordnet
COPY . /app/
EXPOSE 4402
ENTRYPOINT ["/app/entrypoint.sh"]
be0/GOVERNANCE_LAYER_STATUS.md
+172
View File
@@ -0,0 +1,172 @@
# Governance Layer Status in be0
## Current State
### ✅ What EXISTS (Current Implementation)
The current `be0` codebase has:
1. **Basic Workflow System** (`src/domain/entities/workflow.py`, `src/application/services/workflow_service.py`)
- SDLC/RM Integration workflow
- Phase-based progression
- Task/checklist management
- **Location**: `be0/src/domain/entities/workflow.py`
2. **Compliance Verification** (`src/compliance_verifier.py`)
- Ollama-based compliance checking
- Text generation and similarity analysis
- **Location**: `be0/src/compliance_verifier.py`
3. **Chat Assistant** (`src/chat_assistant.py`)
- Policy Q&A functionality
- Content verification
- **Location**: `be0/src/chat_assistant.py`
4. **Architecture Foundation**
- Domain/Application/Infrastructure layers
- Repository pattern
- API routes structure
- **Location**: `be0/src/domain/`, `be0/src/application/`, `be0/src/api/`
---
## ❌ What's MISSING (Governance Layer for Initiatives)
The **Grassroots Initiative Recognition System** governance layer has **NOT been implemented yet**.
### Missing Components:
#### 1. **Initiative Management**
- ❌ Initiative entity (initiative_id, group_type, status, etc.)
- ❌ Author management (contribution percentages, lead author logic)
- ❌ Unit/Appraisal Team entities
- **Should be in**: `be0/src/domain/entities/initiative.py`
#### 2. **Business Rules Engine**
- ❌ Novelty checker (duplicate detection)
- ❌ Scoring algorithm (Group 01 dual/triple reviewer)
- ❌ Auto-classification (Group 02)
- ❌ Author contribution validator
- **Should be in**: `be0/src/domain/rules/` or `be0/src/application/rules/`
#### 3. **Workflow State Machine**
- ❌ Initiative state transitions (DRAFT → SUBMITTED → UNIT_REVIEW → etc.)
- ❌ Deadline enforcement
- ❌ SLA tracking
- **Should be in**: `be0/src/application/state_machine.py` or `be0/src/domain/workflows/initiative_workflow.py`
#### 4. **Review Management**
- ❌ Review assignment logic
- ❌ Blind review enforcement
- ❌ Score conflict detection
- ❌ Reviewer assignment service
- **Should be in**: `be0/src/application/services/review_service.py`
#### 5. **Document Management**
- ❌ Form templates (Form 01, 03, 05, 06)
- ❌ Document versioning
- ❌ File storage integration
- **Should be in**: `be0/src/infrastructure/storage/`
#### 6. **API Endpoints**
- ❌ `/api/v1/initiatives` (CRUD)
- ❌ `/api/v1/initiatives/{id}/submit`
- ❌ `/api/v1/initiatives/{id}/reviews`
- ❌ `/api/v1/reviews/{review_id}/score`
- ❌ `/api/v1/initiatives/{id}/appeal`
- **Should be in**: `be0/src/api/routes/initiatives.py`
---
## Recommended Structure for Governance Layer
```
be0/src/
├── domain/
│ ├── entities/
│ │ ├── initiative.py # ❌ MISSING
│ │ ├── author.py # ❌ MISSING
│ │ ├── review.py # ❌ MISSING
│ │ ├── unit.py # ❌ MISSING
│ │ └── appraisal_team.py # ❌ MISSING
│ ├── rules/
│ │ ├── novelty_checker.py # ❌ MISSING
│ │ ├── scoring_engine.py # ❌ MISSING
│ │ ├── duplicate_detector.py # ❌ MISSING
│ │ └── classification_engine.py # ❌ MISSING
│ └── workflows/
│ └── initiative_workflow.py # ❌ MISSING
├── application/
│ ├── services/
│ │ ├── initiative_service.py # ❌ MISSING
│ │ ├── review_service.py # ❌ MISSING
│ │ ├── notification_service.py # ❌ MISSING
│ │ └── deadline_service.py # ❌ MISSING
│ └── state_machine.py # ❌ MISSING
├── infrastructure/
│ ├── storage/
│ │ └── file_storage.py # ❌ MISSING
│ └── database/
│ └── models.py # ❌ MISSING (SQLAlchemy models)
└── api/
└── routes/
├── initiatives.py # ❌ MISSING
├── reviews.py # ❌ MISSING
└── reports.py # ❌ MISSING
```
---
## What to Build Next
Based on the simplified tech stack we discussed, here's the implementation order:
### Phase 1: Core Entities & Database
1. Create database models (PostgreSQL)
2. Create domain entities (Initiative, Author, Review, etc.)
3. Create repository interfaces
### Phase 2: Business Rules
1. Novelty checker (using PostgreSQL pg_trgm)
2. Scoring engine
3. Auto-classification logic
### Phase 3: Workflow
1. State machine implementation
2. Transition rules
3. Deadline tracking
### Phase 4: API & Services
1. Initiative service
2. Review service
3. API endpoints
4. Document upload
---
## Current vs. Required
| Component | Current | Required | Status |
|-----------|---------|----------|--------|
| Workflow (SDLC) | ✅ | ✅ | Implemented |
| Initiative Management | ❌ | ✅ | **Missing** |
| Business Rules | ❌ | ✅ | **Missing** |
| Review System | ❌ | ✅ | **Missing** |
| State Machine | ❌ | ✅ | **Missing** |
| Document Storage | ❌ | ✅ | **Missing** |
| Scoring Engine | ❌ | ✅ | **Missing** |
---
## Next Steps
To implement the governance layer:
1. **Start with database schema** - Create PostgreSQL tables for initiatives, authors, reviews
2. **Create domain entities** - Python classes for Initiative, Author, Review
3. **Implement business rules** - Novelty checker, scoring engine
4. **Build state machine** - Workflow transitions
5. **Create API endpoints** - RESTful APIs for frontend
6. **Add document storage** - Local filesystem integration
The foundation (layered architecture, FastAPI, PostgreSQL) is already in place - you just need to build the governance-specific components on top of it.
be0/TROUBLESHOOTING_CHAT.md
+150
View File
@@ -0,0 +1,150 @@
# Chat Assistant Troubleshooting Guide
## Common Errors and Solutions
### Error: 500 Internal Server Error
This usually indicates one of the following issues:
#### 1. Ollama Not Running
**Symptoms:**
- 500 error on `/api/v1/chat`
- Error message mentions "connection" or "refused"
**Solution:**
```bash
# Check if Ollama is running in the container
docker exec be0 ps aux | grep ollama
# If not running, restart the container
docker-compose restart be0
# Or start Ollama manually
docker exec be0 ollama serve &
```
#### 2. Model Not Available
**Symptoms:**
- Error mentions "model not found"
- Model name mismatch
**Solution:**
```bash
# Check available models
docker exec be0 ollama list
# Pull the required model
docker exec be0 ollama pull gemma3:270M
# Verify model is available
docker exec be0 ollama list | grep gemma3
```
#### 3. Model Name Mismatch
**Issue:** Code uses `gemma3:27b` but entrypoint pulls `gemma3:270M`
**Solution:**
The code has been updated to use `gemma3:270M` to match the entrypoint script.
#### 4. Network Connectivity
**Symptoms:**
- Connection refused errors
- Timeout errors
**Solution:**
```bash
# Check if Ollama is accessible from within the container
docker exec be0 curl http://localhost:11434/api/tags
# Check Ollama service status
docker exec be0 ollama list
```
## Diagnostic Endpoints
### Health Check
```bash
curl http://localhost:4402/health
```
This will show:
- Overall service status
- Ollama connection status
- Available models
### Test Ollama Directly
```bash
# From inside the container
docker exec be0 ollama run gemma3:270M "Hello"
```
## Debugging Steps
1. **Check Backend Logs:**
```bash
docker-compose logs be0 | tail -50
```
2. **Check Chat Assistant Logs:**
```bash
tail -f be0/logs/ChatAssistant.log
```
3. **Test API Endpoint:**
```bash
curl -X POST http://localhost:4402/api/v1/chat \
-H "Content-Type: application/json" \
-d '{"message": "Hello"}'
```
4. **Verify Ollama Service:**
```bash
docker exec be0 ollama list
docker exec be0 curl http://localhost:11434/api/tags
```
## Common Fixes
### Fix 1: Restart Ollama Service
```bash
docker exec be0 pkill ollama
docker exec be0 ollama serve &
sleep 2
docker exec be0 ollama list
```
### Fix 2: Pull Missing Model
```bash
docker exec be0 ollama pull gemma3:270M
```
### Fix 3: Restart Container
```bash
docker-compose restart be0
```
### Fix 4: Rebuild Container
```bash
docker-compose down
docker-compose build be0
docker-compose up be0
```
## Expected Behavior
When working correctly:
1. Health endpoint shows Ollama as "connected"
2. Available models list includes `gemma3:270M`
3. Chat endpoint returns 200 with a response
4. Logs show successful message processing
## Still Having Issues?
1. Check the full error in logs: `docker-compose logs be0`
2. Verify Ollama is running: `docker exec be0 ps aux | grep ollama`
3. Test Ollama directly: `docker exec be0 ollama run gemma3:270M "test"`
4. Check model availability: `docker exec be0 ollama list`
be0/__init__.py
View File
be0/__pycache__/__init__.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/__pycache__/main.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/__pycache__/main.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/assets/data/pdf/UNObank_IT Governance and Risk Management Policy v1.5.pdf
BIN
View File
Binary file not shown.
be0/entrypoint.sh
Executable
+46
View File
@@ -0,0 +1,46 @@
#!/bin/bash
if command -v ollama >/dev/null 2>&1; then
echo "Starting Ollama server..."
ollama serve &
sleep 1
else
echo "Ollama not installed in this image; skipping."
fi
# if ! ollama list | grep -q "qwen2.5:3b"; then
# echo "Model qwen2.5:3b not found. Pulling..."
# ollama pull qwen2.5:3b
# else
# echo "Model qwen2.5:3b already exists. Skipping pull."
# fi
# #download embedding model
# if ! ollama list | grep -q "embeddinggemma:300m"; then
# echo "Model embeddinggemma:300m not found. Pulling..."
# ollama pull embeddinggemma:300m
# else
# echo "Model embeddinggemma:300m already exists. Skipping pull."
# fi
# NLTK corpora are installed when the image is built (see Dockerfile).
# Bind mount overwrites /app; image site-packages may be stale vs mounted requirements.txt.
if [ -f /app/requirements.txt ]; then
echo "Installing/updating Python deps from mounted /app/requirements.txt..."
pip install --no-cache-dir -r /app/requirements.txt || {
echo "ERROR: pip install -r /app/requirements.txt failed; fix deps and restart be0."
exit 1
}
fi
echo "Applying idempotent initiative DB migrations (008–014 incl. registration_otp_codes) if needed..."
python /app/scripts/apply_initiative_migrations.py || echo "WARNING: apply_initiative_migrations exited non-zero — check be0 logs (API may return 503 for evidence/artifacts until DB is fixed)."
echo "Starting FastAPI..."
if [ "${UVICORN_RELOAD:-0}" = "1" ]; then
exec uvicorn main:app --host 0.0.0.0 --port 4402 --reload
else
exec uvicorn main:app --host 0.0.0.0 --port 4402
fi
be0/main.py
+3726
View File
File diff suppressed because it is too large Load Diff
be0/migrations/001_initiative_schema.sql
+251
View File
@@ -0,0 +1,251 @@
-- Initiative Recognition System — PostgreSQL schema (architecture_plan.md §4)
-- Table order respects FKs (units before users).
CREATE EXTENSION IF NOT EXISTS citext;
-- ========= ENUMS =========
DO $$ BEGIN
CREATE TYPE user_role AS ENUM ('applicant','council_member','editor','admin','viewer');
EXCEPTION WHEN duplicate_object THEN NULL;
END $$;
DO $$ BEGIN
CREATE TYPE initiative_class AS ENUM ('technical','research','textbook');
EXCEPTION WHEN duplicate_object THEN NULL;
END $$;
DO $$ BEGIN
CREATE TYPE research_evidence AS ENUM ('international','domestic','poster');
EXCEPTION WHEN duplicate_object THEN NULL;
END $$;
DO $$ BEGIN
CREATE TYPE eval_level AS ENUM ('high','medium','low');
EXCEPTION WHEN duplicate_object THEN NULL;
END $$;
DO $$ BEGIN
CREATE TYPE submission_status AS ENUM ('draft','submitted','under_review','approved','rejected');
EXCEPTION WHEN duplicate_object THEN NULL;
END $$;
DO $$ BEGIN
CREATE TYPE recognition_tier AS ENUM ('excellent','good');
EXCEPTION WHEN duplicate_object THEN NULL;
END $$;
-- ========= IDENTITY =========
CREATE TABLE IF NOT EXISTS units (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
parent_id UUID REFERENCES units(id),
address TEXT
);
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
email CITEXT UNIQUE NOT NULL,
password_hash TEXT NOT NULL,
full_name TEXT NOT NULL,
phone TEXT,
unit_id UUID REFERENCES units(id),
is_active BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS user_roles (
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role user_role NOT NULL,
PRIMARY KEY (user_id, role)
);
-- System user for anonymous draft saves (no login yet)
INSERT INTO users (id, email, password_hash, full_name)
VALUES (
'00000000-0000-4000-8000-000000000001',
'system@draft.local',
'-',
'System (draft owner)'
)
ON CONFLICT (email) DO NOTHING;
-- ========= CASE / INITIATIVE ROOT =========
CREATE TABLE IF NOT EXISTS initiatives (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
case_code TEXT UNIQUE NOT NULL,
owner_id UUID NOT NULL REFERENCES users(id),
status submission_status NOT NULL DEFAULT 'draft',
recognition_tier recognition_tier,
submitted_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_initiatives_owner_status ON initiatives(owner_id, status);
-- ========= DRAFT SNAPSHOTS =========
CREATE TABLE IF NOT EXISTS drafts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
draft_code TEXT UNIQUE NOT NULL,
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
payload JSONB NOT NULL,
version INTEGER NOT NULL DEFAULT 1,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_drafts_initiative ON drafts(initiative_id);
-- ========= ĐƠN (APPLICATION) =========
CREATE TABLE IF NOT EXISTS applications (
initiative_id UUID PRIMARY KEY REFERENCES initiatives(id) ON DELETE CASCADE,
initiative_name TEXT NOT NULL,
investor_name TEXT,
application_field TEXT,
first_apply_date DATE,
initiative_classification initiative_class,
research_evidence_kind research_evidence,
international_journal_decl TEXT,
content_summary TEXT,
confidential_info TEXT,
conditions TEXT,
author_evaluation TEXT,
trial_evaluation TEXT,
submission_day SMALLINT,
submission_month SMALLINT,
submission_year SMALLINT,
honesty_confirmed BOOLEAN NOT NULL DEFAULT FALSE,
CONSTRAINT chk_first_apply_window
CHECK (first_apply_date IS NULL
OR first_apply_date BETWEEN DATE '2025-04-15' AND DATE '2026-04-15')
);
CREATE TABLE IF NOT EXISTS authors (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
user_id UUID REFERENCES users(id),
ordinal SMALLINT NOT NULL,
full_name TEXT NOT NULL,
dob DATE,
workplace TEXT,
title TEXT,
qualification TEXT,
contribution_percent NUMERIC(5,2) NOT NULL,
is_representative BOOLEAN NOT NULL DEFAULT FALSE,
CHECK (contribution_percent >= 0 AND contribution_percent <= 100)
);
CREATE UNIQUE INDEX IF NOT EXISTS uq_authors_repr ON authors(initiative_id) WHERE is_representative;
CREATE TABLE IF NOT EXISTS support_staff (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
full_name TEXT,
dob DATE,
workplace TEXT,
title TEXT,
qualification TEXT,
support_content TEXT
);
CREATE TABLE IF NOT EXISTS evidence_files (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
kind TEXT NOT NULL CHECK (kind IN ('textbook','research','technical')),
storage_uri TEXT NOT NULL,
original_name TEXT NOT NULL,
mime_type TEXT NOT NULL DEFAULT 'application/pdf',
byte_size BIGINT NOT NULL,
sha256 CHAR(64) NOT NULL,
uploaded_by UUID NOT NULL REFERENCES users(id),
uploaded_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX IF NOT EXISTS uq_evidence_kind ON evidence_files(initiative_id, kind);
-- ========= BÁO CÁO (REPORT) =========
CREATE TABLE IF NOT EXISTS reports (
initiative_id UUID PRIMARY KEY REFERENCES initiatives(id) ON DELETE CASCADE,
introduction TEXT,
representative_phone TEXT,
representative_email TEXT,
current_status TEXT,
purpose TEXT,
implementation_steps TEXT,
first_applied_unit TEXT,
achieved_result TEXT,
novelty TEXT,
effectiveness JSONB NOT NULL DEFAULT '{}'::jsonb,
submission_date DATE
);
CREATE TABLE IF NOT EXISTS trial_units (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
name TEXT NOT NULL,
address TEXT,
field TEXT,
ordinal SMALLINT
);
-- ========= CONTRIBUTION CONFIRMATION =========
CREATE TABLE IF NOT EXISTS contributions (
initiative_id UUID PRIMARY KEY REFERENCES initiatives(id) ON DELETE CASCADE,
main_author TEXT NOT NULL,
position TEXT,
representative_percent NUMERIC(5,2),
submission_date TIMESTAMPTZ,
digital_signature_confirmed BOOLEAN NOT NULL DEFAULT FALSE
);
CREATE TABLE IF NOT EXISTS contribution_participants (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
full_name TEXT,
work_unit TEXT,
contribution_percent NUMERIC(5,2)
);
-- ========= PHIẾU ĐÁNH GIÁ =========
CREATE TABLE IF NOT EXISTS evaluations (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
council_member_id UUID NOT NULL REFERENCES users(id),
position TEXT,
evaluation_date DATE NOT NULL,
novelty_level eval_level,
novelty_score SMALLINT,
novelty_comment TEXT,
effectiveness_level eval_level,
effectiveness_score SMALLINT,
effectiveness_comment TEXT,
total_score SMALLINT GENERATED ALWAYS AS
(COALESCE(novelty_score,0) + COALESCE(effectiveness_score,0)) STORED,
conclusion TEXT,
status submission_status NOT NULL DEFAULT 'draft',
submitted_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CHECK (novelty_score IS NULL OR (novelty_score BETWEEN 0 AND 40)),
CHECK (effectiveness_score IS NULL OR (effectiveness_score BETWEEN 0 AND 60)),
UNIQUE (initiative_id, council_member_id)
);
CREATE INDEX IF NOT EXISTS idx_eval_initiative ON evaluations(initiative_id);
-- ========= ADMIN VERIFY =========
CREATE TABLE IF NOT EXISTS verifications (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
field_name TEXT NOT NULL,
content_hash CHAR(64) NOT NULL,
verified_by UUID NOT NULL REFERENCES users(id),
verified_at TIMESTAMPTZ NOT NULL DEFAULT now(),
result TEXT
);
-- ========= AUDIT TRAIL =========
CREATE TABLE IF NOT EXISTS audit_log (
id BIGSERIAL PRIMARY KEY,
actor_id UUID REFERENCES users(id),
action TEXT NOT NULL,
entity TEXT NOT NULL,
entity_id UUID NOT NULL,
diff JSONB,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_audit_entity ON audit_log(entity, entity_id);
be0/migrations/002_application_storage_extensions.sql
+71
View File
@@ -0,0 +1,71 @@
-- Versioned tab payloads + immutable submit snapshots + workflow/taxonomy + artifact registry.
-- Apply on existing DBs: psql "$INITIATIVE_DATABASE_URL" -f migrations/002_application_storage_extensions.sql
-- (use sync driver URL, not asyncpg, for psql)
-- ========= DRAFT TAB SNAPSHOTS (fe0: report | application | contribution) =========
CREATE TABLE IF NOT EXISTS draft_tab_snapshots (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
draft_id UUID REFERENCES drafts(id) ON DELETE SET NULL,
tab TEXT NOT NULL CHECK (tab IN ('report', 'application', 'contribution')),
tab_version INTEGER NOT NULL DEFAULT 1,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
source TEXT NOT NULL DEFAULT 'autosave',
captured_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_draft_tab_snapshots_init_tab_ver
ON draft_tab_snapshots (initiative_id, tab, tab_version DESC);
CREATE INDEX IF NOT EXISTS idx_draft_tab_snapshots_captured
ON draft_tab_snapshots (captured_at DESC);
-- ========= SUBMIT SNAPSHOTS (immutable row per successful submit) =========
CREATE TABLE IF NOT EXISTS application_submit_snapshots (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
submission_record_id TEXT NOT NULL,
merged_tabs JSONB NOT NULL DEFAULT '{}'::jsonb,
submit_metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
full_pdf_uri TEXT NOT NULL,
captured_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_submit_snapshots_init_time
ON application_submit_snapshots (initiative_id, captured_at DESC);
-- ========= WORKFLOW / LIST PROJECTION (council fields) =========
CREATE TABLE IF NOT EXISTS application_workflow (
initiative_id UUID PRIMARY KEY REFERENCES initiatives(id) ON DELETE CASCADE,
review_status TEXT NOT NULL DEFAULT 'not_reviewed',
review_deadline DATE,
reviewer JSONB,
supervisor JSONB,
conference JSONB,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- ========= TAXONOMY (subjectId, groupId, topicType from fe0 ApplicationItem) =========
CREATE TABLE IF NOT EXISTS application_taxonomy (
initiative_id UUID PRIMARY KEY REFERENCES initiatives(id) ON DELETE CASCADE,
subject_id TEXT NOT NULL DEFAULT '',
group_id TEXT NOT NULL DEFAULT '',
topic_type TEXT NOT NULL DEFAULT '',
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- ========= ARTIFACTS (PDF + future abstract/poster URIs; complements evidence_files) =========
CREATE TABLE IF NOT EXISTS application_artifacts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
role TEXT NOT NULL CHECK (role IN (
'full_pdf', 'abstract', 'poster',
'textbook_evidence', 'research_evidence', 'technical_evidence', 'other'
)),
storage_uri TEXT NOT NULL,
original_name TEXT,
mime_type TEXT NOT NULL DEFAULT 'application/pdf',
byte_size BIGINT,
sha256 CHAR(64),
uploaded_by UUID REFERENCES users(id),
uploaded_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (initiative_id, role)
);
CREATE INDEX IF NOT EXISTS idx_application_artifacts_init ON application_artifacts (initiative_id);
be0/migrations/003_review_documents.sql
+22
View File
@@ -0,0 +1,22 @@
-- Persist ReviewPanel JSON bundles (templateData + officialBieuMau + full trees)
-- Apply on existing DBs:
-- psql "$INITIATIVE_DATABASE_URL" -f migrations/003_review_documents.sql
CREATE TABLE IF NOT EXISTS application_review_documents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
case_id TEXT NOT NULL,
document_version INTEGER NOT NULL DEFAULT 1,
official_bieu_mau JSONB NOT NULL DEFAULT '{}'::jsonb,
template_data JSONB,
full_bundle JSONB,
created_by UUID REFERENCES users(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (initiative_id, document_version)
);
CREATE INDEX IF NOT EXISTS idx_review_docs_initiative_time
ON application_review_documents (initiative_id, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_review_docs_case_time
ON application_review_documents (case_id, created_at DESC);
be0/migrations/004_application_admin_results.sql
+18
View File
@@ -0,0 +1,18 @@
-- Admin-recorded adjudication outcome per initiative (linked to applicant application id API).
-- One row per initiative; CRUD via /api/applications/{applicationId}/admin-result
CREATE TABLE IF NOT EXISTS application_admin_results (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
initiative_id UUID NOT NULL REFERENCES initiatives(id) ON DELETE CASCADE,
decision TEXT NOT NULL CHECK (decision IN ('approved','rejected')),
feedback TEXT NOT NULL DEFAULT '',
rationale TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
created_by UUID REFERENCES users(id),
updated_by UUID REFERENCES users(id),
CONSTRAINT uq_application_admin_results_initiative UNIQUE (initiative_id)
);
CREATE INDEX IF NOT EXISTS idx_application_admin_results_initiative
ON application_admin_results(initiative_id);
be0/migrations/004_evidence_artifact_review.sql
+13
View File
@@ -0,0 +1,13 @@
-- Evidence staff review (approve / reject) on application_artifacts — must match be0/src/initiative_db/models.py ApplicationArtifact
-- New DBs: loaded by docker-compose postgres init (04_...).
-- Existing DBs: run once, e.g.
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/004_evidence_artifact_review.sql
-- # or: psql "$INITIATIVE_DATABASE_URL" -f be0/migrations/004_evidence_artifact_review.sql
ALTER TABLE application_artifacts
ADD COLUMN IF NOT EXISTS review_status TEXT,
ADD COLUMN IF NOT EXISTS reviewed_by UUID REFERENCES users (id) ON DELETE SET NULL,
ADD COLUMN IF NOT EXISTS reviewed_at TIMESTAMPTZ;
CREATE INDEX IF NOT EXISTS idx_application_artifacts_review
ON application_artifacts (initiative_id, review_status);
be0/migrations/006_user_notifications.sql
+26
View File
@@ -0,0 +1,26 @@
-- In-app notifications for applicants (admin adjudication → inbox).
-- Best-effort insert after PUT/POST admin-result; full text duplicated for read UX.
CREATE TABLE IF NOT EXISTS user_notifications (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
recipient_user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type TEXT NOT NULL CHECK (type IN ('admin_application_decision')),
title TEXT NOT NULL,
body TEXT NOT NULL,
application_id TEXT NOT NULL,
related_initiative_id UUID REFERENCES initiatives(id) ON DELETE SET NULL,
source_admin_result_id UUID REFERENCES application_admin_results(id) ON DELETE SET NULL,
decision TEXT NOT NULL CHECK (decision IN ('approved','rejected')),
merit_category_label TEXT,
feedback_text TEXT NOT NULL DEFAULT '',
rationale_text TEXT,
read_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS user_notifications_inbox_idx
ON user_notifications (recipient_user_id, created_at DESC);
CREATE INDEX IF NOT EXISTS user_notifications_unread_idx
ON user_notifications (recipient_user_id)
WHERE read_at IS NULL;
be0/migrations/007_user_roles_email_policy_admin.sql
+33
View File
@@ -0,0 +1,33 @@
-- Policy-sourced admin rows: safe to drop when email leaves AUTH_ADMIN_EMAILS (app reconciliation).
-- Apply on existing DBs: docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/007_user_roles_email_policy_admin.sql
-- Fresh docker-compose init: add this file as docker-entrypoint-initdb.d/07_*.sql
ALTER TABLE user_roles ADD COLUMN IF NOT EXISTS admin_from_email_policy BOOLEAN NOT NULL DEFAULT FALSE;
COMMENT ON COLUMN user_roles.admin_from_email_policy IS
'TRUE when admin was granted by email allow-list (AUTH_ADMIN_EMAILS). Reconciliation may DELETE this row if the user email is no longer in the list. FALSE preserves manually granted admin (future / exceptional).';
-- One-time cleanup: remove admin for addresses not in the default institutional allow-list
-- (must match default in auth_api._DEFAULT_POLICY_ADMIN_EMAILS when AUTH_ADMIN_EMAILS is unset).
DELETE FROM user_roles ur
USING users u
WHERE ur.user_id = u.id
AND ur.role::text = 'admin'
AND lower(u.email::text) NOT IN (
'thaontt@ump.edu.vn',
'nltanh@ump.edu.vn',
'ldbaochau@ump.edu.vn',
'htchuong@ump.edu.vn'
);
UPDATE user_roles ur
SET admin_from_email_policy = TRUE
FROM users u
WHERE ur.user_id = u.id
AND ur.role::text = 'admin'
AND lower(u.email::text) IN (
'thaontt@ump.edu.vn',
'nltanh@ump.edu.vn',
'ldbaochau@ump.edu.vn',
'htchuong@ump.edu.vn'
);
be0/migrations/008_audit_events.sql
+38
View File
@@ -0,0 +1,38 @@
-- Unified append-only audit trail (see assets/docs/audit-log-implementation.md).
-- Application role should be granted INSERT, SELECT only (configure per deployment).
DO $$
BEGIN
CREATE TYPE audit_action AS ENUM (
'create',
'read',
'update',
'delete',
'login',
'logout',
'login_failed'
);
EXCEPTION
WHEN duplicate_object THEN NULL;
END
$$;
CREATE TABLE IF NOT EXISTS audit_events (
id BIGSERIAL PRIMARY KEY,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),
actor_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
actor_email TEXT NOT NULL,
actor_role TEXT NOT NULL,
action audit_action NOT NULL,
entity_type TEXT NOT NULL,
entity_id TEXT,
before JSONB,
after JSONB,
metadata JSONB NOT NULL DEFAULT '{}'::jsonb,
request_id UUID
);
CREATE INDEX IF NOT EXISTS idx_audit_actor_time ON audit_events (actor_user_id, occurred_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_entity ON audit_events (entity_type, entity_id, occurred_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_action_time ON audit_events (action, occurred_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_metadata_gin ON audit_events USING gin (metadata);
be0/migrations/009_backup_artifact_roles_storage_kind.sql
+35
View File
@@ -0,0 +1,35 @@
-- Backup / canonical storage: official printable DOCX+PDF roles + explicit storage_kind.
-- Apply: psql "$INITIATIVE_DATABASE_URL" -f migrations/009_backup_artifact_roles_storage_kind.sql
ALTER TABLE application_artifacts DROP CONSTRAINT IF EXISTS application_artifacts_role_check;
ALTER TABLE application_artifacts ADD CONSTRAINT application_artifacts_role_check CHECK (role IN (
'full_pdf',
'abstract',
'poster',
'textbook_evidence',
'research_evidence',
'technical_evidence',
'other',
'official_form_docx',
'official_form_pdf'
));
ALTER TABLE application_artifacts
ADD COLUMN IF NOT EXISTS storage_kind TEXT;
UPDATE application_artifacts SET storage_kind = CASE
WHEN storage_uri LIKE 'http://%' OR storage_uri LIKE 'https://%' THEN 'external_url'
WHEN storage_uri LIKE '/submitted-initiatives/%' THEN 'filesystem'
WHEN role IN ('research_evidence', 'textbook_evidence', 'technical_evidence') THEN 'minio_attachments'
ELSE 'minio_exports'
END
WHERE storage_kind IS NULL;
ALTER TABLE application_artifacts DROP CONSTRAINT IF EXISTS application_artifacts_storage_kind_check;
ALTER TABLE application_artifacts ADD CONSTRAINT application_artifacts_storage_kind_check
CHECK (storage_kind IS NULL OR storage_kind IN (
'minio_exports',
'minio_attachments',
'filesystem',
'external_url'
));
be0/migrations/010_user_staff_profiles.sql
+114
View File
@@ -0,0 +1,114 @@
-- User staff profiles (1:1 with users) — HR / verification workflow
-- Apply: docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/010_user_staff_profiles.sql
DO $$ BEGIN
CREATE TYPE profile_verification_status AS ENUM ('draft', 'pending', 'verified', 'rejected');
EXCEPTION WHEN duplicate_object THEN NULL;
END $$;
CREATE TABLE IF NOT EXISTS academic_titles (
code TEXT PRIMARY KEY,
label_vi TEXT NOT NULL,
label_en TEXT NOT NULL,
sort_order INTEGER NOT NULL DEFAULT 0,
active BOOLEAN NOT NULL DEFAULT TRUE
);
INSERT INTO academic_titles (code, label_vi, label_en, sort_order) VALUES
('professor', 'Giáo sư', 'Professor', 10),
('associate_professor', 'Phó Giáo sư', 'Associate Professor', 20),
('doctor_sc', 'Tiến sĩ', 'Doctor of Science', 30),
('bsckii', 'BSCKII', 'Specialist level II', 35),
('bscki', 'BSCKI', 'Specialist level I', 36),
('master', 'Thạc sĩ', 'Master', 40),
('doctor_md', 'Bác sĩ', 'Physician', 45),
('pharmacist', 'Dược sĩ', 'Pharmacist', 46),
('bachelor', 'Cử nhân', 'Bachelor', 50),
('other', 'Khác (ghi rõ)', 'Other (specify)', 100)
ON CONFLICT (code) DO NOTHING;
CREATE TABLE IF NOT EXISTS user_staff_profiles (
user_id UUID PRIMARY KEY
REFERENCES users(id) ON DELETE CASCADE,
employee_id TEXT,
academic_title_code TEXT REFERENCES academic_titles(code),
academic_title_other TEXT,
unit_name_freetext TEXT,
job_title TEXT,
profile_verification_status profile_verification_status
NOT NULL DEFAULT 'draft',
verification_submitted_at TIMESTAMPTZ,
verified_at TIMESTAMPTZ,
verified_by_user_id UUID REFERENCES users(id),
rejection_reason TEXT,
version INTEGER NOT NULL DEFAULT 1,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT employee_id_shape
CHECK (employee_id IS NULL OR employee_id ~ '^[A-Z0-9-]{3,32}$'),
CONSTRAINT academic_title_other_invariant CHECK (
CASE
WHEN academic_title_code IS NULL THEN academic_title_other IS NULL
WHEN academic_title_code = 'other' THEN
academic_title_other IS NOT NULL AND length(trim(academic_title_other)) > 0
ELSE academic_title_other IS NULL
END
),
CONSTRAINT verified_requires_metadata CHECK (
profile_verification_status <> 'verified'
OR (verified_at IS NOT NULL AND verified_by_user_id IS NOT NULL)
),
CONSTRAINT rejected_requires_reason CHECK (
profile_verification_status <> 'rejected'
OR (rejection_reason IS NOT NULL AND length(trim(rejection_reason)) > 0)
),
CONSTRAINT non_terminal_clears_verification CHECK (
profile_verification_status NOT IN ('draft', 'pending')
OR (verified_at IS NULL AND verified_by_user_id IS NULL)
),
CONSTRAINT rejected_clears_verification_metadata CHECK (
profile_verification_status <> 'rejected'
OR (verified_at IS NULL AND verified_by_user_id IS NULL)
),
CONSTRAINT verified_clears_rejection CHECK (
profile_verification_status <> 'verified'
OR rejection_reason IS NULL
),
CONSTRAINT job_title_length CHECK (
job_title IS NULL OR length(job_title) <= 120
)
);
CREATE UNIQUE INDEX IF NOT EXISTS ix_usp_employee_id_unique
ON user_staff_profiles (employee_id)
WHERE employee_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS ix_usp_pending_queue
ON user_staff_profiles (verification_submitted_at)
WHERE profile_verification_status = 'pending';
CREATE INDEX IF NOT EXISTS ix_usp_verifier_activity
ON user_staff_profiles (verified_by_user_id, verified_at DESC)
WHERE verified_by_user_id IS NOT NULL;
-- Backfill one row per existing user (draft, NULL fields)
INSERT INTO user_staff_profiles (user_id, profile_verification_status)
SELECT u.id, 'draft'::profile_verification_status
FROM users u
WHERE NOT EXISTS (
SELECT 1 FROM user_staff_profiles p WHERE p.user_id = u.id
);
COMMENT ON TABLE user_staff_profiles IS
'Institutional staff profile and verification state; scalars only — no MinIO.';
be0/migrations/011_academic_titles_vn.sql
+19
View File
@@ -0,0 +1,19 @@
-- Extend / refresh academic_titles for UMP staff profile dropdown (VN labels + BSCK codes).
-- Apply after 010: psql … -f be0/migrations/011_academic_titles_vn.sql
INSERT INTO academic_titles (code, label_vi, label_en, sort_order, active) VALUES
('professor', 'Giáo sư', 'Professor', 10, TRUE),
('associate_professor', 'Phó Giáo sư', 'Associate Professor', 20, TRUE),
('doctor_sc', 'Tiến sĩ', 'Doctor of Science', 30, TRUE),
('bsckii', 'BSCKII', 'Specialist level II', 35, TRUE),
('bscki', 'BSCKI', 'Specialist level I', 36, TRUE),
('master', 'Thạc sĩ', 'Master', 40, TRUE),
('doctor_md', 'Bác sĩ', 'Physician', 45, TRUE),
('pharmacist', 'Dược sĩ', 'Pharmacist', 46, TRUE),
('bachelor', 'Cử nhân', 'Bachelor', 50, TRUE),
('other', 'Khác (ghi rõ)', 'Other (specify)', 100, TRUE)
ON CONFLICT (code) DO UPDATE SET
label_vi = EXCLUDED.label_vi,
label_en = EXCLUDED.label_en,
sort_order = EXCLUDED.sort_order,
active = EXCLUDED.active;
be0/migrations/012_password_reset.sql
+19
View File
@@ -0,0 +1,19 @@
-- Password reset tokens + JWT credential invalidation (see auth_api, auth_credential_middleware).
-- Apply: docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/012_password_reset.sql
ALTER TABLE users ADD COLUMN IF NOT EXISTS credential_version INTEGER NOT NULL DEFAULT 0;
COMMENT ON COLUMN users.credential_version IS
'Incremented on password change/reset. JWT ''cv'' claim must match or token is rejected.';
CREATE TABLE IF NOT EXISTS password_reset_tokens (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
expires_at TIMESTAMPTZ NOT NULL,
used_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_password_reset_tokens_user_id ON password_reset_tokens(user_id);
CREATE INDEX IF NOT EXISTS idx_password_reset_tokens_expires_at ON password_reset_tokens(expires_at);
be0/migrations/013_email_verification.sql
+21
View File
@@ -0,0 +1,21 @@
-- Email verification before login (see auth_api deliver_email_verification_email).
-- Apply: docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/013_email_verification.sql
ALTER TABLE users ADD COLUMN IF NOT EXISTS email_verified BOOLEAN NOT NULL DEFAULT FALSE;
UPDATE users SET email_verified = TRUE WHERE email_verified = FALSE;
COMMENT ON COLUMN users.email_verified IS
'FALSE until user confirms institutional inbox via email link; login and API tokens require TRUE.';
CREATE TABLE IF NOT EXISTS email_verification_tokens (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
expires_at TIMESTAMPTZ NOT NULL,
used_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_email_verification_tokens_user_id ON email_verification_tokens(user_id);
CREATE INDEX IF NOT EXISTS idx_email_verification_tokens_expires_at ON email_verification_tokens(expires_at);
be0/migrations/014_registration_otp.sql
+20
View File
@@ -0,0 +1,20 @@
-- Registration email verification via 6-digit OTP (replaces magic-link issuance on register).
-- Apply after 013_email_verification.sql:
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/014_registration_otp.sql
CREATE TABLE IF NOT EXISTS registration_otp_codes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
otp_hash TEXT NOT NULL,
expires_at TIMESTAMPTZ NOT NULL,
failed_attempts INT NOT NULL DEFAULT 0,
used_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_registration_otp_codes_user_pending
ON registration_otp_codes (user_id)
WHERE used_at IS NULL;
COMMENT ON TABLE registration_otp_codes IS
'Hashed 6-digit OTP for register verification; pending rows deleted when superseded by resend.';
be0/migrations/015_document_templates.sql
+24
View File
@@ -0,0 +1,24 @@
-- Admin-managed document templates: a .docx (stored in MinIO bucket initiative-templates)
-- plus its extracted Jinja placeholder fields. Applicants render a filled PDF by template id.
-- Apply after 014_registration_otp.sql:
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/015_document_templates.sql
CREATE TABLE IF NOT EXISTS document_templates (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
description TEXT,
storage_key TEXT NOT NULL,
original_filename TEXT,
content_sha256 TEXT,
fields JSONB NOT NULL DEFAULT '[]'::jsonb,
is_active BOOLEAN NOT NULL DEFAULT TRUE,
created_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_document_templates_active
ON document_templates (is_active, created_at DESC);
COMMENT ON TABLE document_templates IS
'Admin-managed DOCX templates (file in MinIO initiative-templates) with extracted Jinja placeholder fields. Applicants render filled PDFs by template id.';
be0/migrations/016_research_projects.sql
+133
View File
@@ -0,0 +1,133 @@
-- Research-project proposals (Thuyết minh đề tài, Mẫu III.06-TM.ĐTUD) + the PI "cockpit" entities.
-- A proposal row IS the project across its lifecycle: draft -> submitted -> approved | rejected.
-- On approval the cockpit unlocks; child tables (members/datasets/models/assets/milestones) hang off it.
-- Owner+admin authz (v1): a project is owned by owner_user_id; admins may review/approve/reject.
-- Apply after 015_document_templates.sql:
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/016_research_projects.sql
CREATE TABLE IF NOT EXISTS research_projects (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
owner_user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
status TEXT NOT NULL DEFAULT 'draft' CHECK (status IN ('draft','submitted','approved','rejected')),
code TEXT,
title TEXT NOT NULL DEFAULT '',
level TEXT NOT NULL DEFAULT '',
pi_name TEXT NOT NULL DEFAULT '',
period_months INTEGER,
budget_total NUMERIC(14,2),
content JSONB NOT NULL DEFAULT '{}'::jsonb,
submitted_at TIMESTAMPTZ,
reviewed_by UUID REFERENCES users(id) ON DELETE SET NULL,
reviewed_at TIMESTAMPTZ,
review_note TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_research_projects_owner ON research_projects (owner_user_id, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_research_projects_status ON research_projects (status, created_at DESC);
CREATE TABLE IF NOT EXISTS research_project_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id UUID NOT NULL REFERENCES research_projects(id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
name TEXT NOT NULL DEFAULT '',
role TEXT NOT NULL DEFAULT '',
access TEXT NOT NULL DEFAULT '',
org TEXT NOT NULL DEFAULT '',
email TEXT NOT NULL DEFAULT '',
months INTEGER,
tasks TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT '',
user_id UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_research_project_members_project ON research_project_members (project_id, sort_order);
CREATE TABLE IF NOT EXISTS research_project_datasets (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id UUID NOT NULL REFERENCES research_projects(id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
name TEXT NOT NULL DEFAULT '',
type TEXT NOT NULL DEFAULT '',
records INTEGER,
source TEXT NOT NULL DEFAULT '',
sensitivity TEXT NOT NULL DEFAULT '',
ethics TEXT NOT NULL DEFAULT '',
owner TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_research_project_datasets_project ON research_project_datasets (project_id, sort_order);
CREATE TABLE IF NOT EXISTS research_project_models (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id UUID NOT NULL REFERENCES research_projects(id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
name TEXT NOT NULL DEFAULT '',
task TEXT NOT NULL DEFAULT '',
framework TEXT NOT NULL DEFAULT '',
version TEXT NOT NULL DEFAULT '',
dataset TEXT NOT NULL DEFAULT '',
auc NUMERIC(6,4),
sensitivity NUMERIC(6,4),
specificity NUMERIC(6,4),
accuracy NUMERIC(6,4),
owner TEXT NOT NULL DEFAULT '',
notes TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_research_project_models_project ON research_project_models (project_id, sort_order);
CREATE TABLE IF NOT EXISTS research_project_assets (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id UUID NOT NULL REFERENCES research_projects(id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
name TEXT NOT NULL DEFAULT '',
category TEXT NOT NULL DEFAULT '',
acquisition TEXT NOT NULL DEFAULT '',
value NUMERIC(14,2),
owner TEXT NOT NULL DEFAULT '',
notes TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_research_project_assets_project ON research_project_assets (project_id, sort_order);
CREATE TABLE IF NOT EXISTS research_project_milestones (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
project_id UUID NOT NULL REFERENCES research_projects(id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
title TEXT NOT NULL DEFAULT '',
deliverable TEXT NOT NULL DEFAULT '',
start_period TEXT NOT NULL DEFAULT '',
end_period TEXT NOT NULL DEFAULT '',
owner TEXT NOT NULL DEFAULT '',
budget NUMERIC(14,2),
progress INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_research_project_milestones_project ON research_project_milestones (project_id, sort_order);
CREATE TABLE IF NOT EXISTS research_project_audit (
id BIGSERIAL PRIMARY KEY,
project_id UUID NOT NULL REFERENCES research_projects(id) ON DELETE CASCADE,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),
actor_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
actor_name TEXT NOT NULL DEFAULT '',
role_label TEXT NOT NULL DEFAULT '',
action TEXT NOT NULL,
subject TEXT NOT NULL DEFAULT '',
detail TEXT NOT NULL DEFAULT ''
);
CREATE INDEX IF NOT EXISTS idx_research_project_audit_project ON research_project_audit (project_id, occurred_at DESC);
COMMENT ON TABLE research_projects IS
'Research-project proposals (Thuyet minh de tai) that become managed projects on approval. Owner and admin authz. Content JSONB holds the full proposal form. Child research_project_* tables hold cockpit entities.';
be0/migrations/017_imagehub_datasets.sql
+76
View File
@@ -0,0 +1,76 @@
-- ImageHub: content-addressed imaging dataset versioning (milestone 1 walking skeleton).
-- A dataset is owned by a user (investigator/PI). Files are stored as content-addressed,
-- globally deduped blobs in MinIO (one imagehub_blobs row per distinct sha256). The current
-- working file set lives in imagehub_dataset_files; a version freezes a manifest snapshot.
-- Admin sees all datasets (clinical data repository); owners see their own (research data).
-- Apply after 016_research_projects.sql:
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/017_imagehub_datasets.sql
CREATE TABLE IF NOT EXISTS imagehub_datasets (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
owner_user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL DEFAULT '',
slug TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT '',
visibility TEXT NOT NULL DEFAULT 'private' CHECK (visibility IN ('private','internal','public')),
modality_tags JSONB NOT NULL DEFAULT '[]'::jsonb,
default_branch TEXT NOT NULL DEFAULT 'main',
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_imagehub_datasets_owner ON imagehub_datasets (owner_user_id, created_at DESC);
-- Globally content-addressed blob registry: identical bytes across datasets dedupe to one row.
CREATE TABLE IF NOT EXISTS imagehub_blobs (
sha256 TEXT PRIMARY KEY,
size_bytes BIGINT NOT NULL DEFAULT 0,
media_type TEXT NOT NULL DEFAULT 'application/octet-stream',
storage_bucket TEXT NOT NULL DEFAULT '',
storage_key TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- Current working file set on a dataset default branch (one row per logical path).
CREATE TABLE IF NOT EXISTS imagehub_dataset_files (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
dataset_id UUID NOT NULL REFERENCES imagehub_datasets(id) ON DELETE CASCADE,
logical_path TEXT NOT NULL DEFAULT '',
blob_sha256 TEXT NOT NULL REFERENCES imagehub_blobs(sha256) ON DELETE RESTRICT,
size_bytes BIGINT NOT NULL DEFAULT 0,
media_type TEXT NOT NULL DEFAULT 'application/octet-stream',
imaging_meta JSONB NOT NULL DEFAULT '{}'::jsonb,
uploaded_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX IF NOT EXISTS uq_imagehub_dataset_files_path ON imagehub_dataset_files (dataset_id, logical_path);
-- Frozen version snapshots (the versioning spine; DAG-ready via parent_version_id).
CREATE TABLE IF NOT EXISTS imagehub_versions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
dataset_id UUID NOT NULL REFERENCES imagehub_datasets(id) ON DELETE CASCADE,
seq INTEGER NOT NULL DEFAULT 1,
message TEXT NOT NULL DEFAULT '',
manifest JSONB NOT NULL DEFAULT '[]'::jsonb,
parent_version_id UUID REFERENCES imagehub_versions(id) ON DELETE SET NULL,
author_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX IF NOT EXISTS uq_imagehub_versions_seq ON imagehub_versions (dataset_id, seq);
-- Append-only audit trail per dataset.
CREATE TABLE IF NOT EXISTS imagehub_dataset_audit (
id BIGSERIAL PRIMARY KEY,
dataset_id UUID NOT NULL REFERENCES imagehub_datasets(id) ON DELETE CASCADE,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT now(),
actor_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
actor_name TEXT NOT NULL DEFAULT '',
role_label TEXT NOT NULL DEFAULT '',
action TEXT NOT NULL,
subject TEXT NOT NULL DEFAULT '',
detail TEXT NOT NULL DEFAULT ''
);
CREATE INDEX IF NOT EXISTS idx_imagehub_dataset_audit_dataset ON imagehub_dataset_audit (dataset_id, occurred_at DESC);
COMMENT ON TABLE imagehub_datasets IS
'ImageHub content-addressed imaging datasets. Owner and admin authz. Files dedupe into imagehub_blobs by sha256 — imagehub_versions freezes a manifest snapshot.';
be0/migrations/018_imagehub_segmentation_links.sql
+21
View File
@@ -0,0 +1,21 @@
-- ImageHub: link organ-segmentation masks to their parent image file (Phase D).
-- A mask file (file_kind='segmentation') points at the image it segments via a
-- self-referential parent_file_id (e.g. an organ mask of ct.nii.gz); organ_label
-- names the organ. Regular files stay file_kind='image'. Idempotent (ADD COLUMN IF
-- NOT EXISTS) so the startup runner can apply it to volumes that predate it.
-- Apply after 017_imagehub_datasets.sql (no semicolons inside comments — the runner
-- splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/018_imagehub_segmentation_links.sql
ALTER TABLE imagehub_dataset_files
ADD COLUMN IF NOT EXISTS file_kind TEXT NOT NULL DEFAULT 'image' CHECK (file_kind IN ('image','segmentation'));
ALTER TABLE imagehub_dataset_files
ADD COLUMN IF NOT EXISTS parent_file_id UUID REFERENCES imagehub_dataset_files(id) ON DELETE CASCADE;
ALTER TABLE imagehub_dataset_files
ADD COLUMN IF NOT EXISTS organ_label TEXT NOT NULL DEFAULT '';
-- List all masks of an image efficiently.
CREATE INDEX IF NOT EXISTS idx_imagehub_dataset_files_parent
ON imagehub_dataset_files (parent_file_id);
be0/migrations/019_imagehub_cloud_import.sql
+53
View File
@@ -0,0 +1,53 @@
-- ImageHub: Cloud Import — storage methods + external (referenced, not copied) dataset files.
-- A storage method holds verified credentials (config_encrypted, never returned to the client)
-- for an external bucket (S3/GCS/Azure). A dataset file is then EITHER a local content-addressed
-- blob (blob_sha256 set) OR an external reference (storage_method_id + external_path set) that
-- streams from the bucket and is never copied to our servers (privacy rule C4). Idempotent
-- (CREATE/ADD ... IF NOT EXISTS) so the startup runner can apply it to volumes that predate it.
-- Apply after 018 (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/019_imagehub_cloud_import.sql
CREATE TABLE IF NOT EXISTS imagehub_storage_methods (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
owner_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
name TEXT NOT NULL,
provider TEXT NOT NULL CHECK (provider IN ('s3','gcs','azure')),
access_mode TEXT NOT NULL DEFAULT 'read' CHECK (access_mode IN ('read','readwrite')),
bucket TEXT NOT NULL,
region TEXT,
config_encrypted TEXT NOT NULL,
verification_status TEXT NOT NULL DEFAULT 'pending' CHECK (verification_status IN ('pending','verified','failed')),
verification_reason TEXT,
verification_checked_at TIMESTAMPTZ,
created_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_imagehub_storage_methods_owner
ON imagehub_storage_methods (owner_id);
-- Allow a dataset file to be an external reference instead of a local blob. Existing rows keep
-- blob_sha256 set and the new columns NULL, so they satisfy the local-blob branch of the CHECK.
ALTER TABLE imagehub_dataset_files
ALTER COLUMN blob_sha256 DROP NOT NULL;
ALTER TABLE imagehub_dataset_files
ADD COLUMN IF NOT EXISTS storage_method_id UUID REFERENCES imagehub_storage_methods(id) ON DELETE RESTRICT;
ALTER TABLE imagehub_dataset_files
ADD COLUMN IF NOT EXISTS external_path TEXT;
-- A file is EITHER a local content-addressed blob OR an external reference, never both or neither.
ALTER TABLE imagehub_dataset_files
DROP CONSTRAINT IF EXISTS ck_imagehub_file_storage_mode;
ALTER TABLE imagehub_dataset_files
ADD CONSTRAINT ck_imagehub_file_storage_mode CHECK (
(blob_sha256 IS NOT NULL AND storage_method_id IS NULL AND external_path IS NULL)
OR
(blob_sha256 IS NULL AND storage_method_id IS NOT NULL AND external_path IS NOT NULL)
);
CREATE INDEX IF NOT EXISTS idx_imagehub_dataset_files_storage_method
ON imagehub_dataset_files (storage_method_id);
be0/migrations/020_imagehub_dataset_stages.sql
+26
View File
@@ -0,0 +1,26 @@
-- ImageHub: labeling-pipeline stages on a dataset (Label -> Review_1 -> Review_2 ...). Each stage
-- has a kind (label/review), an order (seq), an optional review_percent (review stages only), and
-- an auto_assign flag (the "Automatic Task Assignment" toggle). Idempotent (CREATE ... IF NOT
-- EXISTS) so the startup runner can apply it to volumes that predate it. Apply after 019 (no
-- semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/020_imagehub_dataset_stages.sql
CREATE TABLE IF NOT EXISTS imagehub_dataset_stages (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
dataset_id UUID NOT NULL REFERENCES imagehub_datasets(id) ON DELETE CASCADE,
name TEXT NOT NULL,
kind TEXT NOT NULL DEFAULT 'label' CHECK (kind IN ('label','review')),
seq INTEGER NOT NULL DEFAULT 0,
review_percent INTEGER CHECK (review_percent IS NULL OR (review_percent >= 0 AND review_percent <= 100)),
auto_assign BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- Stages of a dataset, in pipeline order.
CREATE INDEX IF NOT EXISTS idx_imagehub_dataset_stages_dataset
ON imagehub_dataset_stages (dataset_id, seq);
-- A stage name is unique within its dataset.
CREATE UNIQUE INDEX IF NOT EXISTS uq_imagehub_dataset_stages_name
ON imagehub_dataset_stages (dataset_id, name);
be0/migrations/021_imagehub_task_pipeline.sql
+37
View File
@@ -0,0 +1,37 @@
-- ImageHub: per-file work TASKS that flow through a dataset's pipeline stages (single-user MVP).
-- A task is a NEW join row (one per dataset file) carrying its pipeline position (current_stage_id
-- + pipeline_state), per-user queue status, assignee, priority, and the Ground-Truth reference flag.
-- The file row itself (imagehub_dataset_files) stays a pure storage record. Membership / multi-labeler
-- assignment is a later phase, so for now task access reuses the dataset owner-or-admin gate.
-- Idempotent (CREATE ... IF NOT EXISTS) so the startup runner can apply it to volumes that predate it.
-- Apply after 020 (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/021_imagehub_task_pipeline.sql
CREATE TABLE IF NOT EXISTS imagehub_tasks (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
dataset_id UUID NOT NULL REFERENCES imagehub_datasets(id) ON DELETE CASCADE,
dataset_file_id UUID NOT NULL REFERENCES imagehub_dataset_files(id) ON DELETE CASCADE,
name TEXT NOT NULL DEFAULT '',
current_stage_id UUID REFERENCES imagehub_dataset_stages(id) ON DELETE SET NULL,
pipeline_state TEXT NOT NULL DEFAULT 'inLabel' CHECK (pipeline_state IN ('inLabel','inReview','groundTruth','issue')),
queue_status TEXT NOT NULL DEFAULT 'assigned' CHECK (queue_status IN ('assigned','saved','pendingFinalization','skipped')),
assignee_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
assignment_mode TEXT NOT NULL DEFAULT 'auto' CHECK (assignment_mode IN ('auto','manual')),
priority DOUBLE PRECISION NOT NULL DEFAULT 0 CHECK (priority >= 0 AND priority <= 1),
is_reference_standard BOOLEAN NOT NULL DEFAULT FALSE,
skipped_seq BIGINT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- One task per file (MVP simplification — droppable later for multi-task-per-file).
CREATE UNIQUE INDEX IF NOT EXISTS uq_imagehub_tasks_file
ON imagehub_tasks (dataset_file_id);
-- Queue scan: a dataset's tasks at a given stage and status, highest priority first.
CREATE INDEX IF NOT EXISTS idx_imagehub_tasks_queue
ON imagehub_tasks (dataset_id, current_stage_id, queue_status, priority DESC);
-- A user's personal labeling queue across datasets.
CREATE INDEX IF NOT EXISTS idx_imagehub_tasks_assignee
ON imagehub_tasks (assignee_user_id, queue_status);
be0/migrations/022_imagehub_task_annotations.sql
+8
View File
@@ -0,0 +1,8 @@
-- ImageHub: a task's labeler annotations (bbox / points / pen / brush / polygon) stored as JSON.
-- The shared viewer's annotation overlay emits normalized [0..1] vector geometry per slice — small
-- JSON, persisted on the task so the AnnotationTool can load + save a labeler's work. Idempotent
-- (ADD COLUMN IF NOT EXISTS) so the startup runner can apply it to volumes that predate it. Apply
-- after 021 (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/022_imagehub_task_annotations.sql
ALTER TABLE imagehub_tasks ADD COLUMN IF NOT EXISTS annotations JSONB NOT NULL DEFAULT '[]'::jsonb;
be0/migrations/023_imagehub_dataset_members.sql
+23
View File
@@ -0,0 +1,23 @@
-- ImageHub: dataset membership — lets users other than the owner work a dataset's tasks
-- (multi-labeler). MVP treats all members as labelers: they view the dataset and work tasks
-- assigned to them, while dataset / stage / settings management stays with the owner + platform
-- admins. The role column is reserved for a future project-admin tier. Idempotent. Apply after 022
-- (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/023_imagehub_dataset_members.sql
CREATE TABLE IF NOT EXISTS imagehub_dataset_members (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
dataset_id UUID NOT NULL REFERENCES imagehub_datasets(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
role TEXT NOT NULL DEFAULT 'member' CHECK (role IN ('project_admin','member')),
added_by UUID REFERENCES users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- One membership per user per dataset.
CREATE UNIQUE INDEX IF NOT EXISTS uq_imagehub_dataset_members_user
ON imagehub_dataset_members (dataset_id, user_id);
-- "Datasets I am a member of" lookup (the member's dataset list).
CREATE INDEX IF NOT EXISTS idx_imagehub_dataset_members_user
ON imagehub_dataset_members (user_id);
be0/migrations/024_imagehub_dataset_project_link.sql
+13
View File
@@ -0,0 +1,13 @@
-- ImageHub: link a dataset to a research project (the "workspace" superstructure). Nullable,
-- so existing datasets stay unlinked and a dataset can still exist standalone. A dataset created
-- from a project cockpit attaches to that project. ON DELETE SET NULL so deleting a project
-- orphans its datasets rather than dropping the imaging data. Idempotent. Apply after 023
-- (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/024_imagehub_dataset_project_link.sql
ALTER TABLE imagehub_datasets
ADD COLUMN IF NOT EXISTS research_project_id UUID REFERENCES research_projects(id) ON DELETE SET NULL;
-- "Datasets in this project" lookup (the project-scoped dataset list).
CREATE INDEX IF NOT EXISTS idx_imagehub_datasets_research_project
ON imagehub_datasets (research_project_id);
be0/migrations/025_imagehub_task_review_events.sql
+25
View File
@@ -0,0 +1,25 @@
-- ImageHub: structured review decisions. The task pipeline applies accept/acceptWithCorrections/
-- reject moves, but until now the verdict survived only as a free-text Vietnamese audit string —
-- not queryable, no reviewer/stage FK, no reject reason. This append-only table records every
-- review decision so review history + per-reviewer accept/reject counters become real. Idempotent.
-- Apply after 024 (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/025_imagehub_task_review_events.sql
CREATE TABLE IF NOT EXISTS imagehub_task_review_events (
id BIGSERIAL PRIMARY KEY,
dataset_id UUID NOT NULL REFERENCES imagehub_datasets(id) ON DELETE CASCADE,
task_id UUID NOT NULL REFERENCES imagehub_tasks(id) ON DELETE CASCADE,
stage_id UUID REFERENCES imagehub_dataset_stages(id) ON DELETE SET NULL,
reviewer_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
decision TEXT NOT NULL CHECK (decision IN ('accept','acceptWithCorrections','reject')),
note TEXT NOT NULL DEFAULT '',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
-- Per-reviewer counters over a date window (the productivity panel query).
CREATE INDEX IF NOT EXISTS idx_imagehub_review_events_reviewer
ON imagehub_task_review_events (dataset_id, reviewer_user_id, created_at);
-- A task's review history (chronological).
CREATE INDEX IF NOT EXISTS idx_imagehub_review_events_task
ON imagehub_task_review_events (task_id, created_at);
be0/migrations/026_imagehub_file_folder_path.sql
+21
View File
@@ -0,0 +1,21 @@
-- ImageHub: persist the relative folder path of each uploaded file (Option B — real folders inside
-- a dataset). Until now logical_path was basename-flattened, so an uploaded directory structure
-- (e.g. the nnU-Net imagesTr/labelsTr layout) was lost once files reached MinIO. folder_path keeps
-- the relative directory so the dataset browser can render a real folder tree and the structure
-- round-trips. The working-file natural key moves from (dataset_id, logical_path) to
-- (dataset_id, folder_path, logical_path) so two files sharing a basename in different folders no
-- longer collide and silently merge. Existing rows default folder_path to the empty string, so the
-- new key stays unique wherever the old one was. Idempotent.
-- Apply after 025 (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/026_imagehub_file_folder_path.sql
ALTER TABLE imagehub_dataset_files
ADD COLUMN IF NOT EXISTS folder_path TEXT NOT NULL DEFAULT '';
DROP INDEX IF EXISTS uq_imagehub_dataset_files_path;
CREATE UNIQUE INDEX IF NOT EXISTS uq_imagehub_dataset_files_folder_path
ON imagehub_dataset_files (dataset_id, folder_path, logical_path);
CREATE INDEX IF NOT EXISTS idx_imagehub_dataset_files_folder
ON imagehub_dataset_files (dataset_id, folder_path);
be0/migrations/027_imagehub_dataset_label_map.sql
+12
View File
@@ -0,0 +1,12 @@
-- ImageHub: per-dataset value to name label map for multi-label segmentation masks. A multi-label
-- labelsTr/<case>.nii.gz encodes each organ or structure as an integer voxel value (1, 2, 3 …). Until
-- now the viewer named those values from a fixed TotalSegmentator-v2 117-class map, so a non
-- TotalSegmentator dataset (KiTS = 1 kidney / 2 tumor / 3 cyst, or any custom nnU-Net labels) showed
-- confidently-wrong organ names. label_map stores the dataset own value to name mapping (a JSON object
-- with string keys), so the organ panel labels each overlay correctly and a user can edit them. The
-- empty default keeps the TotalSegmentator fallback for datasets without a map. Idempotent.
-- Apply after 026 (no semicolons inside comments or string literals — the runner splitter is naive):
-- docker exec -i initiative-postgres psql -U initiative -d initiatives < be0/migrations/027_imagehub_dataset_label_map.sql
ALTER TABLE imagehub_datasets
ADD COLUMN IF NOT EXISTS label_map JSONB NOT NULL DEFAULT '{}'::jsonb;
be0/requirements-dev.txt
+6
View File
@@ -0,0 +1,6 @@
# Test-only dependencies for CI (not installed in the runtime image).
# be0 tests are a mix of unittest.TestCase (incl. IsolatedAsyncioTestCase) and
# pytest-style; pytest runs both. pytest-asyncio covers the pytest async tests.
-r requirements.txt
pytest>=8,<9
pytest-asyncio>=0.23,<0.24
be0/requirements.txt
+43
View File
@@ -0,0 +1,43 @@
uvicorn[standard]
httpx
sqlalchemy[asyncio]>=2.0
asyncpg>=0.29
greenlet>=3.0
argon2-cffi>=23.1.0
PyJWT>=2.8.0
ollama
fastapi
asyncio
python-multipart
langchain
langchain-core
langgraph
langchain-community
sentence-transformers
huggingface
scikit-learn
neo4j
nltk
rake-nltk
pypdf
pydantic
pydantic-settings
aioboto3
zipstream-ng
boto3
numpy
pandas
pyvi
docling
pymupdf
docxtpl>=0.16
openpyxl>=3.1.0
# ImageHub: best-effort imaging metadata sniff (DICOM / NIfTI). See src/imagehub_routes.py.
pydicom
nibabel
be0/scripts/__pycache__/apply_initiative_migrations.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/scripts/__pycache__/repair_split_submission.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/scripts/add_ump_ideas.py
+93
View File
@@ -0,0 +1,93 @@
"""
Script to add the 10 UMP innovation ideas to the vector database
"""
import asyncio
import sys
from pathlib import Path
# Add parent directory to path
sys.path.insert(0, str(Path(__file__).parent.parent))
from src.infrastructure.vector_db.qdrant_service import get_qdrant_service
UMP_IDEAS = [
{
"title": "Nền tảng Trợ lý AI học tập lâm sàng (Clinical AI Tutor)",
"description": "Ứng dụng AI đóng vai trò trợ giảng cho sinh viên y, hỗ trợ phân tích ca bệnh giả lập, giải thích cận lâm sàng, và gợi ý chẩn đoán theo phác đồ Việt Nam.",
"category": "Giáo dục - AI"
},
{
"title": "Hệ thống bệnh án điện tử học thuật (Academic EMR Sandbox)",
"description": "Môi trường EMR mô phỏng cho đào tạo và nghiên cứu, cho phép sinh viên và giảng viên thực hành nhập – phân tích – khai thác dữ liệu y khoa mà không ảnh hưởng dữ liệu bệnh nhân thật.",
"category": "Giáo dục - Chuyển đổi số"
},
{
"title": "Trung tâm mô phỏng y khoa bằng AR/VR & Digital Twin",
"description": "Xây dựng phòng lab mô phỏng phẫu thuật, cấp cứu, và quy trình điều trị bằng AR/VR, kết hợp mô hình \"digital twin\" của cơ thể người phục vụ đào tạo nâng cao.",
"category": "Giáo dục - AR/VR"
},
{
"title": "Chương trình Y tế cộng đồng số cho vùng sâu vùng xa",
"description": "Kết hợp telehealth, trợ lý ảo y tế (agentic care) và AI sàng lọc sớm bệnh không lây (NCD) cho người dân vùng nông thôn, miền núi và hải đảo.",
"category": "Tác động xã hội - Telehealth"
},
{
"title": "Nền tảng nghiên cứu AI y sinh dùng chung (UMP AI Research Hub)",
"description": "Cung cấp hạ tầng GPU, kho dữ liệu y khoa ẩn danh, và công cụ phân tích AI cho giảng viên – nghiên cứu sinh – startup hợp tác nghiên cứu.",
"category": "Nghiên cứu - AI"
},
{
"title": "Hệ thống theo dõi và dự báo sức khỏe sinh viên & nhân viên y tế",
"description": "Ứng dụng phân tích dữ liệu và AI để phát hiện sớm stress, burnout, và vấn đề sức khỏe tâm thần trong cộng đồng sinh viên và nhân viên y tế.",
"category": "Tác động xã hội - Sức khỏe"
},
{
"title": "Vườn ươm khởi nghiệp công nghệ y sinh (MedTech Incubator)",
"description": "Hỗ trợ sinh viên, bác sĩ và giảng viên phát triển startup MedTech, HealthTech, AI y tế thông qua mentoring, quỹ seed và kết nối bệnh viện – doanh nghiệp.",
"category": "Khởi nghiệp - MedTech"
},
{
"title": "Hệ thống quản lý chất lượng đào tạo và kiểm định số",
"description": "Số hóa toàn bộ quy trình đảm bảo chất lượng nội bộ (IQA), đánh giá chương trình đào tạo, và chuẩn hóa theo tiêu chuẩn quốc tế (WFME, AUN-QA).",
"category": "Giáo dục - Quản lý chất lượng"
},
{
"title": "Nền tảng dữ liệu lớn phòng chống dịch và bệnh không lây",
"description": "Phân tích dữ liệu dịch tễ, môi trường, và hành vi để dự báo dịch bệnh, hỗ trợ Sở Y tế và Bộ Y tế trong ra quyết định chính sách.",
"category": "Nghiên cứu - Dịch tễ học"
},
{
"title": "Học viện Y học chính xác & Y học cá thể hóa",
"description": "Kết hợp dữ liệu gen, hình ảnh y khoa, lối sống và AI để nghiên cứu và ứng dụng điều trị cá thể hóa cho bệnh ung thư, tim mạch và bệnh mạn tính.",
"category": "Nghiên cứu - Y học chính xác"
}
]
async def main():
"""Add all UMP ideas to the database"""
print("Initializing Qdrant service...")
qdrant_service = get_qdrant_service()
print("Initializing collection...")
await qdrant_service.initialize_collection()
print(f"Adding {len(UMP_IDEAS)} ideas to the database...")
results = []
for i, idea in enumerate(UMP_IDEAS, 1):
try:
print(f"Adding idea {i}/{len(UMP_IDEAS)}: {idea['title']}")
result = await qdrant_service.add_idea(
title=idea['title'],
description=idea['description'],
category=idea['category']
)
results.append(result)
print(f"✓ Added: {result['id']}")
except Exception as e:
print(f"✗ Error adding idea {i}: {e}")
print(f"\n✓ Successfully added {len(results)}/{len(UMP_IDEAS)} ideas")
return results
if __name__ == "__main__":
asyncio.run(main())
be0/scripts/apply-migration-007.sh
Executable
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env bash
# Apply migration 007 (user_roles.admin_from_email_policy) to an EXISTING Postgres.
# initdb scripts in docker-entrypoint-initdb.d run only on first volume creation.
#
# Default (full SQL file): adds column, runs one-time policy DELETE/UPDATE (see
# be0/migrations/007_user_roles_email_policy_admin.sql before running on prod).
#
# Usage (from anywhere):
# ./be0/scripts/apply-migration-007.sh
# ./be0/scripts/apply-migration-007.sh --schema-only # only ADD COLUMN (safest repeat)
#
# On a remote host (SSH to be0/docker host, repo or copy of migrations present):
# export POSTGRES_CONTAINER=initiative-postgres POSTGRES_USER=initiative POSTGRES_DB=initiatives
# ./be0/scripts/apply-migration-007.sh
#
# From repo root (wrapper):
# ./scripts/apply-migration-007-postgres.sh
set -euo pipefail
SCHEMA_ONLY=0
for arg in "$@"; do
case "$arg" in
--schema-only) SCHEMA_ONLY=1 ;;
-h|--help)
sed -n '2,20p' "$0"
exit 0
;;
esac
done
BE0_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
SQL_FULL="$BE0_ROOT/migrations/007_user_roles_email_policy_admin.sql"
CONTAINER="${POSTGRES_CONTAINER:-initiative-postgres}"
PGUSER="${POSTGRES_USER:-initiative}"
PGDATABASE="${POSTGRES_DB:-initiatives}"
if ! docker info >/dev/null 2>&1; then
echo "error: Docker is not reachable (is the daemon running?)" >&2
exit 1
fi
if ! docker inspect "$CONTAINER" >/dev/null 2>&1; then
echo "error: container not found: $CONTAINER (set POSTGRES_CONTAINER)" >&2
exit 1
fi
if [[ "$(docker inspect -f '{{.State.Running}}' "$CONTAINER" 2>/dev/null || echo false)" != "true" ]]; then
echo "error: container is not running: $CONTAINER" >&2
exit 1
fi
apply_schema_only() {
docker exec -i "$CONTAINER" psql -U "$PGUSER" -d "$PGDATABASE" -v ON_ERROR_STOP=1 <<'SQL'
ALTER TABLE user_roles ADD COLUMN IF NOT EXISTS admin_from_email_policy BOOLEAN NOT NULL DEFAULT FALSE;
COMMENT ON COLUMN user_roles.admin_from_email_policy IS
'TRUE when admin was granted by email allow-list (AUTH_ADMIN_EMAILS). Reconciliation may DELETE this row if the user email is no longer in the list. FALSE preserves manually granted admin (future / exceptional).';
SQL
}
apply_full() {
if [[ ! -f "$SQL_FULL" ]]; then
echo "error: missing migration file: $SQL_FULL" >&2
exit 1
fi
docker exec -i "$CONTAINER" psql -U "$PGUSER" -d "$PGDATABASE" -v ON_ERROR_STOP=1 <"$SQL_FULL"
}
verify_column() {
local out
out="$(docker exec "$CONTAINER" psql -U "$PGUSER" -d "$PGDATABASE" -tAc \
"SELECT 1 FROM information_schema.columns WHERE table_schema = 'public' AND table_name = 'user_roles' AND column_name = 'admin_from_email_policy'")"
if [[ "${out//$'\r'/}" != "1" ]]; then
echo "error: verification failed: column admin_from_email_policy missing on public.user_roles" >&2
exit 1
fi
}
if (( SCHEMA_ONLY )); then
echo "Applying schema only (ADD COLUMN + COMMENT) → $CONTAINER / $PGDATABASE"
apply_schema_only
else
echo "Applying full 007_user_roles_email_policy_admin.sql → $CONTAINER / $PGDATABASE"
apply_full
fi
verify_column
echo "ok: user_roles.admin_from_email_policy is present; admin register/login should work with current be0."
be0/scripts/apply_initiative_migrations.py
+533
View File
@@ -0,0 +1,533 @@
"""
Apply idempotent SQL fixes when the DB volume predates newer migrations.
- ``008_audit_events.sql`` when ``audit_events`` is missing (older volumes never
ran ``docker-entrypoint-initdb.d`` for new files).
- ``009_backup_artifact_roles_storage_kind.sql`` when ``storage_kind`` is missing.
- ``010_user_staff_profiles.sql`` + ``011_academic_titles_vn.sql`` when
``academic_titles`` is missing (staff profile / register flow).
- ``013_email_verification.sql`` when ``email_verification_tokens`` is missing.
- ``014_registration_otp.sql`` when ``registration_otp_codes`` is missing.
Run automatically from entrypoint when ``INITIATIVE_DATABASE_URL`` is set.
Standalone:
INITIATIVE_DATABASE_URL=postgresql+asyncpg://user:pass@host:5432/dbname \\
python scripts/apply_initiative_migrations.py
"""
from __future__ import annotations
import asyncio
import os
import sys
from pathlib import Path
def _async_url_to_asyncpg_dsn(url: str) -> str:
u = url.strip()
if "+asyncpg" in u:
u = u.replace("postgresql+asyncpg://", "postgresql://", 1)
return u
def _strip_sql_comments(text: str) -> str:
lines: list[str] = []
for line in text.splitlines():
s = line.strip()
if s.startswith("--"):
continue
lines.append(line)
return "\n".join(lines)
def _split_sql_statements(text: str) -> list[str]:
"""Split on semicolons outside ``$$`` dollar-quoted blocks (008 uses ``DO $$``)."""
statements: list[str] = []
buf: list[str] = []
i = 0
n = len(text)
in_dollar = False
while i < n:
if text.startswith("$$", i):
in_dollar = not in_dollar
buf.append("$$")
i += 2
continue
ch = text[i]
if ch == ";" and not in_dollar:
stmt = "".join(buf).strip()
if stmt:
statements.append(stmt)
buf = []
i += 1
continue
buf.append(ch)
i += 1
tail = "".join(buf).strip()
if tail:
statements.append(tail)
return statements
async def _needs_audit_events_migration(conn) -> bool:
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'audit_events'
LIMIT 1
"""
)
return row is None
async def _needs_backup_migration(conn) -> bool:
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'application_artifacts'
AND column_name = 'storage_kind'
LIMIT 1
"""
)
return row is None
async def _needs_staff_profiles_migration(conn) -> bool:
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'academic_titles'
LIMIT 1
"""
)
return row is None
async def _needs_email_verification_migration(conn) -> bool:
"""True when verification tokens table is missing (013 also adds users.email_verified)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'email_verification_tokens'
LIMIT 1
"""
)
return row is None
async def _needs_registration_otp_migration(conn) -> bool:
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'registration_otp_codes'
LIMIT 1
"""
)
return row is None
async def _needs_document_templates_migration(conn) -> bool:
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'document_templates'
LIMIT 1
"""
)
return row is None
async def _needs_research_projects_migration(conn) -> bool:
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'research_projects'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_datasets_migration(conn) -> bool:
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'imagehub_datasets'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_segmentation_columns_migration(conn) -> bool:
"""True when imagehub_dataset_files lacks the segmentation-link columns (018)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'imagehub_dataset_files'
AND column_name = 'file_kind'
LIMIT 1
"""
)
return row is None
async def _needs_cloud_import_migration(conn) -> bool:
"""True when the cloud-import storage_methods table is absent (019)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'imagehub_storage_methods'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_stages_migration(conn) -> bool:
"""True when the dataset-stages table is absent (020)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'imagehub_dataset_stages'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_tasks_migration(conn) -> bool:
"""True when the per-file task-pipeline table is absent (021)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'imagehub_tasks'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_task_annotations_migration(conn) -> bool:
"""True when imagehub_tasks lacks the annotations column (022)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'imagehub_tasks'
AND column_name = 'annotations'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_members_migration(conn) -> bool:
"""True when the dataset-membership table is absent (023)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'imagehub_dataset_members'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_dataset_project_link_migration(conn) -> bool:
"""True when imagehub_datasets.research_project_id is absent (024)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'imagehub_datasets'
AND column_name = 'research_project_id'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_review_events_migration(conn) -> bool:
"""True when the task-review-events table is absent (025)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name = 'imagehub_task_review_events'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_folder_path_migration(conn) -> bool:
"""True when imagehub_dataset_files.folder_path is absent (026)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'imagehub_dataset_files'
AND column_name = 'folder_path'
LIMIT 1
"""
)
return row is None
async def _needs_imagehub_label_map_migration(conn) -> bool:
"""True when imagehub_datasets.label_map is absent (027)."""
row = await conn.fetchrow(
"""
SELECT 1
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'imagehub_datasets'
AND column_name = 'label_map'
LIMIT 1
"""
)
return row is None
async def _apply_sql_file(conn, path: Path, label: str) -> None:
body = _strip_sql_comments(path.read_text(encoding="utf-8"))
for stmt in _split_sql_statements(body):
await conn.execute(stmt)
print(f"apply_initiative_migrations: {label} applied.")
async def main() -> int:
raw_url = (os.environ.get("INITIATIVE_DATABASE_URL") or "").strip()
if not raw_url.lower().startswith("postgresql"):
print("apply_initiative_migrations: no PostgreSQL URL; skipping.", file=sys.stderr)
return 0
root = Path(__file__).resolve().parent.parent
m008 = root / "migrations" / "008_audit_events.sql"
m009 = root / "migrations" / "009_backup_artifact_roles_storage_kind.sql"
m010 = root / "migrations" / "010_user_staff_profiles.sql"
m011 = root / "migrations" / "011_academic_titles_vn.sql"
for p in (m008, m009, m010, m011):
if not p.is_file():
print(f"apply_initiative_migrations: missing {p}", file=sys.stderr)
return 1
import asyncpg
dsn = _async_url_to_asyncpg_dsn(raw_url)
conn = await asyncpg.connect(dsn, timeout=60)
try:
if await _needs_audit_events_migration(conn):
print("apply_initiative_migrations: applying 008_audit_events …")
await _apply_sql_file(conn, m008, "008_audit_events")
else:
print("apply_initiative_migrations: audit_events present; OK.")
if await _needs_backup_migration(conn):
print("apply_initiative_migrations: applying 009_backup_artifact_roles_storage_kind …")
await _apply_sql_file(conn, m009, "009_backup_artifact_roles_storage_kind")
else:
print("apply_initiative_migrations: application_artifacts.storage_kind present; OK.")
if await _needs_staff_profiles_migration(conn):
print("apply_initiative_migrations: applying 010_user_staff_profiles …")
await _apply_sql_file(conn, m010, "010_user_staff_profiles")
print("apply_initiative_migrations: applying 011_academic_titles_vn …")
await _apply_sql_file(conn, m011, "011_academic_titles_vn")
else:
print("apply_initiative_migrations: academic_titles present; OK.")
m013 = root / "migrations" / "013_email_verification.sql"
if not m013.is_file():
print(f"apply_initiative_migrations: missing {m013}", file=sys.stderr)
return 1
if await _needs_email_verification_migration(conn):
print("apply_initiative_migrations: applying 013_email_verification …")
await _apply_sql_file(conn, m013, "013_email_verification")
else:
print("apply_initiative_migrations: email_verification_tokens present; OK.")
m014 = root / "migrations" / "014_registration_otp.sql"
if not m014.is_file():
print(f"apply_initiative_migrations: missing {m014}", file=sys.stderr)
return 1
if await _needs_registration_otp_migration(conn):
print("apply_initiative_migrations: applying 014_registration_otp …")
await _apply_sql_file(conn, m014, "014_registration_otp")
else:
print("apply_initiative_migrations: registration_otp_codes present; OK.")
m015 = root / "migrations" / "015_document_templates.sql"
if not m015.is_file():
print(f"apply_initiative_migrations: missing {m015}", file=sys.stderr)
return 1
if await _needs_document_templates_migration(conn):
print("apply_initiative_migrations: applying 015_document_templates …")
await _apply_sql_file(conn, m015, "015_document_templates")
else:
print("apply_initiative_migrations: document_templates present; OK.")
m016 = root / "migrations" / "016_research_projects.sql"
if not m016.is_file():
print(f"apply_initiative_migrations: missing {m016}", file=sys.stderr)
return 1
if await _needs_research_projects_migration(conn):
print("apply_initiative_migrations: applying 016_research_projects …")
await _apply_sql_file(conn, m016, "016_research_projects")
else:
print("apply_initiative_migrations: research_projects present; OK.")
m017 = root / "migrations" / "017_imagehub_datasets.sql"
if not m017.is_file():
print(f"apply_initiative_migrations: missing {m017}", file=sys.stderr)
return 1
if await _needs_imagehub_datasets_migration(conn):
print("apply_initiative_migrations: applying 017_imagehub_datasets …")
await _apply_sql_file(conn, m017, "017_imagehub_datasets")
else:
print("apply_initiative_migrations: imagehub_datasets present; OK.")
m018 = root / "migrations" / "018_imagehub_segmentation_links.sql"
if not m018.is_file():
print(f"apply_initiative_migrations: missing {m018}", file=sys.stderr)
return 1
if await _needs_imagehub_segmentation_columns_migration(conn):
print("apply_initiative_migrations: applying 018_imagehub_segmentation_links …")
await _apply_sql_file(conn, m018, "018_imagehub_segmentation_links")
else:
print("apply_initiative_migrations: imagehub_dataset_files.file_kind present; OK.")
m019 = root / "migrations" / "019_imagehub_cloud_import.sql"
if not m019.is_file():
print(f"apply_initiative_migrations: missing {m019}", file=sys.stderr)
return 1
if await _needs_cloud_import_migration(conn):
print("apply_initiative_migrations: applying 019_imagehub_cloud_import …")
await _apply_sql_file(conn, m019, "019_imagehub_cloud_import")
else:
print("apply_initiative_migrations: imagehub_storage_methods present; OK.")
m020 = root / "migrations" / "020_imagehub_dataset_stages.sql"
if not m020.is_file():
print(f"apply_initiative_migrations: missing {m020}", file=sys.stderr)
return 1
if await _needs_imagehub_stages_migration(conn):
print("apply_initiative_migrations: applying 020_imagehub_dataset_stages …")
await _apply_sql_file(conn, m020, "020_imagehub_dataset_stages")
else:
print("apply_initiative_migrations: imagehub_dataset_stages present; OK.")
m021 = root / "migrations" / "021_imagehub_task_pipeline.sql"
if not m021.is_file():
print(f"apply_initiative_migrations: missing {m021}", file=sys.stderr)
return 1
if await _needs_imagehub_tasks_migration(conn):
print("apply_initiative_migrations: applying 021_imagehub_task_pipeline …")
await _apply_sql_file(conn, m021, "021_imagehub_task_pipeline")
else:
print("apply_initiative_migrations: imagehub_tasks present; OK.")
m022 = root / "migrations" / "022_imagehub_task_annotations.sql"
if not m022.is_file():
print(f"apply_initiative_migrations: missing {m022}", file=sys.stderr)
return 1
if await _needs_imagehub_task_annotations_migration(conn):
print("apply_initiative_migrations: applying 022_imagehub_task_annotations …")
await _apply_sql_file(conn, m022, "022_imagehub_task_annotations")
else:
print("apply_initiative_migrations: imagehub_tasks.annotations present; OK.")
m023 = root / "migrations" / "023_imagehub_dataset_members.sql"
if not m023.is_file():
print(f"apply_initiative_migrations: missing {m023}", file=sys.stderr)
return 1
if await _needs_imagehub_members_migration(conn):
print("apply_initiative_migrations: applying 023_imagehub_dataset_members …")
await _apply_sql_file(conn, m023, "023_imagehub_dataset_members")
else:
print("apply_initiative_migrations: imagehub_dataset_members present; OK.")
m024 = root / "migrations" / "024_imagehub_dataset_project_link.sql"
if not m024.is_file():
print(f"apply_initiative_migrations: missing {m024}", file=sys.stderr)
return 1
if await _needs_imagehub_dataset_project_link_migration(conn):
print("apply_initiative_migrations: applying 024_imagehub_dataset_project_link …")
await _apply_sql_file(conn, m024, "024_imagehub_dataset_project_link")
else:
print("apply_initiative_migrations: imagehub_datasets.research_project_id present; OK.")
m025 = root / "migrations" / "025_imagehub_task_review_events.sql"
if not m025.is_file():
print(f"apply_initiative_migrations: missing {m025}", file=sys.stderr)
return 1
if await _needs_imagehub_review_events_migration(conn):
print("apply_initiative_migrations: applying 025_imagehub_task_review_events …")
await _apply_sql_file(conn, m025, "025_imagehub_task_review_events")
else:
print("apply_initiative_migrations: imagehub_task_review_events present; OK.")
m026 = root / "migrations" / "026_imagehub_file_folder_path.sql"
if not m026.is_file():
print(f"apply_initiative_migrations: missing {m026}", file=sys.stderr)
return 1
if await _needs_imagehub_folder_path_migration(conn):
print("apply_initiative_migrations: applying 026_imagehub_file_folder_path …")
await _apply_sql_file(conn, m026, "026_imagehub_file_folder_path")
else:
print("apply_initiative_migrations: imagehub_dataset_files.folder_path present; OK.")
m027 = root / "migrations" / "027_imagehub_dataset_label_map.sql"
if not m027.is_file():
print(f"apply_initiative_migrations: missing {m027}", file=sys.stderr)
return 1
if await _needs_imagehub_label_map_migration(conn):
print("apply_initiative_migrations: applying 027_imagehub_dataset_label_map …")
await _apply_sql_file(conn, m027, "027_imagehub_dataset_label_map")
else:
print("apply_initiative_migrations: imagehub_datasets.label_map present; OK.")
return 0
except Exception as exc:
print(f"apply_initiative_migrations: FAILED: {exc}", file=sys.stderr)
if os.environ.get("INITIATIVE_DB_STRICT_MIGRATE", "").strip().lower() in ("1", "true", "yes"):
return 1
return 0
finally:
await conn.close()
if __name__ == "__main__":
raise SystemExit(asyncio.run(main()))
be0/scripts/repair_split_submission.py
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
"""
CLI: merge a mis-linked submission onto the real CASE-* initiative row and delete the orphan initiative.
Usage (dry-run — default, no writes):
cd be0
export INITIATIVE_DATABASE_URL="postgresql+asyncpg://user:pass@host:5432/initiatives"
python scripts/repair_split_submission.py --submission-id sub-d560fbb6f2944ec6
Apply (commits one transaction):
python scripts/repair_split_submission.py --submission-id sub-... --good-case CASE-YOURCODE --execute
Requires the same Postgres URL as the API (`INITIATIVE_DATABASE_URL` / `DATABASE_URL`).
"""
from __future__ import annotations
import argparse
import asyncio
import os
import sys
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.abspath(os.path.join(SCRIPT_DIR, ".."))
if ROOT not in sys.path:
sys.path.insert(0, ROOT)
async def _main_async() -> int:
p = argparse.ArgumentParser(description="Repair split submission / wrong initiative linkage.")
p.add_argument(
"--submission-id",
required=True,
help="submissionRecord.id (e.g. sub-d560fbb6f2944ec6)",
)
p.add_argument(
"--good-case",
dest="good_case",
default=None,
help="Explicit CASE-* code for the autosave row (recommended if owner has multiple drafts)",
)
p.add_argument(
"--execute",
action="store_true",
help="Apply changes (otherwise dry-run only)",
)
args = p.parse_args()
os.environ.setdefault("INITIATIVE_DATABASE_URL", os.getenv("DATABASE_URL") or "")
from src.initiative_db.engine import get_session, init_engine, is_postgres_enabled
from src.initiative_db.repair_split_submission import repair_submission_cross_initiative_merge
if not is_postgres_enabled():
print("Error: set INITIATIVE_DATABASE_URL=postgresql+asyncpg://.../initiatives", file=sys.stderr)
return 2
await init_engine()
async with get_session() as session:
report = await repair_submission_cross_initiative_merge(
session,
submission_record_id=args.submission_id.strip(),
good_case_code_explicit=(args.good_case or "").strip() or None,
dry_run=not args.execute,
)
lines = [
f"dry_run={report.dry_run}",
f"submission_record_id={report.submission_record_id}",
f"owner_id={report.owner_id or '(n/a)'}",
f"bad_case={report.bad_case_code or '(n/a)'}",
f"good_case={report.good_case_code or '(n/a)'}",
]
if report.skipped:
lines.append(f"SKIPPED: {report.skipped}")
lines.extend(report.actions)
print("\n".join(lines))
if args.execute and report.skipped:
return 3
return 0
def main() -> None:
raise SystemExit(asyncio.run(_main_async()))
if __name__ == "__main__":
main()
be0/src/__init__.py
View File
be0/src/__pycache__/MCP.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/Memory_Manager.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/Memory_Manager.cpython-38.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/Pdf_Manager.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/Response_Manager.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/__init__.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/__init__.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/__init__.cpython-38.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/admin_audit_routes.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/admin_audit_routes.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/admin_user_profile_routes.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/admin_user_profile_routes.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/audit.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/audit.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_api.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_api.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_credential_middleware.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_credential_middleware.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_jwt.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_jwt.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_mail.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_mail.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_rate_limit.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/auth_rate_limit.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/awareness_manager.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/chat_assistant.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/chat_assistant.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/compliance_verifier.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/compliance_verifier.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/config.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/config.cpython-38.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/main_new.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/memory_manager.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/schemas.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/staff_profile_domain.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/staff_profile_domain.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/structure_analysis.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/structure_analysis.cpython-313.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/template_manager.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/text_io.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/text_io.cpython-38.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/utils.cpython-311.pyc
BIN
View File
Binary file not shown.
be0/src/__pycache__/utils.cpython-313.pyc
BIN
View File
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More