ML Batch Demo Seed
Epic #1083 (Batch Inference) M1 의 시드 스크립트를 AKS prod 또는 Kind dev 에 주입하는 절차. Issue #1112 에서 apps/api/Dockerfile 에 COPY scripts/ /app/scripts/ 가 추가된 이후 가능.
빠른 시작 — AKS prod
kubectl --context aks-genos-prod exec -n gend deploy/gend-api -c gend-api \
-- python /app/scripts/seed_ml_batch_demo.py
성공 시 출력:
[seed_ml_batch_demo] inserting demo_credit_risk job + 5 dummy runs
[seed_ml_batch_demo] job_id=<uuid>
[seed_ml_batch_demo] run created status=succeeded
[seed_ml_batch_demo] run created status=succeeded
[seed_ml_batch_demo] run created status=succeeded
[seed_ml_batch_demo] run created status=failed
[seed_ml_batch_demo] run created status=running
[seed_ml_batch_demo] done
재실행 시 idempotent — (workspace_id, name) UNIQUE 제약으로 skip.
검증
TOKEN=$(curl -sk -X POST https://gend.genon.ai/auth/realms/gend/protocol/openid-connect/token \
-d "grant_type=password&client_id=gend-ui&username=admin&password=admin&scope=openid profile email" \
| python3 -c "import sys, json; print(json.load(sys.stdin)['access_token'])")
curl -sk https://gend.genon.ai/api/v1/ml/batch/jobs \
-H "Authorization: Bearer $TOKEN" | python3 -m json.tool
# expected: items[0].name == "demo_credit_risk"
curl -sk https://gend.genon.ai/api/v1/ml/batch/runs \
-H "Authorization: Bearer $TOKEN" | python3 -m json.tool
# expected: items length == 5, status mix (succeeded/failed/running)
Kind dev 환경
동일 명령 with --context kind-datax-local:
kubectl --context kind-datax-local exec -n gend deploy/gend-api -c gend-api \
-- python /app/scripts/seed_ml_batch_demo.py
시드 데이터 명세
| 항목 | 값 | 비고 |
|---|---|---|
| Job 명 | demo_credit_risk | UNIQUE per workspace |
| Model | credit_risk@Production | alias-only 정책 |
| Source asset | iceberg.silver.demo_customers | M2 에서 실제 Iceberg 시드 추가 |
| Output table | iceberg.gold.predictions_demo_credit_risk | gend.layer='gold' |
| Schedule | 0 1 * * * (01:00 UTC daily) | DefaultScheduleStatus.STOPPED |
| Executor | pandas | M3 에서 spark 옵션 |
| Drift check | False | M1 hard-guard |
| Runs | 5건 (succeeded × 3, failed × 1, running × 1) | dummy timestamps |
트러블슈팅
seed_ml_batch_demo.py: No such file or directory
원인: 이미지가 Issue #1112 머지 전 빌드된 버전. apps/api/Dockerfile 에 COPY scripts/ /app/scripts/ 가 없는 상태.
해결: 최신 이미지로 rollout:
TAG="$(cat VERSION)-dev.$(date +%Y%m%d)-$(git rev-parse --short HEAD)" # ADR-0036 태그 규칙
az acr build --registry genosprodacr --platform linux/amd64 \
--build-arg GEND_BUILD_SHA=$(git rev-parse --short HEAD) \
--build-arg GEND_BUILD_DATE=$(date -u +%Y-%m-%dT%H:%MZ) \
--image gend-api:$TAG --file apps/api/Dockerfile .
kubectl --context aks-genos-prod set image deploy/gend-api -n gend \
gend-api=genosprodacr.azurecr.io/gend-api:$TAG
kubectl --context aks-genos-prod rollout status deploy/gend-api -n gend
workspace_id is None 에러
원인: 사용자 gend workspace 가 부재 (#1018 M1 backfill 미수행).
해결: workspace 우선 생성:
kubectl --context aks-genos-prod exec -n gend deploy/gend-api -c gend-api -- \
python -c "from gend_api.db.models import Workspace; ..."
또는 seed 스크립트가 None workspace 로도 동작 (M1 staged backfill posture, feedback_workspace_isolation_status).
UNIQUE constraint violation
이미 시드가 적재됨. 재주입이 필요하면 prior 데이터 삭제 후 재실행:
kubectl --context aks-genos-prod exec -n gend deploy/postgresql -- \
psql -U gend -d gend -c "DELETE FROM ml_batch_run WHERE job_id IN (SELECT id FROM ml_batch_job WHERE name = 'demo_credit_risk');"
kubectl --context aks-genos-prod exec -n gend deploy/postgresql -- \
psql -U gend -d gend -c "DELETE FROM ml_batch_job WHERE name = 'demo_credit_risk';"