GitHub Actions CI/CD 架构文档
概览
本文档描述了 ljwx-shiti (智能错题本) 项目的 CI/CD 流程和 K8s 部署架构。
关键组件:
- 代码仓库: GitHub ([email protected]:BrunoGao/ljwx-shiti.git)
- 部署仓库: GitHub ([email protected]:BrunoGaoSZ/ljwx-deploy.git)
- 镜像仓库: Harbor (https://harbor.omniverseai.net)
- K8s 集群: 本地 K8s (基础设施在 infra namespace)
- GitOps 工具: ArgoCD
1. K8s 集群架构
1.1 Infra Namespace 基础设施
所有共享的基础设施服务都部署在 infra namespace 中，应用通过跨 namespace 服务发现访问。
PostgreSQL 服务
# 内部访问 (应用使用)
postgres-infra-postgresql.infra.svc.cluster.local:5432
ClusterIP: 192.168.194.246
# Headless Service (StatefulSet 使用)
postgres-infra-postgresql-hl.infra.svc.cluster.local
# 外部访问 (开发调试)
postgres-external (NodePort: 32432)
# 部署方式
StatefulSet: postgres-infra-postgresql-0 (1 副本)
Redis 服务
# 主节点访问 (应用使用)
redis-infra-master.infra.svc.cluster.local:6379
ClusterIP: 192.168.194.187
# Headless Service
redis-infra-headless.infra.svc.cluster.local
# 外部访问 (开发调试)
redis-external (NodePort: 32379)
# 部署方式
StatefulSet: redis-infra-master-0 (1 副本)
MySQL 服务
# 内部访问 (应用使用)
mysql-infra.infra.svc.cluster.local:3306
Headless Service
# 外部访问 (开发调试)
mysql-external (NodePort: 32306)
# 部署方式
StatefulSet: mysql-infra-0 (1 副本)
其他基础设施
- MinIO: 对象存储 (S3 兼容)
- ClickHouse: 分析数据库
- Redpanda: Kafka 兼容消息队列
- Ingress Nginx: 入口控制器
1.2 跨 Namespace 服务访问
DNS 格式: <service-name>.<namespace>.svc.cluster.local
应用配置示例:
# Deployment 环境变量
env:
- name: DATABASE_URL
value: postgresql+asyncpg://user:[email protected]:5432/wrongbook
- name: REDIS_URL
value: redis://redis-infra-master.infra.svc.cluster.local:6379/0
InitContainer 等待依赖:
initContainers:
- name: wait-for-postgres
image: busybox:1.35
command:
- sh
- -c
- |
until nc -z postgres-infra-postgresql.infra.svc.cluster.local 5432; do
echo "Waiting for PostgreSQL..."
sleep 2
done
2. CI/CD 完整流程
2.1 流程图
┌─────────────────────────────────────────────────────────────┐
│ 1. 开发者推送代码 │
│ git push origin main │
└────────────────┬────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 2. GitHub Actions 触发 │
│ Workflow: .github/workflows/ci.yml │
│ - Lint & Test (pytest, ruff) │
│ - Build Docker Image │
│ - Push to Harbor │
│ Image: harbor.omniverseai.net/ljwx/ljwx-shiti:${SHA} │
└────────────────┬────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 3. 自动更新部署仓库 │
│ - Clone ljwx-deploy │
│ - Update kustomization.yaml │
│ Path: apps/ljwx-shiti/overlays/prod/kustomization.yaml│
│ - Create PR to main branch │
└────────────────┬────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 4. PR 合并到 ljwx-deploy/main │
│ (手动审核或自动合并) │
└────────────────┬────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 5. ArgoCD 自动同步 (30秒内) │
│ - 检测 Git 变更 │
│ - 应用 Kustomize 配置 │
│ - 滚动更新 Deployment │
│ - 健康检查 (liveness/readiness/startup probes) │
└────────────────┬────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 6. 应用运行在 K8s 集群 │
│ - 连接 infra namespace 的 PostgreSQL/Redis │
│ - 通过 Ingress 暴露服务 │
│ - Prometheus 监控 (ServiceMonitor) │
└─────────────────────────────────────────────────────────────┘
2.2 GitHub Actions Workflow 配置
必需的 GitHub Secrets
在 GitHub 仓库设置中配置以下 Secrets:
# Harbor 镜像仓库凭据
HARBOR_USERNAME=admin
HARBOR_PASSWORD=<harbor-password>
# 部署仓库访问 Token
DEPLOY_REPO_TOKEN=<github-personal-access-token>
# Token 权限: repo, write:repository
Workflow 文件结构
# .github/workflows/ci.yml
name: CI/CD Pipeline
on:
push:
branches: [main, master]
pull_request:
branches: [main, master]
env:
HARBOR_HOST: harbor.omniverseai.net
HARBOR_PROJECT: ljwx
IMAGE_NAME: ljwx-shiti
jobs:
test:
# 运行测试
build-and-push:
# 构建并推送镜像
update-deploy:
# 更新部署配置
2.3 镜像构建策略
Backend (Python FastAPI)
# backend/Dockerfile
FROM python:3.11-slim
WORKDIR /app
# 安装系统依赖
RUN apt-get update && apt-get install -y \
gcc \
postgresql-client \
&& rm -rf /var/lib/apt/lists/*
# 安装 Python 依赖
COPY backend/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# 复制应用代码
COPY backend/ .
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
构建命令
# 单架构构建
docker build -t harbor.omniverseai.net/ljwx/ljwx-shiti:${GITHUB_SHA} \
-f backend/Dockerfile .
docker push harbor.omniverseai.net/ljwx/ljwx-shiti:${GITHUB_SHA}
# 多架构构建 (可选)
docker buildx build \
--platform linux/amd64,linux/arm64 \
--push \
-t harbor.omniverseai.net/ljwx/ljwx-shiti:${GITHUB_SHA} \
-f backend/Dockerfile .
3. ljwx-deploy 仓库结构
3.1 目录结构
ljwx-deploy/
├── apps/
│ └── ljwx-shiti/
│ ├── base/ # 基础配置
│ │ ├── namespace.yaml # Namespace 定义
│ │ ├── configmap.yaml # 非敏感配置
│ │ ├── deployment.yaml # Deployment 配置
│ │ ├── service.yaml # Service 配置
│ │ ├── ingress.yaml # Ingress 配置
│ │ ├── servicemonitor.yaml # Prometheus 监控
│ │ └── kustomization.yaml # Kustomize 配置
│ └── overlays/
│ ├── dev/ # 开发环境
│ │ └── kustomization.yaml
│ ├── staging/ # 预发布环境
│ │ └── kustomization.yaml
│ └── prod/ # 生产环境
│ └── kustomization.yaml
├── argocd-apps/
│ ├── 80-ljwx-shiti-dev.yaml # ArgoCD Application (dev)
│ ├── 81-ljwx-shiti-staging.yaml # ArgoCD Application (staging)
│ └── 82-ljwx-shiti-prod.yaml # ArgoCD Application (prod)
└── cluster/
└── namespace-ljwx-shiti.yaml # 集群级别 Namespace 配置
3.2 Kustomize 配置
base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: ljwx-shiti
resources:
- namespace.yaml
- configmap.yaml
- deployment.yaml
- service.yaml
- ingress.yaml
- servicemonitor.yaml
commonLabels:
app: ljwx-shiti
managed-by: kustomize
# 镜像配置 (由 CI 自动更新)
images:
- name: harbor.omniverseai.net/ljwx/ljwx-shiti
newTag: latest # CI 会更新为具体的 commit SHA
generatorOptions:
disableNameSuffixHash: true
overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: ljwx-shiti
resources:
- ../../base
# 生产环境特定配置
patchesStrategicMerge:
- |-
apiVersion: apps/v1
kind: Deployment
metadata:
name: ljwx-shiti
spec:
replicas: 3 # 生产环境 3 副本
template:
spec:
containers:
- name: ljwx-shiti
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
memory: 2Gi
# 镜像标签 (由 CI 自动更新)
images:
- name: harbor.omniverseai.net/ljwx/ljwx-shiti
newTag: abc123def # CI 更新为最新的 commit SHA
3.3 Deployment 配置示例
# apps/ljwx-shiti/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ljwx-shiti
namespace: ljwx-shiti
labels:
app: ljwx-shiti
component: backend
spec:
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 0
maxSurge: 1
selector:
matchLabels:
app: ljwx-shiti
template:
metadata:
labels:
app: ljwx-shiti
component: backend
spec:
# 等待 PostgreSQL 就绪
initContainers:
- name: wait-for-postgres
image: busybox:1.35
command:
- sh
- -c
- |
until nc -z postgres-infra-postgresql.infra.svc.cluster.local 5432; do
echo "Waiting for PostgreSQL..."
sleep 2
done
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 100m
memory: 128Mi
containers:
- name: ljwx-shiti
image: harbor.omniverseai.net/ljwx/ljwx-shiti:latest
imagePullPolicy: Always
ports:
- name: http
containerPort: 8000
protocol: TCP
# 环境变量 - 从 ConfigMap
envFrom:
- configMapRef:
name: ljwx-shiti-config
# 环境变量 - 从 Secret (敏感信息)
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: ljwx-shiti-secret
key: DATABASE_URL
- name: REDIS_URL
valueFrom:
secretKeyRef:
name: ljwx-shiti-secret
key: REDIS_URL
- name: ANTHROPIC_AUTH_TOKEN
valueFrom:
secretKeyRef:
name: ljwx-shiti-secret
key: ANTHROPIC_AUTH_TOKEN
- name: JWT_SECRET
valueFrom:
secretKeyRef:
name: ljwx-shiti-secret
key: JWT_SECRET
# 资源配置
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
# 健康检查
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 60
periodSeconds: 30
timeoutSeconds: 10
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
startupProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 30
restartPolicy: Always
terminationGracePeriodSeconds: 60
3.4 ArgoCD Application 配置
# argocd-apps/80-ljwx-shiti-dev.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: ljwx-shiti-dev
namespace: argocd
spec:
project: default
source:
repoURL: [email protected]:BrunoGaoSZ/ljwx-deploy.git
targetRevision: main
path: apps/ljwx-shiti/overlays/dev
destination:
server: https://kubernetes.default.svc
namespace: ljwx-shiti
syncPolicy:
automated:
prune: true # 自动删除不在 Git 中的资源
selfHeal: true # 自动修复手动修改
syncOptions:
- CreateNamespace=true
4. Secret 管理
4.1 创建 Secret
# 在 K8s 集群中创建 Secret (不提交到 Git)
kubectl create secret generic ljwx-shiti-secret \
--namespace=ljwx-shiti \
--from-literal=DATABASE_URL='postgresql+asyncpg://wrongbook:[email protected]:5432/wrongbook' \
--from-literal=REDIS_URL='redis://redis-infra-master.infra.svc.cluster.local:6379/0' \
--from-literal=ANTHROPIC_AUTH_TOKEN='sk-ant-xxx' \
--from-literal=JWT_SECRET='your-secret-key' \
--from-literal=MINIO_ENDPOINT='minio.infra.svc.cluster.local:9000' \
--from-literal=MINIO_ACCESS_KEY='minioadmin' \
--from-literal=MINIO_SECRET_KEY='minioadmin'
4.2 Secret 最佳实践
- ❌ 不要: 将 Secret 提交到 Git 仓库
- ✅ 推荐: 使用 External Secrets Operator (未来迁移)
- ✅ 推荐: 使用 Sealed Secrets
- ✅ 当前: 手动在集群中创建 Secret
5. 监控和可观测性
5.1 Prometheus 监控
# apps/ljwx-shiti/base/servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: ljwx-shiti
namespace: ljwx-shiti
spec:
selector:
matchLabels:
app: ljwx-shiti
endpoints:
- port: http
path: /metrics
interval: 30s
5.2 日志收集
# Deployment annotations
annotations:
fluentbit.io/parser: json
5.3 追踪 (OpenTelemetry)
# ConfigMap 环境变量
OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector.tracing.svc.cluster.local:4318"
OTEL_SERVICE_NAME: "ljwx-shiti"
6. 常用命令
6.1 本地开发
# 启动本地环境
docker-compose up -d
# 运行测试
cd backend && pytest -v
# 运行 linter
cd backend && ruff check .
# 构建镜像
docker build -t ljwx-shiti:dev -f backend/Dockerfile .
6.2 K8s 操作
# 查看应用状态
kubectl get pods -n ljwx-shiti
kubectl get svc -n ljwx-shiti
kubectl get ingress -n ljwx-shiti
# 查看日志
kubectl logs -n ljwx-shiti -l app=ljwx-shiti -f
# 查看 Pod 详情
kubectl describe pod -n ljwx-shiti <pod-name>
# 进入容器
kubectl exec -it -n ljwx-shiti <pod-name> -- /bin/bash
# 查看 Secret
kubectl get secret -n ljwx-shiti ljwx-shiti-secret -o yaml
# 测试数据库连接
kubectl exec -it -n ljwx-shiti <pod-name> -- \
psql postgresql://wrongbook:[email protected]:5432/wrongbook
6.3 ArgoCD 操作
# 查看 Application 状态
kubectl get application -n argocd ljwx-shiti-dev
# 手动触发同步
kubectl patch application ljwx-shiti-dev -n argocd \
--type merge \
-p '{"operation":{"initiatedBy":{"username":"admin"},"sync":{"revision":"HEAD"}}}'
# 查看同步历史
kubectl describe application -n argocd ljwx-shiti-dev
6.4 Harbor 操作
# 登录 Harbor
docker login harbor.omniverseai.net
# 查看镜像
docker pull harbor.omniverseai.net/ljwx/ljwx-shiti:latest
# 查看镜像详情
docker buildx imagetools inspect harbor.omniverseai.net/ljwx/ljwx-shiti:latest
7. 故障排查
7.1 Pod 无法启动
# 查看 Pod 事件
kubectl describe pod -n ljwx-shiti <pod-name>
# 查看 init container 日志
kubectl logs -n ljwx-shiti <pod-name> -c wait-for-postgres
# 测试数据库连接
kubectl exec -n ljwx-shiti <pod-name> -- \
nc -zv postgres-infra-postgresql.infra.svc.cluster.local 5432
7.2 镜像拉取失败
# 检查 imagePullSecrets
kubectl get secret -n ljwx-shiti regcred
# 手动拉取镜像测试
docker pull harbor.omniverseai.net/ljwx/ljwx-shiti:latest
7.3 健康检查失败
# 查看健康检查端点
kubectl exec -n ljwx-shiti <pod-name> -- \
curl -v http://localhost:8000/health
# 查看应用日志
kubectl logs -n ljwx-shiti <pod-name> --tail=100
7.4 ArgoCD 同步失败
# 查看 Application 状态
kubectl get application -n argocd ljwx-shiti-dev -o yaml
# 查看 ArgoCD 日志
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller
8. 参考资料
8.1 相关仓库
- 应用仓库: [email protected]:BrunoGao/ljwx-shiti.git
- 部署仓库: [email protected]:BrunoGaoSZ/ljwx-deploy.git
- Harbor: https://harbor.omniverseai.net/harbor/projects/ljwx
8.2 文档
- 项目文档:
/docs/ - API 文档:
/docs/API.md - 架构文档:
/docs/architecture.md - 部署文档:
/docs/DEPLOYMENT.md
8.3 工具
- Kustomize: https://kustomize.io/
- ArgoCD: https://argo-cd.readthedocs.io/
- Harbor: https://goharbor.io/
最后更新: 2026-02-26 维护者: DevOps Team