AI SOC 分析师

AI-driven Security Operations Center analyst platform for intelligent threat detection and analysis.

Overview

This is a complete AI SOC Analyst platform that processes security alerts, builds threat graphs, and generates intelligent analysis reports. The system consists of a Vue 3 frontend and FastAPI backend with LLM-powered report generation.

Features

• Multi-format Alert Ingestion: Support for JSON and CSV alert formats from various SIEM/XDR products
• Intelligent Preprocessing: YAML-driven configuration for alert standardization and entity extraction
• Graph-based Scoring: Entity relationship modeling and attack chain risk assessment
• LLM-enhanced Reports: AI-powered threat analysis and actionable recommendations
• Interactive Pipeline: Complete workflow from data ingestion to report generation
• Real-time Dashboard: Vue 3 frontend for pipeline control and report visualization

Architecture

Frontend (Vue 3 + TypeScript + Vite)

• src/api.ts: HTTP client with mock/real API switching via VITE_USE_MOCK
• src/components/:
  • UploadPanel.vue: File upload with configurable data sources
  • PipelinePanel.vue: AI analysis pipeline control and monitoring
  • ReportList.vue: Threat reports display with filtering
  • ReportDetail.vue: Detailed analysis results drawer

Backend (FastAPI + Python)

• app/api_main.py: Main FastAPI application with all endpoints
• app/storage.py: In-memory storage for development and demo
• app/dsl/: Configuration-driven alert preprocessing engine
• app/edges.py: Graph construction from standardized alerts
• app/scoring.py: Entity and subgraph risk scoring system
• app/generate.py: Report generation with LLM integration
• app/llm.py: OpenAI client for AI-powered analysis

Configuration System (/configs)

• edge_scoring.yaml: Risk scoring weights and thresholds
• llm.yaml: LLM configuration and prompt templates
• splunk-es-alert-preprocessing.yaml: JSON alert preprocessing
• csv-alerts-preprocessing.yaml: CSV alert preprocessing

Quick Start

1. Install Dependencies

Backend:

pip install -r requirements.txt

Frontend:

npm install

2. Start the Backend

# From project root
python run_backend.py

Backend will be available at:

• API: http://localhost:8000
• Documentation: http://localhost:8000/docs

3. Start the Frontend

npm run dev

Frontend will be available at: http://localhost:5173

4. Test the Complete Pipeline

1. Upload Sample Data: Use the provided test-data/sample-alerts.json file
2. Select Data Source: Choose "Splunk ES Alert (JSON)"
3. Run Pipeline: Click "构建威胁图谱" → "风险评分计算" → "生成分析报告"
4. View Results: Check the generated reports in the reports section

Configuration

Environment Variables

LLM Integration (Optional):

export LLM_API_BASE="https://api.openai.com/v1"
export LLM_API_KEY="your-openai-api-key"

Frontend Mock Mode:

# .env file
VITE_USE_MOCK=false  # Use real backend API
VITE_USE_MOCK=true   # Use mock data for testing

Scoring Configuration

Edit configs/edge_scoring.yaml to customize:

• Severity weights (critical: 100, high: 70, etc.)
• Attack stage weights (credential_access: 90, etc.)
• Risk thresholds (entity: 70, subgraph: 300)
• Time decay and burst detection parameters

Data Processing Pipeline

1. Alert Ingestion: Upload JSON/CSV files with security alerts
2. Preprocessing: YAML-driven field extraction and entity standardization
3. Graph Construction: Build entity relationships from alert data
4. Risk Scoring: Calculate entity and subgraph risk scores
5. Report Generation: Create LLM-enhanced threat analysis reports

API Endpoints

Core APIs (/api)

• POST /ingest - Ingest alert data files
• POST /dry_run - Preview preprocessing results
• POST /graph/build - Build entity relationship graph
• POST /scoring/run - Calculate risk scores
• POST /reports/generate - Generate analysis reports
• GET /reports - List generated reports
• GET /reports/{id} - Get detailed report
• GET /status - System status and statistics

Sample Data

The project includes realistic test data (test-data/sample-alerts.json) covering:

• Phishing attack with malware execution
• Cloud credential theft and exfiltration
• Backdoor user creation and privilege escalation
• Kubernetes compromise and container control

This demonstrates a complete attack chain spanning endpoint, cloud, and container environments.

Development Features

Mock Mode

Enable frontend mock mode for development:

// Set in .env or environment
VITE_USE_MOCK=true

In-Memory Storage

The backend uses in-memory storage for development, making it easy to:

• Test different configurations quickly
• Reset state by restarting the service
• Run without external database dependencies

Hot Reload

Both frontend and backend support hot reload during development:

• Frontend: Vite dev server with HMR
• Backend: uvicorn with --reload flag

Production Deployment

For production deployment:

1. Database: Replace in-memory storage with PostgreSQL (models already defined)
2. Configuration: Set proper environment variables for LLM API keys
3. Security: Configure CORS, authentication, and rate limiting
4. Scaling: Use multiple worker processes for FastAPI backend

Advanced Configuration

Custom Data Sources

Add new data source configurations:

1. Create YAML file in configs/sources/
2. Define input mappings, transforms, and entity extraction rules
3. Update frontend source options in src/types.ts

Custom Scoring Rules

Modify configs/edge_scoring.yaml:

• Adjust severity and attack stage weights
• Configure time decay and burst detection
• Set entity-specific thresholds
• Customize cooldown and upgrade policies

LLM Customization

Edit configs/llm.yaml:

• Change model and API settings
• Customize prompt templates
• Adjust temperature and token limits
• Configure retry and fallback behavior

Troubleshooting

Backend won't start:

• Check Python dependencies: pip install -r requirements.txt
• Verify Python version: 3.10+ required

Frontend can't connect to backend:

• Ensure backend is running on port 8000
• Check CORS configuration in backend
• Try mock mode: VITE_USE_MOCK=true

LLM reports fail:

• Verify API key is set: echo $LLM_API_KEY
• Check API endpoint: echo $LLM_API_BASE
• System automatically falls back to template generation

No reports generated:

• Check if entities exceed thresholds in scoring config
• Review preprocessing results in dry-run mode
• Verify graph construction produces edges

For more help, check the API documentation at http://localhost:8000/docs when the backend is running.

Git 多仓库管理

本项目支持同时推送到多个远程仓库（GitHub + Gitea）。

方法一：使用 Git 别名（最简单）

# 推送到所有远程仓库
git pushall main

# 推送当前分支
git pushall

方法二：使用推送脚本

# Python 脚本（功能丰富）
python push_all.py

# Shell 脚本（快速便捷）
./push_all.sh

方法三：手动推送到指定仓库

# 同时推送到两个仓库
git push origin main && git push gitea main

配置远程仓库

# 查看当前远程仓库
git remote -v

# 添加新的远程仓库
git remote add gitea http://192.168.1.83:33000/gao/security_demo.git

# 删除远程仓库
git remote remove <remote-name>

远程仓库地址

• GitHub: [email protected]:BrunoGao/security_demo.git
• Gitea: http://192.168.1.83:33000/gao/security_demo.git

License

MIT License