This is a complete AI SOC Analyst platform that processes security alerts, builds threat graphs, and generates intelligent analysis reports. The system consists of a Vue 3 frontend and FastAPI backend with LLM-powered report generation.

Features

Multi-format Alert Ingestion: Support for JSON and CSV alert formats from various SIEM/XDR products
Intelligent Preprocessing: YAML-driven configuration for alert standardization and entity extraction
Graph-based Scoring: Entity relationship modeling and attack chain risk assessment
LLM-enhanced Reports: AI-powered threat analysis and actionable recommendations
Interactive Pipeline: Complete workflow from data ingestion to report generation
Real-time Dashboard: Vue 3 frontend for pipeline control and report visualization

Architecture

Frontend (Vue 3 + TypeScript + Vite)

src/api.ts: HTTP client with mock/real API switching via VITE_USE_MOCK
src/components/:
- UploadPanel.vue: File upload with configurable data sources
- PipelinePanel.vue: AI analysis pipeline control and monitoring
- ReportList.vue: Threat reports display with filtering
- ReportDetail.vue: Detailed analysis results drawer

Backend (FastAPI + Python)

app/api_main.py: Main FastAPI application with all endpoints
app/storage.py: In-memory storage for development and demo
app/dsl/: Configuration-driven alert preprocessing engine
app/edges.py: Graph construction from standardized alerts
app/scoring.py: Entity and subgraph risk scoring system
app/generate.py: Report generation with LLM integration
app/llm.py: OpenAI client for AI-powered analysis

Configuration System (`/configs`)

edge_scoring.yaml: Risk scoring weights and thresholds
llm.yaml: LLM configuration and prompt templates
splunk-es-alert-preprocessing.yaml: JSON alert preprocessing
csv-alerts-preprocessing.yaml: CSV alert preprocessing

Quick Start

1. Install Dependencies

Backend:

pip install -r requirements.txt

Frontend:

npm install

2. Start the Backend

# From project root
python run_backend.py

Backend will be available at:

API: http://localhost:8000
Documentation: http://localhost:8000/docs

3. Start the Frontend

npm run dev

Frontend will be available at: http://localhost:5173

4. Test the Complete Pipeline

Upload Sample Data: Use the provided test-data/sample-alerts.json file
Select Data Source: Choose "Splunk ES Alert (JSON)"
Run Pipeline: Click "构建威胁图谱" → "风险评分计算" → "生成分析报告"
View Results: Check the generated reports in the reports section

Configuration

Environment Variables

LLM Integration (Optional):

export LLM_API_BASE="https://api.openai.com/v1"
export LLM_API_KEY="your-openai-api-key"

Frontend Mock Mode:

# .env file
VITE_USE_MOCK=false  # Use real backend API
VITE_USE_MOCK=true   # Use mock data for testing

Scoring Configuration

Edit configs/edge_scoring.yaml to customize:

Severity weights (critical: 100, high: 70, etc.)
Attack stage weights (credential_access: 90, etc.)
Risk thresholds (entity: 70, subgraph: 300)
Time decay and burst detection parameters

Data Processing Pipeline

Alert Ingestion: Upload JSON/CSV files with security alerts
Preprocessing: YAML-driven field extraction and entity standardization
Graph Construction: Build entity relationships from alert data
Risk Scoring: Calculate entity and subgraph risk scores
Report Generation: Create LLM-enhanced threat analysis reports

API Endpoints

Core APIs (`/api`)

POST /ingest - Ingest alert data files
POST /dry_run - Preview preprocessing results
POST /graph/build - Build entity relationship graph
POST /scoring/run - Calculate risk scores
POST /reports/generate - Generate analysis reports
GET /reports - List generated reports
GET /reports/{id} - Get detailed report
GET /status - System status and statistics

Sample Data

The project includes realistic test data (test-data/sample-alerts.json) covering:

Phishing attack with malware execution
Cloud credential theft and exfiltration
Backdoor user creation and privilege escalation
Kubernetes compromise and container control

This demonstrates a complete attack chain spanning endpoint, cloud, and container environments.

Development Features

Mock Mode

Enable frontend mock mode for development:

// Set in .env or environment
VITE_USE_MOCK=true

In-Memory Storage

The backend uses in-memory storage for development, making it easy to:

Test different configurations quickly
Reset state by restarting the service
Run without external database dependencies

Hot Reload

Both frontend and backend support hot reload during development:

Frontend: Vite dev server with HMR
Backend: uvicorn with --reload flag

Production Deployment

For production deployment:

Database: Replace in-memory storage with PostgreSQL (models already defined)
Configuration: Set proper environment variables for LLM API keys
Security: Configure CORS, authentication, and rate limiting
Scaling: Use multiple worker processes for FastAPI backend

Advanced Configuration

Custom Data Sources

Add new data source configurations:

Create YAML file in configs/sources/
Define input mappings, transforms, and entity extraction rules
Update frontend source options in src/types.ts

Custom Scoring Rules

Modify configs/edge_scoring.yaml:

Adjust severity and attack stage weights
Configure time decay and burst detection
Set entity-specific thresholds
Customize cooldown and upgrade policies

LLM Customization

Edit configs/llm.yaml:

Change model and API settings
Customize prompt templates
Adjust temperature and token limits
Configure retry and fallback behavior

Troubleshooting

Backend won't start:

Check Python dependencies: pip install -r requirements.txt
Verify Python version: 3.10+ required

Frontend can't connect to backend:

Ensure backend is running on port 8000
Check CORS configuration in backend
Try mock mode: VITE_USE_MOCK=true

LLM reports fail:

Verify API key is set: echo $LLM_API_KEY
Check API endpoint: echo $LLM_API_BASE
System automatically falls back to template generation

No reports generated:

Check if entities exceed thresholds in scoring config
Review preprocessing results in dry-run mode
Verify graph construction produces edges

For more help, check the API documentation at http://localhost:8000/docs when the backend is running.

License

MIT License

README.md

AI SOC 分析师

Overview