ljwx-stock/KEY_FILES_REFERENCE.md

Stock Recommendation System - Key Files Reference

Current Module Files

Core Data Processing Modules

• /root/work/stock/backend/app/modules/data_fetcher.py (355 lines)

  • Fetch OHLCV data from TuShare, yfinance
  • Get real-time quotes
  • Fetch fundamental data

• /root/work/stock/backend/app/modules/feature_engineer.py (244 lines)

  • Generate 40+ technical indicators
  • Normalize features for ML

• /root/work/stock/backend/app/modules/ml_trainer.py (251 lines)

  • Train XGBoost/LightGBM models
  • Predict stock scores (0-100)

• /root/work/stock/backend/app/modules/llm_ranker.py (296 lines)

  • Generate explanations via Claude API
  • Select top recommendations

Supporting Modules

• /root/work/stock/backend/app/modules/fundamental_data.py (468 lines)

  • [Phase 1 Addition] Fetch real TuShare data
  • Calculate derived metrics (ROE, debt ratio, PEG)
  • Interpretation layer (低估/合理/高估)

• /root/work/stock/backend/app/modules/backtester.py (250 lines)

  • Validate strategy with historical data
  • Calculate performance metrics

• /root/work/stock/backend/app/modules/cache_manager.py (127 lines)

  • Redis caching operations
  • TTL management

• /root/work/stock/backend/app/modules/openbb_provider.py (384 lines)

  • Alternative data source
  • Fallback when TuShare unavailable

API Routes

• /root/work/stock/backend/app/routes/ai_assistant.py (397 lines)

  • Chat endpoint with real data integration
  • Context-aware responses
  • Health check

• /root/work/stock/backend/app/routes/recommendations.py

  • GET /api/recommendations
  • GET /api/recommendations/{symbol}

• /root/work/stock/backend/app/routes/analysis.py

  • POST /api/analysis/signals/{symbol}

• /root/work/stock/backend/app/routes/backtest.py

  • POST /api/backtest

Configuration & Database

• /root/work/stock/backend/app/config.py

  • Settings management
  • Environment variables

• /root/work/stock/backend/app/database.py

  • Database connection

• /root/work/stock/backend/app/models.py (if exists)

  • SQLAlchemy models

Documentation Files

• /root/work/stock/docs/SYSTEM_OVERVIEW.md

  • High-level architecture
  • Tech stack overview
  • Data flow diagrams

• /root/work/stock/docs/EXECUTION_PLAN.md

  • 8-phase implementation plan
  • Timeline and checkpoints

• /root/work/stock/docs/MODULE_SPEC.md

  • Module specifications
  • API contracts

• /root/work/stock/PHASE_1_IMPLEMENTATION_COMPLETE.md

  • Phase 1 completion report
  • Test results
  • Performance metrics

New Files to Create (Phase 2)

Priority 1 (Week 1)

• /root/work/stock/backend/app/modules/sentiment_fetcher.py
  • Market sentiment data aggregation
  • Sina, Tencent, East Money integrations

Priority 2 (Week 2)

• /root/work/stock/backend/app/modules/announcement_crawler.py
  • Corporate announcement extraction
  • Impact scoring

Priority 3 (Week 2)

• /root/work/stock/backend/app/modules/sentiment_analyzer.py
  • NLP-based sentiment analysis
  • Keyword extraction

Priority 4 (Week 3)

• /root/work/stock/backend/app/modules/market_context_builder.py
  • Aggregate all context data
  • Generate market overview

New Files to Create (Phase 3)

• /root/work/stock/backend/app/modules/knowledge_base.py

  • Vector database integration
  • Document embeddings

• /root/work/stock/backend/app/modules/cache_optimizer.py

  • Multi-level caching
  • Adaptive TTL

• /root/work/stock/backend/app/modules/performance_monitor.py

  • Metrics tracking
  • Prometheus integration

• /root/work/stock/backend/app/modules/batch_processor.py

  • Parallel data processing
  • Batch operations

Root Directory Structure

/root/work/stock/
├── backend/
│   ├── main.py
│   ├── requirements.txt
│   ├── Dockerfile
│   └── app/
│       ├── __init__.py
│       ├── config.py
│       ├── database.py
│       ├── routes/
│       │   ├── ai_assistant.py (397 lines)
│       │   ├── recommendations.py
│       │   ├── analysis.py
│       │   └── backtest.py
│       └── modules/
│           ├── data_fetcher.py (355 lines)
│           ├── feature_engineer.py (244 lines)
│           ├── ml_trainer.py (251 lines)
│           ├── llm_ranker.py (296 lines)
│           ├── fundamental_data.py (468 lines)
│           ├── backtester.py (250 lines)
│           ├── cache_manager.py (127 lines)
│           └── openbb_provider.py (384 lines)
├── frontend/
│   ├── src/
│   │   ├── components/
│   │   ├── views/
│   │   └── services/
│   └── package.json
├── docs/
│   ├── SYSTEM_OVERVIEW.md
│   ├── EXECUTION_PLAN.md
│   ├── MODULE_SPEC.md
│   └── API_REFERENCE.md
├── PHASE_1_IMPLEMENTATION_COMPLETE.md
├── CODEBASE_ANALYSIS.md (newly generated)
├── docker-compose.yml
└── README.md

Statistics

Current Code:

• Total lines of code: ~2000+
• Number of modules: 8
• API endpoints: 10+
• Data cache patterns: 6

Phase 2 Additions:

• New modules: 4 (~1100 lines)
• New API endpoints: 3
• New database tables: 3
• Estimated effort: 28 hours

Phase 3 Additions:

• New modules: 4 (~1230 lines)
• Database optimizations: Yes
• Infrastructure improvements: Yes
• Estimated effort: 34 hours

API Endpoints Summary

Current (Phase 1)

• GET /api/recommendations - Top 10 stocks
• GET /api/recommendations/{symbol} - Stock detail
• POST /api/analysis/signals/{symbol} - Technical analysis
• POST /api/backtest - Backtest strategy
• POST /api/ai/chat - AI assistant
• GET /api/ai/health - Health check

Phase 2 (New)

• GET /api/market/sentiment - Market sentiment
• GET /api/announcements/{symbol} - Stock announcements
• Updated: GET /api/recommendations (with market_context)

Phase 3 (New)

• GET /api/knowledge/similar - Similar insights
• POST /api/analysis/batch - Batch analysis
• Updated: All endpoints with improved performance

Testing Entry Points

Unit Tests

• Test data_fetcher.py with mock TuShare API
• Test feature_engineer.py with sample OHLCV
• Test ml_trainer.py with dummy data
• Test cache_manager.py with Redis mock

Integration Tests

• Test full recommendation generation flow
• Test AI assistant with real/mock data
• Test backtest with historical data

E2E Tests

• Test frontend-backend communication
• Test WebSocket updates
• Test AI chat interaction

Performance Targets

Current (Phase 1):

• Recommendation: 1-3 seconds
• AI assistant (cache hit): ~100ms
• AI assistant (cache miss): ~1.5-2 seconds

Phase 3 Target:

• Recommendation: <500ms (p95)
• Stock analysis: <1 second (p95)
• Cache hit rate: >80% for quotes
• System throughput: 1000 req/s

Configuration Keys

# API Configuration
APP_ENV=development
DEBUG=True
API_HOST=0.0.0.0
API_PORT=8000

# API Keys
OPENROUTER_KEY=sk-or-v1-...
TUSHARE_TOKEN=...
OPENBB_TOKEN=...

# Database
DATABASE_URL=postgresql://user:pass@localhost:5432/stock_db
REDIS_URL=redis://localhost:6379/0

# Model Settings
MODEL_TYPE=xgboost
TOP_N_RECOMMENDATIONS=10
MIN_CONFIDENCE=0.5

# Caching
CACHE_TTL_QUOTE=60
CACHE_TTL_FEATURES=300
CACHE_TTL_RECOMMENDATION=60
CACHE_TTL_LLM=3600

External Dependencies

Data Sources

• TuShare API (stock data, fundamentals)
• yfinance (alternative stock data)
• OpenBB SDK (alternative provider)
• Sina Finance (sentiment - Phase 2)
• Tencent Stock (sentiment - Phase 2)

APIs

• OpenRouter Claude 4.5 (LLM)
• TuShare Pro (financial data)

Libraries

• pandas, numpy (data processing)
• scikit-learn (feature scaling)
• xgboost, lightgbm (ML models)
• fastapi (backend framework)
• redis, sqlalchemy (persistence)
• requests, httpx (HTTP)
• jieba (NLP - Phase 2)
• sentence-transformers (embeddings - Phase 3)