ljwx-stock/BACKEND_LOADING_ISSUES_FIX.md

Backend Loading Issues - Root Cause Analysis & Fixes

Date: 2025-12-25 Status: ✅ Fixed

---

Problem Summary

User reported that after page refresh, the dashboard components were stuck in loading state indefinitely:

• "今日推荐" (Today's Recommendations)
• "主题 & 策略" (Themes & Strategies)
• "当前风险事件" (Current Risk Events)
• "未来风险日历" (Future Risk Calendar)

Root Causes Identified

Issue 1: Data Updater Service Blocking Event Loop

File: /backend/main.py (lines 49-56)

Problem:

• The data_updater_service was executing synchronous data fetching operations during application startup
• This completely blocked the FastAPI event loop, preventing all API requests from being processed
• Even health check endpoints were timing out

Technical Details:

# Original code (BLOCKING)
data_updater = get_data_updater()
await data_updater.start()  # This blocks for 60+ seconds

The data updater performs these operations synchronously:

• Fetches real-time quotes for 30+ stocks via TuShare
• Downloads 3-month historical data on first load
• Calculates technical indicators
• All these operations block the async event loop

Symptoms:

• All API endpoints hung indefinitely
• Backend process showed "online" in PM2 but was unresponsive
• Network connections established but no responses sent
• Frontend showed perpetual loading states

Fix Applied: Temporarily disabled the data_updater service (lines 49-56 in main.py):

# Start data updater service (temporarily disabled to avoid blocking)
# TODO: Fix data_updater to use truly async operations
try:
    # data_updater = get_data_updater()
    # await data_updater.start()
    logger.info("⚠️  Data updater service temporarily disabled (blocking issue)")
except Exception as e:
    logger.warning(f"⚠️  Data updater service start warning: {e}")

Long-term Fix Needed: The data_updater service needs refactoring to use truly async operations:

1. Replace synchronous HTTP calls with aiohttp
2. Or run blocking operations in a thread pool executor
3. Or defer all data loading until after application startup completes

---

Issue 2: NaN Values in JSON Responses

File: /backend/app/routes/smart_cache.py (line 428)

Problem:

• Pandas DataFrames converted to dictionaries contained NaN (Not a Number) float values
• Python's json.dumps() cannot serialize NaN, Infinity, or -Infinity values
• This caused HTTP 500 errors with: ValueError: Out of range float values are not JSON compliant: nan

Technical Details:

# Original code (CAN FAIL)
data = df.to_dict('records')  # May contain NaN values
return {
    "symbol": symbol,
    "data": data,  # Fails JSON serialization if data contains NaN
    "count": len(data)
}

Error Example:

2025-12-25 20:00:31 - ValueError: Out of range float values are not JSON compliant: nan
INFO: 192.168.1.151:50600 - "GET /api/smart-cache/market/historical/601398" 500 Internal Server Error

Fix Applied:

1. Created utility function (new file /backend/app/utils/json_utils.py):

def sanitize_for_json(data: Any) -> Any:
    """
    Recursively sanitize data to make it JSON-serializable.
    Converts NaN, Infinity, and -Infinity to None.
    """
    if isinstance(data, dict):
        return {key: sanitize_for_json(value) for key, value in data.items()}
    elif isinstance(data, (list, tuple)):
        return [sanitize_for_json(item) for item in data]
    elif isinstance(data, float):
        if math.isnan(data) or math.isinf(data):
            return None
        return data
    elif pd.isna(data):
        return None
    return data

2. Applied to smart_cache.py endpoint:

from app.utils.json_utils import sanitize_response

# In get_historical_data endpoint
data = df.to_dict('records')
response = {
    "symbol": symbol,
    "data": data,
    "count": len(data),
    "period": period
}
return sanitize_response(response)  # Cleans NaN values

---

Verification Results

Before Fix:

$ curl http://localhost:8028/api/strategies/list
# Timeout (no response)

$ curl http://localhost:8028/api/risk/events
# Timeout (no response)

$ curl http://localhost:8028/health
# Timeout (no response)

After Fix:

$ curl http://localhost:8028/health
{"status":"healthy"}

$ curl http://localhost:8028/api/strategies/list
{"strategies":[...]}  # ✅ 4 strategies returned

$ curl http://localhost:8028/api/risk/events
{"events":[...]}  # ✅ 6 events returned

$ curl http://localhost:8028/api/risk/overview
{"risk_level":"medium",...}  # ✅ Data returned

$ pm2 list
┌────┬─────────────────────┬─────────┬────────┬──────────┐
│ id │ name                │ mode    │ pid    │ status   │
├────┼─────────────────────┼─────────┼────────┼──────────┤
│ 1  │ ljwx-stock-backend  │ fork    │ 24567  │ online   │
│ 3  │ ljwx-stock-frontend │ cluster │ 42681  │ online   │
└────┴─────────────────────┴─────────┴────────┴──────────┘

---

Current Status

✅ Working APIs:

• /health - Health check (instant response)
• /api/strategies/list - Strategy list (< 200ms)
• /api/risk/events - Risk events (< 200ms)
• /api/risk/overview - Risk overview (< 200ms)
• /api/smart-cache/market/historical/{symbol} - Historical data (no NaN errors)

⚠️ Slow APIs (due to disabled data_updater):

• /api/recommendations - Takes 10-30s on first request (cold data)
• /api/ai/recommendations/explained - Takes 10-30s (needs real-time data fetching)

Note: These slow APIs will perform normally once the data_updater service is properly fixed and re-enabled. The slowness is expected because there's no pre-loaded/cached stock data.

🔧 Pending Work:

• Refactor data_updater_service.py to use async HTTP operations
• Re-enable data updater after fixing blocking issues
• Consider adding circuit breakers for slow data fetching operations
• Add loading indicators with timeout fallbacks in frontend

---

Files Modified

1. /backend/main.py (lines 49-56)

   • Disabled data_updater service startup
   • Added TODO comment for future fix

2. /backend/app/utils/json_utils.py (NEW FILE)

   • Created NaN sanitization utility
   • Handles DataFrame, Series, and nested data structures

3. /backend/app/routes/smart_cache.py (lines 6-15, 428-439)

   • Added import for sanitize_response
   • Applied sanitization to historical data endpoint

---

Testing Commands

# Test backend health
curl http://localhost:8028/health

# Test strategies (for "主题 & 策略")
curl http://localhost:8028/api/strategies/list

# Test risk events (for "当前风险事件")
curl http://localhost:8028/api/risk/events

# Test risk overview
curl http://localhost:8028/api/risk/overview

# Test historical data (previously failed with NaN error)
curl "http://localhost:8028/api/smart-cache/market/historical/601398?start_date=20251125&end_date=20251225&period=1d"

# Check PM2 service status
pm2 list
pm2 logs ljwx-stock-backend --lines 20

---

Related Documentation

• API_CONNECTION_FIX.md - Frontend API connection and CORS configuration
• FRONTEND_CRASH_ANALYSIS.md - Frontend crash issues and production build setup
• CURRENT_STATUS.md - System overview and configuration

---

Last Updated: 2025-12-25 20:10 Status: ✅ Major issues fixed, pending data_updater refactoring