🔍 Log Classification System
3-tier hybrid pipeline → 🟢 Regex · 🔵 BERT + LogReg · 🟡 LLM Built to mimic production enterprise log monitoring architecture.
Source System
📋 Example Logs (click to try)
| Source System | Log Message |
|---|
Upload a CSV with columns: source, log_message
Download the classified CSV with added columns: predicted_label, tier_used, confidence.
Sample CSV format:
source,log_message
ModernCRM,User User123 logged in.
LegacyCRM,Case escalation for ticket ID 7324 failed.
BillingSystem,GET /api/v2/invoice HTTP/1.1 status: 500
🏗️ 3-Tier Hybrid Pipeline
| Tier | Method | Coverage | Latency | When Used |
|---|---|---|---|---|
| 🟢 Regex | Python re patterns |
~21% | < 1ms | Fixed patterns (login, backup, etc.) |
| 🔵 BERT | all-MiniLM-L6-v2 + LogReg |
~79% | 20–80ms | High-volume categories with 150+ samples |
| 🟡 LLM | HuggingFace Inference API | ~0.3% | 500–2000ms | LegacyCRM logs, rare patterns |
📊 Model Performance (from training)
- BERT + LogReg trained on 2,410 synthetic enterprise logs
- Confidence threshold: 0.5 (below → escalate to LLM)
- Source-aware routing:
LegacyCRMbypasses ML entirely (only 7 training samples)
🔑 Environment Variables
| Secret | Required For |
|---|---|
HF_TOKEN |
LLM inference (LegacyCRM logs) |