feat: Add stale process monitor, users route, landing page, archive old scripts
- Add backend stale process monitoring API (/api/stale-processes) - Add users management route - Add frontend landing page and stale process monitor UI on /scraper-tools - Move old development scripts to backend/archive/ - Update frontend build with new features 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
82
CLAUDE.md
82
CLAUDE.md
@@ -211,3 +211,85 @@
|
||||
- **Trigger schedules manually**: `curl -X POST /api/az/admin/schedules/{id}/trigger`
|
||||
- **Check schedule status**: `curl /api/az/admin/schedules`
|
||||
- **Worker logs**: `kubectl logs -f deployment/scraper-worker -n dispensary-scraper`
|
||||
|
||||
24) **Crawler Maintenance Procedure (Check Jobs, Requeue, Restart)**
|
||||
When crawlers are stuck or jobs aren't processing, follow this procedure:
|
||||
|
||||
**Step 1: Check Job Status**
|
||||
```bash
|
||||
# Port-forward to production
|
||||
kubectl port-forward -n dispensary-scraper deployment/scraper 3099:3010 &
|
||||
|
||||
# Check active/stuck jobs
|
||||
curl -s http://localhost:3099/api/az/monitor/active-jobs | jq .
|
||||
|
||||
# Check recent job history
|
||||
curl -s "http://localhost:3099/api/az/monitor/jobs?limit=20" | jq '.jobs[] | {id, job_type, status, dispensary_id, started_at, products_found, duration_min: (.duration_ms/60000 | floor)}'
|
||||
|
||||
# Check schedule status
|
||||
curl -s http://localhost:3099/api/az/admin/schedules | jq '.schedules[] | {id, jobName, enabled, lastRunAt, lastStatus, nextRunAt}'
|
||||
```
|
||||
|
||||
**Step 2: Reset Stuck Jobs**
|
||||
Jobs are considered stuck if they have `status='running'` but no heartbeat in >30 minutes:
|
||||
```bash
|
||||
# Via API (if endpoint exists)
|
||||
curl -s -X POST http://localhost:3099/api/az/admin/reset-stuck-jobs
|
||||
|
||||
# Via direct DB (if API not available)
|
||||
kubectl exec -n dispensary-scraper deployment/scraper -- psql $DATABASE_URL -c "
|
||||
UPDATE dispensary_crawl_jobs
|
||||
SET status = 'failed',
|
||||
error_message = 'Job timed out - worker stopped sending heartbeats',
|
||||
completed_at = NOW()
|
||||
WHERE status = 'running'
|
||||
AND (last_heartbeat_at < NOW() - INTERVAL '30 minutes' OR last_heartbeat_at IS NULL);
|
||||
"
|
||||
```
|
||||
|
||||
**Step 3: Requeue Jobs (Trigger Fresh Crawl)**
|
||||
```bash
|
||||
# Trigger product crawl schedule (typically ID 1)
|
||||
curl -s -X POST http://localhost:3099/api/az/admin/schedules/1/trigger
|
||||
|
||||
# Trigger menu detection schedule (typically ID 2)
|
||||
curl -s -X POST http://localhost:3099/api/az/admin/schedules/2/trigger
|
||||
|
||||
# Or crawl a specific dispensary
|
||||
curl -s -X POST http://localhost:3099/api/az/admin/crawl/112
|
||||
```
|
||||
|
||||
**Step 4: Restart Crawler Workers**
|
||||
```bash
|
||||
# Restart scraper-worker pods (clears any stuck processes)
|
||||
kubectl rollout restart deployment/scraper-worker -n dispensary-scraper
|
||||
|
||||
# Watch rollout progress
|
||||
kubectl rollout status deployment/scraper-worker -n dispensary-scraper
|
||||
|
||||
# Optionally restart main scraper pod too
|
||||
kubectl rollout restart deployment/scraper -n dispensary-scraper
|
||||
```
|
||||
|
||||
**Step 5: Monitor Recovery**
|
||||
```bash
|
||||
# Watch worker logs
|
||||
kubectl logs -f deployment/scraper-worker -n dispensary-scraper --tail=50
|
||||
|
||||
# Check dashboard for product counts
|
||||
curl -s http://localhost:3099/api/az/dashboard | jq '{totalStores, totalProducts, storesByType}'
|
||||
|
||||
# Verify jobs are processing
|
||||
curl -s http://localhost:3099/api/az/monitor/active-jobs | jq .
|
||||
```
|
||||
|
||||
**Quick One-Liner for Full Reset:**
|
||||
```bash
|
||||
# Reset stuck jobs and restart workers
|
||||
kubectl exec -n dispensary-scraper deployment/scraper -- psql $DATABASE_URL -c "UPDATE dispensary_crawl_jobs SET status='failed', completed_at=NOW() WHERE status='running' AND (last_heartbeat_at < NOW() - INTERVAL '30 minutes' OR last_heartbeat_at IS NULL);" && kubectl rollout restart deployment/scraper-worker -n dispensary-scraper && kubectl rollout status deployment/scraper-worker -n dispensary-scraper
|
||||
```
|
||||
|
||||
**Cleanup port-forwards when done:**
|
||||
```bash
|
||||
pkill -f "port-forward.*dispensary-scraper"
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user