8e2f07c941
Workers can now process multiple tasks concurrently (default: 3 max). Self-regulate based on resource usage - back off at 85% memory or 90% CPU. Backend changes: - TaskWorker handles concurrent tasks using async Maps - Resource monitoring (memory %, CPU %) with backoff logic - Heartbeat reports active_task_count, max_concurrent_tasks, resource stats - Decommission support via worker_commands table Frontend changes: - Workers Dashboard shows tasks per worker (N/M format) - Resource badges with color-coded thresholds - Pod visualization with clickable selection - Decommission controls per worker New env vars: - MAX_CONCURRENT_TASKS (default: 3) - MEMORY_BACKOFF_THRESHOLD (default: 0.85) - CPU_BACKOFF_THRESHOLD (default: 0.90) - BACKOFF_DURATION_MS (default: 10000) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
37 lines
982 B
YAML
37 lines
982 B
YAML
# RBAC configuration for scraper pod to control worker scaling
|
|
# Allows the scraper to read and scale the scraper-worker statefulset
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: scraper-sa
|
|
namespace: dispensary-scraper
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: Role
|
|
metadata:
|
|
name: worker-scaler
|
|
namespace: dispensary-scraper
|
|
rules:
|
|
# Allow reading deployment and statefulset status
|
|
- apiGroups: ["apps"]
|
|
resources: ["deployments", "statefulsets"]
|
|
verbs: ["get", "list"]
|
|
# Allow scaling deployments and statefulsets
|
|
- apiGroups: ["apps"]
|
|
resources: ["deployments/scale", "statefulsets/scale"]
|
|
verbs: ["get", "patch", "update"]
|
|
---
|
|
apiVersion: rbac.authorization.k8s.io/v1
|
|
kind: RoleBinding
|
|
metadata:
|
|
name: scraper-worker-scaler
|
|
namespace: dispensary-scraper
|
|
subjects:
|
|
- kind: ServiceAccount
|
|
name: scraper-sa
|
|
namespace: dispensary-scraper
|
|
roleRef:
|
|
kind: Role
|
|
name: worker-scaler
|
|
apiGroup: rbac.authorization.k8s.io
|