Merge pull request 'fix(workers): Increase max concurrent tasks to 15' (#62) from feat/proxy-reload-and-bulk-import into master

Reviewed-on: https://code.cannabrands.app/Creationshop/dispensary-scraper/pulls/62

CLAUDE.md

@@ -17,6 +17,35 @@ Never deploy unless user explicitly says: "CLAUDE — DEPLOYMENT IS NOW AUTHORIZ
 ### 5. DB POOL ONLY
 Never import `src/db/migrate.ts` at runtime. Use `src/db/pool.ts` for DB access.
 
+### 6. K8S POD LIMITS — CRITICAL
+**MAX 8 PODS** for `scraper-worker` deployment. NEVER EXCEED THIS.
+
+**Pods vs Workers:**
+- **Pod** = Kubernetes container instance (MAX 8)
+- **Worker** = Concurrent task runner INSIDE a pod (controlled by `MAX_CONCURRENT_TASKS` env var)
+- Formula: `8 pods × MAX_CONCURRENT_TASKS = total concurrent workers`
+
+**To increase workers:** Change `MAX_CONCURRENT_TASKS` env var, NOT replicas.
+```bash
+# CORRECT - increase workers per pod
+kubectl set env deployment/scraper-worker -n dispensary-scraper MAX_CONCURRENT_TASKS=5
+
+# WRONG - never scale above 8 replicas
+kubectl scale deployment/scraper-worker --replicas=20  # NEVER DO THIS
+```
+
+**If K8s API returns ServiceUnavailable:** STOP IMMEDIATELY. Do not retry. The cluster is overloaded.
+
+### 7. K8S REQUIRES EXPLICIT PERMISSION
+**NEVER run kubectl commands without explicit user permission.**
+
+Before running ANY `kubectl` command (scale, rollout, set env, delete, apply, etc.):
+1. Tell the user what you want to do
+2. Wait for explicit approval
+3. Only then execute the command
+
+This applies to ALL kubectl operations - even read-only ones like `kubectl get pods`.
+
 ---
 
 ## Quick Reference

backend/src/tasks/task-worker.ts

@@ -98,7 +98,7 @@ const API_BASE_URL = process.env.API_BASE_URL || 'http://localhost:3010';
 
 // Maximum number of tasks this worker will run concurrently
 // Tune based on workload: I/O-bound tasks benefit from higher concurrency
-const MAX_CONCURRENT_TASKS = parseInt(process.env.MAX_CONCURRENT_TASKS || '3');
+const MAX_CONCURRENT_TASKS = parseInt(process.env.MAX_CONCURRENT_TASKS || '15');
 
 // When heap memory usage exceeds this threshold (as decimal 0.0-1.0), stop claiming new tasks
 // Default 85% - gives headroom before OOM