56cc171287
## Worker System - Role-agnostic workers that can handle any task type - Pod-based architecture with StatefulSet (5-15 pods, 5 workers each) - Custom pod names (Aethelgard, Xylos, Kryll, etc.) - Worker registry with friendly names and resource monitoring - Hub-and-spoke visualization on JobQueue page ## Stealth & Anti-Detection (REQUIRED) - Proxies are MANDATORY - workers fail to start without active proxies - CrawlRotator initializes on worker startup - Loads proxies from `proxies` table - Auto-rotates proxy + fingerprint on 403 errors - 12 browser fingerprints (Chrome, Firefox, Safari, Edge) - Locale/timezone matching for geographic consistency ## Task System - Renamed product_resync → product_refresh - Task chaining: store_discovery → entry_point → product_discovery - Priority-based claiming with FOR UPDATE SKIP LOCKED - Heartbeat and stale task recovery ## UI Updates - JobQueue: Pod visualization, resource monitoring on hover - WorkersDashboard: Simplified worker list - Removed unused filters from task list ## Other - IP2Location service for visitor analytics - Findagram consumer features scaffolding - Documentation updates 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
36 lines
1.5 KiB
SQL
36 lines
1.5 KiB
SQL
-- Migration: Add visitor location and dispensary name to click events
|
|
-- Captures where visitors are clicking from and which dispensary
|
|
|
|
-- Add visitor location columns
|
|
ALTER TABLE product_click_events
|
|
ADD COLUMN IF NOT EXISTS visitor_city VARCHAR(100);
|
|
|
|
ALTER TABLE product_click_events
|
|
ADD COLUMN IF NOT EXISTS visitor_state VARCHAR(10);
|
|
|
|
ALTER TABLE product_click_events
|
|
ADD COLUMN IF NOT EXISTS visitor_lat DECIMAL(10, 7);
|
|
|
|
ALTER TABLE product_click_events
|
|
ADD COLUMN IF NOT EXISTS visitor_lng DECIMAL(10, 7);
|
|
|
|
-- Add dispensary name for easier reporting
|
|
ALTER TABLE product_click_events
|
|
ADD COLUMN IF NOT EXISTS dispensary_name VARCHAR(255);
|
|
|
|
-- Create index for location-based analytics
|
|
CREATE INDEX IF NOT EXISTS idx_product_click_events_visitor_state
|
|
ON product_click_events(visitor_state)
|
|
WHERE visitor_state IS NOT NULL;
|
|
|
|
CREATE INDEX IF NOT EXISTS idx_product_click_events_visitor_city
|
|
ON product_click_events(visitor_city)
|
|
WHERE visitor_city IS NOT NULL;
|
|
|
|
-- Add comments
|
|
COMMENT ON COLUMN product_click_events.visitor_city IS 'City where the visitor is located (from IP geolocation)';
|
|
COMMENT ON COLUMN product_click_events.visitor_state IS 'State where the visitor is located (from IP geolocation)';
|
|
COMMENT ON COLUMN product_click_events.visitor_lat IS 'Visitor latitude (from IP geolocation)';
|
|
COMMENT ON COLUMN product_click_events.visitor_lng IS 'Visitor longitude (from IP geolocation)';
|
|
COMMENT ON COLUMN product_click_events.dispensary_name IS 'Name of the dispensary (denormalized for easier reporting)';
|