feat: Stealth worker system with mandatory proxy rotation

## Worker System
- Role-agnostic workers that can handle any task type
- Pod-based architecture with StatefulSet (5-15 pods, 5 workers each)
- Custom pod names (Aethelgard, Xylos, Kryll, etc.)
- Worker registry with friendly names and resource monitoring
- Hub-and-spoke visualization on JobQueue page

## Stealth & Anti-Detection (REQUIRED)
- Proxies are MANDATORY - workers fail to start without active proxies
- CrawlRotator initializes on worker startup
- Loads proxies from `proxies` table
- Auto-rotates proxy + fingerprint on 403 errors
- 12 browser fingerprints (Chrome, Firefox, Safari, Edge)
- Locale/timezone matching for geographic consistency

## Task System
- Renamed product_resync → product_refresh
- Task chaining: store_discovery → entry_point → product_discovery
- Priority-based claiming with FOR UPDATE SKIP LOCKED
- Heartbeat and stale task recovery

## UI Updates
- JobQueue: Pod visualization, resource monitoring on hover
- WorkersDashboard: Simplified worker list
- Removed unused filters from task list

## Other
- IP2Location service for visitor analytics
- Findagram consumer features scaffolding
- Documentation updates

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Kelly
2025-12-10 00:44:59 -07:00
parent 0295637ed6
commit 56cc171287
61 changed files with 8591 additions and 2076 deletions

View File

@@ -213,22 +213,23 @@ CannaiQ has **TWO databases** with distinct purposes:
| Table | Purpose | Row Count | | Table | Purpose | Row Count |
|-------|---------|-----------| |-------|---------|-----------|
| `dispensaries` | Store/dispensary records | ~188+ rows | | `dispensaries` | Store/dispensary records | ~188+ rows |
| `dutchie_products` | Product catalog | ~37,000+ rows | | `store_products` | Product catalog | ~37,000+ rows |
| `dutchie_product_snapshots` | Price/stock history | ~millions | | `store_product_snapshots` | Price/stock history | ~millions |
| `store_products` | Canonical product schema | ~37,000+ rows |
| `store_product_snapshots` | Canonical snapshot schema | growing |
**LEGACY TABLES (EMPTY - DO NOT USE):** **LEGACY TABLES (EMPTY - DO NOT USE):**
| Table | Status | Action | | Table | Status | Action |
|-------|--------|--------| |-------|--------|--------|
| `stores` | EMPTY (0 rows) | Use `dispensaries` instead | | `stores` | EMPTY (0 rows) | Use `dispensaries` instead |
| `products` | EMPTY (0 rows) | Use `dutchie_products` or `store_products` | | `products` | EMPTY (0 rows) | Use `store_products` instead |
| `dutchie_products` | LEGACY (0 rows) | Use `store_products` instead |
| `dutchie_product_snapshots` | LEGACY (0 rows) | Use `store_product_snapshots` instead |
| `categories` | EMPTY (0 rows) | Categories stored in product records | | `categories` | EMPTY (0 rows) | Categories stored in product records |
**Code must NEVER:** **Code must NEVER:**
- Query the `stores` table (use `dispensaries`) - Query the `stores` table (use `dispensaries`)
- Query the `products` table (use `dutchie_products` or `store_products`) - Query the `products` table (use `store_products`)
- Query the `dutchie_products` table (use `store_products`)
- Query the `categories` table (categories are in product records) - Query the `categories` table (categories are in product records)
**CRITICAL RULES:** **CRITICAL RULES:**
@@ -343,23 +344,23 @@ npx tsx src/scripts/etl/042_legacy_import.ts
- SCHEMA ONLY - no data inserts from legacy tables - SCHEMA ONLY - no data inserts from legacy tables
**ETL Script 042** (`backend/src/scripts/etl/042_legacy_import.ts`): **ETL Script 042** (`backend/src/scripts/etl/042_legacy_import.ts`):
- Copies data from `dutchie_products``store_products` - Copies data from legacy `dutchie_legacy.dutchie_products``store_products`
- Copies data from `dutchie_product_snapshots``store_product_snapshots` - Copies data from legacy `dutchie_legacy.dutchie_product_snapshots``store_product_snapshots`
- Extracts brands from product data into `brands` table - Extracts brands from product data into `brands` table
- Links dispensaries to chains and states - Links dispensaries to chains and states
- INSERT-ONLY and IDEMPOTENT (uses ON CONFLICT DO NOTHING) - INSERT-ONLY and IDEMPOTENT (uses ON CONFLICT DO NOTHING)
- Run manually: `cd backend && npx tsx src/scripts/etl/042_legacy_import.ts` - Run manually: `cd backend && npx tsx src/scripts/etl/042_legacy_import.ts`
**Tables touched by ETL:** **Tables touched by ETL:**
| Source Table | Target Table | | Source Table (dutchie_legacy) | Target Table (dutchie_menus) |
|--------------|--------------| |-------------------------------|------------------------------|
| `dutchie_products` | `store_products` | | `dutchie_products` | `store_products` |
| `dutchie_product_snapshots` | `store_product_snapshots` | | `dutchie_product_snapshots` | `store_product_snapshots` |
| (brand names extracted) | `brands` | | (brand names extracted) | `brands` |
| (state codes mapped) | `dispensaries.state_id` | | (state codes mapped) | `dispensaries.state_id` |
| (chain names matched) | `dispensaries.chain_id` | | (chain names matched) | `dispensaries.chain_id` |
**Legacy tables remain intact** - `dutchie_products` and `dutchie_product_snapshots` are not modified. **Note:** The legacy `dutchie_products` and `dutchie_product_snapshots` tables in `dutchie_legacy` are read-only sources. All new crawl data goes directly to `store_products` and `store_product_snapshots`.
**Migration 045** (`backend/migrations/045_add_image_columns.sql`): **Migration 045** (`backend/migrations/045_add_image_columns.sql`):
- Adds `thumbnail_url` to `store_products` and `store_product_snapshots` - Adds `thumbnail_url` to `store_products` and `store_product_snapshots`
@@ -881,7 +882,7 @@ export default defineConfig({
18) **Dashboard Architecture** 18) **Dashboard Architecture**
- **Frontend**: Rebuild the frontend with `VITE_API_URL` pointing to the correct backend and redeploy. - **Frontend**: Rebuild the frontend with `VITE_API_URL` pointing to the correct backend and redeploy.
- **Backend**: `/api/dashboard/stats` MUST use the canonical DB pool. Use the correct tables: `dutchie_products`, `dispensaries`, and views like `v_dashboard_stats`, `v_latest_snapshots`. - **Backend**: `/api/dashboard/stats` MUST use the canonical DB pool. Use the correct tables: `store_products`, `dispensaries`, and views like `v_dashboard_stats`, `v_latest_snapshots`.
19) **Deployment (Gitea + Kubernetes)** 19) **Deployment (Gitea + Kubernetes)**
- **Registry**: Gitea at `code.cannabrands.app/creationshop/dispensary-scraper` - **Registry**: Gitea at `code.cannabrands.app/creationshop/dispensary-scraper`

3
backend/.gitignore vendored Normal file
View File

@@ -0,0 +1,3 @@
# IP2Location database (downloaded separately)
data/ip2location/

View File

@@ -275,6 +275,22 @@ Store metadata:
--- ---
## Worker Roles
Workers pull tasks from the `worker_tasks` queue based on their assigned role.
| Role | Name | Description | Handler |
|------|------|-------------|---------|
| `product_resync` | Product Resync | Re-crawl dispensary products for price/stock changes | `handleProductResync` |
| `product_discovery` | Product Discovery | Initial product discovery for new dispensaries | `handleProductDiscovery` |
| `store_discovery` | Store Discovery | Discover new dispensary locations | `handleStoreDiscovery` |
| `entry_point_discovery` | Entry Point Discovery | Resolve platform IDs from menu URLs | `handleEntryPointDiscovery` |
| `analytics_refresh` | Analytics Refresh | Refresh materialized views and analytics | `handleAnalyticsRefresh` |
**API Endpoint:** `GET /api/worker-registry/roles`
---
## Scheduling ## Scheduling
Crawls are scheduled via `worker_tasks` table: Crawls are scheduled via `worker_tasks` table:
@@ -282,8 +298,219 @@ Crawls are scheduled via `worker_tasks` table:
| Role | Frequency | Description | | Role | Frequency | Description |
|------|-----------|-------------| |------|-----------|-------------|
| `product_resync` | Every 4 hours | Regular product refresh | | `product_resync` | Every 4 hours | Regular product refresh |
| `product_discovery` | On-demand | First crawl for new stores |
| `entry_point_discovery` | On-demand | New store setup | | `entry_point_discovery` | On-demand | New store setup |
| `store_discovery` | Daily | Find new stores | | `store_discovery` | Daily | Find new stores |
| `analytics_refresh` | Daily | Refresh analytics materialized views |
---
## Priority & On-Demand Tasks
Tasks are claimed by workers in order of **priority DESC, created_at ASC**.
### Priority Levels
| Priority | Use Case | Example |
|----------|----------|---------|
| 0 | Scheduled/batch tasks | Daily product_resync generation |
| 10 | On-demand/chained tasks | entry_point → product_discovery |
| Higher | Urgent/manual triggers | Admin-triggered immediate crawl |
### Task Chaining
When a task completes, the system automatically creates follow-up tasks:
```
store_discovery (completed)
└─► entry_point_discovery (priority: 10) for each new store
entry_point_discovery (completed, success)
└─► product_discovery (priority: 10) for that store
product_discovery (completed)
└─► [no chain] Store enters regular resync schedule
```
### On-Demand Task Creation
Use the task service to create high-priority tasks:
```typescript
// Create immediate product resync for a store
await taskService.createTask({
role: 'product_resync',
dispensary_id: 123,
platform: 'dutchie',
priority: 20, // Higher than batch tasks
});
// Convenience methods with default high priority (10)
await taskService.createEntryPointTask(dispensaryId, 'dutchie');
await taskService.createProductDiscoveryTask(dispensaryId, 'dutchie');
await taskService.createStoreDiscoveryTask('dutchie', 'AZ');
```
### Claim Function
The `claim_task()` SQL function atomically claims tasks:
- Respects priority ordering (higher = first)
- Uses `FOR UPDATE SKIP LOCKED` for concurrency
- Prevents multiple active tasks per store
---
## Image Storage
Images are downloaded from Dutchie's AWS S3 and stored locally with on-demand resizing.
### Storage Path
```
/storage/images/products/<state>/<store>/<brand>/<product_id>/image-<hash>.webp
/storage/images/brands/<brand>/logo-<hash>.webp
```
**Example:**
```
/storage/images/products/az/az-deeply-rooted/bud-bros/6913e3cd444eac3935e928b9/image-ae38b1f9.webp
```
### Image Proxy API
Served via `/img/*` with on-demand resizing using **sharp**:
```
GET /img/products/az/az-deeply-rooted/bud-bros/6913e3cd444eac3935e928b9/image-ae38b1f9.webp?w=200
```
| Param | Description |
|-------|-------------|
| `w` | Width in pixels (max 4000) |
| `h` | Height in pixels (max 4000) |
| `q` | Quality 1-100 (default 80) |
| `fit` | cover, contain, fill, inside, outside |
| `blur` | Blur sigma (0.3-1000) |
| `gray` | Grayscale (1 = enabled) |
| `format` | webp, jpeg, png, avif (default webp) |
### Key Files
| File | Purpose |
|------|---------|
| `src/utils/image-storage.ts` | Download & save images to local filesystem |
| `src/routes/image-proxy.ts` | On-demand resize/transform at `/img/*` |
### Download Rules
| Scenario | Image Action |
|----------|--------------|
| **New product (first crawl)** | Download if `primaryImageUrl` exists |
| **Existing product (refresh)** | Download only if `local_image_path` is NULL (backfill) |
| **Product already has local image** | Skip download entirely |
**Logic:**
- Images are downloaded **once** and never re-downloaded on subsequent crawls
- `skipIfExists: true` - filesystem check prevents re-download even if queued
- First crawl: all products get images
- Refresh crawl: only new products or products missing local images
### Storage Rules
- **NO MinIO** - local filesystem only (`STORAGE_DRIVER=local`)
- Store full resolution, resize on-demand via `/img` proxy
- Convert to webp for consistency using **sharp**
- Preserve original Dutchie URL as fallback in `image_url` column
- Local path stored in `local_image_path` column
---
## Stealth & Anti-Detection
**PROXIES ARE REQUIRED** - Workers will fail to start if no active proxies are available in the database. All HTTP requests to Dutchie go through a proxy.
Workers automatically initialize anti-detection systems on startup.
### Components
| Component | Purpose | Source |
|-----------|---------|--------|
| **CrawlRotator** | Coordinates proxy + UA rotation | `src/services/crawl-rotator.ts` |
| **ProxyRotator** | Round-robin proxy selection, health tracking | `src/services/crawl-rotator.ts` |
| **UserAgentRotator** | Cycles through realistic browser fingerprints | `src/services/crawl-rotator.ts` |
| **Dutchie Client** | Curl-based HTTP with auto-retry on 403 | `src/platforms/dutchie/client.ts` |
### Initialization Flow
```
Worker Start
├─► initializeStealth()
│ │
│ ├─► CrawlRotator.initialize()
│ │ └─► Load proxies from `proxies` table
│ │
│ └─► setCrawlRotator(rotator)
│ └─► Wire to Dutchie client
└─► Process tasks...
```
### Stealth Session (per task)
Each crawl task starts a stealth session:
```typescript
// In product-refresh.ts, entry-point-discovery.ts
const session = startSession(dispensary.state || 'AZ', 'America/Phoenix');
```
This creates a new identity with:
- **Random fingerprint:** Chrome/Firefox/Safari/Edge on Win/Mac/Linux
- **Accept-Language:** Matches timezone (e.g., `America/Phoenix``en-US,en;q=0.9`)
- **sec-ch-ua headers:** Proper Client Hints for the browser profile
### On 403 Block
When Dutchie returns 403, the client automatically:
1. Records failure on current proxy (increments `failure_count`)
2. If proxy has 5+ failures, deactivates it
3. Rotates to next healthy proxy
4. Rotates fingerprint
5. Retries the request
### Proxy Table Schema
```sql
CREATE TABLE proxies (
id SERIAL PRIMARY KEY,
host VARCHAR(255) NOT NULL,
port INTEGER NOT NULL,
username VARCHAR(100),
password VARCHAR(100),
protocol VARCHAR(10) DEFAULT 'http', -- http, https, socks5
is_active BOOLEAN DEFAULT true,
last_used_at TIMESTAMPTZ,
failure_count INTEGER DEFAULT 0,
success_count INTEGER DEFAULT 0,
avg_response_time_ms INTEGER,
last_failure_at TIMESTAMPTZ,
last_error TEXT
);
```
### Configuration
Proxies are mandatory. There is no environment variable to disable them. Workers will refuse to start without active proxies in the database.
### Fingerprints Available
The client includes 6 browser fingerprints:
- Chrome 131 on Windows
- Chrome 131 on macOS
- Chrome 120 on Windows
- Firefox 133 on Windows
- Safari 17.2 on macOS
- Edge 131 on Windows
Each includes proper `sec-ch-ua`, `sec-ch-ua-platform`, and `sec-ch-ua-mobile` headers.
--- ---
@@ -293,6 +520,7 @@ Crawls are scheduled via `worker_tasks` table:
- **Normalization errors:** Logged as warnings, continue with valid products - **Normalization errors:** Logged as warnings, continue with valid products
- **Image download errors:** Non-fatal, logged, continue - **Image download errors:** Non-fatal, logged, continue
- **Database errors:** Task fails, will be retried - **Database errors:** Task fails, will be retried
- **403 blocks:** Auto-rotate proxy + fingerprint, retry (up to 3 retries)
--- ---
@@ -305,4 +533,6 @@ Crawls are scheduled via `worker_tasks` table:
| `src/platforms/dutchie/index.ts` | GraphQL client, session management | | `src/platforms/dutchie/index.ts` | GraphQL client, session management |
| `src/hydration/normalizers/dutchie.ts` | Payload normalization | | `src/hydration/normalizers/dutchie.ts` | Payload normalization |
| `src/hydration/canonical-upsert.ts` | Database upsert logic | | `src/hydration/canonical-upsert.ts` | Database upsert logic |
| `src/utils/image-storage.ts` | Image download and local storage |
| `src/routes/image-proxy.ts` | On-demand image resizing |
| `migrations/075_consecutive_misses.sql` | OOS tracking column | | `migrations/075_consecutive_misses.sql` | OOS tracking column |

View File

@@ -0,0 +1,69 @@
apiVersion: batch/v1
kind: CronJob
metadata:
name: ip2location-update
namespace: default
spec:
# Run on the 1st of every month at 3am UTC
schedule: "0 3 1 * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
template:
spec:
containers:
- name: ip2location-updater
image: curlimages/curl:latest
command:
- /bin/sh
- -c
- |
set -e
echo "Downloading IP2Location LITE DB5..."
# Download to temp
cd /tmp
curl -L -o ip2location.zip "https://www.ip2location.com/download/?token=${IP2LOCATION_TOKEN}&file=DB5LITEBIN"
# Extract
unzip -o ip2location.zip
# Find and copy the BIN file
BIN_FILE=$(ls *.BIN 2>/dev/null | head -1)
if [ -z "$BIN_FILE" ]; then
echo "ERROR: No BIN file found"
exit 1
fi
# Copy to shared volume
cp "$BIN_FILE" /data/IP2LOCATION-LITE-DB5.BIN
echo "Done! Database updated: /data/IP2LOCATION-LITE-DB5.BIN"
env:
- name: IP2LOCATION_TOKEN
valueFrom:
secretKeyRef:
name: dutchie-backend-secret
key: IP2LOCATION_TOKEN
volumeMounts:
- name: ip2location-data
mountPath: /data
restartPolicy: OnFailure
volumes:
- name: ip2location-data
persistentVolumeClaim:
claimName: ip2location-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ip2location-pvc
namespace: default
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Mi

View File

@@ -26,6 +26,12 @@ spec:
name: dutchie-backend-config name: dutchie-backend-config
- secretRef: - secretRef:
name: dutchie-backend-secret name: dutchie-backend-secret
env:
- name: IP2LOCATION_DB_PATH
value: /data/ip2location/IP2LOCATION-LITE-DB5.BIN
volumeMounts:
- name: ip2location-data
mountPath: /data/ip2location
resources: resources:
requests: requests:
memory: "256Mi" memory: "256Mi"
@@ -45,3 +51,7 @@ spec:
port: 3010 port: 3010
initialDelaySeconds: 5 initialDelaySeconds: 5
periodSeconds: 5 periodSeconds: 5
volumes:
- name: ip2location-data
persistentVolumeClaim:
claimName: ip2location-pvc

View File

@@ -0,0 +1,71 @@
-- Visitor location analytics for Findagram
-- Tracks visitor locations to understand popular areas
CREATE TABLE IF NOT EXISTS visitor_locations (
id SERIAL PRIMARY KEY,
-- Location data (from IP lookup)
ip_hash VARCHAR(64), -- Hashed IP for privacy (SHA256)
city VARCHAR(100),
state VARCHAR(100),
state_code VARCHAR(10),
country VARCHAR(100),
country_code VARCHAR(10),
latitude DECIMAL(10, 7),
longitude DECIMAL(10, 7),
-- Visit metadata
domain VARCHAR(50) NOT NULL, -- 'findagram.co', 'findadispo.com', etc.
page_path VARCHAR(255), -- '/products', '/dispensaries/123', etc.
referrer VARCHAR(500),
user_agent VARCHAR(500),
-- Session tracking
session_id VARCHAR(64), -- For grouping page views in a session
-- Timestamps
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Indexes for analytics queries
CREATE INDEX IF NOT EXISTS idx_visitor_locations_domain ON visitor_locations(domain);
CREATE INDEX IF NOT EXISTS idx_visitor_locations_city_state ON visitor_locations(city, state_code);
CREATE INDEX IF NOT EXISTS idx_visitor_locations_created_at ON visitor_locations(created_at);
CREATE INDEX IF NOT EXISTS idx_visitor_locations_session ON visitor_locations(session_id);
-- Aggregated daily stats (materialized for performance)
CREATE TABLE IF NOT EXISTS visitor_location_stats (
id SERIAL PRIMARY KEY,
date DATE NOT NULL,
domain VARCHAR(50) NOT NULL,
city VARCHAR(100),
state VARCHAR(100),
state_code VARCHAR(10),
country_code VARCHAR(10),
-- Metrics
visit_count INTEGER DEFAULT 0,
unique_sessions INTEGER DEFAULT 0,
UNIQUE(date, domain, city, state_code, country_code)
);
CREATE INDEX IF NOT EXISTS idx_visitor_stats_date ON visitor_location_stats(date);
CREATE INDEX IF NOT EXISTS idx_visitor_stats_domain ON visitor_location_stats(domain);
CREATE INDEX IF NOT EXISTS idx_visitor_stats_state ON visitor_location_stats(state_code);
-- View for easy querying of top locations
CREATE OR REPLACE VIEW v_top_visitor_locations AS
SELECT
domain,
city,
state,
state_code,
country_code,
COUNT(*) as total_visits,
COUNT(DISTINCT session_id) as unique_sessions,
MAX(created_at) as last_visit
FROM visitor_locations
WHERE created_at > NOW() - INTERVAL '30 days'
GROUP BY domain, city, state, state_code, country_code
ORDER BY total_visits DESC;

View File

@@ -0,0 +1,141 @@
-- Migration 076: Worker Registry for Dynamic Workers
-- Workers register on startup, receive a friendly name, and report heartbeats
-- Name pool for workers (expandable, no hardcoding)
CREATE TABLE IF NOT EXISTS worker_name_pool (
id SERIAL PRIMARY KEY,
name VARCHAR(50) UNIQUE NOT NULL,
in_use BOOLEAN DEFAULT FALSE,
assigned_to VARCHAR(100), -- worker_id
assigned_at TIMESTAMPTZ,
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Seed with initial names (can add more via API)
INSERT INTO worker_name_pool (name) VALUES
('Alice'), ('Bella'), ('Clara'), ('Diana'), ('Elena'),
('Fiona'), ('Grace'), ('Hazel'), ('Iris'), ('Julia'),
('Katie'), ('Luna'), ('Mia'), ('Nora'), ('Olive'),
('Pearl'), ('Quinn'), ('Rosa'), ('Sara'), ('Tara'),
('Uma'), ('Vera'), ('Wendy'), ('Xena'), ('Yuki'), ('Zara'),
('Amber'), ('Blake'), ('Coral'), ('Dawn'), ('Echo'),
('Fleur'), ('Gem'), ('Haven'), ('Ivy'), ('Jade'),
('Kira'), ('Lotus'), ('Maple'), ('Nova'), ('Onyx'),
('Pixel'), ('Quest'), ('Raven'), ('Sage'), ('Terra'),
('Unity'), ('Violet'), ('Willow'), ('Xylo'), ('Yara'), ('Zen')
ON CONFLICT (name) DO NOTHING;
-- Worker registry - tracks active workers
CREATE TABLE IF NOT EXISTS worker_registry (
id SERIAL PRIMARY KEY,
worker_id VARCHAR(100) UNIQUE NOT NULL, -- e.g., "pod-abc123" or uuid
friendly_name VARCHAR(50), -- assigned from pool
role VARCHAR(50) NOT NULL, -- task role
pod_name VARCHAR(100), -- k8s pod name
hostname VARCHAR(100), -- machine hostname
ip_address VARCHAR(50), -- worker IP
status VARCHAR(20) DEFAULT 'starting', -- starting, active, idle, offline, terminated
started_at TIMESTAMPTZ DEFAULT NOW(),
last_heartbeat_at TIMESTAMPTZ DEFAULT NOW(),
last_task_at TIMESTAMPTZ,
tasks_completed INTEGER DEFAULT 0,
tasks_failed INTEGER DEFAULT 0,
current_task_id INTEGER,
metadata JSONB DEFAULT '{}',
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Indexes for worker registry
CREATE INDEX IF NOT EXISTS idx_worker_registry_status ON worker_registry(status);
CREATE INDEX IF NOT EXISTS idx_worker_registry_role ON worker_registry(role);
CREATE INDEX IF NOT EXISTS idx_worker_registry_heartbeat ON worker_registry(last_heartbeat_at);
-- Function to assign a name to a new worker
CREATE OR REPLACE FUNCTION assign_worker_name(p_worker_id VARCHAR(100))
RETURNS VARCHAR(50) AS $$
DECLARE
v_name VARCHAR(50);
BEGIN
-- Try to get an unused name
UPDATE worker_name_pool
SET in_use = TRUE, assigned_to = p_worker_id, assigned_at = NOW()
WHERE id = (
SELECT id FROM worker_name_pool
WHERE in_use = FALSE
ORDER BY RANDOM()
LIMIT 1
FOR UPDATE SKIP LOCKED
)
RETURNING name INTO v_name;
-- If no names available, generate one
IF v_name IS NULL THEN
v_name := 'Worker-' || SUBSTRING(p_worker_id FROM 1 FOR 8);
END IF;
RETURN v_name;
END;
$$ LANGUAGE plpgsql;
-- Function to release a worker's name back to the pool
CREATE OR REPLACE FUNCTION release_worker_name(p_worker_id VARCHAR(100))
RETURNS VOID AS $$
BEGIN
UPDATE worker_name_pool
SET in_use = FALSE, assigned_to = NULL, assigned_at = NULL
WHERE assigned_to = p_worker_id;
END;
$$ LANGUAGE plpgsql;
-- Function to mark stale workers as offline
CREATE OR REPLACE FUNCTION mark_stale_workers(stale_threshold_minutes INTEGER DEFAULT 5)
RETURNS INTEGER AS $$
DECLARE
v_count INTEGER;
BEGIN
UPDATE worker_registry
SET status = 'offline', updated_at = NOW()
WHERE status IN ('active', 'idle', 'starting')
AND last_heartbeat_at < NOW() - (stale_threshold_minutes || ' minutes')::INTERVAL
RETURNING COUNT(*) INTO v_count;
-- Release names from offline workers
PERFORM release_worker_name(worker_id)
FROM worker_registry
WHERE status = 'offline'
AND last_heartbeat_at < NOW() - INTERVAL '30 minutes';
RETURN COALESCE(v_count, 0);
END;
$$ LANGUAGE plpgsql;
-- View for dashboard
CREATE OR REPLACE VIEW v_active_workers AS
SELECT
wr.id,
wr.worker_id,
wr.friendly_name,
wr.role,
wr.status,
wr.pod_name,
wr.hostname,
wr.started_at,
wr.last_heartbeat_at,
wr.last_task_at,
wr.tasks_completed,
wr.tasks_failed,
wr.current_task_id,
EXTRACT(EPOCH FROM (NOW() - wr.last_heartbeat_at)) as seconds_since_heartbeat,
CASE
WHEN wr.status = 'offline' THEN 'offline'
WHEN wr.last_heartbeat_at < NOW() - INTERVAL '2 minutes' THEN 'stale'
WHEN wr.current_task_id IS NOT NULL THEN 'busy'
ELSE 'ready'
END as health_status
FROM worker_registry wr
WHERE wr.status != 'terminated'
ORDER BY wr.status = 'active' DESC, wr.last_heartbeat_at DESC;
COMMENT ON TABLE worker_registry IS 'Tracks all workers that have registered with the system';
COMMENT ON TABLE worker_name_pool IS 'Pool of friendly names for workers - expandable via API';

View File

@@ -0,0 +1,35 @@
-- Migration: Add visitor location and dispensary name to click events
-- Captures where visitors are clicking from and which dispensary
-- Add visitor location columns
ALTER TABLE product_click_events
ADD COLUMN IF NOT EXISTS visitor_city VARCHAR(100);
ALTER TABLE product_click_events
ADD COLUMN IF NOT EXISTS visitor_state VARCHAR(10);
ALTER TABLE product_click_events
ADD COLUMN IF NOT EXISTS visitor_lat DECIMAL(10, 7);
ALTER TABLE product_click_events
ADD COLUMN IF NOT EXISTS visitor_lng DECIMAL(10, 7);
-- Add dispensary name for easier reporting
ALTER TABLE product_click_events
ADD COLUMN IF NOT EXISTS dispensary_name VARCHAR(255);
-- Create index for location-based analytics
CREATE INDEX IF NOT EXISTS idx_product_click_events_visitor_state
ON product_click_events(visitor_state)
WHERE visitor_state IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_product_click_events_visitor_city
ON product_click_events(visitor_city)
WHERE visitor_city IS NOT NULL;
-- Add comments
COMMENT ON COLUMN product_click_events.visitor_city IS 'City where the visitor is located (from IP geolocation)';
COMMENT ON COLUMN product_click_events.visitor_state IS 'State where the visitor is located (from IP geolocation)';
COMMENT ON COLUMN product_click_events.visitor_lat IS 'Visitor latitude (from IP geolocation)';
COMMENT ON COLUMN product_click_events.visitor_lng IS 'Visitor longitude (from IP geolocation)';
COMMENT ON COLUMN product_click_events.dispensary_name IS 'Name of the dispensary (denormalized for easier reporting)';

View File

@@ -1026,6 +1026,17 @@
"url": "https://github.com/sponsors/fb55" "url": "https://github.com/sponsors/fb55"
} }
}, },
"node_modules/csv-parser": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/csv-parser/-/csv-parser-3.2.0.tgz",
"integrity": "sha512-fgKbp+AJbn1h2dcAHKIdKNSSjfp43BZZykXsCjzALjKy80VXQNHPFJ6T9Afwdzoj24aMkq8GwDS7KGcDPpejrA==",
"bin": {
"csv-parser": "bin/csv-parser"
},
"engines": {
"node": ">= 10"
}
},
"node_modules/data-uri-to-buffer": { "node_modules/data-uri-to-buffer": {
"version": "6.0.2", "version": "6.0.2",
"resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-6.0.2.tgz", "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-6.0.2.tgz",
@@ -2235,6 +2246,14 @@
"node": ">= 12" "node": ">= 12"
} }
}, },
"node_modules/ip2location-nodejs": {
"version": "9.7.0",
"resolved": "https://registry.npmjs.org/ip2location-nodejs/-/ip2location-nodejs-9.7.0.tgz",
"integrity": "sha512-eQ4T5TXm1cx0+pQcRycPiuaiRuoDEMd9O89Be7Ugk555qi9UY9enXSznkkqr3kQRyUaXx7zj5dORC5LGTPOttA==",
"dependencies": {
"csv-parser": "^3.0.0"
}
},
"node_modules/ipaddr.js": { "node_modules/ipaddr.js": {
"version": "2.2.0", "version": "2.2.0",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-2.2.0.tgz", "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-2.2.0.tgz",

View File

@@ -21,6 +21,7 @@
"helmet": "^7.1.0", "helmet": "^7.1.0",
"https-proxy-agent": "^7.0.2", "https-proxy-agent": "^7.0.2",
"ioredis": "^5.8.2", "ioredis": "^5.8.2",
"ip2location-nodejs": "^9.7.0",
"ipaddr.js": "^2.2.0", "ipaddr.js": "^2.2.0",
"jsonwebtoken": "^9.0.2", "jsonwebtoken": "^9.0.2",
"minio": "^7.1.3", "minio": "^7.1.3",
@@ -1531,6 +1532,17 @@
"url": "https://github.com/sponsors/fb55" "url": "https://github.com/sponsors/fb55"
} }
}, },
"node_modules/csv-parser": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/csv-parser/-/csv-parser-3.2.0.tgz",
"integrity": "sha512-fgKbp+AJbn1h2dcAHKIdKNSSjfp43BZZykXsCjzALjKy80VXQNHPFJ6T9Afwdzoj24aMkq8GwDS7KGcDPpejrA==",
"bin": {
"csv-parser": "bin/csv-parser"
},
"engines": {
"node": ">= 10"
}
},
"node_modules/data-uri-to-buffer": { "node_modules/data-uri-to-buffer": {
"version": "6.0.2", "version": "6.0.2",
"resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-6.0.2.tgz", "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-6.0.2.tgz",
@@ -2754,6 +2766,14 @@
"node": ">= 12" "node": ">= 12"
} }
}, },
"node_modules/ip2location-nodejs": {
"version": "9.7.0",
"resolved": "https://registry.npmjs.org/ip2location-nodejs/-/ip2location-nodejs-9.7.0.tgz",
"integrity": "sha512-eQ4T5TXm1cx0+pQcRycPiuaiRuoDEMd9O89Be7Ugk555qi9UY9enXSznkkqr3kQRyUaXx7zj5dORC5LGTPOttA==",
"dependencies": {
"csv-parser": "^3.0.0"
}
},
"node_modules/ipaddr.js": { "node_modules/ipaddr.js": {
"version": "2.2.0", "version": "2.2.0",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-2.2.0.tgz", "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-2.2.0.tgz",

View File

@@ -35,6 +35,7 @@
"helmet": "^7.1.0", "helmet": "^7.1.0",
"https-proxy-agent": "^7.0.2", "https-proxy-agent": "^7.0.2",
"ioredis": "^5.8.2", "ioredis": "^5.8.2",
"ip2location-nodejs": "^9.7.0",
"ipaddr.js": "^2.2.0", "ipaddr.js": "^2.2.0",
"jsonwebtoken": "^9.0.2", "jsonwebtoken": "^9.0.2",
"minio": "^7.1.3", "minio": "^7.1.3",

View File

@@ -0,0 +1,65 @@
#!/bin/bash
# Download IP2Location LITE DB3 (City-level) database
# Free for commercial use with attribution
# https://lite.ip2location.com/database/db3-ip-country-region-city
set -e
DATA_DIR="${1:-./data/ip2location}"
DB_FILE="IP2LOCATION-LITE-DB3.BIN"
mkdir -p "$DATA_DIR"
cd "$DATA_DIR"
echo "Downloading IP2Location LITE DB3 database..."
# IP2Location LITE DB3 - includes city, region, country, lat/lng
# You need to register at https://lite.ip2location.com/ to get a download token
# Then set IP2LOCATION_TOKEN environment variable
if [ -z "$IP2LOCATION_TOKEN" ]; then
echo ""
echo "ERROR: IP2LOCATION_TOKEN not set"
echo ""
echo "To download the database:"
echo "1. Register free at https://lite.ip2location.com/"
echo "2. Get your download token from the dashboard"
echo "3. Run: IP2LOCATION_TOKEN=your_token ./scripts/download-ip2location.sh"
echo ""
exit 1
fi
# Download DB3.LITE (IPv4 + City)
DOWNLOAD_URL="https://www.ip2location.com/download/?token=${IP2LOCATION_TOKEN}&file=DB3LITEBIN"
echo "Downloading from IP2Location..."
curl -L -o ip2location.zip "$DOWNLOAD_URL"
echo "Extracting..."
unzip -o ip2location.zip
# Rename to standard name
if [ -f "IP2LOCATION-LITE-DB3.BIN" ]; then
echo "Database ready: $DATA_DIR/IP2LOCATION-LITE-DB3.BIN"
elif [ -f "IP-COUNTRY-REGION-CITY.BIN" ]; then
mv "IP-COUNTRY-REGION-CITY.BIN" "$DB_FILE"
echo "Database ready: $DATA_DIR/$DB_FILE"
else
# Find whatever BIN file was extracted
BIN_FILE=$(ls *.BIN 2>/dev/null | head -1)
if [ -n "$BIN_FILE" ]; then
mv "$BIN_FILE" "$DB_FILE"
echo "Database ready: $DATA_DIR/$DB_FILE"
else
echo "ERROR: No BIN file found in archive"
ls -la
exit 1
fi
fi
# Cleanup
rm -f ip2location.zip *.txt LICENSE* README*
echo ""
echo "Done! Database saved to: $DATA_DIR/$DB_FILE"
echo "Update monthly by re-running this script."

View File

@@ -191,6 +191,23 @@ export async function runFullDiscovery(
} }
} }
// Step 5: Detect dropped stores (in DB but not in discovery results)
if (!dryRun) {
console.log('\n[Discovery] Step 5: Detecting dropped stores...');
const droppedResult = await detectDroppedStores(pool, stateCode);
if (droppedResult.droppedCount > 0) {
console.log(`[Discovery] Found ${droppedResult.droppedCount} dropped stores:`);
droppedResult.droppedStores.slice(0, 10).forEach(s => {
console.log(` - ${s.name} (${s.city}, ${s.state}) - last seen: ${s.lastSeenAt}`);
});
if (droppedResult.droppedCount > 10) {
console.log(` ... and ${droppedResult.droppedCount - 10} more`);
}
} else {
console.log(`[Discovery] No dropped stores detected`);
}
}
return { return {
cities: cityResult, cities: cityResult,
locations: locationResults, locations: locationResults,
@@ -200,6 +217,107 @@ export async function runFullDiscovery(
}; };
} }
// ============================================================
// DROPPED STORE DETECTION
// ============================================================
export interface DroppedStoreResult {
droppedCount: number;
droppedStores: Array<{
id: number;
name: string;
city: string;
state: string;
platformDispensaryId: string;
lastSeenAt: string;
}>;
}
/**
* Detect stores that exist in dispensaries but were not found in discovery.
* Marks them as status='dropped' for manual review.
*
* A store is considered "dropped" if:
* 1. It has a platform_dispensary_id (was verified via Dutchie)
* 2. It was NOT seen in the latest discovery crawl (last_seen_at in discovery < 24h ago)
* 3. It's currently marked as 'open' status
*/
export async function detectDroppedStores(
pool: Pool,
stateCode?: string
): Promise<DroppedStoreResult> {
// Find dispensaries that:
// 1. Have platform_dispensary_id (verified Dutchie stores)
// 2. Are currently 'open' status
// 3. Have a linked discovery record that wasn't seen in the last discovery run
// (last_seen_at in dutchie_discovery_locations is older than 24 hours)
const params: any[] = [];
let stateFilter = '';
if (stateCode) {
stateFilter = ` AND d.state = $1`;
params.push(stateCode);
}
const query = `
WITH recently_seen AS (
SELECT DISTINCT platform_location_id
FROM dutchie_discovery_locations
WHERE last_seen_at > NOW() - INTERVAL '24 hours'
AND active = true
)
SELECT
d.id,
d.name,
d.city,
d.state,
d.platform_dispensary_id,
d.updated_at as last_seen_at
FROM dispensaries d
WHERE d.platform_dispensary_id IS NOT NULL
AND d.platform = 'dutchie'
AND (d.status = 'open' OR d.status IS NULL)
AND d.crawl_enabled = true
AND d.platform_dispensary_id NOT IN (SELECT platform_location_id FROM recently_seen)
${stateFilter}
ORDER BY d.name
`;
const result = await pool.query(query, params);
const droppedStores = result.rows;
// Mark these stores as 'dropped' status
if (droppedStores.length > 0) {
const ids = droppedStores.map(s => s.id);
await pool.query(`
UPDATE dispensaries
SET status = 'dropped', updated_at = NOW()
WHERE id = ANY($1::int[])
`, [ids]);
// Log to promotion log for audit
for (const store of droppedStores) {
await pool.query(`
INSERT INTO dutchie_promotion_log
(dispensary_id, action, state_code, store_name, triggered_by)
VALUES ($1, 'dropped', $2, $3, 'discovery_detection')
`, [store.id, store.state, store.name]);
}
}
return {
droppedCount: droppedStores.length,
droppedStores: droppedStores.map(s => ({
id: s.id,
name: s.name,
city: s.city,
state: s.state,
platformDispensaryId: s.platform_dispensary_id,
lastSeenAt: s.last_seen_at,
})),
};
}
// ============================================================ // ============================================================
// SINGLE CITY DISCOVERY // SINGLE CITY DISCOVERY
// ============================================================ // ============================================================

View File

@@ -140,6 +140,7 @@ import clickAnalyticsRoutes from './routes/click-analytics';
import seoRoutes from './routes/seo'; import seoRoutes from './routes/seo';
import priceAnalyticsRoutes from './routes/price-analytics'; import priceAnalyticsRoutes from './routes/price-analytics';
import tasksRoutes from './routes/tasks'; import tasksRoutes from './routes/tasks';
import workerRegistryRoutes from './routes/worker-registry';
// Mark requests from trusted domains (cannaiq.co, findagram.co, findadispo.com) // Mark requests from trusted domains (cannaiq.co, findagram.co, findadispo.com)
// These domains can access the API without authentication // These domains can access the API without authentication
@@ -216,6 +217,10 @@ console.log('[Workers] Routes registered at /api/workers, /api/monitor, and /api
app.use('/api/tasks', tasksRoutes); app.use('/api/tasks', tasksRoutes);
console.log('[Tasks] Routes registered at /api/tasks'); console.log('[Tasks] Routes registered at /api/tasks');
// Worker registry - dynamic worker registration, heartbeats, and name management
app.use('/api/worker-registry', workerRegistryRoutes);
console.log('[WorkerRegistry] Routes registered at /api/worker-registry');
// Phase 3: Analytics V2 - Enhanced analytics with rec/med state segmentation // Phase 3: Analytics V2 - Enhanced analytics with rec/med state segmentation
try { try {
const analyticsV2Router = createAnalyticsV2Router(getPool()); const analyticsV2Router = createAnalyticsV2Router(getPool());

View File

@@ -5,31 +5,35 @@ import { pool } from '../db/pool';
const router = Router(); const router = Router();
router.use(authMiddleware); router.use(authMiddleware);
// Get categories (flat list) // Get categories (flat list) - derived from actual product data
router.get('/', async (req, res) => { router.get('/', async (req, res) => {
try { try {
const { store_id } = req.query; const { store_id, in_stock_only } = req.query;
let query = ` let query = `
SELECT SELECT
c.*, category_raw as name,
COUNT(DISTINCT p.id) as product_count, category_raw as slug,
pc.name as parent_name COUNT(*) as product_count,
FROM categories c COUNT(*) FILTER (WHERE is_in_stock = true) as in_stock_count
LEFT JOIN store_products p ON c.name = p.category_raw FROM store_products
LEFT JOIN categories pc ON c.parent_id = pc.id WHERE category_raw IS NOT NULL
`; `;
const params: any[] = []; const params: any[] = [];
if (store_id) { if (store_id) {
query += ' WHERE c.store_id = $1';
params.push(store_id); params.push(store_id);
query += ` AND dispensary_id = $${params.length}`;
}
if (in_stock_only === 'true') {
query += ` AND is_in_stock = true`;
} }
query += ` query += `
GROUP BY c.id, pc.name GROUP BY category_raw
ORDER BY c.display_order, c.name ORDER BY category_raw
`; `;
const result = await pool.query(query, params); const result = await pool.query(query, params);
@@ -40,49 +44,85 @@ router.get('/', async (req, res) => {
} }
}); });
// Get category tree (hierarchical) // Get category tree (hierarchical) - category -> subcategory structure from product data
router.get('/tree', async (req, res) => { router.get('/tree', async (req, res) => {
try { try {
const { store_id } = req.query; const { store_id, in_stock_only } = req.query;
if (!store_id) { // Get category + subcategory combinations with counts
return res.status(400).json({ error: 'store_id is required' }); let query = `
}
// Get all categories for the store
const result = await pool.query(`
SELECT SELECT
c.*, category_raw as category,
COUNT(DISTINCT p.id) as product_count subcategory_raw as subcategory,
FROM categories c COUNT(*) as product_count,
LEFT JOIN store_products p ON c.name = p.category_raw AND p.is_in_stock = true AND p.dispensary_id = $1 COUNT(*) FILTER (WHERE is_in_stock = true) as in_stock_count
WHERE c.store_id = $1 FROM store_products
GROUP BY c.id WHERE category_raw IS NOT NULL
ORDER BY c.display_order, c.name `;
`, [store_id]);
// Build tree structure const params: any[] = [];
const categories = result.rows;
const categoryMap = new Map();
const tree: any[] = [];
// First pass: create map if (store_id) {
categories.forEach((cat: { id: number; parent_id?: number }) => { params.push(store_id);
categoryMap.set(cat.id, { ...cat, children: [] }); query += ` AND dispensary_id = $${params.length}`;
});
// Second pass: build tree
categories.forEach((cat: { id: number; parent_id?: number }) => {
const node = categoryMap.get(cat.id);
if (cat.parent_id) {
const parent = categoryMap.get(cat.parent_id);
if (parent) {
parent.children.push(node);
} }
} else {
tree.push(node); if (in_stock_only === 'true') {
query += ` AND is_in_stock = true`;
} }
query += `
GROUP BY category_raw, subcategory_raw
ORDER BY category_raw, subcategory_raw
`;
const result = await pool.query(query, params);
// Build tree structure: category -> subcategories
const categoryMap = new Map<string, {
name: string;
slug: string;
product_count: number;
in_stock_count: number;
subcategories: Array<{
name: string;
slug: string;
product_count: number;
in_stock_count: number;
}>;
}>();
for (const row of result.rows) {
const category = row.category;
const subcategory = row.subcategory;
const count = parseInt(row.product_count);
const inStockCount = parseInt(row.in_stock_count);
if (!categoryMap.has(category)) {
categoryMap.set(category, {
name: category,
slug: category.toLowerCase().replace(/\s+/g, '-'),
product_count: 0,
in_stock_count: 0,
subcategories: []
}); });
}
const cat = categoryMap.get(category)!;
cat.product_count += count;
cat.in_stock_count += inStockCount;
if (subcategory) {
cat.subcategories.push({
name: subcategory,
slug: subcategory.toLowerCase().replace(/\s+/g, '-'),
product_count: count,
in_stock_count: inStockCount
});
}
}
const tree = Array.from(categoryMap.values());
res.json({ tree }); res.json({ tree });
} catch (error) { } catch (error) {
@@ -91,4 +131,91 @@ router.get('/tree', async (req, res) => {
} }
}); });
// Get all unique subcategories for a category
router.get('/:category/subcategories', async (req, res) => {
try {
const { category } = req.params;
const { store_id, in_stock_only } = req.query;
let query = `
SELECT
subcategory_raw as name,
subcategory_raw as slug,
COUNT(*) as product_count,
COUNT(*) FILTER (WHERE is_in_stock = true) as in_stock_count
FROM store_products
WHERE category_raw = $1
AND subcategory_raw IS NOT NULL
`;
const params: any[] = [category];
if (store_id) {
params.push(store_id);
query += ` AND dispensary_id = $${params.length}`;
}
if (in_stock_only === 'true') {
query += ` AND is_in_stock = true`;
}
query += `
GROUP BY subcategory_raw
ORDER BY subcategory_raw
`;
const result = await pool.query(query, params);
res.json({
category,
subcategories: result.rows
});
} catch (error) {
console.error('Error fetching subcategories:', error);
res.status(500).json({ error: 'Failed to fetch subcategories' });
}
});
// Get global category summary (across all stores)
router.get('/summary', async (req, res) => {
try {
const { state } = req.query;
let query = `
SELECT
sp.category_raw as category,
COUNT(DISTINCT sp.id) as product_count,
COUNT(DISTINCT sp.dispensary_id) as store_count,
COUNT(*) FILTER (WHERE sp.is_in_stock = true) as in_stock_count
FROM store_products sp
`;
const params: any[] = [];
if (state) {
query += `
JOIN dispensaries d ON sp.dispensary_id = d.id
WHERE sp.category_raw IS NOT NULL
AND d.state = $1
`;
params.push(state);
} else {
query += ` WHERE sp.category_raw IS NOT NULL`;
}
query += `
GROUP BY sp.category_raw
ORDER BY product_count DESC
`;
const result = await pool.query(query, params);
res.json({
categories: result.rows,
total_categories: result.rows.length
});
} catch (error) {
console.error('Error fetching category summary:', error);
res.status(500).json({ error: 'Failed to fetch category summary' });
}
});
export default router; export default router;

View File

@@ -11,7 +11,7 @@ const VALID_MENU_TYPES = ['dutchie', 'treez', 'jane', 'weedmaps', 'leafly', 'mea
// Get all dispensaries (with pagination) // Get all dispensaries (with pagination)
router.get('/', async (req, res) => { router.get('/', async (req, res) => {
try { try {
const { menu_type, city, state, crawl_enabled, dutchie_verified, limit, offset, search } = req.query; const { menu_type, city, state, crawl_enabled, dutchie_verified, status, limit, offset, search } = req.query;
const pageLimit = Math.min(parseInt(limit as string) || 50, 500); const pageLimit = Math.min(parseInt(limit as string) || 50, 500);
const pageOffset = parseInt(offset as string) || 0; const pageOffset = parseInt(offset as string) || 0;
@@ -100,6 +100,12 @@ router.get('/', async (req, res) => {
} }
} }
// Filter by status (e.g., 'dropped', 'open', 'closed')
if (status) {
conditions.push(`status = $${params.length + 1}`);
params.push(status);
}
// Search filter (name, dba_name, city, company_name) // Search filter (name, dba_name, city, company_name)
if (search) { if (search) {
conditions.push(`(name ILIKE $${params.length + 1} OR dba_name ILIKE $${params.length + 1} OR city ILIKE $${params.length + 1})`); conditions.push(`(name ILIKE $${params.length + 1} OR dba_name ILIKE $${params.length + 1} OR city ILIKE $${params.length + 1})`);
@@ -161,6 +167,7 @@ router.get('/stats/crawl-status', async (req, res) => {
COUNT(*) FILTER (WHERE crawl_enabled = false OR crawl_enabled IS NULL) as disabled_count, COUNT(*) FILTER (WHERE crawl_enabled = false OR crawl_enabled IS NULL) as disabled_count,
COUNT(*) FILTER (WHERE dutchie_verified = true) as verified_count, COUNT(*) FILTER (WHERE dutchie_verified = true) as verified_count,
COUNT(*) FILTER (WHERE dutchie_verified = false OR dutchie_verified IS NULL) as unverified_count, COUNT(*) FILTER (WHERE dutchie_verified = false OR dutchie_verified IS NULL) as unverified_count,
COUNT(*) FILTER (WHERE status = 'dropped') as dropped_count,
COUNT(*) as total_count COUNT(*) as total_count
FROM dispensaries FROM dispensaries
`; `;
@@ -190,6 +197,34 @@ router.get('/stats/crawl-status', async (req, res) => {
} }
}); });
// Get dropped stores count (for dashboard alert)
router.get('/stats/dropped', async (req, res) => {
try {
const result = await pool.query(`
SELECT
COUNT(*) as dropped_count,
json_agg(json_build_object(
'id', id,
'name', name,
'city', city,
'state', state,
'dropped_at', updated_at
) ORDER BY updated_at DESC) FILTER (WHERE status = 'dropped') as dropped_stores
FROM dispensaries
WHERE status = 'dropped'
`);
const row = result.rows[0];
res.json({
dropped_count: parseInt(row.dropped_count) || 0,
dropped_stores: row.dropped_stores || []
});
} catch (error) {
console.error('Error fetching dropped stores:', error);
res.status(500).json({ error: 'Failed to fetch dropped stores' });
}
});
// Get single dispensary by slug or ID // Get single dispensary by slug or ID
router.get('/:slugOrId', async (req, res) => { router.get('/:slugOrId', async (req, res) => {
try { try {

View File

@@ -22,11 +22,17 @@ interface ProductClickEventPayload {
store_id?: string; store_id?: string;
brand_id?: string; brand_id?: string;
campaign_id?: string; campaign_id?: string;
dispensary_name?: string;
action: 'view' | 'open_store' | 'open_product' | 'compare' | 'other'; action: 'view' | 'open_store' | 'open_product' | 'compare' | 'other';
source: string; source: string;
page_type?: string; // Page where event occurred (e.g., StoreDetailPage, BrandsIntelligence) page_type?: string; // Page where event occurred (e.g., StoreDetailPage, BrandsIntelligence)
url_path?: string; // URL path for debugging url_path?: string; // URL path for debugging
occurred_at?: string; occurred_at?: string;
// Visitor location (from frontend IP geolocation)
visitor_city?: string;
visitor_state?: string;
visitor_lat?: number;
visitor_lng?: number;
} }
/** /**
@@ -77,13 +83,14 @@ router.post('/product-click', optionalAuthMiddleware, async (req: Request, res:
// Insert the event with enhanced fields // Insert the event with enhanced fields
await pool.query( await pool.query(
`INSERT INTO product_click_events `INSERT INTO product_click_events
(product_id, store_id, brand_id, campaign_id, action, source, user_id, ip_address, user_agent, occurred_at, event_type, page_type, url_path, device_type) (product_id, store_id, brand_id, campaign_id, dispensary_name, action, source, user_id, ip_address, user_agent, occurred_at, event_type, page_type, url_path, device_type, visitor_city, visitor_state, visitor_lat, visitor_lng)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14)`, VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19)`,
[ [
payload.product_id, payload.product_id,
payload.store_id || null, payload.store_id || null,
payload.brand_id || null, payload.brand_id || null,
payload.campaign_id || null, payload.campaign_id || null,
payload.dispensary_name || null,
payload.action, payload.action,
payload.source, payload.source,
userId, userId,
@@ -93,7 +100,11 @@ router.post('/product-click', optionalAuthMiddleware, async (req: Request, res:
'product_click', // event_type 'product_click', // event_type
payload.page_type || null, payload.page_type || null,
payload.url_path || null, payload.url_path || null,
deviceType deviceType,
payload.visitor_city || null,
payload.visitor_state || null,
payload.visitor_lat || null,
payload.visitor_lng || null
] ]
); );

View File

@@ -1,11 +1,29 @@
import { Router } from 'express'; import { Router } from 'express';
import { authMiddleware } from '../auth/middleware'; import { authMiddleware } from '../auth/middleware';
import { pool } from '../db/pool'; import { pool } from '../db/pool';
import { getImageUrl } from '../utils/minio';
const router = Router(); const router = Router();
router.use(authMiddleware); router.use(authMiddleware);
/**
* Convert local image path to proxy URL
* /images/products/... -> /img/products/...
*/
function getImageUrl(localPath: string): string {
if (!localPath) return '';
// If already a full URL, return as-is
if (localPath.startsWith('http')) return localPath;
// Convert /images/ path to /img/ proxy path
if (localPath.startsWith('/images/')) {
return '/img' + localPath.substring(7);
}
// Handle paths without leading slash
if (localPath.startsWith('images/')) {
return '/img/' + localPath.substring(7);
}
return '/img/' + localPath;
}
// Freshness threshold: data older than this is considered stale // Freshness threshold: data older than this is considered stale
const STALE_THRESHOLD_HOURS = 4; const STALE_THRESHOLD_HOURS = 4;

View File

@@ -463,7 +463,7 @@ router.get('/products', async (req: PublicApiRequest, res: Response) => {
// Filter by on special // Filter by on special
if (on_special === 'true' || on_special === '1') { if (on_special === 'true' || on_special === '1') {
whereClause += ` AND s.is_on_special = TRUE`; whereClause += ` AND s.special = TRUE`;
} }
// Search by name or brand // Search by name or brand
@@ -547,7 +547,7 @@ router.get('/products', async (req: PublicApiRequest, res: Response) => {
const { rows: countRows } = await pool.query(` const { rows: countRows } = await pool.query(`
SELECT COUNT(*) as total FROM store_products p SELECT COUNT(*) as total FROM store_products p
LEFT JOIN LATERAL ( LEFT JOIN LATERAL (
SELECT rec_min_price_cents / 100.0 as price_rec, med_min_price_cents / 100.0 as price_med, special as is_on_special FROM v_product_snapshots SELECT rec_min_price_cents / 100.0 as price_rec, med_min_price_cents / 100.0 as price_med, special FROM v_product_snapshots
WHERE store_product_id = p.id WHERE store_product_id = p.id
ORDER BY crawled_at DESC ORDER BY crawled_at DESC
LIMIT 1 LIMIT 1
@@ -1125,6 +1125,7 @@ router.get('/dispensaries', async (req: PublicApiRequest, res: Response) => {
SELECT SELECT
d.id, d.id,
d.name, d.name,
d.slug,
d.address1, d.address1,
d.address2, d.address2,
d.city, d.city,
@@ -1179,6 +1180,7 @@ router.get('/dispensaries', async (req: PublicApiRequest, res: Response) => {
const transformedDispensaries = dispensaries.map((d) => ({ const transformedDispensaries = dispensaries.map((d) => ({
id: d.id, id: d.id,
name: d.name, name: d.name,
slug: d.slug || null,
address1: d.address1, address1: d.address1,
address2: d.address2, address2: d.address2,
city: d.city, city: d.city,
@@ -1876,7 +1878,7 @@ router.get('/stats', async (req: PublicApiRequest, res: Response) => {
SELECT SELECT
(SELECT COUNT(*) FROM store_products) as product_count, (SELECT COUNT(*) FROM store_products) as product_count,
(SELECT COUNT(DISTINCT brand_name_raw) FROM store_products WHERE brand_name_raw IS NOT NULL) as brand_count, (SELECT COUNT(DISTINCT brand_name_raw) FROM store_products WHERE brand_name_raw IS NOT NULL) as brand_count,
(SELECT COUNT(*) FROM dispensaries WHERE crawl_enabled = true AND product_count > 0) as dispensary_count (SELECT COUNT(DISTINCT dispensary_id) FROM store_products) as dispensary_count
`); `);
const s = stats[0] || {}; const s = stats[0] || {};
@@ -1996,4 +1998,235 @@ router.get('/menu', async (req: PublicApiRequest, res: Response) => {
} }
}); });
// ============================================================
// VISITOR TRACKING & GEOLOCATION
// ============================================================
import crypto from 'crypto';
import { GeoLocation, lookupIP } from '../services/ip2location';
/**
* Get location from IP using local IP2Location database
*/
function getLocationFromIP(ip: string): GeoLocation | null {
return lookupIP(ip);
}
/**
* Hash IP for privacy (we don't store raw IPs)
*/
function hashIP(ip: string): string {
return crypto.createHash('sha256').update(ip).digest('hex').substring(0, 16);
}
/**
* POST /api/v1/visitor/track
* Track visitor location for analytics
*
* Body:
* - domain: string (required) - 'findagram.co', 'findadispo.com', etc.
* - page_path: string (optional) - current page path
* - session_id: string (optional) - client-generated session ID
* - referrer: string (optional) - document.referrer
*
* Returns:
* - location: { city, state, lat, lng } for client use
*/
router.post('/visitor/track', async (req: Request, res: Response) => {
try {
const { domain, page_path, session_id, referrer } = req.body;
if (!domain) {
return res.status(400).json({ error: 'domain is required' });
}
// Get client IP
const clientIp = (req.headers['x-forwarded-for'] as string)?.split(',')[0].trim() ||
req.headers['x-real-ip'] as string ||
req.ip ||
req.socket.remoteAddress ||
'';
// Get location from IP (local database lookup)
const location = getLocationFromIP(clientIp);
// Store visit (with hashed IP for privacy)
await pool.query(`
INSERT INTO visitor_locations (
ip_hash, city, state, state_code, country, country_code,
latitude, longitude, domain, page_path, referrer, user_agent, session_id
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)
`, [
hashIP(clientIp),
location?.city || null,
location?.state || null,
location?.stateCode || null,
location?.country || null,
location?.countryCode || null,
location?.lat || null,
location?.lng || null,
domain,
page_path || null,
referrer || null,
req.headers['user-agent'] || null,
session_id || null
]);
// Return location to client (for nearby dispensary feature)
res.json({
success: true,
location: location ? {
city: location.city,
state: location.state,
stateCode: location.stateCode,
lat: location.lat,
lng: location.lng
} : null
});
} catch (error: any) {
console.error('Visitor tracking error:', error);
// Don't fail the request - tracking is non-critical
res.json({
success: false,
location: null
});
}
});
/**
* GET /api/v1/visitor/location
* Get visitor location without tracking (just IP lookup)
*/
router.get('/visitor/location', (req: Request, res: Response) => {
try {
const clientIp = (req.headers['x-forwarded-for'] as string)?.split(',')[0].trim() ||
req.headers['x-real-ip'] as string ||
req.ip ||
req.socket.remoteAddress ||
'';
const location = getLocationFromIP(clientIp);
res.json({
success: true,
location: location ? {
city: location.city,
state: location.state,
stateCode: location.stateCode,
lat: location.lat,
lng: location.lng
} : null
});
} catch (error: any) {
console.error('Location lookup error:', error);
res.json({
success: false,
location: null
});
}
});
/**
* GET /api/v1/analytics/visitors
* Get visitor analytics (admin only - requires auth)
*
* Query params:
* - domain: filter by domain
* - days: number of days to look back (default: 30)
* - limit: max results (default: 50)
*/
router.get('/analytics/visitors', async (req: PublicApiRequest, res: Response) => {
try {
const scope = req.scope;
// Only allow internal keys
if (!scope || scope.type !== 'internal') {
return res.status(403).json({ error: 'Access denied - internal key required' });
}
const { domain, days = '30', limit = '50' } = req.query;
const daysNum = Math.min(parseInt(days as string, 10) || 30, 90);
const limitNum = Math.min(parseInt(limit as string, 10) || 50, 200);
let whereClause = 'WHERE created_at > NOW() - $1::interval';
const params: any[] = [`${daysNum} days`];
let paramIndex = 2;
if (domain) {
whereClause += ` AND domain = $${paramIndex}`;
params.push(domain);
paramIndex++;
}
// Get top locations
const { rows: topLocations } = await pool.query(`
SELECT
city,
state,
state_code,
country_code,
COUNT(*) as visit_count,
COUNT(DISTINCT session_id) as unique_sessions,
MAX(created_at) as last_visit
FROM visitor_locations
${whereClause}
GROUP BY city, state, state_code, country_code
ORDER BY visit_count DESC
LIMIT $${paramIndex}
`, [...params, limitNum]);
// Get daily totals
const { rows: dailyStats } = await pool.query(`
SELECT
DATE(created_at) as date,
COUNT(*) as visits,
COUNT(DISTINCT session_id) as unique_sessions
FROM visitor_locations
${whereClause}
GROUP BY DATE(created_at)
ORDER BY date DESC
LIMIT 30
`, params);
// Get totals
const { rows: totals } = await pool.query(`
SELECT
COUNT(*) as total_visits,
COUNT(DISTINCT session_id) as total_sessions,
COUNT(DISTINCT city || state_code) as unique_locations
FROM visitor_locations
${whereClause}
`, params);
res.json({
success: true,
period: {
days: daysNum,
domain: domain || 'all'
},
totals: totals[0],
top_locations: topLocations.map(l => ({
city: l.city,
state: l.state,
state_code: l.state_code,
country_code: l.country_code,
visits: parseInt(l.visit_count, 10),
unique_sessions: parseInt(l.unique_sessions, 10),
last_visit: l.last_visit
})),
daily_stats: dailyStats.map(d => ({
date: d.date,
visits: parseInt(d.visits, 10),
unique_sessions: parseInt(d.unique_sessions, 10)
}))
});
} catch (error: any) {
console.error('Visitor analytics error:', error);
res.status(500).json({
error: 'Failed to fetch visitor analytics',
message: error.message
});
}
});
export default router; export default router;

View File

@@ -444,7 +444,7 @@ router.post('/migration/cancel-pending-crawl-jobs', async (_req: Request, res: R
/** /**
* POST /api/tasks/migration/create-resync-tasks * POST /api/tasks/migration/create-resync-tasks
* Create product_resync tasks for all crawl-enabled dispensaries * Create product_refresh tasks for all crawl-enabled dispensaries
*/ */
router.post('/migration/create-resync-tasks', async (req: Request, res: Response) => { router.post('/migration/create-resync-tasks', async (req: Request, res: Response) => {
try { try {
@@ -474,7 +474,7 @@ router.post('/migration/create-resync-tasks', async (req: Request, res: Response
const hasActive = await taskService.hasActiveTask(disp.id); const hasActive = await taskService.hasActiveTask(disp.id);
if (!hasActive) { if (!hasActive) {
await taskService.createTask({ await taskService.createTask({
role: 'product_resync', role: 'product_refresh',
dispensary_id: disp.id, dispensary_id: disp.id,
platform: 'dutchie', platform: 'dutchie',
priority, priority,

View File

@@ -0,0 +1,652 @@
/**
* Worker Registry API Routes
*
* Dynamic worker management - workers register on startup, get assigned names,
* and report heartbeats. Everything is API-driven, no hardcoding.
*
* Endpoints:
* POST /api/worker-registry/register - Worker reports for duty
* POST /api/worker-registry/heartbeat - Worker heartbeat
* POST /api/worker-registry/deregister - Worker signing off
* GET /api/worker-registry/workers - List all workers (for dashboard)
* GET /api/worker-registry/workers/:id - Get specific worker
* POST /api/worker-registry/cleanup - Mark stale workers offline
*
* GET /api/worker-registry/names - List all names in pool
* POST /api/worker-registry/names - Add names to pool
* DELETE /api/worker-registry/names/:name - Remove name from pool
*
* GET /api/worker-registry/roles - List available task roles
* POST /api/worker-registry/roles - Add a new role (future)
*/
import { Router, Request, Response } from 'express';
import { pool } from '../db/pool';
import os from 'os';
const router = Router();
// ============================================================
// WORKER REGISTRATION
// ============================================================
/**
* POST /api/worker-registry/register
* Worker reports for duty - gets assigned a friendly name
*
* Body:
* - role: string (optional) - task role, or null for role-agnostic workers
* - worker_id: string (optional) - custom ID, auto-generated if not provided
* - pod_name: string (optional) - k8s pod name
* - hostname: string (optional) - machine hostname
* - metadata: object (optional) - additional worker info
*
* Returns:
* - worker_id: assigned worker ID
* - friendly_name: assigned name from pool
* - role: confirmed role (or null if agnostic)
* - message: welcome message
*/
router.post('/register', async (req: Request, res: Response) => {
try {
const {
role = null, // Role is now optional - null means agnostic
worker_id,
pod_name,
hostname,
ip_address,
metadata = {}
} = req.body;
// Generate worker_id if not provided
const finalWorkerId = worker_id || `worker-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const finalHostname = hostname || os.hostname();
const clientIp = ip_address || req.ip || req.socket.remoteAddress;
// Check if worker already registered
const existing = await pool.query(
'SELECT id, friendly_name, status FROM worker_registry WHERE worker_id = $1',
[finalWorkerId]
);
if (existing.rows.length > 0) {
// Re-activate existing worker
const { rows } = await pool.query(`
UPDATE worker_registry
SET status = 'active',
role = $1,
pod_name = $2,
hostname = $3,
ip_address = $4,
last_heartbeat_at = NOW(),
started_at = NOW(),
metadata = $5,
updated_at = NOW()
WHERE worker_id = $6
RETURNING id, worker_id, friendly_name, role
`, [role, pod_name, finalHostname, clientIp, metadata, finalWorkerId]);
const worker = rows[0];
const roleMsg = role ? `for ${role}` : 'as role-agnostic';
console.log(`[WorkerRegistry] Worker "${worker.friendly_name}" (${finalWorkerId}) re-registered ${roleMsg}`);
return res.json({
success: true,
worker_id: worker.worker_id,
friendly_name: worker.friendly_name,
role: worker.role,
message: role
? `Welcome back, ${worker.friendly_name}! You are assigned to ${role}.`
: `Welcome back, ${worker.friendly_name}! You are ready to take any task.`
});
}
// Assign a friendly name
const nameResult = await pool.query('SELECT assign_worker_name($1) as name', [finalWorkerId]);
const friendlyName = nameResult.rows[0].name;
// Register the worker
const { rows } = await pool.query(`
INSERT INTO worker_registry (
worker_id, friendly_name, role, pod_name, hostname, ip_address, status, metadata
) VALUES ($1, $2, $3, $4, $5, $6, 'active', $7)
RETURNING id, worker_id, friendly_name, role
`, [finalWorkerId, friendlyName, role, pod_name, finalHostname, clientIp, metadata]);
const worker = rows[0];
const roleMsg = role ? `for ${role}` : 'as role-agnostic';
console.log(`[WorkerRegistry] New worker "${friendlyName}" (${finalWorkerId}) reporting for duty ${roleMsg}`);
res.json({
success: true,
worker_id: worker.worker_id,
friendly_name: worker.friendly_name,
role: worker.role,
message: role
? `Hello ${friendlyName}! You are now registered for ${role}. Ready for work!`
: `Hello ${friendlyName}! You are ready to take any task from the pool.`
});
} catch (error: any) {
console.error('[WorkerRegistry] Registration error:', error);
res.status(500).json({ success: false, error: error.message });
}
});
/**
* POST /api/worker-registry/heartbeat
* Worker sends heartbeat to stay alive
*
* Body:
* - worker_id: string (required)
* - current_task_id: number (optional) - task currently being processed
* - status: string (optional) - 'active', 'idle'
*/
router.post('/heartbeat', async (req: Request, res: Response) => {
try {
const { worker_id, current_task_id, status = 'active', resources } = req.body;
if (!worker_id) {
return res.status(400).json({ success: false, error: 'worker_id is required' });
}
// Store resources in metadata jsonb column
const { rows } = await pool.query(`
UPDATE worker_registry
SET last_heartbeat_at = NOW(),
current_task_id = $1,
status = $2,
metadata = COALESCE(metadata, '{}'::jsonb) || COALESCE($4::jsonb, '{}'::jsonb),
updated_at = NOW()
WHERE worker_id = $3
RETURNING id, friendly_name, status
`, [current_task_id || null, status, worker_id, resources ? JSON.stringify(resources) : null]);
if (rows.length === 0) {
return res.status(404).json({ success: false, error: 'Worker not found - please register first' });
}
res.json({
success: true,
worker: rows[0]
});
} catch (error: any) {
console.error('[WorkerRegistry] Heartbeat error:', error);
res.status(500).json({ success: false, error: error.message });
}
});
/**
* POST /api/worker-registry/task-completed
* Worker reports task completion
*
* Body:
* - worker_id: string (required)
* - success: boolean (required)
*/
router.post('/task-completed', async (req: Request, res: Response) => {
try {
const { worker_id, success } = req.body;
if (!worker_id) {
return res.status(400).json({ success: false, error: 'worker_id is required' });
}
const incrementField = success ? 'tasks_completed' : 'tasks_failed';
const { rows } = await pool.query(`
UPDATE worker_registry
SET ${incrementField} = ${incrementField} + 1,
last_task_at = NOW(),
current_task_id = NULL,
status = 'idle',
updated_at = NOW()
WHERE worker_id = $1
RETURNING id, friendly_name, tasks_completed, tasks_failed
`, [worker_id]);
if (rows.length === 0) {
return res.status(404).json({ success: false, error: 'Worker not found' });
}
res.json({ success: true, worker: rows[0] });
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
/**
* POST /api/worker-registry/deregister
* Worker signing off (graceful shutdown)
*
* Body:
* - worker_id: string (required)
*/
router.post('/deregister', async (req: Request, res: Response) => {
try {
const { worker_id } = req.body;
if (!worker_id) {
return res.status(400).json({ success: false, error: 'worker_id is required' });
}
// Release the name back to the pool
await pool.query('SELECT release_worker_name($1)', [worker_id]);
// Mark as terminated
const { rows } = await pool.query(`
UPDATE worker_registry
SET status = 'terminated',
current_task_id = NULL,
updated_at = NOW()
WHERE worker_id = $1
RETURNING id, friendly_name
`, [worker_id]);
if (rows.length === 0) {
return res.status(404).json({ success: false, error: 'Worker not found' });
}
console.log(`[WorkerRegistry] Worker "${rows[0].friendly_name}" (${worker_id}) signed off`);
res.json({
success: true,
message: `Goodbye ${rows[0].friendly_name}! Thanks for your work.`
});
} catch (error: any) {
console.error('[WorkerRegistry] Deregister error:', error);
res.status(500).json({ success: false, error: error.message });
}
});
// ============================================================
// WORKER LISTING (for Dashboard)
// ============================================================
/**
* GET /api/worker-registry/workers
* List all workers (for dashboard)
*
* Query params:
* - status: filter by status (active, idle, offline, all)
* - role: filter by role
* - include_terminated: include terminated workers (default: false)
*/
router.get('/workers', async (req: Request, res: Response) => {
try {
const { status, role, include_terminated = 'false' } = req.query;
let whereClause = include_terminated === 'true' ? 'WHERE 1=1' : "WHERE status != 'terminated'";
const params: any[] = [];
let paramIndex = 1;
if (status && status !== 'all') {
whereClause += ` AND status = $${paramIndex}`;
params.push(status);
paramIndex++;
}
if (role) {
whereClause += ` AND role = $${paramIndex}`;
params.push(role);
paramIndex++;
}
const { rows } = await pool.query(`
SELECT
id,
worker_id,
friendly_name,
role,
pod_name,
hostname,
ip_address,
status,
started_at,
last_heartbeat_at,
last_task_at,
tasks_completed,
tasks_failed,
current_task_id,
metadata,
EXTRACT(EPOCH FROM (NOW() - last_heartbeat_at)) as seconds_since_heartbeat,
CASE
WHEN status = 'offline' OR status = 'terminated' THEN status
WHEN last_heartbeat_at < NOW() - INTERVAL '2 minutes' THEN 'stale'
WHEN current_task_id IS NOT NULL THEN 'busy'
ELSE 'ready'
END as health_status,
created_at
FROM worker_registry
${whereClause}
ORDER BY
CASE status
WHEN 'active' THEN 1
WHEN 'idle' THEN 2
WHEN 'offline' THEN 3
ELSE 4
END,
last_heartbeat_at DESC
`, params);
// Get summary counts
const { rows: summary } = await pool.query(`
SELECT
COUNT(*) FILTER (WHERE status = 'active') as active_count,
COUNT(*) FILTER (WHERE status = 'idle') as idle_count,
COUNT(*) FILTER (WHERE status = 'offline') as offline_count,
COUNT(*) FILTER (WHERE status != 'terminated') as total_count,
COUNT(DISTINCT role) FILTER (WHERE status IN ('active', 'idle')) as active_roles
FROM worker_registry
`);
res.json({
success: true,
workers: rows,
summary: summary[0]
});
} catch (error: any) {
console.error('[WorkerRegistry] List workers error:', error);
res.status(500).json({ success: false, error: error.message });
}
});
/**
* GET /api/worker-registry/workers/:workerId
* Get specific worker details
*/
router.get('/workers/:workerId', async (req: Request, res: Response) => {
try {
const { workerId } = req.params;
const { rows } = await pool.query(`
SELECT * FROM worker_registry WHERE worker_id = $1
`, [workerId]);
if (rows.length === 0) {
return res.status(404).json({ success: false, error: 'Worker not found' });
}
res.json({ success: true, worker: rows[0] });
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
/**
* DELETE /api/worker-registry/workers/:workerId
* Remove a worker (admin action)
*/
router.delete('/workers/:workerId', async (req: Request, res: Response) => {
try {
const { workerId } = req.params;
// Release name
await pool.query('SELECT release_worker_name($1)', [workerId]);
// Delete worker
const { rows } = await pool.query(`
DELETE FROM worker_registry WHERE worker_id = $1 RETURNING friendly_name
`, [workerId]);
if (rows.length === 0) {
return res.status(404).json({ success: false, error: 'Worker not found' });
}
res.json({ success: true, message: `Worker ${rows[0].friendly_name} removed` });
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
/**
* POST /api/worker-registry/cleanup
* Mark stale workers as offline
*
* Body:
* - stale_threshold_minutes: number (default: 5)
*/
router.post('/cleanup', async (req: Request, res: Response) => {
try {
const { stale_threshold_minutes = 5 } = req.body;
const { rows } = await pool.query(
'SELECT mark_stale_workers($1) as count',
[stale_threshold_minutes]
);
res.json({
success: true,
stale_workers_marked: rows[0].count,
message: `Marked ${rows[0].count} stale workers as offline`
});
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
// ============================================================
// NAME POOL MANAGEMENT
// ============================================================
/**
* GET /api/worker-registry/names
* List all names in the pool
*/
router.get('/names', async (_req: Request, res: Response) => {
try {
const { rows } = await pool.query(`
SELECT
id,
name,
in_use,
assigned_to,
assigned_at
FROM worker_name_pool
ORDER BY in_use DESC, name ASC
`);
const { rows: summary } = await pool.query(`
SELECT
COUNT(*) as total,
COUNT(*) FILTER (WHERE in_use = true) as in_use,
COUNT(*) FILTER (WHERE in_use = false) as available
FROM worker_name_pool
`);
res.json({
success: true,
names: rows,
summary: summary[0]
});
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
/**
* POST /api/worker-registry/names
* Add names to the pool
*
* Body:
* - names: string[] (required) - array of names to add
*/
router.post('/names', async (req: Request, res: Response) => {
try {
const { names } = req.body;
if (!names || !Array.isArray(names) || names.length === 0) {
return res.status(400).json({ success: false, error: 'names array is required' });
}
const values = names.map(n => `('${n.replace(/'/g, "''")}')`).join(', ');
const { rowCount } = await pool.query(`
INSERT INTO worker_name_pool (name)
VALUES ${values}
ON CONFLICT (name) DO NOTHING
`);
res.json({
success: true,
added: rowCount,
message: `Added ${rowCount} new names to the pool`
});
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
/**
* DELETE /api/worker-registry/names/:name
* Remove a name from the pool (only if not in use)
*/
router.delete('/names/:name', async (req: Request, res: Response) => {
try {
const { name } = req.params;
const { rows } = await pool.query(`
DELETE FROM worker_name_pool
WHERE name = $1 AND in_use = false
RETURNING name
`, [name]);
if (rows.length === 0) {
return res.status(400).json({
success: false,
error: 'Name not found or currently in use'
});
}
res.json({ success: true, message: `Name "${name}" removed from pool` });
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
// ============================================================
// ROLE MANAGEMENT
// ============================================================
/**
* GET /api/worker-registry/roles
* List available task roles
*/
router.get('/roles', async (_req: Request, res: Response) => {
// These are the roles the task handlers support
const roles = [
{
id: 'product_refresh',
name: 'Product Refresh',
description: 'Re-crawl dispensary products for price/stock changes',
handler: 'handleProductRefresh'
},
{
id: 'product_discovery',
name: 'Product Discovery',
description: 'Initial product discovery for new dispensaries',
handler: 'handleProductDiscovery'
},
{
id: 'store_discovery',
name: 'Store Discovery',
description: 'Discover new dispensary locations',
handler: 'handleStoreDiscovery'
},
{
id: 'entry_point_discovery',
name: 'Entry Point Discovery',
description: 'Resolve platform IDs from menu URLs',
handler: 'handleEntryPointDiscovery'
},
{
id: 'analytics_refresh',
name: 'Analytics Refresh',
description: 'Refresh materialized views and analytics',
handler: 'handleAnalyticsRefresh'
}
];
// Get active worker counts per role
try {
const { rows } = await pool.query(`
SELECT role, COUNT(*) as worker_count
FROM worker_registry
WHERE status IN ('active', 'idle')
GROUP BY role
`);
const countMap = new Map(rows.map(r => [r.role, parseInt(r.worker_count)]));
const rolesWithCounts = roles.map(r => ({
...r,
active_workers: countMap.get(r.id) || 0
}));
res.json({ success: true, roles: rolesWithCounts });
} catch {
// If table doesn't exist yet, just return roles without counts
res.json({ success: true, roles: roles.map(r => ({ ...r, active_workers: 0 })) });
}
});
/**
* GET /api/worker-registry/capacity
* Get capacity planning info
*/
router.get('/capacity', async (_req: Request, res: Response) => {
try {
// Get worker counts by role
const { rows: workerCounts } = await pool.query(`
SELECT role, COUNT(*) as count
FROM worker_registry
WHERE status IN ('active', 'idle')
GROUP BY role
`);
// Get pending task counts by role (if worker_tasks exists)
let taskCounts: any[] = [];
try {
const result = await pool.query(`
SELECT role, COUNT(*) as pending_count
FROM worker_tasks
WHERE status = 'pending'
GROUP BY role
`);
taskCounts = result.rows;
} catch {
// worker_tasks might not exist yet
}
// Get crawl-enabled store count
const storeCountResult = await pool.query(`
SELECT COUNT(*) as count
FROM dispensaries
WHERE crawl_enabled = true AND platform_dispensary_id IS NOT NULL
`);
const totalStores = parseInt(storeCountResult.rows[0].count);
const workerMap = new Map(workerCounts.map(r => [r.role, parseInt(r.count)]));
const taskMap = new Map(taskCounts.map(r => [r.role, parseInt(r.pending_count)]));
const roles = ['product_refresh', 'product_discovery', 'store_discovery', 'entry_point_discovery', 'analytics_refresh'];
const capacity = roles.map(role => ({
role,
active_workers: workerMap.get(role) || 0,
pending_tasks: taskMap.get(role) || 0,
// Rough estimate: 20 seconds per task, 4-hour cycle
tasks_per_worker_per_cycle: 720,
workers_needed_for_all_stores: Math.ceil(totalStores / 720)
}));
res.json({
success: true,
total_stores: totalStores,
capacity
});
} catch (error: any) {
res.status(500).json({ success: false, error: error.message });
}
});
export default router;

View File

@@ -109,14 +109,14 @@ export class ProxyRotator {
username, username,
password, password,
protocol, protocol,
is_active as "isActive", active as "isActive",
last_used_at as "lastUsedAt", last_tested_at as "lastUsedAt",
failure_count as "failureCount", failure_count as "failureCount",
success_count as "successCount", 0 as "successCount",
avg_response_time_ms as "avgResponseTimeMs" response_time_ms as "avgResponseTimeMs"
FROM proxies FROM proxies
WHERE is_active = true WHERE active = true
ORDER BY failure_count ASC, last_used_at ASC NULLS FIRST ORDER BY failure_count ASC, last_tested_at ASC NULLS FIRST
`); `);
this.proxies = result.rows; this.proxies = result.rows;
@@ -192,11 +192,11 @@ export class ProxyRotator {
UPDATE proxies UPDATE proxies
SET SET
failure_count = failure_count + 1, failure_count = failure_count + 1,
last_failure_at = NOW(), updated_at = NOW(),
last_error = $2, test_result = $2,
is_active = CASE WHEN failure_count >= 4 THEN false ELSE is_active END active = CASE WHEN failure_count >= 4 THEN false ELSE active END
WHERE id = $1 WHERE id = $1
`, [proxyId, error || null]); `, [proxyId, error || 'failed']);
} catch (err) { } catch (err) {
console.error(`[ProxyRotator] Failed to update proxy ${proxyId}:`, err); console.error(`[ProxyRotator] Failed to update proxy ${proxyId}:`, err);
} }
@@ -226,12 +226,13 @@ export class ProxyRotator {
await this.pool.query(` await this.pool.query(`
UPDATE proxies UPDATE proxies
SET SET
success_count = success_count + 1, last_tested_at = NOW(),
last_used_at = NOW(), test_result = 'success',
avg_response_time_ms = CASE response_time_ms = CASE
WHEN avg_response_time_ms IS NULL THEN $2 WHEN response_time_ms IS NULL THEN $2
ELSE (avg_response_time_ms * 0.8) + ($2 * 0.2) ELSE (response_time_ms * 0.8 + $2 * 0.2)::integer
END END,
updated_at = NOW()
WHERE id = $1 WHERE id = $1
`, [proxyId, responseTimeMs || null]); `, [proxyId, responseTimeMs || null]);
} catch (err) { } catch (err) {

View File

@@ -0,0 +1,134 @@
/**
* IP2Location Service
*
* Uses local IP2Location LITE DB3 database for IP geolocation.
* No external API calls, no rate limits.
*
* Database: IP2Location LITE DB3 (free, monthly updates)
* Fields: country, region, city, latitude, longitude
*/
import path from 'path';
import fs from 'fs';
// @ts-ignore - no types for ip2location-nodejs
const { IP2Location } = require('ip2location-nodejs');
const DB_PATH = process.env.IP2LOCATION_DB_PATH ||
path.join(__dirname, '../../data/ip2location/IP2LOCATION-LITE-DB5.BIN');
let ip2location: any = null;
let dbLoaded = false;
/**
* Initialize IP2Location database
*/
export function initIP2Location(): boolean {
if (dbLoaded) return true;
try {
if (!fs.existsSync(DB_PATH)) {
console.warn(`IP2Location database not found at: ${DB_PATH}`);
console.warn('Run: ./scripts/download-ip2location.sh to download');
return false;
}
ip2location = new IP2Location();
ip2location.open(DB_PATH);
dbLoaded = true;
console.log('IP2Location database loaded successfully');
return true;
} catch (err) {
console.error('Failed to load IP2Location database:', err);
return false;
}
}
/**
* Close IP2Location database
*/
export function closeIP2Location(): void {
if (ip2location) {
ip2location.close();
ip2location = null;
dbLoaded = false;
}
}
export interface GeoLocation {
city: string | null;
state: string | null;
stateCode: string | null;
country: string | null;
countryCode: string | null;
lat: number | null;
lng: number | null;
}
/**
* Lookup IP address location
*
* @param ip - IPv4 or IPv6 address
* @returns Location data or null if not found
*/
export function lookupIP(ip: string): GeoLocation | null {
// Skip private/localhost IPs
if (!ip || ip === '127.0.0.1' || ip === '::1' ||
ip.startsWith('192.168.') || ip.startsWith('10.') ||
ip.startsWith('172.16.') || ip.startsWith('172.17.') ||
ip.startsWith('::ffff:127.') || ip.startsWith('::ffff:192.168.') ||
ip.startsWith('::ffff:10.')) {
return null;
}
// Strip IPv6 prefix if present
const cleanIP = ip.replace(/^::ffff:/, '');
// Initialize on first use if not already loaded
if (!dbLoaded) {
if (!initIP2Location()) {
return null;
}
}
try {
const result = ip2location.getAll(cleanIP);
if (!result || result.ip === '?' || result.countryShort === '-') {
return null;
}
// DB3 LITE doesn't include lat/lng - would need DB5+ for that
const lat = typeof result.latitude === 'number' && result.latitude !== 0 ? result.latitude : null;
const lng = typeof result.longitude === 'number' && result.longitude !== 0 ? result.longitude : null;
return {
city: result.city !== '-' ? result.city : null,
state: result.region !== '-' ? result.region : null,
stateCode: null, // DB3 doesn't include state codes
country: result.countryLong !== '-' ? result.countryLong : null,
countryCode: result.countryShort !== '-' ? result.countryShort : null,
lat,
lng,
};
} catch (err) {
console.error('IP2Location lookup error:', err);
return null;
}
}
/**
* Check if IP2Location database is available
*/
export function isIP2LocationAvailable(): boolean {
if (dbLoaded) return true;
return fs.existsSync(DB_PATH);
}
// Export singleton-style interface
export default {
init: initIP2Location,
close: closeIP2Location,
lookup: lookupIP,
isAvailable: isIP2LocationAvailable,
};

View File

@@ -3,7 +3,7 @@ import StealthPlugin from 'puppeteer-extra-plugin-stealth';
import { Browser, Page } from 'puppeteer'; import { Browser, Page } from 'puppeteer';
import { SocksProxyAgent } from 'socks-proxy-agent'; import { SocksProxyAgent } from 'socks-proxy-agent';
import { pool } from '../db/pool'; import { pool } from '../db/pool';
import { uploadImageFromUrl, getImageUrl } from '../utils/minio'; import { downloadProductImageLegacy } from '../utils/image-storage';
import { logger } from './logger'; import { logger } from './logger';
import { registerScraper, updateScraperStats, completeScraper } from '../routes/scraper-monitor'; import { registerScraper, updateScraperStats, completeScraper } from '../routes/scraper-monitor';
import { incrementProxyFailure, getActiveProxy, isBotDetectionError, putProxyInTimeout } from './proxy'; import { incrementProxyFailure, getActiveProxy, isBotDetectionError, putProxyInTimeout } from './proxy';
@@ -767,7 +767,8 @@ export async function saveProducts(storeId: number, categoryId: number, products
if (product.imageUrl && !localImagePath) { if (product.imageUrl && !localImagePath) {
try { try {
localImagePath = await uploadImageFromUrl(product.imageUrl, productId); const result = await downloadProductImageLegacy(product.imageUrl, 0, productId);
localImagePath = result.urls?.original || null;
await client.query(` await client.query(`
UPDATE products UPDATE products
SET local_image_path = $1 SET local_image_path = $1

View File

@@ -1,13 +1,21 @@
/** /**
* Entry Point Discovery Handler * Entry Point Discovery Handler
* *
* Detects menu type and resolves platform IDs for a discovered store. * Resolves platform IDs for a discovered store using Dutchie GraphQL.
* This is the step between store_discovery and product_discovery. * This is the step between store_discovery and product_discovery.
* *
* TODO: Integrate with platform ID resolution when available * Flow:
* 1. Load dispensary info from database
* 2. Extract slug from menu_url
* 3. Start stealth session (fingerprint + optional proxy)
* 4. Query Dutchie GraphQL to resolve slug → platform_dispensary_id
* 5. Update dispensary record with resolved ID
* 6. Queue product_discovery task if successful
*/ */
import { TaskContext, TaskResult } from '../task-worker'; import { TaskContext, TaskResult } from '../task-worker';
import { startSession, endSession } from '../../platforms/dutchie';
import { resolveDispensaryIdWithDetails } from '../../platforms/dutchie/queries';
export async function handleEntryPointDiscovery(ctx: TaskContext): Promise<TaskResult> { export async function handleEntryPointDiscovery(ctx: TaskContext): Promise<TaskResult> {
const { pool, task } = ctx; const { pool, task } = ctx;
@@ -18,9 +26,11 @@ export async function handleEntryPointDiscovery(ctx: TaskContext): Promise<TaskR
} }
try { try {
// Get dispensary info // ============================================================
// STEP 1: Load dispensary info
// ============================================================
const dispResult = await pool.query(` const dispResult = await pool.query(`
SELECT id, name, menu_url, platform_dispensary_id, menu_type SELECT id, name, menu_url, platform_dispensary_id, menu_type, state
FROM dispensaries FROM dispensaries
WHERE id = $1 WHERE id = $1
`, [dispensaryId]); `, [dispensaryId]);
@@ -33,7 +43,7 @@ export async function handleEntryPointDiscovery(ctx: TaskContext): Promise<TaskR
// If already has platform_dispensary_id, we're done // If already has platform_dispensary_id, we're done
if (dispensary.platform_dispensary_id) { if (dispensary.platform_dispensary_id) {
console.log(`[EntryPointDiscovery] Dispensary ${dispensaryId} already has platform ID`); console.log(`[EntryPointDiscovery] Dispensary ${dispensaryId} already has platform ID: ${dispensary.platform_dispensary_id}`);
return { return {
success: true, success: true,
alreadyResolved: true, alreadyResolved: true,
@@ -46,9 +56,12 @@ export async function handleEntryPointDiscovery(ctx: TaskContext): Promise<TaskR
return { success: false, error: `Dispensary ${dispensaryId} has no menu_url` }; return { success: false, error: `Dispensary ${dispensaryId} has no menu_url` };
} }
console.log(`[EntryPointDiscovery] Would resolve platform ID for ${dispensary.name} from ${menuUrl}`); console.log(`[EntryPointDiscovery] Resolving platform ID for ${dispensary.name}`);
console.log(`[EntryPointDiscovery] Menu URL: ${menuUrl}`);
// Extract slug from menu URL // ============================================================
// STEP 2: Extract slug from menu URL
// ============================================================
let slug: string | null = null; let slug: string | null = null;
const embeddedMatch = menuUrl.match(/\/embedded-menu\/([^/?]+)/); const embeddedMatch = menuUrl.match(/\/embedded-menu\/([^/?]+)/);
@@ -61,21 +74,109 @@ export async function handleEntryPointDiscovery(ctx: TaskContext): Promise<TaskR
} }
if (!slug) { if (!slug) {
// Mark as non-dutchie menu type
await pool.query(`
UPDATE dispensaries
SET menu_type = 'unknown', updated_at = NOW()
WHERE id = $1
`, [dispensaryId]);
return { return {
success: false, success: false,
error: `Could not extract slug from menu_url: ${menuUrl}`, error: `Could not extract slug from menu_url: ${menuUrl}`,
}; };
} }
// TODO: Integrate with actual platform ID resolution console.log(`[EntryPointDiscovery] Extracted slug: ${slug}`);
// For now, mark the task as needing manual resolution
console.log(`[EntryPointDiscovery] Found slug: ${slug} - manual resolution needed`); await ctx.heartbeat();
// ============================================================
// STEP 3: Start stealth session
// ============================================================
const session = startSession(dispensary.state || 'AZ', 'America/Phoenix');
console.log(`[EntryPointDiscovery] Session started: ${session.sessionId}`);
try {
// ============================================================
// STEP 4: Resolve platform ID via GraphQL
// ============================================================
console.log(`[EntryPointDiscovery] Querying Dutchie GraphQL for slug: ${slug}`);
const result = await resolveDispensaryIdWithDetails(slug);
if (!result.dispensaryId) {
// Resolution failed - could be 403, 404, or invalid response
const reason = result.httpStatus
? `HTTP ${result.httpStatus}`
: result.error || 'Unknown error';
console.log(`[EntryPointDiscovery] Failed to resolve ${slug}: ${reason}`);
// Mark as failed resolution but keep menu_type as dutchie
await pool.query(`
UPDATE dispensaries
SET
menu_type = CASE
WHEN $2 = 404 THEN 'removed'
WHEN $2 = 403 THEN 'blocked'
ELSE 'dutchie'
END,
updated_at = NOW()
WHERE id = $1
`, [dispensaryId, result.httpStatus || 0]);
return {
success: false,
error: `Could not resolve platform ID: ${reason}`,
slug,
httpStatus: result.httpStatus,
};
}
const platformId = result.dispensaryId;
console.log(`[EntryPointDiscovery] Resolved ${slug} -> ${platformId}`);
await ctx.heartbeat();
// ============================================================
// STEP 5: Update dispensary with resolved ID
// ============================================================
await pool.query(`
UPDATE dispensaries
SET
platform_dispensary_id = $2,
menu_type = 'dutchie',
crawl_enabled = true,
updated_at = NOW()
WHERE id = $1
`, [dispensaryId, platformId]);
console.log(`[EntryPointDiscovery] Updated dispensary ${dispensaryId} with platform ID`);
// ============================================================
// STEP 6: Queue product_discovery task
// ============================================================
await pool.query(`
INSERT INTO worker_tasks (role, dispensary_id, priority, scheduled_for)
VALUES ('product_discovery', $1, 5, NOW())
ON CONFLICT DO NOTHING
`, [dispensaryId]);
console.log(`[EntryPointDiscovery] Queued product_discovery task for dispensary ${dispensaryId}`);
return { return {
success: true, success: true,
message: 'Slug extracted, awaiting platform ID resolution', platformId,
slug, slug,
queuedProductDiscovery: true,
}; };
} finally {
// Always end session
endSession();
}
} catch (error: unknown) { } catch (error: unknown) {
const errorMessage = error instanceof Error ? error.message : 'Unknown error'; const errorMessage = error instanceof Error ? error.message : 'Unknown error';
console.error(`[EntryPointDiscovery] Error for dispensary ${dispensaryId}:`, errorMessage); console.error(`[EntryPointDiscovery] Error for dispensary ${dispensaryId}:`, errorMessage);

View File

@@ -4,7 +4,7 @@
* Exports all task handlers for the task worker. * Exports all task handlers for the task worker.
*/ */
export { handleProductResync } from './product-resync'; export { handleProductRefresh } from './product-refresh';
export { handleProductDiscovery } from './product-discovery'; export { handleProductDiscovery } from './product-discovery';
export { handleStoreDiscovery } from './store-discovery'; export { handleStoreDiscovery } from './store-discovery';
export { handleEntryPointDiscovery } from './entry-point-discovery'; export { handleEntryPointDiscovery } from './entry-point-discovery';

View File

@@ -6,11 +6,11 @@
*/ */
import { TaskContext, TaskResult } from '../task-worker'; import { TaskContext, TaskResult } from '../task-worker';
import { handleProductResync } from './product-resync'; import { handleProductRefresh } from './product-refresh';
export async function handleProductDiscovery(ctx: TaskContext): Promise<TaskResult> { export async function handleProductDiscovery(ctx: TaskContext): Promise<TaskResult> {
// Product discovery is essentially the same as resync for the first time // Product discovery is essentially the same as refresh for the first time
// The main difference is in when this task is triggered (new store vs scheduled) // The main difference is in when this task is triggered (new store vs scheduled)
console.log(`[ProductDiscovery] Starting initial product fetch for dispensary ${ctx.task.dispensary_id}`); console.log(`[ProductDiscovery] Starting initial product fetch for dispensary ${ctx.task.dispensary_id}`);
return handleProductResync(ctx); return handleProductRefresh(ctx);
} }

View File

@@ -1,5 +1,5 @@
/** /**
* Product Resync Handler * Product Refresh Handler
* *
* Re-crawls a store to capture price/stock changes using the GraphQL pipeline. * Re-crawls a store to capture price/stock changes using the GraphQL pipeline.
* *
@@ -31,12 +31,12 @@ import {
const normalizer = new DutchieNormalizer(); const normalizer = new DutchieNormalizer();
export async function handleProductResync(ctx: TaskContext): Promise<TaskResult> { export async function handleProductRefresh(ctx: TaskContext): Promise<TaskResult> {
const { pool, task } = ctx; const { pool, task } = ctx;
const dispensaryId = task.dispensary_id; const dispensaryId = task.dispensary_id;
if (!dispensaryId) { if (!dispensaryId) {
return { success: false, error: 'No dispensary_id specified for product_resync task' }; return { success: false, error: 'No dispensary_id specified for product_refresh task' };
} }
try { try {

View File

@@ -17,7 +17,7 @@ export {
export { TaskWorker, TaskContext, TaskResult } from './task-worker'; export { TaskWorker, TaskContext, TaskResult } from './task-worker';
export { export {
handleProductResync, handleProductRefresh,
handleProductDiscovery, handleProductDiscovery,
handleStoreDiscovery, handleStoreDiscovery,
handleEntryPointDiscovery, handleEntryPointDiscovery,

View File

@@ -0,0 +1,93 @@
#!/usr/bin/env npx tsx
/**
* Start Pod - Simulates a Kubernetes pod locally
*
* Starts 5 workers with a pod name from the predefined list.
*
* Usage:
* npx tsx src/tasks/start-pod.ts <pod-index>
* npx tsx src/tasks/start-pod.ts 0 # Starts pod "Aethelgard" with 5 workers
* npx tsx src/tasks/start-pod.ts 1 # Starts pod "Xylos" with 5 workers
*/
import { spawn } from 'child_process';
import path from 'path';
const POD_NAMES = [
'Aethelgard',
'Xylos',
'Kryll',
'Coriolis',
'Dimidium',
'Veridia',
'Zetani',
'Talos IV',
'Onyx',
'Celestia',
'Gormand',
'Betha',
'Ragnar',
'Syphon',
'Axiom',
'Nadir',
'Terra Nova',
'Acheron',
'Nexus',
'Vespera',
'Helios Prime',
'Oasis',
'Mordina',
'Cygnus',
'Umbra',
];
const WORKERS_PER_POD = 5;
async function main() {
const podIndex = parseInt(process.argv[2] ?? '0', 10);
if (podIndex < 0 || podIndex >= POD_NAMES.length) {
console.error(`Invalid pod index: ${podIndex}. Must be 0-${POD_NAMES.length - 1}`);
process.exit(1);
}
const podName = POD_NAMES[podIndex];
console.log(`[Pod] Starting pod "${podName}" with ${WORKERS_PER_POD} workers...`);
const workerScript = path.join(__dirname, 'task-worker.ts');
const workers: ReturnType<typeof spawn>[] = [];
for (let i = 1; i <= WORKERS_PER_POD; i++) {
const workerId = `${podName}-worker-${i}`;
const worker = spawn('npx', ['tsx', workerScript], {
env: {
...process.env,
WORKER_ID: workerId,
POD_NAME: podName,
},
stdio: 'inherit',
});
workers.push(worker);
console.log(`[Pod] Started worker ${i}/${WORKERS_PER_POD}: ${workerId}`);
}
// Handle shutdown
const shutdown = () => {
console.log(`\n[Pod] Shutting down pod "${podName}"...`);
workers.forEach(w => w.kill('SIGTERM'));
setTimeout(() => process.exit(0), 2000);
};
process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);
// Keep the process alive
await new Promise(() => {});
}
main().catch(err => {
console.error('[Pod] Fatal error:', err);
process.exit(1);
});

View File

@@ -14,7 +14,7 @@ export type TaskRole =
| 'store_discovery' | 'store_discovery'
| 'entry_point_discovery' | 'entry_point_discovery'
| 'product_discovery' | 'product_discovery'
| 'product_resync' | 'product_refresh'
| 'analytics_refresh'; | 'analytics_refresh';
export type TaskStatus = export type TaskStatus =
@@ -29,6 +29,8 @@ export interface WorkerTask {
id: number; id: number;
role: TaskRole; role: TaskRole;
dispensary_id: number | null; dispensary_id: number | null;
dispensary_name?: string; // JOINed from dispensaries
dispensary_slug?: string; // JOINed from dispensaries
platform: string | null; platform: string | null;
status: TaskStatus; status: TaskStatus;
priority: number; priority: number;
@@ -128,9 +130,11 @@ class TaskService {
/** /**
* Claim a task atomically for a worker * Claim a task atomically for a worker
* Uses the SQL function for proper locking * If role is null, claims ANY available task (role-agnostic worker)
*/ */
async claimTask(role: TaskRole, workerId: string): Promise<WorkerTask | null> { async claimTask(role: TaskRole | null, workerId: string): Promise<WorkerTask | null> {
if (role) {
// Role-specific claiming - use the SQL function
const result = await pool.query( const result = await pool.query(
`SELECT * FROM claim_task($1, $2)`, `SELECT * FROM claim_task($1, $2)`,
[role, workerId] [role, workerId]
@@ -138,6 +142,33 @@ class TaskService {
return (result.rows[0] as WorkerTask) || null; return (result.rows[0] as WorkerTask) || null;
} }
// Role-agnostic claiming - claim ANY pending task
const result = await pool.query(`
UPDATE worker_tasks
SET
status = 'claimed',
worker_id = $1,
claimed_at = NOW()
WHERE id = (
SELECT id FROM worker_tasks
WHERE status = 'pending'
AND (scheduled_for IS NULL OR scheduled_for <= NOW())
-- Exclude stores that already have an active task
AND (dispensary_id IS NULL OR dispensary_id NOT IN (
SELECT dispensary_id FROM worker_tasks
WHERE status IN ('claimed', 'running')
AND dispensary_id IS NOT NULL
))
ORDER BY priority DESC, created_at ASC
LIMIT 1
FOR UPDATE SKIP LOCKED
)
RETURNING *
`, [workerId]);
return (result.rows[0] as WorkerTask) || null;
}
/** /**
* Mark a task as running (worker started processing) * Mark a task as running (worker started processing)
*/ */
@@ -206,27 +237,27 @@ class TaskService {
let paramIndex = 1; let paramIndex = 1;
if (filter.role) { if (filter.role) {
conditions.push(`role = $${paramIndex++}`); conditions.push(`t.role = $${paramIndex++}`);
params.push(filter.role); params.push(filter.role);
} }
if (filter.status) { if (filter.status) {
if (Array.isArray(filter.status)) { if (Array.isArray(filter.status)) {
conditions.push(`status = ANY($${paramIndex++})`); conditions.push(`t.status = ANY($${paramIndex++})`);
params.push(filter.status); params.push(filter.status);
} else { } else {
conditions.push(`status = $${paramIndex++}`); conditions.push(`t.status = $${paramIndex++}`);
params.push(filter.status); params.push(filter.status);
} }
} }
if (filter.dispensary_id) { if (filter.dispensary_id) {
conditions.push(`dispensary_id = $${paramIndex++}`); conditions.push(`t.dispensary_id = $${paramIndex++}`);
params.push(filter.dispensary_id); params.push(filter.dispensary_id);
} }
if (filter.worker_id) { if (filter.worker_id) {
conditions.push(`worker_id = $${paramIndex++}`); conditions.push(`t.worker_id = $${paramIndex++}`);
params.push(filter.worker_id); params.push(filter.worker_id);
} }
@@ -235,9 +266,14 @@ class TaskService {
const offset = filter.offset ?? 0; const offset = filter.offset ?? 0;
const result = await pool.query( const result = await pool.query(
`SELECT * FROM worker_tasks `SELECT
t.*,
d.name as dispensary_name,
d.slug as dispensary_slug
FROM worker_tasks t
LEFT JOIN dispensaries d ON d.id = t.dispensary_id
${whereClause} ${whereClause}
ORDER BY created_at DESC ORDER BY t.created_at DESC
LIMIT ${limit} OFFSET ${offset}`, LIMIT ${limit} OFFSET ${offset}`,
params params
); );

View File

@@ -1,26 +1,58 @@
/** /**
* Task Worker * Task Worker
* *
* A unified worker that processes tasks from the worker_tasks queue. * A unified worker that pulls tasks from the worker_tasks queue.
* Replaces the fragmented job systems (job_schedules, dispensary_crawl_jobs, etc.) * Workers register on startup, get a friendly name, and pull tasks.
*
* Architecture:
* - Tasks are generated on schedule (by scheduler or API)
* - Workers PULL tasks from the pool (not assigned to them)
* - Tasks are claimed in order of priority (DESC) then creation time (ASC)
* - Workers report heartbeats to worker_registry
* - Workers are ROLE-AGNOSTIC by default (can handle any task type)
*
* Stealth & Anti-Detection:
* PROXIES ARE REQUIRED - workers will fail to start if no proxies available.
*
* On startup, workers initialize the CrawlRotator which provides:
* - Proxy rotation: Loads proxies from `proxies` table, ALL requests use proxy
* - User-Agent rotation: Cycles through realistic browser fingerprints
* - Fingerprint rotation: Changes browser profile on blocks
* - Locale/timezone: Matches Accept-Language to target state
*
* The CrawlRotator is wired to the Dutchie client via setCrawlRotator().
* Task handlers call startSession() which picks a random fingerprint.
* On 403 errors, the client automatically:
* 1. Records failure on current proxy
* 2. Rotates to next proxy
* 3. Rotates fingerprint
* 4. Retries the request
* *
* Usage: * Usage:
* WORKER_ROLE=product_resync npx tsx src/tasks/task-worker.ts * npx tsx src/tasks/task-worker.ts # Role-agnostic (any task)
* WORKER_ROLE=product_refresh npx tsx src/tasks/task-worker.ts # Role-specific
* *
* Environment: * Environment:
* WORKER_ROLE - Which task role to process (required) * WORKER_ROLE - Which task role to process (optional, null = any task)
* WORKER_ID - Optional custom worker ID * WORKER_ID - Optional custom worker ID (auto-generated if not provided)
* POD_NAME - Kubernetes pod name (optional)
* POLL_INTERVAL_MS - How often to check for tasks (default: 5000) * POLL_INTERVAL_MS - How often to check for tasks (default: 5000)
* HEARTBEAT_INTERVAL_MS - How often to update heartbeat (default: 30000) * HEARTBEAT_INTERVAL_MS - How often to update heartbeat (default: 30000)
* API_BASE_URL - Backend API URL for registration (default: http://localhost:3010)
*/ */
import { Pool } from 'pg'; import { Pool } from 'pg';
import { v4 as uuidv4 } from 'uuid'; import { v4 as uuidv4 } from 'uuid';
import { taskService, TaskRole, WorkerTask } from './task-service'; import { taskService, TaskRole, WorkerTask } from './task-service';
import { getPool } from '../db/pool'; import { getPool } from '../db/pool';
import os from 'os';
// Stealth/rotation support
import { CrawlRotator } from '../services/crawl-rotator';
import { setCrawlRotator } from '../platforms/dutchie';
// Task handlers by role // Task handlers by role
import { handleProductResync } from './handlers/product-resync'; import { handleProductRefresh } from './handlers/product-refresh';
import { handleProductDiscovery } from './handlers/product-discovery'; import { handleProductDiscovery } from './handlers/product-discovery';
import { handleStoreDiscovery } from './handlers/store-discovery'; import { handleStoreDiscovery } from './handlers/store-discovery';
import { handleEntryPointDiscovery } from './handlers/entry-point-discovery'; import { handleEntryPointDiscovery } from './handlers/entry-point-discovery';
@@ -28,6 +60,7 @@ import { handleAnalyticsRefresh } from './handlers/analytics-refresh';
const POLL_INTERVAL_MS = parseInt(process.env.POLL_INTERVAL_MS || '5000'); const POLL_INTERVAL_MS = parseInt(process.env.POLL_INTERVAL_MS || '5000');
const HEARTBEAT_INTERVAL_MS = parseInt(process.env.HEARTBEAT_INTERVAL_MS || '30000'); const HEARTBEAT_INTERVAL_MS = parseInt(process.env.HEARTBEAT_INTERVAL_MS || '30000');
const API_BASE_URL = process.env.API_BASE_URL || 'http://localhost:3010';
export interface TaskContext { export interface TaskContext {
pool: Pool; pool: Pool;
@@ -48,7 +81,7 @@ export interface TaskResult {
type TaskHandler = (ctx: TaskContext) => Promise<TaskResult>; type TaskHandler = (ctx: TaskContext) => Promise<TaskResult>;
const TASK_HANDLERS: Record<TaskRole, TaskHandler> = { const TASK_HANDLERS: Record<TaskRole, TaskHandler> = {
product_resync: handleProductResync, product_refresh: handleProductRefresh,
product_discovery: handleProductDiscovery, product_discovery: handleProductDiscovery,
store_discovery: handleStoreDiscovery, store_discovery: handleStoreDiscovery,
entry_point_discovery: handleEntryPointDiscovery, entry_point_discovery: handleEntryPointDiscovery,
@@ -58,15 +91,160 @@ const TASK_HANDLERS: Record<TaskRole, TaskHandler> = {
export class TaskWorker { export class TaskWorker {
private pool: Pool; private pool: Pool;
private workerId: string; private workerId: string;
private role: TaskRole; private role: TaskRole | null; // null = role-agnostic (any task)
private friendlyName: string = '';
private isRunning: boolean = false; private isRunning: boolean = false;
private heartbeatInterval: NodeJS.Timeout | null = null; private heartbeatInterval: NodeJS.Timeout | null = null;
private registryHeartbeatInterval: NodeJS.Timeout | null = null;
private currentTask: WorkerTask | null = null; private currentTask: WorkerTask | null = null;
private crawlRotator: CrawlRotator;
constructor(role: TaskRole, workerId?: string) { constructor(role: TaskRole | null = null, workerId?: string) {
this.pool = getPool(); this.pool = getPool();
this.role = role; this.role = role;
this.workerId = workerId || `worker-${role}-${uuidv4().slice(0, 8)}`; this.workerId = workerId || `worker-${uuidv4().slice(0, 8)}`;
this.crawlRotator = new CrawlRotator(this.pool);
}
/**
* Initialize stealth systems (proxy rotation, fingerprints)
* Called once on worker startup before processing any tasks.
*
* IMPORTANT: Proxies are REQUIRED. Workers will fail to start if no proxies available.
*/
private async initializeStealth(): Promise<void> {
// Load proxies from database
await this.crawlRotator.initialize();
const stats = this.crawlRotator.proxy.getStats();
if (stats.activeProxies === 0) {
throw new Error('No active proxies available. Workers MUST use proxies for all requests. Add proxies to the database before starting workers.');
}
console.log(`[TaskWorker] Loaded ${stats.activeProxies} proxies (${stats.avgSuccessRate.toFixed(1)}% avg success rate)`);
// Wire rotator to Dutchie client - proxies will be used for ALL requests
setCrawlRotator(this.crawlRotator);
console.log(`[TaskWorker] Stealth initialized: ${this.crawlRotator.userAgent.getCount()} fingerprints, proxy REQUIRED for all requests`);
}
/**
* Register worker with the registry (get friendly name)
*/
private async register(): Promise<void> {
try {
const response = await fetch(`${API_BASE_URL}/api/worker-registry/register`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
role: this.role,
worker_id: this.workerId,
pod_name: process.env.POD_NAME || process.env.HOSTNAME,
hostname: os.hostname(),
metadata: {
pid: process.pid,
node_version: process.version,
started_at: new Date().toISOString()
}
})
});
const data = await response.json();
if (data.success) {
this.friendlyName = data.friendly_name;
console.log(`[TaskWorker] ${data.message}`);
} else {
console.warn(`[TaskWorker] Registration warning: ${data.error}`);
this.friendlyName = this.workerId.slice(0, 12);
}
} catch (error: any) {
// Registration is optional - worker can still function without it
console.warn(`[TaskWorker] Could not register with API (will continue): ${error.message}`);
this.friendlyName = this.workerId.slice(0, 12);
}
}
/**
* Deregister worker from the registry
*/
private async deregister(): Promise<void> {
try {
await fetch(`${API_BASE_URL}/api/worker-registry/deregister`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ worker_id: this.workerId })
});
console.log(`[TaskWorker] ${this.friendlyName} signed off`);
} catch {
// Ignore deregistration errors
}
}
/**
* Send heartbeat to registry with resource usage
*/
private async sendRegistryHeartbeat(): Promise<void> {
try {
const memUsage = process.memoryUsage();
const cpuUsage = process.cpuUsage();
await fetch(`${API_BASE_URL}/api/worker-registry/heartbeat`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
worker_id: this.workerId,
current_task_id: this.currentTask?.id || null,
status: this.currentTask ? 'active' : 'idle',
resources: {
memory_mb: Math.round(memUsage.heapUsed / 1024 / 1024),
memory_total_mb: Math.round(memUsage.heapTotal / 1024 / 1024),
memory_rss_mb: Math.round(memUsage.rss / 1024 / 1024),
cpu_user_ms: Math.round(cpuUsage.user / 1000),
cpu_system_ms: Math.round(cpuUsage.system / 1000),
}
})
});
} catch {
// Ignore heartbeat errors
}
}
/**
* Report task completion to registry
*/
private async reportTaskCompletion(success: boolean): Promise<void> {
try {
await fetch(`${API_BASE_URL}/api/worker-registry/task-completed`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
worker_id: this.workerId,
success
})
});
} catch {
// Ignore errors
}
}
/**
* Start registry heartbeat interval
*/
private startRegistryHeartbeat(): void {
this.registryHeartbeatInterval = setInterval(async () => {
await this.sendRegistryHeartbeat();
}, HEARTBEAT_INTERVAL_MS);
}
/**
* Stop registry heartbeat interval
*/
private stopRegistryHeartbeat(): void {
if (this.registryHeartbeatInterval) {
clearInterval(this.registryHeartbeatInterval);
this.registryHeartbeatInterval = null;
}
} }
/** /**
@@ -74,7 +252,18 @@ export class TaskWorker {
*/ */
async start(): Promise<void> { async start(): Promise<void> {
this.isRunning = true; this.isRunning = true;
console.log(`[TaskWorker] Starting worker ${this.workerId} for role: ${this.role}`);
// Initialize stealth systems (proxy rotation, fingerprints)
await this.initializeStealth();
// Register with the API to get a friendly name
await this.register();
// Start registry heartbeat
this.startRegistryHeartbeat();
const roleMsg = this.role ? `for role: ${this.role}` : '(role-agnostic - any task)';
console.log(`[TaskWorker] ${this.friendlyName} starting ${roleMsg}`);
while (this.isRunning) { while (this.isRunning) {
try { try {
@@ -91,10 +280,12 @@ export class TaskWorker {
/** /**
* Stop the worker * Stop the worker
*/ */
stop(): void { async stop(): Promise<void> {
this.isRunning = false; this.isRunning = false;
this.stopHeartbeat(); this.stopHeartbeat();
console.log(`[TaskWorker] Stopping worker ${this.workerId}...`); this.stopRegistryHeartbeat();
await this.deregister();
console.log(`[TaskWorker] ${this.friendlyName} stopped`);
} }
/** /**
@@ -142,7 +333,8 @@ export class TaskWorker {
if (result.success) { if (result.success) {
// Mark as completed // Mark as completed
await taskService.completeTask(task.id, result); await taskService.completeTask(task.id, result);
console.log(`[TaskWorker] Task ${task.id} completed successfully`); await this.reportTaskCompletion(true);
console.log(`[TaskWorker] ${this.friendlyName} completed task ${task.id}`);
// Chain next task if applicable // Chain next task if applicable
const chainedTask = await taskService.chainNextTask({ const chainedTask = await taskService.chainNextTask({
@@ -156,12 +348,14 @@ export class TaskWorker {
} else { } else {
// Mark as failed // Mark as failed
await taskService.failTask(task.id, result.error || 'Unknown error'); await taskService.failTask(task.id, result.error || 'Unknown error');
console.log(`[TaskWorker] Task ${task.id} failed: ${result.error}`); await this.reportTaskCompletion(false);
console.log(`[TaskWorker] ${this.friendlyName} failed task ${task.id}: ${result.error}`);
} }
} catch (error: any) { } catch (error: any) {
// Mark as failed // Mark as failed
await taskService.failTask(task.id, error.message); await taskService.failTask(task.id, error.message);
console.error(`[TaskWorker] Task ${task.id} threw error:`, error.message); await this.reportTaskCompletion(false);
console.error(`[TaskWorker] ${this.friendlyName} task ${task.id} error:`, error.message);
} finally { } finally {
this.stopHeartbeat(); this.stopHeartbeat();
this.currentTask = null; this.currentTask = null;
@@ -201,7 +395,7 @@ export class TaskWorker {
/** /**
* Get worker info * Get worker info
*/ */
getInfo(): { workerId: string; role: TaskRole; isRunning: boolean; currentTaskId: number | null } { getInfo(): { workerId: string; role: TaskRole | null; isRunning: boolean; currentTaskId: number | null } {
return { return {
workerId: this.workerId, workerId: this.workerId,
role: this.role, role: this.role,
@@ -216,30 +410,27 @@ export class TaskWorker {
// ============================================================ // ============================================================
async function main(): Promise<void> { async function main(): Promise<void> {
const role = process.env.WORKER_ROLE as TaskRole; const role = process.env.WORKER_ROLE as TaskRole | undefined;
if (!role) {
console.error('Error: WORKER_ROLE environment variable is required');
console.error('Valid roles: store_discovery, entry_point_discovery, product_discovery, product_resync, analytics_refresh');
process.exit(1);
}
const validRoles: TaskRole[] = [ const validRoles: TaskRole[] = [
'store_discovery', 'store_discovery',
'entry_point_discovery', 'entry_point_discovery',
'product_discovery', 'product_discovery',
'product_resync', 'product_refresh',
'analytics_refresh', 'analytics_refresh',
]; ];
if (!validRoles.includes(role)) { // If role specified, validate it
if (role && !validRoles.includes(role)) {
console.error(`Error: Invalid WORKER_ROLE: ${role}`); console.error(`Error: Invalid WORKER_ROLE: ${role}`);
console.error(`Valid roles: ${validRoles.join(', ')}`); console.error(`Valid roles: ${validRoles.join(', ')}`);
console.error('Or omit WORKER_ROLE for role-agnostic worker (any task)');
process.exit(1); process.exit(1);
} }
const workerId = process.env.WORKER_ID; const workerId = process.env.WORKER_ID;
const worker = new TaskWorker(role, workerId); // Pass null for role-agnostic, or the specific role
const worker = new TaskWorker(role || null, workerId);
// Handle graceful shutdown // Handle graceful shutdown
process.on('SIGTERM', () => { process.on('SIGTERM', () => {

View File

@@ -113,7 +113,7 @@ class ApiClient {
}); });
} }
async getDispensaries(params?: { limit?: number; offset?: number; search?: string; city?: string; state?: string; crawl_enabled?: string }) { async getDispensaries(params?: { limit?: number; offset?: number; search?: string; city?: string; state?: string; crawl_enabled?: string; status?: string }) {
const searchParams = new URLSearchParams(); const searchParams = new URLSearchParams();
if (params?.limit) searchParams.append('limit', params.limit.toString()); if (params?.limit) searchParams.append('limit', params.limit.toString());
if (params?.offset) searchParams.append('offset', params.offset.toString()); if (params?.offset) searchParams.append('offset', params.offset.toString());
@@ -121,10 +121,15 @@ class ApiClient {
if (params?.city) searchParams.append('city', params.city); if (params?.city) searchParams.append('city', params.city);
if (params?.state) searchParams.append('state', params.state); if (params?.state) searchParams.append('state', params.state);
if (params?.crawl_enabled) searchParams.append('crawl_enabled', params.crawl_enabled); if (params?.crawl_enabled) searchParams.append('crawl_enabled', params.crawl_enabled);
if (params?.status) searchParams.append('status', params.status);
const queryString = searchParams.toString() ? `?${searchParams.toString()}` : ''; const queryString = searchParams.toString() ? `?${searchParams.toString()}` : '';
return this.request<{ dispensaries: any[]; total: number; limit: number; offset: number; hasMore: boolean }>(`/api/dispensaries${queryString}`); return this.request<{ dispensaries: any[]; total: number; limit: number; offset: number; hasMore: boolean }>(`/api/dispensaries${queryString}`);
} }
async getDroppedStores() {
return this.request<{ dropped_count: number; dropped_stores: any[] }>('/api/dispensaries/stats/dropped');
}
async getDispensary(slug: string) { async getDispensary(slug: string) {
return this.request<any>(`/api/dispensaries/${slug}`); return this.request<any>(`/api/dispensaries/${slug}`);
} }

View File

@@ -46,12 +46,33 @@ export function Dashboard() {
const [pendingChangesCount, setPendingChangesCount] = useState(0); const [pendingChangesCount, setPendingChangesCount] = useState(0);
const [showNotification, setShowNotification] = useState(false); const [showNotification, setShowNotification] = useState(false);
const [taskCounts, setTaskCounts] = useState<Record<string, number> | null>(null); const [taskCounts, setTaskCounts] = useState<Record<string, number> | null>(null);
const [droppedStoresCount, setDroppedStoresCount] = useState(0);
const [showDroppedAlert, setShowDroppedAlert] = useState(false);
useEffect(() => { useEffect(() => {
loadData(); loadData();
checkNotificationStatus(); checkNotificationStatus();
checkDroppedStores();
}, []); }, []);
const checkDroppedStores = async () => {
try {
const data = await api.getDroppedStores();
setDroppedStoresCount(data.dropped_count);
// Check if notification was dismissed for this count
const dismissedCount = localStorage.getItem('dismissedDroppedStoresCount');
const isDismissed = dismissedCount && parseInt(dismissedCount) >= data.dropped_count;
setShowDroppedAlert(data.dropped_count > 0 && !isDismissed);
} catch (error) {
console.error('Failed to check dropped stores:', error);
}
};
const handleDismissDroppedAlert = () => {
localStorage.setItem('dismissedDroppedStoresCount', droppedStoresCount.toString());
setShowDroppedAlert(false);
};
const checkNotificationStatus = async () => { const checkNotificationStatus = async () => {
try { try {
// Fetch real pending changes count from API // Fetch real pending changes count from API
@@ -214,6 +235,40 @@ export function Dashboard() {
</div> </div>
)} )}
{/* Dropped Stores Alert */}
{showDroppedAlert && (
<div className="mb-6 bg-red-50 border-l-4 border-red-500 rounded-lg p-3 sm:p-4">
<div className="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-3 sm:gap-4">
<div className="flex items-start sm:items-center gap-3 flex-1">
<Store className="w-5 h-5 text-red-600 flex-shrink-0 mt-0.5 sm:mt-0" />
<div className="flex-1">
<h3 className="text-sm font-semibold text-red-900">
{droppedStoresCount} dropped store{droppedStoresCount !== 1 ? 's' : ''} need{droppedStoresCount === 1 ? 's' : ''} review
</h3>
<p className="text-xs sm:text-sm text-red-700 mt-0.5">
These stores were not found in the latest Dutchie discovery and may have stopped using the platform
</p>
</div>
</div>
<div className="flex items-center gap-2 pl-8 sm:pl-0">
<button
onClick={() => navigate('/dispensaries?status=dropped')}
className="btn btn-sm bg-red-600 hover:bg-red-700 text-white border-none"
>
Review
</button>
<button
onClick={handleDismissDroppedAlert}
className="btn btn-sm btn-ghost text-red-900 hover:bg-red-100"
aria-label="Dismiss notification"
>
<X className="w-4 h-4" />
</button>
</div>
</div>
</div>
)}
<div className="space-y-8"> <div className="space-y-8">
{/* Header */} {/* Header */}
<div className="flex flex-col sm:flex-row sm:justify-between sm:items-center gap-4"> <div className="flex flex-col sm:flex-row sm:justify-between sm:items-center gap-4">

View File

@@ -13,6 +13,7 @@ export function Dispensaries() {
const [searchTerm, setSearchTerm] = useState(''); const [searchTerm, setSearchTerm] = useState('');
const [debouncedSearch, setDebouncedSearch] = useState(''); const [debouncedSearch, setDebouncedSearch] = useState('');
const [filterState, setFilterState] = useState(''); const [filterState, setFilterState] = useState('');
const [filterStatus, setFilterStatus] = useState('');
const [editingDispensary, setEditingDispensary] = useState<any | null>(null); const [editingDispensary, setEditingDispensary] = useState<any | null>(null);
const [editForm, setEditForm] = useState<any>({}); const [editForm, setEditForm] = useState<any>({});
const [total, setTotal] = useState(0); const [total, setTotal] = useState(0);
@@ -51,6 +52,7 @@ export function Dispensaries() {
offset, offset,
search: debouncedSearch || undefined, search: debouncedSearch || undefined,
state: filterState || undefined, state: filterState || undefined,
status: filterStatus || undefined,
crawl_enabled: 'all' crawl_enabled: 'all'
}); });
setDispensaries(data.dispensaries); setDispensaries(data.dispensaries);
@@ -61,7 +63,7 @@ export function Dispensaries() {
} finally { } finally {
setLoading(false); setLoading(false);
} }
}, [offset, debouncedSearch, filterState]); }, [offset, debouncedSearch, filterState, filterStatus]);
useEffect(() => { useEffect(() => {
loadDispensaries(); loadDispensaries();
@@ -110,6 +112,11 @@ export function Dispensaries() {
setOffset(0); // Reset to first page setOffset(0); // Reset to first page
}; };
const handleStatusFilter = (status: string) => {
setFilterStatus(status);
setOffset(0); // Reset to first page
};
return ( return (
<Layout> <Layout>
<div className="space-y-6"> <div className="space-y-6">
@@ -123,7 +130,7 @@ export function Dispensaries() {
{/* Filters */} {/* Filters */}
<div className="bg-white rounded-lg border border-gray-200 p-4"> <div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="grid grid-cols-2 gap-4"> <div className="grid grid-cols-3 gap-4">
<div> <div>
<label className="block text-sm font-medium text-gray-700 mb-2"> <label className="block text-sm font-medium text-gray-700 mb-2">
Search Search
@@ -154,6 +161,23 @@ export function Dispensaries() {
))} ))}
</select> </select>
</div> </div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">
Filter by Status
</label>
<select
value={filterStatus}
onChange={(e) => handleStatusFilter(e.target.value)}
className={`w-full px-3 py-2 border rounded-lg focus:ring-2 focus:ring-blue-500 focus:border-blue-500 ${
filterStatus === 'dropped' ? 'border-red-300 bg-red-50' : 'border-gray-300'
}`}
>
<option value="">All Statuses</option>
<option value="open">Open</option>
<option value="dropped">Dropped (Needs Review)</option>
<option value="closed">Closed</option>
</select>
</div>
</div> </div>
</div> </div>

File diff suppressed because it is too large Load Diff

View File

@@ -69,7 +69,7 @@ const ROLES = [
'store_discovery', 'store_discovery',
'entry_point_discovery', 'entry_point_discovery',
'product_discovery', 'product_discovery',
'product_resync', 'product_refresh',
'analytics_refresh', 'analytics_refresh',
]; ];

File diff suppressed because it is too large Load Diff

353
docs/CRAWL_SYSTEM_V2.md Normal file
View File

@@ -0,0 +1,353 @@
# CannaiQ Crawl System V2
## Overview
The CannaiQ Crawl System is a GraphQL-based data pipeline that discovers and monitors cannabis dispensaries using the Dutchie platform. It operates in two phases:
1. **Phase 1: Store Discovery** - Weekly discovery of Dutchie-powered dispensaries
2. **Phase 2: Product Crawling** - Regular product/price/stock updates (documented separately)
---
## Phase 1: Store Discovery
### Purpose
Automatically discover and maintain a database of dispensaries that use Dutchie menus across all US states.
### Schedule
- **Frequency**: Weekly (typically Sunday night)
- **Duration**: ~2-4 hours for full US coverage
### Flow Diagram
```
┌─────────────────────────────────────────────────────────────────────┐
│ PHASE 1: STORE DISCOVERY │
└─────────────────────────────────────────────────────────────────────┘
1. IDENTITY SETUP
┌──────────────────┐
│ getRandomProxy() │ ──► Random IP from proxy pool
└──────────────────┘
┌──────────────────┐
│ startSession() │ ──► Random UA + fingerprint + locale matching proxy location
└──────────────────┘
2. CITY DISCOVERY (per state)
┌──────────────────────────────┐
│ GraphQL: getAllCitiesByState │ ──► Returns cities with active dispensaries
└──────────────────────────────┘
┌──────────────────────────────┐
│ Upsert dutchie_discovery_ │
│ cities table │
└──────────────────────────────┘
3. STORE DISCOVERY (per city)
┌───────────────────────────────┐
│ GraphQL: ConsumerDispensaries │ ──► Returns store data for city
└───────────────────────────────┘
┌───────────────────────────────┐
│ Upsert dutchie_discovery_ │
│ locations table │
└───────────────────────────────┘
4. VALIDATION & PROMOTION
┌──────────────────────────┐
│ validateForPromotion() │ ──► Check required fields
└──────────────────────────┘
┌──────────────────────────┐
│ promoteLocation() │ ──► Upsert to dispensaries table
└──────────────────────────┘
┌──────────────────────────┐
│ ensureCrawlerProfile() │ ──► Create profile with status='sandbox'
└──────────────────────────┘
5. DROPPED STORE DETECTION
┌──────────────────────────┐
│ detectDroppedStores() │ ──► Find stores missing from discovery
└──────────────────────────┘
┌──────────────────────────┐
│ Mark status='dropped' │ ──► Dashboard alert for review
└──────────────────────────┘
```
---
## Key Files
| File | Purpose |
|------|---------|
| `backend/src/platforms/dutchie/client.ts` | HTTP client with proxy/fingerprint rotation |
| `backend/src/discovery/discovery-crawler.ts` | Main discovery orchestrator |
| `backend/src/discovery/location-discovery.ts` | City/store GraphQL fetching |
| `backend/src/discovery/promotion.ts` | Validation and promotion logic |
| `backend/src/scripts/run-discovery.ts` | CLI entry point |
---
## Identity Masking
Before any GraphQL queries, the system establishes a masked identity:
### 1. Proxy Selection
```typescript
// backend/src/platforms/dutchie/client.ts
// Get random proxy from active pool (NOT state-specific)
const proxy = await getRandomProxy();
setProxy(proxy.url);
```
The proxy is selected randomly from the active proxy pool. It is NOT geo-targeted to the state being crawled.
### 2. Fingerprint + Locale Harmonization
```typescript
// backend/src/platforms/dutchie/client.ts
function startSession(stateCode: string, timezone: string) {
// 1. Random browser fingerprint (Chrome/Firefox/Safari/Edge variants)
const fingerprint = getRandomFingerprint();
// 2. Match Accept-Language to proxy's timezone/location
const locale = getLocaleForTimezone(timezone);
// 3. Set headers for this session
currentSession = {
userAgent: fingerprint.ua,
acceptLanguage: locale,
secChUa: fingerprint.secChUa,
// ... other fingerprint headers
};
}
```
### Fingerprint Pool
6 browser fingerprints rotate on each session and on 403 errors:
| Browser | Version | Platform |
|---------|---------|----------|
| Chrome | 120 | Windows |
| Chrome | 120 | macOS |
| Firefox | 121 | Windows |
| Firefox | 121 | macOS |
| Safari | 17.2 | macOS |
| Edge | 120 | Windows |
### Timezone → Locale Mapping
```typescript
const TIMEZONE_TO_LOCALE: Record<string, string> = {
'America/New_York': 'en-US,en;q=0.9',
'America/Chicago': 'en-US,en;q=0.9',
'America/Denver': 'en-US,en;q=0.9',
'America/Los_Angeles': 'en-US,en;q=0.9',
'America/Phoenix': 'en-US,en;q=0.9',
// ...
};
```
---
## GraphQL Queries
### 1. getAllCitiesByState
Fetches cities with active dispensaries for a state.
```typescript
// backend/src/discovery/location-discovery.ts
const response = await executeGraphQL({
operationName: 'getAllCitiesByState',
variables: {
state: 'AZ',
countryCode: 'US'
}
});
// Returns: { cities: [{ name: 'Phoenix', slug: 'phoenix' }, ...] }
```
**Hash**: `ae547a0466ace5a48f91e55bf6699eacd87e3a42841560f0c0eabed5a0a920e6`
### 2. ConsumerDispensaries
Fetches store data for a city/state.
```typescript
// backend/src/discovery/location-discovery.ts
const response = await executeGraphQL({
operationName: 'ConsumerDispensaries',
variables: {
dispensaryFilter: {
city: 'Phoenix',
state: 'AZ',
activeOnly: true
}
}
});
// Returns: [{ id, name, address, coords, menuUrl, ... }, ...]
```
**Hash**: `0a5bfa6ca1d64ae47bcccb7c8077c87147cbc4e6982c17ceec97a2a4948b311b`
---
## Database Tables
### Discovery Tables (Staging)
| Table | Purpose |
|-------|---------|
| `dutchie_discovery_cities` | Cities known to have dispensaries |
| `dutchie_discovery_locations` | Raw discovered store data |
### Canonical Tables
| Table | Purpose |
|-------|---------|
| `dispensaries` | Promoted stores ready for crawling |
| `dispensary_crawler_profiles` | Crawler configuration per store |
| `dutchie_promotion_log` | Audit trail for all discovery actions |
---
## Validation Rules
A discovery location must have these fields to be promoted:
| Field | Requirement |
|-------|-------------|
| `platform_location_id` | MongoDB ObjectId (24 hex chars) |
| `name` | Non-empty string |
| `city` | Non-empty string |
| `state_code` | Non-empty string |
| `platform_menu_url` | Valid URL |
Invalid records are marked `status='rejected'` with errors logged.
---
## Dropped Store Detection
After discovery, the system identifies stores that may have left the Dutchie platform:
### Detection Criteria
A store is marked as "dropped" if:
1. It has a `platform_dispensary_id` (was previously verified)
2. It's currently `status='open'` and `crawl_enabled=true`
3. It was NOT seen in the latest discovery (not in `dutchie_discovery_locations` with `last_seen_at` in last 24 hours)
### Implementation
```typescript
// backend/src/discovery/discovery-crawler.ts
export async function detectDroppedStores(pool: Pool, stateCode?: string) {
// 1. Find dispensaries not in recent discovery
// 2. Mark status='dropped'
// 3. Log to dutchie_promotion_log
// 4. Return list for dashboard alert
}
```
### Admin UI
- **Dashboard**: Red alert banner when dropped stores exist
- **Dispensaries page**: Filter by `status=dropped` to review
---
## CLI Usage
```bash
# Discover all stores in a state
npx tsx src/scripts/run-discovery.ts discover:state AZ
# Discover all US states
npx tsx src/scripts/run-discovery.ts discover:all
# Dry run (no DB writes)
npx tsx src/scripts/run-discovery.ts discover:state CA --dry-run
# Check stats
npx tsx src/scripts/run-discovery.ts stats
```
---
## Rate Limiting
- **2 seconds** between city requests
- **Exponential backoff** on 429/403 responses
- **Fingerprint rotation** on 403 errors
---
## Error Handling
| Error | Action |
|-------|--------|
| 403 Forbidden | Rotate fingerprint, retry |
| 429 Rate Limited | Wait 30s, retry |
| Network timeout | Retry up to 3 times |
| GraphQL error | Log and continue to next city |
---
## Monitoring
### Logs
Discovery progress is logged to stdout:
```
[Discovery] Starting discovery for state: AZ
[Discovery] Step 1: Initializing proxy...
[Discovery] Step 2: Fetching cities...
[Discovery] Found 45 cities for AZ
[Discovery] Step 3: Discovering locations...
[Discovery] City 1/45: Phoenix - found 28 stores
...
[Discovery] Step 4: Auto-promoting discovered locations...
[Discovery] Created: 5 new dispensaries
[Discovery] Updated: 40 existing dispensaries
[Discovery] Step 5: Detecting dropped stores...
[Discovery] Found 2 dropped stores
```
### Audit Log
All actions logged to `dutchie_promotion_log`:
| Action | Description |
|--------|-------------|
| `promoted_create` | New dispensary created |
| `promoted_update` | Existing dispensary updated |
| `rejected` | Validation failed |
| `dropped` | Store not found in discovery |
---
## Next: Phase 2
See `docs/PRODUCT_CRAWL_V2.md` for the product crawling phase (coming next).

408
docs/WORKER_SYSTEM.md Normal file
View File

@@ -0,0 +1,408 @@
# CannaiQ Worker System
## Overview
The Worker System is a role-based task queue that processes background jobs. All tasks go into a single pool, and workers claim tasks based on their assigned role.
---
## Design Pattern: Single Pool, Role-Based Claiming
```
┌─────────────────────────────────────────┐
│ TASK POOL (worker_tasks) │
│ │
│ ┌─────────────────────────────────┐ │
│ │ role=store_discovery pending │ │
│ │ role=product_resync pending │ │
│ │ role=product_resync pending │ │
│ │ role=product_resync pending │ │
│ │ role=analytics_refresh pending │ │
│ │ role=entry_point_disc pending │ │
│ └─────────────────────────────────┘ │
└─────────────────────────────────────────┘
┌────────────────────────────┼────────────────────────────┐
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ WORKER │ │ WORKER │ │ WORKER │
│ role=product_ │ │ role=product_ │ │ role=store_ │
│ resync │ │ resync │ │ discovery │
│ │ │ │ │ │
│ Claims ONLY │ │ Claims ONLY │ │ Claims ONLY │
│ product_resync │ │ product_resync │ │ store_discovery │
│ tasks │ │ tasks │ │ tasks │
└──────────────────┘ └──────────────────┘ └──────────────────┘
```
**Key Points:**
- All tasks go into ONE table (`worker_tasks`)
- Each worker is assigned ONE role at startup
- Workers only claim tasks matching their role
- Multiple workers can share the same role (horizontal scaling)
---
## Worker Roles
| Role | Purpose | Per-Store? | Schedule |
|------|---------|------------|----------|
| `store_discovery` | Find new dispensaries via GraphQL | No | Weekly |
| `entry_point_discovery` | Resolve platform IDs from menu URLs | Yes | On-demand |
| `product_discovery` | Initial product fetch for new stores | Yes | On-demand |
| `product_resync` | Regular price/stock updates | Yes | Every 4 hours |
| `analytics_refresh` | Refresh materialized views | No | Daily |
---
## Task Lifecycle
```
pending → claimed → running → completed
failed
(retry if < max_retries)
```
| Status | Meaning |
|--------|---------|
| `pending` | Waiting to be claimed |
| `claimed` | Worker has claimed, not yet started |
| `running` | Worker is actively processing |
| `completed` | Successfully finished |
| `failed` | Error occurred |
| `stale` | Worker died (heartbeat timeout) |
---
## Task Chaining
Tasks automatically create follow-up tasks:
```
store_discovery (finds new stores)
├─ Returns newStoreIds[] in result
entry_point_discovery (for each new store)
├─ Resolves platform_dispensary_id
product_discovery (initial crawl)
(store enters regular schedule)
product_resync (every 4 hours)
```
---
## How Claiming Works
### 1. Worker starts with a role
```bash
WORKER_ROLE=product_resync npx tsx src/tasks/task-worker.ts
```
### 2. Worker loop polls for tasks
```typescript
// Simplified worker loop
while (running) {
const task = await claimTask(this.role, this.workerId);
if (!task) {
await sleep(5000); // No tasks, wait 5 seconds
continue;
}
await processTask(task);
}
```
### 3. SQL function claims atomically
```sql
-- claim_task(role, worker_id)
UPDATE worker_tasks
SET status = 'claimed', worker_id = $2, claimed_at = NOW()
WHERE id = (
SELECT id FROM worker_tasks
WHERE role = $1 -- Filter by worker's role
AND status = 'pending'
AND (scheduled_for IS NULL OR scheduled_for <= NOW())
AND dispensary_id NOT IN ( -- Per-store locking
SELECT dispensary_id FROM worker_tasks
WHERE status IN ('claimed', 'running')
)
ORDER BY priority DESC, created_at ASC -- Priority ordering
LIMIT 1
FOR UPDATE SKIP LOCKED -- Atomic, no race conditions
)
RETURNING *;
```
**Key Features:**
- `FOR UPDATE SKIP LOCKED` - Prevents race conditions between workers
- Role filtering - Worker only sees tasks for its role
- Per-store locking - Only one active task per dispensary
- Priority ordering - Higher priority tasks first
- Scheduled tasks - Respects `scheduled_for` timestamp
---
## Heartbeat & Stale Recovery
Workers send heartbeats every 30 seconds while processing:
```typescript
// During task processing
setInterval(() => {
await pool.query(
'UPDATE worker_tasks SET last_heartbeat_at = NOW() WHERE id = $1',
[taskId]
);
}, 30000);
```
If a worker dies, its tasks are recovered:
```sql
-- recover_stale_tasks(threshold_minutes)
UPDATE worker_tasks
SET status = 'pending', worker_id = NULL, retry_count = retry_count + 1
WHERE status IN ('claimed', 'running')
AND last_heartbeat_at < NOW() - INTERVAL '10 minutes'
AND retry_count < max_retries;
```
---
## Scheduling
### Daily Resync Generation
```sql
SELECT generate_resync_tasks(6, CURRENT_DATE); -- 6 batches = every 4 hours
```
Creates staggered tasks:
| Batch | Time | Stores |
|-------|------|--------|
| 1 | 00:00 | 1-50 |
| 2 | 04:00 | 51-100 |
| 3 | 08:00 | 101-150 |
| 4 | 12:00 | 151-200 |
| 5 | 16:00 | 201-250 |
| 6 | 20:00 | 251-300 |
---
## Files
### Core
| File | Purpose |
|------|---------|
| `src/tasks/task-service.ts` | Task CRUD, claiming, capacity metrics |
| `src/tasks/task-worker.ts` | Worker loop, heartbeat, handler dispatch |
| `src/routes/tasks.ts` | REST API endpoints |
| `migrations/074_worker_task_queue.sql` | Database schema + SQL functions |
### Handlers
| File | Role |
|------|------|
| `src/tasks/handlers/store-discovery.ts` | `store_discovery` |
| `src/tasks/handlers/entry-point-discovery.ts` | `entry_point_discovery` |
| `src/tasks/handlers/product-discovery.ts` | `product_discovery` |
| `src/tasks/handlers/product-resync.ts` | `product_resync` |
| `src/tasks/handlers/analytics-refresh.ts` | `analytics_refresh` |
---
## Running Workers
### Local Development
```bash
# Start a single worker
WORKER_ROLE=product_resync npx tsx src/tasks/task-worker.ts
# Start multiple workers (different terminals)
WORKER_ROLE=product_resync WORKER_ID=resync-1 npx tsx src/tasks/task-worker.ts
WORKER_ROLE=product_resync WORKER_ID=resync-2 npx tsx src/tasks/task-worker.ts
WORKER_ROLE=store_discovery npx tsx src/tasks/task-worker.ts
```
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `WORKER_ROLE` | (required) | Which task role to process |
| `WORKER_ID` | auto-generated | Custom worker identifier |
| `POLL_INTERVAL_MS` | 5000 | How often to check for tasks |
| `HEARTBEAT_INTERVAL_MS` | 30000 | How often to update heartbeat |
### Kubernetes
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: task-worker-resync
spec:
replicas: 5 # Scale horizontally
template:
spec:
containers:
- name: worker
image: code.cannabrands.app/creationshop/dispensary-scraper:latest
command: ["npx", "tsx", "src/tasks/task-worker.ts"]
env:
- name: WORKER_ROLE
value: "product_resync"
```
---
## API Endpoints
### Task Management
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/tasks` | List tasks (with filters) |
| POST | `/api/tasks` | Create a task |
| GET | `/api/tasks/:id` | Get task by ID |
| GET | `/api/tasks/counts` | Counts by status |
| GET | `/api/tasks/capacity` | Capacity metrics |
| POST | `/api/tasks/recover-stale` | Recover dead worker tasks |
### Task Generation
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/tasks/generate/resync` | Generate daily resync batch |
| POST | `/api/tasks/generate/discovery` | Create store discovery task |
---
## Capacity Planning
The `v_worker_capacity` view provides metrics:
```sql
SELECT * FROM v_worker_capacity;
```
| Metric | Description |
|--------|-------------|
| `pending_tasks` | Tasks waiting |
| `ready_tasks` | Tasks ready now (scheduled_for passed) |
| `running_tasks` | Tasks being processed |
| `active_workers` | Workers with recent heartbeat |
| `tasks_per_worker_hour` | Throughput estimate |
| `estimated_hours_to_drain` | Time to clear queue |
### Scaling API
```bash
GET /api/tasks/capacity/product_resync
```
```json
{
"pending_tasks": 500,
"active_workers": 3,
"workers_needed": {
"for_1_hour": 10,
"for_4_hours": 3,
"for_8_hours": 2
}
}
```
---
## Database Schema
### worker_tasks
```sql
CREATE TABLE worker_tasks (
id SERIAL PRIMARY KEY,
-- Task identification
role VARCHAR(50) NOT NULL,
dispensary_id INTEGER REFERENCES dispensaries(id),
platform VARCHAR(20),
-- State
status VARCHAR(20) DEFAULT 'pending',
priority INTEGER DEFAULT 0,
scheduled_for TIMESTAMPTZ,
-- Ownership
worker_id VARCHAR(100),
claimed_at TIMESTAMPTZ,
started_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ,
last_heartbeat_at TIMESTAMPTZ,
-- Results
result JSONB,
error_message TEXT,
retry_count INTEGER DEFAULT 0,
max_retries INTEGER DEFAULT 3,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
```
### Key Indexes
```sql
-- Fast claiming by role
CREATE INDEX idx_worker_tasks_pending
ON worker_tasks(role, priority DESC, created_at ASC)
WHERE status = 'pending';
-- Prevent duplicate active tasks per store
CREATE UNIQUE INDEX idx_worker_tasks_unique_active_store
ON worker_tasks(dispensary_id)
WHERE status IN ('claimed', 'running') AND dispensary_id IS NOT NULL;
```
---
## Monitoring
### Logs
```
[TaskWorker] Starting worker worker-product_resync-a1b2c3d4 for role: product_resync
[TaskWorker] Claimed task 123 (product_resync) for dispensary 456
[TaskWorker] Task 123 completed successfully
```
### Health Check
```sql
-- Active workers
SELECT worker_id, role, COUNT(*), MAX(last_heartbeat_at)
FROM worker_tasks
WHERE last_heartbeat_at > NOW() - INTERVAL '5 minutes'
GROUP BY worker_id, role;
-- Task counts by role/status
SELECT role, status, COUNT(*)
FROM worker_tasks
GROUP BY role, status;
```

View File

@@ -33,8 +33,8 @@ or overwrites of existing data.
| Table | Purpose | Key Columns | | Table | Purpose | Key Columns |
|-------|---------|-------------| |-------|---------|-------------|
| `dispensaries` | Store locations | id, name, slug, city, state, platform_dispensary_id | | `dispensaries` | Store locations | id, name, slug, city, state, platform_dispensary_id |
| `dutchie_products` | Canonical products | id, dispensary_id, external_product_id, name, brand_name, stock_status | | `store_products` | Canonical products | id, dispensary_id, external_product_id, name, brand_name, stock_status |
| `dutchie_product_snapshots` | Historical snapshots | dutchie_product_id, crawled_at, rec_min_price_cents | | `store_product_snapshots` | Historical snapshots | store_product_id, crawled_at, rec_min_price_cents |
| `brands` (view: v_brands) | Derived from products | brand_name, brand_id, product_count | | `brands` (view: v_brands) | Derived from products | brand_name, brand_id, product_count |
| `categories` (view: v_categories) | Derived from products | type, subcategory, product_count | | `categories` (view: v_categories) | Derived from products | type, subcategory, product_count |
@@ -147,12 +147,10 @@ CREATE TABLE IF NOT EXISTS products_from_legacy (
--- ---
### 3. Dutchie Products ### 3. Products (Legacy dutchie_products)
**Source:** `dutchie_legacy.dutchie_products` **Source:** `dutchie_legacy.dutchie_products`
**Target:** `cannaiq.dutchie_products` **Target:** `cannaiq.store_products`
These tables have nearly identical schemas. The mapping is direct:
| Legacy Column | Canonical Column | Notes | | Legacy Column | Canonical Column | Notes |
|---------------|------------------|-------| |---------------|------------------|-------|
@@ -180,15 +178,15 @@ ON CONFLICT (dispensary_id, external_product_id) DO NOTHING
--- ---
### 4. Dutchie Product Snapshots ### 4. Product Snapshots (Legacy dutchie_product_snapshots)
**Source:** `dutchie_legacy.dutchie_product_snapshots` **Source:** `dutchie_legacy.dutchie_product_snapshots`
**Target:** `cannaiq.dutchie_product_snapshots` **Target:** `cannaiq.store_product_snapshots`
| Legacy Column | Canonical Column | Notes | | Legacy Column | Canonical Column | Notes |
|---------------|------------------|-------| |---------------|------------------|-------|
| id | - | Generate new | | id | - | Generate new |
| dutchie_product_id | dutchie_product_id | Map via product lookup | | dutchie_product_id | store_product_id | Map via product lookup |
| dispensary_id | dispensary_id | Map via dispensary lookup | | dispensary_id | dispensary_id | Map via dispensary lookup |
| crawled_at | crawled_at | Direct | | crawled_at | crawled_at | Direct |
| rec_min_price_cents | rec_min_price_cents | Direct | | rec_min_price_cents | rec_min_price_cents | Direct |
@@ -201,7 +199,7 @@ ON CONFLICT (dispensary_id, external_product_id) DO NOTHING
```sql ```sql
-- No unique constraint on snapshots - all are historical records -- No unique constraint on snapshots - all are historical records
-- Just INSERT, no conflict handling needed -- Just INSERT, no conflict handling needed
INSERT INTO dutchie_product_snapshots (...) VALUES (...) INSERT INTO store_product_snapshots (...) VALUES (...)
``` ```
--- ---

View File

@@ -288,3 +288,89 @@ export async function getStates() {
return []; return [];
} }
} }
// ============================================================
// PRODUCTS
// ============================================================
/**
* Fetch products for a specific dispensary
* @param {number} dispensaryId - Dispensary ID
* @param {Object} params - Query parameters
* @returns {Promise<{products: Array, pagination: Object}>}
*/
export async function getDispensaryProducts(dispensaryId, params = {}) {
const queryParams = new URLSearchParams();
if (params.category) queryParams.append('category', params.category);
if (params.brand) queryParams.append('brand', params.brand);
if (params.search) queryParams.append('search', params.search);
if (params.inStockOnly) queryParams.append('in_stock_only', 'true');
if (params.limit) queryParams.append('limit', params.limit);
if (params.offset) queryParams.append('offset', params.offset);
const queryString = queryParams.toString();
const endpoint = `/api/v1/dispensaries/${dispensaryId}/products${queryString ? `?${queryString}` : ''}`;
return apiRequest(endpoint);
}
/**
* Get categories available at a dispensary
* @param {number} dispensaryId - Dispensary ID
* @returns {Promise<Array>}
*/
export async function getDispensaryCategories(dispensaryId) {
return apiRequest(`/api/v1/dispensaries/${dispensaryId}/categories`);
}
/**
* Get brands available at a dispensary
* @param {number} dispensaryId - Dispensary ID
* @returns {Promise<Array>}
*/
export async function getDispensaryBrands(dispensaryId) {
return apiRequest(`/api/v1/dispensaries/${dispensaryId}/brands`);
}
/**
* Map API product to UI format
* @param {Object} apiProduct - Product from API
* @returns {Object} - Product formatted for UI
*/
export function mapProductForUI(apiProduct) {
const p = apiProduct;
// Parse price from string or number
const parsePrice = (val) => {
if (val === null || val === undefined) return null;
const num = typeof val === 'string' ? parseFloat(val) : val;
return isNaN(num) ? null : num;
};
return {
id: p.id,
name: p.name || p.name_raw,
brand: p.brand || p.brand_name || p.brand_name_raw,
category: p.category || p.type || p.category_raw,
subcategory: p.subcategory || p.subcategory_raw,
strainType: p.strain_type,
image: p.image_url || p.primary_image_url,
thc: p.thc || p.thc_percent || p.thc_percentage,
cbd: p.cbd || p.cbd_percent || p.cbd_percentage,
price: parsePrice(p.price_rec) || parsePrice(p.regular_price) || parsePrice(p.price),
salePrice: parsePrice(p.price_rec_special) || parsePrice(p.sale_price),
inStock: p.in_stock !== undefined ? p.in_stock : p.stock_status === 'in_stock',
stockStatus: p.stock_status,
onSale: p.on_special || p.special || false,
updatedAt: p.updated_at || p.snapshot_at,
};
}
/**
* Get aggregate stats (product count, brand count, dispensary count)
* @returns {Promise<Object>}
*/
export async function getStats() {
return apiRequest('/api/v1/stats');
}

View File

@@ -1,10 +1,11 @@
import React, { useState, useEffect } from 'react'; import React, { useState, useEffect } from 'react';
import { useParams, Link } from 'react-router-dom'; import { useParams, Link } from 'react-router-dom';
import { MapPin, Phone, Clock, Star, Navigation, ArrowLeft, Share2, Heart, Loader2 } from 'lucide-react'; import { MapPin, Phone, Clock, Star, Navigation, ArrowLeft, Share2, Heart, Loader2, Search, Package } from 'lucide-react';
import { Button } from '../../components/ui/button'; import { Button } from '../../components/ui/button';
import { Input } from '../../components/ui/input';
import { Card, CardContent, CardHeader, CardTitle } from '../../components/ui/card'; import { Card, CardContent, CardHeader, CardTitle } from '../../components/ui/card';
import { Badge } from '../../components/ui/badge'; import { Badge } from '../../components/ui/badge';
import { getDispensaryBySlug, mapDispensaryForUI } from '../../api/client'; import { getDispensaryBySlug, mapDispensaryForUI, getDispensaryProducts, getDispensaryCategories, mapProductForUI } from '../../api/client';
import { formatDistance } from '../../lib/utils'; import { formatDistance } from '../../lib/utils';
export function DispensaryDetail() { export function DispensaryDetail() {
@@ -13,6 +14,14 @@ export function DispensaryDetail() {
const [loading, setLoading] = useState(true); const [loading, setLoading] = useState(true);
const [error, setError] = useState(null); const [error, setError] = useState(null);
// Products state
const [products, setProducts] = useState([]);
const [categories, setCategories] = useState([]);
const [productsLoading, setProductsLoading] = useState(false);
const [selectedCategory, setSelectedCategory] = useState('all');
const [searchQuery, setSearchQuery] = useState('');
const [productCount, setProductCount] = useState(0);
useEffect(() => { useEffect(() => {
const fetchDispensary = async () => { const fetchDispensary = async () => {
try { try {
@@ -30,6 +39,35 @@ export function DispensaryDetail() {
fetchDispensary(); fetchDispensary();
}, [slug]); }, [slug]);
// Fetch products when dispensary is loaded
useEffect(() => {
if (!dispensary?.id) return;
const fetchProducts = async () => {
try {
setProductsLoading(true);
const [productsRes, categoriesRes] = await Promise.all([
getDispensaryProducts(dispensary.id, {
category: selectedCategory !== 'all' ? selectedCategory : undefined,
search: searchQuery || undefined,
limit: 50,
}),
getDispensaryCategories(dispensary.id),
]);
setProducts((productsRes.products || []).map(mapProductForUI));
setProductCount(productsRes.pagination?.total || productsRes.products?.length || 0);
setCategories(categoriesRes.categories || []);
} catch (err) {
console.error('Error fetching products:', err);
} finally {
setProductsLoading(false);
}
};
fetchProducts();
}, [dispensary?.id, selectedCategory, searchQuery]);
if (loading) { if (loading) {
return ( return (
<div className="container mx-auto px-4 py-16 text-center"> <div className="container mx-auto px-4 py-16 text-center">
@@ -158,16 +196,66 @@ export function DispensaryDetail() {
</Card> </Card>
)} )}
{/* Products Section Placeholder */} {/* Products Section */}
<Card> <Card>
<CardHeader> <CardHeader>
<CardTitle>Available Products</CardTitle> <div className="flex flex-col sm:flex-row sm:items-center sm:justify-between gap-4">
<CardTitle>Available Products ({productCount})</CardTitle>
<div className="relative w-full sm:w-64">
<Search className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<Input
type="text"
placeholder="Search products..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="pl-10"
/>
</div>
</div>
{/* Category Filters */}
<div className="flex flex-wrap gap-2 mt-4">
<Button
size="sm"
variant={selectedCategory === 'all' ? 'default' : 'outline'}
onClick={() => setSelectedCategory('all')}
>
All
</Button>
{categories.map((cat) => (
<Button
key={cat.type || cat.name}
size="sm"
variant={selectedCategory === (cat.type || cat.name) ? 'default' : 'outline'}
onClick={() => setSelectedCategory(cat.type || cat.name)}
>
{cat.type || cat.name} ({cat.count || cat.product_count || 0})
</Button>
))}
</div>
</CardHeader> </CardHeader>
<CardContent> <CardContent>
<div className="text-center py-8 text-gray-500"> {productsLoading ? (
<p>Product menu coming soon</p> <div className="text-center py-8">
<p className="text-sm mt-2">Connect to API to view available products</p> <Loader2 className="h-8 w-8 mx-auto animate-spin text-primary" />
<p className="text-gray-500 mt-2">Loading products...</p>
</div> </div>
) : products.length === 0 ? (
<div className="text-center py-8 text-gray-500">
<Package className="h-12 w-12 mx-auto mb-4 text-gray-300" />
<p>No products found</p>
{searchQuery && (
<Button variant="link" onClick={() => setSearchQuery('')}>
Clear search
</Button>
)}
</div>
) : (
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 gap-4">
{products.map((product) => (
<ProductCard key={product.id} product={product} />
))}
</div>
)}
</CardContent> </CardContent>
</Card> </Card>
</div> </div>
@@ -225,4 +313,63 @@ export function DispensaryDetail() {
); );
} }
// Product Card Component
function ProductCard({ product }) {
const formatPrice = (price) => {
if (price === null || price === undefined) return null;
return `$${parseFloat(price).toFixed(2)}`;
};
return (
<Card className="overflow-hidden hover:shadow-md transition-shadow">
<div className="aspect-square bg-gray-100 relative">
{product.image ? (
<img
src={product.image}
alt={product.name}
className="w-full h-full object-cover"
/>
) : (
<div className="w-full h-full flex items-center justify-center">
<Package className="h-12 w-12 text-gray-300" />
</div>
)}
{product.onSale && (
<Badge className="absolute top-2 right-2 bg-red-500">Sale</Badge>
)}
{!product.inStock && (
<div className="absolute inset-0 bg-black/50 flex items-center justify-center">
<Badge variant="secondary">Out of Stock</Badge>
</div>
)}
</div>
<CardContent className="p-4">
<p className="text-xs text-gray-500 uppercase tracking-wide mb-1">
{product.brand || 'Unknown Brand'}
</p>
<h4 className="font-medium text-gray-900 line-clamp-2 mb-2">{product.name}</h4>
<div className="flex items-center gap-2 text-sm text-gray-500 mb-2">
{product.category && <Badge variant="outline" className="text-xs">{product.category}</Badge>}
{product.strainType && <Badge variant="outline" className="text-xs">{product.strainType}</Badge>}
</div>
{product.thc && (
<p className="text-xs text-gray-500 mb-2">THC: {product.thc}%</p>
)}
<div className="flex items-baseline gap-2">
{product.salePrice ? (
<>
<span className="font-bold text-red-600">{formatPrice(product.salePrice)}</span>
<span className="text-sm text-gray-400 line-through">{formatPrice(product.price)}</span>
</>
) : product.price ? (
<span className="font-bold text-gray-900">{formatPrice(product.price)}</span>
) : (
<span className="text-sm text-gray-400">Price not available</span>
)}
</div>
</CardContent>
</Card>
);
}
export default DispensaryDetail; export default DispensaryDetail;

114
findagram/FINDAGRAM.md Normal file
View File

@@ -0,0 +1,114 @@
# Findagram Development Notes
## Overview
Findagram (findagram.co) is a consumer-facing cannabis product discovery app. Users can search products across dispensaries, set price alerts, and save favorites.
## Architecture
- **Frontend**: React (Create React App) at `findagram/frontend/`
- **Backend**: Shared CannaiQ Express API at `backend/`
- **Auth**: JWT-based consumer auth via `/api/consumer/auth/*`
- **Domain**: `findagram.co` (passed in all auth requests)
## Key Files
| File | Purpose |
|------|---------|
| `src/context/AuthContext.js` | Global auth state, login/register, token management |
| `src/components/findagram/AuthModal.jsx` | Login/signup modal popup |
| `src/api/client.js` | API client for products, dispensaries, categories, brands |
| `src/api/consumer.js` | API client for favorites, alerts, saved searches (auth required) |
## Backend Consumer API Endpoints
All require JWT token in `Authorization: Bearer <token>` header.
### Auth (`/api/consumer/auth/*`)
- `POST /register` - Create account (requires `domain: 'findagram.co'`)
- `POST /login` - Login (requires `domain: 'findagram.co'`)
- `GET /me` - Get current user
- `PUT /me` - Update profile
### Favorites (`/api/consumer/favorites/*`)
- `GET /` - Get user's favorites
- `POST /` - Add favorite (`{ productId, dispensaryId? }`)
- `DELETE /:id` - Remove by favorite ID
- `DELETE /product/:productId` - Remove by product ID
- `GET /check/product/:id` - Check if product is favorited
### Alerts (`/api/consumer/alerts/*`)
- `GET /` - Get user's alerts
- `POST /` - Create alert (`{ alertType, productId, targetPrice }`)
- Alert types: `price_drop`, `back_in_stock`, `product_on_special`
- `PUT /:id` - Update alert
- `DELETE /:id` - Delete alert
- `POST /:id/toggle` - Toggle active status
### Saved Searches (`/api/consumer/saved-searches/*`)
- `GET /` - Get user's saved searches
- `POST /` - Create saved search
- `PUT /:id` - Update
- `DELETE /:id` - Delete
- `POST /:id/run` - Get search params for execution
## Database Tables (Consumer)
| Table | Purpose |
|-------|---------|
| `users` | User accounts (shared across domains via `domain` column) |
| `findagram_users` | Findagram-specific user profile data |
| `findagram_favorites` | Product favorites |
| `findagram_alerts` | Price/stock alerts |
| `findagram_saved_searches` | Saved search filters |
## Auth Flow
1. User clicks favorite/alert on a product
2. If not logged in → AuthModal opens
3. User logs in or creates account
4. JWT token stored in localStorage (`findagram_auth`)
5. Pending action (favorite/alert) executes automatically after auth
6. All subsequent API calls include `Authorization: Bearer <token>`
## Environment Variables
```bash
# Frontend (.env)
REACT_APP_API_URL=http://localhost:3010 # Local
REACT_APP_API_URL=https://cannaiq.co # Production
```
## Future: Migration to cannabrands.app
Currently uses CannaiQ backend. Later will migrate auth to cannabrands.app:
- Update `API_BASE_URL` for auth endpoints
- Keep product/dispensary API pointing to CannaiQ
- May need to sync user accounts between systems
## Important Notes
1. **Domain is critical** - All auth requests must include `domain: 'findagram.co'`
2. **Favorites are product-based** (unlike findadispo which is dispensary-based)
3. **Price alerts** require `targetPrice` for `price_drop` type
4. **Mock data** in `src/mockData.js` is no longer imported - can be safely deleted
5. **Token expiry** is 30 days (`JWT_EXPIRES_IN` in backend)
## Pages Using Real API
All pages are now wired to the real CannaiQ API:
| Page | API Endpoint | Notes |
|------|--------------|-------|
| Home | `/api/products`, `/api/dispensaries` | Featured products, deals |
| Products | `/api/products` | Search, filters, pagination |
| ProductDetail | `/api/products/:id` | Single product with dispensaries |
| Deals | `/api/products?hasSpecial=true` | Products on sale |
| Brands | `/api/brands` | Brand listing |
| BrandDetail | `/api/brands/:name` | Brand products |
| Categories | `/api/categories` | Category listing |
| CategoryDetail | `/api/products?category=...` | Category products |
| Dashboard | `/api/consumer/favorites`, alerts, searches | User dashboard (auth) |
| Favorites | `/api/consumer/favorites` | User favorites (auth) |
| Alerts | `/api/consumer/alerts` | Price alerts (auth) |
| SavedSearches | `/api/consumer/saved-searches` | Saved searches (auth) |

View File

@@ -1,7 +1,9 @@
import React, { useState } from 'react'; import React from 'react';
import { BrowserRouter as Router, Routes, Route } from 'react-router-dom'; import { BrowserRouter as Router, Routes, Route } from 'react-router-dom';
import { AuthProvider } from './context/AuthContext';
import Header from './components/findagram/Header'; import Header from './components/findagram/Header';
import Footer from './components/findagram/Footer'; import Footer from './components/findagram/Footer';
import AuthModal from './components/findagram/AuthModal';
// Pages // Pages
import Home from './pages/findagram/Home'; import Home from './pages/findagram/Home';
@@ -12,6 +14,7 @@ import Brands from './pages/findagram/Brands';
import BrandDetail from './pages/findagram/BrandDetail'; import BrandDetail from './pages/findagram/BrandDetail';
import Categories from './pages/findagram/Categories'; import Categories from './pages/findagram/Categories';
import CategoryDetail from './pages/findagram/CategoryDetail'; import CategoryDetail from './pages/findagram/CategoryDetail';
import DispensaryDetail from './pages/findagram/DispensaryDetail';
import About from './pages/findagram/About'; import About from './pages/findagram/About';
import Contact from './pages/findagram/Contact'; import Contact from './pages/findagram/Contact';
import Login from './pages/findagram/Login'; import Login from './pages/findagram/Login';
@@ -23,32 +26,11 @@ import SavedSearches from './pages/findagram/SavedSearches';
import Profile from './pages/findagram/Profile'; import Profile from './pages/findagram/Profile';
function App() { function App() {
const [isLoggedIn, setIsLoggedIn] = useState(false);
const [user, setUser] = useState(null);
// Mock login function
const handleLogin = (email, password) => {
// In a real app, this would make an API call
setUser({
id: 1,
name: 'John Doe',
email: email,
avatar: null,
});
setIsLoggedIn(true);
return true;
};
// Mock logout function
const handleLogout = () => {
setUser(null);
setIsLoggedIn(false);
};
return ( return (
<AuthProvider>
<Router> <Router>
<div className="flex flex-col min-h-screen"> <div className="flex flex-col min-h-screen">
<Header isLoggedIn={isLoggedIn} user={user} onLogout={handleLogout} /> <Header />
<main className="flex-grow"> <main className="flex-grow">
<Routes> <Routes>
@@ -61,12 +43,13 @@ function App() {
<Route path="/brands/:slug" element={<BrandDetail />} /> <Route path="/brands/:slug" element={<BrandDetail />} />
<Route path="/categories" element={<Categories />} /> <Route path="/categories" element={<Categories />} />
<Route path="/categories/:slug" element={<CategoryDetail />} /> <Route path="/categories/:slug" element={<CategoryDetail />} />
<Route path="/dispensaries/:slug" element={<DispensaryDetail />} />
<Route path="/about" element={<About />} /> <Route path="/about" element={<About />} />
<Route path="/contact" element={<Contact />} /> <Route path="/contact" element={<Contact />} />
{/* Auth Routes */} {/* Auth Routes */}
<Route path="/login" element={<Login onLogin={handleLogin} />} /> <Route path="/login" element={<Login />} />
<Route path="/signup" element={<Signup onLogin={handleLogin} />} /> <Route path="/signup" element={<Signup />} />
{/* Dashboard Routes */} {/* Dashboard Routes */}
<Route path="/dashboard" element={<Dashboard />} /> <Route path="/dashboard" element={<Dashboard />} />
@@ -78,8 +61,10 @@ function App() {
</main> </main>
<Footer /> <Footer />
<AuthModal />
</div> </div>
</Router> </Router>
</AuthProvider>
); );
} }

View File

@@ -130,7 +130,7 @@ export async function getStoreProducts(storeId, params = {}) {
offset: params.offset || 0, offset: params.offset || 0,
}); });
return request(`/api/v1/stores/${storeId}/products${queryString}`); return request(`/api/v1/dispensaries/${storeId}/products${queryString}`);
} }
// ============================================================ // ============================================================
@@ -149,47 +149,49 @@ export async function getStoreProducts(storeId, params = {}) {
export async function getDispensaries(params = {}) { export async function getDispensaries(params = {}) {
const queryString = buildQueryString({ const queryString = buildQueryString({
city: params.city, city: params.city,
state: params.state,
hasPlatformId: params.hasPlatformId, hasPlatformId: params.hasPlatformId,
has_products: params.hasProducts ? 'true' : undefined,
limit: params.limit || 100, limit: params.limit || 100,
offset: params.offset || 0, offset: params.offset || 0,
}); });
return request(`/api/v1/stores${queryString}`); return request(`/api/v1/dispensaries${queryString}`);
} }
/** /**
* Get a single dispensary by ID * Get a single dispensary by ID
*/ */
export async function getDispensary(id) { export async function getDispensary(id) {
return request(`/api/v1/stores/${id}`); return request(`/api/v1/dispensaries/${id}`);
} }
/** /**
* Get dispensary by slug or platform ID * Get dispensary by slug or platform ID
*/ */
export async function getDispensaryBySlug(slug) { export async function getDispensaryBySlug(slug) {
return request(`/api/v1/stores/slug/${slug}`); return request(`/api/v1/dispensaries/slug/${slug}`);
} }
/** /**
* Get dispensary summary (product counts, categories, brands) * Get dispensary summary (product counts, categories, brands)
*/ */
export async function getDispensarySummary(id) { export async function getDispensarySummary(id) {
return request(`/api/v1/stores/${id}/summary`); return request(`/api/v1/dispensaries/${id}/summary`);
} }
/** /**
* Get brands available at a specific dispensary * Get brands available at a specific dispensary
*/ */
export async function getDispensaryBrands(id) { export async function getDispensaryBrands(id) {
return request(`/api/v1/stores/${id}/brands`); return request(`/api/v1/dispensaries/${id}/brands`);
} }
/** /**
* Get categories available at a specific dispensary * Get categories available at a specific dispensary
*/ */
export async function getDispensaryCategories(id) { export async function getDispensaryCategories(id) {
return request(`/api/v1/stores/${id}/categories`); return request(`/api/v1/dispensaries/${id}/categories`);
} }
// ============================================================ // ============================================================
@@ -224,29 +226,48 @@ export async function getBrands(params = {}) {
} }
// ============================================================ // ============================================================
// DEALS / SPECIALS // STATS
// Note: The /api/az routes don't have a dedicated specials endpoint yet.
// For now, we can filter products with sale prices or use dispensary-specific specials.
// ============================================================ // ============================================================
/** /**
* Get products on sale (products where sale_price exists) * Get aggregate stats (product count, brand count, dispensary count)
* This is a client-side filter until a dedicated endpoint is added.
*/ */
export async function getDeals(params = {}) { export async function getStats() {
// For now, get products and we'll need to filter client-side return request('/api/v1/stats');
// or we could use the /api/dispensaries/:slug/specials endpoint if we have a dispensary context }
const result = await getProducts({
...params, // ============================================================
// DEALS / SPECIALS
// ============================================================
/**
* Get products on special/sale
* Uses the on_special filter parameter on the products endpoint
*
* @param {Object} params
* @param {string} [params.type] - Category type filter
* @param {string} [params.brandName] - Brand name filter
* @param {number} [params.limit=100] - Page size
* @param {number} [params.offset=0] - Offset for pagination
*/
export async function getSpecials(params = {}) {
const queryString = buildQueryString({
on_special: 'true',
type: params.type,
brandName: params.brandName,
stockStatus: params.stockStatus || 'in_stock',
limit: params.limit || 100, limit: params.limit || 100,
offset: params.offset || 0,
}); });
// Filter to only products with a sale price return request(`/api/v1/products${queryString}`);
// Note: This is a temporary solution - ideally the backend would support this filter }
return {
...result, /**
products: result.products.filter(p => p.sale_price || p.med_sale_price), * Alias for getSpecials for backward compatibility
}; */
export async function getDeals(params = {}) {
return getSpecials(params);
} }
// ============================================================ // ============================================================
@@ -278,27 +299,40 @@ export function mapProductForUI(apiProduct) {
// Handle both direct product and transformed product formats // Handle both direct product and transformed product formats
const p = apiProduct; const p = apiProduct;
// Helper to parse price (API returns strings like "29.99" or null)
const parsePrice = (val) => {
if (val === null || val === undefined) return null;
const num = typeof val === 'string' ? parseFloat(val) : val;
return isNaN(num) ? null : num;
};
const regularPrice = parsePrice(p.regular_price);
const salePrice = parsePrice(p.sale_price);
const medPrice = parsePrice(p.med_price);
const medSalePrice = parsePrice(p.med_sale_price);
const regularPriceMax = parsePrice(p.regular_price_max);
return { return {
id: p.id, id: p.id,
name: p.name, name: p.name,
brand: p.brand || p.brand_name, brand: p.brand || p.brand_name,
category: p.type || p.category, category: p.type || p.category || p.category_raw,
subcategory: p.subcategory, subcategory: p.subcategory || p.subcategory_raw,
strainType: p.strain_type || null, strainType: p.strain_type || null,
// Images // Images
image: p.image_url || p.primary_image_url || null, image: p.image_url || p.primary_image_url || null,
// Potency // Potency
thc: p.thc_percentage || p.thc_content || null, thc: p.thc_percentage || p.thc_content || null,
cbd: p.cbd_percentage || p.cbd_content || null, cbd: p.cbd_percentage || p.cbd_content || null,
// Prices (API returns dollars as numbers or null) // Prices (parsed to numbers)
price: p.regular_price || null, price: regularPrice,
priceRange: p.regular_price_max && p.regular_price priceRange: regularPriceMax && regularPrice
? { min: p.regular_price, max: p.regular_price_max } ? { min: regularPrice, max: regularPriceMax }
: null, : null,
onSale: !!(p.sale_price || p.med_sale_price), onSale: !!(salePrice || medSalePrice),
salePrice: p.sale_price || null, salePrice: salePrice,
medPrice: p.med_price || null, medPrice: medPrice,
medSalePrice: p.med_sale_price || null, medSalePrice: medSalePrice,
// Stock // Stock
inStock: p.in_stock !== undefined ? p.in_stock : p.stock_status === 'in_stock', inStock: p.in_stock !== undefined ? p.in_stock : p.stock_status === 'in_stock',
stockStatus: p.stock_status, stockStatus: p.stock_status,
@@ -354,23 +388,41 @@ export function mapBrandForUI(apiBrand) {
* Map API dispensary to UI-compatible format * Map API dispensary to UI-compatible format
*/ */
export function mapDispensaryForUI(apiDispensary) { export function mapDispensaryForUI(apiDispensary) {
// Handle location object from API (location.latitude, location.longitude)
const lat = apiDispensary.location?.latitude || apiDispensary.latitude;
const lng = apiDispensary.location?.longitude || apiDispensary.longitude;
return { return {
id: apiDispensary.id, id: apiDispensary.id,
name: apiDispensary.dba_name || apiDispensary.name, name: apiDispensary.dba_name || apiDispensary.name,
slug: apiDispensary.slug, slug: apiDispensary.slug,
city: apiDispensary.city, city: apiDispensary.city,
state: apiDispensary.state, state: apiDispensary.state,
address: apiDispensary.address, address: apiDispensary.address1 || apiDispensary.address,
zip: apiDispensary.zip, zip: apiDispensary.zip,
latitude: apiDispensary.latitude, latitude: lat,
longitude: apiDispensary.longitude, longitude: lng,
website: apiDispensary.website, website: apiDispensary.website,
menuUrl: apiDispensary.menu_url, menuUrl: apiDispensary.menu_url,
// Summary data (if fetched with summary) imageUrl: apiDispensary.image_url,
productCount: apiDispensary.totalProducts, rating: apiDispensary.rating,
reviewCount: apiDispensary.review_count,
// Product data from API
productCount: apiDispensary.product_count || apiDispensary.totalProducts || 0,
inStockCount: apiDispensary.in_stock_count || apiDispensary.inStockCount || 0,
brandCount: apiDispensary.brandCount, brandCount: apiDispensary.brandCount,
categoryCount: apiDispensary.categoryCount, categoryCount: apiDispensary.categoryCount,
inStockCount: apiDispensary.inStockCount, // Services
services: apiDispensary.services || {
pickup: false,
delivery: false,
curbside: false
},
// License type
licenseType: apiDispensary.license_type || {
medical: false,
recreational: false
},
}; };
} }
@@ -386,6 +438,68 @@ function formatCategoryName(type) {
.replace(/\b\w/g, c => c.toUpperCase()); .replace(/\b\w/g, c => c.toUpperCase());
} }
// ============================================================
// CLICK TRACKING
// ============================================================
/**
* Get cached visitor location from sessionStorage
*/
function getCachedVisitorLocation() {
try {
const cached = sessionStorage.getItem('findagram_location');
if (cached) {
return JSON.parse(cached);
}
} catch (err) {
// Ignore errors
}
return null;
}
/**
* Track a product click event
* Fire-and-forget - doesn't block UI
*
* @param {Object} params
* @param {string} params.productId - Product ID (required)
* @param {string} [params.storeId] - Store/dispensary ID
* @param {string} [params.brandId] - Brand name/ID
* @param {string} [params.dispensaryName] - Dispensary name
* @param {string} params.action - Action type: view, open_product, open_store, compare
* @param {string} params.source - Source identifier (e.g., 'findagram')
* @param {string} [params.pageType] - Page type (e.g., 'home', 'dispensary', 'deals')
*/
export function trackProductClick(params) {
// Get visitor's cached location
const visitorLocation = getCachedVisitorLocation();
const payload = {
product_id: String(params.productId),
store_id: params.storeId ? String(params.storeId) : undefined,
brand_id: params.brandId || undefined,
dispensary_name: params.dispensaryName || undefined,
action: params.action || 'view',
source: params.source || 'findagram',
page_type: params.pageType || undefined,
url_path: window.location.pathname,
// Visitor location from IP geolocation
visitor_city: visitorLocation?.city || undefined,
visitor_state: visitorLocation?.state || undefined,
visitor_lat: visitorLocation?.lat || undefined,
visitor_lng: visitorLocation?.lng || undefined,
};
// Fire and forget - don't await
fetch(`${API_BASE_URL}/api/events/product-click`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
}).catch(() => {
// Silently ignore errors - analytics shouldn't break UX
});
}
// Default export for convenience // Default export for convenience
const api = { const api = {
// Products // Products
@@ -405,13 +519,18 @@ const api = {
// Categories & Brands // Categories & Brands
getCategories, getCategories,
getBrands, getBrands,
// Stats
getStats,
// Deals // Deals
getDeals, getDeals,
getSpecials,
// Mappers // Mappers
mapProductForUI, mapProductForUI,
mapCategoryForUI, mapCategoryForUI,
mapBrandForUI, mapBrandForUI,
mapDispensaryForUI, mapDispensaryForUI,
// Tracking
trackProductClick,
}; };
export default api; export default api;

View File

@@ -0,0 +1,302 @@
/**
* Consumer API Client for Findagram
*
* Handles authenticated requests for:
* - Favorites
* - Price Alerts
* - Saved Searches
*
* All methods require auth token (use with AuthContext's authFetch)
*/
// ============================================================
// FAVORITES
// ============================================================
/**
* Get all user's favorites
* @param {Function} authFetch - Authenticated fetch from AuthContext
*/
export async function getFavorites(authFetch) {
return authFetch('/api/consumer/favorites');
}
/**
* Add product to favorites
* @param {Function} authFetch
* @param {number} productId
* @param {number} [dispensaryId] - Optional dispensary context
*/
export async function addFavorite(authFetch, productId, dispensaryId = null) {
return authFetch('/api/consumer/favorites', {
method: 'POST',
body: JSON.stringify({ productId, dispensaryId }),
});
}
/**
* Remove favorite by favorite ID
* @param {Function} authFetch
* @param {number} favoriteId
*/
export async function removeFavorite(authFetch, favoriteId) {
return authFetch(`/api/consumer/favorites/${favoriteId}`, {
method: 'DELETE',
});
}
/**
* Remove favorite by product ID
* @param {Function} authFetch
* @param {number} productId
*/
export async function removeFavoriteByProduct(authFetch, productId) {
return authFetch(`/api/consumer/favorites/product/${productId}`, {
method: 'DELETE',
});
}
/**
* Check if product is favorited
* @param {Function} authFetch
* @param {number} productId
* @returns {Promise<{isFavorited: boolean}>}
*/
export async function checkFavorite(authFetch, productId) {
return authFetch(`/api/consumer/favorites/check/product/${productId}`);
}
// ============================================================
// ALERTS
// ============================================================
/**
* Get all user's alerts
* @param {Function} authFetch
*/
export async function getAlerts(authFetch) {
return authFetch('/api/consumer/alerts');
}
/**
* Get alert statistics
* @param {Function} authFetch
*/
export async function getAlertStats(authFetch) {
return authFetch('/api/consumer/alerts/stats');
}
/**
* Create a price drop alert
* @param {Function} authFetch
* @param {Object} params
* @param {number} params.productId - Product to track
* @param {number} params.targetPrice - Price to alert at
* @param {number} [params.dispensaryId] - Optional dispensary context
*/
export async function createPriceAlert(authFetch, { productId, targetPrice, dispensaryId }) {
return authFetch('/api/consumer/alerts', {
method: 'POST',
body: JSON.stringify({
alertType: 'price_drop',
productId,
targetPrice,
dispensaryId,
}),
});
}
/**
* Create a back-in-stock alert
* @param {Function} authFetch
* @param {Object} params
* @param {number} params.productId - Product to track
* @param {number} [params.dispensaryId] - Optional dispensary context
*/
export async function createStockAlert(authFetch, { productId, dispensaryId }) {
return authFetch('/api/consumer/alerts', {
method: 'POST',
body: JSON.stringify({
alertType: 'back_in_stock',
productId,
dispensaryId,
}),
});
}
/**
* Create a brand/category alert
* @param {Function} authFetch
* @param {Object} params
* @param {string} [params.brand] - Brand to track
* @param {string} [params.category] - Category to track
*/
export async function createBrandCategoryAlert(authFetch, { brand, category }) {
return authFetch('/api/consumer/alerts', {
method: 'POST',
body: JSON.stringify({
alertType: 'product_on_special',
brand,
category,
}),
});
}
/**
* Update an alert
* @param {Function} authFetch
* @param {number} alertId
* @param {Object} updates
* @param {boolean} [updates.isActive]
* @param {number} [updates.targetPrice]
*/
export async function updateAlert(authFetch, alertId, updates) {
return authFetch(`/api/consumer/alerts/${alertId}`, {
method: 'PUT',
body: JSON.stringify(updates),
});
}
/**
* Toggle alert active status
* @param {Function} authFetch
* @param {number} alertId
*/
export async function toggleAlert(authFetch, alertId) {
return authFetch(`/api/consumer/alerts/${alertId}/toggle`, {
method: 'POST',
});
}
/**
* Delete an alert
* @param {Function} authFetch
* @param {number} alertId
*/
export async function deleteAlert(authFetch, alertId) {
return authFetch(`/api/consumer/alerts/${alertId}`, {
method: 'DELETE',
});
}
// ============================================================
// SAVED SEARCHES
// ============================================================
/**
* Get all user's saved searches
* @param {Function} authFetch
*/
export async function getSavedSearches(authFetch) {
return authFetch('/api/consumer/saved-searches');
}
/**
* Create a saved search
* @param {Function} authFetch
* @param {Object} params
* @param {string} params.name - Display name
* @param {string} [params.query] - Search query
* @param {string} [params.category] - Category filter
* @param {string} [params.brand] - Brand filter
* @param {string} [params.strainType] - Strain type filter
* @param {number} [params.minPrice] - Min price filter
* @param {number} [params.maxPrice] - Max price filter
* @param {number} [params.minThc] - Min THC filter
* @param {number} [params.maxThc] - Max THC filter
* @param {boolean} [params.notifyOnNew] - Notify on new products
* @param {boolean} [params.notifyOnPriceDrop] - Notify on price drops
*/
export async function createSavedSearch(authFetch, params) {
return authFetch('/api/consumer/saved-searches', {
method: 'POST',
body: JSON.stringify(params),
});
}
/**
* Update a saved search
* @param {Function} authFetch
* @param {number} searchId
* @param {Object} updates
*/
export async function updateSavedSearch(authFetch, searchId, updates) {
return authFetch(`/api/consumer/saved-searches/${searchId}`, {
method: 'PUT',
body: JSON.stringify(updates),
});
}
/**
* Delete a saved search
* @param {Function} authFetch
* @param {number} searchId
*/
export async function deleteSavedSearch(authFetch, searchId) {
return authFetch(`/api/consumer/saved-searches/${searchId}`, {
method: 'DELETE',
});
}
/**
* Run a saved search (get search params)
* @param {Function} authFetch
* @param {number} searchId
* @returns {Promise<{searchParams: Object, searchUrl: string}>}
*/
export async function runSavedSearch(authFetch, searchId) {
return authFetch(`/api/consumer/saved-searches/${searchId}/run`, {
method: 'POST',
});
}
// ============================================================
// HELPER: Generate search name from filters
// ============================================================
/**
* Generate a display name for a search based on filters
* @param {Object} filters
* @returns {string}
*/
export function generateSearchName(filters) {
const parts = [];
if (filters.query || filters.search) parts.push(`"${filters.query || filters.search}"`);
if (filters.category || filters.type) parts.push(filters.category || filters.type);
if (filters.brand || filters.brandName) parts.push(filters.brand || filters.brandName);
if (filters.strainType) parts.push(filters.strainType);
if (filters.maxPrice) parts.push(`Under $${filters.maxPrice}`);
if (filters.minThc) parts.push(`${filters.minThc}%+ THC`);
return parts.length > 0 ? parts.join(' - ') : 'All Products';
}
// Default export
const consumerApi = {
// Favorites
getFavorites,
addFavorite,
removeFavorite,
removeFavoriteByProduct,
checkFavorite,
// Alerts
getAlerts,
getAlertStats,
createPriceAlert,
createStockAlert,
createBrandCategoryAlert,
updateAlert,
toggleAlert,
deleteAlert,
// Saved Searches
getSavedSearches,
createSavedSearch,
updateSavedSearch,
deleteSavedSearch,
runSavedSearch,
// Helpers
generateSearchName,
};
export default consumerApi;

View File

@@ -0,0 +1,315 @@
/**
* AuthModal - Login/Signup modal for Findagram
*
* Shows when user tries to:
* - Favorite a product
* - Set a price alert
* - Save a search
* - Access dashboard features
*/
import React, { useState } from 'react';
import { useAuth } from '../../context/AuthContext';
import { Card, CardContent, CardHeader, CardTitle, CardDescription } from '../ui/card';
import { Button } from '../ui/button';
import { X, Mail, Lock, User, Phone, MapPin, Loader2, Eye, EyeOff } from 'lucide-react';
const AuthModal = () => {
const {
showAuthModal,
authModalMode,
setAuthModalMode,
closeAuthModal,
login,
register,
} = useAuth();
const [formData, setFormData] = useState({
email: '',
password: '',
firstName: '',
lastName: '',
phone: '',
city: '',
state: '',
});
const [showPassword, setShowPassword] = useState(false);
const [loading, setLoading] = useState(false);
const [error, setError] = useState('');
if (!showAuthModal) return null;
const handleChange = (e) => {
const { name, value } = e.target;
setFormData(prev => ({ ...prev, [name]: value }));
setError('');
};
const handleSubmit = async (e) => {
e.preventDefault();
setError('');
setLoading(true);
try {
if (authModalMode === 'login') {
await login(formData.email, formData.password);
} else {
// Validate signup fields
if (!formData.firstName || !formData.lastName) {
throw new Error('First and last name are required');
}
if (formData.password.length < 6) {
throw new Error('Password must be at least 6 characters');
}
await register(formData);
}
} catch (err) {
setError(err.message);
} finally {
setLoading(false);
}
};
const switchMode = () => {
setAuthModalMode(authModalMode === 'login' ? 'signup' : 'login');
setError('');
};
return (
<div className="fixed inset-0 z-50 flex items-center justify-center p-4">
{/* Backdrop */}
<div
className="absolute inset-0 bg-black/50 backdrop-blur-sm"
onClick={closeAuthModal}
/>
{/* Modal */}
<Card className="relative w-full max-w-md bg-white shadow-2xl animate-in fade-in zoom-in duration-200">
{/* Close button */}
<button
onClick={closeAuthModal}
className="absolute top-4 right-4 p-1 rounded-full hover:bg-gray-100 transition-colors"
>
<X className="h-5 w-5 text-gray-500" />
</button>
<CardHeader className="text-center pb-2">
<div className="mx-auto w-12 h-12 bg-gradient-to-br from-purple-500 to-pink-500 rounded-full flex items-center justify-center mb-4">
<User className="h-6 w-6 text-white" />
</div>
<CardTitle>
{authModalMode === 'login' ? 'Welcome Back' : 'Create Account'}
</CardTitle>
<CardDescription>
{authModalMode === 'login'
? 'Sign in to save favorites and set price alerts'
: 'Join Findagram to track products and get notified of deals'}
</CardDescription>
</CardHeader>
<CardContent>
<form onSubmit={handleSubmit} className="space-y-4">
{/* Error message */}
{error && (
<div className="p-3 bg-red-50 border border-red-200 rounded-lg text-red-600 text-sm">
{error}
</div>
)}
{/* Signup only fields */}
{authModalMode === 'signup' && (
<>
<div className="grid grid-cols-2 gap-3">
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
First Name
</label>
<div className="relative">
<User className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<input
type="text"
name="firstName"
value={formData.firstName}
onChange={handleChange}
className="w-full pl-10 pr-3 py-2 border border-gray-300 rounded-lg focus:ring-2 focus:ring-purple-500 focus:border-transparent"
placeholder="John"
required
/>
</div>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Last Name
</label>
<input
type="text"
name="lastName"
value={formData.lastName}
onChange={handleChange}
className="w-full px-3 py-2 border border-gray-300 rounded-lg focus:ring-2 focus:ring-purple-500 focus:border-transparent"
placeholder="Doe"
required
/>
</div>
</div>
</>
)}
{/* Email */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Email
</label>
<div className="relative">
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<input
type="email"
name="email"
value={formData.email}
onChange={handleChange}
className="w-full pl-10 pr-3 py-2 border border-gray-300 rounded-lg focus:ring-2 focus:ring-purple-500 focus:border-transparent"
placeholder="you@example.com"
required
/>
</div>
</div>
{/* Password */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Password
</label>
<div className="relative">
<Lock className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<input
type={showPassword ? 'text' : 'password'}
name="password"
value={formData.password}
onChange={handleChange}
className="w-full pl-10 pr-10 py-2 border border-gray-300 rounded-lg focus:ring-2 focus:ring-purple-500 focus:border-transparent"
placeholder={authModalMode === 'signup' ? 'Min 6 characters' : 'Your password'}
required
minLength={authModalMode === 'signup' ? 6 : undefined}
/>
<button
type="button"
onClick={() => setShowPassword(!showPassword)}
className="absolute right-3 top-1/2 -translate-y-1/2 text-gray-400 hover:text-gray-600"
>
{showPassword ? <EyeOff className="h-4 w-4" /> : <Eye className="h-4 w-4" />}
</button>
</div>
</div>
{/* Signup only: Phone & Location */}
{authModalMode === 'signup' && (
<>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Phone <span className="text-gray-400">(optional)</span>
</label>
<div className="relative">
<Phone className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<input
type="tel"
name="phone"
value={formData.phone}
onChange={handleChange}
className="w-full pl-10 pr-3 py-2 border border-gray-300 rounded-lg focus:ring-2 focus:ring-purple-500 focus:border-transparent"
placeholder="(555) 123-4567"
/>
</div>
<p className="text-xs text-gray-500 mt-1">For SMS alerts about price drops</p>
</div>
<div className="grid grid-cols-2 gap-3">
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
City <span className="text-gray-400">(optional)</span>
</label>
<div className="relative">
<MapPin className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<input
type="text"
name="city"
value={formData.city}
onChange={handleChange}
className="w-full pl-10 pr-3 py-2 border border-gray-300 rounded-lg focus:ring-2 focus:ring-purple-500 focus:border-transparent"
placeholder="Phoenix"
/>
</div>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
State
</label>
<select
name="state"
value={formData.state}
onChange={handleChange}
className="w-full px-3 py-2 border border-gray-300 rounded-lg focus:ring-2 focus:ring-purple-500 focus:border-transparent"
>
<option value="">Select...</option>
<option value="AZ">Arizona</option>
<option value="CA">California</option>
<option value="CO">Colorado</option>
<option value="MI">Michigan</option>
<option value="NV">Nevada</option>
<option value="OR">Oregon</option>
<option value="WA">Washington</option>
</select>
</div>
</div>
</>
)}
{/* Submit button */}
<Button
type="submit"
disabled={loading}
className="w-full bg-gradient-to-r from-purple-600 to-pink-600 hover:from-purple-700 hover:to-pink-700 text-white py-2.5"
>
{loading ? (
<>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
{authModalMode === 'login' ? 'Signing in...' : 'Creating account...'}
</>
) : (
authModalMode === 'login' ? 'Sign In' : 'Create Account'
)}
</Button>
{/* Switch mode link */}
<div className="text-center text-sm text-gray-600">
{authModalMode === 'login' ? (
<>
Don't have an account?{' '}
<button
type="button"
onClick={switchMode}
className="text-purple-600 hover:text-purple-700 font-medium"
>
Sign up
</button>
</>
) : (
<>
Already have an account?{' '}
<button
type="button"
onClick={switchMode}
className="text-purple-600 hover:text-purple-700 font-medium"
>
Sign in
</button>
</>
)}
</div>
</form>
</CardContent>
</Card>
</div>
);
};
export default AuthModal;

View File

@@ -1,5 +1,6 @@
import React, { useState } from 'react'; import React, { useState } from 'react';
import { Link, useLocation } from 'react-router-dom'; import { Link, useLocation } from 'react-router-dom';
import { useAuth } from '../../context/AuthContext';
import { Button } from '../ui/button'; import { Button } from '../ui/button';
import { Input } from '../ui/input'; import { Input } from '../ui/input';
import { import {
@@ -27,7 +28,8 @@ import {
Store, Store,
} from 'lucide-react'; } from 'lucide-react';
const Header = ({ isLoggedIn = false, user = null }) => { const Header = () => {
const { isAuthenticated, user, logout, openAuthModal } = useAuth();
const [mobileMenuOpen, setMobileMenuOpen] = useState(false); const [mobileMenuOpen, setMobileMenuOpen] = useState(false);
const [searchQuery, setSearchQuery] = useState(''); const [searchQuery, setSearchQuery] = useState('');
const location = useLocation(); const location = useLocation();
@@ -99,7 +101,7 @@ const Header = ({ isLoggedIn = false, user = null }) => {
{/* Right side actions */} {/* Right side actions */}
<div className="flex items-center space-x-4"> <div className="flex items-center space-x-4">
{isLoggedIn ? ( {isAuthenticated ? (
<> <>
{/* Favorites */} {/* Favorites */}
<Link to="/dashboard/favorites" className="hidden sm:block"> <Link to="/dashboard/favorites" className="hidden sm:block">
@@ -121,9 +123,9 @@ const Header = ({ isLoggedIn = false, user = null }) => {
<DropdownMenuTrigger asChild> <DropdownMenuTrigger asChild>
<Button variant="ghost" className="relative h-10 w-10 rounded-full"> <Button variant="ghost" className="relative h-10 w-10 rounded-full">
<Avatar className="h-10 w-10 border-2 border-primary"> <Avatar className="h-10 w-10 border-2 border-primary">
<AvatarImage src={user?.avatar} alt={user?.name} /> <AvatarImage src={user?.avatar} alt={user?.firstName} />
<AvatarFallback className="bg-primary text-white"> <AvatarFallback className="bg-primary text-white">
{user?.name?.charAt(0) || 'U'} {user?.firstName?.charAt(0) || 'U'}
</AvatarFallback> </AvatarFallback>
</Avatar> </Avatar>
</Button> </Button>
@@ -131,9 +133,11 @@ const Header = ({ isLoggedIn = false, user = null }) => {
<DropdownMenuContent className="w-56" align="end" forceMount> <DropdownMenuContent className="w-56" align="end" forceMount>
<DropdownMenuLabel className="font-normal"> <DropdownMenuLabel className="font-normal">
<div className="flex flex-col space-y-1"> <div className="flex flex-col space-y-1">
<p className="text-sm font-medium leading-none">{user?.name || 'User'}</p> <p className="text-sm font-medium leading-none">
{user?.firstName} {user?.lastName}
</p>
<p className="text-xs leading-none text-muted-foreground"> <p className="text-xs leading-none text-muted-foreground">
{user?.email || 'user@example.com'} {user?.email}
</p> </p>
</div> </div>
</DropdownMenuLabel> </DropdownMenuLabel>
@@ -169,7 +173,10 @@ const Header = ({ isLoggedIn = false, user = null }) => {
Settings Settings
</Link> </Link>
</DropdownMenuItem> </DropdownMenuItem>
<DropdownMenuItem className="text-red-600"> <DropdownMenuItem
className="text-red-600 cursor-pointer"
onClick={logout}
>
<LogOut className="mr-2 h-4 w-4" /> <LogOut className="mr-2 h-4 w-4" />
Log out Log out
</DropdownMenuItem> </DropdownMenuItem>
@@ -178,16 +185,19 @@ const Header = ({ isLoggedIn = false, user = null }) => {
</> </>
) : ( ) : (
<> <>
<Link to="/login" className="hidden sm:block"> <Button
<Button variant="ghost" className="text-gray-600"> variant="ghost"
className="hidden sm:block text-gray-600"
onClick={() => openAuthModal('login')}
>
Log in Log in
</Button> </Button>
</Link> <Button
<Link to="/signup"> className="gradient-purple text-white hover:opacity-90"
<Button className="gradient-purple text-white hover:opacity-90"> onClick={() => openAuthModal('signup')}
>
Sign up Sign up
</Button> </Button>
</Link>
</> </>
)} )}
@@ -241,7 +251,7 @@ const Header = ({ isLoggedIn = false, user = null }) => {
<span className="font-medium">{item.name}</span> <span className="font-medium">{item.name}</span>
</Link> </Link>
))} ))}
{isLoggedIn && ( {isAuthenticated && (
<> <>
<div className="border-t border-gray-200 my-2" /> <div className="border-t border-gray-200 my-2" />
<Link <Link

View File

@@ -1,16 +1,59 @@
import React from 'react'; import React, { useState, useEffect } from 'react';
import { Link } from 'react-router-dom'; import { Link, useLocation } from 'react-router-dom';
import { Card, CardContent } from '../ui/card'; import { Card, CardContent } from '../ui/card';
import { Badge } from '../ui/badge'; import { Badge } from '../ui/badge';
import { Button } from '../ui/button'; import { Button } from '../ui/button';
import { Heart, Star, MapPin, TrendingDown } from 'lucide-react'; import { Heart, Star, MapPin, TrendingDown, Loader2 } from 'lucide-react';
import { useAuth } from '../../context/AuthContext';
import { addFavorite, removeFavoriteByProduct, checkFavorite } from '../../api/consumer';
import { trackProductClick } from '../../api/client';
const ProductCard = ({ const ProductCard = ({
product, product,
onFavorite, onFavoriteChange,
isFavorite = false, initialIsFavorite,
showDispensaryCount = true showDispensaryCount = true,
pageType = 'browse'
}) => { }) => {
const location = useLocation();
const { isAuthenticated, requireAuth, authFetch } = useAuth();
const [isFavorite, setIsFavorite] = useState(initialIsFavorite || false);
const [favoriteLoading, setFavoriteLoading] = useState(false);
// Check favorite status on mount if authenticated
useEffect(() => {
if (isAuthenticated && product?.id && initialIsFavorite === undefined) {
checkFavorite(authFetch, product.id)
.then(data => setIsFavorite(data.isFavorited))
.catch(() => {}); // Ignore errors
}
}, [isAuthenticated, product?.id, authFetch, initialIsFavorite]);
const handleFavoriteClick = async (e) => {
e.preventDefault();
e.stopPropagation();
// If not authenticated, show auth modal with pending action
if (!requireAuth(() => handleFavoriteClick({ preventDefault: () => {}, stopPropagation: () => {} }))) {
return;
}
setFavoriteLoading(true);
try {
if (isFavorite) {
await removeFavoriteByProduct(authFetch, product.id);
setIsFavorite(false);
} else {
await addFavorite(authFetch, product.id, product.dispensaryId);
setIsFavorite(true);
}
onFavoriteChange?.(product.id, !isFavorite);
} catch (error) {
console.error('Failed to update favorite:', error);
} finally {
setFavoriteLoading(false);
}
};
const { const {
id, id,
name, name,
@@ -35,11 +78,24 @@ const ProductCard = ({
hybrid: 'bg-green-100 text-green-800', hybrid: 'bg-green-100 text-green-800',
}; };
const savings = onSale && salePrice ? ((price - salePrice) / price * 100).toFixed(0) : 0; const savings = onSale && salePrice && price ? ((price - salePrice) / price * 100).toFixed(0) : 0;
// Track product click
const handleProductClick = () => {
trackProductClick({
productId: id,
storeId: product.dispensaryId,
brandId: brand,
dispensaryName: product.storeName,
action: 'open_product',
source: 'findagram',
pageType: pageType || location.pathname.split('/')[1] || 'home',
});
};
return ( return (
<Card className="product-card group overflow-hidden"> <Card className="product-card group overflow-hidden">
<Link to={`/products/${id}`}> <Link to={`/products/${id}`} onClick={handleProductClick}>
{/* Image Container */} {/* Image Container */}
<div className="relative aspect-square overflow-hidden bg-gray-100"> <div className="relative aspect-square overflow-hidden bg-gray-100">
<img <img
@@ -70,13 +126,14 @@ const ProductCard = ({
className={`absolute top-3 right-3 h-8 w-8 rounded-full bg-white/80 hover:bg-white ${ className={`absolute top-3 right-3 h-8 w-8 rounded-full bg-white/80 hover:bg-white ${
isFavorite ? 'text-red-500' : 'text-gray-400' isFavorite ? 'text-red-500' : 'text-gray-400'
}`} }`}
onClick={(e) => { onClick={handleFavoriteClick}
e.preventDefault(); disabled={favoriteLoading}
e.stopPropagation();
onFavorite?.(id);
}}
> >
{favoriteLoading ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Heart className={`h-4 w-4 ${isFavorite ? 'fill-current' : ''}`} /> <Heart className={`h-4 w-4 ${isFavorite ? 'fill-current' : ''}`} />
)}
</Button> </Button>
</div> </div>
</Link> </Link>
@@ -88,7 +145,7 @@ const ProductCard = ({
</p> </p>
{/* Product Name */} {/* Product Name */}
<Link to={`/products/${id}`}> <Link to={`/products/${id}`} onClick={handleProductClick}>
<h3 className="font-semibold text-gray-900 line-clamp-2 hover:text-primary transition-colors mb-2"> <h3 className="font-semibold text-gray-900 line-clamp-2 hover:text-primary transition-colors mb-2">
{name} {name}
</h3> </h3>
@@ -124,27 +181,31 @@ const ProductCard = ({
</div> </div>
)} )}
{/* Price */} {/* Price - only show if we have price data */}
{(price != null || salePrice != null || priceRange != null) && (
<div className="flex items-baseline gap-2 mb-3"> <div className="flex items-baseline gap-2 mb-3">
{onSale && salePrice ? ( {onSale && salePrice ? (
<> <>
<span className="text-lg font-bold text-pink-600"> <span className="text-lg font-bold text-pink-600">
${salePrice.toFixed(2)} ${salePrice.toFixed(2)}
</span> </span>
{price && (
<span className="text-sm text-gray-400 line-through"> <span className="text-sm text-gray-400 line-through">
${price.toFixed(2)} ${price.toFixed(2)}
</span> </span>
)}
</> </>
) : priceRange ? ( ) : priceRange && priceRange.min != null && priceRange.max != null ? (
<span className="text-lg font-bold text-gray-900"> <span className="text-lg font-bold text-gray-900">
${priceRange.min.toFixed(2)} - ${priceRange.max.toFixed(2)} ${priceRange.min.toFixed(2)} - ${priceRange.max.toFixed(2)}
</span> </span>
) : ( ) : price != null ? (
<span className="text-lg font-bold text-gray-900"> <span className="text-lg font-bold text-gray-900">
${price.toFixed(2)} ${price.toFixed(2)}
</span> </span>
)} ) : null}
</div> </div>
)}
{/* Dispensary Count */} {/* Dispensary Count */}
{showDispensaryCount && dispensaries.length > 0 && ( {showDispensaryCount && dispensaries.length > 0 && (

View File

@@ -0,0 +1,258 @@
/**
* AuthContext - Global authentication state for Findagram
*
* Manages user login state, JWT token, and provides auth methods.
* Persists auth state in localStorage for session continuity.
*/
import React, { createContext, useContext, useState, useEffect, useCallback } from 'react';
const AuthContext = createContext(null);
const API_BASE_URL = process.env.REACT_APP_API_URL || '';
const STORAGE_KEY = 'findagram_auth';
const DOMAIN = 'findagram.co';
/**
* AuthProvider component - wrap your app with this
*/
export function AuthProvider({ children }) {
const [user, setUser] = useState(null);
const [token, setToken] = useState(null);
const [loading, setLoading] = useState(true);
const [showAuthModal, setShowAuthModal] = useState(false);
const [authModalMode, setAuthModalMode] = useState('login'); // 'login' or 'signup'
const [pendingAction, setPendingAction] = useState(null); // Action to perform after login
// Load auth state from localStorage on mount
useEffect(() => {
const stored = localStorage.getItem(STORAGE_KEY);
if (stored) {
try {
const { user: storedUser, token: storedToken } = JSON.parse(stored);
setUser(storedUser);
setToken(storedToken);
} catch (e) {
console.error('Failed to parse stored auth:', e);
localStorage.removeItem(STORAGE_KEY);
}
}
setLoading(false);
}, []);
// Save auth state to localStorage when it changes
useEffect(() => {
if (user && token) {
localStorage.setItem(STORAGE_KEY, JSON.stringify({ user, token }));
} else {
localStorage.removeItem(STORAGE_KEY);
}
}, [user, token]);
/**
* Make authenticated API request
*/
const authFetch = useCallback(async (endpoint, options = {}) => {
const url = `${API_BASE_URL}${endpoint}`;
const headers = {
'Content-Type': 'application/json',
...options.headers,
};
if (token) {
headers['Authorization'] = `Bearer ${token}`;
}
const response = await fetch(url, { ...options, headers });
// Handle 401 - token expired
if (response.status === 401) {
setUser(null);
setToken(null);
throw new Error('Session expired. Please log in again.');
}
if (!response.ok) {
const error = await response.json().catch(() => ({ error: 'Request failed' }));
throw new Error(error.error || `HTTP ${response.status}`);
}
return response.json();
}, [token]);
/**
* Register a new user
*/
const register = useCallback(async ({ firstName, lastName, email, password, phone, city, state }) => {
const response = await fetch(`${API_BASE_URL}/api/consumer/auth/register`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
firstName,
lastName,
email,
password,
phone,
city,
state,
domain: DOMAIN,
notificationPreference: phone ? 'both' : 'email',
}),
});
const data = await response.json();
if (!response.ok) {
throw new Error(data.error || 'Registration failed');
}
setUser(data.user);
setToken(data.token);
setShowAuthModal(false);
// Execute pending action if any
if (pendingAction) {
setTimeout(() => {
pendingAction();
setPendingAction(null);
}, 100);
}
return data;
}, [pendingAction]);
/**
* Login user
*/
const login = useCallback(async (email, password) => {
const response = await fetch(`${API_BASE_URL}/api/consumer/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
email,
password,
domain: DOMAIN,
}),
});
const data = await response.json();
if (!response.ok) {
throw new Error(data.error || 'Login failed');
}
setUser(data.user);
setToken(data.token);
setShowAuthModal(false);
// Execute pending action if any
if (pendingAction) {
setTimeout(() => {
pendingAction();
setPendingAction(null);
}, 100);
}
return data;
}, [pendingAction]);
/**
* Logout user
*/
const logout = useCallback(() => {
setUser(null);
setToken(null);
localStorage.removeItem(STORAGE_KEY);
}, []);
/**
* Update user profile
*/
const updateProfile = useCallback(async (updates) => {
const data = await authFetch('/api/consumer/auth/me', {
method: 'PUT',
body: JSON.stringify(updates),
});
// Refresh user data
const meData = await authFetch('/api/consumer/auth/me');
setUser(meData.user);
return data;
}, [authFetch]);
/**
* Require auth - shows modal if not logged in
* Returns true if authenticated, false if modal was shown
*
* @param {Function} action - Optional action to perform after successful auth
*/
const requireAuth = useCallback((action = null) => {
if (user && token) {
return true;
}
setPendingAction(() => action);
setAuthModalMode('login');
setShowAuthModal(true);
return false;
}, [user, token]);
/**
* Open auth modal in specific mode
*/
const openAuthModal = useCallback((mode = 'login') => {
setAuthModalMode(mode);
setShowAuthModal(true);
}, []);
/**
* Close auth modal
*/
const closeAuthModal = useCallback(() => {
setShowAuthModal(false);
setPendingAction(null);
}, []);
const value = {
// State
user,
token,
loading,
isAuthenticated: !!user && !!token,
showAuthModal,
authModalMode,
// Auth methods
register,
login,
logout,
updateProfile,
authFetch,
// Modal control
requireAuth,
openAuthModal,
closeAuthModal,
setAuthModalMode,
};
return (
<AuthContext.Provider value={value}>
{children}
</AuthContext.Provider>
);
}
/**
* Hook to use auth context
*/
export function useAuth() {
const context = useContext(AuthContext);
if (!context) {
throw new Error('useAuth must be used within an AuthProvider');
}
return context;
}
export default AuthContext;

View File

@@ -0,0 +1,314 @@
import { useState, useEffect, useCallback } from 'react';
// Default location: Phoenix, AZ (fallback if all else fails)
const DEFAULT_LOCATION = {
lat: 33.4484,
lng: -112.0740,
city: 'Phoenix',
state: 'AZ'
};
const LOCATION_STORAGE_KEY = 'findagram_location';
const SESSION_ID_KEY = 'findagram_session_id';
const API_BASE_URL = process.env.REACT_APP_API_URL || '';
/**
* Get or create session ID
*/
function getSessionId() {
let sessionId = sessionStorage.getItem(SESSION_ID_KEY);
if (!sessionId) {
sessionId = `${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
sessionStorage.setItem(SESSION_ID_KEY, sessionId);
}
return sessionId;
}
/**
* Get cached location from sessionStorage
*/
function getCachedLocation() {
try {
const cached = sessionStorage.getItem(LOCATION_STORAGE_KEY);
if (cached) {
return JSON.parse(cached);
}
} catch (err) {
console.error('Error reading cached location:', err);
}
return null;
}
/**
* Save location to sessionStorage
*/
function cacheLocation(location) {
try {
sessionStorage.setItem(LOCATION_STORAGE_KEY, JSON.stringify(location));
} catch (err) {
console.error('Error caching location:', err);
}
}
/**
* Track visitor and get location from our backend API
* This logs the visit for analytics and returns location from IP
*/
async function trackVisitorAndGetLocation() {
try {
const response = await fetch(`${API_BASE_URL}/api/v1/visitor/track`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
domain: 'findagram.co',
page_path: window.location.pathname,
session_id: getSessionId(),
referrer: document.referrer || null,
}),
});
const data = await response.json();
if (data.success && data.location) {
return {
lat: data.location.lat,
lng: data.location.lng,
city: data.location.city,
state: data.location.state,
stateCode: data.location.stateCode,
source: 'api'
};
}
} catch (err) {
console.error('Visitor tracking error:', err);
}
return null;
}
/**
* Custom hook for getting user's geolocation
*
* @param {Object} options
* @param {boolean} options.autoRequest - Whether to request location automatically on mount
* @param {boolean} options.useIPFallback - Whether to use IP geolocation as fallback (default: true)
* @param {Object} options.defaultLocation - Default location if all methods fail
* @returns {Object} { location, loading, error, requestLocation, hasPermission }
*/
export function useGeolocation(options = {}) {
const {
autoRequest = false,
useIPFallback = true,
defaultLocation = DEFAULT_LOCATION
} = options;
const [location, setLocation] = useState(null);
const [loading, setLoading] = useState(false);
const [error, setError] = useState(null);
const [hasPermission, setHasPermission] = useState(null);
const [locationSource, setLocationSource] = useState(null); // 'gps', 'ip', or 'default'
// Try IP geolocation first (no permission needed)
const getIPLocation = useCallback(async () => {
if (!useIPFallback) return null;
const ipLoc = await getLocationFromIP();
if (ipLoc) {
setLocation(ipLoc);
setLocationSource('ip');
return ipLoc;
}
return null;
}, [useIPFallback]);
// Request precise GPS location (requires permission)
const requestLocation = useCallback(async () => {
setLoading(true);
setError(null);
// First try browser geolocation
if (navigator.geolocation) {
return new Promise(async (resolve) => {
navigator.geolocation.getCurrentPosition(
(position) => {
const { latitude, longitude } = position.coords;
const loc = { lat: latitude, lng: longitude, source: 'gps' };
setLocation(loc);
setLocationSource('gps');
setHasPermission(true);
setLoading(false);
resolve(loc);
},
async (err) => {
console.error('Geolocation error:', err);
if (err.code === err.PERMISSION_DENIED) {
setHasPermission(false);
}
// Fall back to IP geolocation
if (useIPFallback) {
const ipLoc = await getLocationFromIP();
if (ipLoc) {
setLocation(ipLoc);
setLocationSource('ip');
setLoading(false);
resolve(ipLoc);
return;
}
}
// Last resort: default location
setError('Unable to determine location');
setLocation(defaultLocation);
setLocationSource('default');
setLoading(false);
resolve(defaultLocation);
},
{
enableHighAccuracy: false,
timeout: 5000,
maximumAge: 600000 // Cache for 10 minutes
}
);
});
}
// No browser geolocation, try IP
if (useIPFallback) {
const ipLoc = await getLocationFromIP();
if (ipLoc) {
setLocation(ipLoc);
setLocationSource('ip');
setLoading(false);
return ipLoc;
}
}
// Fallback to default
setLocation(defaultLocation);
setLocationSource('default');
setLoading(false);
return defaultLocation;
}, [defaultLocation, useIPFallback]);
// Auto-request location on mount if enabled
useEffect(() => {
if (autoRequest) {
const init = async () => {
// Check for cached location first
const cached = getCachedLocation();
if (cached) {
setLocation(cached);
setLocationSource(cached.source || 'api');
setLoading(false);
return;
}
setLoading(true);
// Track visitor and get location from our backend API
const apiLoc = await trackVisitorAndGetLocation();
if (apiLoc) {
setLocation(apiLoc);
setLocationSource('api');
cacheLocation(apiLoc); // Save for session
} else {
// Fallback to default
setLocation(defaultLocation);
setLocationSource('default');
}
setLoading(false);
};
init();
}
}, [autoRequest, defaultLocation]);
return {
location,
loading,
error,
requestLocation,
hasPermission,
locationSource,
isDefault: locationSource === 'default',
isFromIP: locationSource === 'ip',
isFromGPS: locationSource === 'gps'
};
}
/**
* Calculate distance between two points using Haversine formula
*
* @param {number} lat1 - First point latitude
* @param {number} lng1 - First point longitude
* @param {number} lat2 - Second point latitude
* @param {number} lng2 - Second point longitude
* @returns {number} Distance in miles
*/
export function calculateDistance(lat1, lng1, lat2, lng2) {
const R = 3959; // Earth's radius in miles
const dLat = toRad(lat2 - lat1);
const dLng = toRad(lng2 - lng1);
const a =
Math.sin(dLat / 2) * Math.sin(dLat / 2) +
Math.cos(toRad(lat1)) * Math.cos(toRad(lat2)) *
Math.sin(dLng / 2) * Math.sin(dLng / 2);
const c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1 - a));
return R * c;
}
function toRad(deg) {
return deg * (Math.PI / 180);
}
/**
* Sort items by distance from a location
*
* @param {Array} items - Array of items with location data
* @param {Object} userLocation - User's location { lat, lng }
* @param {Function} getItemLocation - Function to extract lat/lng from item
* @returns {Array} Items sorted by distance with distance property added
*/
export function sortByDistance(items, userLocation, getItemLocation = (item) => item.location) {
if (!userLocation || !items?.length) return items;
return items
.map(item => {
const itemLoc = getItemLocation(item);
if (!itemLoc?.latitude || !itemLoc?.longitude) {
return { ...item, distance: null };
}
const distance = calculateDistance(
userLocation.lat,
userLocation.lng,
itemLoc.latitude,
itemLoc.longitude
);
return { ...item, distance: Math.round(distance * 10) / 10 };
})
.sort((a, b) => {
if (a.distance === null) return 1;
if (b.distance === null) return -1;
return a.distance - b.distance;
});
}
/**
* Filter items within a radius from user location
*
* @param {Array} items - Array of items with location data
* @param {Object} userLocation - User's location { lat, lng }
* @param {number} radiusMiles - Max distance in miles
* @param {Function} getItemLocation - Function to extract lat/lng from item
* @returns {Array} Items within radius, sorted by distance
*/
export function filterByRadius(items, userLocation, radiusMiles = 50, getItemLocation = (item) => item.location) {
const sorted = sortByDistance(items, userLocation, getItemLocation);
return sorted.filter(item => item.distance !== null && item.distance <= radiusMiles);
}
export default useGeolocation;

View File

@@ -0,0 +1,363 @@
/**
* localStorage helpers for user data persistence
*
* Manages favorites, price alerts, and saved searches without requiring authentication.
* All data is stored locally in the browser.
*/
const STORAGE_KEYS = {
FAVORITES: 'findagram_favorites',
ALERTS: 'findagram_alerts',
SAVED_SEARCHES: 'findagram_saved_searches',
};
// ============================================================
// FAVORITES
// ============================================================
/**
* Get all favorite product IDs
* @returns {number[]} Array of product IDs
*/
export function getFavorites() {
try {
const data = localStorage.getItem(STORAGE_KEYS.FAVORITES);
return data ? JSON.parse(data) : [];
} catch (e) {
console.error('Error reading favorites:', e);
return [];
}
}
/**
* Check if a product is favorited
* @param {number} productId
* @returns {boolean}
*/
export function isFavorite(productId) {
const favorites = getFavorites();
return favorites.includes(productId);
}
/**
* Add a product to favorites
* @param {number} productId
*/
export function addFavorite(productId) {
const favorites = getFavorites();
if (!favorites.includes(productId)) {
favorites.push(productId);
localStorage.setItem(STORAGE_KEYS.FAVORITES, JSON.stringify(favorites));
}
}
/**
* Remove a product from favorites
* @param {number} productId
*/
export function removeFavorite(productId) {
const favorites = getFavorites();
const updated = favorites.filter(id => id !== productId);
localStorage.setItem(STORAGE_KEYS.FAVORITES, JSON.stringify(updated));
}
/**
* Toggle a product's favorite status
* @param {number} productId
* @returns {boolean} New favorite status
*/
export function toggleFavorite(productId) {
if (isFavorite(productId)) {
removeFavorite(productId);
return false;
} else {
addFavorite(productId);
return true;
}
}
/**
* Clear all favorites
*/
export function clearFavorites() {
localStorage.setItem(STORAGE_KEYS.FAVORITES, JSON.stringify([]));
}
// ============================================================
// PRICE ALERTS
// ============================================================
/**
* @typedef {Object} PriceAlert
* @property {string} id - Unique alert ID
* @property {number} productId - Product ID to track
* @property {string} productName - Product name (for display when offline)
* @property {string} productImage - Product image URL
* @property {string} brandName - Brand name
* @property {number} targetPrice - Target price to alert at
* @property {number} originalPrice - Price when alert was created
* @property {boolean} active - Whether alert is active
* @property {string} createdAt - ISO date string
*/
/**
* Get all price alerts
* @returns {PriceAlert[]}
*/
export function getAlerts() {
try {
const data = localStorage.getItem(STORAGE_KEYS.ALERTS);
return data ? JSON.parse(data) : [];
} catch (e) {
console.error('Error reading alerts:', e);
return [];
}
}
/**
* Get alert for a specific product
* @param {number} productId
* @returns {PriceAlert|null}
*/
export function getAlertForProduct(productId) {
const alerts = getAlerts();
return alerts.find(a => a.productId === productId) || null;
}
/**
* Create a new price alert
* @param {Object} params
* @param {number} params.productId
* @param {string} params.productName
* @param {string} params.productImage
* @param {string} params.brandName
* @param {number} params.targetPrice
* @param {number} params.originalPrice
* @returns {PriceAlert}
*/
export function createAlert({ productId, productName, productImage, brandName, targetPrice, originalPrice }) {
const alerts = getAlerts();
// Check if alert already exists for this product
const existingIndex = alerts.findIndex(a => a.productId === productId);
const alert = {
id: existingIndex >= 0 ? alerts[existingIndex].id : `alert_${Date.now()}`,
productId,
productName,
productImage,
brandName,
targetPrice,
originalPrice,
active: true,
createdAt: existingIndex >= 0 ? alerts[existingIndex].createdAt : new Date().toISOString(),
};
if (existingIndex >= 0) {
alerts[existingIndex] = alert;
} else {
alerts.push(alert);
}
localStorage.setItem(STORAGE_KEYS.ALERTS, JSON.stringify(alerts));
return alert;
}
/**
* Update an existing alert
* @param {string} alertId
* @param {Partial<PriceAlert>} updates
*/
export function updateAlert(alertId, updates) {
const alerts = getAlerts();
const index = alerts.findIndex(a => a.id === alertId);
if (index >= 0) {
alerts[index] = { ...alerts[index], ...updates };
localStorage.setItem(STORAGE_KEYS.ALERTS, JSON.stringify(alerts));
}
}
/**
* Toggle alert active status
* @param {string} alertId
* @returns {boolean} New active status
*/
export function toggleAlertActive(alertId) {
const alerts = getAlerts();
const alert = alerts.find(a => a.id === alertId);
if (alert) {
alert.active = !alert.active;
localStorage.setItem(STORAGE_KEYS.ALERTS, JSON.stringify(alerts));
return alert.active;
}
return false;
}
/**
* Delete an alert
* @param {string} alertId
*/
export function deleteAlert(alertId) {
const alerts = getAlerts();
const updated = alerts.filter(a => a.id !== alertId);
localStorage.setItem(STORAGE_KEYS.ALERTS, JSON.stringify(updated));
}
/**
* Clear all alerts
*/
export function clearAlerts() {
localStorage.setItem(STORAGE_KEYS.ALERTS, JSON.stringify([]));
}
// ============================================================
// SAVED SEARCHES
// ============================================================
/**
* @typedef {Object} SavedSearch
* @property {string} id - Unique search ID
* @property {string} name - User-defined name for the search
* @property {Object} filters - Search filter parameters
* @property {string} [filters.search] - Search term
* @property {string} [filters.type] - Category type
* @property {string} [filters.brandName] - Brand filter
* @property {string} [filters.strainType] - Strain type filter
* @property {number} [filters.priceMax] - Max price filter
* @property {number} [filters.thcMin] - Min THC filter
* @property {string} createdAt - ISO date string
*/
/**
* Get all saved searches
* @returns {SavedSearch[]}
*/
export function getSavedSearches() {
try {
const data = localStorage.getItem(STORAGE_KEYS.SAVED_SEARCHES);
return data ? JSON.parse(data) : [];
} catch (e) {
console.error('Error reading saved searches:', e);
return [];
}
}
/**
* Create a new saved search
* @param {Object} params
* @param {string} params.name - Display name for the search
* @param {Object} params.filters - Search filters
* @returns {SavedSearch}
*/
export function createSavedSearch({ name, filters }) {
const searches = getSavedSearches();
const search = {
id: `search_${Date.now()}`,
name,
filters,
createdAt: new Date().toISOString(),
};
searches.push(search);
localStorage.setItem(STORAGE_KEYS.SAVED_SEARCHES, JSON.stringify(searches));
return search;
}
/**
* Update a saved search
* @param {string} searchId
* @param {Partial<SavedSearch>} updates
*/
export function updateSavedSearch(searchId, updates) {
const searches = getSavedSearches();
const index = searches.findIndex(s => s.id === searchId);
if (index >= 0) {
searches[index] = { ...searches[index], ...updates };
localStorage.setItem(STORAGE_KEYS.SAVED_SEARCHES, JSON.stringify(searches));
}
}
/**
* Delete a saved search
* @param {string} searchId
*/
export function deleteSavedSearch(searchId) {
const searches = getSavedSearches();
const updated = searches.filter(s => s.id !== searchId);
localStorage.setItem(STORAGE_KEYS.SAVED_SEARCHES, JSON.stringify(updated));
}
/**
* Clear all saved searches
*/
export function clearSavedSearches() {
localStorage.setItem(STORAGE_KEYS.SAVED_SEARCHES, JSON.stringify([]));
}
// ============================================================
// UTILITY FUNCTIONS
// ============================================================
/**
* Build a URL with search params from filters
* @param {Object} filters
* @returns {string}
*/
export function buildSearchUrl(filters) {
const params = new URLSearchParams();
Object.entries(filters).forEach(([key, value]) => {
if (value !== undefined && value !== null && value !== '') {
params.set(key, value);
}
});
return `/products?${params.toString()}`;
}
/**
* Generate a name for a search based on its filters
* @param {Object} filters
* @returns {string}
*/
export function generateSearchName(filters) {
const parts = [];
if (filters.search) parts.push(`"${filters.search}"`);
if (filters.type) parts.push(filters.type);
if (filters.brandName) parts.push(filters.brandName);
if (filters.strainType) parts.push(filters.strainType);
if (filters.priceMax) parts.push(`Under $${filters.priceMax}`);
if (filters.thcMin) parts.push(`${filters.thcMin}%+ THC`);
return parts.length > 0 ? parts.join(' - ') : 'All Products';
}
// Default export
const storage = {
// Favorites
getFavorites,
isFavorite,
addFavorite,
removeFavorite,
toggleFavorite,
clearFavorites,
// Alerts
getAlerts,
getAlertForProduct,
createAlert,
updateAlert,
toggleAlertActive,
deleteAlert,
clearAlerts,
// Saved Searches
getSavedSearches,
createSavedSearch,
updateSavedSearch,
deleteSavedSearch,
clearSavedSearches,
// Utilities
buildSearchUrl,
generateSearchName,
};
export default storage;

View File

@@ -1,28 +1,90 @@
import React, { useState } from 'react'; import React, { useState, useEffect } from 'react';
import { Link } from 'react-router-dom'; import { Link, useNavigate } from 'react-router-dom';
import { Button } from '../../components/ui/button'; import { Button } from '../../components/ui/button';
import { Card, CardContent } from '../../components/ui/card'; import { Card, CardContent } from '../../components/ui/card';
import { Badge } from '../../components/ui/badge'; import { Badge } from '../../components/ui/badge';
import { mockAlerts, mockProducts } from '../../mockData'; import { useAuth } from '../../context/AuthContext';
import { Bell, Trash2, Pause, Play, TrendingDown } from 'lucide-react'; import { getAlerts, toggleAlert, deleteAlert } from '../../api/consumer';
import { Bell, Trash2, Pause, Play, TrendingDown, Loader2 } from 'lucide-react';
const Alerts = () => { const Alerts = () => {
const [alerts, setAlerts] = useState(mockAlerts); const { isAuthenticated, authFetch, requireAuth } = useAuth();
const navigate = useNavigate();
const toggleAlert = (alertId) => { const [alerts, setAlerts] = useState([]);
setAlerts((prev) => const [loading, setLoading] = useState(true);
prev.map((alert) => const [error, setError] = useState(null);
alert.id === alertId ? { ...alert, active: !alert.active } : alert const [togglingId, setTogglingId] = useState(null);
) const [deletingId, setDeletingId] = useState(null);
// Redirect to home if not authenticated
useEffect(() => {
if (!isAuthenticated) {
requireAuth(() => navigate('/dashboard/alerts'));
}
}, [isAuthenticated, requireAuth, navigate]);
// Fetch alerts
useEffect(() => {
if (!isAuthenticated) return;
const fetchAlerts = async () => {
setLoading(true);
setError(null);
try {
const data = await getAlerts(authFetch);
setAlerts(data.alerts || []);
} catch (err) {
setError(err.message);
} finally {
setLoading(false);
}
};
fetchAlerts();
}, [isAuthenticated, authFetch]);
const handleToggleAlert = async (alertId) => {
setTogglingId(alertId);
try {
const result = await toggleAlert(authFetch, alertId);
setAlerts(prev =>
prev.map(a => a.id === alertId ? { ...a, isActive: result.isActive } : a)
); );
} catch (err) {
setError(err.message);
} finally {
setTogglingId(null);
}
}; };
const deleteAlert = (alertId) => { const handleDeleteAlert = async (alertId) => {
setAlerts((prev) => prev.filter((alert) => alert.id !== alertId)); setDeletingId(alertId);
try {
await deleteAlert(authFetch, alertId);
setAlerts(prev => prev.filter(a => a.id !== alertId));
} catch (err) {
setError(err.message);
} finally {
setDeletingId(null);
}
}; };
const activeAlerts = alerts.filter((a) => a.active); if (!isAuthenticated) {
const pausedAlerts = alerts.filter((a) => !a.active); return null;
}
if (loading) {
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
);
}
const activeAlerts = alerts.filter(a => a.isActive);
const pausedAlerts = alerts.filter(a => !a.isActive);
return ( return (
<div className="min-h-screen bg-gray-50"> <div className="min-h-screen bg-gray-50">
@@ -50,6 +112,12 @@ const Alerts = () => {
</section> </section>
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
{error && (
<div className="mb-6 p-4 bg-red-50 border border-red-200 rounded-lg text-red-600">
{error}
</div>
)}
{alerts.length > 0 ? ( {alerts.length > 0 ? (
<div className="space-y-8"> <div className="space-y-8">
{/* Active Alerts */} {/* Active Alerts */}
@@ -61,40 +129,46 @@ const Alerts = () => {
</h2> </h2>
<div className="space-y-4"> <div className="space-y-4">
{activeAlerts.map((alert) => { {activeAlerts.map((alert) => {
const product = mockProducts.find((p) => p.id === alert.productId); const isTriggered = alert.isTriggered;
const priceDiff = product ? product.price - alert.targetPrice : 0;
const isTriggered = priceDiff <= 0;
return ( return (
<Card key={alert.id} className={isTriggered ? 'border-green-500 bg-green-50' : ''}> <Card key={alert.id} className={isTriggered ? 'border-green-500 bg-green-50' : ''}>
<CardContent className="p-4"> <CardContent className="p-4">
<div className="flex items-center gap-4"> <div className="flex items-center gap-4">
<Link to={`/products/${product?.id}`}> <Link to={`/products/${alert.productId}`}>
<img <img
src={product?.image || '/placeholder-product.jpg'} src={alert.productImage || '/placeholder-product.jpg'}
alt={product?.name} alt={alert.productName}
className="w-16 h-16 rounded-lg object-cover" className="w-16 h-16 rounded-lg object-cover"
/> />
</Link> </Link>
<div className="flex-1 min-w-0"> <div className="flex-1 min-w-0">
<Link <Link
to={`/products/${product?.id}`} to={`/products/${alert.productId}`}
className="font-medium text-gray-900 hover:text-primary truncate block" className="font-medium text-gray-900 hover:text-primary truncate block"
> >
{product?.name} {alert.productName}
</Link> </Link>
<p className="text-sm text-gray-500">{product?.brand}</p> <p className="text-sm text-gray-500">{alert.productBrand}</p>
<div className="flex items-center gap-4 mt-1"> <div className="flex items-center gap-4 mt-1">
<span className="text-sm"> <span className="text-sm">
Current: <span className="font-medium">${product?.price.toFixed(2)}</span> Current:{' '}
<span className="font-medium">
{alert.currentPrice
? `$${parseFloat(alert.currentPrice).toFixed(2)}`
: 'N/A'}
</span>
</span> </span>
<span className="text-sm"> <span className="text-sm">
Target: <span className="font-medium text-primary">${alert.targetPrice.toFixed(2)}</span> Target:{' '}
<span className="font-medium text-primary">
${parseFloat(alert.targetPrice).toFixed(2)}
</span>
</span> </span>
</div> </div>
</div> </div>
{isTriggered && ( {isTriggered && (
<Badge variant="success" className="flex items-center gap-1"> <Badge className="bg-green-500 text-white flex items-center gap-1">
<TrendingDown className="h-3 w-3" /> <TrendingDown className="h-3 w-3" />
Price Dropped! Price Dropped!
</Badge> </Badge>
@@ -103,19 +177,29 @@ const Alerts = () => {
<Button <Button
variant="ghost" variant="ghost"
size="icon" size="icon"
onClick={() => toggleAlert(alert.id)} onClick={() => handleToggleAlert(alert.id)}
disabled={togglingId === alert.id}
title="Pause alert" title="Pause alert"
> >
{togglingId === alert.id ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Pause className="h-4 w-4" /> <Pause className="h-4 w-4" />
)}
</Button> </Button>
<Button <Button
variant="ghost" variant="ghost"
size="icon" size="icon"
onClick={() => deleteAlert(alert.id)} onClick={() => handleDeleteAlert(alert.id)}
disabled={deletingId === alert.id}
className="text-red-600 hover:text-red-700 hover:bg-red-50" className="text-red-600 hover:text-red-700 hover:bg-red-50"
title="Delete alert" title="Delete alert"
> >
{deletingId === alert.id ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Trash2 className="h-4 w-4" /> <Trash2 className="h-4 w-4" />
)}
</Button> </Button>
</div> </div>
</div> </div>
@@ -135,25 +219,25 @@ const Alerts = () => {
Paused Alerts ({pausedAlerts.length}) Paused Alerts ({pausedAlerts.length})
</h2> </h2>
<div className="space-y-4"> <div className="space-y-4">
{pausedAlerts.map((alert) => { {pausedAlerts.map((alert) => (
const product = mockProducts.find((p) => p.id === alert.productId);
return (
<Card key={alert.id} className="opacity-75"> <Card key={alert.id} className="opacity-75">
<CardContent className="p-4"> <CardContent className="p-4">
<div className="flex items-center gap-4"> <div className="flex items-center gap-4">
<img <img
src={product?.image || '/placeholder-product.jpg'} src={alert.productImage || '/placeholder-product.jpg'}
alt={product?.name} alt={alert.productName}
className="w-16 h-16 rounded-lg object-cover grayscale" className="w-16 h-16 rounded-lg object-cover grayscale"
/> />
<div className="flex-1 min-w-0"> <div className="flex-1 min-w-0">
<p className="font-medium text-gray-900 truncate"> <p className="font-medium text-gray-900 truncate">
{product?.name} {alert.productName}
</p> </p>
<p className="text-sm text-gray-500">{product?.brand}</p> <p className="text-sm text-gray-500">{alert.productBrand}</p>
<span className="text-sm"> <span className="text-sm">
Target: <span className="font-medium">${alert.targetPrice.toFixed(2)}</span> Target:{' '}
<span className="font-medium">
${parseFloat(alert.targetPrice).toFixed(2)}
</span>
</span> </span>
</div> </div>
<Badge variant="secondary">Paused</Badge> <Badge variant="secondary">Paused</Badge>
@@ -161,26 +245,35 @@ const Alerts = () => {
<Button <Button
variant="ghost" variant="ghost"
size="icon" size="icon"
onClick={() => toggleAlert(alert.id)} onClick={() => handleToggleAlert(alert.id)}
disabled={togglingId === alert.id}
title="Resume alert" title="Resume alert"
> >
{togglingId === alert.id ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Play className="h-4 w-4" /> <Play className="h-4 w-4" />
)}
</Button> </Button>
<Button <Button
variant="ghost" variant="ghost"
size="icon" size="icon"
onClick={() => deleteAlert(alert.id)} onClick={() => handleDeleteAlert(alert.id)}
disabled={deletingId === alert.id}
className="text-red-600 hover:text-red-700 hover:bg-red-50" className="text-red-600 hover:text-red-700 hover:bg-red-50"
title="Delete alert" title="Delete alert"
> >
{deletingId === alert.id ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Trash2 className="h-4 w-4" /> <Trash2 className="h-4 w-4" />
)}
</Button> </Button>
</div> </div>
</div> </div>
</CardContent> </CardContent>
</Card> </Card>
); ))}
})}
</div> </div>
</div> </div>
)} )}

View File

@@ -1,14 +1,10 @@
import React from 'react'; import React, { useState, useEffect } from 'react';
import { Link } from 'react-router-dom'; import { Link, useNavigate } from 'react-router-dom';
import { Card, CardContent, CardHeader, CardTitle } from '../../components/ui/card'; import { Card, CardContent, CardHeader, CardTitle } from '../../components/ui/card';
import { Badge } from '../../components/ui/badge'; import { Badge } from '../../components/ui/badge';
import { Button } from '../../components/ui/button'; import { Button } from '../../components/ui/button';
import { import { useAuth } from '../../context/AuthContext';
mockFavorites, import { getFavorites, getAlerts, getAlertStats, getSavedSearches } from '../../api/consumer';
mockAlerts,
mockSavedSearches,
mockProducts,
} from '../../mockData';
import { import {
Heart, Heart,
Bell, Bell,
@@ -17,23 +13,80 @@ import {
ChevronRight, ChevronRight,
TrendingDown, TrendingDown,
Search, Search,
Loader2,
} from 'lucide-react'; } from 'lucide-react';
const Dashboard = () => { const Dashboard = () => {
// Get favorite products const { isAuthenticated, user, authFetch, requireAuth } = useAuth();
const favoriteProducts = mockProducts.filter((p) => const navigate = useNavigate();
mockFavorites.includes(p.id)
);
// Get active alerts const [favorites, setFavorites] = useState([]);
const activeAlerts = mockAlerts.filter((a) => a.active); const [alerts, setAlerts] = useState([]);
const [alertStats, setAlertStats] = useState({ active: 0, triggeredThisWeek: 0 });
const [savedSearches, setSavedSearches] = useState([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState(null);
// Redirect to home if not authenticated
useEffect(() => {
if (!isAuthenticated) {
requireAuth(() => navigate('/dashboard'));
}
}, [isAuthenticated, requireAuth, navigate]);
// Fetch all dashboard data
useEffect(() => {
if (!isAuthenticated) return;
const fetchData = async () => {
setLoading(true);
setError(null);
try {
const [favData, alertData, statsData, searchData] = await Promise.all([
getFavorites(authFetch).catch(() => ({ favorites: [] })),
getAlerts(authFetch).catch(() => ({ alerts: [] })),
getAlertStats(authFetch).catch(() => ({ active: 0, triggeredThisWeek: 0 })),
getSavedSearches(authFetch).catch(() => ({ savedSearches: [] })),
]);
setFavorites(favData.favorites || []);
setAlerts(alertData.alerts || []);
setAlertStats(statsData);
setSavedSearches(searchData.savedSearches || []);
} catch (err) {
setError(err.message);
} finally {
setLoading(false);
}
};
fetchData();
}, [isAuthenticated, authFetch]);
if (!isAuthenticated) {
return null; // Will redirect via useEffect
}
if (loading) {
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
);
}
const activeAlerts = alerts.filter(a => a.isActive);
const triggeredAlerts = alerts.filter(a => a.isTriggered);
return ( return (
<div className="min-h-screen bg-gray-50"> <div className="min-h-screen bg-gray-50">
{/* Header */} {/* Header */}
<section className="bg-white border-b"> <section className="bg-white border-b">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
<h1 className="text-3xl font-bold text-gray-900">Dashboard</h1> <h1 className="text-3xl font-bold text-gray-900">
Welcome back, {user?.firstName || 'there'}!
</h1>
<p className="text-gray-600 mt-2"> <p className="text-gray-600 mt-2">
Manage your favorites, alerts, and saved searches Manage your favorites, alerts, and saved searches
</p> </p>
@@ -41,6 +94,12 @@ const Dashboard = () => {
</section> </section>
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
{error && (
<div className="mb-6 p-4 bg-red-50 border border-red-200 rounded-lg text-red-600">
{error}
</div>
)}
{/* Quick Stats */} {/* Quick Stats */}
<div className="grid grid-cols-1 md:grid-cols-4 gap-4 mb-8"> <div className="grid grid-cols-1 md:grid-cols-4 gap-4 mb-8">
<Card> <Card>
@@ -48,7 +107,7 @@ const Dashboard = () => {
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div> <div>
<p className="text-sm text-gray-500">Favorites</p> <p className="text-sm text-gray-500">Favorites</p>
<p className="text-2xl font-bold">{mockFavorites.length}</p> <p className="text-2xl font-bold">{favorites.length}</p>
</div> </div>
<Heart className="h-8 w-8 text-red-500" /> <Heart className="h-8 w-8 text-red-500" />
</div> </div>
@@ -60,7 +119,7 @@ const Dashboard = () => {
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div> <div>
<p className="text-sm text-gray-500">Active Alerts</p> <p className="text-sm text-gray-500">Active Alerts</p>
<p className="text-2xl font-bold">{activeAlerts.length}</p> <p className="text-2xl font-bold">{alertStats.active}</p>
</div> </div>
<Bell className="h-8 w-8 text-primary" /> <Bell className="h-8 w-8 text-primary" />
</div> </div>
@@ -72,7 +131,7 @@ const Dashboard = () => {
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div> <div>
<p className="text-sm text-gray-500">Saved Searches</p> <p className="text-sm text-gray-500">Saved Searches</p>
<p className="text-2xl font-bold">{mockSavedSearches.length}</p> <p className="text-2xl font-bold">{savedSearches.length}</p>
</div> </div>
<Bookmark className="h-8 w-8 text-indigo-500" /> <Bookmark className="h-8 w-8 text-indigo-500" />
</div> </div>
@@ -84,7 +143,7 @@ const Dashboard = () => {
<div className="flex items-center justify-between"> <div className="flex items-center justify-between">
<div> <div>
<p className="text-sm text-gray-500">Price Drops</p> <p className="text-sm text-gray-500">Price Drops</p>
<p className="text-2xl font-bold">3</p> <p className="text-2xl font-bold">{triggeredAlerts.length}</p>
</div> </div>
<TrendingDown className="h-8 w-8 text-green-500" /> <TrendingDown className="h-8 w-8 text-green-500" />
</div> </div>
@@ -108,28 +167,39 @@ const Dashboard = () => {
</Link> </Link>
</CardHeader> </CardHeader>
<CardContent> <CardContent>
{favoriteProducts.length > 0 ? ( {favorites.length > 0 ? (
<div className="space-y-4"> <div className="space-y-4">
{favoriteProducts.slice(0, 3).map((product) => ( {favorites.slice(0, 3).map((fav) => (
<Link <Link
key={product.id} key={fav.id}
to={`/products/${product.id}`} to={`/products/${fav.productId}`}
className="flex items-center gap-4 p-3 rounded-lg hover:bg-gray-50 transition-colors" className="flex items-center gap-4 p-3 rounded-lg hover:bg-gray-50 transition-colors"
> >
<img <img
src={product.image || '/placeholder-product.jpg'} src={fav.imageUrl || '/placeholder-product.jpg'}
alt={product.name} alt={fav.savedName}
className="w-12 h-12 rounded-lg object-cover" className="w-12 h-12 rounded-lg object-cover"
/> />
<div className="flex-1 min-w-0"> <div className="flex-1 min-w-0">
<p className="font-medium text-gray-900 truncate"> <p className="font-medium text-gray-900 truncate">
{product.name} {fav.currentName || fav.savedName}
</p> </p>
<p className="text-sm text-gray-500">{product.brand}</p> <p className="text-sm text-gray-500">{fav.currentBrand || fav.savedBrand}</p>
</div> </div>
<div className="text-right">
{fav.currentPrice ? (
<p className="font-bold text-primary"> <p className="font-bold text-primary">
${product.price.toFixed(2)} ${parseFloat(fav.currentPrice).toFixed(2)}
</p> </p>
) : (
<p className="text-sm text-gray-400">No price</p>
)}
{fav.priceDrop && (
<Badge variant="success" className="text-xs">
Price dropped!
</Badge>
)}
</div>
</Link> </Link>
))} ))}
</div> </div>
@@ -160,34 +230,40 @@ const Dashboard = () => {
</Link> </Link>
</CardHeader> </CardHeader>
<CardContent> <CardContent>
{mockAlerts.length > 0 ? ( {alerts.length > 0 ? (
<div className="space-y-4"> <div className="space-y-4">
{mockAlerts.slice(0, 3).map((alert) => { {alerts.slice(0, 3).map((alert) => (
const product = mockProducts.find((p) => p.id === alert.productId);
return (
<div <div
key={alert.id} key={alert.id}
className="flex items-center gap-4 p-3 rounded-lg bg-gray-50" className={`flex items-center gap-4 p-3 rounded-lg ${
alert.isTriggered ? 'bg-green-50' : 'bg-gray-50'
}`}
> >
<img <img
src={product?.image || '/placeholder-product.jpg'} src={alert.productImage || '/placeholder-product.jpg'}
alt={product?.name} alt={alert.productName}
className="w-12 h-12 rounded-lg object-cover" className="w-12 h-12 rounded-lg object-cover"
/> />
<div className="flex-1 min-w-0"> <div className="flex-1 min-w-0">
<p className="font-medium text-gray-900 truncate"> <p className="font-medium text-gray-900 truncate">
{product?.name} {alert.productName}
</p> </p>
<p className="text-sm text-gray-500"> <p className="text-sm text-gray-500">
Alert at ${alert.targetPrice.toFixed(2)} Target: ${parseFloat(alert.targetPrice).toFixed(2)}
</p> </p>
</div> </div>
<Badge variant={alert.active ? 'default' : 'secondary'}> {alert.isTriggered ? (
{alert.active ? 'Active' : 'Paused'} <Badge variant="success" className="flex items-center gap-1">
<TrendingDown className="h-3 w-3" />
Triggered!
</Badge> </Badge>
) : (
<Badge variant={alert.isActive ? 'default' : 'secondary'}>
{alert.isActive ? 'Active' : 'Paused'}
</Badge>
)}
</div> </div>
); ))}
})}
</div> </div>
) : ( ) : (
<div className="text-center py-8"> <div className="text-center py-8">
@@ -216,22 +292,41 @@ const Dashboard = () => {
</Link> </Link>
</CardHeader> </CardHeader>
<CardContent> <CardContent>
{mockSavedSearches.length > 0 ? ( {savedSearches.length > 0 ? (
<div className="space-y-3"> <div className="space-y-3">
{mockSavedSearches.slice(0, 4).map((search) => ( {savedSearches.slice(0, 4).map((search) => {
// Build search URL
const params = new URLSearchParams();
if (search.query) params.set('search', search.query);
if (search.category) params.set('type', search.category);
if (search.brand) params.set('brandName', search.brand);
const searchUrl = `/products?${params.toString()}`;
return (
<Link <Link
key={search.id} key={search.id}
to={`/products?${new URLSearchParams(search.filters).toString()}`} to={searchUrl}
className="flex items-center gap-3 p-3 rounded-lg hover:bg-gray-50 transition-colors" className="flex items-center gap-3 p-3 rounded-lg hover:bg-gray-50 transition-colors"
> >
<Search className="h-5 w-5 text-gray-400" /> <Search className="h-5 w-5 text-gray-400" />
<div className="flex-1"> <div className="flex-1">
<p className="font-medium text-gray-900">{search.name}</p> <p className="font-medium text-gray-900">{search.name}</p>
<p className="text-sm text-gray-500">{search.resultCount} results</p> <div className="flex flex-wrap gap-1 mt-1">
{search.category && (
<Badge variant="secondary" className="text-xs">{search.category}</Badge>
)}
{search.brand && (
<Badge variant="outline" className="text-xs">{search.brand}</Badge>
)}
{search.strainType && (
<Badge variant="outline" className="text-xs">{search.strainType}</Badge>
)}
</div>
</div> </div>
<ChevronRight className="h-4 w-4 text-gray-400" /> <ChevronRight className="h-4 w-4 text-gray-400" />
</Link> </Link>
))} );
})}
</div> </div>
) : ( ) : (
<div className="text-center py-8"> <div className="text-center py-8">

View File

@@ -3,35 +3,63 @@ import { Link } from 'react-router-dom';
import { Button } from '../../components/ui/button'; import { Button } from '../../components/ui/button';
import { Badge } from '../../components/ui/badge'; import { Badge } from '../../components/ui/badge';
import ProductCard from '../../components/findagram/ProductCard'; import ProductCard from '../../components/findagram/ProductCard';
import { getDeals, getProducts, mapProductForUI } from '../../api/client'; import { getSpecials, getDispensaries, mapProductForUI, mapDispensaryForUI } from '../../api/client';
import { Tag, TrendingDown, Clock, Flame, Loader2 } from 'lucide-react'; import { useGeolocation, sortByDistance } from '../../hooks/useGeolocation';
import { Tag, TrendingDown, Clock, Flame, Loader2, MapPin, Navigation } from 'lucide-react';
const Deals = () => { const Deals = () => {
const [favorites, setFavorites] = useState([]); const [favorites, setFavorites] = useState([]);
const [filter, setFilter] = useState('all'); const [filter, setFilter] = useState('all');
// API state // Geolocation
const [allProducts, setAllProducts] = useState([]); const { location, loading: locationLoading, requestLocation } = useGeolocation({ autoRequest: true });
const [dealsProducts, setDealsProducts] = useState([]);
const [loading, setLoading] = useState(true);
// Fetch data on mount // API state
const [specials, setSpecials] = useState([]);
const [nearbyDispensaryIds, setNearbyDispensaryIds] = useState([]);
const [loading, setLoading] = useState(true);
const [totalCount, setTotalCount] = useState(0);
const [hasMore, setHasMore] = useState(false);
const [loadingMore, setLoadingMore] = useState(false);
// Fetch nearby dispensaries when location is available
useEffect(() => {
const fetchNearbyDispensaries = async () => {
if (!location) return;
try {
const res = await getDispensaries({ limit: 100, hasProducts: true });
const dispensaries = (res.dispensaries || []).map(mapDispensaryForUI);
// Sort by distance and get IDs of nearest 20
const sorted = sortByDistance(dispensaries, location, (d) => ({
latitude: d.latitude,
longitude: d.longitude
}));
const nearbyIds = sorted.slice(0, 20).map(d => d.id);
setNearbyDispensaryIds(nearbyIds);
} catch (err) {
console.error('Error fetching nearby dispensaries:', err);
}
};
fetchNearbyDispensaries();
}, [location]);
// Fetch specials on mount
useEffect(() => { useEffect(() => {
const fetchData = async () => { const fetchData = async () => {
try { try {
setLoading(true); setLoading(true);
const [dealsRes, productsRes] = await Promise.all([ const res = await getSpecials({ limit: 100 });
getDeals({ limit: 50 }),
getProducts({ limit: 50 }),
]);
// Set deals products (products with sale_price) const products = (res.products || []).map(mapProductForUI);
setDealsProducts((dealsRes.products || []).map(mapProductForUI)); setSpecials(products);
setTotalCount(res.pagination?.total || products.length);
// Set all products for fallback display setHasMore(res.pagination?.has_more || false);
setAllProducts((productsRes.products || []).map(mapProductForUI));
} catch (err) { } catch (err) {
console.error('Error fetching deals data:', err); console.error('Error fetching specials:', err);
} finally { } finally {
setLoading(false); setLoading(false);
} }
@@ -39,6 +67,22 @@ const Deals = () => {
fetchData(); fetchData();
}, []); }, []);
const loadMore = async () => {
if (loadingMore || !hasMore) return;
try {
setLoadingMore(true);
const res = await getSpecials({ limit: 50, offset: specials.length });
const newProducts = (res.products || []).map(mapProductForUI);
setSpecials(prev => [...prev, ...newProducts]);
setHasMore(res.pagination?.has_more || false);
} catch (err) {
console.error('Error loading more specials:', err);
} finally {
setLoadingMore(false);
}
};
const toggleFavorite = (productId) => { const toggleFavorite = (productId) => {
setFavorites((prev) => setFavorites((prev) =>
prev.includes(productId) prev.includes(productId)
@@ -47,31 +91,39 @@ const Deals = () => {
); );
}; };
// Use dealsProducts if available, otherwise fall back to allProducts // Filter specials to only show products from nearby dispensaries
const displayProducts = dealsProducts.length > 0 ? dealsProducts : allProducts; const nearbySpecials = nearbyDispensaryIds.length > 0
? specials.filter(p => nearbyDispensaryIds.includes(p.dispensaryId))
: specials;
// Create some "deal categories" from available products // Filter products by category for display sections
const hotDeals = displayProducts.slice(0, 4); const flowerSpecials = nearbySpecials.filter(p =>
const todayOnly = displayProducts.slice(4, 8); p.category?.toLowerCase().includes('flower')
const weeklySpecials = displayProducts.slice(0, 8); ).slice(0, 8);
const edibleSpecials = nearbySpecials.filter(p =>
p.category?.toLowerCase().includes('edible')
).slice(0, 8);
const concentrateSpecials = nearbySpecials.filter(p =>
p.category?.toLowerCase().includes('concentrate') || p.category?.toLowerCase().includes('vape')
).slice(0, 8);
const filterOptions = [ const filterOptions = [
{ id: 'all', label: 'All Deals', icon: Tag }, { id: 'all', label: 'All Specials', icon: Tag },
{ id: 'hot', label: 'Hot Deals', icon: Flame }, { id: 'flower', label: 'Flower', icon: Flame },
{ id: 'today', label: 'Today Only', icon: Clock }, { id: 'edibles', label: 'Edibles', icon: Clock },
{ id: 'weekly', label: 'Weekly Specials', icon: TrendingDown }, { id: 'concentrates', label: 'Concentrates', icon: TrendingDown },
]; ];
const getFilteredProducts = () => { const getFilteredProducts = () => {
switch (filter) { switch (filter) {
case 'hot': case 'flower':
return hotDeals; return flowerSpecials;
case 'today': case 'edibles':
return todayOnly; return edibleSpecials;
case 'weekly': case 'concentrates':
return weeklySpecials; return concentrateSpecials;
default: default:
return displayProducts; return nearbySpecials;
} }
}; };
@@ -89,6 +141,18 @@ const Deals = () => {
<p className="text-lg text-pink-100 max-w-2xl mx-auto"> <p className="text-lg text-pink-100 max-w-2xl mx-auto">
Save big on top cannabis products. Prices updated daily from dispensaries near you. Save big on top cannabis products. Prices updated daily from dispensaries near you.
</p> </p>
{location && (
<div className="mt-4 inline-flex items-center gap-2 bg-white/20 rounded-full px-4 py-2">
<MapPin className="h-4 w-4" />
<span className="text-sm">Showing deals near your location</span>
</div>
)}
{locationLoading && (
<div className="mt-4 inline-flex items-center gap-2 bg-white/20 rounded-full px-4 py-2">
<Loader2 className="h-4 w-4 animate-spin" />
<span className="text-sm">Finding deals near you...</span>
</div>
)}
</div> </div>
</div> </div>
</section> </section>
@@ -98,17 +162,25 @@ const Deals = () => {
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-4"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-4">
<div className="grid grid-cols-2 md:grid-cols-4 gap-4 text-center"> <div className="grid grid-cols-2 md:grid-cols-4 gap-4 text-center">
<div> <div>
<p className="text-2xl font-bold text-pink-600">{loading ? '...' : dealsProducts.length > 0 ? `${dealsProducts.length}+` : `${allProducts.length}+`}</p> <p className="text-2xl font-bold text-pink-600">
<p className="text-sm text-gray-600">Products on Sale</p> {loading ? '...' : nearbySpecials.length > 0 ? nearbySpecials.length : '0'}
</p>
<p className="text-sm text-gray-600">
{location ? 'Specials Near You' : 'Products on Special'}
</p>
</div>
<div>
<p className="text-2xl font-bold text-pink-600">
{nearbyDispensaryIds.length > 0 ? nearbyDispensaryIds.length : '20+'}
</p>
<p className="text-sm text-gray-600">
{location ? 'Nearby Dispensaries' : 'Dispensaries'}
</p>
</div> </div>
<div> <div>
<p className="text-2xl font-bold text-pink-600">Up to 40%</p> <p className="text-2xl font-bold text-pink-600">Up to 40%</p>
<p className="text-sm text-gray-600">Savings</p> <p className="text-sm text-gray-600">Savings</p>
</div> </div>
<div>
<p className="text-2xl font-bold text-pink-600">200+</p>
<p className="text-sm text-gray-600">Dispensaries</p>
</div>
<div> <div>
<p className="text-2xl font-bold text-pink-600">Daily</p> <p className="text-2xl font-bold text-pink-600">Daily</p>
<p className="text-sm text-gray-600">Price Updates</p> <p className="text-sm text-gray-600">Price Updates</p>
@@ -153,74 +225,28 @@ const Deals = () => {
</div> </div>
)} )}
{/* Hot Deals Section */} {/* Products Section */}
{(filter === 'all' || filter === 'hot') && (
<section className="mb-12"> <section className="mb-12">
<div className="flex items-center gap-2 mb-6"> <div className="flex items-center justify-between mb-6">
<Flame className="h-6 w-6 text-orange-500" /> <div className="flex items-center gap-2">
<h2 className="text-2xl font-bold text-gray-900">Hot Deals</h2>
<Badge variant="deal">Limited Time</Badge>
</div>
{loading ? (
<div className="flex justify-center py-8">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
) : (
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-4 gap-6">
{(filter === 'hot' ? getFilteredProducts() : hotDeals).map((product) => (
<ProductCard
key={product.id}
product={product}
onFavorite={toggleFavorite}
isFavorite={favorites.includes(product.id)}
/>
))}
</div>
)}
</section>
)}
{/* Today Only Section */}
{(filter === 'all' || filter === 'today') && (
<section className="mb-12">
<div className="flex items-center gap-2 mb-6">
<Clock className="h-6 w-6 text-red-500" />
<h2 className="text-2xl font-bold text-gray-900">Today Only</h2>
<Badge className="bg-red-100 text-red-800">Ends at Midnight</Badge>
</div>
{loading ? (
<div className="flex justify-center py-8">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
) : (
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-4 gap-6">
{(filter === 'today' ? getFilteredProducts() : todayOnly).map((product) => (
<ProductCard
key={product.id}
product={product}
onFavorite={toggleFavorite}
isFavorite={favorites.includes(product.id)}
/>
))}
</div>
)}
</section>
)}
{/* Weekly Specials Section */}
{(filter === 'all' || filter === 'weekly') && (
<section className="mb-12">
<div className="flex items-center gap-2 mb-6">
<TrendingDown className="h-6 w-6 text-green-500" /> <TrendingDown className="h-6 w-6 text-green-500" />
<h2 className="text-2xl font-bold text-gray-900">Weekly Specials</h2> <h2 className="text-2xl font-bold text-gray-900">
{filter === 'all' ? 'All Specials' :
filter === 'flower' ? 'Flower Specials' :
filter === 'edibles' ? 'Edible Specials' : 'Concentrate Specials'}
</h2>
<Badge variant="deal">{getFilteredProducts().length} products</Badge>
</div> </div>
</div>
{loading ? ( {loading ? (
<div className="flex justify-center py-8"> <div className="flex justify-center py-12">
<Loader2 className="h-8 w-8 animate-spin text-primary" /> <Loader2 className="h-8 w-8 animate-spin text-primary" />
</div> </div>
) : ( ) : getFilteredProducts().length > 0 ? (
<>
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-4 gap-6"> <div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-4 gap-6">
{(filter === 'weekly' ? getFilteredProducts() : weeklySpecials).map((product) => ( {getFilteredProducts().map((product) => (
<ProductCard <ProductCard
key={product.id} key={product.id}
product={product} product={product}
@@ -229,9 +255,38 @@ const Deals = () => {
/> />
))} ))}
</div> </div>
{/* Load More Button */}
{filter === 'all' && hasMore && (
<div className="text-center mt-8">
<Button
variant="outline"
size="lg"
onClick={loadMore}
disabled={loadingMore}
>
{loadingMore ? (
<>
<Loader2 className="h-4 w-4 animate-spin mr-2" />
Loading...
</>
) : (
'Load More Specials'
)}
</Button>
</div>
)}
</>
) : (
<div className="text-center py-12 bg-white rounded-lg">
<Tag className="h-12 w-12 text-gray-300 mx-auto mb-4" />
<p className="text-gray-500 mb-4">No specials found in this category.</p>
<Button variant="outline" onClick={() => setFilter('all')}>
View All Specials
</Button>
</div>
)} )}
</section> </section>
)}
{/* CTA Section */} {/* CTA Section */}
<section className="bg-white rounded-xl p-8 text-center"> <section className="bg-white rounded-xl p-8 text-center">

View File

@@ -0,0 +1,654 @@
import React, { useState, useEffect } from 'react';
import { useParams, Link } from 'react-router-dom';
import { Button } from '../../components/ui/button';
import { Badge } from '../../components/ui/badge';
import { Card, CardContent } from '../../components/ui/card';
import ProductCard from '../../components/findagram/ProductCard';
import {
getDispensaryBySlug,
getDispensarySummary,
getDispensaryBrands,
getDispensaryCategories,
getStoreProducts,
mapDispensaryForUI,
mapProductForUI,
} from '../../api/client';
import {
Store,
ChevronRight,
MapPin,
Phone,
Globe,
Clock,
Truck,
ShoppingBag,
Car,
Loader2,
Package,
Tag,
Filter,
X,
Search,
ChevronDown,
} from 'lucide-react';
const DispensaryDetail = () => {
const { slug } = useParams();
const [dispensary, setDispensary] = useState(null);
const [summary, setSummary] = useState(null);
const [brands, setBrands] = useState([]);
const [categories, setCategories] = useState([]);
const [products, setProducts] = useState([]);
const [loading, setLoading] = useState(true);
const [productsLoading, setProductsLoading] = useState(false);
const [error, setError] = useState(null);
// Filters
const [searchTerm, setSearchTerm] = useState('');
const [selectedCategory, setSelectedCategory] = useState('');
const [selectedBrand, setSelectedBrand] = useState('');
const [stockFilter, setStockFilter] = useState('in_stock');
const [showFilters, setShowFilters] = useState(false);
// Pagination
const [offset, setOffset] = useState(0);
const [hasMore, setHasMore] = useState(true);
const LIMIT = 24;
// Fetch dispensary data
useEffect(() => {
const fetchDispensaryData = async () => {
try {
setLoading(true);
setError(null);
// First get dispensary by slug
const dispRes = await getDispensaryBySlug(slug);
const mappedDispensary = mapDispensaryForUI(dispRes);
setDispensary(mappedDispensary);
// Then fetch related data using the dispensary ID
const dispensaryId = mappedDispensary.id;
const [summaryRes, brandsRes, catsRes] = await Promise.all([
getDispensarySummary(dispensaryId).catch(() => null),
getDispensaryBrands(dispensaryId).catch(() => []),
getDispensaryCategories(dispensaryId).catch(() => []),
]);
setSummary(summaryRes);
setBrands(brandsRes.brands || brandsRes || []);
setCategories(catsRes.categories || catsRes || []);
} catch (err) {
console.error('Error fetching dispensary:', err);
setError(err.message || 'Failed to load dispensary');
} finally {
setLoading(false);
}
};
fetchDispensaryData();
}, [slug]);
// Fetch products when filters change
useEffect(() => {
const fetchProducts = async () => {
if (!dispensary) return;
try {
setProductsLoading(true);
const res = await getStoreProducts(dispensary.id, {
search: searchTerm || undefined,
type: selectedCategory || undefined,
brandName: selectedBrand || undefined,
stockStatus: stockFilter || undefined,
limit: LIMIT,
offset: 0,
});
const mapped = (res.products || []).map(mapProductForUI);
setProducts(mapped);
setOffset(0);
setHasMore(mapped.length === LIMIT);
} catch (err) {
console.error('Error fetching products:', err);
} finally {
setProductsLoading(false);
}
};
fetchProducts();
}, [dispensary, searchTerm, selectedCategory, selectedBrand, stockFilter]);
const loadMore = async () => {
if (productsLoading || !hasMore || !dispensary) return;
try {
setProductsLoading(true);
const newOffset = offset + LIMIT;
const res = await getStoreProducts(dispensary.id, {
search: searchTerm || undefined,
type: selectedCategory || undefined,
brandName: selectedBrand || undefined,
stockStatus: stockFilter || undefined,
limit: LIMIT,
offset: newOffset,
});
const mapped = (res.products || []).map(mapProductForUI);
setProducts((prev) => [...prev, ...mapped]);
setOffset(newOffset);
setHasMore(mapped.length === LIMIT);
} catch (err) {
console.error('Error loading more products:', err);
} finally {
setProductsLoading(false);
}
};
const clearFilters = () => {
setSearchTerm('');
setSelectedCategory('');
setSelectedBrand('');
setStockFilter('in_stock');
};
const hasActiveFilters = searchTerm || selectedCategory || selectedBrand || stockFilter !== 'in_stock';
if (loading) {
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center">
<div className="text-center">
<Loader2 className="h-12 w-12 animate-spin text-primary mx-auto mb-4" />
<p className="text-gray-600">Loading dispensary...</p>
</div>
</div>
);
}
if (error || !dispensary) {
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center">
<div className="text-center">
<h2 className="text-2xl font-bold text-gray-900 mb-2">Dispensary Not Found</h2>
<p className="text-gray-600 mb-4">
{error || "The dispensary you're looking for doesn't exist."}
</p>
<Link to="/">
<Button>Back to Home</Button>
</Link>
</div>
</div>
);
}
return (
<div className="min-h-screen bg-gray-50">
{/* Breadcrumb */}
<div className="bg-white border-b">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-3">
<nav className="flex items-center space-x-2 text-sm">
<Link to="/" className="text-gray-500 hover:text-primary">
Home
</Link>
<ChevronRight className="h-4 w-4 text-gray-400" />
<Link to="/dispensaries" className="text-gray-500 hover:text-primary">
Dispensaries
</Link>
<ChevronRight className="h-4 w-4 text-gray-400" />
<span className="text-gray-900 truncate max-w-[200px]">{dispensary.name}</span>
</nav>
</div>
</div>
{/* Dispensary Header */}
<section className="bg-white border-b">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
<div className="flex flex-col lg:flex-row gap-8">
{/* Left: Main Info */}
<div className="flex-1">
<div className="flex items-start gap-4">
{/* Dispensary Image/Icon */}
<div className="w-20 h-20 rounded-xl bg-gradient-to-br from-green-500 to-green-600 flex items-center justify-center shrink-0">
{dispensary.imageUrl ? (
<img
src={dispensary.imageUrl}
alt={dispensary.name}
className="w-full h-full object-cover rounded-xl"
onError={(e) => {
e.target.style.display = 'none';
}}
/>
) : (
<Store className="h-10 w-10 text-white" />
)}
</div>
<div className="flex-1 min-w-0">
<h1 className="text-2xl sm:text-3xl font-bold text-gray-900 mb-2">
{dispensary.name}
</h1>
{/* Location */}
<div className="flex items-center text-gray-600 mb-3">
<MapPin className="h-4 w-4 mr-1 shrink-0" />
<span className="truncate">
{dispensary.address && `${dispensary.address}, `}
{dispensary.city}, {dispensary.state} {dispensary.zip}
</span>
</div>
{/* Badges */}
<div className="flex flex-wrap gap-2">
{dispensary.licenseType?.recreational && (
<Badge className="bg-green-100 text-green-800">Recreational</Badge>
)}
{dispensary.licenseType?.medical && (
<Badge className="bg-blue-100 text-blue-800">Medical</Badge>
)}
{dispensary.services?.delivery && (
<Badge variant="outline" className="flex items-center gap-1">
<Truck className="h-3 w-3" />
Delivery
</Badge>
)}
{dispensary.services?.pickup && (
<Badge variant="outline" className="flex items-center gap-1">
<ShoppingBag className="h-3 w-3" />
Pickup
</Badge>
)}
{dispensary.services?.curbside && (
<Badge variant="outline" className="flex items-center gap-1">
<Car className="h-3 w-3" />
Curbside
</Badge>
)}
</div>
</div>
</div>
{/* Stats Cards */}
<div className="grid grid-cols-2 sm:grid-cols-4 gap-4 mt-6">
<Card>
<CardContent className="p-4 text-center">
<Package className="h-6 w-6 text-primary mx-auto mb-1" />
<p className="text-2xl font-bold text-gray-900">
{summary?.productCount || dispensary.productCount || 0}
</p>
<p className="text-xs text-gray-500">Products</p>
</CardContent>
</Card>
<Card>
<CardContent className="p-4 text-center">
<Tag className="h-6 w-6 text-primary mx-auto mb-1" />
<p className="text-2xl font-bold text-gray-900">
{brands.length || dispensary.brandCount || 0}
</p>
<p className="text-xs text-gray-500">Brands</p>
</CardContent>
</Card>
<Card>
<CardContent className="p-4 text-center">
<Filter className="h-6 w-6 text-primary mx-auto mb-1" />
<p className="text-2xl font-bold text-gray-900">
{categories.length || dispensary.categoryCount || 0}
</p>
<p className="text-xs text-gray-500">Categories</p>
</CardContent>
</Card>
<Card>
<CardContent className="p-4 text-center">
<ShoppingBag className="h-6 w-6 text-green-500 mx-auto mb-1" />
<p className="text-2xl font-bold text-gray-900">
{summary?.inStockCount || dispensary.inStockCount || 0}
</p>
<p className="text-xs text-gray-500">In Stock</p>
</CardContent>
</Card>
</div>
</div>
{/* Right: Contact Info */}
<div className="lg:w-80">
<Card>
<CardContent className="p-6 space-y-4">
<h3 className="font-semibold text-gray-900">Store Info</h3>
{dispensary.website && (
<a
href={dispensary.website}
target="_blank"
rel="noopener noreferrer"
className="flex items-center text-sm text-primary hover:underline"
>
<Globe className="h-4 w-4 mr-2" />
Visit Website
</a>
)}
{dispensary.menuUrl && (
<a
href={dispensary.menuUrl}
target="_blank"
rel="noopener noreferrer"
className="flex items-center text-sm text-primary hover:underline"
>
<ShoppingBag className="h-4 w-4 mr-2" />
View Full Menu
</a>
)}
<div className="flex items-start text-sm text-gray-600">
<MapPin className="h-4 w-4 mr-2 mt-0.5 shrink-0" />
<div>
{dispensary.address && <p>{dispensary.address}</p>}
<p>
{dispensary.city}, {dispensary.state} {dispensary.zip}
</p>
</div>
</div>
{/* Map placeholder */}
{dispensary.latitude && dispensary.longitude && (
<a
href={`https://www.google.com/maps/search/?api=1&query=${dispensary.latitude},${dispensary.longitude}`}
target="_blank"
rel="noopener noreferrer"
className="block"
>
<div className="h-32 bg-gray-100 rounded-lg flex items-center justify-center hover:bg-gray-200 transition-colors">
<div className="text-center">
<MapPin className="h-8 w-8 text-gray-400 mx-auto mb-1" />
<span className="text-xs text-gray-500">View on Map</span>
</div>
</div>
</a>
)}
</CardContent>
</Card>
</div>
</div>
</div>
</section>
{/* Products Section */}
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
<div className="flex flex-col lg:flex-row gap-8">
{/* Filters Sidebar - Desktop */}
<div className="hidden lg:block lg:w-64 shrink-0">
<div className="sticky top-4 space-y-6">
<Card>
<CardContent className="p-4">
<h3 className="font-semibold text-gray-900 mb-4">Filters</h3>
{/* Search */}
<div className="mb-4">
<label className="block text-sm font-medium text-gray-700 mb-1">
Search
</label>
<div className="relative">
<Search className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<input
type="text"
value={searchTerm}
onChange={(e) => setSearchTerm(e.target.value)}
placeholder="Search products..."
className="w-full pl-9 pr-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
/>
</div>
</div>
{/* Category */}
<div className="mb-4">
<label className="block text-sm font-medium text-gray-700 mb-1">
Category
</label>
<select
value={selectedCategory}
onChange={(e) => setSelectedCategory(e.target.value)}
className="w-full px-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
>
<option value="">All Categories</option>
{categories.map((cat) => (
<option key={cat.type || cat} value={cat.type || cat}>
{cat.type || cat}
</option>
))}
</select>
</div>
{/* Brand */}
<div className="mb-4">
<label className="block text-sm font-medium text-gray-700 mb-1">
Brand
</label>
<select
value={selectedBrand}
onChange={(e) => setSelectedBrand(e.target.value)}
className="w-full px-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
>
<option value="">All Brands</option>
{brands.map((brand) => (
<option key={brand.brand_name || brand} value={brand.brand_name || brand}>
{brand.brand_name || brand}
</option>
))}
</select>
</div>
{/* Stock Status */}
<div className="mb-4">
<label className="block text-sm font-medium text-gray-700 mb-1">
Availability
</label>
<select
value={stockFilter}
onChange={(e) => setStockFilter(e.target.value)}
className="w-full px-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
>
<option value="in_stock">In Stock</option>
<option value="">All Products</option>
<option value="out_of_stock">Out of Stock</option>
</select>
</div>
{hasActiveFilters && (
<Button
variant="outline"
size="sm"
onClick={clearFilters}
className="w-full"
>
<X className="h-4 w-4 mr-1" />
Clear Filters
</Button>
)}
</CardContent>
</Card>
</div>
</div>
{/* Main Content */}
<div className="flex-1">
{/* Mobile Filter Toggle */}
<div className="lg:hidden mb-4">
<Button
variant="outline"
onClick={() => setShowFilters(!showFilters)}
className="w-full justify-between"
>
<span className="flex items-center">
<Filter className="h-4 w-4 mr-2" />
Filters
{hasActiveFilters && (
<Badge className="ml-2 bg-primary text-white">Active</Badge>
)}
</span>
<ChevronDown className={`h-4 w-4 transition-transform ${showFilters ? 'rotate-180' : ''}`} />
</Button>
{/* Mobile Filters */}
{showFilters && (
<Card className="mt-2">
<CardContent className="p-4 space-y-4">
{/* Search */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Search
</label>
<div className="relative">
<Search className="absolute left-3 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400" />
<input
type="text"
value={searchTerm}
onChange={(e) => setSearchTerm(e.target.value)}
placeholder="Search products..."
className="w-full pl-9 pr-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
/>
</div>
</div>
<div className="grid grid-cols-2 gap-4">
{/* Category */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Category
</label>
<select
value={selectedCategory}
onChange={(e) => setSelectedCategory(e.target.value)}
className="w-full px-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
>
<option value="">All</option>
{categories.map((cat) => (
<option key={cat.type || cat} value={cat.type || cat}>
{cat.type || cat}
</option>
))}
</select>
</div>
{/* Brand */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Brand
</label>
<select
value={selectedBrand}
onChange={(e) => setSelectedBrand(e.target.value)}
className="w-full px-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
>
<option value="">All</option>
{brands.map((brand) => (
<option key={brand.brand_name || brand} value={brand.brand_name || brand}>
{brand.brand_name || brand}
</option>
))}
</select>
</div>
</div>
{/* Stock Status */}
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
Availability
</label>
<select
value={stockFilter}
onChange={(e) => setStockFilter(e.target.value)}
className="w-full px-3 py-2 border rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-primary"
>
<option value="in_stock">In Stock</option>
<option value="">All Products</option>
<option value="out_of_stock">Out of Stock</option>
</select>
</div>
{hasActiveFilters && (
<Button
variant="outline"
size="sm"
onClick={clearFilters}
className="w-full"
>
<X className="h-4 w-4 mr-1" />
Clear Filters
</Button>
)}
</CardContent>
</Card>
)}
</div>
{/* Results Header */}
<div className="flex items-center justify-between mb-6">
<h2 className="text-xl font-bold text-gray-900">
Products
{products.length > 0 && (
<span className="text-gray-500 font-normal ml-2">
({products.length}{hasMore ? '+' : ''})
</span>
)}
</h2>
</div>
{/* Products Grid */}
{productsLoading && products.length === 0 ? (
<div className="flex items-center justify-center py-12">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
) : products.length > 0 ? (
<>
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 gap-6">
{products.map((product) => (
<ProductCard
key={product.id}
product={product}
showDispensaryCount={false}
/>
))}
</div>
{/* Load More */}
{hasMore && (
<div className="text-center mt-8">
<Button
variant="outline"
onClick={loadMore}
disabled={productsLoading}
>
{productsLoading ? (
<>
<Loader2 className="h-4 w-4 animate-spin mr-2" />
Loading...
</>
) : (
'Load More Products'
)}
</Button>
</div>
)}
</>
) : (
<div className="text-center py-12 bg-white rounded-lg">
<Package className="h-12 w-12 text-gray-300 mx-auto mb-4" />
<p className="text-gray-500 mb-4">
{hasActiveFilters
? 'No products match your filters.'
: 'No products available at this dispensary.'}
</p>
{hasActiveFilters && (
<Button variant="outline" onClick={clearFilters}>
Clear Filters
</Button>
)}
</div>
)}
</div>
</div>
</div>
</div>
);
};
export default DispensaryDetail;

View File

@@ -1,27 +1,86 @@
import React, { useState } from 'react'; import React, { useState, useEffect } from 'react';
import { Link } from 'react-router-dom'; import { Link, useNavigate } from 'react-router-dom';
import { Button } from '../../components/ui/button'; import { Button } from '../../components/ui/button';
import ProductCard from '../../components/findagram/ProductCard'; import { useAuth } from '../../context/AuthContext';
import { mockFavorites, mockProducts } from '../../mockData'; import { getFavorites, removeFavorite } from '../../api/consumer';
import { Heart, Trash2 } from 'lucide-react'; import { Heart, Trash2, Loader2 } from 'lucide-react';
const Favorites = () => { const Favorites = () => {
const [favorites, setFavorites] = useState(mockFavorites); const { isAuthenticated, authFetch, requireAuth } = useAuth();
const navigate = useNavigate();
const favoriteProducts = mockProducts.filter((p) => favorites.includes(p.id)); const [favorites, setFavorites] = useState([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState(null);
const [removingId, setRemovingId] = useState(null);
const toggleFavorite = (productId) => { // Redirect to home if not authenticated
setFavorites((prev) => useEffect(() => {
prev.includes(productId) if (!isAuthenticated) {
? prev.filter((id) => id !== productId) requireAuth(() => navigate('/dashboard/favorites'));
: [...prev, productId] }
); }, [isAuthenticated, requireAuth, navigate]);
// Fetch favorites
useEffect(() => {
if (!isAuthenticated) return;
const fetchFavorites = async () => {
setLoading(true);
setError(null);
try {
const data = await getFavorites(authFetch);
setFavorites(data.favorites || []);
} catch (err) {
setError(err.message);
} finally {
setLoading(false);
}
}; };
const clearAllFavorites = () => { fetchFavorites();
}, [isAuthenticated, authFetch]);
const handleRemoveFavorite = async (favoriteId) => {
setRemovingId(favoriteId);
try {
await removeFavorite(authFetch, favoriteId);
setFavorites(prev => prev.filter(f => f.id !== favoriteId));
} catch (err) {
setError(err.message);
} finally {
setRemovingId(null);
}
};
const clearAllFavorites = async () => {
if (!window.confirm('Are you sure you want to remove all favorites?')) return;
setLoading(true);
try {
// Remove all favorites one by one
await Promise.all(favorites.map(f => removeFavorite(authFetch, f.id)));
setFavorites([]); setFavorites([]);
} catch (err) {
setError(err.message);
} finally {
setLoading(false);
}
}; };
if (!isAuthenticated) {
return null;
}
if (loading) {
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
);
}
return ( return (
<div className="min-h-screen bg-gray-50"> <div className="min-h-screen bg-gray-50">
{/* Header */} {/* Header */}
@@ -34,10 +93,10 @@ const Favorites = () => {
My Favorites My Favorites
</h1> </h1>
<p className="text-gray-600 mt-2"> <p className="text-gray-600 mt-2">
{favoriteProducts.length} {favoriteProducts.length === 1 ? 'product' : 'products'} saved {favorites.length} {favorites.length === 1 ? 'product' : 'products'} saved
</p> </p>
</div> </div>
{favoriteProducts.length > 0 && ( {favorites.length > 0 && (
<Button <Button
variant="outline" variant="outline"
onClick={clearAllFavorites} onClick={clearAllFavorites}
@@ -52,15 +111,86 @@ const Favorites = () => {
</section> </section>
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
{favoriteProducts.length > 0 ? ( {error && (
<div className="mb-6 p-4 bg-red-50 border border-red-200 rounded-lg text-red-600">
{error}
</div>
)}
{favorites.length > 0 ? (
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-4 gap-6"> <div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-4 gap-6">
{favoriteProducts.map((product) => ( {favorites.map((fav) => (
<ProductCard <div
key={product.id} key={fav.id}
product={product} className="bg-white rounded-lg shadow-sm border overflow-hidden group"
onFavorite={toggleFavorite} >
isFavorite={true} <Link to={`/products/${fav.productId}`}>
<div className="relative aspect-square overflow-hidden bg-gray-100">
<img
src={fav.imageUrl || '/placeholder-product.jpg'}
alt={fav.savedName}
className="w-full h-full object-cover group-hover:scale-105 transition-transform duration-300"
/> />
{fav.priceDrop && (
<div className="absolute top-2 left-2 bg-green-500 text-white text-xs px-2 py-1 rounded">
Price dropped!
</div>
)}
</div>
</Link>
<div className="p-4">
<p className="text-xs text-gray-500 uppercase tracking-wide mb-1">
{fav.currentBrand || fav.savedBrand}
</p>
<Link to={`/products/${fav.productId}`}>
<h3 className="font-semibold text-gray-900 line-clamp-2 hover:text-primary transition-colors mb-2">
{fav.currentName || fav.savedName}
</h3>
</Link>
{fav.dispensaryName && (
<p className="text-sm text-gray-500 mb-2">
at {fav.dispensaryName}
</p>
)}
<div className="flex items-center justify-between">
<div>
{fav.currentPrice ? (
<p className="text-lg font-bold text-primary">
${parseFloat(fav.currentPrice).toFixed(2)}
</p>
) : fav.savedPrice ? (
<p className="text-lg font-bold text-gray-600">
${parseFloat(fav.savedPrice).toFixed(2)}
<span className="text-xs text-gray-400 ml-1">(saved)</span>
</p>
) : (
<p className="text-sm text-gray-400">No price</p>
)}
{fav.priceDrop && fav.savedPrice && fav.currentPrice && (
<p className="text-xs text-green-600">
Was ${parseFloat(fav.savedPrice).toFixed(2)}
</p>
)}
</div>
<Button
variant="ghost"
size="icon"
onClick={() => handleRemoveFavorite(fav.id)}
disabled={removingId === fav.id}
className="text-red-500 hover:text-red-700 hover:bg-red-50"
>
{removingId === fav.id ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Heart className="h-4 w-4 fill-current" />
)}
</Button>
</div>
</div>
</div>
))} ))}
</div> </div>
) : ( ) : (

View File

@@ -10,10 +10,14 @@ import {
getDeals, getDeals,
getCategories, getCategories,
getBrands, getBrands,
getDispensaries,
getStats,
mapProductForUI, mapProductForUI,
mapCategoryForUI, mapCategoryForUI,
mapBrandForUI, mapBrandForUI,
mapDispensaryForUI,
} from '../../api/client'; } from '../../api/client';
import { useGeolocation, sortByDistance } from '../../hooks/useGeolocation';
import { import {
Search, Search,
Leaf, Leaf,
@@ -26,6 +30,9 @@ import {
ShoppingBag, ShoppingBag,
MapPin, MapPin,
Loader2, Loader2,
Navigation,
Clock,
Store,
} from 'lucide-react'; } from 'lucide-react';
const Home = () => { const Home = () => {
@@ -33,17 +40,22 @@ const Home = () => {
const [searchQuery, setSearchQuery] = useState(''); const [searchQuery, setSearchQuery] = useState('');
const [favorites, setFavorites] = useState([]); const [favorites, setFavorites] = useState([]);
// Geolocation - auto-request on mount
const { location, loading: locationLoading, error: locationError, requestLocation, hasPermission } = useGeolocation({ autoRequest: true });
// API state // API state
const [featuredProducts, setFeaturedProducts] = useState([]); const [featuredProducts, setFeaturedProducts] = useState([]);
const [dealsProducts, setDealsProducts] = useState([]); const [dealsProducts, setDealsProducts] = useState([]);
const [categories, setCategories] = useState([]); const [categories, setCategories] = useState([]);
const [brands, setBrands] = useState([]); const [brands, setBrands] = useState([]);
const [nearbyDispensaries, setNearbyDispensaries] = useState([]);
const [stats, setStats] = useState({ const [stats, setStats] = useState({
products: 0, products: 0,
brands: 0, brands: 0,
dispensaries: 0, dispensaries: 0,
}); });
const [loading, setLoading] = useState(true); const [loading, setLoading] = useState(true);
const [dispensariesLoading, setDispensariesLoading] = useState(false);
// Fetch data on mount // Fetch data on mount
useEffect(() => { useEffect(() => {
@@ -52,11 +64,12 @@ const Home = () => {
setLoading(true); setLoading(true);
// Fetch all data in parallel // Fetch all data in parallel
const [productsRes, dealsRes, categoriesRes, brandsRes] = await Promise.all([ const [productsRes, dealsRes, categoriesRes, brandsRes, statsRes] = await Promise.all([
getProducts({ limit: 4 }), getProducts({ limit: 4 }),
getDeals({ limit: 4 }), getDeals({ limit: 4 }),
getCategories(), getCategories(),
getBrands({ limit: 100 }), getBrands({ limit: 500 }),
getStats(),
]); ]);
// Set featured products // Set featured products
@@ -75,15 +88,17 @@ const Home = () => {
); );
// Set brands (first 6 as popular) // Set brands (first 6 as popular)
const allBrands = brandsRes.brands || [];
setBrands( setBrands(
(brandsRes.brands || []).slice(0, 6).map(mapBrandForUI) allBrands.slice(0, 6).map(mapBrandForUI)
); );
// Set stats // Set stats from dedicated stats endpoint
const statsData = statsRes.stats || {};
setStats({ setStats({
products: productsRes.pagination?.total || 0, products: statsData.products || 0,
brands: brandsRes.pagination?.total || 0, brands: statsData.brands || 0,
dispensaries: 200, // Hardcoded for now - could add API endpoint dispensaries: statsData.dispensaries || 0,
}); });
} catch (err) { } catch (err) {
console.error('Error fetching home data:', err); console.error('Error fetching home data:', err);
@@ -95,6 +110,35 @@ const Home = () => {
fetchData(); fetchData();
}, []); }, []);
// Fetch nearby dispensaries when location is available
useEffect(() => {
const fetchNearbyDispensaries = async () => {
if (!location) return;
try {
setDispensariesLoading(true);
// Fetch dispensaries with products
const res = await getDispensaries({ limit: 100, hasProducts: true });
const dispensaries = (res.dispensaries || []).map(mapDispensaryForUI);
// Sort by distance from user
const sorted = sortByDistance(dispensaries, location, (d) => ({
latitude: d.latitude,
longitude: d.longitude
}));
// Take top 6 nearest
setNearbyDispensaries(sorted.slice(0, 6));
} catch (err) {
console.error('Error fetching nearby dispensaries:', err);
} finally {
setDispensariesLoading(false);
}
};
fetchNearbyDispensaries();
}, [location]);
const handleSearch = (e) => { const handleSearch = (e) => {
e.preventDefault(); e.preventDefault();
if (searchQuery.trim()) { if (searchQuery.trim()) {
@@ -203,8 +247,136 @@ const Home = () => {
</div> </div>
</section> </section>
{/* Featured Products */} {/* Nearby Dispensaries Section */}
<section className="py-12 bg-gray-50"> <section className="py-12 bg-gray-50">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
<div className="flex items-center justify-between mb-8">
<div>
<h2 className="text-2xl font-bold text-gray-900 flex items-center gap-2">
<MapPin className="h-6 w-6 text-primary" />
{location ? 'Dispensaries Near You' : 'Find Dispensaries Near You'}
</h2>
<p className="text-gray-600 mt-1">
{location
? 'Sorted by distance from your location'
: 'Enable location to see nearby stores'}
</p>
</div>
<div className="flex items-center gap-3">
{!location && (
<Button
onClick={requestLocation}
disabled={locationLoading}
variant="outline"
className="text-primary border-primary"
>
{locationLoading ? (
<Loader2 className="h-4 w-4 animate-spin mr-2" />
) : (
<Navigation className="h-4 w-4 mr-2" />
)}
Use My Location
</Button>
)}
<Link to="/dispensaries">
<Button variant="ghost" className="text-primary">
View All
<ArrowRight className="ml-2 h-4 w-4" />
</Button>
</Link>
</div>
</div>
{dispensariesLoading ? (
<div className="flex justify-center py-8">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
) : nearbyDispensaries.length > 0 ? (
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 gap-6">
{nearbyDispensaries.map((dispensary) => (
<Link
key={dispensary.id}
to={`/dispensaries/${dispensary.slug || dispensary.id}`}
className="group"
>
<Card className="h-full transition-all hover:shadow-lg hover:-translate-y-1">
<CardContent className="p-5">
<div className="flex items-start gap-4">
<div className="w-14 h-14 rounded-lg bg-gray-100 flex items-center justify-center flex-shrink-0 overflow-hidden">
{dispensary.imageUrl ? (
<img
src={dispensary.imageUrl}
alt={dispensary.name}
className="w-full h-full object-cover"
/>
) : (
<Store className="h-7 w-7 text-gray-400" />
)}
</div>
<div className="flex-1 min-w-0">
<h3 className="font-semibold text-gray-900 group-hover:text-primary transition-colors truncate">
{dispensary.name}
</h3>
<p className="text-sm text-gray-500 mt-1 flex items-center gap-1">
<MapPin className="h-3 w-3" />
{dispensary.city}, {dispensary.state}
</p>
{dispensary.distance !== null && (
<p className="text-sm text-primary font-medium mt-1">
{dispensary.distance} mi away
</p>
)}
</div>
</div>
<div className="flex items-center gap-2 mt-4 pt-4 border-t">
{dispensary.productCount > 0 && (
<Badge variant="secondary" className="text-xs">
{dispensary.productCount} products
</Badge>
)}
{dispensary.rating && (
<Badge variant="outline" className="text-xs flex items-center gap-1">
<Star className="h-3 w-3 fill-yellow-400 text-yellow-400" />
{dispensary.rating}
</Badge>
)}
</div>
</CardContent>
</Card>
</Link>
))}
</div>
) : location ? (
<div className="text-center py-8">
<Store className="h-12 w-12 text-gray-300 mx-auto mb-4" />
<p className="text-gray-500">No dispensaries found nearby</p>
</div>
) : (
<div className="text-center py-8 bg-white rounded-lg border-2 border-dashed border-gray-200">
<Navigation className="h-12 w-12 text-gray-300 mx-auto mb-4" />
<p className="text-gray-600 font-medium mb-2">Find dispensaries near you</p>
<p className="text-gray-500 text-sm mb-4 max-w-md mx-auto">
We'll use your location to show the closest dispensaries, their products, and prices. Your location is never stored or shared.
</p>
<Button
onClick={requestLocation}
disabled={locationLoading}
className="gradient-purple"
>
{locationLoading ? (
<Loader2 className="h-4 w-4 animate-spin mr-2" />
) : (
<MapPin className="h-4 w-4 mr-2" />
)}
Share My Location
</Button>
</div>
)}
</div>
</section>
{/* Featured Products */}
<section className="py-12 bg-white">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
<div className="flex items-center justify-between mb-8"> <div className="flex items-center justify-between mb-8">
<div> <div>
@@ -236,7 +408,7 @@ const Home = () => {
</section> </section>
{/* Deals Section */} {/* Deals Section */}
<section className="py-12 bg-white"> <section className="py-12 bg-gray-50">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
<div className="flex items-center justify-between mb-8"> <div className="flex items-center justify-between mb-8">
<div> <div>
@@ -280,7 +452,7 @@ const Home = () => {
</section> </section>
{/* Browse by Category */} {/* Browse by Category */}
<section className="py-12 bg-gray-50"> <section className="py-12 bg-white">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
<div className="text-center mb-8"> <div className="text-center mb-8">
<h2 className="text-2xl font-bold text-gray-900">Browse by Category</h2> <h2 className="text-2xl font-bold text-gray-900">Browse by Category</h2>
@@ -320,7 +492,7 @@ const Home = () => {
</section> </section>
{/* Popular Brands */} {/* Popular Brands */}
<section className="py-12 bg-white"> <section className="py-12 bg-gray-50">
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8">
<div className="flex items-center justify-between mb-8"> <div className="flex items-center justify-between mb-8">
<div> <div>

View File

@@ -1,23 +1,84 @@
import React, { useState } from 'react'; import React, { useState, useEffect } from 'react';
import { Link } from 'react-router-dom'; import { Link, useNavigate } from 'react-router-dom';
import { Button } from '../../components/ui/button'; import { Button } from '../../components/ui/button';
import { Card, CardContent } from '../../components/ui/card'; import { Card, CardContent } from '../../components/ui/card';
import { Badge } from '../../components/ui/badge'; import { Badge } from '../../components/ui/badge';
import { mockSavedSearches } from '../../mockData'; import { useAuth } from '../../context/AuthContext';
import { Bookmark, Search, Trash2, ChevronRight } from 'lucide-react'; import { getSavedSearches, deleteSavedSearch } from '../../api/consumer';
import { Bookmark, Search, Trash2, ChevronRight, Loader2 } from 'lucide-react';
const SavedSearches = () => { const SavedSearches = () => {
const [searches, setSearches] = useState(mockSavedSearches); const { isAuthenticated, authFetch, requireAuth } = useAuth();
const navigate = useNavigate();
const deleteSearch = (searchId) => { const [searches, setSearches] = useState([]);
setSearches((prev) => prev.filter((search) => search.id !== searchId)); const [loading, setLoading] = useState(true);
const [error, setError] = useState(null);
const [deletingId, setDeletingId] = useState(null);
// Redirect to home if not authenticated
useEffect(() => {
if (!isAuthenticated) {
requireAuth(() => navigate('/dashboard/searches'));
}
}, [isAuthenticated, requireAuth, navigate]);
// Fetch saved searches
useEffect(() => {
if (!isAuthenticated) return;
const fetchSearches = async () => {
setLoading(true);
setError(null);
try {
const data = await getSavedSearches(authFetch);
setSearches(data.savedSearches || []);
} catch (err) {
setError(err.message);
} finally {
setLoading(false);
}
}; };
const buildSearchUrl = (filters) => { fetchSearches();
const params = new URLSearchParams(filters); }, [isAuthenticated, authFetch]);
const handleDeleteSearch = async (searchId) => {
setDeletingId(searchId);
try {
await deleteSavedSearch(authFetch, searchId);
setSearches(prev => prev.filter(s => s.id !== searchId));
} catch (err) {
setError(err.message);
} finally {
setDeletingId(null);
}
};
const buildSearchUrl = (search) => {
const params = new URLSearchParams();
if (search.query) params.set('search', search.query);
if (search.category) params.set('type', search.category);
if (search.brand) params.set('brandName', search.brand);
if (search.strainType) params.set('strainType', search.strainType);
if (search.minPrice) params.set('minPrice', search.minPrice);
if (search.maxPrice) params.set('maxPrice', search.maxPrice);
return `/products?${params.toString()}`; return `/products?${params.toString()}`;
}; };
if (!isAuthenticated) {
return null;
}
if (loading) {
return (
<div className="min-h-screen bg-gray-50 flex items-center justify-center">
<Loader2 className="h-8 w-8 animate-spin text-primary" />
</div>
);
}
return ( return (
<div className="min-h-screen bg-gray-50"> <div className="min-h-screen bg-gray-50">
{/* Header */} {/* Header */}
@@ -44,6 +105,12 @@ const SavedSearches = () => {
</section> </section>
<div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8"> <div className="max-w-7xl mx-auto px-4 sm:px-6 lg:px-8 py-8">
{error && (
<div className="mb-6 p-4 bg-red-50 border border-red-200 rounded-lg text-red-600">
{error}
</div>
)}
{searches.length > 0 ? ( {searches.length > 0 ? (
<div className="space-y-4"> <div className="space-y-4">
{searches.map((search) => ( {searches.map((search) => (
@@ -56,31 +123,33 @@ const SavedSearches = () => {
<div className="flex-1 min-w-0"> <div className="flex-1 min-w-0">
<h3 className="font-medium text-gray-900">{search.name}</h3> <h3 className="font-medium text-gray-900">{search.name}</h3>
<div className="flex flex-wrap gap-2 mt-2"> <div className="flex flex-wrap gap-2 mt-2">
{search.filters.category && ( {search.query && (
<Badge variant="secondary">{search.filters.category}</Badge> <Badge variant="secondary">"{search.query}"</Badge>
)} )}
{search.filters.strainType && ( {search.category && (
<Badge variant="outline">{search.filters.strainType}</Badge> <Badge variant="secondary">{search.category}</Badge>
)} )}
{search.filters.priceMax && ( {search.brand && (
<Badge variant="outline">Under ${search.filters.priceMax}</Badge> <Badge variant="outline">{search.brand}</Badge>
)} )}
{search.filters.thcMin && ( {search.strainType && (
<Badge variant="outline">THC {search.filters.thcMin}%+</Badge> <Badge variant="outline">{search.strainType}</Badge>
)} )}
{search.filters.search && ( {search.maxPrice && (
<Badge variant="secondary">"{search.filters.search}"</Badge> <Badge variant="outline">Under ${search.maxPrice}</Badge>
)}
{search.minThc && (
<Badge variant="outline">THC {search.minThc}%+</Badge>
)} )}
</div> </div>
</div> </div>
<div className="text-right"> <div className="text-right">
<p className="text-sm text-gray-500">{search.resultCount} results</p> <p className="text-xs text-gray-400">
<p className="text-xs text-gray-400 mt-1">
Saved {new Date(search.createdAt).toLocaleDateString()} Saved {new Date(search.createdAt).toLocaleDateString()}
</p> </p>
</div> </div>
<div className="flex items-center gap-2"> <div className="flex items-center gap-2">
<Link to={buildSearchUrl(search.filters)}> <Link to={buildSearchUrl(search)}>
<Button variant="outline" size="sm"> <Button variant="outline" size="sm">
Run Search Run Search
<ChevronRight className="h-4 w-4 ml-1" /> <ChevronRight className="h-4 w-4 ml-1" />
@@ -89,11 +158,16 @@ const SavedSearches = () => {
<Button <Button
variant="ghost" variant="ghost"
size="icon" size="icon"
onClick={() => deleteSearch(search.id)} onClick={() => handleDeleteSearch(search.id)}
disabled={deletingId === search.id}
className="text-red-600 hover:text-red-700 hover:bg-red-50" className="text-red-600 hover:text-red-700 hover:bg-red-50"
title="Delete search" title="Delete search"
> >
{deletingId === search.id ? (
<Loader2 className="h-4 w-4 animate-spin" />
) : (
<Trash2 className="h-4 w-4" /> <Trash2 className="h-4 w-4" />
)}
</Button> </Button>
</div> </div>
</div> </div>

View File

@@ -1,65 +1,162 @@
# Hydration Worker Deployment # Task Worker Pods
# These workers process raw_payloads → canonical tables. # Each pod runs 5 role-agnostic workers that pull tasks from worker_tasks queue.
# Scale this deployment to increase hydration throughput.
# #
# Architecture: # Architecture:
# - The main 'scraper' deployment runs the API server + scheduler (1 replica) # - Pods are named from a predefined list (Aethelgard, Xylos, etc.)
# - This 'scraper-worker' deployment runs hydration workers (5 replicas) # - Each pod spawns 5 worker processes
# - Workers use DB-level locking to prevent double-processing # - Workers register with API and show their pod name
# - Each worker processes payloads in batches with configurable limits # - HPA scales pods 5-15 based on pending task count
apiVersion: apps/v1 # - Workers use DB-level locking (FOR UPDATE SKIP LOCKED) to prevent conflicts
kind: Deployment #
# Pod Names (up to 25):
# Aethelgard, Xylos, Kryll, Coriolis, Dimidium, Veridia, Zetani, Talos IV,
# Onyx, Celestia, Gormand, Betha, Ragnar, Syphon, Axiom, Nadir, Terra Nova,
# Acheron, Nexus, Vespera, Helios Prime, Oasis, Mordina, Cygnus, Umbra
---
apiVersion: v1
kind: ConfigMap
metadata: metadata:
name: scraper-worker name: pod-names
namespace: dispensary-scraper
data:
names: |
Aethelgard
Xylos
Kryll
Coriolis
Dimidium
Veridia
Zetani
Talos IV
Onyx
Celestia
Gormand
Betha
Ragnar
Syphon
Axiom
Nadir
Terra Nova
Acheron
Nexus
Vespera
Helios Prime
Oasis
Mordina
Cygnus
Umbra
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: worker-pod
namespace: dispensary-scraper namespace: dispensary-scraper
spec: spec:
serviceName: worker-pods
replicas: 5 replicas: 5
podManagementPolicy: Parallel
selector: selector:
matchLabels: matchLabels:
app: scraper-worker app: worker-pod
template: template:
metadata: metadata:
labels: labels:
app: scraper-worker app: worker-pod
spec: spec:
imagePullSecrets: imagePullSecrets:
- name: regcred - name: regcred
containers: containers:
- name: worker - name: workers
image: code.cannabrands.app/creationshop/dispensary-scraper:latest image: code.cannabrands.app/creationshop/dispensary-scraper:latest
# Run the hydration worker in loop mode # Run 5 workers per pod
command: ["node"] command: ["/bin/sh", "-c"]
args: ["dist/scripts/run-hydration.js", "--mode=payload", "--loop"] args:
- |
# Get pod ordinal (0, 1, 2, etc.)
ORDINAL=$(echo $HOSTNAME | rev | cut -d'-' -f1 | rev)
# Get pod name from configmap
POD_NAME=$(sed -n "$((ORDINAL + 1))p" /etc/pod-names/names)
echo "Starting pod: $POD_NAME (ordinal: $ORDINAL)"
# Start 5 workers in this pod
for i in 1 2 3 4 5; do
WORKER_ID="${POD_NAME}-worker-${i}" \
POD_NAME="$POD_NAME" \
node dist/tasks/task-worker.js &
done
# Wait for all workers
wait
envFrom: envFrom:
- configMapRef: - configMapRef:
name: scraper-config name: scraper-config
- secretRef: - secretRef:
name: scraper-secrets name: scraper-secrets
env: env:
# Worker-specific environment variables - name: API_BASE_URL
- name: WORKER_MODE value: "http://scraper:3010"
value: "true" - name: WORKERS_PER_POD
# Pod name becomes part of worker ID for debugging value: "5"
- name: POD_NAME volumeMounts:
valueFrom: - name: pod-names
fieldRef: mountPath: /etc/pod-names
fieldPath: metadata.name
resources: resources:
requests: requests:
memory: "256Mi"
cpu: "100m"
limits:
memory: "512Mi" memory: "512Mi"
cpu: "500m" cpu: "200m"
# Health check - workers don't expose ports, but we can use a file check limits:
memory: "1Gi"
cpu: "1000m"
livenessProbe: livenessProbe:
exec: exec:
command: command:
- /bin/sh - /bin/sh
- -c - -c
- "pgrep -f 'run-hydration' > /dev/null" - "pgrep -f 'task-worker' > /dev/null"
initialDelaySeconds: 10 initialDelaySeconds: 15
periodSeconds: 30 periodSeconds: 30
failureThreshold: 3 failureThreshold: 3
# Graceful shutdown - give workers time to complete current batch volumes:
- name: pod-names
configMap:
name: pod-names
terminationGracePeriodSeconds: 60 terminationGracePeriodSeconds: 60
---
# Headless service for StatefulSet
apiVersion: v1
kind: Service
metadata:
name: worker-pods
namespace: dispensary-scraper
spec:
clusterIP: None
selector:
app: worker-pod
ports:
- port: 80
name: placeholder
---
# HPA to scale pods based on pending tasks
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: worker-pod-hpa
namespace: dispensary-scraper
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: StatefulSet
name: worker-pod
minReplicas: 5
maxReplicas: 15
metrics:
- type: External
external:
metric:
name: pending_tasks
selector:
matchLabels:
queue: worker_tasks
target:
type: AverageValue
averageValue: "10"