- 008: Add IF NOT EXISTS to ALTER TABLE ADD COLUMN
- 011: Add IF NOT EXISTS to CREATE TABLE and INDEX
- 012: Add IF NOT EXISTS, DROP TRIGGER IF EXISTS
- 013: Add ON CONFLICT (azdhs_id) DO NOTHING
- 014: Add IF NOT EXISTS to ALTER TABLE ADD COLUMN
All migrations can now be safely re-run without errors.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- ai_provider and ai_model stored in settings table
- Editable via /settings page in admin UI
- API keys remain in env vars for security
- Falls back to env vars if settings not in DB
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Configure via env vars:
- AI_PROVIDER=claude|openai (default: claude)
- ANTHROPIC_API_KEY=sk-ant-...
- OPENAI_API_KEY=sk-...
Falls back to template generation if no API key configured.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move pool initialization inside functions (lazy loading)
- Fix page_key parsing for state-XX format
- Add fallback query if mv_state_metrics doesn't exist
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- GET /api/seo/pages - List all SEO pages with filters
- POST /api/seo/sync-state-pages - Create pages for states with dispensaries
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove unused MapPin import from Dashboard.jsx
- Remove unused Clock import from DashboardHome.jsx
- Remove unused Mail import from Profile.jsx
- Remove unused CardHeader, CardTitle imports from SavedSearches.jsx
- Add eslint-disable for heading-has-content in card.jsx (shadcn pattern)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- ci.yml: Runs on all branches - typecheck backend, build all 3 frontends
- deploy.yml: Runs on master only after CI passes
- Builds and pushes 4 Docker images to Gitea registry
- Deploys to Kubernetes (scraper, scraper-worker, 3 frontends)
Required secrets:
- REGISTRY_USERNAME: Gitea username
- REGISTRY_PASSWORD: Gitea password/token
- KUBECONFIG: Base64-encoded kubeconfig
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
CLAUDE.md now requires explicit local mode confirmation before
running any crawler, orchestrator, sandbox test, or image scrape.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add findagram.co React frontend with product search, brands, categories
- Add findadispo.com React frontend with dispensary locator
- Wire findagram to backend /api/az/* endpoints
- Update category/brand links to route to /products with filters
- Add k8s manifests for both frontends
- Add multi-domain user support migrations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add .dockerignore to exclude .env.local from Docker builds
- Replace http://localhost:9020/dutchie/ with /api/images/dutchie/ in:
- StoreDetail.tsx
- ProductDetail.tsx
- StoreView.tsx
This fixes production builds connecting to localhost for images.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add key_type column to wp_dutchie_api_permissions (internal/wordpress)
- Create apiScope middleware with scope types and helpers
- Internal keys: full access to ALL dispensaries
- WordPress keys: restricted to single dispensary
- Update all /api/v1 handlers to honor scope
- Add /dispensaries and /search endpoints to public API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add backend stale process monitoring API (/api/stale-processes)
- Add users management route
- Add frontend landing page and stale process monitor UI on /scraper-tools
- Move old development scripts to backend/archive/
- Update frontend build with new features
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Tab-specific data loading: Only fetch APIs needed for the active tab
- AZ Live tab fetches only AZ monitor APIs
- Dispensary Jobs tab fetches only legacy job APIs
- Crawl History tab fetches only scraper history APIs
- Auto-refresh now respects active tab
- Added New and Updated columns to Crawl History table with color coding
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add curaleaf.com and livewithsol.com to dutchie detection patterns
- Update crawl-five-sequential.ts with all 57 dutchie store IDs for batch crawling
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- GET /api/az/admin/dutchie-stores - Lists all Dutchie stores with crawl status
- POST /api/az/admin/crawl-all - Enqueues product crawl jobs for all ready stores
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Menu detection now always crawls websites to find actual embedded menu
providers instead of marking stores as proprietary based on domain alone.
This fixes detection for stores like Curaleaf that may use Dutchie embeds.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add extractFromMenuUrl() to discovery.ts that extracts either cName or platformId directly
from Dutchie URLs (handles /api/v2/embedded-menu/<id>.js pattern)
- Add isObjectId() helper to identify MongoDB ObjectIds in URLs
- Update menu-detection.ts to skip GraphQL resolution when URL contains platformId directly
- For proprietary domains (curaleaf, sol), crawl website to find actual menu provider
instead of blindly marking as not_crawlable
- If website crawl finds Dutchie embedded menu, set menu_type='dutchie' and resolve platform ID
- Tested successfully with consumeaz.com which discovers Dutchie embedded menu JS URL
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When onlyUnknown=true and onlyMissingPlatformId=true, the query now
uses OR logic instead of AND to include:
- Stores with unknown menu_type (new stores needing detection)
- Stores with menu_type='dutchie' but no platform_dispensary_id
This allows re-detection to:
1. Resolve platform IDs for actual Dutchie stores
2. Reclassify stores that migrated away from Dutchie (e.g. to Curaleaf)
Also changed default for onlyMissingPlatformId to true so scheduled
detection jobs always attempt to resolve missing platform IDs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add dba_name to DISPENSARY_COLUMNS for the /api/az/stores list endpoint
and getDispensaryById for the single store endpoint. This allows the
frontend to display DBA names (trade names) when available, falling
back to the legal entity name.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The UPDATE query was trying to set a column that doesn't exist in the database
schema, causing platform ID resolution to fail silently. Now stores the
resolved_at timestamp in provider_detection_data JSONB instead.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The job_run_logs table tracks scheduled job orchestration, not individual
worker jobs. Worker info (worker_id, worker_hostname) belongs on
dispensary_crawl_jobs, not job_run_logs.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Product crawl job now only targets ready dispensaries (menu_type=dutchie
AND platform_dispensary_id IS NOT NULL). Detection is handled by the
separate menu_detection schedule.
This ensures:
- Single path for menu discovery (menu_detection job)
- Product crawl only processes ready dutchie stores
- All 5 workers claim jobs via queue/locking
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Main dashboard now uses /api/az/dashboard for dispensary, product, and
brand counts instead of the legacy /api/dashboard/stats endpoint. This
ensures the dashboard displays consistent data with the /az pages.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Reorder processProducts() to create snapshots BEFORE attempting image downloads.
Previously, if image downloads hung or failed, the process would be killed before
snapshots were created, resulting in 0 snapshots despite successful product upserts.
Changes:
- Move Step 3 (snapshot creation) before Step 4 (image downloads)
- Ensures core crawl data (products + snapshots) is persisted even if images fail
- Adds chunked batch processing for improved memory management
Tested locally: 771 snapshots created for dispensary 112 with quantity data populated.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Changed missing_from_feed=true to stock_status='missing_from_feed'
- Changed products_inserted/snapshots_created to products_new/products_updated
- Changed crawl_jobs table reference to dispensary_crawl_jobs
- Fixed product query to use actual snapshot columns (price in cents, etc.)
- Added explicit column list for dispensaries to avoid SELECT * issues
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add 'curaleaf' to MenuProvider type enum
- Add Curaleaf URL patterns BEFORE Dutchie in PROVIDER_URL_PATTERNS for proper precedence
- Add isCuraleafUrl() and extractCuraleafStoreUrl() helper functions
- Check website field for Curaleaf pattern before any Dutchie resolution
- Clear stale Dutchie menu_url when store is identified as Curaleaf
- Mark Curaleaf stores as not_crawlable with reason until crawler is built
- This prevents 60s Dutchie timeouts for stores that migrated to Curaleaf
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
