2.2 KiB
2.2 KiB
Dutchie Menus Project - Session Notes
Important Rules & Context
Container Management
- ONLY manage containers that start with
dutchie- - User has multiple projects running - be careful not to affect other containers
Scraping Setup
- Uses fingerprints and antidetect for web scraping
- Bypasses age gates using real Puppeteer clicks (not page.evaluate)
- Mobile Chrome user agent:
Mozilla/5.0 (Linux; Android 10; SM-G973F) AppleWebKit/537.36
Database Configuration
- Correct DATABASE_URL:
postgresql://sail:password@localhost:5432/dutchie_menus - ⚠️ Common Issue: Backend sometimes connects to wrong DB:
postgresql://kelly:kelly@localhost:5432/hub - Always verify backend is using correct .env file from
/home/kelly/dutchie-menus/backend/.env
Project Structure
- Backend:
/home/kelly/dutchie-menus/backend(Port 3010) - Frontend:
/home/kelly/dutchie-menus/frontend(Port 5174) - Database: PostgreSQL on localhost:5432
Key Technical Decisions
- Brands Table: Stores brand names separately from products
- Unique constraint on
(store_id, name) - Index on
(store_id)for fast queries
- Unique constraint on
- Specials Table: Daily tracking with date-based filtering
- Composite index on
(store_id, valid_date DESC)for performance - Designed for "specials change every day" requirement
- Composite index on
Scraping Breakthroughs
- Age Gate Solution: Use
await option.click()instead ofpage.evaluate(() => element.click())- React synthetic events require real browser interactions
- Two-gate system: State selector dropdown → "I'm over 21" button
Current Implementation Status
- ✅ StoreBrands and StoreSpecials pages created
- ✅ API endpoints for /brands and /specials
- ✅ Routes added to App.tsx
- ✅ Navigation working from StoreDetail page
- ✅ 127 brands scraped and saved for Curaleaf Phoenix Airport
Common Issues & Solutions
Issue: "0 brands" showing on page
Cause: Backend connected to wrong database Solution: Restart backend from correct directory with .env file:
cd /home/kelly/dutchie-menus/backend
npm run dev
Issue: Scraper not bypassing age gate
Cause: Using page.evaluate for clicks
Solution: Use real Puppeteer clicks: await element.click()