57 lines
2.2 KiB
Markdown
57 lines
2.2 KiB
Markdown
# Dutchie Menus Project - Session Notes
|
|
|
|
## Important Rules & Context
|
|
|
|
### Container Management
|
|
- **ONLY manage containers that start with `dutchie-`**
|
|
- User has multiple projects running - be careful not to affect other containers
|
|
|
|
### Scraping Setup
|
|
- Uses fingerprints and antidetect for web scraping
|
|
- Bypasses age gates using real Puppeteer clicks (not page.evaluate)
|
|
- Mobile Chrome user agent: `Mozilla/5.0 (Linux; Android 10; SM-G973F) AppleWebKit/537.36`
|
|
|
|
### Database Configuration
|
|
- **Correct DATABASE_URL**: `postgresql://sail:password@localhost:5432/dutchie_menus`
|
|
- ⚠️ **Common Issue**: Backend sometimes connects to wrong DB: `postgresql://kelly:kelly@localhost:5432/hub`
|
|
- Always verify backend is using correct .env file from `/home/kelly/dutchie-menus/backend/.env`
|
|
|
|
### Project Structure
|
|
- Backend: `/home/kelly/dutchie-menus/backend` (Port 3010)
|
|
- Frontend: `/home/kelly/dutchie-menus/frontend` (Port 5174)
|
|
- Database: PostgreSQL on localhost:5432
|
|
|
|
### Key Technical Decisions
|
|
- **Brands Table**: Stores brand names separately from products
|
|
- Unique constraint on `(store_id, name)`
|
|
- Index on `(store_id)` for fast queries
|
|
- **Specials Table**: Daily tracking with date-based filtering
|
|
- Composite index on `(store_id, valid_date DESC)` for performance
|
|
- Designed for "specials change every day" requirement
|
|
|
|
### Scraping Breakthroughs
|
|
- **Age Gate Solution**: Use `await option.click()` instead of `page.evaluate(() => element.click())`
|
|
- React synthetic events require real browser interactions
|
|
- Two-gate system: State selector dropdown → "I'm over 21" button
|
|
|
|
### Current Implementation Status
|
|
- ✅ StoreBrands and StoreSpecials pages created
|
|
- ✅ API endpoints for /brands and /specials
|
|
- ✅ Routes added to App.tsx
|
|
- ✅ Navigation working from StoreDetail page
|
|
- ✅ 127 brands scraped and saved for Curaleaf Phoenix Airport
|
|
|
|
## Common Issues & Solutions
|
|
|
|
### Issue: "0 brands" showing on page
|
|
**Cause**: Backend connected to wrong database
|
|
**Solution**: Restart backend from correct directory with .env file:
|
|
```bash
|
|
cd /home/kelly/dutchie-menus/backend
|
|
npm run dev
|
|
```
|
|
|
|
### Issue: Scraper not bypassing age gate
|
|
**Cause**: Using page.evaluate for clicks
|
|
**Solution**: Use real Puppeteer clicks: `await element.click()`
|