Commit Graph

14 Commits

Author SHA1 Message Date
Kelly
98970acf13 fix: Update Dockerfiles to use git.spdy.io registry
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
2025-12-15 08:18:53 -07:00
Kelly
af859a85f9 feat: Add Real-Time Inventory Tracking infrastructure
Implements per-store high-frequency crawl scheduling and inventory
snapshot tracking for sales velocity estimation (Hoodie Analytics parity).

Database migrations:
- 117: Per-store crawl_interval_minutes and next_crawl_at columns
- 118: inventory_snapshots table (30-day retention)
- 119: product_visibility_events table for OOS/brand alerts (90-day)

Backend changes:
- inventory-snapshots.ts: Shared utility normalizing Dutchie/Jane/Treez
- visibility-events.ts: Detects OOS, price changes, brand drops
- task-scheduler.ts: checkHighFrequencyStores() runs every 60s
- Handler updates: 2-line additions to save snapshots/events

API endpoints:
- GET /api/tasks/schedules/high-frequency
- PUT /api/tasks/schedules/high-frequency/:id
- DELETE /api/tasks/schedules/high-frequency/:id

Frontend:
- TasksDashboard: Per-Store Schedules section with stats

Features:
- Per-store intervals (15/30/60 min configurable)
- Jitter (0-20%) to avoid detection patterns
- Cross-platform support (Dutchie, Jane, Treez)
- No crawler core changes - scheduling/post-crawl only

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 15:53:04 -07:00
Kelly
15cb657f13 fix(docker): Revert to libasound2 for Debian bookworm
- libasound2t64 is for Debian trixie (13), not bookworm (12)
- Keep build tools fix for native modules

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 11:07:03 -07:00
Kelly
f15920e508 fix(docker): Add build tools for native modules and fix Debian package name
- Add python3 and build-essential to builder stage for bcrypt/sharp compilation
- Change libasound2 to libasound2t64 for Debian bookworm compatibility
- Copy pre-built node_modules from builder instead of re-running npm install
- Prune dev dependencies in builder for smaller production image

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 11:01:22 -07:00
Kelly
824d48fd85 fix: Add curl to Docker, add active flag to worker_tasks
- Install curl in Docker container for Dutchie HTTP requests
- Add 'active' column to worker_tasks (default false) to prevent
  accidental task execution on startup
- Update task-service to only claim tasks where active=true

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 23:12:09 -07:00
Kelly
2ee2ba6b8c fix(docker): Use npm install instead of npm ci for reliability
npm ci can fail when package-lock.json has minor mismatches with
package.json. npm install is more forgiving and appropriate for
Docker builds where determinism is less critical than reliability.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-10 11:28:29 -07:00
Kelly
9647f94f89 fix: Copy migrations folder to Docker image + fix SQL FILTER syntax
- Dockerfile: Add COPY migrations ./migrations so auto-migrate works on remote
- intelligence.ts: Fix FILTER clause placement in aggregate functions
  - FILTER must be inside AVG(), not wrapping ROUND()
  - Remove redundant FILTER on MIN (already filtered by WHERE)
  - Remove unsupported FILTER on PERCENTILE_CONT

These fixes resolve:
- "Failed to get task counts" (worker_tasks table missing)
- "FILTER specified but round is not an aggregate function" errors
- /national page "column m.state does not exist" (mv_state_metrics missing)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 09:38:05 -07:00
Kelly
7e517b5801 ci: Use self-hosted base images to avoid Docker Hub rate limits
Cached node:20, node:20-slim, and nginx:alpine to code.cannabrands.app.
No more Docker Hub dependency for builds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-09 13:07:21 -07:00
Kelly
d91c55a344 feat: Add stale process monitor, users route, landing page, archive old scripts
- Add backend stale process monitoring API (/api/stale-processes)
- Add users management route
- Add frontend landing page and stale process monitor UI on /scraper-tools
- Move old development scripts to backend/archive/
- Update frontend build with new features

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-05 04:07:31 -07:00
Kelly
511629b4e6 Rename WordPress plugin from Dutchie Menus to Crawlsy Menus v1.3.0
- Renamed plugin to avoid WordPress.org naming conflict causing false update notifications
- Added /downloads static route to serve plugin zip file
- Updated all CSS classes from dutchie- to crawlsy- prefix
- Added plugin zip to backend/public/downloads for hosting
- Plugin available at: https://dispos.crawlsy.com/downloads/crawlsy-menus-1.3.0.zip

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 16:29:21 -07:00
Kelly
0083f6a510 Add version display in admin sidebar footer
- Add /api/version endpoint that returns build info from env vars
- Add version footer to Layout.tsx showing build version, git SHA, and image tag
- Update Dockerfile to accept build args for version info (APP_BUILD_VERSION, APP_GIT_SHA, APP_BUILD_TIME, CONTAINER_IMAGE_TAG)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 10:01:10 -07:00
Kelly
8b4292fbb2 Add local product detail page with Dutchie comparison
- Add ProductDetail page for viewing products locally
- Add Dutchie and Details buttons to product cards in Products and StoreDetail pages
- Add Last Updated display showing data freshness
- Add parallel scrape scripts and routes
- Add K8s deployment configurations
- Add frontend Dockerfile with nginx

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 06:34:38 -07:00
Kelly
6e597f15ca Fix Dockerfile for Puppeteer/Chromium support 2025-11-28 20:08:57 -07:00
Kelly
5757a8e9bd Initial commit - Dutchie dispensary scraper 2025-11-28 19:45:44 -07:00