Kelly 1fb8f84929 Add crawl fields directly to dispensaries table ...

Migration 025: Dispensaries ARE stores - add crawl metadata fields
(menu_url, provider_type, scrape_enabled, crawl_status, etc.)
directly to dispensaries table instead of maintaining separate stores table.

- Copies menu_url from 22 existing stores to their dispensaries
- Migrates products from store_id to dispensary_id
- Detects provider_type from menu_url domain
- Adds indexes for crawl scheduling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

2025-11-30 23:52:48 -07:00

..

dist

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

k8s

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

migrations

Add crawl fields directly to dispensaries table

2025-11-30 23:52:48 -07:00

node_modules

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

src

Switch scheduler UI to dispensary-based API

2025-11-30 18:54:52 -07:00

.dockerignore

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

.env

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-curaleaf-az-stores.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-curaleaf-docker.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-discount-columns.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-geo-fields.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-price-history.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-proxy-location-columns.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-sol-flower-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-test-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

API_TOKENS_GUIDE.md

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

API_USAGE.md

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

cancel-pending-job.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

capture-age-gate-cookies.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-48th-store.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brand-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brand-names.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brands-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-db.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-deeply-rooted.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-discounts.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-enrichment-progress.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-for-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-jobs.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-leaks.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-product-data.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-product-detail-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-product-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-products-schema.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-proxies.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-proxy-stats.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-scraper-data.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-select-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-special-product.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-store.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-tables.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-treez-categories.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-treez-data.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-treez-pagination.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

complete-schema-migration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

count-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-azdhs-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-brands-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-brands-tables.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-table.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

curaleaf-cookies.json

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-after-state-select.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-age-gate-detailed.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-age-gate-elements.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-azdhs-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-brands-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-curaleaf-buttons.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-dutchie-detection.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-dutchie-selectors.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-google-scraper.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-product-card.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-products-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-sale-price.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-scrape.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-sol-flower.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-specials-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

delete-dispensary-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

diagnose-curaleaf-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

Dockerfile

Add version display in admin sidebar footer

2025-11-30 10:01:10 -07:00

Dockerfile.dev

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-azdhs-from-google-maps.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-dispensaries-from-google.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-single-dispensary.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

explore-curaleaf-menu.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

find-dutchie-menu-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

find-price-location.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

find-products-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-brand-names.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-brands-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-category-urls.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-docker-db-urls.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-stuck-job.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

get-best-dispensary.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

get-dispensary-ids.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

import-proxies.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

inspect-treez-structure.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

launch-25-scrapers.sh

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

list-dispensaries.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

list-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

migrate-proxy-locations.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

openapi.yaml

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

package-lock.json

Add local product detail page with Dutchie comparison

2025-11-30 06:34:38 -07:00

package.json

Add local product detail page with Dutchie comparison

2025-11-30 06:34:38 -07:00

parse-azdhs-copied-data.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

populate-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

populate-proxy-locations.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

preview-azdhs-import.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

reset-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

reset-one-job.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-location-migration.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-logo-migration.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-migration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-scrape-jobs-migration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-48th-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-api.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-auto.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-better.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-manual-scroll.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-map.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-brands-48th.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-bypass-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-simple.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-with-proxy.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-dutchie-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-google-maps.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-leafly-arizona.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-leafly-map-markers.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-leafly-map.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-mobile-chrome.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-parallel-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-real-clicks.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-treez.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-two-gates.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-with-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-with-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

SESSION_SUMMARY.md

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

setup-api-permissions-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

start-proxy-test.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

sync-by-address.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

sync-dispensaries-to-dutchie.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-azdhs-parser.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-brand-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-brand-scrape.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-brands-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-category-discovery.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-curaleaf-playwright-advanced.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-curaleaf-playwright-final.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-curaleaf-playwright.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-deeply-rooted.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-dutchie-simple.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-dutchie-template.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-filter-interaction.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-improved-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-no-proxy.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-playwright-scraper-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-puppeteer-click.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-scrape-flower.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-sol-flower-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-sol-flower-category-discovery.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-sol-flower-scraper.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-stealth-sol-flower.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-stock-status.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-template-integration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-treez-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

tsconfig.json

Add local product detail page with Dutchie comparison

2025-11-30 06:34:38 -07:00

update-best-dispensary.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

update-curaleaf-dutchie-urls.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-categories.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-curaleaf-urls.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-treez-total-count.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00