Kelly 33e12de3f2 Remove domain-based shortcuts for Curaleaf/Sol detection ...

Menu detection now always crawls websites to find actual embedded menu
providers instead of marking stores as proprietary based on domain alone.
This fixes detection for stores like Curaleaf that may use Dutchie embeds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

2025-12-03 21:55:31 -07:00

..

dist

fix(menu-detection): remove non-existent platform_dispensary_id_resolved_at column

2025-12-03 19:40:22 -07:00

k8s

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

migrations

fix(monitor): remove non-existent worker columns from job_run_logs query

2025-12-03 18:45:05 -07:00

new-scrapers

fix(monitor): remove non-existent worker columns from job_run_logs query

2025-12-03 18:45:05 -07:00

node_modules

fix(monitor): remove non-existent worker columns from job_run_logs query

2025-12-03 18:45:05 -07:00

public

fix(monitor): remove non-existent worker columns from job_run_logs query

2025-12-03 18:45:05 -07:00

src

Remove domain-based shortcuts for Curaleaf/Sol detection

2025-12-03 21:55:31 -07:00

.dockerignore

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

.env

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-curaleaf-az-stores.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-curaleaf-docker.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-discount-columns.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-geo-fields.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-price-history.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-proxy-location-columns.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-sol-flower-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

add-test-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

API_TOKENS_GUIDE.md

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

API_USAGE.md

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

cancel-pending-job.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

capture-age-gate-cookies.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-48th-store.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brand-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brand-names.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brands-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-db.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-deeply-rooted.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-discounts.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-enrichment-progress.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-for-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-jobs.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-leaks.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-product-data.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-product-detail-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-product-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-products-schema.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-proxies.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-proxy-stats.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-scraper-data.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-select-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-special-product.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-store.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-tables.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-treez-categories.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-treez-data.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

check-treez-pagination.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

complete-schema-migration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

count-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-azdhs-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-brands-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-brands-tables.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

create-table.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

curaleaf-cookies.json

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-after-state-select.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-age-gate-detailed.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-age-gate-elements.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-azdhs-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-brands-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-curaleaf-buttons.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-dutchie-detection.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-dutchie-selectors.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-google-scraper.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-product-card.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-products-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-sale-price.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-scrape.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-sol-flower.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

debug-specials-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

delete-dispensary-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

diagnose-curaleaf-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

Dockerfile

Rename WordPress plugin from Dutchie Menus to Crawlsy Menus v1.3.0

2025-12-01 16:29:21 -07:00

Dockerfile.dev

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-azdhs-from-google-maps.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-dispensaries-from-google.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-prices.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

enrich-single-dispensary.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

explore-curaleaf-menu.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

find-dutchie-menu-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

find-price-location.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

find-products-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-brand-names.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-brands-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-category-urls.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-docker-db-urls.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

fix-stuck-job.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

get-best-dispensary.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

get-dispensary-ids.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

import-proxies.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

inspect-treez-structure.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

launch-25-scrapers.sh

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

list-dispensaries.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

list-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

migrate-proxy-locations.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

openapi.yaml

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

package-lock.json

Add local product detail page with Dutchie comparison

2025-11-30 06:34:38 -07:00

package.json

Add AZ schema migration script and health check

2025-12-02 10:40:06 -07:00

parse-azdhs-copied-data.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

populate-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

populate-proxy-locations.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

preview-azdhs-import.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

reset-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

reset-one-job.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-location-migration.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-logo-migration.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-migration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

run-scrape-jobs-migration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

schema-dump.sql

fix(monitor): remove non-existent worker columns from job_run_logs query

2025-12-03 18:45:05 -07:00

scrape-48th-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-api.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-auto.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-better.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-manual-scroll.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-azdhs-map.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-brands-48th.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-bypass-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-simple.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-stores.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-curaleaf-with-proxy.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-dutchie-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-google-maps.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-leafly-arizona.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-leafly-map-markers.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-leafly-map.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-mobile-chrome.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-parallel-brands.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-real-clicks.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-treez.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-two-gates.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-with-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

scrape-with-jobs.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

SESSION_SUMMARY.md

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

setup-api-permissions-table.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

start-proxy-test.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

sync-by-address.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

sync-dispensaries-to-dutchie.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-azdhs-parser.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-brand-products.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-brand-scrape.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-brands-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-category-discovery.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-curaleaf-playwright-advanced.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-curaleaf-playwright-final.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-curaleaf-playwright.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-deeply-rooted.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-dutchie-simple.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-dutchie-template.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-filter-interaction.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-graphql-curl.sh

fix(monitor): remove non-existent worker columns from job_run_logs query

2025-12-03 18:45:05 -07:00

test-improved-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-llm-scraper-deeply-rooted.ts

fix(monitor): remove non-existent worker columns from job_run_logs query

2025-12-03 18:45:05 -07:00

test-no-proxy.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-playwright-scraper-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-puppeteer-click.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-scrape-flower.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-sol-flower-age-gate.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-sol-flower-category-discovery.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-sol-flower-scraper.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-stealth-sol-flower.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-stock-status.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-template-integration.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

test-treez-page.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

tsconfig.json

Add local product detail page with Dutchie comparison

2025-11-30 06:34:38 -07:00

update-best-dispensary.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

update-curaleaf-dutchie-urls.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-categories.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-curaleaf-urls.js

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-curaleaf.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00

verify-treez-total-count.ts

Initial commit - Dutchie dispensary scraper

2025-11-28 19:45:44 -07:00