Kelly 95fc8bb4cc Improve menu detection to extract platform ID from URL and crawl proprietary domains
- Add extractFromMenuUrl() to discovery.ts that extracts either cName or platformId directly
  from Dutchie URLs (handles /api/v2/embedded-menu/<id>.js pattern)
- Add isObjectId() helper to identify MongoDB ObjectIds in URLs
- Update menu-detection.ts to skip GraphQL resolution when URL contains platformId directly
- For proprietary domains (curaleaf, sol), crawl website to find actual menu provider
  instead of blindly marking as not_crawlable
- If website crawl finds Dutchie embedded menu, set menu_type='dutchie' and resolve platform ID
- Tested successfully with consumeaz.com which discovers Dutchie embedded menu JS URL

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 21:41:45 -07:00
2025-12-03 18:45:05 -07:00
Description
CannaiQ Scraper
117 MiB
Languages
TypeScript 54.5%
JavaScript 37.5%
PLpgSQL 3.6%
PHP 3.1%
Python 0.5%
Other 0.7%