Creationshop/cannaiq
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Increase navigation timeout to 90s for parallel browser crawling

Browse Source 
Dutchie pages can take 30-40s to load, and when running 3 parallel browsers
they compete for resources, causing some to hit the 60s timeout. Increased
to 90s to accommodate slow-loading React SPAs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Kelly
2025-12-01 10:37:52 -07:00
parent 81606447d2
commit d614979569
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
backend/src/scraper-v2/downloader.ts
View File

@@ -379,9 +379,10 @@ export class Downloader {
} }
// Navigate to page - use networkidle2 for SPAs like Dutchie // Navigate to page - use networkidle2 for SPAs like Dutchie
// Increased timeout to 90s - Dutchie pages can take 30-40s to fully load
const navigationPromise = page.goto(request.url, { const navigationPromise = page.goto(request.url, {
waitUntil: 'networkidle2', waitUntil: 'networkidle2',
timeout: 60000 timeout: 90000
}); });
const response = await navigationPromise; const response = await navigationPromise;