Website Crawler
App IntegrationsCrawl multiple pages with depth control
Overview
Crawl multiple pages with depth control.
Configuration
| Property | Type | Default | Description |
|---|---|---|---|
url | string | — | Url for the Website Crawler node |
maxDepth | number | — | MaxDepth for the Website Crawler node |
maxPages | number | — | MaxPages for the Website Crawler node |
Output Variables
Reference these variables in downstream nodes using the {{node.field}} syntax.
{{crawler.pages}}arrayCrawled page data
Examples
Basic Usage
Example configuration:
{
"url": "value",
"maxDepth": 2,
"maxPages": 50
}Tips & Best Practices
- •Ensure all required fields for Website Crawler are configured.
- •Check the output variables to see available data.
- •Verify your connection credentials are set up in Settings > Integrations.
Try Website Crawler in Your Workflow
Create a free account and start building AI-powered workflows with the Website Crawler node.
Open Workflow Editor