FireCrawl

FireCrawl API

Actions6

Overview

The FireCrawl node's "Map A Website And Get Urls" operation allows you to scan a website and retrieve a list of URLs, optionally filtered by a search term. This is useful for scenarios such as:

Crawling documentation sites to index or analyze their structure.
Gathering all accessible pages from a given domain for SEO or content audits.
Filtering URLs that match specific keywords (e.g., only documentation or blog pages).

Practical examples:

Mapping https://firecrawl.dev to get all URLs containing the word "docs".
Auditing a website to find all product-related pages by searching for "product".

Properties

Display Name	Type	Description
Url	String	The URL of the website to map.
Search	String	Search term to filter specific URLs.
Use Custom Body	Boolean	Whether to use a custom body for the request.
Custom Body	JSON	Custom JSON body to send instead of standard parameters.

Notes:

If "Use Custom Body" is enabled, the node will send the contents of "Custom Body" as the request payload, overriding the standard "Url" and "Search" fields.

Output

The node outputs a json field containing the results from the FireCrawl API. While the exact structure depends on the API response, typically you can expect:

{
  "urls": [
    "https://example.com/docs/page1",
    "https://example.com/docs/page2"
    // ... more URLs matching the search criteria
  ],
  // Additional metadata may be present depending on the API
}

The main output is an array of URLs found on the mapped website, possibly filtered by the provided search term.

Dependencies

External Service: Requires access to the FireCrawl API.
API Key: You must configure the "FireCrawl API" credentials in n8n.
Environment Variable: The base URL for the API is set via the credentials (baseUrl).

Troubleshooting

Common Issues:

Invalid or missing API key:
Error message: "401 Unauthorized"
Resolution: Ensure your FireCrawl API credentials are correctly configured in n8n.
Malformed URL or unreachable site:
Error message: "400 Bad Request" or "Failed to fetch"
Resolution: Double-check the "Url" property for typos and ensure the target site is accessible.
Incorrect Custom Body format:
Error message: "Invalid JSON" or unexpected API errors
Resolution: Make sure the "Custom Body" field contains valid JSON matching the API's expected schema.
No URLs returned:
Possible cause: The search term is too restrictive or the site has no matching pages.
Resolution: Try broadening the search term or verifying the site's content.