ScrapegraphAI icon

ScrapegraphAI

Turn any webpage into usable data in one shot – ScrapegraphAI explores the website and extracts the content you need.

Actions7

Overview

The node integrates with the ScrapegraphAI API to convert a given website URL into markdown format. It sends the URL to the API, optionally rendering JavaScript-heavy websites if requested, and receives the webpage content converted into markdown. This is useful for scenarios where you want to extract clean, readable markdown content from web pages for documentation, note-taking, or further processing in markdown-compatible tools.

Practical examples:

  • Converting blog posts or articles into markdown for easy editing or publishing.
  • Extracting product descriptions or FAQs from websites into markdown format for internal knowledge bases.
  • Archiving web content in markdown for offline reading or version control.

Properties

Name Meaning
Website URL The URL of the website to convert to markdown.
Render Heavy JS Whether to render JavaScript-heavy websites (this may consume additional API credits).

Output

The output is a JSON object containing the response from the ScrapegraphAI /markdownify endpoint. This typically includes the markdown-converted content of the specified website.

  • json: Contains the markdown data extracted from the website.
  • No binary data output is indicated by the code.

Dependencies

  • Requires an API key credential for authenticating with the ScrapegraphAI service.
  • The node makes HTTP POST requests to https://api.scrapegraphai.com/v1/markdownify.
  • Proper configuration of the API key credential in n8n is necessary for successful execution.

Troubleshooting

  • Invalid API Key or Authentication Errors: Ensure that the API key credential is correctly configured and has the necessary permissions.
  • Invalid Website URL: If the URL is malformed or unreachable, the API call will fail. Verify the URL correctness and network accessibility.
  • Rendering Heavy JS Sites: Enabling "Render Heavy JS" may consume more API credits and take longer. Use only when necessary.
  • API Response Errors: If the API returns errors, check the error message for details. Common issues include rate limits or invalid parameters.
  • JSON Parsing Errors: Not applicable here since the node does not parse user-provided JSON for this operation.

Links and References

Discussion