Apify icon

Apify

Access Apify tools for web scraping, data extraction, and automation.

Overview

The node provides integration with Apify, a platform for web scraping, data extraction, and automation. Specifically, the "Dataset - Get Items" operation allows users to retrieve items from a specified dataset stored on Apify. This is useful when you want to programmatically access structured data collected by Apify actors or other processes.

Common scenarios include:

  • Fetching scraped data from an Apify dataset for further processing or analysis.
  • Automating workflows that depend on data extracted via Apify.
  • Integrating Apify data into other systems or dashboards within n8n.

Example: Retrieve the first 50 records from a dataset identified by its ID or name, then use this data downstream in your workflow.

Properties

Name Meaning
Dataset ID The unique identifier of the dataset or a string in the format username~dataset-name.
Offset Number of items to skip at the start of the dataset (default is 0).
Limit Maximum number of results to return (minimum 1, default 50).
Authentication Method of authentication to use: either an API Key or OAuth2 token for accessing Apify APIs.

Output

The node outputs JSON data representing the items retrieved from the specified Apify dataset. Each item corresponds to a record stored in the dataset and can contain arbitrary structured data depending on what was scraped or stored.

If the dataset contains binary data references, those would typically be represented as URLs or metadata; however, this node primarily returns JSON objects.

Output structure example (conceptual):

[
  {
    "field1": "value1",
    "field2": "value2",
    ...
  },
  ...
]

Dependencies

  • Requires an active connection to Apify via either an API key credential or OAuth2 authentication.
  • The user must have access rights to the specified dataset on Apify.
  • No additional external dependencies beyond the Apify API.

Troubleshooting

  • Invalid Dataset ID: If the dataset ID is incorrect or does not exist, the node will fail to retrieve data. Verify the dataset identifier format and existence.
  • Authentication Errors: Using invalid or expired credentials will cause authentication failures. Ensure the API key or OAuth2 token is valid and has necessary permissions.
  • Offset/Limit Issues: Providing negative numbers or zero for limit may cause errors. Use positive integers, with limit >= 1.
  • Empty Results: If offset exceeds the number of items in the dataset, the output will be empty. Adjust offset accordingly.

Links and References

Discussion