BrightData icon

BrightData

Interact with Bright Data to scrape websites or use existing datasets from the marketplace to generate adapted snapshots

Actions14

Overview

The node interacts with the Bright Data platform, specifically focusing on the "Marketplace Dataset" resource and its "Get Dataset Metadata" operation. It allows users to retrieve metadata information about a selected dataset from Bright Data's marketplace. This is useful for scenarios where you want to understand the details or attributes of a dataset before using it in data processing or analysis workflows.

Practical examples include:

  • Fetching metadata to verify dataset contents before running data extraction.
  • Automating dataset selection by retrieving metadata dynamically within an n8n workflow.
  • Integrating dataset metadata retrieval as part of a larger data pipeline that uses Bright Data datasets.

Properties

Name Meaning
Dataset Select the dataset from the Bright Data marketplace to get metadata for. The dataset can be chosen from a searchable list of available datasets.

Output

The node outputs JSON data containing the metadata of the selected dataset. This metadata typically includes descriptive information such as dataset name, description, fields, update frequency, size, and other relevant attributes provided by Bright Data’s API.

If the node supports binary data output (not explicitly shown here), it would represent any associated files or snapshots related to the dataset, but this operation primarily focuses on metadata in JSON form.

Dependencies

  • Requires an API key credential for authenticating with the Bright Data API.
  • The node makes HTTP requests to https://api.brightdata.com.
  • The user must have access rights to the Bright Data marketplace datasets.
  • No additional environment variables are indicated beyond the API authentication.

Troubleshooting

  • Common issues:

    • Invalid or missing API credentials will cause authentication failures.
    • Selecting a dataset that no longer exists or is inaccessible may result in errors or empty responses.
    • Network connectivity problems can prevent successful API calls.

  • Error messages and resolutions:

    • Authentication failed: Verify that the API key credential is correctly configured and has necessary permissions.
    • Dataset not found: Confirm the dataset ID is valid and the dataset is available in the marketplace.
    • Timeout or network error: Check internet connection and API endpoint accessibility.

Links and References

Discussion