Skip to main content

ScrapeGraph AI

ScrapeGraph AI is a service that provides AI-powered web scraping capabilities. It offers tools for extracting structured data, converting webpages to markdown, and processing local HTML content using natural language prompts.

Installation and Setup​

Install the required packages:

pip install langchain-scrapegraph

Set up your API key:

export SGAI_API_KEY="your-scrapegraph-api-key"

Tools​

See a usage example.

There are four tools available:

from langchain_scrapegraph.tools import (
    SmartScraperTool,    # Extract structured data from websites
    MarkdownifyTool,     # Convert webpages to markdown
    LocalScraperTool,    # Process local HTML content
    GetCreditsTool,      # Check remaining API credits
)

Each tool serves a specific purpose:

  • SmartScraperTool: Extract structured data from websites given a URL, prompt and optional output schema
  • MarkdownifyTool: Convert any webpage to clean markdown format
  • LocalScraperTool: Extract structured data from a local HTML file given a prompt and optional output schema
  • GetCreditsTool: Check your remaining ScrapeGraph AI credits

Was this page helpful?