NAiOS IconNAiOS Logo
Contacto
Volver al catálogo

Scrape Do

scrape_do

Scrape.do is a web scraping API offering rotating residential, data-center, and mobile proxies with headless browser support and session management to bypass anti-bot protections (e.g., Cloudflare, Akamai) and extract data at scale in formats like JSON and HTML.

AI Web ScrapingDeveloper ToolsWeb del proveedor
Acciones
19
Triggers
0
Autenticación
—
OAuth gestionado
No
Información técnica: el detalle de parámetros, schemas y triggers de esta página está pensado para equipos de integración. Si solo necesitas saber si tu herramienta favorita está disponible, basta con ver la lista de acciones.

Acciones disponibles (19)

Cada acción es una operación que el agente puede ejecutar contra este conector. Haz clic en una acción para ver sus parámetros.

Get Account InformationSCRAPE_DO_GET_ACCOUNT_INFOAcción

Retrieves account information and usage statistics from scrape.do. this action makes a get request to the scrape.do info endpoint to fetch: - subscription status - concurrent request limits and usage - monthly request limits and remaining requests - real-time usage statistics rate limit: maximum 10 requests per minute

Parámetros de entrada

  • tokenstringObligatorio

    Authentication token for Scrape.do API

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Get rendered page contentSCRAPE_DO_GET_RENDER_PAGEAcción

This tool allows you to scrape web pages with javascript rendering enabled. it's particularly useful for scraping dynamic websites where content is loaded through javascript. the tool will wait for the javascript to execute and return the fully rendered html content.

Parámetros de entrada

  • urlstringObligatorio

    The target web page URL to scrape

  • widthinteger

    Browser viewport width in pixels

  • devicestringenum

    Device type to emulate (desktop/mobile/tablet)

    desktopmobiletablet
  • heightinteger

    Browser viewport height in pixels

  • renderboolean

    Enable JavaScript rendering

  • timeoutinteger

    Maximum timeout in milliseconds (5000-120000)

  • waitUntilstringenum

    When to consider navigation succeeded

    domcontentloadednetworkidle0networkidle2load
  • customWaitinteger

    Additional wait time in milliseconds after page load (0-35000)

  • waitSelectorstring

    CSS selector to wait for before returning content

  • customHeadersboolean

    Enable custom header forwarding

  • blockResourcesboolean

    Block CSS and image resources

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Scrape webpage using scrape.doSCRAPE_DO_SCRAPE_DO_GET_PAGEAcción

A tool to scrape web pages using scrape.do's api service. it makes a basic get request to fetch the content of a target webpage while handling anti-bot protections and proxy rotation automatically.

Parámetros de entrada

  • urlstringObligatorio

    Target web page URL to scrape

  • superboolean

    Use residential & mobile proxy networks

  • widthinteger

    Browser viewport width

  • devicestring

    Specify device type (desktop, mobile, tablet)

  • heightinteger

    Browser viewport height

  • outputstring

    Output format (raw or markdown)

  • renderboolean

    Enable headless browser rendering

  • timeoutinteger

    Maximum request timeout in ms (5000-120000)

  • geo_codestring

    Choose country for target web page (e.g. 'us', 'gb')

  • return_jsonboolean

    Return network requests in JSON format

  • set_cookiesstring

    Set cookies for target web page

  • extra_headersboolean

    Add/modify headers

  • retry_timeoutinteger

    Maximum retry timeout in ms (5000-55000)

  • custom_headersboolean

    Handle all request headers

  • block_resourcesboolean

    Block CSS and image sources

  • disable_redirectionboolean

    Disable request redirection

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Use Scrape.do Proxy ModeSCRAPE_DO_SCRAPE_DO_PROXY_MODEAcción

This tool implements the proxy mode functionality of scrape.do, which allows routing requests through their proxy server. it provides an alternative way to access web scraping capabilities by handling complex javascript-rendered pages, geolocation-based routing, device simulation, and built-in anti-bot and retry mechanisms.

Parámetros de entrada

  • urlstringObligatorio

    The target URL to scrape

  • devicestring

    Device type to simulate (desktop, mobile, tablet)

  • renderboolean

    Enable/disable JavaScript rendering

  • geo_codestring

    Geographic location for the request (e.g., 'us', 'uk')

  • custom_headersboolean

    Whether to forward custom headers to the target website

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Cookies for ScrapingSCRAPE_DO_SCRAPE_DO_SET_COOKIESAcción

This tool allows users to set specific cookies for their scraping requests to a target website. it is useful for maintaining session states or authentication through cookies.

Parámetros de entrada

  • urlstringObligatoriouri

    Target web page URL where cookies will be set

  • cookiesstringObligatorio

    Cookie string in format 'name1=value1;name2=value2' (will be URL-encoded)

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Scrape.do Super ModeSCRAPE_DO_SCRAPE_DO_SET_SUPER_MODEAcción

The scrape do set super mode tool enables enhanced scraping by using residential and mobile proxies, bypassing blocks and restrictions associated with datacenter ips. when the 'super' parameter is set to true, it activates a mode that leverages a network of residential ip addresses, which is particularly useful to bypass strict anti-bot measures and for accessing websites that block datacenter ips.

Parámetros de entrada

  • super_modebooleanObligatorio

    Enable/disable Super Mode for enhanced scraping using residential and mobile proxies

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Block specific URLs during scrapingSCRAPE_DO_SET_BLOCK_URLSAcción

This tool allows users to block specific urls during the scraping process. it's particularly useful for blocking unwanted resources like analytics scripts, advertisements, or any other urls that might interfere with the scraping process or slow it down. it provides granular control by allowing users to specify url patterns to block, thereby improving scraping performance and maintaining privacy.

Parámetros de entrada

  • urlsstring[]Obligatorio

    List of URL patterns to block during scraping. Can be full URLs or patterns.

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set custom headers for scrape.do requestSCRAPE_DO_SET_CUSTOM_HEADERSAcción

A tool to send custom headers with scrape.do requests. this allows simulating specific browser behaviors or adding authentication headers by controlling all headers sent to the target website.

Parámetros de entrada

  • urlstringObligatorio

    Target web page URL to scrape

  • headersobjectObligatorio

    Dictionary of custom headers to send with the request

  • custom_headersboolean

    Enable custom headers mode (default: True)

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Custom Wait TimeSCRAPE_DO_SET_CUSTOM_WAITAcción

This tool sets the custom wait time in milliseconds after page load when using the render option in scrape.do. it is particularly useful for dealing with dynamic content to ensure that it is fully loaded before scraping, especially on javascript-heavy websites or single-page applications. the action allows fine-tuned control over the rendering wait time and must be used with render=true.

Parámetros de entrada

  • custom_waitinteger

    The time to wait in milliseconds after page load when using render option (between 0-35000 ms)

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Device Type for ScrapingSCRAPE_DO_SET_DEVICE_TYPEAcción

This tool allows users to set the device type (desktop, mobile, or tablet) for making scraping requests. it is used to emulate different devices, which helps in testing responsive designs or fetching device-specific content.

Parámetros de entrada

  • urlstringObligatorio

    The target URL to scrape

  • device_typestringObligatorio

    The type of device to emulate for scraping requests

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Disable RedirectionSCRAPE_DO_SET_DISABLE_REDIRECTIONAcción

Controls the automatic redirection behavior of scrape.do requests. when enabled (disable redirection=true), prevents the automatic following of redirects during web scraping operations. this allows: - inspection of the redirect chain - capturing intermediate redirect responses - manual control of redirection flow - analysis of http status codes of redirect responses the redirect url will be available in the scrape.do-target-redirected-location response header.

Parámetros de entrada

  • disable_redirectionboolean

    Whether to disable automatic redirection following. When true, prevents automatic following of redirects and allows inspection of redirect responses.

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Pure Cookies ModeSCRAPE_DO_SET_PURE_COOKIESAcción

This tool enables getting the original set-cookie headers from target websites instead of the processed scrape.do-cookies format. when enabled, this parameter returns the original set-cookie headers from the target website rather than using the default scrape.do-cookies header format.

Parámetros de entrada

  • pure_cookiesbooleanObligatorio

    When enabled, returns the original Set-Cookie headers from the target website instead of the processed Scrape.do-Cookies format.

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Regional Geolocation for ScrapingSCRAPE_DO_SET_REGIONAL_GEO_CODEAcción

This tool allows users to set a broader geographical targeting by specifying a region code instead of a specific country code. this is useful when you want to scrape content from an entire region rather than a specific country. note that this feature requires super mode to be enabled and is only available for business plan or higher subscriptions.

Parámetros de entrada

  • urlstringObligatorio

    The target URL to scrape with the specified regional geo code

  • regional_geo_codestringObligatorio

    The region code to target for scraping requests

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Retry TimeoutSCRAPE_DO_SET_RETRY_TIMEOUTAcción

This tool allows users to set the maximum wait time (in milliseconds) before retrying a failed request in scrape.do. it requires a parameter 'retry timeout' (integer) which specifies the maximum time to wait before retrying, with a default of 15000 ms. it is designed to improve the reliability of web scraping operations, especially when dealing with unstable or slow-responding websites.

Parámetros de entrada

  • retry_timeoutinteger

    The maximum time in milliseconds to wait before retrying a failed request (between 5000-55000 ms)

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Screenshot Capture for ScrapingSCRAPE_DO_SET_SCREENSHOTAcción

This tool enables the screenshot functionality for the scrape.do api, allowing users to capture a visual representation of the scraped webpage. when enabled, the api will return a screenshot of the rendered page along with the regular response. features: - basic screenshot capture - full page screenshot capture - capture specific area using css selector

Parámetros de entrada

  • urlstringObligatorio

    The URL of the webpage to take a screenshot of

  • enabledboolean

    Whether to enable screenshot capture for scraping requests

  • selectorstring

    CSS selector to capture specific area of the page

  • full_pageboolean

    Whether to capture full page screenshot

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Session ID for Sticky SessionsSCRAPE_DO_SET_SESSION_IDAcción

This tool implements the session id functionality for scrape.do to maintain a sticky session with the same proxy ip across multiple requests. it achieves this by adding a sessionid parameter to the query parameters of any scraping request, which is crucial for ensuring session consistency when scraping websites with stringent session requirements.

Parámetros de entrada

  • session_idintegerObligatorio

    An integer between 0 and 1,000,000 that will be used as the session identifier. The same session ID will maintain the same proxy IP for up to 5 minutes of inactivity.

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Wait For SelectorSCRAPE_DO_SET_WAIT_FOR_SELECTORAcción

This action allows setting a css selector to wait for before considering the page load complete. it is particularly useful when scraping javascript-heavy pages to ensure that certain elements have loaded dynamically.

Parámetros de entrada

  • timeoutinteger

    Maximum time to wait in milliseconds (between 1000 and 35000)

  • selectorstringObligatorio

    CSS selector to wait for in the target web page

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Set Wait Until ConditionSCRAPE_DO_SET_WAIT_UNTILAcción

This tool sets the waituntil parameter for the scrape.do api, defining when the rendering should consider the page loaded during javascript execution. it is particularly useful for handling dynamic websites by specifying conditions such as 'domcontentloaded', 'networkidle0', or 'networkidle2'.

Parámetros de entrada

  • wait_untilstringObligatorioenum

    The condition to determine when the page is considered loaded. 'domcontentloaded': Waits for DOMContentLoaded event. 'networkidle0': Waits until no network connections for 500ms. 'networkidle2': Waits until ≤2 network connections for 500ms.

    domcontentloadednetworkidle0networkidle2

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not

Monitor WebSocket requests using scrape.doSCRAPE_DO_SHOW_WEBSOCKET_REQUESTSAcción

This tool provides the ability to view websocket requests made by a webpage. it requires using render=true and returnjson=true parameters along with showwebsocketrequests=true to enable logging of websocket requests.

Parámetros de entrada

  • urlstringObligatorio

    Target web page URL to monitor websocket requests

  • timeoutinteger

    Maximum request timeout in ms (5000-120000)

  • session_idstring

    Optional session ID for maintaining state across requests

Parámetros de salida

  • dataobjectObligatorio

    Data from the action execution

  • errorstring

    Error if any occurred during the execution of the action

  • successfulbooleanObligatorio

    Whether or not the action execution was successful or not