crawlapi


Updated: Tuesday, 02. June 2020 03:42 PM UTC


Base API package

The base API package starts at $75/month. Contact us to subscribe. Currently, we only allow customers that build and run their scrapers with us.

Features:

  1. You don't need to worry about proxy rotation. You only pay for threads and we manage the proxy rotation. Our API is Backed by a mix of residential and datacenter proxy pools that use backconnect rotating proxy. Proxy changes on every request.
  2. You don't need to worry about user-agent/header rotation. We rotate user agents and request headers with different browser signature on every request.
  3. Invalid pages such as try again later, captcha page, something went wrong, etc. are handled for you. Our API provides a JSON output and you only watch for a success response. A success response should always give you a valid page.
  4. US geolocation targetting.
  5. JS rendering usually takes a lot of threads (async). A basic JS rendering is provided with the base package which is managed by us. We have to make sure that the JS request is minimal enough so that it does not eat up all of our allocated threads.
  6. Unlimited API call. If you want to have this feature with other providers, you are gonna have to spend a lot. We make some compromises and use a thread-based usage model to provide unlimited API call for clients who need to run scrapers 24 hours a day and 7 days a week.

Addon package for CrawlAPI

No. of threads