WebHook API

This API allows you to auto-import data into your application whenever fresh ones become available.


Registering a webhook

To register a webhook, you will need to provide a URL that accepts HTTP POST requests. You can provide it when you are creating or updating a data source.

Webhook url


The Webhook Request

A HTTP POST request is sent to the registered URL each time fresh data becomes available. This is how the body of the request looks like for a data source

POST HTTP/1.1
Content-Type: application/json

{
  "krake_name":     "Hacker News - News Headlines",
  "krake_handle":   "n3901_f3e4cbce5dccaf886b0b6f02b4d24509eses",
  "event_name":     "complete",
  "batch_time_sec": 1526515200,
  "batch_time_str": "2018-05-17 00:00:00",
  "data_url_json": "http://cache.getdata.io/n3901_f3e4cbce5dccaf886b0b6f02b4d24509eses/1526515200_page_1.json",
  "data_url_csv": "http://cache.getdata.io/n3901_f3e4cbce5dccaf886b0b6f02b4d24509eses/1526515200_all.csv"
}

Attributes in the payload
krake_name String The name of your data source
krake_handle String The handle of your data source
event_name String The event that triggered off this webhook. Types include:
  • complete - when crawling was successfully completed
  • aborted - when crawling was aborted by the user
batch_time_sec Integer The Unix time stamp for when this batch of crawling started
batch_time_str String The human readable time stamp for when this batch of crawling started
data_url_json String The URL to the cache to download the first batch of results in JSON format
data_url_csv String The URL to the cache to download the entire batch of results in CSV format

Testing Your Webhook

You can test your integration with your registered webhook when a batch of data is available. To do so, simply click on test button of your data source

Webhook test


Next Steps

To find out more about how GetData.IO works