Monitoring

Monitor a dataset for real-time updates and receive the changes immediately into the channel of your choice. Stay up to date with the information you need the most.

Monitoring is like an alarm system that alerts you to any activity that occurs in the datasets you want to watch. Let's take a look at how to set up a monitoring rule.

After you have saved your dataset, you can set up a monitoring rule by clicking on the blue Monitor button on the top right-hand side of your screen.

monitoring1

Creating a monitoring rule

Follow these steps to create a monitoring rule:

  1. Select what to monitor
    The first step allows you to select what you want to monitor. Do you want to get notified of new websites matching your dataset fields? Or updates when a website changes its address or payment method or perhaps when records have been removed and no longer match your criteria? 

    Monitoring-Dataprovider-com (1)
  2. Destination
    Now that you have selected the criteria you want to monitor, when an activity occurs, the insights will be sent to the channel of your choice using a webhook. With a webhook we can easily send new or changed data to your system. The data will be sent as a JSON object.

    Monitoring-Dataprovider-com (2)
  3. Select fields

    Choose the fields that you want to monitor for any changes. You can check the boxes per category or select a category and choose specific fields. Also, you can search for a field using the search bar. 


    Monitoring-Dataprovider-com (3)
  4. Adjust output settings
    Lastly, you should provide a couple of details to get the Monitor running. Insert a unique URL, like www.yourcompany.com/webhook, to receive the data (in a JSON format) and add an extra layer of security by using the custom headers option to identify your request. You can also set up a rate limit or a records restriction per month.
    Monitoring-Dataprovider-com (4)
    Don't forget to test the monitoring rule by clicking on Send test data: the test checks if your webhook is working and ensures you will receive the data.

    JSON object layout
    Depending on the event type you have selected in Step 1 ('What to monitor'), the JSON object will contain the following elements:
    "event", "datasetId", "hostname" and "data".

    For new websites:
    {
    "event": "new",
    "datasetId": 12345,
    "hostname": "testhostname.com",
    "data": {
    "country": "US"
    }
    }
    For changed websites:
    {
    "event": "change",
    "datasetId": 12345,
    "hostname": "testhostname.com",
    "data": {
    "new": {
    "country": "US",
    "region": null
    },
    "old": {
    "country": "NL",
    "region": "Groningen"
    }
    },
    "dataFull": {
    "changedfields": [
    "country",
    "region"
    ],
    "hostname": "testhostname.com",
    "country": "US"
    }
    }

    For removed websites:

    {
    "event": "remove",
    "datasetId": 12345,
    "hostname": "testhostname.com",
    "data": null
    }
    The "data" element tells you what is new or has been changed (new vs. old).
    For changed websites there is an extra element called "dataFull". This element resends the complete set of fields of the changed website.

  5. Start monitoring
    Check the summary, save the monitoring rule and you are all set!

Editing a monitoring rule

If you go back to the dataset, you will notice a yellow notification icon on the Monitor button.

monitoringrule

By clicking on the Monitor menu, you can add or edit a new monitoring rule and delete an existing one.

monitoringrule2-1

Deleting a monitoring rule

If you want to delete a monitoring rule, click on the respective rule. The red Delete button is located at the bottom of the page.

Monitoring-Dataprovider-com_delete

Deleting your monitoring rule can't be undone. Data will no longer be sent to the selected channel.