Get started with Monitors

Turn a Bigdata Search into a recurring feed of structured, source-backed events.

Monitors turn a Bigdata Search into a recurring feed of structured events. Describe what you want to track, choose how often to search, and define the information you want returned. Each event links back to the source content.

You can test a monitor against recent historical data before turning it on. This lets you check the search results, extracted fields, and duplicate detection immediately instead of waiting for new content to arrive.

How Monitors work

1. Describe what you want to track

   Save a search, a schedule, and the fields you want to extract. New monitors should start as inactive so they cannot produce scheduled results before you test them.

2. Test it with historical data

   Run a simulation over recent data. The simulation uses the same configuration as the live monitor but does not activate it.

3. Review the events

   Check the results and their source links. Refine the search or extraction instructions if important developments are missing or irrelevant ones appear.

4. Turn on continuous monitoring

   Change the monitor’s status to active. Bigdata then evaluates it automatically at the frequency you selected.

5. Use the results

   Retrieve completed runs and send their structured events to your application, alerting system, or research workflow.

Test before activating. A short simulation makes it much easier to spot a search that is too broad, missing results, or extracting the wrong information.

Create your first monitor

This example watches for credit rating downgrades and negative watch placements affecting BlackRock. It is inactive when created and will run once per hour after activation.

Only intent and schedule are required to create a monitor. The remaining fields let you customize its search and output:

• name: a short label that helps you recognize the monitor
• intent: what counts as a relevant event; this is set at creation and cannot be changed
• status: active or inactive. active also queues the first run
• search_queries: the search to run on each check. Fast mode takes your query and filters. Smart mode builds the search from a question. See Search query
• schedule: how often an active monitor searches. Use a positive number of hours, days, or weeks, such as 1h, 2d, or 1w. The minimum is 1 hour
• structured_output: the fields returned with each event
• extraction_instructions: what to include and exclude
• watchlists: groups of entity IDs to track

curl --request POST \
  --url https://api.bigdata.com/v1/search-monitoring/monitors \
  --header 'Content-Type: application/json' \
  --header 'X-API-KEY: <your-api-key>' \
  --data '{
    "name": "Credit Rating Downgrade or Negative Watch",
    "intent": "Watch for rating downgrades and negative watch placements affecting BlackRock",
    "status": "inactive",
    "search_queries": {
      "search_mode": "fast",
      "query": {
        "text": "credit rating downgrade negative watch",
        "max_chunks": 100
      }
    },
    "schedule": {
      "frequency": "1h"
    },
    "structured_output": {
      "company": {
        "description": "The company affected by the rating action.",
        "field_type": "entity_reference"
      },
      "rating_agency": {
        "description": "The credit rating agency announcing the action.",
        "field_type": "text"
      },
      "action": {
        "description": "The announced credit rating action.",
        "field_type": "enum",
        "values": [
          "downgrade",
          "negative_watch"
        ]
      },
      "new_rating": {
        "description": "The new rating, when stated.",
        "field_type": "text"
      }
    },
    "extraction_instructions": "Only return a new event when a rating agency announces a downgrade or places the company on negative watch. Do not treat analyst recommendations as credit rating actions.",
    "watchlists": [
      {
        "entities": ["HJ95FV"]
      }
    ]
  }'

The response includes the new monitor in data. Save data.id; you will use it to test, update, and retrieve the monitor.

A watchlist is a group of Bigdata entity IDs. Use the Knowledge Graph API to find the ID for each company you want to track. At most 100 IDs are kept across all groups.

Test the monitor

A simulation replays the monitor over a past window without turning it on. In this example, an hourly monitor is replayed from 06:00 to 12:00 UTC on August 27, 2026. The window is split into frequency-sized runs, and the first run can look back up to one frequency before start_timestamp. A replay can request at most 90 runs.

curl --request POST \
  --url 'https://api.bigdata.com/v1/search-monitoring/monitors/<monitor-id>/simulate?start_timestamp=2026-08-27T06:00:00Z&end_timestamp=2026-08-27T12:00:00Z' \
  --header 'X-API-KEY: <your-api-key>'

end_timestamp is required. Both timestamps are ISO 8601 with a timezone, and end_timestamp must be at least five minutes in the past. Omit start_timestamp to replay one period ending at end_timestamp.

The API returns 202 Accepted because the test keeps running. Save simulation_id, then read the runs:

curl --request GET \
  --url 'https://api.bigdata.com/v1/search-monitoring/monitors/<monitor-id>/runs?type=simulation&simulation_id=<simulation-id>&include_events=true' \
  --header 'X-API-KEY: <your-api-key>'

Each run reports a status:

• PENDING: still processing
• COMPLETED: ready to review
• FAILED: could not be completed

Start continuous monitoring

When the simulation produces the events you expect, turn on the monitor by changing its status:

curl --request PATCH \
  --url https://api.bigdata.com/v1/search-monitoring/monitors/<monitor-id> \
  --header 'Content-Type: application/json' \
  --header 'X-API-KEY: <your-api-key>' \
  --data '{
    "status": "active"
  }'

To pause future runs without losing the configuration, change the status back to inactive.

Key concepts

Monitor

A monitor combines a saved search, extraction instructions, an output format, and a schedule. An active monitor evaluates newly available content automatically.

Simulation

A simulation tests the monitor against historical data without activating it. Use simulations to improve the quality of your results before sending them into a live workflow.

Run

A run is one evaluation period. For an hourly monitor, each run searches one hour of newly available content. Runs can come from an active monitor or a simulation.

Event

An event is a relevant development extracted during a run. It contains a summary, your structured fields, and links to the source content that supports it.

In API responses, events appear in the events array.

Search query

Send a single search query, or a list of search queries when the monitor should check more than one. The list repeats the same object, with search_mode, query, and response_detail on every item.

Fast mode runs the query you write. It is the default:

{
  "search_mode": "fast",
  "query": {
    "text": "executive departure",
    "max_chunks": 100,
    "filters": {
      "entity": { "any_of": ["HJ95FV"] }
    }
  },
  "response_detail": "standard"
}

text is 4 to 1400 characters. Use texts instead when you need up to five phrases, not both. A fast search needs text or at least one filter. max_chunks is 0 to 1000. response_detail is light, standard, or extended.

filters is the same object as in Search documents. Every filter is also listed on Create a monitor, so you can try it in the playground. The monitor already picks the time window for each run, so leave timestamp empty unless you want a narrower slice.

To add web results, set external_search.mode to INCLUDE or ONLY and values to web. ONLY needs text, and it is not available in smart mode.

Smart mode builds the search from your question. It does not take filters:

{
  "search_mode": "smart",
  "query": {
    "text": "executive departures at large asset managers",
    "max_chunks": 100
  },
  "response_detail": "standard"
}

When you read the monitor back, the search is one object and unused fields are null.

Structured output

Choose the fields you want on every event. Include at least one entity_reference, or a field named reference_period. The monitor uses that field to tell a new event from one it has already reported.

Field type	What you get	values
entity_reference	The company or other entity, as name and rp_entity_id, or null	Leave out
reference_period	The period named in the source. Name this field reference_period	Leave out
enum	One of the values you listed, in the source’s own wording	Required
ordered_enum	One of the values you listed, where the order matters	Required
number	A numeric fact, in the source’s own wording	Leave out
boolean	True or false	Leave out
text	Free text. This is shown on the event and is not used to detect duplicates	Leave out

Do not use string, datetime, list, range, or currency. These names are reserved: announcement_date, effective_date, event_text, date_basis, summary, source_item_ids, and retract.

Delivery

A new monitor does not send email. To email yourself, update the monitor and set delivery.type to email. The message goes to the address on your account. Send null to turn email off.

What a change does

• Turning a monitor on queues one run. Sending active again does not queue another.
• Changing the search, output fields, watchlists, or extraction instructions clears run history. Later matches are treated as new.
• Changing the name, status, schedule, or email delivery keeps that history.
• You cannot delete a monitor while a run is still in progress. Set status to inactive if you only want to pause it.

What you can monitor

Once the first monitor is working, the same workflow can support:

M&A Activity Monitor

Track announced acquisitions, divestitures, bids, and material deal-status changes.

Central Bank Policy Decisions

Capture rate decisions and changes in forward guidance from major central banks.

Executive Departures

Detect confirmed departures, retirements, and leadership transitions.

Regulatory & Legal Actions

Watch for investigations, enforcement actions, judgments, and settlements.

Layoffs & Restructuring

Identify announced workforce reductions and major operating restructurings.

AI Product & Model Launches

Follow commercially relevant AI launches, releases, and availability changes.