Batch Search

Process large volumes of search queries asynchronously at a 50% cost reduction.

Overview

The Batch Search API lets you submit a file containing multiple search queries and retrieve all results once processing is complete. Instead of making thousands of individual API calls, you upload a single .jsonl file, poll for completion, and download the results.

Ideal for financial institutions that want to stay up to date on the latest news and events related to their entire portfolio.

For example, you can schedule a daily batch of queries to run early in the morning, retrieve all the information that could impact your portfolio, and present it to your team right when they start their day.

Key benefits

✅ 50% cost reduction: $0.0075 per query unit vs $0.015 in fast mode
✅ No client-side scaling: no rate limiting, connection pooling, or retry logic
✅ Single upload, single download: one .jsonl file in, one .jsonl file out
✅ Same query capabilities: supports all Search API filters of the fast search mode

How it works

The Batch Search workflow has four steps:

1. Create a batch job

   The endpoint will return a batch_id and a presigned_url for uploading your input file.

2. Upload your input file

   Upload your .jsonl file to the presigned_url.

3. Poll for completion

   Check the job status using the batch_id. The status will progress through pending → processing → completed. During this time, you can monitor the estimated_completion_time which is a dynamic ISO 8601 timestamp that continuously updates to reflect when the job is expected to finish.

4. Download results

   Once the status is completed, the response includes an output_file_url to download your results.

Input file format

Your input file must be in .jsonl format (JSON Lines), where each line is a JSON object containing a query field with a standard Search Documents request body.

{"query": {"text": "Impact of tariffs on semiconductor industry", "filters": {"timestamp": {"start": "2026-01-01", "end": "2026-02-24"}}, "max_chunks": 10}}
{"query": {"text": "Central bank interest rate decisions", "filters": {"timestamp": {"start": "2026-01-01", "end": "2026-02-24"}}, "max_chunks": 10}}
{"query": {"text": "Oil supply disruptions", "filters": {"timestamp": {"start": "2026-01-01", "end": "2026-02-24"}}, "max_chunks": 10}}

Each line is processed as an independent search request, so you can use different filters, topics, and time ranges across queries within the same batch.

Output file format

The output file is also in .jsonl format. Each line contains:

Field	Description
status	success, error, timeout, or exception
response	The Search response for this query
line_number	Matches the line number from your original input file
query	The original search query you submitted
error	Error description (only present if the request failed)

Output file example

The output file contains a JSON object for each line of the input file:

Try it out

Postman

We provide a ready-to-use Postman collection that includes all Batch Search endpoints pre-configured.

Download Postman Collection

Get the Postman collection and environment files from our GitHub repository.

The collection includes four requests:

1. Create pre-signed URL to upload the batch: Creates a new batch job
2. Upload search batch: Uploads your .jsonl file
3. Get batch status: Checks the current status
4. Get response: Downloads the results

Import both the collection (.postman_collection.json) and the environment (.postman_environment.json) files into Postman. Set your API key in the environment variables before running the requests.

Python

We provide a ready-to-use Python script that runs the complete Batch Search workflow end to end.

Download Python Script

Get the Python script from our GitHub repository.

1. Install dependencies

pip install requests

2. Set your API key

export BIGDATA_API_KEY="your-api-key"

3. Run the script

python how-to-guide_batch-search.py

The script will create an example input file with three search queries, schedule the batch, monitor its status, and download the results once processing completes.

Related resources

• Batch Search cookbook: Real-world use case of Batch Search
• Batch Search API Reference: Official API reference