Skip to main content

​
Overview

The Bigdata Content API lets you manage and query private content that you or your organization uploads to Bigdata, making it available for search and chat. You can onboard content in two ways:
  • Connectors API: For asynchronous, unsupervised, or automated workflows where content is not directly accessible by the user (e.g. email inbox, SharePoint). You configure a connector and the service ingests the content from that source.
  • Direct upload (POST /documents): For clients that manage their own corpus and want to build the ingestion workflow manually. You request a pre-signed URL and document id; you upload each file via PUT to that URL, then use the id to poll for processing status.
Both paths produce documents that you can list, tag, filter, and use with the Search and Research Agent Services. You can create and manage Connectors in the Developer Platform > Connectors. Developer Platform > Connectors

​
Connectors

Connectors are for ingestion that is asynchronous, unsupervised, or part of a workflow where content is not directly in the user’s hands, for example, an email inbox or a SharePoint library.
  • You create a connector with a type, label, and type-specific configuration. Supported types include Email Inbox and (coming soon) Microsoft SharePoint.
    • For Email, the API returns an inbox address; you forward messages from allowed senders to that address. Set up automatic forwarding rules in your email client to start building your corpus, Bigdata will ingest and process content automatically.
  • Content arriving through the connector is turned into documents you can manage and use via Search and Research Agent Services.
Typical operations:
  • Create connector: Register a new ingestion source. For email connectors, the response includes a connector ID and an inbox address; forward messages from allowed senders to that address to ingest content into Bigdata.
  • List connectors: List all connectors you can access; use connector IDs to filter documents or to update/delete.
  • Get connector: Fetch full details for one connector (config, share settings, timestamps).
  • Update connector: Change label, description, sharing, or configuration of the connector.
  • Delete connector: Soft-delete (archive) a connector by default—it stops ingesting new content but existing documents remain. Use ?force_delete=true to trigger a hard delete: all files related to the connector are deleted asynchronously, then the connector is removed from the database.
Once a connector is set up, documents are ingested according to its configuration.

​
Documents

Documents are the items Bigdata has processed and made available for search and analysis, whether they came from a connector (e.g. email, SharePoint) or from direct upload. Direct upload is intended for clients that manage their corpus and want to build the ingestion workflow from scratch: you choose what to send, when, and with what metadata (file name, tags, sharing). Use Upload document to get a pre-signed URL and document id; PUT the file to that URL, then use the id with Get document to poll for status until the document is ready. The following operations apply to all documents, whether they were ingested via a connector or direct upload:
  • List documents: Paginated list of documents you can access. Supports filtering and sorting; each item includes a content ID for use with the other document endpoints.
  • Get document: Return metadata for one document by content ID (status, file name, connector, tags, timestamps). Use this to check processing status before downloading content.
  • Get annotated document: Return a time-limited pre-signed URL. GET that URL to download the document as structured JSON (metadata, title, body blocks, entities, sentences with sentiment). Use for search indexing, entity extraction, or structured display.
  • Get original document: Return a time-limited pre-signed URL. GET that URL to download the original file in its native format (e.g. .eml, .pdf).
  • Delete document: Remove a document from the platform. Deletes the original document, annotated version, and chunks from the vector database.
Documents can be tagged (e.g. by sender/recipient for emails). Use List tags to discover tag names and List documents with the tags parameter to filter by them.

​
Typical flow

Using connectors (async / unsupervised):
  1. Create a connector (e.g. email) with type, label, and config; for email, use the returned inbox address to forward messages.
  2. Optionally check that content has been processed by listing your recent documents.
  3. Use your content in the Search or Research Agent Service (e.g. filter by Emails in the playgrounds below).
Using direct upload (manual corpus):
  1. Call Upload document with metadata (file name, tags, sharing); use the returned URL to PUT the file.
  2. Poll Get document with the returned id to check status until it is completed. See Get document in the API reference.
  3. Use your content in the Search or Research Agent Service (e.g. filter by My Files in the playgrounds below).

Search Service Playground

Search across your private content and other sources. In the playground, open the source selector and choose My Files to limit results to your uploaded documents.

Research Agent Playground

Run research over your private content and real-time data. In the playground, use the source selector and filter by My Files to ground answers in your documents.

​
Authentication and reference

All endpoints require an API key sent in the X-API-KEY header. See Authentication for details. Use the API reference (endpoints listed in this section) for request/response schemas, parameters, and examples. You can try the API from the Developer Platform playgrounds.