What is a webhook?

Carbon employs webhooks to promptly transmit real-time notifications regarding your Carbon actions. These webhooks consistently utilize HTTPS for secure data transmission and deliver JSON payloads, seamlessly integrable into your application. Leverage webhook feeds to streamline various tasks, including:

  • Retrieval of documents, chunks and embeddings.
  • Generation of alerts within messaging or incident management tools, triggered by specific event types.
  • Comprehensive storage of all transmission events in your database for tailored reporting and extended data retention.

Steps to receive webhooks

Currently, there is no way to choose which events to receive webhooks for - it’s on our roadmap to allow for more fine-grained filtering.

Set up the webhook

Jupyter Notebook

Please refer to this Jupyter notebook for code snippets that can be executed for setting up webhooks.

The steps in setting up webhooks for your service are:

  1. Add a URL to which webhoooks should be sent. This can be done using the /add_webhook endpoint.
  2. Save the signing_key in the response somewhere safe - this can’t be retrieved again.
You can create a tunnel to your localhost server using a tool like ngrok. For example: https://5f9f-98-14-198-85.ngrok-free.app/api/webhook

At this point, all events will be sent to the URL specified in Step 1. An event - sent via an HTTP POST request - contains two important elements, a Carbon-Signature header and a body with a single key-value pair. You can validate the authenticity and integrity of the webhook by calculating its signature.

Extract the timestamp and signature.

  1. Extract the timestamp and signature from the Carbon-Signature header. It will be of the form Carbon-Signature:t=1492774577,v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd.
  2. For those using JavaScript and decoding the request json via JSON.stringify, we suggest using the Carbon-Signature-Compact header instead. It will have the form Carbon-Signature:t=1492774577,v2=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd.

Secure your endpoint

You need to secure your integration by making sure your handler verifies that all webhook requests are generated by Carbon. You can verify webhook signatures using our official libraries or verify them manually.

You can manually verify the webhook or use our official webhook libraries.

Carbon requires the raw body of the request to perform signature verification. If you’re using a framework, make sure it doesn’t manipulate the raw body. Any manipulation to the raw body of the request causes the verification to fail.

Webhook Behavior

Once you receive and handle the webhook, you should respond to the POST request with a 200 status - otherwise the webhook will be retried (up to three times).

An organization is allowed up to 3 webhook URLs. To delete an existing URL, you can use the /delete_webhook endpoint. To view all existing webhooks, use the /webhooks endpoint.

All webhook payloads have the form

{
    "payload": str
}

The value of payload (which is a stringified json object) will always have the form

{
    "webhook_type": WebhookType,
    "obj": {
        "object_type": WebhookObjectType,
        "object_id": str,
        "additional_information": dict[str, (str | bool | list[str])] | str | None
    },
    "timestamp": str
}

The WebhookObjectType values are as follows:

class WebhookObjectType(StrEnum):
    FILE = "FILE"
    FILE_LIST = "FILE_LIST"
    UPLOAD_REQUEST_ID = "UPLOAD_REQUEST_ID"
    DATA_SOURCE = "DATA_SOURCE"
    ORGANIZATION_USER = "ORGANIZATION_USER"

Please note that the WebhookType refers to specific events outlined in the Event Types documentation. Presently, the WebhookObjectType is restricted to FILE.

The field additional_information consistently contains a dictionary of string-string key-value pairs. If empty, it defaults to the string value “null”.

The timestamp field is an integer POSIX timestamp presented as a string. To utilize object_id in subsequent API requests, it’s advisable to convert it into an integer. This object_id corresponds to the file_id of the file in question.

FAQ