Realtime Transcription

Real-time transcription allows you to receive live transcript data as the meeting progresses, rather than waiting for the meeting to end. This is enabled by using any of the realtime transcription models.

Key Features

• Live transcript streaming via WebSocket connection
• Immediate access to spoken content as it happens
• Suitable for applications requiring real-time processing

How it works

After creating a new meeting bot with a realtime transcription model, you'll receive websocket_url & websocket_read_only_url (both provide the same information, read_only_url denies any Actions, this is therefor safe to provide to customers front-end directly). Simply connect to the websocket url and you'll start receiving live updates. Keep in mind that old events will not be pushed through, only live events are provided.

Websocket Events

Events will be websocket messages as strings. Parse these to JSON to receive the payload. The JSON will always be structured as follow;

{
  "type": "[event]",
  "data": {...}
}

Ping

This event is simply making sure the connection stays alive. We're pinging all active clients once every minute. Does not contain much interesting information so may be ignored.

{
  "type": "ping"
}

Start

Once the meeting bot has joined the meeting and started recording, it'll send this event.
Just to inform you that from now on you'll start receiving transcripts. Refer to Bot Lifecycle Documentation for a better understanding of the joining process

{
  "type": "start"
}

Transcript

This event contains the live transcript of the meeting as well as timestamps and speaker information. speaker_name may be null at the start, this is because we're live calculating which name belongs to which speaker id. After a couple of sentences speaker_name should be provided.

{
  "type": "ts",
  "data": {
    "transcript": "This contains the spoken text.",
    "start": 1.23,
    "end": 4.56,
    "speaker": 0,
    "speaker_name": "John Doe"
  }
}

Chat Message

This event triggers when a new chat message is sent

{
  "type": "chat-message",
  "data": {
    "username": "John Doe",
    "content": "Foo bar.",
    "user_avatar": null // Either a URL or null. Do not rely on this field for permanent access.
  }
}

Participant Tracked

Whenever we detect a new participant in the meeting, we'll throw the participant-tracked event. This gets triggered for all existing participants at the start of the meeting.

{
  "type": "participant-tracked",
  "data": {
    "participantId": "John Doe",
    "participantName": "John Doe"
  }
}

Started Speaking

When we detect a participant as actively speaking, we'll throw this event.

{
  "type": "started-speaking",
  "data": {
    "participantId": "John Doe",
    "participantName": "John Doe"
  }
}

Stopped Speaking

When we detect a participant has stopped speaking, we'll throw this event.

{
  "type": "stopped-speaking",
  "data": {
    "participantId": "John Doe",
    "participantName": "John Doe"
  }
}

Stop

Once the meeting is over and therefor the recording has stopped, this event will be fired. Informing you that no further transcripts will be shared and you're safe to disconnect from the websocket server.

{
  "type": "stop"
}

Error

If anything goes wrong with our transcription provider, then this error will event will be fired. This can be considered the same as stop, no further transcription will be shared. We will have been alerted right away to solve the issue.

{
  "type": "error",
  "data": {
    "message": "Error message"
  }
}

Websocket Actions

You can also interact with the bot via the websocket to provide feedback as soon as possible. This once again works via a pre-defined JSON structure you send as a message. Examples below will contain JSON, but keep in mind that websocket messages are strings. So you'll need to parse this JSON to string before sending. Validation is complicated with websocket events. So at this point in time there is no validation, if your action does not contain the required data then the action will not be performed.

{
  "action": "[action]",
  "data": {...}
}

Send Chat Message

Via this action you can send a message to the chat of the meeting.

{
  "action": "chat-message",
  "data": {
    "content": "Welcome to the meeting!"
  }
}

Stop the bot

This will simply stop the bot.

{
  "action": "stop"
}

More events coming soon. If you have specific feature requests, please let us know via Discord.