How to develop your OpenVidu application#

This page is a collection of the most common operations you may want to perform in your application while integrating OpenVidu. Depending on the scope of the operation, these operations will be performed on the client side using a LiveKit Client SDK, or on the server side using a LiveKit Server SDK (or directly using the HTTP server API). Consider the architecture of an OpenVidu application:

You can use this page as a cheat sheet to know at a glance how to do something, and you have links to the LiveKit reference documentation of each operation for a more detailed explanation.

Generate access tokens#

The application client needs an access token to connect to a Room. This token must be generated by the application server. Visit LiveKit reference documentation to learn how to generate access tokens:

Manage Rooms#

Connect to a Room#

To connect to a Room you need the URL of your OpenVidu deployment (which is a WebSocket URL) and the access token generated by your application server.

import { Room } from "livekit-client";

const room = new Room();
await room.connect(wsUrl, token);

Disconnect from a Room#

await room.disconnect();

Publish a Track#

You can directly publish the default camera and microphone of the device using methods setCameraEnabled and setMicrophoneEnabled of the LocalParticipant object:

// Publish a video track from the default camera
await room.localParticipant.setCameraEnabled(true);
// Publish an audio track from the default microphone
await room.localParticipant.setMicrophoneEnabled(true);

It is also possible to publish both of them at the same time using method LocalParticipant.enableCameraAndMicrophone, which has the advantage of showing a single permission dialog to the user:

// Publish both default video and audio tracks triggering a single permission dialog
await room.localParticipant.enableCameraAndMicrophone();

To craft a custom Track, you can use the LocalParticipant.createTracks method and publish them with LocalParticipant.publishTrack:

var tracks = await room.localParticipant.createTracks({
  audio: {
    deviceId: "default",
    autoGainControl: true,
    echoCancellation: true,
    noiseSuppression: true
  },
  video: {
    deviceId: 'frontcamera';
    facingMode: 'user'
  },
});
await Promise.all([
    room.localParticipant.publishTrack(tracks[0]),
    room.localParticipant.publishTrack(tracks[1]),
]);

Mute/Unmute a Track#

To mute the default camera and microphone Tracks:

await room.localParticipant.setCameraEnabled(false);
await room.localParticipant.setMicrophoneEnabled(false);

To mute/unmute a custom Track:

// Mute the track
await track.mute();

// Unmute the track
await track.unmute();

Unpublish a Track#

To completely stop sending a Track to the Room, you must unpublish it:

await room.localParticipant.unpublishTrack(track, true);

The second boolean parameter indicates if the local Track should be stopped. This usually means freeing the device capturing it (switching off the camera LED, for example).

Send messages#

You can share information between Participants of a Room in different ways.

First of all, you can set Room metadata that will be available for all clients in the Room object. You do so in your application server when calling the CreateRoom API (available for all LiveKit Server SDKs and the HTTP Server API). The Room metadata will be available in the client side like this:

console.log(room.metadata);

You can update the Room metadata at any time from your application server with the UpdateRoomMetadata API (available for all LiveKit Server SDKs and the HTTP Server API). The client side will be notified of the change with the roomMetadataChanged event of the Room object:

room.on("roomMetadataChanged", (metadata) => {
  console.log("New room metadata", metadata);
});

Secondly, you can also set Participant metadata. You do so when creating an access token in your application server, setting metadata field of the JWT.

Participants can also update their own metadata from the client side, if their access token was created with grant canUpdateOwnMetadata.

room.localParticipant.setMetadata("new metadata");

The client side will be notified of the change with the participantMetadataChanged event of the Room and/or Participant object:

// To handle all metadata changes of all participants
room.on(
  RoomEvent.ParticipantMetadataChanged,
  (previousMetadata: string, participant) => {
    console.log(
      "New metadata for participant",
      participant.identity,
      participant.metadata
    );
  }
);

// To handle only metadata changes of a specific participant
participant.on(
  ParticipantEvent.ParticipantMetadataChanged,
  (previousMetadata) => {
    console.log(
      "New metadata for participant",
      participant.identity,
      participant.metadata
    );
  }
);

Finally, you can send messages to Participants in the Room using the LocalParticipant.publishData method:

const data: Uint8Array = new TextEncoder().encode(JSON.stringify(""));
room.localParticipant.publishData(data, {
  reliable: true,
  topic: "chat",
  destinationIdentities: ["participant-identity"],
});

The DataPublishOptions allow setting the reliability of the message (depending on the nature of the message it can be sent as a reliable or lossy message), a topic to easily filter messages, and the participants that will receive the message.

The client side will be notified of the message with the dataReceived event of the Room and/or Participant object:

// To receive all messages from the Room
room.on(
  RoomEvent.DataReceived,
  (payload: Uint8Array, participant: Participant, kind: DataPacket_Kind) => {
    const strData = new TextDecoder().decode(payload);
    console.log("Received data from", participant.identity, strData);
  }
);

// To receive messages only from a specific participant
participant.on(
  ParticipantEvent.DataReceived,
  (payload: Uint8Array, kind: DataPacket_Kind) => {
    const strData = new TextDecoder().decode(payload);
    console.log("Received data from", participant.identity, strData);
  }
);

From your application server#

Except for the generation of access tokens, it is possible for all the logic of your application to be contained entirely on the client side. Nonetheless, some use cases may require the management of the Rooms from the server side.

These operations are only available in the server SDKs, and not in the client SDKs:

Closing a Room. From the client side, a user can only leave his own Room.
Removing any Participant from a Room. From the client side, a user can only leave his own Room.
Muting any Track of any Participant. From the client side, a user can only mute/unmute his own Tracks.
Updating the metadata of any Participant. From the client side, a user can only update his own metadata.
Updating the metadata of the Room. From the client side this is not possible.
Egress operations. Egress cannot be started and stopped on demand by users from the client side.
Ingress operations. Ingress cannot be started and stopped on demand by users from the client side.

You have here the complete list of the server-side operations, documented for the HTTP Server API. All the LiveKit Server SDKs have the same operations.

RoomService: to manage Rooms and Participants.
Egress: to manage egress operations.
Ingress: to manage ingress operations.

To quickly publish a screen sharing Track:

await room.localParticipant.setScreenShareEnabled(true);

You can also create custom screen tracks, for example capturing the audio of the screen and fine-tuning the video capture options (checkout the ScreenTrackOptions interface for detailed information):

const screenTracks = await room.localParticipant.createScreenTracks({
  audio: true,
  contentHint: "detail",
  preferCurrentTab: true,
  video: {
    displaySurface: "window",
  },
});
await Promise.all([
  room.localParticipant.publishTrack(screenTracks[0]),
  room.localParticipant.publishTrack(screenTracks[1]),
]);

Virtual Background#

It is possible to apply a virtual background to video tracks. In this way you can blur the background or replace it with an image.

It is necessary to install an additional dependency to use this feature:

npm add @livekit/track-processors

To blur the background:

import { BackgroundBlur } from "@livekit/track-processors";

const videoTrack = await createLocalVideoTrack();
const blur = BackgroundBlur(10);
await videoTrack.setProcessor(blur);

To replace the background with an image:

import { VirtualBackground } from "@livekit/track-processors";

const videoTrack = await createLocalVideoTrack();
const image = VirtualBackground("https://picsum.photos/400");
await videoTrack.setProcessor(image);

Recording#

You can record your Rooms using the Egress module. Egress allows exporting media from a Room in different formats, including

Room Composite Egress: a single video output with all the Tracks of a Room composited in a layout. You can even create your custom layouts.
Track Composite Egress: a single video output combining an audio Track and a video Track.
Track Egress: individual outputs for each Track of a Room.

Visit the LiveKit reference documentation for a detailed explanation of Egress:

Stream ingestion#

You can ingest media streams into your Rooms using the Ingress module. It supports different sources, including:

RTMP: the Ingress module exposes an RTMP endpoint to which your user can stream their content. The ingress module will transcode and publish the stream to the Room, making it available to all participants.
WHIP: the Ingress module exposes a WHIP endpoint to which your user can stream their content directly via WebRTC. You can choose whether the Ingress module should transcode the stream or directly relay it to the Room. Avoiding transcodification is the best option to minimize latency when ingesting media to a Room.
Media files serve by an HTTP server: the Ingress module will fetch a media file, transcode it and publish it to the Room.
Media served by a SRT server: the Ingress module will pull the media from an SRT server, transcode it and publish it to the Room.

Visit the LiveKit reference documentation for a detailed explanation of Ingress:

IP Cameras#

With OpenVidu you can ingest RTSP streams into your Rooms. To do so, simply use the Ingress API to create and ingress of input type URL, providing the IP camera RTSP URL as value:

Node.js Go Ruby Java Rust PHP .NET Server API

Using LiveKit Node SDK

import { IngressClient, IngressInfo, IngressInput } from 'livekit-server-sdk';

const ingressClient = new IngressClient('https://my-openvidu-host', 'api-key', 'secret-key');

const ingress = {
  name: 'my-ingress',
  roomName: 'my-room',
  participantIdentity: 'my-participant',
  participantName: 'My Participant',
  url: 'rtsp://admin:pass@192.168.1.79/mystream'
};

await ingressClient.createIngress(IngressInput.URL_INPUT, ingress);

Using LiveKit Go SDK

import (
  lksdk "github.com/livekit/server-sdk-go/v2"
  livekit "github.com/livekit/protocol/livekit"
)

ingressClient := lksdk.NewIngressClient(
    "https://my-openvidu-host",
    "livekit-api-key",
    "livekit-api-secret",
)

ingressRequest := &livekit.CreateIngressRequest{
    InputType:           livekit.IngressInput_URL_INPUT,
    Name:                "my-ingress",
    RoomName:            "my-room",
    ParticipantIdentity: "my-participant",
    ParticipantName:     "My Participant",
    Url:                 "rtsp://admin:pass@192.168.1.79/mystream",
}

ingressInfo, err := ingressClient.CreateIngress(context.Background(), ingressRequest)

Using LiveKit Ruby SDK

require 'livekit'

ingressClient = LiveKit::IngressServiceClient.new("https://my-openvidu-host", api_key: "api-key", api_secret: "secret-key")
ingressInfo = ingressClient.create_ingress(
  :URL_INPUT,
  name: "my-ingress",
  room_name: "my-room",
  participant_identity: "my-participant",
  participant_name: "My Participant",
  url: "rtsp://admin:pass@192.168.1.79/mystream",
)

Using LiveKit Kotlin SDK

import io.livekit.server.IngressServiceClient;
import livekit.LivekitIngress.IngressInfo;
import livekit.LivekitIngress.IngressInput;

IngressServiceClient ingressService = IngressServiceClient.createClient("https://my-openvidu-host", "api-key", "secret-key");

IngressInfo ingressInfo = ingressService.createIngress(
    "my-ingress", // Ingress name
    "my-room", // Room name
    "my-participant", // Ingress participant identity
    "My Participant", // Ingress participant name
    IngressInput.URL_INPUT, // Ingress input type
    null, null, null, null, // Other default options
    "rtsp://admin:pass@192.168.1.79/mystream" // Input URL
).execute().body();

Using LiveKit Rust SDK

use livekit_api::services::ingress::*;
use livekit_protocol::*;

let ingress_client = IngressClient::with_api_key(
    "https://my-openvidu-host",
    "api-key",
    "secret-key",
);
let ingress_info = ingress_client.create_ingress(
    IngressInput::UrlInput,
    CreateIngressOptions {
        name: "my-ingress".to_string(),
        room_name: "my-room".to_string(),
        participant_identity: "my-participant".to_string(),
        participant_name: "My Participant".to_string(),
        url: "rtsp://admin:pass@192.168.1.79/mystream".to_string(),
        ..Default::default()
    }).await;

Using LiveKit PHP SDK

<?php
use Agence104\LiveKit\IngressServiceClient;
use Livekit\IngressInput;

$ingress_client = new IngressServiceClient("https://my-openvidu-host", "api-key", "secret-key");
$ingress_info = $ingress_client->createIngress(
  IngressInput::URL_INPUT,
  "my-ingress", // Ingress name
  "my-room", // Room name
  "my-participant", // Ingress participant identity
  "My Participant", // Ingress participant name
  NULL, NULL, NULL, // Other default options
  "rtsp://admin:pass@192.168.1.79/mystream" // Input URL
);

Using LiveKit .NET SDK

using Livekit.Server.Sdk.Dotnet;

IngressServiceClient ingressServiceClient = new IngressServiceClient(
    "https://my-openvidu-host",
    LIVEKIT_API_KEY,
    LIVEKIT_API_SECRET
);
var ingressInfo = await ingressServiceClient.CreateIngress(new CreateIngressRequest
{
    Name = "my-ingress",
    RoomName = "my-room",
    ParticipantIdentity = "my-participant",
    ParticipantName = "My Participant",
    InputType = IngressInput.UrlInput,
    Url = "rtsp://admin:pass@192.168.1.79/mystream",
});

If your backend technology does not have its own SDK, you have two different options:

Consume the Ingress API directly: Reference Docs

Use the livekit-cli:

Create a file at ingress.json with the following content:

{
  "input_type": "URL_INPUT",
  "name": "Name of the Ingress goes here",
  "room_name": "Name of the room to connect to",
  "participant_identity": "Unique identity for the room participant the Ingress service will connect as",
  "participant_name": "Name displayed in the room for the participant",
  "url": "rtsp://admin:pass@192.168.1.79/mystream"
}

Then run the following commands:

export LIVEKIT_URL=https://my-openvidu-host
export LIVEKIT_API_KEY=api-key
export LIVEKIT_API_SECRET=secret-key

lk ingress create ingress.json

Many different audio and video codecs are supported for ingesting IP cameras:

For video:

H264
VP8
VP9
MPEG4
MJPEG

For audio:

AAC
MP3
OPUS
G711

Webhooks#

Your application server may receive webhooks coming from the OpenVidu deployment. These webhooks inform about events happening in the Rooms, including when a Room is created and finished, when a Participant joins and leaves a Room, when a Track is published and unpublished, and when Egress/Ingress operations take place in a Room.

Every application server tutorial here is ready to receive webhooks: Application Server Tutorials.

Visit the LiveKit reference documentation for a detailed explanation of each webhook event: