Skip to content

Blog#

How we develop and test complex OpenVidu clusters in our laptops

Developing and testing complex OpenVidu clusters on a laptop: production topologies inside Docker-in-Docker, and the chaos tests that break them on purpose

Any modern software system of a certain size that aims to be scalable and fault-tolerant consists of multiple services running on different nodes. OpenVidu is no exception, and it goes even further: as a real-time system, there are other factors that become critical: shared state, concurrency, and response time.

Developing and testing such a distributed system is not trivial. In this post, we will explain how we are able to replicate production OpenVidu clusters on a single machine using Docker-in-Docker. This allows us to test and find bugs that only appear in a distributed environment, without the need for multiple VMs or physical machines.

The idea is that this short post will serve as inspiration for developers and sysadmins who may face similar challenges when working with distributed systems and need a practical solution to chaos test them, in a cost-effective and efficient way.

The Black Hole Node: how one full disk broke recordings in an entire WebRTC cluster (Bug stories #1)

The Black Hole Node: one full disk swallowed every recording in the cluster The Black Hole Node: one full disk swallowed every recording in the cluster

OpenVidu records media from its rooms with a service called Egress, and spreads that work across multiple Media Nodes, each one running an Egress of its own. Our Egress is built on LiveKit's, but the algorithm that decides which node takes each recording is our own implementation. LiveKit does not choose by CPU: it weights each recording by type to decide which nodes can take it at all, then hands the job to one that is already recording, picking at random when every candidate is idle. We added a CPU-aware strategy on top and made it the default, so a new recording lands on the node with the most headroom rather than on the busiest one.

We did that to keep the load even across the cluster and get more out of every Media Node. But in systems as complex as WebRTC it is easy to miss second-order effects: parts that behave perfectly on their own and break once you put them together. Our strategy broke when it met a disk limit nobody had accounted for, and it took the recording capability of a whole cluster with it.

The metrics never pointed at it. What they measure is whether a cluster is healthy, and this one was. The failure was somewhere else, in the logs, and that is where we traced it, node by node, until we found what to fix.

Introducing the OpenVidu Agent Plugin for coding agents

The OpenVidu Agent Plugin: the OpenVidu documentation server and the version, edition and product skill, installed in one step in Claude Code, VS Code, GitHub Copilot, Codex, Cursor and Kiro The OpenVidu Agent Plugin: the OpenVidu documentation server and the version, edition and product skill, installed in one step in Claude Code, VS Code, GitHub Copilot, Codex, Cursor and Kiro

Many developers already let a coding agent write the code that connects their application to OpenVidu, like the backend that creates the rooms or the page where the meeting is embedded. To get that code right, it needs to know how OpenVidu works in the version you're running. That's why today we're releasing the OpenVidu Agent Plugin. Install it once in Claude Code, VS Code, GitHub Copilot, Codex, Cursor or Kiro, and your agent will read the official OpenVidu documentation for your version whenever a task needs it, so the code it writes is up to date and accurate.

The plugin bundles two components: a documentation server with the OpenVidu Meet and OpenVidu Platform docs of every release from 3.4, and a skill that finds out the version, edition and product of the OpenVidu deployment your project uses. In this post we explain how they work together, how to install the plugin, and how much it improved the answers in our tests.

  • in Release
  • 9 min read

OpenVidu 3.9.0 is now available

OpenVidu 3.9.0 is a comprehensive collection of improvements, bug fixes, and stability enhancements. It brings a renovated OpenVidu Meet with programmatic control over live meetings, and a better OpenVidu Platform that behaves more predictably in production under network changes, high load and node restarts.

On the OpenVidu Meet side, the new Meetings REST API lets your backend read and moderate live meetings, and moderators can mute participants from the meeting itself. Webhooks can now be filtered per event and per room, and each room can set participant and duration limits, automatic recording and the initial state of microphones and cameras. Improved UI and a set of bug fixes complete this release.

On the OpenVidu Platform side, Live Captions add NVIDIA Nemotron, the most accurate of the local models, with 40 languages in a single model. The rest of the release is hardening work: mediasoup fixes for SVC video in VP9 and AV1, Firefox and ICE, graceful restarts that take half the time, and services that no longer make background calls to third parties.

Here are the highlights.

3 ways to integrate video conferencing into your app with OpenVidu

Three stacked integration levels, from embedding OpenVidu Meet to Angular Components to low-level SDKs, all running on one self-hosted OpenVidu deployment Three stacked integration levels, from embedding OpenVidu Meet to Angular Components to low-level SDKs, all running on one self-hosted OpenVidu deployment

Most products reach a point where a chat window or a phone number is no longer enough, and people need to see each other. Sooner or later the ticket lands on your board: "Add video calls to the app". The WebRTC part is a solved problem. The question that actually shapes the project is a different one: how much of the meeting do you want to own? The buttons, the layout, the media tracks themselves? Or just a <div> where a meeting shows up?

There are three ways to integrate video conferencing into your app with OpenVidu, and all of them run on the same self-hosted deployment. You can embed OpenVidu Meet, a finished meeting UI, with one HTML tag. You can assemble your own meeting screen from Angular Components. Or you can go down to the OpenVidu Platform SDKs and handle every audio and video track yourself.

How Novakid runs 30 million live English lessons on OpenVidu

Novakid and OpenVidu customer success story Novakid and OpenVidu customer success story

Running a live video product where the users are seven years old is a challenge. Adults may tolerate a frozen frame or a reconnect spinner, but a seven-year-old just gives up, and the parent doesn't book a second lesson. That's what Novakid , an online English school for kids aged 4–12, has been doing since 2017. Over a million students, 50+ countries, more than 30 million lessons delivered, and up to 2,300 lessons running at once at peak.

We talked to Andrei Iakimov, DevOps Lead at Novakid, Inc., about moving their live classroom off a self-managed Kurento deployment and onto OpenVidu: why they migrated, why they didn't take the SaaS path, what that migration actually looked like, how they built scheduled autoscaling on top of it, what broke (and got fixed) along the way, and what they value most of.

Low Latency Live Streaming: Ingest WHIP into OpenVidu (Part 2)

A browser and OBS Studio pushing video into an OpenVidu Room over WHIP, and a viewer subscribing to it A browser and OBS Studio pushing video into an OpenVidu Room over WHIP, and a viewer subscribing to it

Part 1 of this series argued that if your video has to close a feedback loop with the person watching it, HLS and DASH structurally can't get you there and WebRTC can. That's the theory, and theory is cheap. So let's do the thing itself: take a webcam, push it into a self-hosted OpenVidu Platform Room over WHIP, and watch it come out the other side fast enough to have a conversation through. Then do it again from OBS Studio, which has spoken WHIP natively since version 30 and needs no plugin, no SDK and no code at all.

Low Latency Live Streaming: WebRTC vs. HLS and DASH (Part 1)

A split-screen graphic comparing a near-instant video call with a delayed live broadcast A split-screen graphic comparing a near-instant video call with a delayed live broadcast

When Spain was playing the World Cup, we noticed something annoying. We celebrated each of Spain's scores by shouting "GOOOOOL", 15 seconds before our neighbors saw the goal on their TV. Obviously, we ruined their experience watching the game, so much that they asked us where we were watching it from to avoid the gap. Now picture that same 15-second gap on a live shopping stream where you're typing "does it come in blue?", or in a video call where you keep talking over the other person because their audio hasn't reached you yet. That's the difference between "live" and low latency live streaming.

Debugging WebRTC with an AI agent and Grafana MCP

Debugging WebRTC with an AI agent and Grafana MCP: read-only Grafana, a broken deployment, and an agent that works through the metrics to find each root cause

What if you gave an AI agent nothing but read-only access to your Grafana, pointed it at a WebRTC deployment it had never seen, and asked what was broken? No shell, no source code, no config files, nothing but the dashboards and logs any on-call engineer would stare at. Could it actually find the root cause?

That is the experiment we ran at OpenVidu. We took a real OpenVidu deployment, broke it on purpose in five different ways, and handed a blind Claude Code session a single vague complaint and a link to Grafana. This post walks through what it found, where it shone and where it fell flat, and it ships with a companion repo so you can reproduce every bit of it yourself.

Deploy OpenVidu on Hetzner Cloud in 15 Minutes

OpenVidu servers inside a Hetzner cloud serving a video call

This post is a getting-started guide to OpenVidu on Hetzner Cloud. It gathers in one place all the steps needed to go from an empty Hetzner account to a working OpenVidu deployment in a few minutes: which instance to pick, which ports to open, and the one command that installs everything. It is deliberately shorter than the official self-hosting documentation; the goal here is a running deployment today, not covering every option.