Getting Started

In this section, we will have a quick view about Weever Streaming.

Introduction

Weever Streaming is a Open Source WebRTC SFU.

It serves for broadcasting, video conferencing, or regular video/audio/data streaming.

It's easy to deploy and scale.

You can come up with your own streaming products/platforms easily with it, as long as you are familiar with HTML/CSS/JavaScript, and web service hosting.

You can also come up with non-web products/platforms, as long as you can run HTTP client with WebRTC capability.

Concepts

We will explain some WebRTC related concepts and Weever Streaming only concepts in this sections.

WebRTC SFU (Selective Forwarding Unit)

SFU is a server to help forwarding media. It can communicate with clients to decide which media need forwarding.

For more details information please checkout SFU (Selective Forwarding Unit) • BlogGeek.me.

SDP (Session Description Protocol)

SDP is a widely adopted standard for describing the multimedia of the connection.

For more details information please checkout Anatomy of a WebRTC SDP.

ICE (Interactive Connectivity Establishment) Protocol

ICE is a framework to allow peer to connect with another peer. The ICE candidate will describe potential network path for connection.

STUN (Session Traversal Utilities for NAT)

STUN is a protocol to discover peer's public address and determine any restrictions in the router.

For more details information please checkout STUN • BlogGeek.me.

TURN (Traversal Using Relays around NAT)

TURN is a relay server to bypass the Symmetric NAT restriction.

For more details information please checkout WebRTC TURN: Why you NEED it and when you DON’T need it.

Publisher & Subscriber

Publisher and Subscriber are the only two roles in Weever Streaming.

Depends on use cases, the application can run both publisher and subscriber at the same time.

Publisher

Publisher push media to Weever Streaming. The media can be audio, video, or both.

Subscriber

Subscriber pull media from Weever Streaming. The media can be audio, video, or both.

Room

A room in Weever Streaming maintains states of participating publishers/subscribers.

There is no different for hosting audio or video for a room. A room can serve both audio and video.

There is no publishers or subscribers amount limitation for a room.

Multistream

Multistream means putting arbitrary amount of audio/video tracks in the same PeerConnection.
(It's done via Unified Plan of SDP)

Combine with the widely adopted RTP BUNDLE and RTCP multiplexing. This means we can serve both audio and video at the same time, plus the control protocol, for any amount of peers' media all in "one single socket connection".

In Weever Streaming, the subscribers will leverage multistream for receiving all audio/video (from multiple publishers) in single connection.

By Weever Streaming convention, the stream id in SDP will be the publisher id. So subscriber can identify each stream after parsing the SDP. This also means subscribers can use info in SDP to know all the publishers.

To keep it simple, publishers in Weever Streaming still use 1 connection for each outgoing streams. Only subscribers fully leverage the multistream.

When a publisher join or leave, server will trigger WebRTC Renegotiation for subscribers. New SDP will be sent via Data Channel. Subscribers can use new SDP to get latest streams status in the room.

For more information about multistream, you can checkout Janus's article Multistream is here!.

WebRTC Renegotiation

After WebRTC is connected, you can still update the SDP for latest media status.

To achieve WebRTC Renegotiation, you simply go through the WebRTC Negotiation flow one more time. Send SDP Offer to the other peer. Receive and set SDP Answer.

The SDP Offer can be initiated by any peer, not necessary the peer who sent SDP Offer in first setup.

WHIP & WHEP

WHIP (WebRTC-HTTP Ingestion Protocol)

The WHIP is a spec for pushing WebRTC-based media into services. It provides a spec for WebRTC signaling with HTTP protocol and Bearer Token in pushing to services scenario.

Weever Streaming has WHIP-like media ingress. It's WHIP-"like" because there is no guarantee about spec compliance at the moment. But this project learned the idea from there.

And it's possible to use WHIP clients to publish audio/video to Weever Streaming.

WHIP Clients:

• simple-whip-client (GStreamer based)
  • ./whip-client -u https://localhost:8443/pub/testroom/user1 -t mytoken -V "videotestsrc is-live=true pattern=smpte ! videoconvert ! video/x-raw,width=1280,height=720,framerate=60/1 ! queue ! vp8enc deadline=1 ! rtpvp8pay pt=96 ssrc=2 ! queue ! application/x-rtp,media=video,encoding-name=VP8,payload=96"

WHEP (WebRTC-HTTP Egress Protocol)

The WHEP is a spec for pulling WebRTC-based media from services. It provides a spec for WebRTC signaling with HTTP protocol and Bearer Token in pulling from services scenario. (The pulling version of WHIP.)

Weever Streaming has WHEP-like media egress. It's WHEP-"like" because there is no guarantee about spec compliance at the moment. Weever Streaming implemented similar idea before there is WHEP spec release.

And it's possible to use WHEP clients to pull audio/video from Weever Streaming. But WHEP doesn't include renegotiation, so WHEP clients can't update media on the fly.

WHEP Clients:

• simple-whep-client (GStreamer based)
  • ./whep-client -u https://localhost:8443/sub/testroom/user2 -t mytoken

Client as SDP Offer first

In Weever Streaming, the clients always will be the SDP Offer side for connection setup. No matter the clients want to be publisher or subscriber. This make clients WebRTC connection setup to be 1 HTTP POST and response.

After WebRTC is connected, Weever Streaming will use Data Channel for following WebRTC Renegotiation.

Authentication

Weever Streaming's authentication is done via Bearer Token.

Server can set the token for each publisher/subscriber.

Client connect to Weever Streaming with Authorization: Bearer <token> in HTTP headers.

Server

This section will introduce the Weever Streaming server.

Room Management

In Weever Streaming there is "Room" concept. A room can host audio and video streaming.

Server can set Authentication to control who can join room. (via Private HTTP API)

ID & Full ID

In Weever Streaming, Publisher/Subscriber ID can have trailing +<RANDOM> pattern (called it Full ID). So <ID>+<RANDOM> is considered to be the same publisher/subscriber as <ID>. In the Weever Streaming example, we use this to identify each connection from same user. Even if a user reconnect, there will be unique Full ID from server perspective.

Dependencies

Weever Streaming has 2 dependencies at the moment.

One is for states sharing, Weever Streaming use Redis at the moment.

Another is for media routing, Weever Streaming use NATS at the moment..

States Sharing

Weever Streaming use external service for states sharing.

States sharing will handle information like:

• media count in a room
• publishers list in a room
• subscribers list in a room

It possible to have a internal states sharing implementation in the future.

Redis

When launching Weever Streaming, you can use CLI argument --redis or environment variable REDIS to assign Redis URL.

Media Routing

Weever Streaming use external service for media routing.

Media routing will handle audio/video streams routing. Pass the streams across Weever Streaming instances. Forward streams to the Weever Streaming instances that need them.

It possible to have a internal media routing implementation in the future.

NATS

When launching Weever Streaming, you can use CLI argument --nats or environment variable NATS to assign NATS URL.

Public HTTP API

Public HTTP APIs in Weever Streaming for publisher/subscriber connection.

Private HTTP API

There is one private HTTP server in Weever Streaming. It servers for management. You can use these API to manage the room from other services.

Data Channel API

A data channel labeled as "control" will be setup from server. This data channel will be used for WebRTC Renegotiation or other connection actions.

Publisher

Subscriber

Scaling

Weever Streaming scaling can be done by adding more instances.

Possible setup:

• All Weever Streaming shared the same external NATS service. In this style, Weever Streaming scaling doesn't bind with NATS scaling.
• Each Weever Streaming has its own NATS sidecar. And NATS sidecars form a cluster by themselves. In this style, Weever Streaming scaling bind with NATS scaling.

Client

Weever Streaming client can be any programming language that supports HTTP and WebRTC.

Weever Streaming also shares many aspect with WHIP & WHEP.

Join as Publisher

JavaScript example (based on browser API):

// create WebRTC peer connection
let pc = new RTCPeerConnection();
// create a data channel labeled as "control"
let dc = pc.createDataChannel("control");

// setup callback
pc.onicecandidate = function(event) {
    if (event.candidate) {
        // ...
    } else {
        let offer = pc.localDescription;

        let type = "pub";
        let room = "testroom";  // modify this
        let id = "testuser";    // modify this
        let token = "mysecret"; // modify this
        let url = `/${type}/${room}/${id}`;   // modify this

        // send to Weever Streaming
        fetch(url, {
            method: "POST",
            headers: {
                "Content-Type": "application/sdp",
                "Authorization": `Bearer ${token}`,
            },
            body: offer.sdp,
        }).then(res => {
            res.text().then(function(answer) {
                pc.setRemoteDescription(new RTCSessionDescription({"type": "answer", "sdp": answer}));
            })
        });
    }
}

// grab media (audio/video)
navigator.mediaDevices.getUserMedia({ audio: true, video: true })
    .then(stream => {
        stream.getTracks().forEach(track => pc.addTrack(track, stream));
        pc.createOffer()
            .then(offer => {
                // set local SDP offer
                // this will trigger ICE gathering, and then onicecandidate callback
                pc.setLocalDescription(offer);
            });
    });

Join as Subscriber

JavaScript example (based on browser API):

// create WebRTC peer connection
let pc = new RTCPeerConnection();
// create a data channel labeled as "control"
let dc = pc.createDataChannel("control");

// setup callback
pc.onicecandidate = function(event) {
    if (event.candidate) {
        // ...
    } else {
        let offer = pc.localDescription;

        let type = "sub";
        let room = "testroom";  // modify this
        let id = "testuser";    // modify this
        let token = "mysecret"; // modify this
        let url = `/${type}/${room}/${id}`;   // modify this

        // send to Weever Streaming
        fetch(url, {
            method: "POST",
            headers: {
                "Content-Type": "application/sdp",
                "Authorization": `Bearer ${token}`,
            },
            body: offer.sdp,
        }).then(res => {
            res.text().then(function(answer) {
                pc.setRemoteDescription(new RTCSessionDescription({"type": "answer", "sdp": answer}));
            })
        });
    }
}

// data channel messages
dc.onmessage = e => {
    // WebRTC Renegotiation
    if (e.data.startsWith("SDP_OFFER ") == true) {
        let offer = e.data.slice(10);
        pc.setRemoteDescription(new RTCSessionDescription({"type": "offer", "sdp": offer}));
        pc.createAnswer()
            .then(answer => {
                pc.setLocalDescription(answer);
                dc.send("SDP_ANSWER " + answer.sdp);
            });
    }
}

pc.createOffer()
    .then(offer => {
        // set local SDP offer
        // this will trigger ICE gathering, and then onicecandidate callback
        pc.setLocalDescription(offer);
    });

Use Cases

This section will show some setup examples for different use cases.

Public Audio Chat

Target

An audio only chatting application.

Anyone can join the room and speak.
Anyone can join the room and listen.

Setup

Application run both publisher and subscriber clients.

Publisher client only enable audio, grab audio from device's microphone.

Subscriber client play all the incoming audio via speaker.

Private Audio Chat

Target

An audio only chatting application.

Only people with invitation can join the room and speak.
Only people with invitation can join the room and listen.
One person might have permission to listen, but not necessary have permission to listen.

Setup

Server set authentication for clients.

For person who has both speak and listen permission, application run both publisher and subscriber clients.

For person who only has listen permission, application only run subscriber client.

Video Conferencing

Target

An meeting application with both audio and video.

Only people with invitation can join the room and speak.
Only people with invitation can join the room and listen.

Setup

Server set authentication for clients.

Application run both publisher and subscriber clients.

Publishers enable both audio and video.

Broadcasting

Target

Only a few people have permission to speak and show video.
Most of the people are just watching and listening.

Setup

Server set authentication for clients.

Application run both publisher and subscriber clients for broadcasting organizer.

Publishers enable both audio and video.

Application only run subscriber for all the viewers.

Deployment

This section will show how to deploy Weever Streaming in different platforms.

Single Machine

To run Weever Streaming with single machine, you can:

1. run Redis
2. run NATS
3. grab Weever Streaming binary (a single file) (You can download from GitHub Actions' artifacts)
4. execute Weever Streaming binary with CLI paramenters

Deploy with Docker Compose

this will launch: 1 Redis, 3 NATS, 3 WebRTC SFU (Weever Streaming)

  ┌──────┐    ┌──────┐    ┌──────┐
  │ SFU1 ├──┐ │ SFU2 ├──┐ │ SFU3 ├──┐
  └───┬──┘  │ └───┬──┘  │ └───┬──┘  │
      │     │     │     │     │     │
  ┌───▼───┐ │ ┌───▼───┐ │ ┌───▼───┐ │
  │ NATS1 ◄─┼─► NATS2 ◄─┼─► NATS3 │ │
  └───────┘ │ └───────┘ │ └───────┘ │
            │           │           │
  ┌─────────▼───────────▼───────────▼──┐
  │               Redis                │
  └────────────────────────────────────┘

Start your docker daemon

# start your docker daemon
sudo systemctl start docker

Bring up Weever Streaming

# run in the root folder of the project, the one with "docker-compose.yml" in it
docker-compose up
# check status
docker-compose ls
docker-compose ps
docker-compose logs -f sfu1

Remove Weever Streaming

docker-compose down

Deploy with Docker Swarm

Start up docker swarm mode

# start your docker daemon
sudo systemctl start docker
# run docker swarm node
docker swarm init

Have all the nodes you want join the swarm

# (optional) join the cluster in another machine
docker swarm join --token <token> <ip:port>

Deploy Weever Streaming up

# run in the root folder of the project, the one with "docker-compose.yml" in it
docker stack deploy --compose-file docker-compose.yml weever-streaming

# check status
docker stack ls
docker stack services weever-streaming
docker stack ps weever-streaming
docker service logs -f weever-streaming_sfu1

Remove Weever Streaming from the swarm

docker stack rm weever-streaming

Kubernetes

(TODO: add helm chart in)

• SFU can be deployed with StatefulSet.
• Override the SSL certs in Pod.
• A LoadBalancer in front of public web server.
• Setup liveness/readiness probe, point to SFU private web server.
• Setup preStop hook, point to SFU private web server.
• NATS instance can be another service or SFU Pod sidecar.
• Redis instance can be another service.
• HPA based on CPU usage, dynamically grows the Pod count.
• Can use other complex mechanism to control minimum Pod count if you need to schedule big events. For example, a cronjob to read business data and calculate a desired minimum Pod count.
• Use/Setup TURN server if needed.
• Authentication can be enabled via --auth. It's using Bearer Token mechanism. Token can be set via private web server. Data will be saved in Redis (1 day TTL).
• Management can be done via another service that talks to SFU via private web server.
• CORS can be set via --cors-domain <DOMAIN>
• Most of configs can be set via both CLI arguments and environment variables.

Metrics

This section will show some way to monitor Weever Streaming status.

Prometheus

In Weever Streaming Private HTTP API, there is Prometheus metrics at /metrics.

There is only publishers/subscribers count at the moment. The clients count is categorized based on room and role type.