MediaEngine

Configure codecs, RTP parameters, and header extensions

Overview

The MediaEngine defines the codecs and RTP capabilities supported by a PeerConnection. It controls which audio and video codecs are negotiated, which RTP header extensions are enabled, and which RTCP feedback mechanisms are available.

By default, Pion WebRTC includes a comprehensive set of codecs. You only need to configure the MediaEngine if you want to customize codec support or add custom header extensions.

Creating a MediaEngine

basic-usage.go

package main

import "github.com/pion/webrtc/v4"

func main() {
    // Create a MediaEngine
    m := &webrtc.MediaEngine{}
    
    // Register default codecs (Opus, VP8, VP9, H.264, AV1)
    if err := m.RegisterDefaultCodecs(); err != nil {
        panic(err)
    }
    
    // Create an API with the MediaEngine
    api := webrtc.NewAPI(webrtc.WithMediaEngine(m))
    
    // Create PeerConnection
    peerConnection, err := api.NewPeerConnection(webrtc.Configuration{})
    if err != nil {
        panic(err)
    }
}

Default Codecs

Pion WebRTC supports these codecs by default:

Audio Codecs

1. Opus

   MimeTypeOpus - 48kHz, stereo, with in-band FEC

   RTPCodecParameters{
       RTPCodecCapability: RTPCodecCapability{
           MimeType:    "audio/opus",
           ClockRate:   48000,
           Channels:    2,
           SDPFmtpLine: "minptime=10;useinbandfec=1",
       },
       PayloadType: 111,
   }
   
2. G.722

   MimeTypeG722 - 8kHz narrowband audio

3. PCMU

   MimeTypePCMU - 8kHz μ-law encoding (G.711)

4. PCMA

   MimeTypePCMA - 8kHz A-law encoding (G.711)

From mediaengine.go:63-88.

Video Codecs

1. VP8

   MimeTypeVP8 - Popular codec with RTX support

2. VP9

   MimeTypeVP9 - Profile 0 and Profile 2 with RTX

3. H.264

   MimeTypeH264 - Multiple profiles:

   • Baseline (profile-level-id=42001f, 42e01f)
   • Main (profile-level-id=4d001f)
   • High (profile-level-id=64001f)

   Both packetization modes 0 and 1 supported

4. H.265

   MimeTypeH265 - HEVC codec

5. AV1

   MimeTypeAV1 - Next-generation codec

All video codecs include RTCP feedback for:

• goog-remb - Receiver Estimated Maximum Bitrate
• ccm fir - Full Intra Request
• nack - Negative Acknowledgment
• nack pli - Picture Loss Indication

From mediaengine.go:90-234.

Custom Codec Registration

Register a Single Codec

custom-codec.go

m := &webrtc.MediaEngine{}

// Register only VP8 codec
codec := webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:     webrtc.MimeTypeVP8,
        ClockRate:    90000,
        RTCPFeedback: []webrtc.RTCPFeedback{
            {Type: "nack"},
            {Type: "nack", Parameter: "pli"},
        },
    },
    PayloadType: 96,
}

if err := m.RegisterCodec(codec, webrtc.RTPCodecTypeVideo); err != nil {
    panic(err)
}

From mediaengine.go:256-274.

Register Specific Codecs Only

specific-codecs.go

m := &webrtc.MediaEngine{}

// Audio: Only Opus
m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:    webrtc.MimeTypeOpus,
        ClockRate:   48000,
        Channels:    2,
        SDPFmtpLine: "minptime=10;useinbandfec=1",
    },
    PayloadType: 111,
}, webrtc.RTPCodecTypeAudio)

// Video: Only VP8
m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:  webrtc.MimeTypeVP8,
        ClockRate: 90000,
        RTCPFeedback: []webrtc.RTCPFeedback{
            {Type: "goog-remb"},
            {Type: "ccm", Parameter: "fir"},
            {Type: "nack"},
            {Type: "nack", Parameter: "pli"},
        },
    },
    PayloadType: 96,
}, webrtc.RTPCodecTypeVideo)

// Don't forget RTX for VP8
m.RegisterCodec(webrtc.RTPCodecParameters{
    RTPCodecCapability: webrtc.RTPCodecCapability{
        MimeType:    webrtc.MimeTypeRTX,
        ClockRate:   90000,
        SDPFmtpLine: "apt=96",
    },
    PayloadType: 97,
}, webrtc.RTPCodecTypeVideo)

RTP Header Extensions

Header extensions provide additional metadata in RTP packets.

Registering Header Extensions

header-extensions.go

import "github.com/pion/sdp/v3"

m := &webrtc.MediaEngine{}
m.RegisterDefaultCodecs()

// Register Transport-CC for bandwidth estimation
err := m.RegisterHeaderExtension(
    webrtc.RTPHeaderExtensionCapability{URI: sdp.TransportCCURI},
    webrtc.RTPCodecTypeVideo,
)
if err != nil {
    panic(err)
}

// Register audio level indicator
err = m.RegisterHeaderExtension(
    webrtc.RTPHeaderExtensionCapability{URI: sdp.AudioLevelURI},
    webrtc.RTPCodecTypeAudio,
)

From mediaengine.go:276-324.

Common Header Extensions

Transport-CC (sdp.TransportCCURI)

Used for bandwidth estimation and congestion control. Required for TWCC (Transport Wide Congestion Control).

sdp.TransportCCURI = "http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01"

Audio Level (sdp.AudioLevelURI)

Indicates the audio level of the RTP packet.

sdp.AudioLevelURI = "urn:ietf:params:rtp-hdrext:ssrc-audio-level"

MID (sdp.SDESMidURI)

Media stream identification. Required for simulcast.

sdp.SDESMidURI = "urn:ietf:params:rtp-hdrext:sdes:mid"

RTP Stream ID (sdp.SDESRTPStreamIDURI)

RTP stream identifier. Required for simulcast.

sdp.SDESRTPStreamIDURI = "urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id"

Direction-Specific Extensions

You can limit header extensions to specific transceiver directions:

directional-extensions.go

// Only use for sending
m.RegisterHeaderExtension(
    webrtc.RTPHeaderExtensionCapability{URI: sdp.TransportCCURI},
    webrtc.RTPCodecTypeVideo,
    webrtc.RTPTransceiverDirectionSendonly,
)

// Only use for receiving
m.RegisterHeaderExtension(
    webrtc.RTPHeaderExtensionCapability{URI: sdp.AudioLevelURI},
    webrtc.RTPCodecTypeAudio,
    webrtc.RTPTransceiverDirectionRecvonly,
)

From mediaengine.go:292-300.

RTCP Feedback

Add RTCP feedback mechanisms to codecs:

rtcp-feedback.go

m := &webrtc.MediaEngine{}
m.RegisterDefaultCodecs()

// Add additional feedback to all video codecs
m.RegisterFeedback(
    webrtc.RTCPFeedback{Type: "ccm", Parameter: "fir"},
    webrtc.RTPCodecTypeVideo,
)

// Add NACK to all audio codecs
m.RegisterFeedback(
    webrtc.RTCPFeedback{Type: "nack"},
    webrtc.RTPCodecTypeAudio,
)

From mediaengine.go:326-354.

Common Feedback Types

goog-remb · string

Google’s Receiver Estimated Maximum Bitrate - for bandwidth estimation

nack · string

Negative Acknowledgment - request retransmission of lost packets

nack pli · string

Picture Loss Indication - request keyframe when packet loss affects decoding

ccm fir · string

Full Intra Request - request keyframe

transport-cc · string

Transport-wide Congestion Control - modern bandwidth estimation

Advanced Usage

Disable MediaEngine Copy

By default, the MediaEngine is copied for each PeerConnection. You can disable this:

disable-copy.go

s := webrtc.SettingEngine{}

// Allow modifying MediaEngine after PeerConnection creation
s.DisableMediaEngineCopy(true)

m := &webrtc.MediaEngine{}
m.RegisterDefaultCodecs()

api := webrtc.NewAPI(
    webrtc.WithMediaEngine(m),
    webrtc.WithSettingEngine(s),
)

peerConnection, _ := api.NewPeerConnection(webrtc.Configuration{})

// Now you can modify m after creation
// Warning: Don't share MediaEngine between PeerConnections!

When disabling MediaEngine copy, do not share the same MediaEngine instance between multiple PeerConnections.

From settingengine.go:496-501.

Query Negotiated Codecs

query-codecs.go

// After signaling is complete, you can query what was negotiated
codecs := m.getCodecsByKind(webrtc.RTPCodecTypeVideo)
for _, codec := range codecs {
    fmt.Printf("Negotiated: %s (PT: %d)\n", codec.MimeType, codec.PayloadType)
}

Complete Example

Custom MediaEngine with Specific Codecs

complete-example.go

package main

import (
    "fmt"
    "github.com/pion/sdp/v3"
    "github.com/pion/webrtc/v4"
)

func main() {
    // Create MediaEngine
    m := &webrtc.MediaEngine{}
    
    // Register only Opus for audio
    if err := m.RegisterCodec(webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{
            MimeType:    webrtc.MimeTypeOpus,
            ClockRate:   48000,
            Channels:    2,
            SDPFmtpLine: "minptime=10;useinbandfec=1",
        },
        PayloadType: 111,
    }, webrtc.RTPCodecTypeAudio); err != nil {
        panic(err)
    }
    
    // Register VP8 for video
    if err := m.RegisterCodec(webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{
            MimeType:  webrtc.MimeTypeVP8,
            ClockRate: 90000,
            RTCPFeedback: []webrtc.RTCPFeedback{
                {Type: "goog-remb"},
                {Type: "ccm", Parameter: "fir"},
                {Type: "nack"},
                {Type: "nack", Parameter: "pli"},
            },
        },
        PayloadType: 96,
    }, webrtc.RTPCodecTypeVideo); err != nil {
        panic(err)
    }
    
    // Register RTX for VP8
    if err := m.RegisterCodec(webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{
            MimeType:    webrtc.MimeTypeRTX,
            ClockRate:   90000,
            SDPFmtpLine: "apt=96",
        },
        PayloadType: 97,
    }, webrtc.RTPCodecTypeVideo); err != nil {
        panic(err)
    }
    
    // Register header extensions
    if err := m.RegisterHeaderExtension(
        webrtc.RTPHeaderExtensionCapability{URI: sdp.TransportCCURI},
        webrtc.RTPCodecTypeVideo,
    ); err != nil {
        panic(err)
    }
    
    if err := m.RegisterHeaderExtension(
        webrtc.RTPHeaderExtensionCapability{URI: sdp.AudioLevelURI},
        webrtc.RTPCodecTypeAudio,
    ); err != nil {
        panic(err)
    }
    
    // Create API
    api := webrtc.NewAPI(webrtc.WithMediaEngine(m))
    
    // Create PeerConnection
    config := webrtc.Configuration{
        ICEServers: []webrtc.ICEServer{
            {URLs: []string{"stun:stun.l.google.com:19302"}},
        },
    }
    
    peerConnection, err := api.NewPeerConnection(config)
    if err != nil {
        panic(err)
    }
    defer peerConnection.Close()
    
    fmt.Println("PeerConnection created with custom codecs")
}

Related Resources

SettingEngine

Configure advanced WebRTC behavior

Interceptors

Process RTP/RTCP packets