@vpalmisano/webrtcperf
Advanced tools
 # WebRtcPerf [GitHub page](https://github.com/vpalmisano/webrtcperf) | [Documentation](https://vpalmisano.github.io/webrtcperf)
WebRtcPerf is an open-source tool designed for testing WebRTC services with multiple concurrent client connections, measuring the most important RTC statistics and collecting them in an easy way. This documentation will dive into its multiple features and configuration options, showing you how to leverage this tool to gain valuable insights into your real-time communication solutions.
The webrtcperf tool is a NodeJS application spawning multiple Puppeteer headless browsers that will actually start the WebRTC connections, so it could ideally run on every platform where NodeJS and Chromium browser could run. Anyway, to gain advantage of some specific features, using a Linux OS is the suggested way to use the tool. If you plan to run multiple concurrent connections to an external host running the WebRTC service, you need to make sure that your machine has enough network bandwidth to send and receive the audio/video streams without affecting the quality and an amount of CPU and memory proportional to the number of clients that you want to start. Anyway, you can also use a limited machine at the purpose, in order to evaluate how the WebRTC server and the client react to the constrained environment. Consider also that the tool uses the Traffic Controller (TC) framework on Linux to apply network constraints and change them while running the tests.
The command to install the tool:
npm install -g @vpalmisano/webrtcperf
webrtcperf --help
Make sure to have FFMpeg installed on the same machine.
Alternatively, use the Docker pre-built image:
docker pull ghcr.io/vpalmisano/webrtcperf:devel
docker run -it ghcr.io/vpalmisano/webrtcperf:devel --help
The tool can also be used from the source code:
git clone https://github.com/vpalmisano/webrtcperf.git
cd webrtcperf
yarn build
yarn start --help
The command line arguments can be explicitly provided using the lower snake case format (e.g. --run-duration= or setting the corresponding environment variable in upper snake case format (RUN_DURATION=). The command line params have precedence over the corresponding environment variables. Alternatively, it is possible to load the configuration variables saved in lower camel case into a local JSON/JSON5, YAML or TOML file; the file can also be stored remotely and loaded using its public HTTP URL.
Example running a simple scenario with Mediasoup demo:
webrtcperf https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/scenarios/mediasoup.json
Stop the tool pressing q (normal browser close) or x (it will close the process immediately).
Let’s start running some simple test scenarios with only 2 participants. The command line arguments required to join a Google Meet public room are:
webrtcperf \
--sessions=1 \
--run-duration=600 \
--url=https://meet.google.com/<ID> \
--debugging-port=9000 \
--script-path=https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/google-meet.js
In particular, the mandatory arguments are:
--sessions: the total number of chromium browser sessions that we want to start.--run-duration: the test run duration in seconds.--url: the remote URL that we want to visit.Other arguments required to join a Google Meet room:
--debugging-port: set to a value different from zero in order to disable the webdriver feature (https://issues.chromium.org/issues/40746300).--script-path: it injects a javascript code into the running page. It accepts a comma separated list of local file paths or http urls. In this case, the code automatically fills in the user name and it joins the room.By default, the tool will periodically output some metrics fetched from the running RTC PeerConnection objects. The metrics are aggregated across all the running sessions, showing the sum (when it makes sense), mean, standard deviation, 5th and 95th percentiles, min and max values.
-- Wed, 04 Jun 2025 11:04:30 GMT -------------------------------------------------------------------
name count sum mean stddev 5p 95p min max
System CPU 1 10.20 0.00 10.20 10.20 10.20 10.20 %
System GPU 1 22.00 0.00 22.00 22.00 22.00 22.00 %
System Memory 1 33.64 0.00 33.64 33.64 33.64 33.64 %
CPU/page 1 92.29 92.29 0.00 92.29 92.29 92.29 92.29 %
Memory/page 1 2021.88 2021.88 0.00 2021.88 2021.88 2021.88 2021.88 MB
Pages 1 1 1 0 1 1 1 1
Errors 1 0 0 0 0 0 0 0
Warnings 1 0 0 0 0 0 0 0
Peer Connections 1 1 1 0 1 1 1 1
-- Inbound audio -----------------------------------------------------------------------------------
-- Inbound video -----------------------------------------------------------------------------------
-- Outbound audio ----------------------------------------------------------------------------------
rate 1 73.81 73.81 0.00 73.81 73.81 73.81 73.81 Kbps
lost 1 0.00 0.00 0.00 0.00 0.00 0.00 %
roundTripTime 1 0.033 0.000 0.033 0.033 0.033 0.033 s
-- Outbound video ----------------------------------------------------------------------------------
sent 1 2.62 2.62 0.00 2.62 2.62 2.62 2.62 MB
rate 1 1209.16 1209.16 0.00 1209.16 1209.16 1209.16 1209.16 Kbps
lost 1 0.00 0.00 0.00 0.00 0.00 0.00 %
roundTripTime 1 0.000 0.000 0.000 0.000 0.000 0.000 s
qualityLimitResolutionChanges 1 0 0 0 0 0 0 0
qualityLimitationCpu 1 0 0 0 0 0 0 0 %
qualityLimitationBandwidth 1 0 0 0 0 0 0 0 %
width 1 1280 0 1280 1280 1280 1280 px
height 1 720 0 720 720 720 720 px
fps 1 25 0 25 25 25 25 fps
firCountReceived 1 0 0 0 0 0 0
pliCountReceived 1 0 0 0 0 0 0
The above metrics represent:
--sessions provided value).Let’s see what happens when we start increasing the number of sessions, introducing more participants in the same room (--sessions=2):
-- Wed, 04 Jun 2025 11:34:00 GMT -------------------------------------------------------------------
name count sum mean stddev 5p 95p min max
System CPU 2 18.92 0.00 18.92 18.92 18.92 18.92 %
System GPU 2 0.00 0.00 0.00 0.00 0.00 0.00 %
System Memory 2 33.98 0.00 33.98 33.98 33.98 33.98 %
CPU/page 2 370.03 185.01 1.91 183.10 186.93 183.10 186.93 %
Memory/page 2 3914.67 1957.34 10.50 1946.84 1967.83 1946.84 1967.83 MB
Pages 2 2 1 0 1 1 1 1
Errors 2 0 0 0 0 0 0 0
Warnings 2 0 0 0 0 0 0 0
Peer Connections 2 2 1 0 1 1 1 1
-- Inbound audio -----------------------------------------------------------------------------------
rate 2 89.64 44.82 0.02 44.80 44.83 44.80 44.83 Kbps
lost 2 0.00 0.00 0.00 0.00 0.00 0.00 %
jitter 2 0.00 0.00 0.00 0.00 0.00 0.00 s
avgJitterBufferDelay 2 65.87 13.25 52.62 79.12 52.62 79.12 ms
-- Inbound video -----------------------------------------------------------------------------------
received 2 12.93 6.47 0.01 6.46 6.48 6.46 6.48 MB
rate 2 2985.18 1492.59 15.29 1477.30 1507.88 1477.30 1507.88 Kbps
lost 2 0.00 0.00 0.00 0.00 0.00 0.00 %
jitter 2 0.01 0.00 0.01 0.01 0.01 0.01 s
avgJitterBufferDelay 2 52.64 4.28 48.36 56.92 48.36 56.92 ms
width 2 1280 0 1280 1280 1280 1280 px
height 2 720 0 720 720 720 720 px
fps 2 25 0 24 25 24 25 fps
-- Outbound audio ----------------------------------------------------------------------------------
rate 2 154.32 77.16 0.01 77.15 77.17 77.15 77.17 Kbps
lost 2 0.00 0.00 0.00 0.00 0.00 0.00 %
roundTripTime 2 0.032 0.001 0.031 0.033 0.031 0.033 s
-- Outbound video ----------------------------------------------------------------------------------
sent 2 13.72 6.86 0.03 6.83 6.90 6.83 6.90 MB
rate 2 2986.60 1493.30 15.40 1477.91 1508.70 1477.91 1508.70 Kbps
lost 2 0.00 0.00 0.00 0.00 0.00 0.00 %
roundTripTime 2 0.032 0.001 0.031 0.033 0.031 0.033 s
qualityLimitResolutionChanges 2 0 0 0 0 0 0 0
qualityLimitationCpu 2 0 0 0 0 0 0 0 %
qualityLimitationBandwidth 2 0 0 0 0 0 0 0 %
width 2 1280 0 1280 1280 1280 1280 px
height 2 720 0 720 720 720 720 px
fps 2 25 0 25 25 25 25 fps
firCountReceived 2 0 0 0 0 0 0
pliCountReceived 2 0 0 0 0 0 0
As we can see from the command output, now we have count=2 for all the metrics and we start noticing some differences between the sum, average, min, max and percentile values.
Collecting and analyzing the metrics using the command console output could be a difficult task, especially when we increase the number of sessions. At this purpose, webrtcperf allows sending the metrics values to an external Prometheus/Grafana service, through a Pushgateway service. An example docker compose configuration to run all the required services is available at https://github.com/vpalmisano/webrtcperf/tree/devel/prometheus-stack.
Assuming that the Pushgateway service is running on localhost:9091, the additional param to include to the command line is: --prometheus-pushgateway=http://localhost:9091
If the Pushgateway server is protected by username and password credentials, just add the --prometheus-pushgateway-auth=username:password param.
Additionally, we can change the Prometheus job=default tag using the --prometheus-pushgateway-job-name=custom-tag param; this way, we can easily differentiate the metrics coming from different or concurrent test runs.
Visiting the Grafana service started with the previous docker compose it is possible to access the example dashboard, showing a number of metrics collected by webrtcperf:
In the previous examples we used the --url option to set a page target URL that is used by all the running sessions. The tool has other two options in order to customize the destination url:
--url-query: It allows you to specify an additional URL query string that will be attached to the provided --url. The string can include some template variables that will be replaced at run time:
$p the process pid$s the session index$S the total sessions$t the tab index$T the total tabs per session$i the tab absolute indexExample: --url-query=”participantName=Test-$i” will attach ?participantName=Test-0, ?participantName=Test-1 to the corresponding session url.
--custom-url-handler: This option accepts a javascript module path containing a single exported function that will be invoked for each started session. Use this option without setting the --url option.
The function will be invoked with a single object with the following properties:
sessions: the total number of sessions;
tabsPerSession: the total number of tabs per session;
id: the session global index (0-indexed);
index: the tab global index (0-indexed);
tabIndex: the tab index in the current session (0-indexed);
pid: the process pid;
env: the environment variables object;
params: the script parameters object.
Example script:
export default function ({ id, sessions, tabIndex, tabsPerSession, index, env, pid }) {
return `https://example.com/${id}/${sessions}/${tabIndex}/${tabsPerSession}/${index}/${pid}`
}
The tool leverages several options offered by chromium to specify an audio/video input file that will be used as getUserMedia source. Using fake audio and video allows us to run consistent and repeatable tests, without introducing the implicit randomness that will be caused by running tests with a local webcam/microphone inputs.
The tool allows to configure the following options:
--video-path: A file path used as source for the audio and video streams. FFMpeg will be used to convert this file into a format that is accepted by the chromium options, thus any input format accepted by ffmpeg is a valid value (e.g. https://…). Additionally, using generate:null will generate a black video and silent audio track; with generate:test it will generate an ffmpeg testsrc video pattern and a single frequency audio with beep pattern. The option is set by default to this test video https://github.com/vpalmisano/webrtcperf/releases/download/videos-1.0/kt.mp4. More test videos are available under this release tag: https://github.com/vpalmisano/webrtcperf/releases/tag/videos-1.0.--video-width, --video-height, --video-framerate: the resolution and the video frame rate that will be used to upscale or downscale the input video.--video-seek: the seek time in seconds from the start in the provided video.--video-duration: the total duration used to trim the input video.--video-cache-raw: when true (default value) the generated files will be cached into the ~/.webrtcperf/cache directory.--video-format: the video format used for video track conversion; it could be “y4m” (raw yuv video, the default value) or “mjpeg”.--use-fake-media:
use-fake-ui-for-media-stream, use-file-for-fake-video-capture and use-file-for-fake-audio-capture params. The getUserMedia calls will output tracks generated from the provided raw files.The chromium default behaviour to fake the display media track allows to get a generated video animation that is usually very different from what we have in a regular video conference when users share a static presentation with slide transitions. In fact, the chromium generated animation produces a constant bitrate stream, while a real presentation generates a low bitrate at average with big spikes caused by the slide transitions. Emulating this behavior is essential to assess the video quality of the screen sharing in a video conference service.
In webrtcperf, the getDisplayMedia calls will output a video track that is obtained from sharing the current browser window (when --use-fake-media=true) or a new browser tab (--use-fake-media=false). In both cases, the tool will use a page script automation that generates a fake screenshare sequence that can be customized adding a fakeScreenshare option in the --script-params command line option. The fakeScreenshare option accepts the following params:
animationDuration: the slide animation duration expressed in milliseconds (default: 1000).delay: the slide change delay in milliseconds (default: 5000).embed: an optional URL specifying a document or an external Google Drive presentation that will be loaded inside an iframe element.width, height: the screen share resolution (default: 1920x1080)pointerAnimation: if different from zero, it will draw a random animation emulating the mouse movement.slides: the total slides to generate (default: 4).urls: an optional array of strings that could include (even in mixed order):
an image or video URL that will be used as slide content;
a document that will be embedded into an iframe element and it will be used as slide content.
When both slides and embed are not provided, the tool will load the images from https://picsum.photos.
Internet available bandwidth, network latency and packet loss are factors that usually could impact the perceived quality of a video conference service. Replicating the network conditions that affect real users is usually a good way to assess if the WebRTC service is able to adapt to the available bandwidth (increasing or decreasing the video resolution and the framerate) and deal with the packet loss and delay.
The webrtcperf tool includes a network throttling functionality offered by the https://github.com/vpalmisano/throttler project, that is essentially a NodeJS wrapper of the Linux Traffic Controller framework (https://tldp.org/HOWTO/Traffic-Control-HOWTO/intro.html) and NetEm. The --throttle-config param accepts a JSON/JSON5 encoded string containing an array of throttler configuration; each configuration item is an object containing the following properties:
Each throttle rule could include the following properties that controls the corresponding value in the NetEm tool:
Examples
Run a two participants video conference on Google Meet, throttling the the first participant uplink at 500Kbps, 100ms delay and 2% packet loss:
webrtcperf \
--sessions=2 \
--run-duration=300 \
--url=https://meet.google.com/<ID> \
--debugging-port=9000 \
--script-path=https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/google-meet.js \
--prometheus-pushgateway=http://localhost:9091 \
--throttle-config='[{sessions:"0",protocol:"udp",up:[{rate:500,delay:100,loss:2,queue:50}]}]'
The “Participant-000000” video stream has a visible lower quality compared with “Participant-000001”:
Comparing the same test run without and with network throttler option, we can see how the sent bitrate decreased from an average of ~600Kbps down to ~300Kbps; packet loss increased up to 1-2% and the round trip time increase from ~40ms up to ~150ms:
Let’s run another test with a different throttle configuration. In this case we specify multiple rules for the upstream limitation, using a different at value for each one, in particular what we want to test is:
webrtcperf \
--sessions=2 \
--run-duration=300 \
--url=https://meet.google.com/<ID> \
--debugging-port=9000 \
--script-path=https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/google-meet.js \
--prometheus-pushgateway=http://localhost:9091 \
--throttle-config='[{
sessions:"0",
protocol:"udp",
up: [
{ rate:1000, delay:100, queue:50 },
{ rate:500, delay:100, queue:50, at:60 },
{ rate:1000, delay:100, queue:50, at:120 },
]
}]'
Looking at the measured metrics, we see the effect of the bandwidth limitation change:
Using a fake media file with continuous speech when we run tests with multiple participants is not close to what happens in a real video conference room, when participants usually talk at turn without overlapping. The webrtcperf page automation permits to automatically activate/deactivate the audio tracks captured with getUserMedia changing the track.enabled property; the mechanism can be controlled with the following command line options:
--random-audio-period: It specifies the maximum period in seconds after which a new random active session is selected, enabling the getUserMedia audio tracks in that session and disabling all of the others.--random-audio-probability: it defines the probability % that the selected audio will be activated (default: 100%). Setting this value to value lower than 100, will have the effect of deactivating the audio for all the sessions.--random-audio-range: It defines the session indexes that should be included into the random selection mechanism (default: include all the sessions).Additionally to the track.enabled property, the tool will invoke the publisherSetMuted(true | false) function, if that function has been implemented in a custom page script (loaded with --script-path); this way it is possible to trigger the muted state clicking on the corresponding mute/unmute button inside the page, depending on the particular service we are testing.
Running tests with many participants increases the amount of CPU required to render the page, encode the audio/video streams and decode them at the receiver side. The webrtcperf docker image includes a modified Chromium version that allows the tool to specify a maximum number of RTC video streams to decode in parallel for each browser session. The value can be controlled using the following command line options:
--max-video-decoders: The maximum number of RTC video streams that will be decoded by each browser session (default: 0, decode all the videos).--max-video-decoders-range: It applies the max video decoders option to the sessions included into this list (default: “true” that will include all the sessions).It is worth noticing that disabling the video stream decoding would cause some RTC metrics (e.g. the received width, height, framerate) to always report a zero value. The suggested way to use this feature is keeping the video decoder enabled for a percentage of sessions, depending on the amount of cpu that we want to save.
The modified Chromium version can be build from sources using the following commands:
git clone https://github.com/vpalmisano/webrtcperf.git
cd webrtcperf/chromium
./build.sh setup
# Use a valid tag version
./build.sh update "tags/139.0.7230.1"
./build.sh build
# install the package (on Ubuntu/Debian)
sudo dpkg -i ./chromium-browser-unstable_139.0.7230.1-1_amd64.deb
Once that the custom Chromium package has been installed, specify the executable path with the --chromium-path option (or setting the CHROMIUM_PATH env variable):
export CHROMIUM_PATH=/usr/bin/chromium-browser-unstable
webrtcperf https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/scenarios/mediasoup.json
The tool offers several logging options to save the opened pages logs and, optionally, the full chromium process output.
--show-page-log: If true (default) the pages console logs will be shown on console.--page-log-filter: If set, only the logs with the matching text or regular expression will be printed on the console.--page-log-path: If set, the page console logs will be saved on the selected file path.--enable-browser-logging: It enables the Chromium browser logging for the specified session indexes. It requires the --page-log-path option to be set. When enabled, the tool will enable the chromium verbose logging (with --vmodule=*/webrtc/*=5) that will be written into the page log path directory. It also activates the Chromium --webrtc-event-logging option, that will save the event logging file in the same directory (you can use https://github.com/fippo/dump-webrtc-event-log to parse this file).In the previous examples we used the --script-path option providing a javascript page that runs some page automation (like automatically filling in the participant name and clicking on the join button). By default, the webrtcperf tool will load the webrtcperf-js script bundle, that provides a number of utilities to automate the testing tasks and run advanced measurements into the page context. The custom script provided with --script-path option can access the webrtcperf-js utilities under the webrtcperf global object. Some of the features offered by webrtcperf-js needs to be configured at the page load time; at this purpose, webrtcperf offers the --script-params option that can be used to provide a JSON/JSON5 object with the values that will be set into the webrtcperf.params object.
Setting the timestampWatermarkAudio and timestampWatermarkVideo script params will enable a webrtcperf-js feature that automatically applies a video watermark overlay and an audio modulated signal containing the sender UNIX timestamp. At the receiver side, the video overlay will be recognized using the Tesseract.js library, while the audio signal will be decoded with the ggwave library. Subtracting the decoded value from the UNIX timestamp at receiver side we will obtain the total end to end delay that occurs from the audio/video frame capture time to the display time (please note that the sender and receiver machine clocks need to be synchronized, otherwise we will introduce an error in the delay estimation).
Example usage:
webrtcperf \
--sessions=2 \
--run-duration=300 \
--url=https://meet.google.com/<ID> \
--debugging-port=9000 \
--prometheus-pushgateway=http://localhost:9091 \
--script-path=https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/google-meet.js \
--script-params='{timestampWatermarkAudio:true,timestampWatermarkVideo:true}'
The webrtcperf-js library comes with another mechanism that can be used to estimate the network end-to-end delay. Setting the absCaptureTime script param, the library will modify the SDP offer inserting the the abs-capture-time header extension (https://datatracker.ietf.org/doc/draft-ietf-avtcore-abs-capture-time/) for audio and video media sections. If the SFU supports this extension header, the receiver side will receive the captureTimestamp value into the corresponding contributing source. Consider also that some video conferencing applications (e.g. Google Meet) already uses this header, so you don’t need to add any script param to get the measurement.
The code used to calculate the delay is the following:
const contributingSource = receiver.getSynchronizationSources()[0]
const captureTimestamp = contributingSource?.captureTimestamp
const senderCaptureTimeOffset = contributingSource?.senderCaptureTimeOffset
if (contributingSource && captureTimestamp && senderCaptureTimeOffset !== undefined) {
endToEndDelay = (contributingSource.timestamp - (captureTimestamp + senderCaptureTimeOffset - 2208988800000)) / 1000
}
As we can see, the delay is calculated as the time difference from the current timestamp and the capture timestamp, using the time offset to compensate the clock difference between the sender and the receiver machines; both the capture and the offset are NTP time based (midnight UTC on January 1, 1900), so we need to rebase them to the UNIX start time (midnight UTC on January 1, 1970).
The End-to-End delay calculated with the abs-capture-time extension in a Google Meet test (audioEndToEndDelay and videoEndToEndDelay metrics):
The advantage of using this option is that it does not require any audio or video watermarking, and it will save some CPU at the receiver side required to run the image recognition. Anyway, this method takes into account only the RTP one-way delay on the network path; if you want to get a more accurate delay you should add the encoding (videoEncodeLatency) and the decoding (videoDecodeLatency) metrics values to the videoEndToEndDelay value.
Setting the actions script params value allows us to run several page tasks at different times, starting from the webrtcperf launch time. Each action object could include the following properties:
Example: assuming you have defined a muteParticipant custom function to mute/unmute the microphone, use this script actions to mute “Participant-000000” after 60 seconds and unmute it after 180 seconds form the test launch time:
webrtcperf \
--sessions=2 \
--run-duration=300 \
--url=https://meet.google.com/<ID> \
--debugging-port=9000 \
--prometheus-pushgateway=http://localhost:9091 \
--script-path=https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/google-meet.js \
--script-params='{
actions: [
{ name: "muteParticipant", params: [true], index: "0", at: 60 },
{ name: "muteParticipant", params: [false], index: "0", at: 180 },
]
}'
The audio bitrate sent by Participant-000000 reports an empty period in the muted period:
The audio bitrates received by Participant-000001 reflect the audio paused period:
Using the --response-modifiers option it is possible to run a string replacement of one or more HTTP requests made by the page (including XHR, assets, etc.). It accepts a JSON object with the URL request pattern and a list of one of more desired replacements.
Examples:
{ "https://url.com/*.js": [{ search: "searchString": replace: "anotherString" }] }{ "https://url.com/file.js": [{ file: "path/to/newFile.js" }] }Another useful option to debug the page requests is --download-reponses. It accepts an array of URL patterns that we want to save as local files. Example, saving the HLS mpegts fragments download while watching a live stream will require something like this:
[{ urlPattern: "https://url.com/*.ts", output: "save/directory" }]
--extra-headers: Specify this option to add custom HTTP headers to each request. Example:{ "https://url.com/*": { "Authorization": "Bearer secret" } }--extra-css: A string containing the CSS rules that will be injected inside the page content.--cookies: An array of CookieParam JSON objects ( https://pptr.dev/api/puppeteer.cookieparam) that will be set into the running web page.The tool accepts the following options that allow us to access the Chromium opened pages with the devtools inspector (chrome://inspect/#devices). This way we can access the page context while running the test to debug the web application.
The options that allow to configure the debugger are:
--debugging-port: If not zero, it will enable the Chromium debugger listening on the selected port.--debugging-address: The Chromium browser allows listening only on 127.0.0.1 address, so it is not possible to reach a session running inside a docker container or on a remote machine. When this option is set, webrtcperf will start a port forwarder allowing you to connect to the devtool protocol even if the page is running remotely.Other misc options that simplify the page debugging are:
--emulate-cpu-throttling: When different from zero, it will use the devtools CPU throttling feature in order to throttle the page javascript execution, emulating a slow device.--hardware-concurrency: It overrides the navigator.hardwareConcurrency property. Some web conference providers use this option to infer the machine capabilities and reduce the video resolution and framerate accordingly with the detected number of CPU cores.--device-scale-factor: The browser device scale factor to use (default: 1).--user-agent: A custom user agent string to use.--chromium-field-trials: A valid Chromium field trials override string.--local-storage and --session-storage: A JSON object that will be set in the page localStorage and sessionStorage at the page load.--override-permissions: A comma separated list of permissions to grant to the page. The possible values are listed here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Permissions-Policy#directives.--debug: It allows to increase the tool logging messages verbosity. See https://github.com/commenthol/debug-level#readme for the syntax.After running tests we need to usually look at the collected metrics in the Grafana interface or directly from the output logs and inspect them looking at packet loss values, low bandwidth or resolutions, etc.
Webrtcperf comes with a functionality that tries to automate the metrics inspection and the failure detection by running a value checking step every time the metrics are collected. The value checks can be specified providing a valid JSON string to the --alert-rules option containing a single object with the metrics that we want to check with the corresponding alert rule definition.
Example 1: we want to check if the sum of the active peer connections is equal to 100 after 1 minute from the test start.
--alert-rules='{
pages: {
tags: ["connectivity"],
sum: { "$eq": 100, "$after": 60 },
},
}'
Example 2: we want to check if, after 2 minutes, the number of received videos is equal to 25 and if the 5th percentile of the received bitrates is included into the 2-3 Mbps range.
--alert-rules='{
videoRecvBitrates: {
tags: ["quality"],
length: { "$eq": 25, "$after": 120 },
p5: { "$gt": 2000000, "$lt": 3000000, "$after": 120 },
},
}'
The full list of the metrics collected by webrtcperf is available here. The metric rules could include these specifiers:
The value checks could include:
The optional “tag” rule can be used to group the test rule into a custom category. If the --alert-rules-output is specified, at the end of the test the tool will write a JSON file with the test results grouped by category tag, reporting the percentage of time the metrics checked in each category failed to match the corresponding rules. You can use this output to trigger an automated alert in a monitoring system every time the test results are different from the expected values. Additionally, for each alert rule check, the tool will add a Prometheus _alert metric that can be used to visualize the alert threshold in Grafana and report if the metric check is failing or not.
Example 3: running a test with 2 participants visiting a Mediasoup demo page; the first participant will run with limited downlink network (1Mbps, 100ms delay) and we want to check if the video received bitrate falls into the interval 400Kbps - 1Mbps:
mediasoup_r1000-d100.yaml scenario configuration:
sessions: 2
runDuration: 300
url: 'https://v3demo.mediasoup.org/?roomId=webrtcperf-test-12345&displayName=Participant-$i'
debuggingPort: 9000
prometheusPushgateway: 'http://localhost:9091'
showPageLog: false
showStats: false
statsInterval: 5
throttleConfig: |
[{
sessions: "0",
protocol: "udp",
down: [
{ rate: 1000, delay: 100, queue: 50 },
]
}]
alertRules: |
{
peerConnections: {
tags: ["connectivity"],
sum: { "$eq": 4, "$after": 30 },
},
videoRecvBitrates: {
tags: ["quality"],
length: { "$eq": 2, "$after": 30 },
p5: { "$gt": 400000, "$lt": 1000000, "$after": 30 },
},
}
alertRulesOutput: 'results.json'
Test command:
webrtcperf mediasoup_r1000-d100.yaml
We can see the Grafana metric and the corresponding alert rule state; in this case the 5th percentile doesn't match the expected values, so we have some alert check failures.
The results.json content and the end of the test reports a test failure percentage of 48%, with the details about how many times and the total duration the check failed:
{
"tags": {
"connectivity": 0,
"quality": 48
},
"reports": {
"videoRecvBitrates p5 > 400000 and < 1000000 after 30s": {
"totalFails": 18,
"totalFailsTime": 50,
"valueAverage": 419731.18521030556,
"totalFailsTimePerc": 19,
"failAmount": 48,
"count": 54
}
}
}
If we change the videoRecvBitrates alert rule using "$gt": 100000 the test checks are successful instead:
Evaluating the video quality in most cases requires a visual inspection of the output streams searching for artifacts, deblocking effects, etc. Running this comparison in an automated way is preferable and it allows the test infrastructure to immediately report an alert when the detected quality doesn’t match the expected value while running the same test scenario in the same conditions.
VMAF (Video Multimethod Assessment Fusion) is a perceptual video quality assessment algorithm developed by Netflix (https://github.com/Netflix/vmaf). It's a full-reference metric, meaning it requires both a reference (original) and a distorted (encoded) video for evaluation. VMAF aims to predict video quality by combining several metrics into a single score value (ranging from 0 to 100). VMAF has been designed to assess the video encoding quality assuming that the distorted version of the video contains exactly the same video frames of the input video. This is not true in the WebRTC context, where the received video could be affected by delay (implicit network delay, jitter buffer, encode/decode operations) and packet loss. The missing perfect relationship between the sent and the received video frames makes it impossible to use the VMAF tool as it is, after simply recording the video streams at sender (reference) and at the receiver side (distorted). The webrtcperf tool integrates a video watermarking mechanism that we previously used to measure the end-to-end video delay (timestampWatermarkVideo). Having both the reference and the distorted video files with an overlay timestamp on each frame, the tool is able to process the video files, align them and filter the frames that have the same timestamp overlay. After the alignment step, the tool evaluates the video score using the libvmaf ffmpeg filter. What we get here is a perceptual indication of the video degradation caused by the encoder that could reduce the bitrate to match the available bandwidth. It is worth noticing that the tool will not evaluate the video quality for dropped (lost) frames, so the obtained VMAF score must be combined with the other WebRTC metrics (like the packet loss and the received video frame rate) to have a complete quality evaluation.
The suggested way to use this feature is running the webrtcperf docker image that already includes a patched ffmpeg version with the libvmaf filter.
Example test running the VMAF quality evaluation (start the prometheus stack before running the test):
docker run -it --rm \
-v /dev/shm:/dev/shm \
-v $PWD/data:/data \
-p 9000 \
--net=prometheus-stack_default \
ghcr.io/vpalmisano/webrtcperf:devel \
--sessions=2 \
--run-duration=180 \
--url=https://meet.google.com/<ID> \
--debugging-port=9000 \
--script-path=https://raw.githubusercontent.com/vpalmisano/webrtcperf/refs/heads/devel/examples/google-meet.js \
--prometheus-pushgateway=http://pushgateway:9091 \
--script-params="{timestampWatermarkVideo:true,saveSendVideoTrack:'0',saveRecvVideoTrack:'1'}" \
--server-port=5000 \
--server-use-https \
--server-data=/data \
--vmaf-path=/data \
--vmaf-preview
The tool will run a Google Meet conference with 2 participants, saving the “Participant-000000” sent video (saveSendVideoTrack:’0’) and the “Participant-000001” received video (saveRecvVideoTrack:'1'). We need to activate the tool in server mode (--server-port) in order to save the video frames captured inside the web page automation. Adding the --vmaf-path option will make the tool running the VMAF evaluation at the end of the test and --vmaf-preview will generate a side-by-side comparison between the reference and the degraded video.
The VMAF outputs are:
<Sender Participant>_recv-by_<Receiver Participant>; for this simple test it will be Participant-000000_recv-by_Participant-000001. In each folder we will find the following files:
psnr.log (the PSNR log);vmaf-log.json (the full VMAF output);vmaf-log.png (a graph made with the frame-by-frame scores);comparison.mp4 with the side-by-side comparison.vmaf-log.png:
vmaf.json:
[
{
"sender": "Participant-000000",
"receiver": "Participant-000001",
"min": 61.083063,
"max": 87.46276,
"mean": 76.260023,
"harmonic_mean": 76.118533
}
]
Other options for controlling the VMAF evaluation are:
--vmaf-keep-source-files: True by default, it keeps the recorded raw video files in the destination folder after the evaluation.--vmaf-keep-intermediate-files: False by default, when true all the intermediate video files used by the VMAF tool will be kept on the destination folder, allowing to debug them.--vmaf-skip-duplicated: False by default, when true the tool will skip the frames with the same recognized timestamp. Use this option if the encoder used by the service you are testing applies a frame duplication to match the configured framerate (e.g. this could happen when testing HLS videos).--vmaf-crop: a JSON string with a crop configuration to be applied to reference and/or degraded videos. The crop configuration should be expressed using the ffmpeg crop filter syntax (https://ffmpeg.org/ffmpeg-filters.html#crop).{
"Participant-000001_recv-by_Participant-000000": {
ref: { w: "iw-10", h: "ih-5" },
deg: { w: "200", h: "200" }
}
}
Running tests with multiple participants usually requires an amount of CPU and memory that could not match the host machine capabilities. Additionally, running tests with high resolution video and/or receiving streams from many remote participants in the same conference room will require an amount of bandwidth that could not be available on a single host. For these reason, when running tests with many concurrent participants, the preferred way to use webrtcperf is activating the server mode option in one instance (we will call it “collector’) and run all the other required webrtcperf test instances in different hosts (“worker’); the worker instances will collect the metrics scraped from the local running sessions and will send them to the collector, that will aggregate and send them to the Prometheus Pushgateway service.
Example command to start webrtcperf in “collector” mode on host 192.168.0.1:
webrtcperf \
--run-duration=200 \
--prometheus-pushgateway=http://127.0.0.1:9091 \
--server-port=5000 \
--server-use-https \
--server-secret=<SECRET>
Example command to start two “worker” instances on different hosts, running 50 participants each one:
# Worker 1
webrtcperf \
--sessions=50 \
--run-duration=180 \
--url=<URL> \
--push-stats-url=https://192.168.0.1:5000 \
--server-secret=<SECRET> \
--push-stats-id=1 \
--start-session-id=0 \
--start-timestamp=1749212181000
# Worker 2
webrtcperf \
--sessions=50 \
--run-duration=180 \
--url=<URL> \
--push-stats-url=https://192.168.0.1:5000 \
--server-secret=<SECRET> \
--push-stats-id=2 \
--start-session-id=50 \
--start-timestamp=1749212181000
Please note that you need to add the following options for each worker:
--push-stats-url.--push-stats-id that will be used by the collector to aggregate the metrics coming from that worker.--start-session-id to 50 in the second host, so the participants will be indexed starting from Participant-000050 to Participant-000099.--start-timestamp for both hosts and set it to the same UNIX timestamp expressed in milliseconds. This way all the action automation running on each host will be time synchronized (example: unmuting Participant-000001 and Participant-000051 at time 60 will be executed at the same clock time on both instances). Worker machines' clocks must be synchronized.--server-secret value for both collector and workers.For the collector configuration, use a longer --run-duration option just to avoid missing the last metric updates from the worker hosts.
The --prompt option allows you to run a test with an AI prompt that will be used to generate the test scenario configuration.
The prompt should be an accurate description of the test scenario that we want to run, including the number of participants, the service URL, the network throttling configuration, etc. The prompt will be sent to the Google Gemini AI service and the response will be parsed to generate a valid test configuration.
Example usage:
export GEMINI_API_KEY=<key>
webrtcperf --prompt "run a 2min test with 2 sessions on 'https://v3demo.mediasoup.org/?roomId=webrtcperf-test-12345&displayName=Participant-$i' sending the stats to pushgateway at http://localhost:9091 and limiting the 2nd session upstream at 1Mbps with 1% packet loss for 30s, 2Mbps for 30s and 1Mbps for all the remaining time and disable page logs"
FAQs
 # WebRtcPerf [GitHub page](https://github.com/vpalmisano/webrtcperf) | [Documentation](https://vpalmisano.github.io/webrtcperf)
We found that @vpalmisano/webrtcperf demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ECMAScript 2025 introduces Iterator Helpers, Set methods, JSON modules, and more in its latest spec update approved by Ecma in June 2025.
Security News
A new Node.js homepage button linking to paid support for EOL versions has sparked a heated discussion among contributors and the wider community.
Research
North Korean threat actors linked to the Contagious Interview campaign return with 35 new malicious npm packages using a stealthy multi-stage malware loader.