Support H265 for WebRTC in Safari with go2rtc

go2rtc is one of the first projects in the World, that support HEVC/H.265 for WebRTC in web browser.

Maybe you can see articles about WebRTC H265 earlier. But I haven't seen any examples where the video was displayed in a popular browser. Some projects can support WebRTC H265 if they are used on the receiving and transmitting sides. But what if there is a real browser on one side?

I tried streaming the RTSP from the Dahua camera to Safari with Janus without success. This works for some H264 cameras, but not all of them. This can work for streaming H265 between two Apple devices. But this does not work for H265 cameras.

All because Apple doesn't respect world standards. You can forgot RTP Payload Format for High Efficiency Video Coding (HEVC) (RFC 7798) then work with Apple devices. They invented their own payload:

You need a correct H265 stream: VPS, SPS, PPS, I-frame, P-frame(s).
You need it with Annex-B headers 00 00 00 01 before each NAL unit.
You need split your stream on RTP payloads with one byte header:
- 03 for payload with VPS, SPS, PPS, I-frame start
- 01 for all next packets from this I-frame
- 02 for payload with P-frame
- 00 for all next packets from this P-frame
Don't forget set marker flag only for last packet of each Access Units

That's all. Now anyone can try to watch WebRTC H265 in Safari even if don't have H265 cameras. This is because go2rtc has optional transcode feature for almost any stream source:

streams:
  sonoff_camera: ffmpeg:rtsp://rtsp:12345678@192.168.1.123:554/av_stream/ch0#video=h265#audio=copy

If you have H265 camera, you can stream it without transcoding, even from Xiaomi Gateway 3. Because go2rtc is a small application that you can install almost anywhere.

Unfortunately Safari is the only browser at the moment that supports H265 for WebRTC.

Codecs madness

Device	WebRTC	MSE	MP4
latency	best	medium	bad
Desktop Chrome	H264	H264, H265*	H264, H265*
Desktop Safari	H264, H265*	H264	no
Desktop Edge	H264	H264, H265*	H264, H265*
Desktop Firefox	H264	H264	H264
Desktop Opera	no	H264	H264
iPhone Safari	H264, H265*	no	no
iPad Safari	H264, H265*	H264	no
Android Chrome	H264	H264	H264
masOS Hass App	no	no	no

Спасибо Вам,Вы делаете огромную и трудоемкую работу!За Getwei 3 Отдельное СПАСИБИЩЕ!

use webcodecs

Edge support H265 as well.

@itispip

@itispip can you show SDP that confirms your words?

[Assistance needed]: Only audio played during WebRTC test via RTSP stream from H.265 file on Safari

Description: I encountered an issue while testing the WebRTC via RTSP stream from an H.265 file on Safari. When I tried to stream from a file encoded with libx265 and hevc_nvenc with opus for audio, only the audio is played for WebRTC, and only the first picture is shown for MSE.
Here's what I did:

Transcoded the file to an mp4 using libx265 and hevc_nvenc:

ffmpeg -re -i puppets_with_greenbg_hevc.mov -pix_fmt yuv420p -c:v libx265 -g 50 -preset:v superfast -tune:v zerolatency -c:a libopus -ar:a 48000 -ac:a 2 puppets_with_greenbg_x265_opus.mp4
ffmpeg -re -i puppets_with_greenbg_hevc.mov -c:v hevc_nvenc -g 50 -level:v auto -strict -2 -c:a libopus -ar:a 48000 -ac:a 2 puppets_with_alpha_hevc_nvenc_opus.mp4

Allowed WebRTC h265 in "Experimental Features" and tried H.264 streaming from a file, which worked well using the same settings in the go2rtc.yaml and ffmpeg commands for streaming.
Streamed RTSP from a file to go2rtc:
ffmpeg -re -stream_loop -1 -i puppets_with_greenbg_hevc_nvenc_opus.mp4 -c copy -f rtsp rtsp://localhost:8554/stream1
Used the following go2rtc.yaml:

streams:
  stream1: ""

info: http://127.0.0.1:1984/api/streams?src=stream1

{
  "producers": [
    {},
    {
      "type": "RTSP source",
      "url": "rtsp://localhost:8554/stream1",
      "remote_addr": "[::1]:65513",
      "user_agent": "Lavf59.27.100",
      "medias": [
        "video, sendonly, 96 H265/90000",
        "audio, sendonly, 97 OPUS/48000/2"
      ],
      "tracks": [
        "96 H265/90000, sinks=1",
        "97 OPUS/48000/2, sinks=1"
      ],
      "recv": 6700364
    }
  ],
  "consumers": [
    {
      "type": "WebRTC client",
      "remote_addr": "udp4 prflx 192.168.2.10:60248 related :0",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15",
      "send": 2613086
    }
  ]
}

I would appreciate your guidance on what I might be doing wrong. Thank you in advance for your help

H264 and H265 from files may not be compatible for streaming. They may have B-frames or some others unsupported NAL types.

I don't support hardware encoding for H265 NVENC exact because it produce some kind of "bad" stream. Haven't had time to figure out the problem and fix it yet.

The best stream is made by cameras of good brands, such as Dahua.

Dear AlexxIT,

Thank you for your prompt response. Do you have an example of an H.265+opus file that works well with go2rtc? Alternatively, could you provide me with a test stream from a Dahua camera or any other HEVC RTSP test stream that I can use for testing purposes? I ask this as I am currently unable to purchase a Dahua camera.

Also, regarding NVENC, I don't require a hardware video encoder. Instead, a software video encoder such as libx265 would suffice for my current needs.

Thank you for your assistance.
Best regards, Evgeny

Dear AlexxIT,

I hope you are doing well. I have been attempting to create an RTSP stream without audio, and while it worked for MSE, I have yet to make it work for WebRTC. So I have a question about MSE - is it considered a Progressive Download?

Please provide me with some assistance with WebRTC. I would appreciate your help.
Thank you and best regards, Evgeny

Here are the ffmpeg commands I used for creating the RTSP stream without audio:

ffmpeg -re -i puppets_with_greenbg_hevc.mov -pix_fmt yuv420p -c:v libx265 -g 50 -preset:v superfast -tune:v zerolatency -bsf:v hevc_mp4toannexb -c:a libopus -ar:a 48000 -ac:a 2  -f mp4 -y puppets_with_greenbg_libx265_opus.mp4

ffmpeg -re -stream_loop -1 -i puppets_with_greenbg_libx265_opus.mp4 -c copy -an -rtsp_transport tcp -f rtsp rtsp://localhost:8554/stream1

Thank you.

Are you sure you using WebRTC in Apple Safari with enabled H265 for WebRTC checkbox?

Hi AlexxIT,
I enabled the "WebRTC H265 codec" feature under "Develop" > "Experimental Features." After that, I attempted to restart Safari, but should I do anything else?

Hi AlexxIT,

I'm having trouble finding a working RTSP H.265 stream for Safari browser. I've tested my RTSP stream on Safari versions 16.3 (Apple M1, macOS Ventura v13.2.1) and 14.1.2 (macOS Mojave v10.14.6), but haven't had any success with WebRTC. Do you know where I can find a working stream?

Thank you, Evgeny

Hi AlexxIT,

I hope this message finds you well. I have successfully tested the rtsp stream from go2rtc on the VLC media player. Here is the URL: rtsp://192.168.2.30:8554/stream1

I have tried it on both my MacBook Air and iPad, and everything appears to be working well, including the original HEVC file with alpha channel "puppets_with_alpha_hevc.mov" with AAC from Apple.

ffmpeg -re -stream_loop -1 -i puppets_with_alpha_hevc.mov -c copy -f rtsp rtsp://localhost:8554/stream1

However, I am experiencing an issue making WebRTC with HEVC work on the Safari browser. I would like your assistance in resolving this issue. I would appreciate your prompt response and help.

Thank you very much.
Best regards,
Evgeny

I implemented the same at June, 2022 https://blog.wmspanel.com/2022/07/hevc-webrtc-apple.html
I tried to handle H265 RTP from Apple using standard Pion unpackager and of course failed.
I had to process it manually the same way and was reading RFC 7798) again and again thinking I am doing something wrong. I tried to emphasize this issue here pion/webrtc#2271 and here https://groups.google.com/g/discuss-webrtc/c/Zc3-3hddEn0 but nobody cares. Unfortunately I was the first who implemented this.
If I wait a few month and found your post this would save me a lot of time.

@alexpokotilo

@alexpokotilo fair enough. I have removed phrase about "first in the World" from readme. Go2rtc has many others "first in the World". No any problem :)

If the client supports only H264, main has H265 codec and sub has H264 codec, everything should work. Go2rtc supports this.

@AlexxIT

Thanks @AlexxIT , that was my understanding as well. Ah, I was struggling a bit with turning on logging for RTSP but got it going now, and I indeed see that go2rtc queries (Describe) both main and sub and guess what, the camera (foscam x5) reports only h265 for both main and sub (grrrr). Reolink TrackMix seems to do the same on all 3 substreams. hmmm what's up with that....

If your browser advertises H.265 as playable, then it should play it. Otherwise it should have advertised as non playable.

It may be worth reporting a chromium bug for this.

@AlexxIT

If your device doesn't support the H265 codec, it's not go2rtc's problem. No need to create new issues :)

Due to licensing issues, this codec must be supported not only at the software level (such as the latest version of the Chrome browser), but also at the hardware level. The hardware developer has to pay money to the patent owner of this codec.

The problem is not that it does not support it. The problem is that it advertises as supported:

receiver video
video/H265,,90000,level-id=90;profile-id=1;tier-flag=0;tx-mode=SRST

video.canPlayType
video/mp4; codecs="hev1.1.6.L93.B0" = true / probably

@AlexxIT, it could either be an issue with go2rtc or an issue with Chromium as well. I think it should be reported so that it can be investigated.

And if there's indeed no issue with go2rtc, I would recommend reporting a bug to Chromium. The browser shouldn't advertise H.265 as playable if it is not.

I reported the issue with my tablet to Chromium, and it just got fixed (only in Chrome Canary for now). Now my Teclast P20HD with Android 10 and an Unisoc SC9863A can play H.265 over WebRTC.

If the browser shows that it can play this codec - then of course it could be a go2rtc problem.

But, for example, there's a browser like Safari. The worst browser to date, in my opinion (because IE is no longer relevant).
Its older versions are lying about the list of supported codecs. So I had to write a workaround code that solves this issue:
https://github.com/AlexxIT/go2rtc/blob/ae8145f26696ffc19a13fea4a21b4a5306fc0e3a/www/video-rtc.js#L256-L262

It is better to create a separate issue in the go2rtc project to clarify your question. There is a lot of offtopic stuff here, which does not concern the original post.
Your offer and answer contain H265 codec. I don't see any problems here.

Navigation Menu

Repository navigation

Support H265 for WebRTC in Safari with go2rtc #5

18 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Support H265 for WebRTC in Safari with go2rtc #5

Description

Activity

Frend13 commented on Sep 22, 2022

xiangxud commented on Oct 4, 2022

itispip commented on Mar 5, 2023

AlexxIT commented on Mar 5, 2023

compressionist commented on Mar 5, 2023

AlexxIT commented on Mar 7, 2023

compressionist commented on Mar 7, 2023

compressionist commented on Mar 7, 2023

AlexxIT commented on Mar 8, 2023

compressionist commented on Mar 8, 2023

compressionist commented on Mar 10, 2023

compressionist commented on Mar 14, 2023

alexpokotilo commented on Jul 21, 2023

AlexxIT commented on Jul 21, 2023

18 remaining items

AlexxIT commented on Jun 3, 2025

dcorsus commented on Jun 3, 2025

felipecrs commented on Jun 3, 2025

felipecrs commented on Jun 4, 2025

AlexxIT commented on Jun 4, 2025

dcorsus commented on Jun 4, 2025

dcorsus commented on Jun 4, 2025

AlexxIT commented on Jun 4, 2025

dcorsus commented on Jun 4, 2025

AlexxIT commented on Jun 4, 2025

dcorsus commented on Jun 4, 2025

AlexxIT commented on Jun 5, 2025

dcorsus commented on Jun 5, 2025

dcorsus commented on Jun 5, 2025

AlexxIT commented on Jun 5, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions