SlideShare a Scribd company logo
Bringing real-time text to WebRTC
for NG Emergency Services
Lorenzo Miniero
@lminiero@fosstodon.org
Kamailio World
June 7th 2023,
Who am I?
Lorenzo Miniero
• Ph.D @ UniNA
• Chairman @ Meetecho
• Main author of Janus
Contacts and info
• lorenzo@meetecho.com
• https://fosstodon.org/@lminiero
• https://www.slideshare.net/LorenzoMiniero
• https://lminiero.bandcamp.com
Just a few words on Meetecho
• Co-founded in 2009 as an academic spin-off
• University research efforts brought to the market
• Completely independent from the University
• Focus on real-time multimedia applications
• Strong perspective on standardization and open source
• Several activities
• Consulting services
• Commercial support and Janus licenses
• Streaming of live events (IETF, ACM, etc.)
• Proudly brewed in sunny Napoli(*), Italy
((*)
A different picture, for once!)
Emergency Services for the Hearing Impaired
• Ad-hoc text telephones and telecommunication devices
• Originally conceived and implemented in the 60’s
• Commonly known as TDD / TTY / TT
• “Telecommunications Device for the Deaf”
• Units made of keyboard, small screen and modem
• Messages converted to electric signals
• Phone line used to deliver messages
• Widely used in legacy Emergency Services
• e.g., to allow 911 operators to engage with hard of hearing people
• Uses phone line + batteries −→ still works in power failure
Emergency Services for the Hearing Impaired
• Ad-hoc text telephones and telecommunication devices
• Originally conceived and implemented in the 60’s
• Commonly known as TDD / TTY / TT
• “Telecommunications Device for the Deaf”
• Units made of keyboard, small screen and modem
• Messages converted to electric signals
• Phone line used to deliver messages
• Widely used in legacy Emergency Services
• e.g., to allow 911 operators to engage with hard of hearing people
• Uses phone line + batteries −→ still works in power failure
Emergency Services for the Hearing Impaired
• Ad-hoc text telephones and telecommunication devices
• Originally conceived and implemented in the 60’s
• Commonly known as TDD / TTY / TT
• “Telecommunications Device for the Deaf”
• Units made of keyboard, small screen and modem
• Messages converted to electric signals
• Phone line used to deliver messages
• Widely used in legacy Emergency Services
• e.g., to allow 911 operators to engage with hard of hearing people
• Uses phone line + batteries −→ still works in power failure
TDD/TTY/TT (PSTN)
TDD/TTY/TT (PSTN)
• Different acronyms for mostly similar things
• TDD (Telecommunications Device for the Deaf)
• TTY (TeleTYpe)
• TT (Text Telephone)
• Some important limitations
• Half-duplex (GA = GO AHEAD)
• Can’t be interrupted (garbled output)
• Limited set of “characters” (e.g., Baudot)
• No specific handshake, and no error correction
• Requires specific device
TDD/TTY/TT (PSTN)
• Different acronyms for mostly similar things
• TDD (Telecommunications Device for the Deaf)
• TTY (TeleTYpe)
• TT (Text Telephone)
• Some important limitations
• Half-duplex (GA = GO AHEAD)
• Can’t be interrupted (garbled output)
• Limited set of “characters” (e.g., Baudot)
• No specific handshake, and no error correction
• Requires specific device
Next Generation Emergency Services
• Multiple efforts in different areas
• NG9-1-1 (https://www.911.gov/issues/ng911/)
• NG112 (https://eena.org/next-generation-112/)
• Attempt to switch to modern ways of communication
• Existing NG services based on PSTN
• Internet-based communication allows for richer media, though
• Location, , photos, videos, etc.
Next Generation Emergency Services
• Multiple efforts in different areas
• NG9-1-1 (https://www.911.gov/issues/ng911/)
• NG112 (https://eena.org/next-generation-112/)
• Attempt to switch to modern ways of communication
• Existing NG services based on PSTN
• Internet-based communication allows for richer media, though
• Location, , photos, videos, etc.
Next Generation Emergency Services
• Multiple efforts in different areas
• NG9-1-1 (https://www.911.gov/issues/ng911/)
• NG112 (https://eena.org/next-generation-112/)
• Attempt to switch to modern ways of communication
• Existing NG services based on PSTN
• Internet-based communication allows for richer media, though
• Location, real-time text, photos, videos, etc.
Next Generation Emergency Services
• Multiple efforts in different areas
• NG9-1-1 (https://www.911.gov/issues/ng911/)
• NG112 (https://eena.org/next-generation-112/)
• Attempt to switch to modern ways of communication
• Existing NG services based on PSTN
• Internet-based communication allows for richer media, though
• Location, real-time text, photos, videos, etc.
Real-Time Text (RTT)
• Text transmitted instantly, as it is typed or created
• TDD/TTY work that way too
• How to do it over IP, though?
• Text over IP (ToIP)
• ITU-T T.140 (Protocol for multimedia application text conversation)
• Allows for real-time editing (e.g., backspace, rewriting)
• T.140 messages transported over RTP
• RFC 4103 (RTP Payload for Text Conversation)
• Redundancy implemented via RED (RFC 2198)
Real-Time Text (RTT)
• Text transmitted instantly, as it is typed or created
• TDD/TTY work that way too
• How to do it over IP, though?
• Text over IP (ToIP)
• ITU-T T.140 (Protocol for multimedia application text conversation)
• Allows for real-time editing (e.g., backspace, rewriting)
• T.140 messages transported over RTP
• RFC 4103 (RTP Payload for Text Conversation)
• Redundancy implemented via RED (RFC 2198)
Real-Time Text (RTT)
• Text transmitted instantly, as it is typed or created
• TDD/TTY work that way too
• How to do it over IP, though?
• Text over IP (ToIP)
• ITU-T T.140 (Protocol for multimedia application text conversation)
• Allows for real-time editing (e.g., backspace, rewriting)
• T.140 messages transported over RTP
• RFC 4103 (RTP Payload for Text Conversation)
• Redundancy implemented via RED (RFC 2198)
T.140
• ITU-T T.140 (Protocol for multimedia application text conversation)
• https://www.itu.int/rec/T-REC-T.140/
• Protocol based on the concept of T140blocks
• Set of characters or special codes that one party is delivering to one or more others
• Participant typing −→ typed characters bundled together in a T.140 block
• Buffering size up to sender (overhead vs. latency)
• UTF-8 used for characters, including special codes (actions)
• e.g., Byte Order Mark (BOM), backspace, etc.
T.140
• ITU-T T.140 (Protocol for multimedia application text conversation)
• https://www.itu.int/rec/T-REC-T.140/
• Protocol based on the concept of T140blocks
• Set of characters or special codes that one party is delivering to one or more others
• Participant typing −→ typed characters bundled together in a T.140 block
• Buffering size up to sender (overhead vs. latency)
• UTF-8 used for characters, including special codes (actions)
• e.g., Byte Order Mark (BOM), backspace, etc.
T.140
• ITU-T T.140 (Protocol for multimedia application text conversation)
• https://www.itu.int/rec/T-REC-T.140/
• Protocol based on the concept of T140blocks
• Set of characters or special codes that one party is delivering to one or more others
• Participant typing −→ typed characters bundled together in a T.140 block
• Buffering size up to sender (overhead vs. latency)
• UTF-8 used for characters, including special codes (actions)
• e.g., Byte Order Mark (BOM), backspace, etc.
T.140 over RTP
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC=0 |M| T140 PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp (1000Hz) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| T.140 encoded data |
+ +---------------+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Dealing with packet loss using RED
Dealing with packet loss using RED
Dealing with packet loss using RED
Dealing with packet loss using RED
Dealing with packet loss using RED
T.140 over RTP (with redundancy)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC=0 |M| "RED" PT | sequence number of primary |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp of primary encoding "P" |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| T140 PT | timestamp offset of "R" | "R" block length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| T140 PT | "R" T.140 encoded redundant data |
+-+-+-+-+-+-+-+-+ +---------------+
+ | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+
| "P" T.140 encoded primary data |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
RTT and SIP/SDP
• As an RTP-based stream, easy to negotiate via SDP
• Needs text as media in m-line
• t140 used to advertise codec in rtpmap
• Sampling rate always 1000 (also for RED)
SDP example
m=text 11000 RTP/AVP 98 100
a=rtpmap:98 t140/1000
a=rtpmap:100 red/1000
a=fmtp:100 98/98/98
RTT and SIP/SDP
• As an RTP-based stream, easy to negotiate via SDP
• Needs text as media in m-line
• t140 used to advertise codec in rtpmap
• Sampling rate always 1000 (also for RED)
SDP example
m=text 11000 RTP/AVP 98 100
a=rtpmap:98 t140/1000
a=rtpmap:100 red/1000
a=fmtp:100 98/98/98
RTT and SIP/SDP
• As an RTP-based stream, easy to negotiate via SDP
• Needs text as media in m-line
• t140 used to advertise codec in rtpmap
• Sampling rate always 1000 (also for RED)
SDP example
m=text 11000 RTP/AVP 98 100
a=rtpmap:98 t140/1000
a=rtpmap:100 red/1000
a=fmtp:100 98/98/98
RTT and SIP/SDP
• As an RTP-based stream, easy to negotiate via SDP
• Needs text as media in m-line
• t140 used to advertise codec in rtpmap
• Sampling rate always 1000 (also for RED)
SDP example
m=text 11000 RTP/AVP 98 100
a=rtpmap:98 t140/1000
a=rtpmap:100 red/1000
a=fmtp:100 98/98/98
RTT and SIP/SDP
• As an RTP-based stream, easy to negotiate via SDP
• Needs text as media in m-line
• t140 used to advertise codec in rtpmap
• Sampling rate always 1000 (also for RED)
SDP example
m=text 11000 RTP/AVP 98 100
a=rtpmap:98 t140/1000
a=rtpmap:100 red/1000
a=fmtp:100 98/98/98
RTT and SIP/SDP
• As an RTP-based stream, easy to negotiate via SDP
• Needs text as media in m-line
• t140 used to advertise codec in rtpmap
• Sampling rate always 1000 (also for RED)
SDP example
m=text 11000 RTP/AVP 98 100
a=rtpmap:98 t140/1000
a=rtpmap:100 red/1000
a=fmtp:100 98/98/98
RTT and SIP/SDP
• As an RTP-based stream, easy to negotiate via SDP
• Needs text as media in m-line
• t140 used to advertise codec in rtpmap
• Sampling rate always 1000 (also for RED)
SDP example
m=text 11000 RTP/AVP 98 100
a=rtpmap:98 t140/1000
a=rtpmap:100 red/1000
a=fmtp:100 98/98/98
What about WebRTC?
• WebRTC heavily based on existing VoIP technologies
• SDP (on “steroids”) is mandatory
• RTP used for media (encrypted via DTLS-SRTP)
• That said, not all RTP media is supported
• Only m=audio and m=video supported out of the box
• No support at all for m=text, so RTT won’t work
• Only m=application media that can be used is UDP/DTLS/SCTP
• WebRTC data channels
• Generic data sent via SCTP and encapsulated in DTLS
What about WebRTC?
• WebRTC heavily based on existing VoIP technologies
• SDP (on “steroids”) is mandatory
• RTP used for media (encrypted via DTLS-SRTP)
• That said, not all RTP media is supported
• Only m=audio and m=video supported out of the box
• No support at all for m=text, so RTT won’t work
• Only m=application media that can be used is UDP/DTLS/SCTP
• WebRTC data channels
• Generic data sent via SCTP and encapsulated in DTLS
What about WebRTC?
• WebRTC heavily based on existing VoIP technologies
• SDP (on “steroids”) is mandatory
• RTP used for media (encrypted via DTLS-SRTP)
• That said, not all RTP media is supported
• Only m=audio and m=video supported out of the box
• No support at all for m=text, so RTT won’t work
• Only m=application media that can be used is UDP/DTLS/SCTP
• WebRTC data channels
• Generic data sent via SCTP and encapsulated in DTLS
T.140 RTT over WebRTC Data Channels
• Why not use data channels for RTT, then?
• T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865)
• https://www.rfc-editor.org/rfc/rfc8865.html
• Sending T.140 frames directly in data channels, not on RTP
• Data channels can be configured to be ordered and reliable
• Helps avoiding hoops needed for RTP (e.g., RED)
• A specific label can be associated with a specific RTT session
• Of course, needs gatewaying to be backwards compatible
• WebRTC endpoints can’t do RTP/RTT, and legacy ones don’t do data channels
• Something needs to “translate” (media and SDP)
T.140 RTT over WebRTC Data Channels
• Why not use data channels for RTT, then?
• T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865)
• https://www.rfc-editor.org/rfc/rfc8865.html
• Sending T.140 frames directly in data channels, not on RTP
• Data channels can be configured to be ordered and reliable
• Helps avoiding hoops needed for RTP (e.g., RED)
• A specific label can be associated with a specific RTT session
• Of course, needs gatewaying to be backwards compatible
• WebRTC endpoints can’t do RTP/RTT, and legacy ones don’t do data channels
• Something needs to “translate” (media and SDP)
T.140 RTT over WebRTC Data Channels
• Why not use data channels for RTT, then?
• T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865)
• https://www.rfc-editor.org/rfc/rfc8865.html
• Sending T.140 frames directly in data channels, not on RTP
• Data channels can be configured to be ordered and reliable
• Helps avoiding hoops needed for RTP (e.g., RED)
• A specific label can be associated with a specific RTT session
• Of course, needs gatewaying to be backwards compatible
• WebRTC endpoints can’t do RTP/RTT, and legacy ones don’t do data channels
• Something needs to “translate” (media and SDP)
T.140/data channels SDP usage
m=application 911 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP6 2001:db8::3
a=max-message-size:1000
a=sctp-port 5000
a=setup:actpass
a=dcmap:2 label="ACME customer service";subprotocol="t140"
a=dcsa:2 fmtp:t140 cps=20
a=dcsa:2 hlang-send:es eo
a=dcsa:2 hlang-recv:es eo
Prototyping T.140/WebRTC in Janus
• Created a prototype in Janus a few years ago
• Old PR: https://github.com/meetecho/janus-gateway/pull/1898 (closed)
• Blog post: https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• Used the SIP plugin in Janus as the necessary gateway
• Negotiates text m-lines and the text/t140 format on the SDP side
• Negotiates data channels on the WebRTC side (binary)
• T140blocks are then bridged (no translation) as they are
• T140blocks decapsulated from RTP packets −→ data channels
• Data channels −→ crafting RTP packets with T140blocks
• RED optionally supported in both directions on the RTP side
Prototyping T.140/WebRTC in Janus
• Created a prototype in Janus a few years ago
• Old PR: https://github.com/meetecho/janus-gateway/pull/1898 (closed)
• Blog post: https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• Used the SIP plugin in Janus as the necessary gateway
• Negotiates text m-lines and the text/t140 format on the SDP side
• Negotiates data channels on the WebRTC side (binary)
• T140blocks are then bridged (no translation) as they are
• T140blocks decapsulated from RTP packets −→ data channels
• Data channels −→ crafting RTP packets with T140blocks
• RED optionally supported in both directions on the RTP side
Prototyping T.140/WebRTC in Janus
• Created a prototype in Janus a few years ago
• Old PR: https://github.com/meetecho/janus-gateway/pull/1898 (closed)
• Blog post: https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• Used the SIP plugin in Janus as the necessary gateway
• Negotiates text m-lines and the text/t140 format on the SDP side
• Negotiates data channels on the WebRTC side (binary)
• T140blocks are then bridged (no translation) as they are
• T140blocks decapsulated from RTP packets −→ data channels
• Data channels −→ crafting RTP packets with T140blocks
• RED optionally supported in both directions on the RTP side
SIP plugin in Janus
https://janus.conf.meetecho.com/docs/sip
SIP plugin in Janus + RTT
Example: SDP offer from SIP/RTT endpoint
v=0
o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74
s=Omnitor_SDP_v1.1
c=IN IP4 192.168.1.74
t=0 0
m=text 1024 RTP/AVP 99 98
a=rtpmap:99 red/1000
a=fmtp:99 98/98/98
a=rtpmap:98 t140/1000
Example: SDP offer to WebRTC endpoint
v=0
o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74
s=Omnitor_SDP_v1.1 t=0 0
a=group:BUNDLE data
a=msid-semantic: WMS janus
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 192.168.1.74
a=sendrecv
a=sctp-port:5000
a=mid:0
[.. ICE/DTLS details follow..]
Testing: TIPcon1 (SIP/RTT)
https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
Testing: Janus SIP demo (WebRTC)
https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
T.140 conversation on the wire
https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
What’s next?
• First of all, revive the effort
• The original PR was closed a few months ago
• Little to no feedback, and core had changed a lot in the meanwhile
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• New revision can take advantage of some new features
• e.g., RED supported in the Janus core, now
• Maybe write a prettier and more functional UI too...
• Possibly test with something more complex than TIPcon1
• TIPcon1 good for basic functional testing, but very limited
• RTT endpoints very hard to find, especially in FOSS
• Possibly integration tests with actual RTT deployments?
What’s next?
• First of all, revive the effort
• The original PR was closed a few months ago
• Little to no feedback, and core had changed a lot in the meanwhile
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• New revision can take advantage of some new features
• e.g., RED supported in the Janus core, now
• Maybe write a prettier and more functional UI too...
• Possibly test with something more complex than TIPcon1
• TIPcon1 good for basic functional testing, but very limited
• RTT endpoints very hard to find, especially in FOSS
• Possibly integration tests with actual RTT deployments?
What’s next?
• First of all, revive the effort
• The original PR was closed a few months ago
• Little to no feedback, and core had changed a lot in the meanwhile
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• New revision can take advantage of some new features
• e.g., RED supported in the Janus core, now
• Maybe write a prettier and more functional UI too...
• Possibly test with something more complex than TIPcon1
• TIPcon1 good for basic functional testing, but very limited
• RTT endpoints very hard to find, especially in FOSS
• Possibly integration tests with actual RTT deployments?
What’s next?
• First of all, revive the effort
• The original PR was closed a few months ago
• Little to no feedback, and core had changed a lot in the meanwhile
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• New revision can take advantage of some new features
• e.g., RED supported in the Janus core, now
• Maybe write a prettier and more functional UI too...
• Possibly test with something more complex than TIPcon1
• TIPcon1 good for basic functional testing, but very limited
• RTT endpoints very hard to find, especially in FOSS
• Possibly integration tests with actual RTT deployments?
What’s next?
• First of all, revive the effort
• The original PR was closed a few months ago
• Little to no feedback, and core had changed a lot in the meanwhile
• New PR: https://github.com/meetecho/janus-gateway/pull/3231
• New revision can take advantage of some new features
• e.g., RED supported in the Janus core, now
• Maybe write a prettier and more functional UI too...
• Possibly test with something more complex than TIPcon1
• TIPcon1 good for basic functional testing, but very limited
• RTT endpoints very hard to find, especially in FOSS
• Possibly integration tests with actual RTT deployments?
Thanks! Questions? Comments?
Get in touch!
• https://fosstodon.org/@lminiero
• https://twitter.com/elminiero
• https://twitter.com/meetecho
• https://www.meetecho.com/blog/

More Related Content

PDF
Sipwise rtpengine
Andreas Granig
 
PDF
rtpengine and kamailio - or how to simulate calls at scale
Andreas Granig
 
PDF
SIP Testing with FreeSWITCH
Moises Silva
 
PDF
Three Ways Kamailio Can Help Your FreeSWITCH Deployment
Fred Posner
 
PDF
Janus/SIP @ OpenSIPS 2019
Lorenzo Miniero
 
PPTX
FreeSWITCH as a Kickass SBC
Moises Silva
 
PDF
rtpengine - Media Relaying and Beyond
Andreas Granig
 
PDF
Apache kafka
Shravan (Sean) Pabba
 
Sipwise rtpengine
Andreas Granig
 
rtpengine and kamailio - or how to simulate calls at scale
Andreas Granig
 
SIP Testing with FreeSWITCH
Moises Silva
 
Three Ways Kamailio Can Help Your FreeSWITCH Deployment
Fred Posner
 
Janus/SIP @ OpenSIPS 2019
Lorenzo Miniero
 
FreeSWITCH as a Kickass SBC
Moises Silva
 
rtpengine - Media Relaying and Beyond
Andreas Granig
 
Apache kafka
Shravan (Sean) Pabba
 

What's hot (20)

PDF
FreeSWITCH on Docker
建澄 吳
 
PDF
Scaling Asterisk with Kamailio
Fred Posner
 
PDF
Kamailio, FreeSWITCH, and You
Fred Posner
 
PPTX
Hashicorp Vault Open Source vs Enterprise
Stenio Ferreira
 
PDF
Astricon 10 (October 2013) - SIP over WebSocket on Kamailio
Crocodile WebRTC SDK and Cloud Signalling Network
 
PDF
Continuous Integration and Kamailio
Giacomo Vacca
 
PDF
SIP transfer with Janus/WebRTC @ OpenSIPS 2022
Lorenzo Miniero
 
PDF
Asterisk WebRTC frontier: make client SIP Phone with sipML5 and Janus Gateway
Alessandro Polidori
 
ODP
Expanding Asterisk with Kamailio
Fred Posner
 
PDF
Introduction to Open Telemetry as Observability Library
Tonny Adhi Sabastian
 
PDF
Kamailio :: A Quick Introduction
Olle E Johansson
 
PDF
Introduction to Istio on Kubernetes
Jonh Wendell
 
PDF
RootedCON 2017 - Docker might not be your friend. Trojanizing Docker images
Daniel Garcia (a.k.a cr0hn)
 
PDF
Linux Linux Traffic Control
SUSE Labs Taipei
 
PPTX
FreeSBC How To - Advanced SIP Routing
Alan Percy
 
PDF
Scaling FreeSWITCH Performance
Moises Silva
 
PPT
Linux networking
Armando Reis
 
PDF
Homer - Workshop at Kamailio World 2017
Giacomo Vacca
 
PDF
OPENMARU APM 브로셔
Opennaru, inc.
 
PDF
The internals and the latest trends of container runtimes
Akihiro Suda
 
FreeSWITCH on Docker
建澄 吳
 
Scaling Asterisk with Kamailio
Fred Posner
 
Kamailio, FreeSWITCH, and You
Fred Posner
 
Hashicorp Vault Open Source vs Enterprise
Stenio Ferreira
 
Astricon 10 (October 2013) - SIP over WebSocket on Kamailio
Crocodile WebRTC SDK and Cloud Signalling Network
 
Continuous Integration and Kamailio
Giacomo Vacca
 
SIP transfer with Janus/WebRTC @ OpenSIPS 2022
Lorenzo Miniero
 
Asterisk WebRTC frontier: make client SIP Phone with sipML5 and Janus Gateway
Alessandro Polidori
 
Expanding Asterisk with Kamailio
Fred Posner
 
Introduction to Open Telemetry as Observability Library
Tonny Adhi Sabastian
 
Kamailio :: A Quick Introduction
Olle E Johansson
 
Introduction to Istio on Kubernetes
Jonh Wendell
 
RootedCON 2017 - Docker might not be your friend. Trojanizing Docker images
Daniel Garcia (a.k.a cr0hn)
 
Linux Linux Traffic Control
SUSE Labs Taipei
 
FreeSBC How To - Advanced SIP Routing
Alan Percy
 
Scaling FreeSWITCH Performance
Moises Silva
 
Linux networking
Armando Reis
 
Homer - Workshop at Kamailio World 2017
Giacomo Vacca
 
OPENMARU APM 브로셔
Opennaru, inc.
 
The internals and the latest trends of container runtimes
Akihiro Suda
 
Ad

Similar to Real-Time Text and WebRTC @ Kamailio World 2023 (20)

PPT
Transporting voice by using IP
Muhammad Jahangir
 
PDF
WebRTC and SIP not just audio and video @ OpenSIPS 2024
Lorenzo Miniero
 
PDF
JavaOne - A Sip Of Java - RJ Auburn
Voxeo Corp
 
PDF
Philippe Langlois - SCTPscan Finding entry points to SS7 Networks & Telecommu...
P1Security
 
PDF
GENBAND G2 Compact Gateway
GENBANDcorporate
 
PDF
Real time text communication - making it real ITAG 2011
AEGIS-ACCESSIBLE Projects
 
PPT
ie55104.ppt
WasimIqbal42
 
PPT
Telecom1.ppt
RohanBorgalli
 
PDF
Isup t-rec-q 931-199805-i!!pdf-e
Sidhartha Muraleedharan
 
PDF
Janus/Asterisk @ Astricon 2017
Lorenzo Miniero
 
PDF
EENA 2021: How to improve accessibility for people with disabilities? (1/3)
EENA (European Emergency Number Association)
 
PDF
VoLTE Flows and CS network
Karel Berkovec
 
PDF
lecture 1: introduction to wired and wireless communications
ssuserf58df5
 
PPTX
Cable TV Network.pptx
LankeshHJ1
 
PPT
Download
Videoguy
 
PPTX
Telecommunication basics
Yoohyun Kim
 
PDF
Rayo for XMPP Folks
Mojo Lingo
 
PDF
Matrix - One-year in, Matthew Hodgson, Matrix.org
Alan Quayle
 
PPT
How PSTN phone works?
mahipal9
 
Transporting voice by using IP
Muhammad Jahangir
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
Lorenzo Miniero
 
JavaOne - A Sip Of Java - RJ Auburn
Voxeo Corp
 
Philippe Langlois - SCTPscan Finding entry points to SS7 Networks & Telecommu...
P1Security
 
GENBAND G2 Compact Gateway
GENBANDcorporate
 
Real time text communication - making it real ITAG 2011
AEGIS-ACCESSIBLE Projects
 
ie55104.ppt
WasimIqbal42
 
Telecom1.ppt
RohanBorgalli
 
Isup t-rec-q 931-199805-i!!pdf-e
Sidhartha Muraleedharan
 
Janus/Asterisk @ Astricon 2017
Lorenzo Miniero
 
EENA 2021: How to improve accessibility for people with disabilities? (1/3)
EENA (European Emergency Number Association)
 
VoLTE Flows and CS network
Karel Berkovec
 
lecture 1: introduction to wired and wireless communications
ssuserf58df5
 
Cable TV Network.pptx
LankeshHJ1
 
Download
Videoguy
 
Telecommunication basics
Yoohyun Kim
 
Rayo for XMPP Folks
Mojo Lingo
 
Matrix - One-year in, Matthew Hodgson, Matrix.org
Alan Quayle
 
How PSTN phone works?
mahipal9
 
Ad

More from Lorenzo Miniero (20)

PDF
Multistream in SIP and NoSIP @ OpenSIPS Summit 2025
Lorenzo Miniero
 
PDF
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
Lorenzo Miniero
 
PDF
WebRTC and QUIC: how hard can it be? @ RTC.ON 2024
Lorenzo Miniero
 
PDF
SIP trunking in Janus @ Kamailio World 2024
Lorenzo Miniero
 
PDF
Getting AV1/SVC to work in the Janus WebRTC Server
Lorenzo Miniero
 
PDF
WebRTC Broadcasting @ TADSummit 2023
Lorenzo Miniero
 
PDF
BWE in Janus
Lorenzo Miniero
 
PDF
The challenges of hybrid meetings @ CommCon 2023
Lorenzo Miniero
 
PDF
Become a rockstar using FOSS!
Lorenzo Miniero
 
PDF
Janus SFU cascading @ IIT-RTC 2022
Lorenzo Miniero
 
PDF
WHIP WebRTC Broadcasting @ FOSDEM 2022
Lorenzo Miniero
 
PDF
WebRTC, RED and Janus @ ClueCon21
Lorenzo Miniero
 
PDF
WHIP and Janus @ IIT-RTC 2021
Lorenzo Miniero
 
PDF
Write a SocialTV app @ OpenSIPS 2021
Lorenzo Miniero
 
PDF
Janus + Audio @ Open Source World
Lorenzo Miniero
 
PDF
JamRTC @ Wonder WebRTC unConference
Lorenzo Miniero
 
PDF
Scaling WebRTC deployments with multicast @ IETF 110 MBONED
Lorenzo Miniero
 
PDF
Janus Workshop pt.2 @ ClueCon 2021
Lorenzo Miniero
 
PDF
Janus + NDI @ ClueCon 2021
Lorenzo Miniero
 
PDF
Can WebRTC help musicians? @ FOSDEM 2021
Lorenzo Miniero
 
Multistream in SIP and NoSIP @ OpenSIPS Summit 2025
Lorenzo Miniero
 
RTP Over QUIC: An Interesting Opportunity Or Wasted Time?
Lorenzo Miniero
 
WebRTC and QUIC: how hard can it be? @ RTC.ON 2024
Lorenzo Miniero
 
SIP trunking in Janus @ Kamailio World 2024
Lorenzo Miniero
 
Getting AV1/SVC to work in the Janus WebRTC Server
Lorenzo Miniero
 
WebRTC Broadcasting @ TADSummit 2023
Lorenzo Miniero
 
BWE in Janus
Lorenzo Miniero
 
The challenges of hybrid meetings @ CommCon 2023
Lorenzo Miniero
 
Become a rockstar using FOSS!
Lorenzo Miniero
 
Janus SFU cascading @ IIT-RTC 2022
Lorenzo Miniero
 
WHIP WebRTC Broadcasting @ FOSDEM 2022
Lorenzo Miniero
 
WebRTC, RED and Janus @ ClueCon21
Lorenzo Miniero
 
WHIP and Janus @ IIT-RTC 2021
Lorenzo Miniero
 
Write a SocialTV app @ OpenSIPS 2021
Lorenzo Miniero
 
Janus + Audio @ Open Source World
Lorenzo Miniero
 
JamRTC @ Wonder WebRTC unConference
Lorenzo Miniero
 
Scaling WebRTC deployments with multicast @ IETF 110 MBONED
Lorenzo Miniero
 
Janus Workshop pt.2 @ ClueCon 2021
Lorenzo Miniero
 
Janus + NDI @ ClueCon 2021
Lorenzo Miniero
 
Can WebRTC help musicians? @ FOSDEM 2021
Lorenzo Miniero
 

Recently uploaded (20)

PDF
SparkLabs Primer on Artificial Intelligence 2025
SparkLabs Group
 
PDF
Make GenAI investments go further with the Dell AI Factory
Principled Technologies
 
PDF
Economic Impact of Data Centres to the Malaysian Economy
flintglobalapac
 
PDF
Data_Analytics_vs_Data_Science_vs_BI_by_CA_Suvidha_Chaplot.pdf
CA Suvidha Chaplot
 
PPTX
Applied-Statistics-Mastering-Data-Driven-Decisions.pptx
parmaryashparmaryash
 
PPTX
Introduction to Flutter by Ayush Desai.pptx
ayushdesai204
 
PDF
AI-Cloud-Business-Management-Platforms-The-Key-to-Efficiency-Growth.pdf
Artjoker Software Development Company
 
PPTX
OA presentation.pptx OA presentation.pptx
pateldhruv002338
 
PDF
NewMind AI Weekly Chronicles - July'25 - Week IV
NewMind AI
 
PPTX
The Future of AI & Machine Learning.pptx
pritsen4700
 
PDF
Google I/O Extended 2025 Baku - all ppts
HusseinMalikMammadli
 
PPTX
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
PDF
Presentation about Hardware and Software in Computer
snehamodhawadiya
 
PDF
Trying to figure out MCP by actually building an app from scratch with open s...
Julien SIMON
 
PDF
CIFDAQ's Market Wrap : Bears Back in Control?
CIFDAQ
 
PPTX
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 
PDF
Automating ArcGIS Content Discovery with FME: A Real World Use Case
Safe Software
 
PPTX
IT Runs Better with ThousandEyes AI-driven Assurance
ThousandEyes
 
PDF
Get More from Fiori Automation - What’s New, What Works, and What’s Next.pdf
Precisely
 
PDF
Unlocking the Future- AI Agents Meet Oracle Database 23ai - AIOUG Yatra 2025.pdf
Sandesh Rao
 
SparkLabs Primer on Artificial Intelligence 2025
SparkLabs Group
 
Make GenAI investments go further with the Dell AI Factory
Principled Technologies
 
Economic Impact of Data Centres to the Malaysian Economy
flintglobalapac
 
Data_Analytics_vs_Data_Science_vs_BI_by_CA_Suvidha_Chaplot.pdf
CA Suvidha Chaplot
 
Applied-Statistics-Mastering-Data-Driven-Decisions.pptx
parmaryashparmaryash
 
Introduction to Flutter by Ayush Desai.pptx
ayushdesai204
 
AI-Cloud-Business-Management-Platforms-The-Key-to-Efficiency-Growth.pdf
Artjoker Software Development Company
 
OA presentation.pptx OA presentation.pptx
pateldhruv002338
 
NewMind AI Weekly Chronicles - July'25 - Week IV
NewMind AI
 
The Future of AI & Machine Learning.pptx
pritsen4700
 
Google I/O Extended 2025 Baku - all ppts
HusseinMalikMammadli
 
cloud computing vai.pptx for the project
vaibhavdobariyal79
 
Presentation about Hardware and Software in Computer
snehamodhawadiya
 
Trying to figure out MCP by actually building an app from scratch with open s...
Julien SIMON
 
CIFDAQ's Market Wrap : Bears Back in Control?
CIFDAQ
 
AI in Daily Life: How Artificial Intelligence Helps Us Every Day
vanshrpatil7
 
Automating ArcGIS Content Discovery with FME: A Real World Use Case
Safe Software
 
IT Runs Better with ThousandEyes AI-driven Assurance
ThousandEyes
 
Get More from Fiori Automation - What’s New, What Works, and What’s Next.pdf
Precisely
 
Unlocking the Future- AI Agents Meet Oracle Database 23ai - AIOUG Yatra 2025.pdf
Sandesh Rao
 

Real-Time Text and WebRTC @ Kamailio World 2023

  • 1. Bringing real-time text to WebRTC for NG Emergency Services Lorenzo Miniero @[email protected] Kamailio World June 7th 2023,
  • 2. Who am I? Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Main author of Janus Contacts and info • [email protected] • https://fosstodon.org/@lminiero • https://www.slideshare.net/LorenzoMiniero • https://lminiero.bandcamp.com
  • 3. Just a few words on Meetecho • Co-founded in 2009 as an academic spin-off • University research efforts brought to the market • Completely independent from the University • Focus on real-time multimedia applications • Strong perspective on standardization and open source • Several activities • Consulting services • Commercial support and Janus licenses • Streaming of live events (IETF, ACM, etc.) • Proudly brewed in sunny Napoli(*), Italy
  • 5. Emergency Services for the Hearing Impaired • Ad-hoc text telephones and telecommunication devices • Originally conceived and implemented in the 60’s • Commonly known as TDD / TTY / TT • “Telecommunications Device for the Deaf” • Units made of keyboard, small screen and modem • Messages converted to electric signals • Phone line used to deliver messages • Widely used in legacy Emergency Services • e.g., to allow 911 operators to engage with hard of hearing people • Uses phone line + batteries −→ still works in power failure
  • 6. Emergency Services for the Hearing Impaired • Ad-hoc text telephones and telecommunication devices • Originally conceived and implemented in the 60’s • Commonly known as TDD / TTY / TT • “Telecommunications Device for the Deaf” • Units made of keyboard, small screen and modem • Messages converted to electric signals • Phone line used to deliver messages • Widely used in legacy Emergency Services • e.g., to allow 911 operators to engage with hard of hearing people • Uses phone line + batteries −→ still works in power failure
  • 7. Emergency Services for the Hearing Impaired • Ad-hoc text telephones and telecommunication devices • Originally conceived and implemented in the 60’s • Commonly known as TDD / TTY / TT • “Telecommunications Device for the Deaf” • Units made of keyboard, small screen and modem • Messages converted to electric signals • Phone line used to deliver messages • Widely used in legacy Emergency Services • e.g., to allow 911 operators to engage with hard of hearing people • Uses phone line + batteries −→ still works in power failure
  • 9. TDD/TTY/TT (PSTN) • Different acronyms for mostly similar things • TDD (Telecommunications Device for the Deaf) • TTY (TeleTYpe) • TT (Text Telephone) • Some important limitations • Half-duplex (GA = GO AHEAD) • Can’t be interrupted (garbled output) • Limited set of “characters” (e.g., Baudot) • No specific handshake, and no error correction • Requires specific device
  • 10. TDD/TTY/TT (PSTN) • Different acronyms for mostly similar things • TDD (Telecommunications Device for the Deaf) • TTY (TeleTYpe) • TT (Text Telephone) • Some important limitations • Half-duplex (GA = GO AHEAD) • Can’t be interrupted (garbled output) • Limited set of “characters” (e.g., Baudot) • No specific handshake, and no error correction • Requires specific device
  • 11. Next Generation Emergency Services • Multiple efforts in different areas • NG9-1-1 (https://www.911.gov/issues/ng911/) • NG112 (https://eena.org/next-generation-112/) • Attempt to switch to modern ways of communication • Existing NG services based on PSTN • Internet-based communication allows for richer media, though • Location, , photos, videos, etc.
  • 12. Next Generation Emergency Services • Multiple efforts in different areas • NG9-1-1 (https://www.911.gov/issues/ng911/) • NG112 (https://eena.org/next-generation-112/) • Attempt to switch to modern ways of communication • Existing NG services based on PSTN • Internet-based communication allows for richer media, though • Location, , photos, videos, etc.
  • 13. Next Generation Emergency Services • Multiple efforts in different areas • NG9-1-1 (https://www.911.gov/issues/ng911/) • NG112 (https://eena.org/next-generation-112/) • Attempt to switch to modern ways of communication • Existing NG services based on PSTN • Internet-based communication allows for richer media, though • Location, real-time text, photos, videos, etc.
  • 14. Next Generation Emergency Services • Multiple efforts in different areas • NG9-1-1 (https://www.911.gov/issues/ng911/) • NG112 (https://eena.org/next-generation-112/) • Attempt to switch to modern ways of communication • Existing NG services based on PSTN • Internet-based communication allows for richer media, though • Location, real-time text, photos, videos, etc.
  • 15. Real-Time Text (RTT) • Text transmitted instantly, as it is typed or created • TDD/TTY work that way too • How to do it over IP, though? • Text over IP (ToIP) • ITU-T T.140 (Protocol for multimedia application text conversation) • Allows for real-time editing (e.g., backspace, rewriting) • T.140 messages transported over RTP • RFC 4103 (RTP Payload for Text Conversation) • Redundancy implemented via RED (RFC 2198)
  • 16. Real-Time Text (RTT) • Text transmitted instantly, as it is typed or created • TDD/TTY work that way too • How to do it over IP, though? • Text over IP (ToIP) • ITU-T T.140 (Protocol for multimedia application text conversation) • Allows for real-time editing (e.g., backspace, rewriting) • T.140 messages transported over RTP • RFC 4103 (RTP Payload for Text Conversation) • Redundancy implemented via RED (RFC 2198)
  • 17. Real-Time Text (RTT) • Text transmitted instantly, as it is typed or created • TDD/TTY work that way too • How to do it over IP, though? • Text over IP (ToIP) • ITU-T T.140 (Protocol for multimedia application text conversation) • Allows for real-time editing (e.g., backspace, rewriting) • T.140 messages transported over RTP • RFC 4103 (RTP Payload for Text Conversation) • Redundancy implemented via RED (RFC 2198)
  • 18. T.140 • ITU-T T.140 (Protocol for multimedia application text conversation) • https://www.itu.int/rec/T-REC-T.140/ • Protocol based on the concept of T140blocks • Set of characters or special codes that one party is delivering to one or more others • Participant typing −→ typed characters bundled together in a T.140 block • Buffering size up to sender (overhead vs. latency) • UTF-8 used for characters, including special codes (actions) • e.g., Byte Order Mark (BOM), backspace, etc.
  • 19. T.140 • ITU-T T.140 (Protocol for multimedia application text conversation) • https://www.itu.int/rec/T-REC-T.140/ • Protocol based on the concept of T140blocks • Set of characters or special codes that one party is delivering to one or more others • Participant typing −→ typed characters bundled together in a T.140 block • Buffering size up to sender (overhead vs. latency) • UTF-8 used for characters, including special codes (actions) • e.g., Byte Order Mark (BOM), backspace, etc.
  • 20. T.140 • ITU-T T.140 (Protocol for multimedia application text conversation) • https://www.itu.int/rec/T-REC-T.140/ • Protocol based on the concept of T140blocks • Set of characters or special codes that one party is delivering to one or more others • Participant typing −→ typed characters bundled together in a T.140 block • Buffering size up to sender (overhead vs. latency) • UTF-8 used for characters, including special codes (actions) • e.g., Byte Order Mark (BOM), backspace, etc.
  • 21. T.140 over RTP 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC=0 |M| T140 PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp (1000Hz) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | T.140 encoded data | + +---------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  • 22. Dealing with packet loss using RED
  • 23. Dealing with packet loss using RED
  • 24. Dealing with packet loss using RED
  • 25. Dealing with packet loss using RED
  • 26. Dealing with packet loss using RED
  • 27. T.140 over RTP (with redundancy) 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC=0 |M| "RED" PT | sequence number of primary | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp of primary encoding "P" | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| T140 PT | timestamp offset of "R" | "R" block length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| T140 PT | "R" T.140 encoded redundant data | +-+-+-+-+-+-+-+-+ +---------------+ + | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+ | "P" T.140 encoded primary data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  • 28. RTT and SIP/SDP • As an RTP-based stream, easy to negotiate via SDP • Needs text as media in m-line • t140 used to advertise codec in rtpmap • Sampling rate always 1000 (also for RED) SDP example m=text 11000 RTP/AVP 98 100 a=rtpmap:98 t140/1000 a=rtpmap:100 red/1000 a=fmtp:100 98/98/98
  • 29. RTT and SIP/SDP • As an RTP-based stream, easy to negotiate via SDP • Needs text as media in m-line • t140 used to advertise codec in rtpmap • Sampling rate always 1000 (also for RED) SDP example m=text 11000 RTP/AVP 98 100 a=rtpmap:98 t140/1000 a=rtpmap:100 red/1000 a=fmtp:100 98/98/98
  • 30. RTT and SIP/SDP • As an RTP-based stream, easy to negotiate via SDP • Needs text as media in m-line • t140 used to advertise codec in rtpmap • Sampling rate always 1000 (also for RED) SDP example m=text 11000 RTP/AVP 98 100 a=rtpmap:98 t140/1000 a=rtpmap:100 red/1000 a=fmtp:100 98/98/98
  • 31. RTT and SIP/SDP • As an RTP-based stream, easy to negotiate via SDP • Needs text as media in m-line • t140 used to advertise codec in rtpmap • Sampling rate always 1000 (also for RED) SDP example m=text 11000 RTP/AVP 98 100 a=rtpmap:98 t140/1000 a=rtpmap:100 red/1000 a=fmtp:100 98/98/98
  • 32. RTT and SIP/SDP • As an RTP-based stream, easy to negotiate via SDP • Needs text as media in m-line • t140 used to advertise codec in rtpmap • Sampling rate always 1000 (also for RED) SDP example m=text 11000 RTP/AVP 98 100 a=rtpmap:98 t140/1000 a=rtpmap:100 red/1000 a=fmtp:100 98/98/98
  • 33. RTT and SIP/SDP • As an RTP-based stream, easy to negotiate via SDP • Needs text as media in m-line • t140 used to advertise codec in rtpmap • Sampling rate always 1000 (also for RED) SDP example m=text 11000 RTP/AVP 98 100 a=rtpmap:98 t140/1000 a=rtpmap:100 red/1000 a=fmtp:100 98/98/98
  • 34. RTT and SIP/SDP • As an RTP-based stream, easy to negotiate via SDP • Needs text as media in m-line • t140 used to advertise codec in rtpmap • Sampling rate always 1000 (also for RED) SDP example m=text 11000 RTP/AVP 98 100 a=rtpmap:98 t140/1000 a=rtpmap:100 red/1000 a=fmtp:100 98/98/98
  • 35. What about WebRTC? • WebRTC heavily based on existing VoIP technologies • SDP (on “steroids”) is mandatory • RTP used for media (encrypted via DTLS-SRTP) • That said, not all RTP media is supported • Only m=audio and m=video supported out of the box • No support at all for m=text, so RTT won’t work • Only m=application media that can be used is UDP/DTLS/SCTP • WebRTC data channels • Generic data sent via SCTP and encapsulated in DTLS
  • 36. What about WebRTC? • WebRTC heavily based on existing VoIP technologies • SDP (on “steroids”) is mandatory • RTP used for media (encrypted via DTLS-SRTP) • That said, not all RTP media is supported • Only m=audio and m=video supported out of the box • No support at all for m=text, so RTT won’t work • Only m=application media that can be used is UDP/DTLS/SCTP • WebRTC data channels • Generic data sent via SCTP and encapsulated in DTLS
  • 37. What about WebRTC? • WebRTC heavily based on existing VoIP technologies • SDP (on “steroids”) is mandatory • RTP used for media (encrypted via DTLS-SRTP) • That said, not all RTP media is supported • Only m=audio and m=video supported out of the box • No support at all for m=text, so RTT won’t work • Only m=application media that can be used is UDP/DTLS/SCTP • WebRTC data channels • Generic data sent via SCTP and encapsulated in DTLS
  • 38. T.140 RTT over WebRTC Data Channels • Why not use data channels for RTT, then? • T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865) • https://www.rfc-editor.org/rfc/rfc8865.html • Sending T.140 frames directly in data channels, not on RTP • Data channels can be configured to be ordered and reliable • Helps avoiding hoops needed for RTP (e.g., RED) • A specific label can be associated with a specific RTT session • Of course, needs gatewaying to be backwards compatible • WebRTC endpoints can’t do RTP/RTT, and legacy ones don’t do data channels • Something needs to “translate” (media and SDP)
  • 39. T.140 RTT over WebRTC Data Channels • Why not use data channels for RTT, then? • T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865) • https://www.rfc-editor.org/rfc/rfc8865.html • Sending T.140 frames directly in data channels, not on RTP • Data channels can be configured to be ordered and reliable • Helps avoiding hoops needed for RTP (e.g., RED) • A specific label can be associated with a specific RTT session • Of course, needs gatewaying to be backwards compatible • WebRTC endpoints can’t do RTP/RTT, and legacy ones don’t do data channels • Something needs to “translate” (media and SDP)
  • 40. T.140 RTT over WebRTC Data Channels • Why not use data channels for RTT, then? • T.140 Real-Time Text Conversation over WebRTC Data Channels (RFC 8865) • https://www.rfc-editor.org/rfc/rfc8865.html • Sending T.140 frames directly in data channels, not on RTP • Data channels can be configured to be ordered and reliable • Helps avoiding hoops needed for RTP (e.g., RED) • A specific label can be associated with a specific RTT session • Of course, needs gatewaying to be backwards compatible • WebRTC endpoints can’t do RTP/RTT, and legacy ones don’t do data channels • Something needs to “translate” (media and SDP)
  • 41. T.140/data channels SDP usage m=application 911 UDP/DTLS/SCTP webrtc-datachannel c=IN IP6 2001:db8::3 a=max-message-size:1000 a=sctp-port 5000 a=setup:actpass a=dcmap:2 label="ACME customer service";subprotocol="t140" a=dcsa:2 fmtp:t140 cps=20 a=dcsa:2 hlang-send:es eo a=dcsa:2 hlang-recv:es eo
  • 42. Prototyping T.140/WebRTC in Janus • Created a prototype in Janus a few years ago • Old PR: https://github.com/meetecho/janus-gateway/pull/1898 (closed) • Blog post: https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/ • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • Used the SIP plugin in Janus as the necessary gateway • Negotiates text m-lines and the text/t140 format on the SDP side • Negotiates data channels on the WebRTC side (binary) • T140blocks are then bridged (no translation) as they are • T140blocks decapsulated from RTP packets −→ data channels • Data channels −→ crafting RTP packets with T140blocks • RED optionally supported in both directions on the RTP side
  • 43. Prototyping T.140/WebRTC in Janus • Created a prototype in Janus a few years ago • Old PR: https://github.com/meetecho/janus-gateway/pull/1898 (closed) • Blog post: https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/ • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • Used the SIP plugin in Janus as the necessary gateway • Negotiates text m-lines and the text/t140 format on the SDP side • Negotiates data channels on the WebRTC side (binary) • T140blocks are then bridged (no translation) as they are • T140blocks decapsulated from RTP packets −→ data channels • Data channels −→ crafting RTP packets with T140blocks • RED optionally supported in both directions on the RTP side
  • 44. Prototyping T.140/WebRTC in Janus • Created a prototype in Janus a few years ago • Old PR: https://github.com/meetecho/janus-gateway/pull/1898 (closed) • Blog post: https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/ • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • Used the SIP plugin in Janus as the necessary gateway • Negotiates text m-lines and the text/t140 format on the SDP side • Negotiates data channels on the WebRTC side (binary) • T140blocks are then bridged (no translation) as they are • T140blocks decapsulated from RTP packets −→ data channels • Data channels −→ crafting RTP packets with T140blocks • RED optionally supported in both directions on the RTP side
  • 45. SIP plugin in Janus https://janus.conf.meetecho.com/docs/sip
  • 46. SIP plugin in Janus + RTT
  • 47. Example: SDP offer from SIP/RTT endpoint v=0 o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74 s=Omnitor_SDP_v1.1 c=IN IP4 192.168.1.74 t=0 0 m=text 1024 RTP/AVP 99 98 a=rtpmap:99 red/1000 a=fmtp:99 98/98/98 a=rtpmap:98 t140/1000
  • 48. Example: SDP offer to WebRTC endpoint v=0 o=Lorenzo_Miniero 1 1 IN IP4 192.168.1.74 s=Omnitor_SDP_v1.1 t=0 0 a=group:BUNDLE data a=msid-semantic: WMS janus m=application 9 UDP/DTLS/SCTP webrtc-datachannel c=IN IP4 192.168.1.74 a=sendrecv a=sctp-port:5000 a=mid:0 [.. ICE/DTLS details follow..]
  • 50. Testing: Janus SIP demo (WebRTC) https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
  • 51. T.140 conversation on the wire https://www.meetecho.com/blog/realtime-text-sip-and-webrtc/
  • 52. What’s next? • First of all, revive the effort • The original PR was closed a few months ago • Little to no feedback, and core had changed a lot in the meanwhile • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • New revision can take advantage of some new features • e.g., RED supported in the Janus core, now • Maybe write a prettier and more functional UI too... • Possibly test with something more complex than TIPcon1 • TIPcon1 good for basic functional testing, but very limited • RTT endpoints very hard to find, especially in FOSS • Possibly integration tests with actual RTT deployments?
  • 53. What’s next? • First of all, revive the effort • The original PR was closed a few months ago • Little to no feedback, and core had changed a lot in the meanwhile • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • New revision can take advantage of some new features • e.g., RED supported in the Janus core, now • Maybe write a prettier and more functional UI too... • Possibly test with something more complex than TIPcon1 • TIPcon1 good for basic functional testing, but very limited • RTT endpoints very hard to find, especially in FOSS • Possibly integration tests with actual RTT deployments?
  • 54. What’s next? • First of all, revive the effort • The original PR was closed a few months ago • Little to no feedback, and core had changed a lot in the meanwhile • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • New revision can take advantage of some new features • e.g., RED supported in the Janus core, now • Maybe write a prettier and more functional UI too... • Possibly test with something more complex than TIPcon1 • TIPcon1 good for basic functional testing, but very limited • RTT endpoints very hard to find, especially in FOSS • Possibly integration tests with actual RTT deployments?
  • 55. What’s next? • First of all, revive the effort • The original PR was closed a few months ago • Little to no feedback, and core had changed a lot in the meanwhile • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • New revision can take advantage of some new features • e.g., RED supported in the Janus core, now • Maybe write a prettier and more functional UI too... • Possibly test with something more complex than TIPcon1 • TIPcon1 good for basic functional testing, but very limited • RTT endpoints very hard to find, especially in FOSS • Possibly integration tests with actual RTT deployments?
  • 56. What’s next? • First of all, revive the effort • The original PR was closed a few months ago • Little to no feedback, and core had changed a lot in the meanwhile • New PR: https://github.com/meetecho/janus-gateway/pull/3231 • New revision can take advantage of some new features • e.g., RED supported in the Janus core, now • Maybe write a prettier and more functional UI too... • Possibly test with something more complex than TIPcon1 • TIPcon1 good for basic functional testing, but very limited • RTT endpoints very hard to find, especially in FOSS • Possibly integration tests with actual RTT deployments?
  • 57. Thanks! Questions? Comments? Get in touch! • https://fosstodon.org/@lminiero • https://twitter.com/elminiero • https://twitter.com/meetecho • https://www.meetecho.com/blog/