gzip notify/listen payloads

[Pedr0Rocha]

Gzip the postgres NOTIFY/LISTEN payloads. This is mostly about the 8kb limit that we hit sometimes depending on the network you are connected to, but other payloads can also benefit from it.

[Pedr0Rocha]

All tests passed but I'm not sure if they cover the postgres notify/listen system completely. I can expand the steps to try to check those payloads if necessary.

[erikdubbelboer]

The postgres notify/listen system is tested in the feature tests. They spin up a database and actually do webrtc handshakes etc.

[Copilot]

Pull request overview

This PR adds gzip compression to Postgres NOTIFY/LISTEN payloads to address the 8KB size limit that can be hit on certain networks. The implementation introduces compression at the publish side and decompression at the listen side.

Key Changes:

• New gzip utility functions for compressing and decompressing byte arrays
• Modified Postgres NOTIFY/LISTEN to compress payloads before base64 encoding and decompress after decoding
• Removed debug logging code for a specific Poki topic

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
internal/util/gzip.go	Adds new utility functions GzipCompress and GzipDecompress using gzip.BestSpeed compression level
internal/signaling/stores/postgres.go	Integrates gzip compression/decompression into the publish and listen flow, removes debug logging for oversized payloads

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[koenbollen]

From https://www.postgresql.org/docs/current/sql-notify.html

(If binary data or large amounts of information need to be communicated, it's best to put it in a database table and send the key of the record.)

Maybe this is the time to add a payloads table (key int, data bytea). You can in one transaction (and roundtrip) store the data in a table and NOTIFY only the key of the payloads table and then after the LISTEN you query the payload (maybe even that can be done in one call/roundtrip with a function or something).

[erikdubbelboer]

A payloads table sounds a bit overkill for now. This is just for a couple of times when the webrtc descriptor object is larger than 8kb. gzipping with bestspeed is super fast and probably makes the whole thing faster in the end.

[koenbollen]

Fair.
It's just awkward to have to base64 it again to play with NOTIFY's api, you loose 33% of your compression again. 😅

[erikdubbelboer]

We already base64 any payload at the moment (here). I guess that's one of the issues why that webrtc descriptor message gets too big. So doing a quick gzip before the base64 is not a bad idea.

[koenbollen]

We already base64 any payload at the moment (here). I guess that's one of the issues why that webrtc descriptor message gets too big. So doing a quick gzip before the base64 is not a bad idea.

I know we already did that. And gzip is a good idea.
I didn't realize it was just the rtc descriptors. Now I wonder why they grew in size, if if they will grow in the future.
Anyway, this is a good improvement. My earlier idea of a payload table was just a idea incase the payloads keep growing.

[erikdubbelboer]

The webrtc descriptor can apparently be very big in certain network conditions. Poki was trying this during an event in a public space and ran into this issue. Judging from the logs it happens once in a while, not often.