Support additional OSC 52 clipboard types

[mgulick]

Fixes #967.
Fixes #1090.
Fixes #637.

NOTE: This pull request is similar to #1054, but improves it by preserving the selection buffer into which to place the copied text. #1054 always places the copied text into CLIPBOARD, but this preserves the OSC 52 parameter so that the copied text may go into CLIPBOARD, PRIMARY, SECONDARY, or one of the cut buffers. This matches the behavior of ssh and plain XTerm.

The OSC 52 escape sequence supports specifying which X selection buffer to place the selection into. The protocol format is:

\033]52;C;D\007

The C parameter determines the selection buffer. It consists of zero or more of the following characters:

c: CLIPBOARD
p: PRIMARY
q: SECONDARY
s: XTerm-selected (based on XTerm.vt100.selectToClipboard)
0-7: Numbered cut buffer

The default if left blank is 's0', representing the configurable primary/clipboard selection and cut buffer 0. [1]

D is the base64 encoded text to place in the selection buffer.

This patch modifies the transferred clipboard data to include both the selection parameter and the base64 text. I.e. previously the
transferred clipboard data only contained 'D', and now it contains 'C;D'.

To test this functionality:

1. Open XTerm
2. Ctrl-Right Click and select 'Allow Window Ops'
3. Connect to a server w/ mosh
4. Run the following in the remote connection:
   $ printf "\033]52;c;$(echo test1234 | base64)\007"
$ printf "\033]52;p;$(echo test2345 | base64)\007"
$ printf "\033]52;q;$(echo test3456 | base64)\007"
5. Open another terminal on the local machine and run:

   $ xclip -o -selection clipboard
   test1234
   $ xclip -o -selection primary
   test2345
   $ xclip -o -selection secondary
   test3456
   

6. You can also try:
   $ printf "\033]52;;$(echo testdefault | base64)\007"
   (This should update either the clipboard or primary selection based on the Xterm settings)
7. To test the cut buffers you can use the same procedure, substituting the cut buffer number in C, and then use xcutsel to transfer the data from the cut buffer to the PRIMARY selection, where it can then be viewed with xclip.

Note, I observed that XTerm does not currently (as of XTerm patch 358) support specifying more than one value in C. The specification does support it, and this appears to just be a simple bug in XTerm (a patch has been submitted to the maintainer to fix it).

[1] https://github.com/ThomasDickey/xterm-snapshots/blob/master/ctlseqs.txt

This PR also adds a clipboard sequence number to allow duplicate copied text to update the client's clipboard.

[mgulick]

Added a new commit which fixes issue #1090. I believe this PR now addresses all outstanding issues in #637.

[3gx]

will it be merged? this MR works fine on my Linux server & Mac client.

[gwicke]

Seems to work okay with tmux copy mode. If you are migrating from #1054, then make sure to rebuild both client & server from this PR, since the wire format changed.

[mgulick]

Seems to work okay with tmux copy mode. If you are migrating from #1054, then make sure to rebuild both client & server from this PR, since the wire format changed.

That is true about the wire format. I believe it is incompatible with the current wire format used by the master branch. I don't quite grok the wire format used by mosh and haven't been able to figure out where in the code fields used by the framebuffer get turned into bits sent on the wire. Since there hasn't been an official mosh release since clipboard support was added, I don't believe it causes any compatibility issues with previous releases. If it does introduce incompatibility, then I believe the original commit adding clipboard support would also have introduced the same sort of incompatibility issues.

If this wire format compatibility is a concern, it might be possible to make this backwards compatible, but I might need some help understanding the wire format a little more.

[JonathanWheeler]

Confirming this fixes all remaining clipboard issues for me on a local client running macOS Catalina v10.15.6 + iTerm2 v3.3.12 + mosh v1.3.2 [build mosh-1.3.2-92-ge922128 using: git clone https://github.com/mobile-shell/mosh && cd mosh && gh pr checkout 1104 && ./autogen.sh && ./configure && make && sudo make install] to a remote Ubuntu 20.04.1 LTS server running mosh v1.3.2 [ditto as before] + tmux v3.2. Works with both copy-mode-vi key bindings and simple mouse selection. It even works with remote (and local) vim sessions when combined with @sunaku's post at https://sunaku.github.io/tmux-yank-osc52.html.

It's taken over ten years to get to this point; the holy grail has finally been achieved!!! Well done, @mgulick!!

[LubosRemplik]

+1 to merge

[arg]

I hope I'm wrong but this project looks dead. Most recent release: ~4 years ago. Most recent commit to master: 1 year ago. No maintainers activity for several months. Would be great if @keithw and @cgull could check this indeed useful PR and make it merged (not even mentioning the new release).

[xPMo]

I hope I'm wrong but this project looks dead. Most recent release: ~4 years ago. Most recent commit to master: 1 year ago. No maintainers activity for several months.

I saw from other issues that there were a couple of blockers to a new release. Creating a pinned issue and a milestone for the next release would help communicate this. (This is offtopic to this PR though)

[catskul]

@eminence any chance you might have some time to get eyes on this?

Also, it seems like you might be the only active maintainer right now, any chance of getting another to help chew through the PR and issue backlog? Perhaps @agriffis or @njaard ?

[scjones-attunely]

@eminence another ping - would be great to have this/ and sounds like you have offers for more help to support mosh.

[ezzieyguywuf]

Can anyone familiar with this PR see if 4b240ac resolves the same issues noted in the first message?

[mgulick]

Can anyone familiar with this PR see if 4b240ac resolves the same issues noted in the first message?

No, this PR builds on that commit to add some additional functionality that is missing.

[delbao]

This almost works. But I cannot make it work in one scenario

What worked
in mosh + tmux

esc="\033]52;c;$(printf %s "blabla" | base64)\a"
esc="\033Ptmux;\033$esc\033\\" # again for tmux!
printf "$esc"

successfully remote copy to local clipboard.

But if I put this in ~/.tmux.conf.

bind-key -T copy-mode-vi Y send-keys -X copy-pipe '~/bin/yank > #{pane_tty}'

(~/bin/yank is copied from this well-known script: https://sunaku.github.io/tmux-yank-osc52.html). However, in a ssh session, this works without any issue.

I tweaked it further.

bind-key -T copy-mode-vi Y send-keys -X copy-pipe '~/bin/yank > ~/test.out'

then in another window, if I cat ~/test.out. Surprisingly, the copied text was sent to local clipboard.

I dont know what the root cause here. but hunch is related to tmux's pane_tty.

[sunaku]

Hi @delbao, what happens if you change the yank > #{pane_tty} redirection to yank > $SSH_TTY instead? 🤔 And which version of tmux are you using?

[delbao]

Hi @delbao, what happens if you change the yank > #{pane_tty} redirection to yank > $SSH_TTY instead? 🤔 And which version of tmux are you using?

Tried SSH_TTY, nothing changed. This issue is specific to this patch of mosh. SSH works fine.

$ tmux -V
tmux 3.2a

[andrewmzhang]

@delbao
This patch works fine on my machine for your use case: mosh + tmux + tmux-copy-paste.

Here is a copy of my ~/.tmux.conf

# sets tmux to use OSC 52 escape
set -g set-clipboard on
setw -g mode-keys vi
bind-key -T copy-mode-vi v send-keys -X begin-selection
bind-key -T copy-mode-vi y send-keys -X copy-selection

# OSC 52 haxor. I don't think you need this line, but try it if the above doesn't work.
# set -ag terminal-overrides "vte*:XT:Ms=\\E]52;c;%p2%s\\7,xterm*:XT:Ms=\\E]52;c;%p2%s\\7"

(disclaimer: the following info may be out of date) Older(?) tmux needs the last line because:

Tmux supports OSC 52 but does not pass the “c;” option, according to yudai’s post. However we can force it to pass the “c;” option.

[delbao]

@delbao This patch works fine on my machine for your use case: mosh + tmux + tmux-copy-paste.

Here is a copy of my ~/.tmux.conf

# sets tmux to use OSC 52 escape
set -g set-clipboard on
setw -g mode-keys vi
bind-key -T copy-mode-vi v send-keys -X begin-selection
bind-key -T copy-mode-vi y send-keys -X copy-selection

# OSC 52 haxor. I don't think you need this line, but try it if the above doesn't work.
# set -ag terminal-overrides "vte*:XT:Ms=\\E]52;c;%p2%s\\7,xterm*:XT:Ms=\\E]52;c;%p2%s\\7"

(disclaimer: the following info may be out of date) Older(?) tmux needs the last line because:

Tmux supports OSC 52 but does not pass the “c;” option, according to yudai’s post. However we can force it to pass the “c;” option.

thanks for the help. However, sunaku's script already did what you described. it doesn't work for Mosh with this patch.

[xero]

bumping this thread b/c this pr saved me hours of time. working flawlessly on a debian vps connected from my ipad using blink shell in zsh, tmux, and neovim using osc52 tooling

[kkonevets]

I have applied the patch but when copying to clipboard in emacs I get this error

clipetty-cut: Opening output file: Permission denied, /dev/pts/1

But it can be fixed if I start emacs like so

unset SSH_TTY; emacs

Anyway, thanks for this PR, it works now but not out of the box.

[markx]

Scrolled through this thread but failed to figure out: Is there any good reason why this was not merged?

If this repo is abandoned, is there another more actively-maintained fork?

[achernya]

Hi,

This PR hasn't been merged for two reasons:

1. The maintainers haven't gotten to it yet. Merging a PR isn't as sttaight-forward as clicking the merge button; we have to ensure that the code does what is intended and the PR meets our standards for inclusion and do a complete code review. Like most folks here, the maintainers don't work on mosh full-time, but rather as volunteers in their spare time.
2. There are merge conflicts, which increase the workload of (1), above.

Just because there are PRs open doesn't mean the repository is abandoned.

[markx]

I absolutely respect maintainers' right to handle any PRs in any way. Glad to know there's no technical reason against this PR so far.

OSC 52 is not working for me (and many others I believe), and this is the only blocker for me to use mosh. I hope maintainers consider prioritizing this PR. After all it's a relatively small code change for such an important feature.

To be fair, the second reason doesn't stand to me - the merge conflicts are more of a consequence, not reason, of that this PR hasn't been got to for 3+ years.

[mgulick]

I just rebased these patches onto master. There were no "real" conflicts, just some indentation changes. I built these changes again and did a quick test to confirm they still appear to be working. I've been using these patches in my daily workflow for a few years, albeit on an older mosh version based on the original PR patch series.

The XTerm bug mentioned at the end of the PR description has been fixed as of xterm version 359.

I did a quick test using this patched mosh and an unpatched server. While clipboard syncing does not work (to be expected), mosh itself appears to function normally, so maybe there are no backwards-compatibility issues with this change.

[joaotavora]

I'd like to try out this PR, but I cannot easily compile the client. I can easily compile the server, though. Is that enough?

[mgulick]

@joaotavora both the client and server need this change, however you don't need to install them in the usual locations. As long as you have the mosh-client binary with this patch somewhere on the client, and the mosh-server binary with this patch somewhere on the server, you can connect using a command like:

mosh --client=/home/client/mosh-client --server=/home/server/mosh-server server

This does not require updating the mosh command on the client. The mosh command is just a perl script that coordinates setting up the connection, and isn't affected by this patch. In my case, I'm successful even with an old version of mosh, such as mosh 1.3.2 on Debian 11.

[joaotavora]

Thanks @mgulick. Though somehow, compiling just the server worked. at least for very limited testing. I was able to copy to the clipboard from Emacs and paste in my host's browser. I can see the client binary linking in the terminal subsystem though, but I don't know enough of the architecture to understand why that is or why my test worked.

[mgulick]

@joaotavora I think on recent mosh releases, standard clipboard copying will work without this patch. I'm able to do the standard clipboard updates going from a 1.4.0 client to a 1.4.0 server without this patch. However the other issues, such as primary, secondary, and cutbuffer support, as well as the issue with duplicate clipboard updates not getting properly synchronized (#1090), will still exist.

[joaotavora]

Thanks, could have sworn it didn't work before I compiled and installed your version though, maybe I misdiagnosed 👍

[xero]

so i've been using the older version of this patch for a long time with no issues beyond when copying higher level unicode and emojis. strings like this would not be correctly copied to the clipboard instead replaced with a placeholder for each character.

гєŦ๏г๓คՇ Շђє קɭคภєՇ 💾

[markx]

@ezzieyguywuf If you have permission to merge, PTAL

[AkechiShiro]

@achernya any news on this PR merge possibility ??

[3ynm]

@mgulick Clipboard does not work for me when using mosh 1.4.0 on both client and server. It works with SSH.

Perhaps something relevant about this, is that I'm using it on a fairly old CentOS 7 server.

[mgulick]

It doesn't look like mosh has much activity in terms of commits these days, but I'm still interested in getting this PR merged if there is an opportunity to do so. It would be great to be able to eventually have this upstreamed and in distros so that I can stop building a custom mosh on every system that I access.

[isker]

FWIW I have run @jdrouhard's fork https://github.com/jdrouhard/mosh/tree/patched containing this PR and some others for over a year. Thanks to everyone that has produced these fixes. Community in action 🌞.

[rnk]

It doesn't look like mosh has much activity in terms of commits these days, but I'm still interested in getting this PR merged if there is an opportunity to do so.

There is recent activity from maintainers, so I hope they can take another look at this. They aren't listed as assigned reviewers, so I'm not sure they get pings. Speaking for myself, when acting as a maintainer, it's easy for these much-requested PRs to fly under the radar, so pinging @achernya @vapier @eminence

I applied the predecessor PR locally and confirmed it made things work and I was preparing to rebase and submit a new PR, but I hadn't dug up any actual material on how the OSC protocol is supposed to work to really understand why these changes (ignoring 'c' options and skipping to the next ';') are correct.

[rnk]

I hadn't dug up any actual material on how the OSC protocol is supposed to work to really understand why these changes (ignoring 'c' options and skipping to the next ';') are correct.

I've actually reviewed the PR code, pull request message, referenced the original archived xterm plain text docs (helpfully linked in the PR) , and the ghostty docs on OSC 52, which are more readable and modern.

I think there's no wire protocol backwards compatibility concern. I tried to dig into how the clipboard gets serialized, but and it looks like the OSC string goes directly over the wire, so if we have an old client with a non 'c' clipboard identifier, it will just get dropped, but that's what would happen anyway. At least, an LLM agrees with this analysis, but they tend to use motivated reasoning.

Regarding testing, I used an LLM to add tests using the existing testing conventions and pushed them in this commit. I confirm the tests pass after and fail before these source changes as expected, and the error messages are useful. If that's a blocking concern, feel free to merge those changes into this PR.

[mgulick]

If there is renewed interest from a maintainer on getting this merged, I'm happy to help with whatever is necessary as well.

[vapier]

i'm not a maintainer, just a rando who got lucky and had some fixes merged

[mpasternak]

@mgulick thanks for the patch, looks like I could use it too. Does it work with tmux?

[ezzieyguywuf]

I'm also happy to help! (also not a maintainer I think)

[mpasternak]

Thanks for this, @mgulick and @ezzieyguywuf — preserving the OSC 52 selection parameter is exactly the behaviour I was after. This works perfectly with tmux. I love it🙏

For anyone who'd like to try the PR before it's merged, here's how to build a working mosh from this branch. The OSC 52 changes live in both mosh-client and mosh-server, so you'll want this build on both ends of your connection.

1. Install build dependencies

macOS (Homebrew):

brew install autoconf automake pkgconf protobuf

(On macOS mosh uses the system CommonCrypto and ncurses, so OpenSSL isn't needed.)

Debian / Ubuntu:

sudo apt-get install autoconf automake pkg-config protobuf-compiler \
  libprotobuf-dev libutempter-dev zlib1g-dev libncurses-dev libssl-dev

2. Get the PR and build it

git clone https://github.com/mobile-shell/mosh.git
cd mosh
git fetch origin pull/1104/head:osc52 && git checkout osc52
./autogen.sh
./configure
make

This produces src/frontend/mosh-client and src/frontend/mosh-server. To put them on your PATH:

sudo make install   # installs to /usr/local by default

3. Use it

mosh user@your-server

mosh runs the client locally and the server on the host you connect into, so install this build on both machines. If you only built the new server on the remote host, you can point an existing local mosh at it with:

mosh --server=/path/to/mosh-server user@your-server

Verified on macOS (Apple Silicon, Homebrew) and the standard autotools flow on Linux. 🎉

macOS: make a binary package you can copy to another Mac (no recompiling)

If you want to build once and copy the result to other Macs, wrap the branch in a tiny local Homebrew formula and bottle it:

brew tap-new "$USER/local"
cat > "$(brew --repository "$USER/local")/Formula/mosh-osc52.rb" <<'RUBY'
class MoshOsc52 < Formula
  desc "mosh with OSC 52 clipboard types (PR #1104)"
  homepage "https://mosh.org"
  url "https://github.com/mgulick/mosh.git", branch: "osc-52-clipboard-types"
  version "1.4.0-osc52"
  license "GPL-3.0-or-later"
  depends_on "autoconf" => :build
  depends_on "automake" => :build
  depends_on "pkgconf"  => :build
  depends_on "protobuf"
  uses_from_macos "ncurses"
  def install
    ENV.append_to_cflags "-DNDEBUG"
    system "./autogen.sh"
    system "./configure", "--prefix=#{prefix}"
    system "make", "install"
  end
end
RUBY

brew install --build-bottle "$USER/local/mosh-osc52"
brew bottle "$USER/local/mosh-osc52"   # emits a relocatable .bottle.tar.gz

Copy that .bottle.tar.gz to the other Mac, add it to the formula's bottle do block (or just brew install the formula there to rebuild). The only runtime dependency is Homebrew protobuf, which brew pulls in automatically.

[ezzieyguywuf]

I built and tested this, it seems to work as expected. I can provide a code review too if that will help move this along.

[ezzieyguywuf]

@mgulick I rebased this on master (it has some commits a few months ago) and ran the tests. A few are failing:

$ make -j$(nproc) check
<snip>
PASS: ocb-aes
PASS: base64
PASS: encrypt-decrypt
PASS: local.test
PASS: nonce-incr
PASS: emulation-scroll.test
PASS: emulation-back-tab.test
PASS: emulation-80th-column.test
PASS: unicode-combine-fallback-assert.test
PASS: emulation-ascii-iso-8859.test
PASS: emulation-attributes-vt100.test
XFAIL: emulation-attributes-256color8.test
PASS: emulation-attributes-256color248.test
PASS: emulation-attributes-16color.test
PASS: emulation-attributes-truecolor.test
PASS: emulation-attributes-bce.test
PASS: emulation-multiline-scroll.test
XFAIL: e2e-failure.test
PASS: unicode-later-combining.test
PASS: e2e-success.test
PASS: emulation-cursor-motion.test
PASS: emulation-wrap-across-frames.test
PASS: network-no-diff.test
PASS: window-resize.test
PASS: repeat-with-input.test
PASS: prediction-unicode.test
PASS: server-network-timeout.test
PASS: server-signal-timeout.test
PASS: repeat.test
PASS: pty-deadlock.test
============================================================================
Testsuite summary for mosh 1.4.0
============================================================================
# TOTAL: 31
# PASS:  29
# SKIP:  0
# XFAIL: 2
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

I'm not sure if these are related to your changes or not, but it may be worth looking into

[achernya]

XFAIL is "expected to fail and did"

[ezzieyguywuf]

XFAIL is "expected to fail and did"

Oh sweet, then everything passes!