Exploring Wayland: A Modern Display Server for the Future

Integrated Computer Solutions Inc. www.ics.com
Exploring Wayland
A Modern Display Server for the Future

www.ics.com
About ICS
Established in 1987, Integrated
Computer Solutions, Inc. (ICS)
delivers innovative software
solutions with a full suite of
services to accelerate development
of successful next-gen products.
ICS is headquartered outside
Boston in Waltham, Mass. with
oﬃces in California, Canada and
Europe. Currently 160 people.

Pre Graphics

1973: Xerox Alto
Xerox Alto

1983: Apple Lisa

1984: Apple Macintosh

1985: MS Windows

1981: Stamford The V-System
- Distributed OS
- Used a synchronous message system
- Used “w” windowing system

1983: Beginning of X
- Ported to Unix System V
- W is forked into an asynchronous
Window server that will be called X
- Unix was mostly run on large
computers
- User access was via a serial terminal
- The graphics were still an add on to
the terminal not the computer

Early X Development
- Gets adopted by universities
- Version 10 was released with some issues.
- Gets adopted by UNIX vendors as an open standard, most
having already having their own window system
- X development is moved to a X Group in MIT, with
developers paid by the various UNIX vendors.
- Others are not invited to participate
- Version 11
- Unix Vendors would be able to expand X by adding their own
extensions on top of the main protocol
- Due to this the core protocol gets little to no changes

X Becomes a proper standard
- Eventually Motif would become the default widget set
- GTK, Qt and others would be built off of Xlib
- Extensions would expand the protocol with optional
features.
- After sometime XFree86 would become de facto X provider
- Eventually this would all collapse into a single group the
Xorg Foundation taking over maintenance of X
- X is ported to Linux

XServer vs XClient
- Talks to the kernel and provides a socket for clients to connect to
- Figure out what window gets an event
- Provides input to the “client” with focus
- Keyboard events
- Mouse events
- Sends render request to the hardware
- Talks to the compositor and tells it about changes on screen *
- Gets any changes from the compositor and sends them to hardware to be
rendered *
X Server

X Client
- Communicates with the XServer
- Receives events like clicks and button presses.
- Handles them
- Sends any drawing instructions back to the XServer
- Clients are window

Window manager
Window manager
- Handles the management of windows
- Where are they placed
- Draws things like title bars and borders
- If it does then its SSD
- If not CSD
- Handles how windows interact with each other
- Can you move them?
- Do they stack?
- Transforms the windows in some cases
- Maximize , Minimize , Iconify , Shade etc.. are all examples

Compositor
- Optional part used with X System
- An offscreen buffer that contains and renders all the windows.
- Started off as a solution for screen tearing
- Compositors allow for various backend. One could write to X and let it draw to
the screen
- Allows for transparency and shadows
- Backends are things like OpenGL or Vulkan , XRender
- More common on systems with graphical hardware

How X11 works
1. The kernel gets an event from an input device and sends it to X through the evdev input
driver. The kernel does all the hard work here by driving the device and translating the
different device specific event protocols to the linux evdev input event standard.
2. The X server determines which window the event affects and sends it to the clients that
have selected for the event in question on that window. The X server doesn't actually know
how to do this right, since the window location on screen is controlled by the compositor and
may be transformed in a number of ways that the X server doesn't understand (scaled
down, rotated, wobbling, etc).
3. The client looks at the event and decides what to do. Often the UI will have to change in
response to the event - perhaps a checkbox was clicked or the pointer entered a button that
must be highlighted. Thus the client sends a rendering request back to the X server.
4. When the X server receives the rendering request, it sends it to the driver to let it program
the hardware to do the rendering. The X server also calculates the bounding region of the
rendering, and sends that to the compositor as a damage event.
5. The damage event tells the compositor that something changed in the window and that it
has to recomposit the part of the screen where that window is visible. The compositor is
responsible for rendering the entire screen contents based on its scenegraph and the
contents of the X windows. Yet, it has to go through the X server to render this.
6. The X server receives the rendering requests from the compositor and either copies the
compositor back buffer to the front buffer or does a pageflip. In the general case, the X
server has to do this step so it can account for overlapping windows, which may require
clipping and determine whether or not it can page flip. However, for a compositor, which is
always fullscreen, this is another unnecessary context switch.

X11 Cons
1. Insecure (any client can read the IO of any other client)
2. Impossible to have multiple monitors with different refresh rates or scales
3. Drivers for modern hardware are not as simple as the 2d vectors X wants
4. Introduces lots of latency, messages take several round trips
5. Messages are blocking
6. Does both nothing and too much at the same time
a. Used for IPC
b. Used as middle man between client and compositor
7. X extensions are not all network transparent
a. DRM / DRI, XSHM being used will force the remote to use the normal render methods

A better way forward
- Written by the same people who were working on X
- Wayland is designed to be simple
- With “Every frame being perfect”
- Streamlined no more middle man
- Leverage existing libs in IO stack
- Names come from towns by Boston
- Wayland - Protocol
- Weston - Reference compositor
- Maynard - Shell plugin for Weston
- Waltham - wayland like remote IPC protocol

Wayland Changes
- There is no “Wayland” server its just a protocol
- The compositor is central to the window system
- The compositor does the window management, render , etc..
- Each compositor is free to do their own thing
- Apps render to a buffer
- Subsurfaces can be placed relative to the Top Level
- As well as layer surfaces (i.e panels)
- Protocols are able to evolve over time

Wayland
High level description of how wayland works
1. The kernel gets an event and sends it to the compositor. This is similar to the X case, which is
great, since we get to reuse all the input drivers in the kernel.
2. The compositor looks through its scenegraph to determine which window should receive the
event. The scenegraph corresponds to what's on screen and the compositor understands the
transformations that it may have applied to the elements in the scenegraph. Thus, the
compositor can pick the right window and transform the screen coordinates to window-local
coordinates, by applying the inverse transformations. The types of transformation that can be
applied to a window is only restricted to what the compositor can do, as long as it can compute
the inverse transformation for the input events.
3. As in the X case, when the client receives the event, it updates the UI in response. But in the
wayland case, the rendering happens in the client, and the client just sends a request to the
compositor to indicate the region that was updated.
4. The compositor collects damage requests from its clients and then recomposites the screen.
The compositor can then directly issue an ioctl to schedule a pageflip with KMS.

Wayland is protocols
Wayland is a series of protocols some required some are optional
- Stable
- Basic required to be supported
- Can be slow to get adopted all parties have to agree on the stable protocol
- Development / Experimental / Staging
- In progress protocols some may be supported
- Requires more work to keep up as the protocol evolves.
- Anyone can make a propose a protocol
- May or may not become stable one day
- Deprecated
- Should be removed from existing compostors

Wayland issues
- Can’t move or place your window?
- Can’t get window icon?
- Cant get random input so Hotkeys are out
- Copy paste between apps
- No network transparency
- Who draws decorations?

What does this mean for users ?
- Most distros will switch to or have switched to wayland by default
- Most applications should be just ﬁne if they are using GTK or Qt or so similar
toolkit
- Be aware of limits placed on app by wayland
- Apps might be placed in unusual location or unable to restore a window
- Apps can not un-minimize themselves
- Apps can’t read keyboard input without focus so hotkeys may not work in older apps
- Icons have to set set by the desktop ﬁle so apps may get a default icon

What does this mean for Developers ?
Be aware of limits placed on app by wayland
- Apps might be placed in unusual location or unable to restore a window
- Apps can not un-minimize themselves
- Apps can’t read keyboard input without focus so hotkeys may not work in
older apps
- Icons have to set set by the desktop file so apps may get a default icon
- Install a desktop file for your application
- Use reverse domain name for the icon file org.dev.myapp or com.company.product

XDG Portals
Xdg-portals provide a common api for desktop resources
- Designed for the needs of ﬂatpaks
- Allows for things like native ﬁle dialogs
- Easy to use calls to dbus interfaces
- Several helper libs like libportal
Used by
- Flatpaks
- Snaps
- Some Toolkits (Qt)

XDG Portals
Xdg-portals itself is just a spec you still need an implementation, there are several
to choose from. These portals need to communicate with the compositor so they
are also made by the various compositor making groups
Generic:
- Xapp
- Gtk
DE Speciﬁc:
- Gnome, KDE, LxQt, Hyperland, Patheon, Cosmic
Up to the implementation to support any given portal.

Two kinds of portals
1. Ones that “just work” with no interaction from the user or ability for the user to
not allow the access
2. Ones that require user input, either to ask the user for access or ask the user
to clarify what of the possible resources to access

Portal Example: File picker
One of the more commonly used portals is the ﬁle picker
- File picker is now native no matter what toolkit was used
- Get Qt Picker from GTK apps or vice versa
For most toolkits you don’t have to do anything extra to use it.

Portal Example: ScreenCast / Screenshot
- Provides access to an application to capture a
window or your whole desktop
- User is prompted on request and the portal impl
creates the dialog and the app just
- ScreenCast is for video
- Screenshot is for a frame
- Application sends a request and waits for the
response

Portal Example: Global Hotkeys
Lets your application register for and use hotkeys when not in focus.
Description
Portal for managing global shortcuts
This portal lets applications create global shortcuts sessions, and register shortcuts to them. These shortcuts are activated
regardless of the focused state of the application window.
To use this portal, applications have to create a session under which its actions will be collected. Then shortcuts can be
bound (see org.freedesktop.portal.GlobalShortcuts.Bind), and listed (see
org.freedesktop.portal.GlobalShortcuts.ListShortcuts).
All global shortcuts are bound to a session, and all sessions are bound to the application that created them.
The org.freedesktop.portal.GlobalShortcuts::Activated and org.freedesktop.portal.GlobalShortcuts::Deactivated signals are
emitted, respecitvely, whenever a shortcut is activated and deactivated.

Portal Example: Camera Access
One of the commonly used portals is the camera access
- Lets your application access a camera
- Requires the user to accept the application can access the camera.
- When active the compositor may draw a indicator to visually show the user
the camera is in use.

Portals
● Account
● Background
● Camera
● Clipboard
● Documents
● Dynamic Launcher
● Email
● File Chooser
● File Transfer
● Game Mode
● Global Shortcuts
● Inhibit
● Input Capture
● Location
● Memory Monitor
● Network Monitor
● Notification
● OpenURI
● Power Profile Monitor
● Print
● Proxy Resolver
● Realtime
● Remote Desktop
● Request
● ScreenCast
● Screenshot
● Secret
● Session
● Settings
● Trash
● Usb
● Wallpaper

Any questions?
Thanks for attending.
Slides and recording will be sent via
email tomorrow.

Exploring Wayland: A Modern Display Server for the Future

More Related Content

Similar to Exploring Wayland: A Modern Display Server for the Future (20)

More from ICS (20)

Recently uploaded (20)

Exploring Wayland: A Modern Display Server for the Future