[cairo] Wayland remote rendering: Cairo/Pango as Remote Calls over VNC

[Rick van Rein]

Hi all,

I am better with security protocols than desktop rendering, but
I found a few loose ends that seem interesting to discuss here.

Wayland has no remote mode because:

 1. X11 remote rendering is geared towards Motif.  We now prefer
    more advanced widgets drawn with libraries like Pango/Cairo.

 2. VNC is inefficient because it transfers pixels, instead of the
    rendering instructions that made remote X11 efficient.  This
    even gets worse with higher display resolutions.

This gave me the idea that remote procedure calls for Cairo/Pango
APIs could support remote Wayland.  VNC may be designed for pixels,
but it is open to extensions and an RPC protocol could be added as
a pseudo-encoding for pixels.

VNC is officially defined as Remote Framebuffer in RFC 6143, with
IANA-registered unique numbers under Expert Review, so it is quite
doable to extend (Specification Required).  Extensions are offered
by the client and may be ignored by the server, which means that
such extensions need not break compatibility.  If I correctly
understood that Pango builds on top of Cairo, then it might be an
idea to define a VNC API for each, and leave it to the client how
much it wants to offer for local rendering.  Pixel rendering would
be a fallback mode, slow but useful for compatibility.

This could make VNC/RFB a solution for remote Wayland dektops, and
provide a new approach for thin desktops, which can range from mere
pixel rendering to PangoCairo rendering with, presumably, the speed
that X11 remote rendering once had.  Especially with anti-aliasing,
gradients and alpha layers it is very difficult to compress images,
but encoding the API calls would be very compact.

I am considering to extend VNC/RFB with authentication and encryption
that facilitates our open source work for Bring Your Own IDentity
support.  This would make it easy to expose a VNC server to anyone
able to decrypt, possibly to groups, much like SSH does now for text
sessions.  There does not seem to be a problem to mix windows from
various such sessions in one desktop.

Another thought I had is that SCTP may be a useful transport; its
user frames have guaranteed delivery, but ordered delivery can be
relaxed in a few ways.


Does a Pango/Cairo API over VNC make any sense to you?


Best wishes,

Rick van Rein
InternetWide.org [blog]
gitlab.com/arpa2 [code]