2005 was the year of Compiz. 20 years later, we’re doing it all over again. Designed with the power user in mind, Wayfire continues to push the bar with astounding animations, effects and functionality. In today’s post, we’re talking about live window previews. The concept is that you have a window button in a taskbar and hovering over it yields a tooltip containing a live preview of the corresponding window. This is a sought after feature but there are many ways to skin this cat and wayland makes for an interesting ride. You can have the client tell the compositor where to render the tooltip preview but where’s the fun in that? You could have the client request a buffer stream and send the bits to the client so it can render them as it sees fit. So how did we do it? The latter seemed to be the more reasonable choice, so that’s how it started. Wayfire has the ability (for a plugin) to communicate with a client using ipc, side banding messages not on the wayland socket wire. The way this works is: On button enter, the client sends an ipc message with the view id. The wf-live-previews plugin receives this message and creates a view-sized headless output named “live-preview-#” where “#” is the view id. Then the plugin sets up rendering the view to the output. The client gets the new wl_output global and checks the name, then uses this to begin requesting frames for the hidden output using the standard wlr_screencopy protocol. Finally, it gets the frames and displays them (in this case) in a tooltip. And if that wasn’t enough, the client in question is a python panel called panorama-panel. It uses pywayland, pywayfire ipc and gtk4 in a python client, which make for a powerful combination. Now to drive the point home, is a video:
All posts by mrscottm
Wayfire Animation Updates
Wayfire is constantly improving and new interesting animations are constantly pushing the bar. This time, we introduce a new animation part of wayfire-plugins-extra extra-animations plugin. It is called melt, and has an interesting story. Long story short, it was created by an AI LLM. Spoiler alert: It was Google ‘AI Mode’. I fed it the current fire shader and asked it to make an improved version. It spit back a shader which I massaged into an instance that wayfire could consume. In addition to the distortion there was a lot of code to render a gradient of color to simulate some fire, but I felt the distorted surfaces were spot on for a wayfire open/close animation. So I ripped the gradient out and left it as a shader that melts the view to/from existence.
Additionally, a long time wayfire contributor Daniel Kondor filed this issue. At first I dismissed it because the squeezimize shader is quite difficult to work with in its current state. However I am really glad that someone brought this up because 1) it pushed me to improve squeezimize shader and 2) Daniel wrote a companion shader which allows the squeezimize animation to reasonably handle left and right minimize targets. Before, it only handled top and bottom panels. Of course all of this text can be boring and cumbersome, so here are some videos:
Melt Animation
Improved Squeezimize
Wayfire on WSLg
You heard right, wayfire and other wlroots compositors can now run with vulkan or gles renderers on WSLg (Windows Subsystem for Linux GUI)! Before getting too excited, there is a catch.. it runs entirely in software without a render node, on wlroots headless backend. It is viewable with wayvnc (VNC) or wf-ogon plugin (RDP). This is possible with the advent of the udmabuf allocator in wlroots. It is used in conjunction with llvmpipe to render directly into RAM. The udmabuf allocator is new in latest wlroots, so it has yet to be released. However, I have taken the liberty of backporting the code to wlroots 0.18.2. I have uploaded the code to the udmabuf branch on github here. Without any modifications to wayfire GL code, wayfire runs in software. To do this, the WSL kernel must be compiled with `CONFIG_UDMABUF=y`, since this option is not default. This gives us /dev/udmabuf which key for the functionality needed for userspace dynamically allocated buffers. To get this working, latest mesa 25.x.x is required. Build wayfire with the following:
meson setup -Duse_system_wlroots=disabled -Dwlroots:c_std=c11 -Dwlroots:allocators=gbm,udmabuf --prefix=/usr build ninja -C build sudo -E ninja -C build install
and run it with
LP_NUM_THREADS=0 WLR_RENDERER_FORCE_SOFTWARE=1 WLR_BACKENDS=headless wayfire
Don’t forget to load wayvnc in [autostart] section to connect with a vnc client or load the wf-ogon plugin and connect with an rdp client. I didn’t actually try this on WSLg, but I helped those that did get it working. So I don’t have a video of my own, but this is from a user that did run it:
Cutting Corners
At this point, wayfire has been gaining momentum and the pixdecor plugin is becoming more popular. One unsung hero has been lurking however, and that is the filters plugin. The filters plugin allows a glsl shader to be applied from a file on disk to a window, or the entire screen. It comes with a few shaders to do basic things like CRT effect, color inversion and borders. The borders shader is interesting, since it renders a border of color and width on a window surface. However, with pixdecor shadows, the border doesn’t wrap the decoration, it wraps the outermost part of the shadow region. This is not what we want to happen. Meanwhile, in the CSD (client side decorations) case, the shadow area must be computed to know where to render the border. At first, this task seemed nearly impossible. But for the SSD (server side decorations) case of pixdecor, I was able to side band the shadow margin size to the filters plugin using a globally scoped class, shared between the two out-of-tree plugins. This is as if the class were defined once in a common header. Since no such header exists, we just copy and share it. This happens to work very well and allows filters to get the shadow margin size that pixdecor set. For CSD windows, the filter shader can find the shadow margins easily, with a couple of calls using wayfire’s robust plugin API. After all the rest was ironed out, the results are quite awesome. Here is a basic video to showcase the two plugins working simultaneously.
Here is the pixdecor shade effect with filters plugin:
More Wayfire Animations
In this post, we introduce two more animations, shatter and vortex. These two build on the concepts and infrastructure found in the previous animation code. I will focus more on the shatter animation, as it is more technically involving.
Shatter, as you might have guessed from the name, is an animation that fragments the surface into polygons and explodes them into the scene, when closing a window. When opening a window, the reverse implosion is in order. To do this, a way was needed to fragment the rectangular surface into irregular convex polygons. After searching around a bit on the web, I noticed the Voronoi diagrams concept. After learning that boost library offered some useful functions, it was off to test them.
At first, creating a Voronoi diagram was confusing. But after rendering it visually, it started to make sense. Input random points over the surface geometry and get output of convex polygons, suitable for rendering each as a triangle fan. With this in hand, the next step was obvious: rotate the polygons and translate them into an exploding fashion. However there was one small problem. The library doesn’t offer the center point of the polygon, required for rotation. After thinking a bit, I found a simple solution: compute the bounding box of each polygon and take the center of that. This worked remarkably well as it was fast. The only caveat is that the code walk the cells twice, once to get all the points required to know the bounding box and thus center point, and another to use this information when writing the vertices that the shader will use. The first iteration probably can be done once up front at animation creation instead of each frame, but this is yet to be implemented.
Vortex is a fragment shader that sucks the pixels into the center on close and the opposite on open. Now let’s watch some videos!
Wayfire Animations
Wayfire is maturing nicely with an animation infrastructure that makes it relatively easy to create new animations. However, even though wayfire has the nice fire animation, it only has a few other basic ones. Recently, wayfire gained support for a new option type of ‘animation’. This type allows you to select an easing as well as duration. One animation that wayfire users have been talking about for years, is the classic magic lamp minimize animation. And secretly, I’ve been wanting to write this animation. After a comment from a user in wayfire chat about this animation being the only thing missing from their wayifre-dots setup, I figured it was time to try my hand at implementing the effect. I decided to call it ‘squeezimize’, and started coding.
At first, I rendered the view in a loop, scissoring the transformed surface into horizontal lines. This worked ok, but the sigmoid curve seemed to be lost in the math, and worst of all, the implementation was slow. After bouncing around some ideas to make it faster, it seemed the way forward was to write a fragment shader to render the effect in a single pass per frame. After a lot more math and coffee, I came up with this very thing. A simple fragment shader that requires minimum glsl version and only uses basic math. It does not use builtin shader functions other than clamp(). This means it’s fast. And, the shader renders better curves than the slower line by line proof of concept.
TL;DR: Wayfire has a new shader called squeezimize that renders a magic lamp effect. Yes it’s been done, time and time again, but because it’s a single pass shader with very little input, it’s low on resources. After writing squeezimize, I wrote two more animations for wayfire open/close windows, called zap and spin. Now for a video:
EDIT: There are now two more animations, in addition to the above, helix and blinds:
Streaming RTSP directly to browsers
Let’s not sugar coat it, streaming video is a complex topic. If every browser had the same video player, it might make things a bit easier. But of course, they don’t. So, I’ll try to dumb this down, keep it short and make it as simple as possible.
If you have some RTSP H.264 streams that you want to stream to Firefox or Chrome with minimal latency, you might be in for a treat. Using nodejs and ffmpeg, we can serve up the streams while convincing the browser to play the stream immediately while buffering with minimal latency. How? As I found, it is not easy. But let’s act like it was, shall we?
In a nutshell, a browser first makes an initial GET for the url and the server replies with a video/mp4 mime type header with 200 status. We start up ffmpeg and have it write to stdout but we don’t send any of the data on this request. This behooves the client to make a range request. Next, on the range request, the client is expecting a file so it asks for portions of it. Here, we always tell it the start is 0 and do some math to tell it what range and length to expect, based on the bytes we have so far, and send 206 status. Then, we start sending the data. Getting this right was only half the battle though..
TL;DR, I checked the agent string to see if it’s chrome or not. If it’s chrome, we use `ffmpeg -f matroska` but for firefox, we have to use `ffmpeg -f mp4` since it doesn’t have matroska support. The confusing thing was that the mime type could not be video/x-matroska because that caused the browser to attempt downloading the file. Now on to the code.
The following is a snippet to stream RTSP directly to firefox or chrome browsers:
function serve_rtsp(req, res, ip, user, pass) {
user_agent_string = req.get('user-agent');
const range = req.headers.range;
if (range)
{
if (!running)
{
res.end();
return;
}
const start = 0;
const end = Math.max(total_length, 1024 * 1024 * 32);
const chunk_size = end + 1;
res.writeHead(206, {'Accept-Ranges': 'bytes', 'Content-Range': 'bytes ' + start + '-' + end + '/' + chunk_size, 'Content-length': chunk_size, 'Content-Type': 'video/mp4', 'Connection': 'keep-alive'});
ffmpeg.stdout.unpipe();
ffmpeg.stdout.pipe(res);
return;
} else
{
res.writeHead(200, {'Accept-Ranges': 'bytes', 'Content-Type': 'video/mp4', 'Connection': 'keep-alive'});
}
if (running)
{
ffmpeg.kill("SIGINT");
total_length = 0;
clearTimeout(ffmpeg_timer);
}
if (user_agent_string.includes("Chrome"))
{
ffmpeg = child_process.spawn("ffmpeg",[
"-i", 'rtsp://' + user + ':' + pass + '@' + ip + ':80',
"-f", "matroska", // File format
"-c", "copy", // No transcoding
"-t", "60", // Timeout
"-" // Output (to stdout)
]);
}
else
{
ffmpeg = child_process.spawn("ffmpeg",[
"-i", 'rtsp://' + user + ':' + pass + '@' + ip + ':80',
"-f", "mp4", // File format
"-movflags", "empty_moov+frag_keyframe",
"-c", "copy", // No transcoding
"-t", "60", // Timeout
"-" // Output (to stdout)
]);
}
running = true;
res.end();
ffmpeg.stdout.on('data', ffmpeg_handle_data);
ffmpeg_timer = setTimeout(function() {
running = false;
}, 60000);
}
I hope this is helpful to someone out there. If you do find it useful and want to say thanks, you can donate a coffee on northfield.ws.
Decoration Adventures
Wayfire has decoration plugins and even a shadow plugin available. The shadow plugin works great, for square decorations at least. Adding rounded corners with shadows that hug them would be a nice addition. So, I set out to do that, and the results even surprised me.
I started hacking on pixdecor plugin, first porting it to work with latest wayfire, then adding compute shader support for decoration animations, patterns and overlay effects. Out of all the things, none of the effects had a need to render beyond the decoration area, into the shadow region. This basically meant making the rendering area larger, to have a place to scribble in some shadows. The concept is simple but implementing it required changing the code very carefully in many different places. Even after getting it right, changing some settings might make the decoration the wrong size or some other problem (like crashing). After shaking out the bugs for the shadow padding, it was time to shade some shadows.
I knew I wanted simple code that got straight to the point. After poking around a bit on shadertoy and not finding what I wanted, I decided to write the shader from scratch. Here’s the path I ended up with. First, the decoration is cleared using a clear color or otherwise filled in with an effect shader. Then, the overlay shader is applied. The overlay has eight conditionals, one for each side and one for each corner. For the sides, it measures the distance from the edge to the pixel color being written and mixes transparent with the shadow color, depending on how far away it is. The corner is the same recipe but it picks a single point in the corner and uses it to measure the corner and shadow radius. This means that even though the corners are rounded, the shadows follow the curvature. The other nice thing about the overlay is that is automatically anti-aliases the edges of the rounded corners by mixing, instead of just discarding pixels or other method that leaves jagged corner edges.
All of this, and everything is configurable in real time, from the border size to the corner radius and shadow radius, color, etc. It took some careful coding to make everything work together but in the end, it is working nicely, with great performance. Now, for the video:
Wayfire Smoke
A wayfire user decided to port weston smoke into wayfire and while it worked, it was quite slow, even at modest sizes. I patched weston smoke to be resizable but this only showed how slow the cpu-bound algorithm is. So, I decided to try my hand at making it faster, using a compute shader. At first, there was a single shader but realized that each loop needed its own shader. It was also slow at first but after tweaking the workgroups per dispatch and the workgroup size in each shader, the difference was multi-fold. It took some time to implement and optimize but it worked out pretty well. The code has been uploaded here. The performance improvement results are astounding and there isn’t much else to do than watch some comparison videos:
Original Weston Smoke Client:
Implementation in a Decorator:
Side by Side Standalone Clients:
Wayfire Ubuntu Packages
After more than a few requests for wayfire deb packages that work on ubuntu, I decided to try creating some. The result is a success. This is how I did it.
First, I started with the latest code. I will cover wayland and wayfire packages. There are others that are required, but these two are different in that wayland packages are already in ubuntu but there are none for wayfire.
For wayland, we will copy the debian directory, containing the interesting files for building a package from upstream, and place it in the upstream code directory.
$ git clone https://git.launchpad.net/ubuntu/+source/wayland wayland-ubuntu $ git clone https://gitlab.freedesktop.org/wayland/wayland wayland-upstream $ cp -r wayland-ubuntu/debian wayland-upstream
Next, we want to prepare the environment by installing the build dependencies.
$ sudo apt-get build-dep wayland $ sudo apt install gcc g++ meson ninja debhelper dh-make
Finally we want to configure and build the packages.
$ dh_auto_configure --buildsystem=meson $ dpkg-buildpackage -rfakeroot -us -uc -b
This results in the following packages:
libwayland-bin_1.19.90-1_amd64.deb libwayland-client0_1.19.90-1_amd64.deb libwayland-cursor0_1.19.90-1_amd64.deb libwayland-dev_1.19.90-1_amd64.deb libwayland-egl1_1.19.90-1_amd64.deb libwayland-egl-backend-dev_1.19.90-1_amd64.deb libwayland-server0_1.19.90-1_amd64.deb
For wayfire, the process is similar but we need to generate the debian files. Install dependencies manually, by satisfying meson first.
$ git clone https://github.com/WayfireWM/wayfire && cd wayfire $ dh_make --createorig -p wayfire_0.8.0 $ dh_auto_configure --buildsystem=meson $ dpkg-buildpackage -rfakeroot -us -uc -b $ ls ../*.deb ../wayfire_0.8.0-1_amd64.deb
Here is a PPA containing all the packages needed for wayfire on Ubuntu >=20.04. To install wayfire:
$ sudo add-apt-repository ppa:soreau/wayfirewm $ sudo apt-get update $ sudo apt install wayfire wayfire-plugins-extra wf-shell wcm xwayland
Launch wayfire by running it from X or tty, or simply logout and select the Wayfire session.
To remove completely:
$ sudo apt install ppa-purge $ sudo ppa-purge ppa:soreau/wayfirewm