Uncategorized | Soreau's Blog

In this post, we introduce two more animations, shatter and vortex. These two build on the concepts and infrastructure found in the previous animation code. I will focus more on the shatter animation, as it is more technically involving.

Shatter, as you might have guessed from the name, is an animation that fragments the surface into polygons and explodes them into the scene, when closing a window. When opening a window, the reverse implosion is in order. To do this, a way was needed to fragment the rectangular surface into irregular convex polygons. After searching around a bit on the web, I noticed the Voronoi diagrams concept. After learning that boost library offered some useful functions, it was off to test them.

At first, creating a Voronoi diagram was confusing. But after rendering it visually, it started to make sense. Input random points over the surface geometry and get output of convex polygons, suitable for rendering each as a triangle fan. With this in hand, the next step was obvious: rotate the polygons and translate them into an exploding fashion. However there was one small problem. The library doesn’t offer the center point of the polygon, required for rotation. After thinking a bit, I found a simple solution: compute the bounding box of each polygon and take the center of that. This worked remarkably well as it was fast. The only caveat is that the code walk the cells twice, once to get all the points required to know the bounding box and thus center point, and another to use this information when writing the vertices that the shader will use. The first iteration probably can be done once up front at animation creation instead of each frame, but this is yet to be implemented.

Vortex is a fragment shader that sucks the pixels into the center on close and the opposite on open. Now let’s watch some videos!

Let’s not sugar coat it, streaming video is a complex topic. If every browser had the same video player, it might make things a bit easier. But of course, they don’t. So, I’ll try to dumb this down, keep it short and make it as simple as possible.

If you have some RTSP H.264 streams that you want to stream to Firefox or Chrome with minimal latency, you might be in for a treat. Using nodejs and ffmpeg, we can serve up the streams while convincing the browser to play the stream immediately while buffering with minimal latency. How? As I found, it is not easy. But let’s act like it was, shall we?

In a nutshell, a browser first makes an initial GET for the url and the server replies with a video/mp4 mime type header with 200 status. We start up ffmpeg and have it write to stdout but we don’t send any of the data on this request. This behooves the client to make a range request. Next, on the range request, the client is expecting a file so it asks for portions of it. Here, we always tell it the start is 0 and do some math to tell it what range and length to expect, based on the bytes we have so far, and send 206 status. Then, we start sending the data. Getting this right was only half the battle though..

TL;DR, I checked the agent string to see if it’s chrome or not. If it’s chrome, we use `ffmpeg -f matroska` but for firefox, we have to use `ffmpeg -f mp4` since it doesn’t have matroska support. The confusing thing was that the mime type could not be video/x-matroska because that caused the browser to attempt downloading the file. Now on to the code.

The following is a snippet to stream RTSP directly to firefox or chrome browsers:

function serve_rtsp(req, res, ip, user, pass) {
    user_agent_string = req.get('user-agent');
    const range = req.headers.range;
    if (range)
    {
        if (!running)
        {
            res.end();
            return;
        }
        const start = 0;
        const end = Math.max(total_length, 1024 * 1024 * 32);
        const chunk_size = end + 1;
        res.writeHead(206, {'Accept-Ranges': 'bytes', 'Content-Range': 'bytes ' + start + '-' + end + '/' + chunk_size, 'Content-length': chunk_size, 'Content-Type': 'video/mp4', 'Connection': 'keep-alive'});
        ffmpeg.stdout.unpipe();
        ffmpeg.stdout.pipe(res);
        return;
    } else
    {
        res.writeHead(200, {'Accept-Ranges': 'bytes', 'Content-Type': 'video/mp4', 'Connection': 'keep-alive'});
    }
    if (running)
    {
        ffmpeg.kill("SIGINT");
        total_length = 0;
        clearTimeout(ffmpeg_timer);
    }
    if (user_agent_string.includes("Chrome"))
    {
        ffmpeg = child_process.spawn("ffmpeg",[
        "-i", 'rtsp://' + user + ':' + pass + '@' + ip + ':80',
        "-f", "matroska",  // File format
        "-c", "copy",      // No transcoding
        "-t", "60",        // Timeout
        "-"                // Output (to stdout)
        ]);
    }
    else
    {
        ffmpeg = child_process.spawn("ffmpeg",[
        "-i", 'rtsp://' + user + ':' + pass + '@' + ip + ':80',
        "-f", "mp4",       // File format
        "-movflags", "empty_moov+frag_keyframe",
        "-c", "copy",      // No transcoding
        "-t", "60",        // Timeout
        "-"                // Output (to stdout)
        ]);
    }
    running = true;
    res.end();
    ffmpeg.stdout.on('data', ffmpeg_handle_data);
    ffmpeg_timer = setTimeout(function() {
        running = false;
    }, 60000);
}

I hope this is helpful to someone out there. If you do find it useful and want to say thanks, you can donate a coffee on northfield.ws.

Soreau's Blog

Category Archives: Uncategorized

More Wayfire Animations

Streaming RTSP directly to browsers

Cleverly Playful