Planet Collabora

July 28, 2014

Edward Hervey

GUADEC 2014

Well hello there, dear Internet.

So yes, it’s been quite some time since the last blogpost. A lot has been going on, but that will be for future blogposts.

Currently at GUADEC 2014 in Strasbourg, France. Lots of interesting talks and people around.

Quite a few discussions with several people regarding future GStreamer improvements (1.4.0 is freshly out, we need to prepare 1.6 already). I’ll most likely be concentrating on the whole QA and CI side of things (more builds, make them clearer, make them do more), plan how to do nightly/weekly releases (yes, finally, I kid you not !). We also have a plan to make it faster/easier/possible for end-users (non-technical ones) to get GStreamer in their hands. More on that soon (unless Sebastian blogs about it first).

If you want to come hack on GStreamer and Pitivi, or discuss with the various contributors, there’s a Hackfest taking place at GUADEC from Wednesday to Friday. More info at https://wiki.gnome.org/GUADEC/2014/BOFs/Pitivi

by Edward Hervey at July 28, 2014 10:18 AM

July 25, 2014

Pekka Paalanen

Wayland protocol design: object lifespan

Now that we have a few years of experience with the Wayland protocol, I thought I would put some of my observations in writing. This, what will hopefully become a series rather than just one post, considers how to design Wayland protocol extensions the right way.

This first post considers protocol object lifespan and the related races between the compositor/server and the client. I assume that the reader is already aware of the Wayland protocol basics. If not, I suggest reading Chapter 4. Wayland Protocol and Model of Operation.

How protocol objects are created

On a new Wayland connection, the only object that exists is the wl_display which is a specially constructed object. You always have it, and there is no wire protocol for creating it.

The only thing the client can create next is a wl_registry through the wl_display. Registry is the root of the whole interface (class) hierarchy. Wl_registry advertises the global objects by numerical name, and using wl_registry.bind request to bind to a global is the first normal way to create a protocol object.

Binding is slightly special still, as the protocol specification in XML for wl_registry uses the new_id argument type, but does not specify the interface (class) for the new object. In the wire protocol, this special argument gets turned into three arguments: interface name (string), interface version (uint32_t), and the new object ID (uint32_t). This is unique in the Wayland core protocol.

The usual way to create a new protocol object is for the client to send a request that has a new_id type of argument. The protocol specification (XML) defines what the interface is, so there is no need to communicate the interface type over the wire. All that is needed on the wire is the new object ID. Almost all object creation happens this way.

Although rare, also the server may create protocol objects for the client. This happens by having a new_id type of argument in an event. Every time the client receives this event, it receives a new protocol object.

As all requests and events are always part of some interface (like a member of a class), this creates an interface hierarchy. For example, wl_compositor objects are created from wl_registry, and wl_surface objects are created from wl_compositor.

Object creation never fails. Once the request or event is sent, the new objects it creates exists, period. This keeps the protocol asynchronous, as there is no need to reply or check that the creation succeeded.

How protocol objects are destroyed

There are two ways to destroy a protocol object. By far the most common one is to have a request in the interface that is specified to be a destructor. Most often this request is called "destroy". When the client code calls the function wl_foobar_destroy(), the request is sent to the server and the client side proxy (struct wl_proxy) for the object gets destroyed. The server then handles the destructor request at some point in the future.

The other way is to destroy the object by an event. In that case, no destructor must be defined in the interface's protocol specification, and the event must be clearly documented to be destructive as there is no automation nor safeties for this. This is for cases where the server decides when an object dies, and requires extreme care in protocol design to work right in all cases. When a client receives such an event, all it can do is destroy the proxy. The (in)famous example of an interface like this is wl_callback.

Enter the boogeyman: races

It is very important that both the client and the server agree on which protocol objects exist. If the client sends a request on, or references as an argument, an object that does not exist in the server's opinion, the server raises a protocol error, and disconnects the client. Obviously this should never happen, nor should it happen that the server sends an event to an object that the client destroyed.

Wayland being a completely asynchronous protocol, we have no implicit guarantees. The server may send an event at the same time as the client destroys the object, and now the event targets an object the client does not know about anymore. Rather than the client shooting itself dead (that's the server's job), we have a trick in libwayland-client: it silently ignores events to destroyed objects, until the server confirms that the object is truly gone.

This works very well for interfaces where the destructor is a request. If the client first sends the destructor request and then sends another request on the destroyed object, it just shot its own head off - no race needed.

Things get tricky for the other case, destructor events. The server may send the destructor event at the same time the client is sending a request on the same object. When the server finally gets the request, the object is already gone, and the client gets taken behind the shed and shot. Therefore pretty much the only safe way to use destructor events is if the interface does not define any requests at all. Ever, not even in future extensions. Furthermore, objects with that interface should not be used as arguments anywhere, or you may hit the race. That is why destructor events are difficult to use right.

The boogeyman's brother

There is yet another nasty race with events that create objects, i.e. server-created objects. If the client is destroying the (parent) object at the same time as the server is sending an event on that object, creating a new (child) object, the server cannot know if the client actually handled the event or not. If the client ignored the event, it will never tell the server to destroy that new object, and you leak in the server.

You could try to make your way out of that pitfall by writing in your protocol specification, that when the (parent) object is destroyed, all the child objects will be destroyed implicitly. But then the client must not send the destructor request for the child objects after it has destroyed the parent, because otherwise the server sees requests on objects it does not know about, and kicks you in the groin, hard. If the child interface defines a destructor, the client cannot destroy its proxies after destroying the parent object. If the child interface does not define a destructor, you can never free the server-side resources until the parent gets destroyed.

The client could destroy all the child objects with a defined destructor in one go, and then immediately destroy the parent object. I am not sure if that works, but it might. If it does not, you have to specify a whole tear-down protocol sequence. The client tells the server it wants to destroy the parent object, the server acks and guarantees it no longer sends any events on it, then the client actually destroys the parent object. Hey, you have a round-trip and just turned a beautiful asynchronous protocol into synchronous, congratulations!

Concluding with recommendations

Here are my recommendations when designing Wayland protocol extensions:
  • Always make sure there is a guaranteed way to destroy all objects. This may sound obvious, but we have fixed several cases in the Wayland core protocol where there was no way to destroy a created protocol object such, that all resources on both server and client side could be freed. And there are still some cases not fixed.
  • Always define a destructor request. If you have any doubt whether your new interface needs a destructor request, just put it there. It is more awkward to add later than normal requests. If you do not have one, the client cannot tell the server to free those protocol object resources.
  • Do not use destructor events. They are hard to design right, and extending the interface later will be a bitch. The client cannot tell the server to free the resources, so objects with destructor events should be short-lived, and the destruction must be guaranteed.
  • Do not use server-side created objects without a serious thought. Designing the destruction sequence such that it never leaks nor explodes is tricky.

by pq (noreply@blogger.com) at July 25, 2014 07:01 PM

July 16, 2014

Vincent Sanders

It is no great secret that my colleagues at Collabora have been doing work with the Raspberry Pi Foundation.

My desk is very near Marco and I often see him working with the various Pi boards. Recently he obtained one of the new B+ units for testing and I thought it looked a little sad sat naked on his desk.

To remedy this bare board problem I designed and built a laser cut a case for him and now the B+ has been publicly released I can make the design freely available.

The design is completely original though is inspired by several other plastic "clip" type designs I have seen. Originally I created and debugged the case design for my parallella though tweaking it for the Pi was pretty easy.

The design is under a CC attribution licence and I ought to say that my employer is in no way responsible for this, its all my own fault.

by Vincent Sanders (noreply@blogger.com) at July 16, 2014 11:04 AM

July 10, 2014

Jeremy Whiting

Plasma Next is pretty darn stable

Today I wanted to share some of my experiences with using Plasma Next for the past couple of weeks. Since I had been working on some frameworks development (just a small bit here and there), I thought I'd try running Plasma Next a couple of weeks ago to see how things were coming along and to be able to work on and test some things I helped with back in KDE 4.0 days.

I have to say I'm very impressed with the stability.  I hit two issues since then, and one of the issues has been fixed. The issue I hit that has already been fixed was a crash in yakuake or konsole when closing a tab that caused the whole application to crash. I looked into the Konsole codebase with Eike Hein's guidance, but Argonel ultimately found the best patch to fix the problem.
The second issue I hit with Plasma Next has to do with disconnecting and reconnecting an external monitor. I don't do that very often at all, but when I tried last weekend I got a variety of issues. For example sometimes when disconnecting Plasma (or maybe KSmServer?) crashes and I am taken back to the sddm login screen. Other times when connecting my external screen my panel ends up floating on the external monitor but nothing on it is clickable.

I just realized this post probably sounds like a rant or complaining about Plasma Next, but that's not what I intended at all. The main point I wanted to get across is that I haven't used Plasma more than once in the past 2 weeks since Plasma Next is stable enough for my usage.

Now for the obligatory desktop screenshot:

So good job to all the people that have worked on this new iteration of the Plasma Desktop.

P.S. One other minor thing I miss from plasma is the ability to show multiple timezone's times in the clock's tooltip. I'll see if I can get that fixed though. :)

by Jeremy Whiting (noreply@blogger.com) at July 10, 2014 04:21 AM

July 09, 2014

Jeremy Whiting

Qt5/KDE Frameworks porting steps

As I said in my last post I would elaborate about how porting of libkeduvocdocument (name pending currently) from Qt4 and kdelibs4 to Qt5 and KDE Frameworks happened.

Commits can be seen here but it went like this:
1. Change CMakeLists.txt to look for frameworks and Qt5 packages.
2. Try to build, fix any errors. All while checking the Porting Notes.
3. Port away from deprecated methods.
4. Port away from kdelibs4support.

I forget which part of the above involved each of these, but this is what was changed:
Ported from KUrl to QUrl.
Ported from KStandardDirs to QStandardPaths
Ported from KGlobal::locale() to QLocale
Ported away from other deprecated methods and classes.

So rinse and repeat until it's in a state where you are happy with it.
Note that step 4 above isn't strictly necessary, and is similar to porting Qt4 applications away from Qt3Support (Some kde4 applicationss never were ported away from Qt3Support sadly...) Yes KMouth, I'm looking at you.

by Jeremy Whiting (noreply@blogger.com) at July 09, 2014 05:42 AM

July 07, 2014

Jeremy Whiting

Libkeduvocdocument Qt5/KDE Frameworks port

Hello all. Yes I'm still alive. Yes I'm still doing KDE stuff as I find time or make time. I'll report in the next few posts about what is happening and where we are going.
One of the things that happened recently was the port of libkeduvocdocument to Qt5 and frameworks. Vishesh started the effort, and I completed it with some review by Aleix Pol. It was decided as documented here that since libkdeedu only contains libkeduvocdocument it should be split up. Upon further investigation we realized that the other parts of libkdeedu are not used anymore. Besides the icons subfolder that are still looking for a home, the rest of the git repo is only libkeduvocdocument related, so we decided to just rename the git repository for the frameworks and going forward release. So the libkdeedu git repository holds the kde sc 4 codebase, while the libkeduvocdocument git repo holds the qt5 and frameworks based code. Both contain the history so all history is preserved. 

I'll write next time about the steps taken to port the library to Qt5 and KDE Frameworks.

by Jeremy Whiting (noreply@blogger.com) at July 07, 2014 01:49 AM

June 25, 2014

Emilio Pozuelo Monfort

Firefox and GTK+ 3

Lately at Collabora I have been working on helping Mozilla with the GTK+ 3 port of Firefox.

The problem

The issue we had to solve is that GTK+ 2 and GTK+ 3 cannot be loaded in the same address space. Moving Firefox from GTK+ 2 to GTK+ 3 isn’t a problem, as only GTK+ 3 gets loaded in its address space, and everything is fine. The problem comes when you load a plugin that links to GTK+ 2, e.g. Flash. Then, GTK+ 2 and GTK+ 3 get both loaded, GTK+ detects that, and aborts to avoid bigger problems. This was tracked as bug #624422.

More specifically, Firefox links to libxul.so, which in turn links to GTK+. These days, the plugins are loaded in a separate process, plugin-container, which communicates with the Firefox process through IPC. If plugin-container didn’t link to GTK+, there would be absolutely no problem, as the browser (Firefox) process could link to GTK+ 3 and plugin-container could load any plugin, including GTK+ 2 ones. However, although plugin-container doesn’t directly use GTK+, it links to libxul.so for IPC, which brings GTK+ into its address space.

The solution

In order to solve this, we evaluated various options. The first one was to split libxul.so in two parts, one with the IPC code and lower level stuff, which wouldn’t link to GTK+, and another side with the rest of the code, including all the widget and toolkit integration, which would obviously link to GTK+. However this turned not to be possible as the libxul code was too intricate.

In the end, we decided to add a thin layer between libxul and GTK+, which we called libmozgtk.so. This small layer links to GTK+ 3, and provides stubs for GTK+ 2 specific symbols. Additionally, there is a libmozgtk2.so with SONAME “libmozgtk.so”, which links to GTK+ 2 and provides stubs for GTK+ 3 symbols. We made libxul link against libmozgtk.so, and so when Firefox runs, libxul.so, libmozgtk.so, and GTK+ 3 are loaded, and Firefox uses GTK+ 3. However when plugin-container is executed, we add LD_PRELOAD=libmozgtk2.so in the environment. Since libmozgtk2.so has a libmozgtk.so SONAME, the libxul.so dependency is satisfied, and the plugin-container process ends with GTK+ 2. Since plugin-container doesn’t make use of the GTK+ code in libxul, this is safe, and we end up with a GTK+ 3 Firefox that can load GTK+ 2 plugins. The end result is that you can watch Youtube videos again!

While this solution is somewhat hacky, it means we didn’t need to mess with libxul, splitting it in two just for the Linux/GTK+ port’s sake. And when the GTK+ 2 plugins become irrelevant, or NPAPI support is removed (as it recently happened in Chrome), we should be able to easily revert this and use GTK+ 3 everywhere.

Wayland

On an unrelated note, we have looked a bit at porting Firefox to Wayland. Wayland is designed to be a replacement for X11, and is becoming very popular in the digital TV and set top box space. Those obviously need HTML engines and web browsers, and with WebKit and Chrome already having Wayland ports, we think Firefox shouldn’t fall behind.

For this, the GTK+ 3 port was a prerequisite, but that isn’t enough. There are many X11 uses on the Firefox codebase, most of which are guarded by #ifdef MOZ_X11, though not all of them are. We got Firefox to start on Weston (the Wayland reference compositor) with a bunch of hacks, one of which broke keyboard input (but avoided a segfault). As you can see from the screenshot, things aren’t perfect, but it’s at least a good start!

Firefox running on Weston

by Emilio Pozuelo Monfort at June 25, 2014 09:25 AM

June 21, 2014

Nick Richards

Pinpoint COPR Repo

A few years ago I worked with a number of my former colleagues to create Pinpoint, a quick hack that made it easier for us to give presentations that didn't suck. Now that I'm at Collabora I have a couple of presentations to make and using pinpoint was a natural choice. I've been updating our internal templates to use our shiny new brand and wanted to use some newer features that weren't available in Fedora's version of pinpoint.

There hasn't been an official release for a little while and a few useful patches have built up on the master branch. I've packaged a git snapshot and created a COPR repo for Fedora so you can use these snapshots yourself. They're good.

by Nick Richards at June 21, 2014 08:59 PM

June 20, 2014

Philip Withnall

gobject-introspection gets a development mailing list

Public service announcement: if you’re a bindings author, or are otherwise interested in the development of GIR annotations, the GIR format or typelib format, please subscribe to the gir-devel-list mailing list. It’s shiny and new, and will hopefully serve as a useful way to announce and discuss changes to GIR so that they’re suitable for all bindings.

Currently under discussion (mostly in bug #719966): changes to the default nullability of gpointer nodes, and the addition of a (never-null) annotation to complement (nullable).

by Philip Withnall at June 20, 2014 01:13 PM

June 10, 2014

George Kiagiadakis

GStreamer on wayland with GTK+

During the past few months I’ve been occasionally working on integrating GStreamer better with wayland. GStreamer already has a ‘waylandsink’ element in gst-plugins-bad, but the implementation is very limited. One of the things I’ve been working on was to add GstVideoOverlay support in it, and recently, I managed to make this work nicely embedded in a GTK+ window.

GStreamer Wayland GTK Demo

gtk+ video player demo running on weston

I’m happy to say that it works pretty well, even though GTK does not support wayland sub-surfaces, which was being thought of as a problem initially. It turns out there is no problem with that, and both the GTK and GstVideoOverlay APIs are sufficient to make this work. However, there needs to be a small addition in GstVideoOverlay to allow smooth resizing. Currently, I have a GstWaylandVideo API that includes those additions.

This essentially means that as soon as this work is merged (hopefully soon), there is nothing stopping applications like totem from being ported to wayland :D

I believe embedding waylandsink in Qt should also work without any problems, I just haven’t tested it.

If you are interested, check the code. The code of the above running demo can also be found here, and the ticket for merging this branch is being tracked here.

I should say many thanks here to my employer, Collabora, for sponsoring this work.


by gkiagia at June 10, 2014 12:24 PM

June 05, 2014

Nick Richards

Suburbs Driven by Trains Not Cars

I was reading Maciej Cegłowski's excellent talk, 'The Internet With A Human Face' and aside from it's persuasive argument found something interesting in the tension between his argument rooted in American reality and his German audience. I've also recently been reading 'Concretopia', a book about the post war rebuilding of Britain and realised that there, in the gap between the wars was the suburbia he wasn't aware of. I am of course talking of Metroland, where I grew up.

Clouds, Pond, Trees

This environment was enabled by a fascinating business model where the railway company created their own demand. They built houses in the middle of nowhere near London where transport to work required a train journey, giving themselves ongoing revenue and financing the building of the railway (and more). Most recently London has seen a similar excercise in commercial demand creation with Arsenal financing their new stadium (and a bit more) with property development on the old stadium.

As such Metro-land was inherently commuter based from the beginning and unsurprisingly it remains so today, although the nearby presence of the M25 has drawn off some of the rail traffic that existed before. But what's different about a train driven suburbia? It's still strip development, as the logic of the tracks wants a direct and ideally flat route. However it also leads to a more freeway style of node location due to the distance required between stations if the network is to be efficient. There is a greater density of housing than you'd expect of car driven suburbia as well since most houses needed to be within walking or cycling range of the station. This distinctive landscape of small towns and villages where there's nothing to do is characteristic of Metro-land. A parade is different to a strip mall, but they're two sides of the same coin.

All that aside, the artistic flowering of this railway suburbia wasn't just in the lovely graphic design and dead on branding of the Metropolitan Railway company, the poetry of Betjeman, music of Elton John and writing of J.G Ballard are just as distinctively suburban and just as meaningful. In the end railway suburbia is fundamentally different and it comes down to a key phrase in Maciej's talk:

When everyone has a car, it means you can't get anywhere without one. Instead of freeing you, the car becomes a cage.

The railway isn't a cage, it's a highway to possibility.

by Nick Richards at June 05, 2014 07:40 PM

Pekka Paalanen

From pre-history to beyond the global thermonuclear war

This is a short and vague glimpse to the interfaces that the Linux kernel offers to user space for display and graphics management, from the history to what is hot and new, to what might perhaps be coming after. The topic came current for me when I started preparing Weston for global thermonuclear war.

The pre-history


In the age of dragons, kernel mode setting did not exist. There was only user space mode setting, where the job of the kernel driver (if any) was simply to give user space direct access to the graphics card registers. A user space driver (well, Xorg video DDX, really, err... or what it was at the time of XFree86) would then poke the card registers to set a mode. The kernel had no idea of anything.

The kernel DRM infrastructure was started as an out-of-tree kernel module for cooperating between multiple programs wanting to access the graphics card's resources. Later it was (partially?) merged into the kernel tree (the year is a lie, 2.3.18 came out in 1999), and much much later it was finally deleted from the libdrm repository.

The middle age


For some time, the kernel DRM existed alongside user space mode setting. It was a dark time full of crazy hacks to keep it all together with duct tape, barbwire and luck. GPUs and hardware accelerated OpenGL started to come up.

The new age


With the invent of kernel mode setting (KMS), the DRM kernel drivers got in charge of the graphics card resources: outputs, video modes, memory allocations, hotplug! User space mode setting became obsolete and was eventually killed. The kernel driver was finally actually in control of the graphics hardware.

KMS probably started with just setting the main framebuffer (primary plane) for each "CRTC" and programming the video mode. A CRTC is for "cathode-ray tube controller", but essentially means a block that reads memory (a framebuffer) and produces a bitstream according to video mode timings. The bitstream is directed into an "encoder", which turns it into a proper physical/analogue signal, like VGA or digital DVI. The signal then exits the graphics card though a "connector". CRTC, encoder, and connector are the basic concepts in KMS API. Quite often these can be combined in some restricted ways, like a single CRTC feeding two encoders for clone mode.

Even ancient hardware supported hardware cursors: a small sprite that was composited into the outgoing video signal on the fly, which meant that it was very cheap to move around. Cursor being so special, and often with funny color format (alpha!), got its very own DRM ioctl.

There were also hardware overlays (additional or secondary planes) on some hardware. While the primary framebuffer covers the whole display, an overlay is another buffer (just like the cursor) that gets mixed into the bitstream at the CRTC level. It is like basic compositing done on the scanout hardware level. Overlays usually had additional benefits, for example they could apply scaling or color space conversion (hello, video players) very efficiently. Overlays being different, they too got their very own DRM ioctls.

The KMS user space ABI was anything but atomic. With the X11 tradition, it wasn't too important how to update the displays, as long as the end result eventually was what you wanted. Race conditions in content updates didn't matter too much either, as X was racy as hell anyway. You update the CRTC. Then you update each overlay. You might update the cursor, too. By luck, all these updates could hit the same vblank. Or not. Or you don't hit vblank at all, and get tearing. No big deal, as X was essentially all about front-buffer rendering anyway. (And then there were huge efforts in trying to fix it all up with X, GLX, Mesa and GL-compositors, and avoid tearing, and it ended up complicated.)

With the advent of X compositing managers, that did not play well with the  awkward X11 protocol (Xv) or the hardware overlays, and with rise of the  GPU power and OpenGL, it was thought that hardware overlays would  eventually die out. Turned out the benefits of hardware overlays were too great to abandon, and with Wayland we again have a decent chance to make the most of them while still enjoying compositing.

The global thermonuclear war (named after a git branch by Rob Clark)


The quality of display updates became important. People do not like tearing. Someone actually wanted to update the primary framebuffer and the overlays on the same vblank, guaranteed. And the cursor as the cherry on top.

We needed one ABI to rule them all.

Universal planes brings framebuffers (primary planes), overlays (secondary planes) and cursors (cursor planes) together under the same API. No more type specific ioctls, but common ioctls shared by them all. As these objects are still somewhat different, overlays having wildly differing features and vendors wanting to expose their own stuff, object properties were invented.

An object property is essentially a {key, value} pair. In the API, the name of a key is a string. Each object has its own set of keys. To use a key, you must know it by name, fetch the handle, and then use the handle when setting the value. Handles seem to be per-object, so make sure to fetch them separately for each.

Atomic mode setting and nuclear pageflip are two sides of the same feature. Atomicity is achieved by gathering a set of property changes, and then pushing them all into the kernel in a single ioctl call. Then that call either succeeds or fails as a whole. Libdrm offers a drmModePropertySet for gathering the changes. Everything is exposed as properties: the attached FB, overlay position, video mode, etc.

Atomic mode setting means setting the output modes of a single graphics device, more or less. Devices may have hard to express limitations. A simple example is the available scanout memory bandwidth: You can drive either two mid-resolution outputs, or one high-resolution output. Or maybe some crtc-encoder-connector combination is not possible with a particular other combination for another output. Collecting the video mode, encoder and connector setup over the whole grahics card into a single operation avoids flicker. Either the whole set succeeds, or it fails. Without atomic mode setting, changing multiple outputs would not only take longer, but if some step failed, you'd have to undo all earlier steps (and hope the undo steps don't fail). Plus, there would be no way to easily test if a certain combination is possible. Atomic mode setting fixes all this.

Nuclear pageflip is about synchronizing the update of a single output (monitor) and making that atomic. This means that when user space wants to update the primary framebuffer, move the cursor, and update a couple of overlays, all those changes happen at the same vblank. Again it all either succeeds or fails. "Every frame is perfect."

And then there shall be ponies (at the end of the rainbow)


Once the global thermonuclear war is over, we have the perfect ABI for driving display updates.

Well, almost. Enter NVidia G-Sync, or AMD's FreeSync which is actually backed by a VESA standard. Dynamically variable refresh rate. We have no way yet for timing display updates in DRM. All we can do is kick out a display update, and it will hopefully land on the next vblank, whenever that is. But we can't tell the DRM when we would like it to be. Everything so far assumes, that the display refresh rate is a constant, apart from an explicit mode switch. Though I have heard that e.g. Chrome for Intel (i915, LVDS/eDP reclocking) has some hacks that opportunistically drops the refresh rate to save power.

There is also a culprit in the DRM of today (Jun 3rd, 2014). You can schedule a pageflip, but if you have pending rendering on that framebuffer for the same GPU as were you are presenting it, the pageflip will not happen until the rendering completes. And you do not know when it will complete, which means you do not know if you will hit the very next vblank or something later.

If the rendering GPU is not the same graphics device that presents the framebuffer, you do not get synchronization at all. That means that you may be scanning out an incomplete rendering for a frame or two, or you have to stall the GPU to make sure it is done before scheduling the page flip. This should be fixed with the fences related to dma-bufs (Hi, Maarten Lankhorst).

And so the unicorn keeps on running.

by pq (noreply@blogger.com) at June 05, 2014 11:00 AM

May 15, 2014

Luis de Bethencourt

GStreamer and emacs

Debug logs are an extremely helpful tool in the GStreamer developer's toolbox.
Most will say you can't live without them.

Something I've always missed when reading them is a convenient way to jump back and forth between the logs and the source code. So I went ahead and wrote an emacs mini mode that does exactly this:

emacs-gstreamer
an emacs mini module to navigate GStreamer debug logs.


When hitting Enter or M-. in a log file it will open the source code to the line that generated that debug message. If you have multiple emacs windows open, it will open the GStreamer source code file in the second to last active so you can continue reading the log in the active window. If you only have one window open it will open the source code file in the current one. After that you can use your favorite window and buffer handling to surf the files. Read, learn, write, and develop.

Click here to watch a screencast


To get it running you need to have loaded a tags table with the source code. Read this other article to learn how. I run it as part of my gst-uninstalled script.
Then just run M-x gst-debug in the debug log file's buffer.

Let me know if it helps your development workflow!

May 15, 2014 06:30 PM

May 08, 2014

Philip Withnall

How widely is the GNOME stack used?

After a couple of discussions at the DX hackfest about cross-platform-ness and deployment of GLib, I started wondering: we often talk about how GNOME developers work at all levels of the stack, but how much of that actually qualifies as ‘core’ work which is used in web servers, in cross-platform desktop software1, or commonly in embedded systems, and which is security critical?

On desktop systems (taking my Fedora 19 installation as representative), we can compare GLib usage to other packages, taking GLib as the lowest layer of the GNOME stack:

Package Reverse dependencies Recursive reverse dependencies
glib2 4001
qt 2003
libcurl 628
boost-system 375
gnutls 345
openssl 101 1022

(Found with repoquery --whatrequires [--recursive] [package name] | wc -l. Some values omitted because they took too long to query, so can be assumed to be close to the entire universe of packages.)

Obviously GLib is depended on by many more packages here than OpenSSL, which is definitely a core piece of software. However, those packages may not be widely used or good attack targets. Higher layers of the GNOME stack see widespread use too:

Package Reverse dependencies
cairo 2348
gdk-pixbuf2 2301
pango 2294
gtk3 801
libsoup 280
gstreamer 193
librsvg2 155
gstreamer1 136
clutter 90

(Found with repoquery --whatrequires [package name] | wc -l.)

Widely-used cross-platform software which interfaces with servers2 includes PuTTY and Wireshark, both of which use GTK+3. However, other major cross-platform FOSS projects such as Firefox and LibreOffice, which are arguably more ‘core’, only use GNOME libraries on Linux.

How about on embedded systems? It’s hard to produce exact numbers here, since as far as I know there’s no recent survey of open source software use on embedded products. However, some examples:

So there are some sample points which suggest moderately widespread usage of GNOME technologies in open-source-oriented embedded systems. For more proprietary embedded systems it’s hard to tell. If they use Qt for their UI, they may well use GLib’s main loop implementation. I tried sampling GPL firmware releases from gpl-devices.org and gpl.nas-central.org, but both are quite out of date. There seem to be a few releases there which use GLib, and a lot which don’t (though in many cases they’re just kernel releases).

Servers are probably the largest attack surface for core infrastructure. How do GNOME technologies fare there? On my CentOS server:

  • GLib is used by the popular web server lighttpd (via gamin),
  • the widespread logging daemon syslog-ng,
  • all MySQL load balancing via mysql-proxy, and
  • also by QEMU.
  • VMware ESXi seems to use GLib (both versions 2.22 and 2.24!), as determined from looking at its licencing file. This is quite significant — ESXi is used much more widely than QEMU/KVM.
  • The Amanda backup server uses GLib extensively,
  • as do the clustering solutions Heartbeat and Pacemaker.

I can’t find much evidence of other GNOME libraries in use, though, since there isn’t much call for them in a non-graphical server environment. That said, there has been heavy development of server-grade features in the NetworkManager stack, which will apparently be in RHEL 7 (thanks Jon).

So it looks like GLib, if not other GNOME technologies, is a plausible candidate for being core infrastructure. Why haven’t other GNOME libraries seen more widespread usage? Possibly they have, and it’s too hard to measure. Or perhaps they fulfill a niche which is too small. Most server technology was written before GNOME came along and its libraries matured, so any functionality which could be provided by them has already been implemented in other ways. Embedded systems seem to shun desktop libraries for being too big and slow. The cross-platform support in most GNOME libraries is poorly maintained or non-existent, limiting them to use on UNIX systems only, and not the large OS X or Windows markets. At the really low levels, though, there’s solid evidence that GNOME has produced core infrastructure in the form of GLib.


  1. As much as 2014 is the year of Linux on the desktop, Windows and Mac still have a much larger market share. 

  2. And hence is security critical. 

  3. Though Wireshark is switching to Qt. 

by Philip Withnall at May 08, 2014 05:10 PM

Tomeu Vizoso

GNOME API reference at the DevX hackfest

Last week I spent a few days at the Developer Experience hackfest and got to have some fun again with the API reference generator in gobject-introspection.

Jon intended to hack on the XSLT stylesheets in yelp-tools/xsl, but after some trying he got discouraged by the amount of work that would take to get them to generate the HTML that we are interested in. We also discussed the benefits of using Mallard for generating the reference docs, and given that we want to generate a single output, we couldn't see much value in the level of indirection that Mallard adds.

Thus, we considered generating HTML directly from the GIR files, but shortly after Alberto Ruiz came by and offered to explore a client-side-only solution involving processing JSON files with JavaScript.

He very quickly got something relatively complete, which is very encouraging, but even more so is seeing how other projects are generating their API references that way, for example: http://docs.sencha.com/extjs/4.2.2/#!/api/Ext.Class

Aspects such as as-you-type search would bring the online documentation on par with the Devhelp experience. Plus they have some niceties such as a symbol browser and links to annotated source code.

I have hacked a (yet another) --write-json-files switch to g-ir-doc-tool that will output the content of the GIR file to JSON, but indexed and formatted as needed by the JS side of things. See this branch for the code.

That branch also adds support for Markdown rendering using python-markdown, but some more code needs to be written to implement the extensions to Markdown that the Gtk+ docs are using.

It was great to talk about all this and more with old and new friends in Berlin, so I'm very grateful to the GNOME Foundation for organizing it and sponsoring travel, and to Endocode for providing a venue. And special thanks to Chris Kühl for the great organization!



by Tomeu Vizoso (noreply@blogger.com) at May 08, 2014 12:41 PM

May 06, 2014

Philip Withnall

Berlin DX hackfest and Clang

Last week I was in Berlin at the GNOME DX hackfest. My goal for the hackfest was to do further work on the fledgling gnome-clang, and work out ways of integrating it into GNOME. There were several really fruitful discussions about GIR, static analysis, Clang ASTs, and integration into Builder which have really helped flesh out my plans for gnome-clang.

The idea we have settled on is to use static analysis more pervasively in the GNOME build process. I will be looking into setting up a build bot to do static analysis on all GNOME modules, with the dual aims of catching bugs and improving the static analyser. Eventually I hope the analysis will become fast enough and accurate enough to be enabled on developers’ machines — but that’s a while away yet.

(For those who have no idea what gnome-clang is: it’s a plugin for the Clang static analyser I’ve been working on, which adds GLib- and GObject-specific checks to the static analysis process.)

One key feature I was working on throughout the hackfest was support for GVariant format string checking, which has now landed in git master. This will automatically check variadic parameters against a static GVariant format string in calls to g_variant_new(), g_variant_get() and other similar methods.

For example, this can statically catch when you forget to add one of the elements:

/*
 * Expected a GVariant variadic argument of type ‘int’ but there wasn’t one.
 *         floating_variant = g_variant_new ("(si)", "blah");
 *                                           ^
 */
{
	floating_variant = g_variant_new ("(si)", "blah");
}

Or the inevitable time you forget the tuple brackets:

/*
 * Unexpected GVariant format strings ‘i’ with unpaired arguments. If using multiple format strings, they should be enclosed in brackets to create a tuple (e.g. ‘(si)’).
 *         floating_variant = g_variant_new ("si", "blah", 56);
 *                                           ^
 */
{
	floating_variant = g_variant_new ("si", "blah", 56);
}

After Zeeshan did some smoketesting of it (and I fixed the bugs he found), I think gnome-clang is ready for slightly wider usage. If you’re interested, please install it and try it out! Instructions are on its home page. Let me know if you have any problems getting it running — I want it to be as easy to use as possible.

Another topic I discussed with Ryan and Christian at the hackfest was the idea of a GMainContext visualiser and debugger. I’ve got some ideas for this, and will hopefully find time to work on them in the near future.

Huge thanks to Chris Kühl and Endocode for the use of their offices and their unrivalled hospitality. Thanks to the GNOME Foundation for kindly sponsoring my accommodation; and thanks to my employer, Collabora, for letting me take community days to attend the hackfest.

by Philip Withnall at May 06, 2014 09:49 AM

May 04, 2014

Xavier Claessens

OTR in Empathy

It was a really popular Empathy feature request, it even has a bounty on it, and today I just did it! The patches are available in bugzilla, apply on telepathy-gabble and empathy master.

Here is a screencast of OTR in action between Empathy and Pidgin:

I hacked it on my free time, let’s see if crowd funded projects really works. If you like that feature feel free to make a donation on the bounty: https://freedomsponsors.org/core/issue/333/telepathy-should-support-otr-encryption.

by xclaesse at May 04, 2014 01:59 AM

May 01, 2014

Philip Withnall

Developer experience hackfest 2014

Here in sunny Berlin, progress is being made on documentation, developer tools, and Club Mate. I’ve heard rumours of plans for an updated GTK+ data model and widgets. The documentationists are busy alternating between massaging and culling documentation pages. There are excited discussions about the possibilities created by Builder.

I’ve been keeping working on gnome-clang, and have reached a milestone with GVariant error checking:

gvariant-test.c:10:61: error: [gnome]: Expected a GVariant variadic argument of type ‘char *’ but saw one of type ‘guint’.
        some_variant = g_variant_new ("(sss)", "hello", my_string, a_little_int_short_and_stout);
                                                                   ^

More details coming soon, once I’ve tidied it all up and committed it.

A GNOME Foundation sponsorship badge.

by Philip Withnall at May 01, 2014 10:35 AM

April 29, 2014

George Kiagiadakis

CommonsFest

Hello all! Long time no blog. The reason I decided to blog this time is that I want to spread the word about CommonsFest, a different festival that will take place in less than two weeks (9-11 May) in my home city, Heraklion, in Crete.

What is it?

CommonsFest-Logo-BlueCommonsFest is a festival that aims to promote freedom of knowledge (or free knowledge) and peer-to-peer collaboration for the creation and management of the Commons. A philosophy that has spread through free software communities and extends to many aspects of our daily lives, such as the arts, governance, construction of machinery, tools and other goods. In other words, it aims to raise the awareness of people about the philosophy of open source in its generalisation, involving open source code, open hardware, creative commons and similar initiatives like open source ecology, open governance, etc.

Why?

This festival was conceived and organized for the first time in Heraklion last year by a group of volunteers, myself being one of them, who felt that it is worth promoting this philosophy to people that are unaware of its existence, especially in this time of economic crisis where it is clear that people need to cooperate more and share in order to go forward. This year we are repeating it, aiming to be better and achieve even more. Our long term vision is to spread this philosophy to as many people as possible around the world and eventually improve our lives by changing the way people think. We believe in this idea that the world would be a better place if we all shared our knowledge and worked together, equally, for the well-being of all of us.

I love the idea, how can I be part of it?

There are many ways you can help. First of all, this year, the festival is crowd-funded to cover our expenses (mainly printing and transportation for the speakers). If you like the idea and would like to support us directly, you could give a small donation. Another way you can help is to just share this with other people that may (or may not) be interested – even if it’s just to raise their awareness about the subject ;). Finally, you could help to achieve our goals by organizing something similar in your area. All our material (logos, texts, slideshows, etc) is freely licensed under creative commons licences, so you can also use them if you want.

Where can I learn more?

Have a look at our website, our fund-raising campaign, our facebook, google+ and twitter pages. You can also contact the organizing team by email.

CF-CrowdFunding-Campaign-Banner


by gkiagia at April 29, 2014 10:10 AM

April 23, 2014

Nick Richards

My Incredible Journey

Yesterday I resigned from Lumi, ending my startup journey for a while. I'll be starting a new job working for Collabora in a month. I'm really looking forward to be back working on open source software and the new challenge.

This'll be my first job without 'Designer' in the title for about 10 years. Although I've been doing lots of solution definition, product/project managing and suchlike this'll be the first time it actually says that on a business card (we didn't have job titles or business cards at Lumi, it was that sort of place).

Other than having a new desk, and travelling to Cambridge on a regular basis I'm reluctant to predict too far ahead. I'm certainly looking forward to getting a bit of perspective on Lumi and hopefully writing something about the patterns and anti-patterns that I saw whilst working there in the future. In the interim, suggestions on today's best Linux laptops are gratefully received.

by Nick Richards at April 23, 2014 11:30 AM

April 19, 2014

Philip Withnall

Ensuring functions are called in the right context

Continuing in this fledgling series of examining GLib’s GMainContext, this post looks at ensuring that functions are called in the right main context when programming with multiple threads.

tl;dr: Use g_main_context_invoke_full() or GTask. See the end of the post for some guidelines about multi-threaded programming using GLib and main contexts.

To begin with, what is ‘the right context’? Taking a multi-threaded GLib program, let’s assume that each thread has a single GMainContext running in a main loop — this is the thread default main context.((Why use main contexts? A main context effectively provides a work or message queue for a thread — something which the thread can periodically check to determine if there is work pending from another thread. It’s not possible to pre-empt a thread’s execution without using hideous POSIX signalling). I’m ignoring the case of non-default contexts, but their use is similar.)) So ‘the right context’ is the one in the thread you want a function to execute in. For example, if I’m doing a long and CPU-intensive computation I will want to schedule this in a background thread so that it doesn’t block UI updates from the main thread. The results from this computation, however, might need to be displayed in the UI, so some UI update function has to be called in the main thread once the computation’s complete. Furthermore, if I can limit a function to being executed in a single thread, it becomes easy to eliminate the need for locking a lot of the data it accesses((Assuming that other threads are implemented similarly and hence most data is accessed by a single thread, with threads communicating by message passing, allowing each thread to update its data at its leisure.)), which makes multi-threaded programming a whole lot simpler.

For some functions, I might not care which context they’re executed in, perhaps because they’re asynchronous and hence do not block the context. However, it still pays to be explicit about which context is used, since those functions may emit signals or invoke callbacks, and for reasons of thread safety it’s necessary to know which threads those signal handlers or callbacks are going to be invoked in. For example, the progress callback in g_file_copy_async() is documented as being called in the thread default main context at the time of the initial call.

The core principle of invoking a function in a specific context is simple, and I’ll walk through it as an example before demonstrating the convenience methods which should actually be used in practice. A GSource has to be added to the specified GMainContext, which will invoke the function when it’s dispatched. This GSource should almost always be an idle source created with g_idle_source_new(), but this doesn’t have to be the case. It could be a timeout source so that the function is executed after a delay, for example.

As described previously, this GSource will be added to the specified GMainContext and dispatched as soon as it’s ready((In the case of an idle source, this will be as soon as all sources at a higher priority have been dispatched — this can be tweaked using the idle source’s priority parameter with g_source_set_priority(). I’m assuming the specified GMainContext is being run in a GMainLoop all the time, which should be the case for the default context in a thread.)), calling the function on the thread’s stack. The source will typically then be destroyed so the function is only executed once (though again, this doesn’t have to be the case).

Data can be passed between threads in this manner in the form of the user_data passed to the GSource’s callback. This is set on the source using g_source_set_callback(), along with the callback function to invoke. Only a single pointer is provided, so if multiple bits of data need passing, they must be packaged up in a custom structure first.

Here’s an example. Note that this is to demonstrate the underlying principles, and there are convenience methods explained below which make this simpler.

/* Main function for the background thread, thread1. */
static gpointer
thread1_main (gpointer user_data)
{
	GMainContext *thread1_main_context = user_data;
	GMainLoop *main_loop;

	/* Set up the thread’s context and run it forever. */
	g_main_context_push_thread_default (thread1_main_context);

	main_loop = g_main_loop_new (thread1_main_context, FALSE);
	g_main_loop_run (main_loop);
	g_main_loop_unref (main_loop);

	g_main_context_pop_thread_default (thread1_main_context);
	g_main_context_unref (thread1_main_context);

	return NULL;
}

/* A data closure structure to carry multiple variables between
 * threads. */
typedef struct {
	gchar *some_string;  /* owned */
	guint some_int;
	GObject *some_object;  /* owned */
} MyFuncData;

static void
my_func_data_free (MyFuncData *data)
{
	g_free (data->some_string);
	g_clear_object (&data->some_object);
	g_slice_free (MyFuncData, data);
}

static void
my_func (const gchar *some_string, guint some_int,
         GObject *some_object)
{
	/* Do something long and CPU intensive! */
}

/* Convert an idle callback into a call to my_func(). */
static gboolean
my_func_idle (gpointer user_data)
{
	MyFuncData *data = user_data;

	my_func (data->some_string, data->some_int, data->some_object);

	return G_SOURCE_REMOVE;
}

/* Function to be called in the main thread to schedule a call to
 * my_func() in thread1, passing the given parameters along. */
static void
invoke_my_func (GMainContext *thread1_main_context,
                const gchar *some_string, guint some_int,
                GObject *some_object)
{
	GSource *idle_source;
	MyFuncData *data;

	/* Create a data closure to pass all the desired variables
	 * between threads. */
	data = g_slice_new0 (MyFuncData);
	data->some_string = g_strdup (some_string);
	data->some_int = some_int;
	data->some_object = g_object_ref (some_object);

	/* Create a new idle source, set my_func() as the callback with
	 * some data to be passed between threads, bump up the priority
	 * and schedule it by attaching it to thread1’s context. */
	idle_source = g_idle_source_new ();
	g_source_set_callback (idle_source, my_func_idle, data,
	                       (GDestroyNotify) my_func_data_free);
	g_source_set_priority (idle_source, G_PRIORITY_DEFAULT);
	g_source_attach (idle_source, thread1_main_context);
	g_source_unref (idle_source);
}

/* Main function for the main thread. */
static void
main (void)
{
	GThread *thread1;
	GMainContext *thread1_main_context;

	/* Spawn a background thread and pass it a reference to its
	 * GMainContext. Retain a reference for use in this thread
	 * too. */
	thread1_main_context = g_main_context_new ();
	g_thread_new ("thread1", thread1_main,
	              g_main_context_ref (thread1_main_context));

	/* Maybe set up your UI here, for example. */

	/* Invoke my_func() in the other thread. */
	invoke_my_func (thread1_main_context,
	                "some data which needs passing between threads",
	                123456, some_object);

	/* Continue doing other work. */
}

That’s a lot of code, and it doesn’t look fun. There are several points of note here:

  • This invocation is uni-directional: it calls my_func() in thread1, but there’s no way to get a return value back to the main thread. To do that, the same principle needs to be used again, invoking a callback function in the main thread. It’s a straightforward extension which isn’t covered here.
  • Thread safety: This is a vast topic, but the key principle is that data which is potentially accessed by multiple threads must have mutual exclusion enforced on those accesses using a mutex. What data is potentially accessed by multiple threads here? thread1_main_context, which is passed in the fork call to thread1_main; and some_object, a reference to which is passed in the data closure. Critically, GLib guarantees that GMainContext is thread safe, so sharing thread1_main_context between threads is fine. The other code in this example must ensure that some_object is thread safe too, but that’s a topic for another blog post. Note that some_string and some_int cannot be accessed from both threads, because copies of them are passed to thread1, rather than the originals. This is a standard technique for making cross-thread calls thread safe without requiring locking. It also avoids the problem of synchronising freeing some_string. Similarly, a reference to some_object is transferred to thread1, which works around the issue of synchronising destruction of the object.
  • Specificity: g_idle_source_new() was used rather than the simpler g_idle_add() so that the GMainContext the GSource is attached to could be specified.

With those principles and mechanisms in mind, let’s take a look at a convenience method which makes this a whole lot easier: g_main_context_invoke_full().((Why not g_main_context_invoke()? It doesn’t allow a GDestroyNotify function for the user data to be specified, limiting its use in the common case of passing data between threads.)) As stated in its documentation, it invokes a callback so that the specified GMainContext is owned during the invocation. In almost all cases, the context being owned is equivalent to it being run, and hence the function must be being invoked in the thread for which the specified context is the thread default.

Modifying the earlier example, the invoke_my_func() function can be replaced by the following:

static void
invoke_my_func (GMainContext *thread1_main_context,
                const gchar *some_string, guint some_int,
                GObject *some_object)
{
	MyFuncData *data;

	/* Create a data closure to pass all the desired variables
	 * between threads. */
	data = g_slice_new0 (MyFuncData);
	data->some_string = g_strdup (some_string);
	data->some_int = some_int;
	data->some_object = g_object_ref (some_object);

	/* Invoke the function. */
	g_main_context_invoke_full (thread1_main_context,
	                            G_PRIORITY_DEFAULT, my_func_idle,
	                            data,
	                            (GDestroyNotify) my_func_data_free);
}

That’s a bit simpler. Let’s consider what happens if invoke_my_func() were to be called from thread1, rather than from the main thread. With the original implementation, the idle source would be added to thread1’s context and dispatched on the context’s next iteration (assuming no pending dispatches with higher priorities). With the improved implementation, g_main_context_invoke_full() will notice that the specified context is already owned by the thread (or can be acquired by it), and will call my_func_idle() directly, rather than attaching a source to the context and delaying the invocation to the next context iteration. This subtle behaviour difference doesn’t matter in most cases, but is worth bearing in mind since it can affect blocking behaviour (i.e. invoke_my_func() would go from taking negligible time, to taking the same amount of time as my_func() before returning).

How can I be sure a function is always executed in the thread I expect? Since I’m now thinking about which thread each function could be called in, it would be useful to document this in the form of an assertion:

g_assert (g_main_context_is_owner (expected_main_context));

If that’s put at the top of each function, any assertion failure will highlight a case where a function has been called directly from the wrong thread. This technique was invaluable to me recently when writing code which used upwards of four threads with function invocations between all of them. It’s a whole lot easier to put the assertions in when initially writing the code than it is to debug the race conditions which easily result from a function being called in the wrong thread.

This can also be applied to signal emissions and callbacks. As well as documenting which contexts a signal or callback will be emitted in, assertions can be added to ensure that this is always the case. For example, instead of using the following when emitting a signal:

guint param1;  /* arbitrary example parameters */
gchar *param2;
guint retval = 0;

g_signal_emit_by_name (my_object, "some-signal",
                       param1, param2, &retval);

it would be better to use the following:

static guint
emit_some_signal (GObject *my_object, guint param1,
                  const gchar *param2)
{
	guint retval = 0;

	g_assert (g_main_context_is_owner (expected_main_context));

	g_signal_emit_by_name (my_object, "some-signal",
	                       param1, param2, &retval);

	return retval;
}

As well as asserting emission happens in the right context, this improves type safety. Bonus! Note that signal emission via g_signal_emit() is synchronous, and doesn’t involve a main context at all. As signals are a more advanced version of callbacks, this approach can be applied to those as well.

Before finishing, it’s worth mentioning GTask. This provides a slightly different approach to invoking functions in other threads, which is more suited to the case where you want your function to be executed in some background thread, but don’t care exactly which one. GTask will take a data closure, a function to execute, and provide ways to return the result from this function; and will then handle everything necessary to run that function in a thread belonging to some thread pool internal to GLib. Although, by combining g_main_context_invoke_full() and GTask, it should be possible to run a task in a specific context and effortlessly return its result to the current context:

/* This will be invoked in thread1. */
static gboolean
my_func_idle (gpointer user_data)
{
	GTask *task = G_TASK (user_data);
	MyFuncData *data;
	gboolean retval;

	/* Call my_func() and propagate its returned boolean to
	 * the main thread. */
	data = g_task_get_task_data (task);
	retval = my_func (data->some_string, data->some_int,
	                  data->some_object);
	g_task_return_boolean (task, retval);

	return G_SOURCE_REMOVE;
}

/* Whichever thread is invoked in, the @callback will be invoked in
 * once my_func() has finished and returned a result. */
static void
invoke_my_func_with_result (GMainContext *thread1_main_context,
                            const gchar *some_string, guint some_int,
                            GObject *some_object,
                            GAsyncReadyCallback callback,
                            gpointer user_data)
{
	MyFuncData *data;

	/* Create a data closure to pass all the desired variables
	 * between threads. */
	data = g_slice_new0 (MyFuncData);
	data->some_string = g_strdup (some_string);
	data->some_int = some_int;
	data->some_object = g_object_ref (some_object);

	/* Create a GTask to handle returning the result to the current
	 * thread default main context. */
	task = g_task_new (NULL, NULL, callback, user_data);
	g_task_set_task_data (task, data,
	                      (GDestroyNotify) my_func_data_free);

	/* Invoke the function. */
	g_main_context_invoke_full (thread1_main_context,
	                            G_PRIORITY_DEFAULT, my_func_idle,
	                            task,
	                            (GDestroyNotify) g_object_unref);
}

So in summary:

  • Use g_main_context_invoke_full() to invoke functions in other threads, under the assumption that every thread has a thread default main context which runs throughout the lifetime of that thread.
  • Use GTask if you only want to run a function in the background and don’t care about the specifics of which thread is used.
  • In any case, liberally use assertions to check which context is executing a function, and do this right from the start of a project.
  • Explicitly document contexts a function is expected to be called in, a callback will be invoked in, or a signal will be emitted in.
  • Beware of g_idle_add() and similar functions which use the global default main context.

by Philip Withnall at April 19, 2014 06:52 PM

April 16, 2014

Marco Barisione

Maynard: a Wayland desktop shell for the Raspberry Pi

In the last year or so, Collabora has been working with the Raspberry Pi Foundation on a web browser and on Wayland. See Daniel’s and Pekka’s blog posts about their Wayland work.

To make Wayland on the Raspberry Pi actually usable, we needed a shell, but lightweight desktop environments (like LXDE) don’t support Wayland and normal desktops (like Gnome and KDE) are just too heavy.
This meant we ended up writing our own shell based on Tiago Vignatti’s gtk-shell, so Maynard was born!

No video displayed here? Watch the video on Youtube.
Maynard running on my laptop (webm video file)

No video displayed here? Watch the video on Youtube.
Maynard running on a Pi (mp4 video file)

Maynard is far from complete, but it’s already starting to take shape nicely. Its goals are to be functional, light and pretty, so it will never see some of the features one might expect from Gnome or KDE for instance.

The main current limitations are:

  • No XWayland support, so non-Wayland applications cannot run (issue #1).
  • GTK applications take too long to start (issue #2).
  • Active apps are not shown in the panel (issue #3).
  • No configurability (issue #7). I hope you like the background from kdewallpapers we use as you cannot change it for now ;)

Interested in the project? Follow these links:

by barisione at April 16, 2014 12:37 PM

April 06, 2014

Philip Withnall

Rate limiting of asynchronous calls in Vala

Following on from single-thread synchronisation in Vala, I just used a similar technique to easily implement rate limiting of concurrent calls to the same function in Vala. The use case for this was bug #705742, caused by attempting to write out many avatars to libfolks’ on-disk avatar cache simultaneously. So many files were opened simultaneously that the process’ file descriptor limit was hit and subsequent writes failed.

The principle for fixing this is to maintain a counter of the number of ongoing operations. If this hits a limit, operations which are started subsequently are added to a work queue and yielded. Any ongoing operation which finishes will pop a yielded operation off the work queue (if it’s non-empty) and resume that operation. This forms a FIFO queue which is guaranteed to progress due to each completed operation causing the next to resume (which will, in turn, cause the next to resume, etc., until the queue is empty).

A code example:

using GLib;

public class AvatarCache : Object
{
  private uint _n_ongoing_stores = 0;
  private Queue<DelegateWrapper> _pending_stores =
      new Queue<DelegateWrapper> ();

  /* Change this to change the cap on the number of concurrent
   * operations. */
  private const uint _max_n_ongoing_stores = 10;

  public async string store_avatar (string id, LoadableIcon avatar)
      throws GLib.Error
    {
      string retval = "";

      /* If the number of ongoing operations is at the limit,
       * queue this operation and yield. */
      if (this._n_ongoing_stores >
          AvatarCache._max_n_ongoing_stores)
        {
          /* Add to the pending queue. */
          var wrapper = new DelegateWrapper ();
          wrapper.cb = store_avatar.callback;
          this._pending_stores.push_tail ((owned) wrapper);
          yield;
        }

      /* Do the actual store operation. */
      try
        {
          this._n_ongoing_stores++;
          retval = yield this._store_avatar_unlimited (id, avatar);
        }
      finally
        {
          this._n_ongoing_stores--;

          /* If there is a store operation pending, resume it,
           * FIFO-style. */
          var wrapper = this._pending_stores.pop_head ();
          if (wrapper != null)
            {
              wrapper.cb ();
            }
        }

      return retval;
    }

  private async string _store_avatar_unlimited (string id,
                                                LoadableIcon avatar)
      throws GLib.Error
    {
      return /* the actual computation goes here */;
    }
}

/* See:
 * https://mail.gnome.org/archives/vala-list/2011-June/msg00005.html */
[Compact]
private class DelegateWrapper
{
  public SourceFunc cb;
}

This is all done using Vala’s asynchronous operation support, so runs in a single thread using the global default main context, and no locking or thread synchronisation is needed. If you were to use AvatarCache.store_avatar() from multiple threads, locking would have to be added and things would become more complex.

As with the single-thread synchronisation example from before, the key lines are: wrapper.cb = store_avatar.callback; yield, which stores the current function pointer and its closure (in Vala terminology, a delegate with target for the current method); and wrapper.cb (), which calls that function pointer with the stored closure (in Vala terminology, executes the delegate), effectively resuming computation from the yield statement.

So that’s it: a way to rate-limit concurrent method calls in Vala, ensuring they all block correctly (i.e. calls which are waiting for earlier ones to complete continue to block until they themselves are resumed and complete computation). By changing the scheduling function applied to the GQueue, priorities can be applied to queued calls if desired.

by Philip Withnall at April 06, 2014 11:03 AM

April 02, 2014

Philip Withnall

What’s the difference between int and size_t?

Passing variable-sized buffers around is a common task in C, especially in networking code. Ignoring GLib convenience API like GBytes, GInputVector and GOutputVector, a buffer is simply an offset and length describing a block of memory. This is just a void* and int, right? What’s the problem?

tl;dr: I’m suggesting to use (uint8_t*, size_t) to describe buffers in C code, or (guint8*, gsize) if you’re using GLib. This article is aimed at those who are still getting to grips with C idioms. See also: (const gchar*) vs. (gchar*) and other memory management stories.

There are two problems here: this effectively describes an array of elements, but void* doesn’t describe the width of each element; and (assuming each element is a byte) an int isn’t wide enough to index every element in memory on modern systems.

But void* could be assumed to refer to an array of bytes, and an int is probably big enough for any reasonable situation, right? Probably, but not quite: a 32-bit signed integer can address 2 GiB (assuming negative indices are ignored) and an unsigned integer can address 4 GiB1. Fine for network processing, but not when handling large files.

How is size_t better? It’s defined as being big enough to refer to every addressable byte of memory in the current computer system (caveat: this means it’s architecture-specific and not suitable for direct use in network protocols or file formats). Better yet, it’s unsigned, so negative indices aren’t wasted.

What about the offset of the buffer — the void*? Better to use a uint8_t*, I think. This has two advantages: it explicitly defines the width of each element to be one byte; and it’s distinct from char* (it’s unsigned rather than signed2), which makes it more obvious that the data being handled is not necessarily human-readable or nul-terminated.

Why is it important to define the width of each element? This is a requirement of C: it’s impossible to do pointer arithmetic without knowing the size of an element, and hence the C standard forbids arithmetic on void* pointers, since void doesn’t have a width (C11 standard, §6.2.5¶¶19,1). So if you use void* as your offset type, your code will end up casting to uint8_t* as soon as an offset into the buffer is needed anyway.

A note about GLib: guint8 and uint8_t are equivalent, as are gsize and size_t — so if you’re using GLib, you may want to use those type aliases instead.

For a more detailed explanation with some further arguments, see this nice article about size_t and ptrdiff_t by Karpov Andrey.


  1. int doesn’t have a guaranteed width (C11 standard, §6.2.5¶5), which is another reason to use size_t. However, on any relevant modern platform it is at least 32 bits. 

  2. Technically, it’s architecture-dependent whether char is signed or unsigned (C11 standard, §6.2.5¶15), and whether it’s actually eight bits wide (though in practice it is really always eight bits wide), which is another reason to use uint8_t

by Philip Withnall at April 02, 2014 08:05 AM

March 27, 2014

Luis de Bethencourt

snappy has arrived to 1.0

snappy is an open source media player that gathers the power and flexibility of GStreamer inside the comfort of a minimalistic Clutter interface.

The snappy development team is proud to announce it's 1.0 release.
Codename: "I'll be back", Terminator

We think the project has achieved the maturity worthy of a 1.0 release. It does one thing and it does it well.



Some of the changes you will notice are:
  • It’s been given some needed visual polish
  • Playback speed adjustable
  • Video and audio synchronization tweeking
  • Time left of stream viewer
  • Better drag and drop
  • Better media history handling
  • More features accessible from Clutter interface
  • Bug fixes


Features already included from previous releases:
  • Subtitle support
  • Desktop launcher
  • Video and audio synchronization tweeking.
  • Multi-screen full-screen
  • Media queues
  • History of played media
  • Seeking/muting/cycling through languages (audio streams)
  • Frame stepping
  • Much more


Download a tarball: xz
Clone the git repo
Packages in distributions will be updated soon


Thanks to all who helped in snappy's creation!


Disclaimer: No moose were harmed during the making of this release. One got homesick and an other disappeared for days in a Breaking Bad marathon, but that's about it.

March 27, 2014 08:00 PM

Philip Withnall

What is GMainContext?

GMainContext is at the core of almost every GLib application, yet it was only recently that I took the time to fully explore it — the details of it have always been a mystery. Doing some I/O work required me to look a little closer and try to get my head around the ins and outs of GMainContext, GMainLoop and GSources. Here I’ll try and write down a bit of what I’ve learned. If you want to skip to the conclusion, there’s a list of key points for using GMainContexts in libraries at the bottom of the post.

What is GMainContext? It’s a generalised implementation of an event loop, useful for implementing polled file I/O or event-based widget systems (i.e. GTK+). If you don’t know what poll() does, read about that first, since GMainContext can’t be properly understood without understanding polled I/O. A GMainContext has a set of GSources which are ‘attached’ to it, each of which can be thought of as an expected event with an associated callback function which will be invoked when that event is received; or equivalently as a set of file descriptors (FDs) to check. An event could be a timeout or data being received on a socket, for example. One iteration of the event loop will:

  1. Prepare sources, determining if any of them are ready to dispatch immediately.
  2. Poll the sources, blocking the current thread until an event is received for one of the sources.
  3. Check which of the sources received an event (several could have).
  4. Dispatch callbacks from those sources.

This is explained very well in the GLib documentation.

At its core, GMainContext is just a poll() loop, with the preparation, check and dispatch stages of the loop corresponding to the normal preamble and postamble in a typical poll() loop implementation, such as listing 1 from http://www.linux-mag.com/id/357/. Typically, some complexity is needed in non-trivial poll()-using applications to track the lists of FDs which are being polled. Additionally, GMainContext adds a lot of useful functionality which vanilla poll() doesn’t support. Most importantly, it adds thread safety.

GMainContext is completely thread safe, meaning that a GSource can be created in one thread and attached to a GMainContext running in another thread. A typical use for this might be to allow worker threads to control which sockets are being listened to by a GMainContext in a central I/O thread. Each GMainContext is ‘acquired’ by a thread for each iteration it’s put through. Other threads cannot iterate a GMainContext without acquiring it, which guarantees that a GSource and its FDs will only be polled by one thread at once (since each GSource is attached to at most one GMainContext). A GMainContext can be swapped between threads across iterations, but this is expensive.

Why use GMainContext instead of poll()? Mostly for convenience, as it takes all the grunt work out of dynamically managing the array of FDs to pass to poll(), especially when operating over multiple threads. This is done by encapsulating FDs in GSources, which decide whether those FDs should be passed to the poll() call on each ‘prepare’ stage of the main context iteration.

So if that’s GMainContext, what’s GMainLoop? Ignoring reference counting and locking gubbins, it is essentially just the following three lines of code (in g_main_loop_run()):

loop->is_running = TRUE;
while (loop->is_running)
	g_main_context_iteration (context, TRUE);

Plus a fourth line in g_main_loop_quit() which sets loop->is_running = FALSE and which will cause the loop to terminate once the current main context iteration ends. i.e. GMainLoop is a convenient, thread-safe way of running a GMainContext to process events until a desired exit condition is met, at which point you call g_main_loop_quit(). Typically, in a UI program, this will be the user clicking ‘exit’. In a socket handling program, this might be the final socket closing.

It is important not to confuse main contexts with main loops. Main contexts do the bulk of the work: preparing source lists, waiting for events, and dispatching callbacks. A main loop just iterates a context.

One of the important features of GMainContext is its support for ‘default’ contexts. There are two levels of default context: the thread-default, and the global-default. The global-default (accessed using g_main_context_default()) is what’s run by GTK+ when you call gtk_main(). It’s also used for timeouts (g_timeout_add()) and idle callbacks (g_idle_add()) — these won’t be dispatched unless the default context is running!

What are the thread-default contexts then? These are a later addition to GLib (since version 2.22), and are generally used for I/O operations which need to run and dispatch callbacks in a thread. By calling g_main_context_push_thread_default() before starting an I/O operation, the thread-default context has been set, and the I/O operation can add its sources to that context. The context can then be run in a new main loop in an I/O thread, causing the callbacks to be dispatched on that thread’s stack rather than on the stack of the thread running the global-default main context. This allows I/O operations to be run entirely in a separate thread without explicitly passing a specific GMainContext pointer around everywhere.

Conversely, by starting a long-running operation with a specific thread-default context set, your code can guarantee that the operation’s callbacks will be emitted in that context, even if the operation itself runs in a worker thread. This is the principle behind GTask: when a new GTask is created, it stores a reference to the current thread-default context, and dispatches its completion callback in that context, even if the task itself is run using g_task_run_in_thread().

For example, the code below will run a GTask which performs two writes in parallel from a thread. The callbacks for the writes will be dispatched in the worker thread, whereas the callback from the task as a whole will be dispatched in the interesting context.

typedef struct {
	GMainLoop *main_loop;
	guint n_remaining;
} WriteData;

/* This is always called in the same thread as thread_cb() because
 * it’s always dispatched in the @worker_context. */
static void
write_cb (GObject *source_object, GAsyncResult *result,
          gpointer user_data)
{
	WriteData *data = user_data;
	GOutputStream *stream = G_OUTPUT_STREAM (source_object);
	GError *error = NULL;
	gssize len;

	/* Finish the write. */
	len = g_output_stream_write_finish (stream, result, &error);
	if (error != NULL) {
		g_error ("Error: %s", error->message);
		g_error_free (error);
	}

	/* Check whether all parallel operations have finished. */
	write_data->n_remaining--;

	if (write_data->n_remaining == 0) {
		g_main_loop_quit (write_data->main_loop);
	}
}

/* This is called in a new thread. */
static void
thread_cb (GTask *task, gpointer source_object, gpointer task_data,
           GCancellable *cancellable)
{
	/* These streams come from somewhere else in the program: */
	GOutputStream *output_stream1, *output_stream;
	GMainContext *worker_context;
	GBytes *data;
	const guint8 *buf;
	gsize len;

	/* Set up a worker context for the writes’ callbacks. */
	worker_context = g_main_context_new ();
	g_main_context_push_thread_default (worker_context);

	/* Set up the writes. */
	write_data.n_remaining = 2;
	write_data.main_loop = g_main_loop_new (worker_context, FALSE);

	data = g_task_get_task_data (task);
	buf = g_bytes_get_data (data, &len);

	g_output_stream_write_async (output_stream1, buf, len,
	                             G_PRIORITY_DEFAULT, NULL, write_cb,
	                             &write_data);
	g_output_stream_write_async (output_stream2, buf, len,
	                             G_PRIORITY_DEFAULT, NULL, write_cb,
	                             &write_data);

	/* Run the main loop until both writes have finished. */
	g_main_loop_run (write_data.main_loop);
	g_task_return_boolean (task, TRUE);  /* ignore errors */

	g_main_loop_unref (write_data.main_loop);

	g_main_context_pop_thread_default (worker_context);
	g_main_context_unref (worker_context);
}

/* This can be called from any thread. Its @callback will always be
 * dispatched in the thread which currently owns
 * @interesting_context. */
void
parallel_writes_async (GBytes *data,
                       GMainContext *interesting_context,
                       GCancellable *cancellable,
                       GAsyncReadyCallback callback,
                       gpointer user_data)
{
	GTask *task;

	task = g_task_new (NULL, cancellable, callback, user_data);
	g_task_set_task_data (task, data,
	                      (GDestroyNotify) g_bytes_unref);
	g_task_run_in_thread (task, thread_cb);
	g_object_unref (task);
}

From the work I’ve been doing recently with GMainContext, here are a few rules of thumb for using main contexts in libraries which I’m going to follow in future:

  • Never iterate a context you don’t own, including the global-default or thread-default contexts, or you can cause the user’s sources to be dispatched unexpectedly and cause re-entrancy problems.
  • Always remove GSources from a main context once you’re done with them, especially if that context may have been exposed to the user (e.g. as a thread-default). Otherwise the user may keep a reference to the main context and continue iterating it after your code expects it to have been destroyed, potentially causing unexpected source dispatches in your code.
  • If your API is designed to be used in threads, or in a context-aware fashion, always document which context callbacks will be dispatched in. For example, “callbacks will always be dispatched in the context which is the thread-default at the time of the object’s construction”. Users of your API need to know this information.
  • Use g_main_context_invoke() to ensure callbacks are dispatched in the right context. It’s much easier than manually using g_idle_source_new().
  • Libraries should never use g_main_context_default() (or, equivalently, pass NULL to a GMainContext-typed parameter). Always store and explicitly use a specific GMainContext, even if that reduces to being some default context. This makes your code easier to split out into threads in future, if needed, without causing hard-to-debug problems with callbacks being invoked in the wrong context.
  • Always write things asynchronously internally (using the amazing GTask where appropriate), and keep synchronous wrappers to the very top level, where they can be implemented by calling g_main_context_iteration() on a specific GMainContext. Again, this makes future refactoring easier. You can see it in the above example: the thread uses g_output_stream_write_async() rather than g_output_stream_write().
  • Always match pushes and pops of the thread-default main context.

In a future post, I hope to explain in detail what’s in a GSource, and how to implement one, plus do some more in-depth comparison of poll() and GMainContext. Any feedback or corrections are gratefully received!

by Philip Withnall at March 27, 2014 11:06 AM

March 24, 2014

Luis de Bethencourt

Resilience

"It always seems impossible until its done."
Nelson Mandela



My first Soyuz simulator! Summer 1964, nearly 5 years old."
Chris Hadfield

March 24, 2014 07:00 PM

March 19, 2014

Alban Crequy

Traffic control for multimedia devices


Multimedia devices traditionally don't manage the network bandwidth required by applications. This causes a problem when users try to watch a streaming video or listen to a web radio seamlessly while other applications are downloading other content in the background. The background downloads can use too much bandwidth for the streaming video or web radio to keep up and users notice unnecessary interruptions in the playback.

I have been working on an approach to improve this using traffic control on Linux. This work was sponsored by Collabora.

What is traffic control


Traffic control is a technique to control network traffic in order to optimise or guarantee performance, low-latency, and/or bandwidth. This includes deciding which packets to accept at what rate in an input interface and determining which packets to transmit in what order at what rate on an output interface. 

On Linux, applications can send the traffic control configuration to the kernel using a Netlink socket with  the NETLINK_ROUTE protocol. By default, traffic control on Linux consists of a single queue which collects entering packets and dequeues them as quickly as the underlying device can accept them. The tc tool (from the iproute2 package) or the more recent "nl-*" tools (part of libnl) are different implementations but they can both be used to configure traffic control. Libnl has an incomplete support for traffic control but is in active development and progressing quickly.

Difficulty of shaping ingress traffic


Traffic control and shaping comes in two forms, the control of packets being received by the system (ingress) and the control of packets being sent out by the system (egress). Shaping outgoing traffic is reasonably straight-forward, as the system is in direct control of the traffic sent out through its interfaces. Shaping incoming traffic is however much harder as the decision on which packets to sent over the medium is controlled by the sending side and can't be directly controlled by the system itself.

However, for multimedia devices, control over incoming traffic is far more important then controlling outgoing traffic. Our use-case is ensuring glitch-free playback of a media stream (e.g. internet radio). In such a case, essentially, a minimal amount of incoming bandwidth needs to be reserved for the media stream.

For shaping (or rather influencing or policing) incoming traffic, the only practical approach is to put a fake bottleneck in place on the local system and rely on TCP congestion control to adjust its rate to match the intended rate as enforced by this bottleneck. With such a system it's possible to, for example, implement a policy where traffic that is not important for the current media stream (background traffic) can be limited, leaving the remaining available bandwidth for the more critical streams.

On Linux, ingress traffic control ("ingress qdisc" on the graph) happens before the Netfilter subsystem:
Netfilter-packet-flow
By Jengelh (Own work, Origin SVG PNG) [CC-BY-SA-3.0], via Wikimedia Commons



Difficulty of shaping on mobile networks


However, to complicate matters further, in mobile systems which are connected wirelessly to the internet and have a tendency to move around it's not possible to know the total amount of available bandwidth at any specific time as it's constantly changing. Which means, a simple strategy of capping background traffic at a static limit simply can't work.

The implemented solution


To cope with the dynamic nature, a traffic control daemon (tcmmd) has been implemented which can dynamically update the kernel configuration to match the current needs of the playback applications and adapt to the current network conditions. Furthermore to address the issues mentioned above, the implementation will use the following strategy:

  • Split the traffic streams into critical traffic and background traffic. Police the incoming traffic by limiting the bandwidth available to background traffic with the goal of leaving enough bandwidth available for critical streams.
  • Instead of having static configuration, let applications (e.g. a media player) indicate when the current traffic rate is too low for their purposes. This both means the daemon doesn't have to actively measure the traffic rate and allows it cope with streams that don't have a constant bitrate more naturally.
Communication between the traffic control daemon and the applications is done via D-Bus. The  D-Bus interface allow applications to register critical streams by passing the standard 5-tuple (source ip and port, destination ip and port and protocol) which uniquely identify a stream and indicate when a particular stream bandwidth is too low.

To allow the daemon to effectively control the incoming traffic, a so-called Intermediate Functional Block device (ifb0) is used to provide a virtual network device to provide an artificial bottleneck. This is done by transparently redirecting the incoming traffic from the physical network device through the virtual network device and shape the traffic as it leaves the virtual device again. The reason for the traffic redirection is to allow the usage of the kernels egress traffic control to effectively be used on incoming traffic. The results in the example setup shown below (with eth0 being a physical interface and ifb0 the accompanying virtual interface).




To demonstrate the functionality as described above, a simple demonstration media application using Gstreamer (tcdemo) has been written that communicates with the Traffic control daemon in the manner described.

Testing, the set-up


The traffic control feature in tcdemo can be enabled or disabled on the command line. This allowed me to compare the behaviour in both cases. 

On my left, I have a web server serving both the files for a video stream and the files for background downloads. On my right, I have a multimedia device rendering a video stream while downloading other files on the same web server.

Traffic control is only useful when the available bandwidth is limited. In order to have meaningful tests, I simulated a low bandwidth with the following commands on the web server:
tc qdisc add dev wlan0 root handle 1: cbq avpkt 1000 bandwidth 10Mbit
tc class add dev wlan0 parent 1: classid 1:1 cbq rate 3Mbit allot 1500 prio 3 bounded isolated
tc filter add dev wlan0 parent 1: protocol ip u32 match ip protocol 6 0xff match ip sport 80 0xffff flowid 1:1

Only the traffic from port 80/http was limited. It is important to note that the background traffic and the stream traffic were both going through the same bottleneck.

Tcdemo was playing a video file streamed over http while 8 wgets were downloading the same file continuously. The 9 connections were competing for the limited bandwidth. Without traffic control, tcdemo would not have got enough bandwidth.

The following graph shows what happened with traffic control. The video streaming is composed of several phases:
  1. tcdemo opened the HTTP connection and its GStreamer pipeline started downloading. At the same time, tcmmd was notified there was a new stream connection and it restricted any potential background traffic to a very low limit. As long as the initial GStreamer queue was buffering, the background traffic limit did not change.
  2. The GStreamer queue became full at t=4s and the video started to be played on the screen. The daemon increased the limit on the background traffic exponentially and the stream bandwidth got reduced as a consequence.
  3. Despite the stream bandwidth degrading slowly, GStreamer managed to keep its queue over 75% full until t=25s. When the queue is more than 75% full, Gstreamer does not report it because tcdemo chose that threshold with the low-percent property on GstQueue2 (the graph shows 100% in this case). 
  4. At t=30s, the GStreamer queue was less than 70% full and that threshold triggered tcmmd to restrict the background traffic to its minimum.
  5. The stream could use most of the bandwidth and the GStreamer queue became full quickly at t=31s. The background traffic could start its exponential growth again.
traffic control stats
Thanks to traffic control, the GStreamer queue never got empty in my test.

Get the sources


git clone git://git.collabora.co.uk/git/user/alban/tcmmd
git clone https://github.com/alban/tcmmd

FAQ


Q: Do I need any privileges to run this?
A: No privileges required for tcdemo, the GStreamer application. But tcmmd needs CAP_NET_ADMIN to change the TC rules.

Q: The 5-tuple contains the TCP source port. How does the application know that number?
A: The application can either call bind(2) before connect(2) to choose a TCP source port, or call getsockname(2) after connect(2) to retrieve the TCP source port assigned automatically by the kernel. The former allows to install the traffic control rules before the call to connect(2) triggers the emission and reception of the first packets on the network. The latter means the first few packets will be exchanged without being shaped by the traffic control. Tcdemo implements the latter to avoid more invasive changes in the souphttpsrc GStreamer element and libsoup. See bgo#721807.

Q: What happens if an application forgets to unregister a 5-tuple when the video stream finishes?
A: That would be bad manners from the application. The current traffic control settings would remain.  And if the application notifies tcmmd that its buffer was empty and forgets to notify any changes, the background traffic would be severely throttled. However, if the application just terminates or crashes, tcmmd would notice it immediately on D-Bus and the traffic control rules would be removed.

Q: Does tcmmd remove its traffic control rules when terminated?
A: It depends how it is terminated. Tcmmd removes its traffic control rules on SIGINT and SIGTERM. But the rules remain in other cases (SIGSEGV, SIGKILL, etc.). If it is a problem in case of crash, tcmmd initialisation properly removes previous rules, so you could start tcmmd and interrupt it with ctrl-c.

Q: Instead of using the 5-tuple, why not using setsockopt-SO_MARK?
A: First, SO_MARK requires CAP_NET_ADMIN which is not something that media player should have. It could be worked around by fd-passing the socket to a more privileged daemon to call setsockopt-SO_MARK but it's not elegant. More importantly, tcmmd's goal is not to shape the egress traffic but the ingress traffic. The shaping of incoming packets is performed very early in the Linux network stack: it happens before Netfilter, and before the packet is associated to a socket. So we can't check the SO_MARK of a socket to shape incoming packets.

Q: Instead of using the 5-tuple, why not using cgroups?
A: The granularity of cgroups are only per-process. So the traffic control would not be able to distinguish between different HTTP connections in a web browser used to render a video stream and used for background downloads. And for the same reason as setsockopt-SO_MARK, it would not work for shaping ingress traffic: we would not be able to link the packet to any process or cgroup.

Q: Instead of sending the 5-tuple to tcmmd, why not set the IP type-of-service (TOS) on outgoing packets with setsockopt-SO_PRIORITY to avoid changes in the application and have an iptables target to feed that information about connections back to the ingress traffic control?
A: It could be possible if the bandwidth was fixed, but on mobile networks, the application needs to be changed anyway to give feedback when the queue in the GStreamer pipeline get emptied.

Q: Why not play with the TCP windows instead shaping the ingress traffic?
A: As far as I know, Linux does not have the infrastructure for that. The TCP windows to manipulate would not be from the GStreamer application but from all other connections, so it can't be done from userspace.

Q: Does tcdemo require any new feature in GStreamer?
A: Yes, souphttpsrc needs this patch: bgo#721807

Q: Does tcmmd require any new feature in the Linux kernel?
A: No.

Q: Does tcmmd work on several network interfaces (e.g. eth0 + wlan0)?
A: No, at the moment tcmmd only support one interface and it has to be started after the interface is up. Patches welcome!

Q: Tcmmd uses both libnl and /sbin/tc via system() calls. Why?
A: My goal is to use libnl and avoid spawning processes to call /sbin/tc. I just didn't have time to finish this. It will involve checking that libnl has the right features. Some needed features such as u32 action support were implemented recently in the last version.

Q: How did you get the graphs?
A: I used tcmmd's --save-stats option and the script tests/plot-tcmmd-log.sh.

Q: Why is there so frequent Netlink communication between tcmmd and the kernel?
A: One part of this is to gather regular statistics in order to generate graphs if the option --save-stats was used. The other part is for implementing the exponential progression of the bandwidth allocated to the background traffic: at regular interval, tcmmd changes the rate assigned to a qdisc. It could be avoided by implementing a specialised qdisc in the kernel for our use case. It would require more thinking how to design the API for that new qdisc.

Q: Does it work with IPv6?
A: No. The architecture is not specific to IPv4 but it is just not implemented yet for IPv6. Tcmmd would need to generate new TC rules because the IP headers are different between IPv4 and IPv6.


Thanks Sjoerd for the architecture diagram and proof-reading.

by Alban Crequy (noreply@blogger.com) at March 19, 2014 02:11 PM

March 14, 2014

Nick Richards

Some Computations

Sheep per capita in the UK (0.49), Australia (3.32), New Zealand (7.49) and Wales (10.2). All based on 2009 data.

by Nick Richards at March 14, 2014 04:09 PM

March 12, 2014

Neil McGovern

Entering the fray – 2014 DPL election

After a number of years in Debian, in various roles, I decided to put my name forward for election to Debian Project Leader. My platform has now been published, and campaigning is now under way!

Featured image is CC-BY by State Library of Victoria Collections

by Neil McGovern at March 12, 2014 12:42 PM