# Planet Collabora

## November 20, 2014

### Neil McGovern

#### Barbie the Debian Developer

Some people may have seen recently that the Barbie series has a rather sexist book out about Barbie the Computer Engineer. Fortunately, there’s a way to improve this by making your own version.

Thus, I made a short version about Barbie the Debian Developer and init system packager.

(For those who don’t know me, this is satirical. Any resemblance to people is purely coincidental.)

Edit: added text in alt tags. Also, hai reddit!

## November 19, 2014

### Simon McVittie

#### still aiming to be the universal operating system

Debian's latest round of angry mailing list threads have been about some combination of init systems, future direction and project governance. The details aren't particularly important here, and pretty much everything worthwhile in favour of or against each position has already been said several times, but I think this bit is important enough that it bears repeating: the reason I voted "we didn't need this General Resolution" ahead of the other options is that I hope we can continue to use our normal technical and decision-making processes to make Debian 8 the best possible OS distribution for everyone. That includes people who like systemd, people who dislike systemd, people who don't care either way and just want the OS to work, and everyone in between those extremes.

I think that works best when we do things, and least well when a lot of time and energy get diverted into talking about doing things. I've been trying to do my small part of the former by fixing some release-critical bugs so we can release Debian 8. Please join in, and remember to write good unblock requests so our hard-working release team can get through them in a finite time. I realise not everyone will agree with my idea of which bugs, which features and which combinations of packages are highest-priority; that's fine, there are plenty of bugs to go round!

Regarding init systems specifically, Debian 'jessie' currently works with at least systemd-sysv or sysvinit-core as pid 1 (probably also Upstart, but I haven't tried that) and I'm confident that Debian developers won't let either of those regress before it's released as Debian 8.

I expect the freeze for Debian 'stretch' (presumably Debian 9) to be a couple of years away, so it seems premature to say anything about what will or won't be supported there; that depends on what upstream developers do, and what Debian developers do, between now and then. What I can predict is that the components that get useful bug reports, active maintenance, thorough testing, careful review, and similar help from contributors will work better than the things that don't; so if you like a component and want it to be supported in Debian, you can help by, well, supporting it.

PS. If you want the Debian 8 installer to leave you running sysvinit as pid 1 after the first reboot, here's a suitable incantation to add to the kernel command-line in the installer's bootloader. This one certainly worked when KiBi asked for testing a few days ago:

preseed/late_command="in-target apt-get install -y sysvinit-core"


I think that corresponds to this line in a preseeding file, if you use those:

d-i preseed/late_command string in-target apt-get install -y sysvinit-core


A similar apt-get command, without the in-target prefix, should work on an installed system that already has systemd-sysv. Depending on other installed software, you might need to add systemd-shim to the command line too, but when I tried it, apt-get was able to work that out for itself.

If you use aptitude instead of apt-get, double-check what it will do before saying "yes" to this particular switchover: its heuristic for resolving conflicts seems to be rather more trigger-happy about removing packages than the one in apt-get.

## November 16, 2014

### Tollef Fog Heen

#### Resigning as a Debian systemd maintainer

Apparently, people care when you, as privileged person (white, male, long-time Debian Developer) throw in the towel because the amount of crap thrown your way just becomes too much. I guess that's good, both because it gives me a soap box for a short while, but also because if enough people talk about how poisonous the well that Debian is has become, we can fix it.

This morning, I resigned as a member of the systemd maintainer team. I then proceeded to leave the relevant IRC channels and announced this on twitter. The responses I've gotten have been almost all been heartwarming. People have generally been offering hugs, saying thanks for the work put into systemd in Debian and so on. I've greatly appreciated those (and I've been getting those before I resigned too, so this isn't just a response to that). I feel bad about leaving the rest of the team, they're a great bunch: competent, caring, funny, wonderful people. On the other hand, at some point I had to draw a line and say "no further".

Debian and its various maintainer teams are a bunch of tribes (with possibly Debian itself being a supertribe). Unlike many other situations, you can be part of multiple tribes. I'm still a member of the DSA tribe for instance. Leaving pkg-systemd means leaving one of my tribes. That hurts. It hurts even more because it feels like a forced exit rather than because I've lost interest or been distracted by other shiny things for long enough that you don't really feel like part of a tribe. That happened with me with debian-installer. It was my baby for a while (with a then quite small team), then a bunch of real life thing interfered and other people picked it up and ran with it and made it greater and more fantastic than before. I kinda lost touch, and while it's still dear to me, I no longer identify as part of the debian-boot tribe.

Now, how did I, standing stout and tall, get forced out of my tribe? I've been a DD for almost 14 years, I should be able to weather any storm, shouldn't I? It turns out that no, the mountain does get worn down by the rain. It's not a single hurtful comment here and there. There's a constant drum about this all being some sort of conspiracy and there are sometimes flares where people wish people involved in systemd would be run over by a bus or just accusations of incompetence.

Our code of conduct says, "assume good faith". If you ever find yourself not doing that, step back, breathe. See if there's a reasonable explanation for why somebody is saying something or behaving in a way that doesn't make sense to you. It might be as simple as your native tongue being English and their being something else.

If you do genuinely disagree with somebody (something which is entirely fine), try not to escalate, even if the stakes are high. Examples from the last year include talking about this as a war and talking about "increasingly bitter rear-guard battles". By using and accepting this terminology, we, as a project, poison ourselves. Sam Hartman puts this better than me:

I'm hoping that we can all take a few minutes to gain empathy for those who disagree with us. Then I'm hoping we can use that understanding to reassure them that they are valued and respected and their concerns considered even when we end up strongly disagreeing with them or valuing different things.

I'd be lying if I said I didn't ever feel the urge to demonise my opponents in discussions. That they're worse, as people, than I am. However, it is imperative to never give in to this, since doing that will diminish us as humans and make the entire project poorer. Civil disagreements with reasonable discussions lead to better technical outcomes, happier humans and a healthier projects.

## November 12, 2014

### Jeremy Whiting

#### KDE Fundraiser 2014

In case you didn't hear, KDE is running a fundraiser which helps fund sprints and other expenses of your favorite desktop/applications/games brought to you by the KDE community. Plus there are cool konqi postcards!

Ok, back to your regularly scheduled blog reading, have a nice day.

## November 10, 2014

### Gustavo Noronha Silva

#### Yay, the left won! Or did it?

Originally published on politi.kov

I have been asked by a bunch of friends from outside of Brazil for my opinion regarding the recent elections we had in Brazil, and it is a bit complicated to explain it without some background, so I decided to write this piece providing a bit of history so that people can understand my opinion.

The elections this year were a rematch of our traditional polarization between the workers party (PT) and the social democracy party (PSDB), which has been going on since 1994. PT and PSDB used to be allies. In the 80s, when the dictatorship dropped the law that forbade more than 2 parties, the opposition party, MDB, began breaking up in several smaller ones.

PSDB was founded by politicians and intelectuals who were inspired by Europe’s social democracy and political systems. Parliamentarism, for instance, is one of the historical causes of the party. The workers party had a more grassroots origin, with union leaders, marxist intelectuals and marxist-inspired catholic priests being the main founders. They drew their inspiration from the USSR and Cuba, and were very close to social movements.

Lula (PT) and FHC (PSDB) campaigning together in 1981, by Clóvis Cranchi Sobrinho

Some people have celebrated the reelection of Dilma Roussef as a victory of the left against the right. In my opinion that view is wrong for several reasons. First, because I disagree that PSDB and Aécio Neves in particular are right-wing, both in terms of economics and social/moral issues. Second, because I believe Dilma’s first government has taken a quite severe turn to the right in several topics that matter a lot to me. Since comparisons with PSDB’s government during the 90s has been one of the main strategies of the campaign this year, I’ll argue why I think it was actually a pretty good government with a lot of left in it.

Unlike what happens in most other places, Brazil does not really have an actual right-wing party, economics-wise. Although we might see the birth of a couple in the near future, no current party is really against public health, education and social security being provided by the state as rights, or wants to decrease state size and lower taxes significantly. It should come as no surprise that even though it has undergone a lot of liberal reforms over the last 20 years, Brazil is still a very closed country, with very high import tariffs and a huge presence of the state in the economy. There is a certain consensus about all of that, with disagreements being essentially on implementation details, not goals.

On the other hand, and contrary to popular belief, when it comes to social and moral issues we are a very conservative people. Ironically, the two parties which have been in power over the last 20 years are quite progressive, being historically proponents of diversity, minorities rights, reproductive rights. They have had to compromise on those causes to become viable alternatives, given the conservative nature of the majority of the voters.

Despite their different origins and beliefs, both parties share socialist inclinations and were allies from the onset. That changed in 1992, when president Collor, who had been elected on a runoff against Lula (who PSDB supported), was impeached by Congress for corruption. With no formal political support and a chaotic situation in his hands, Itamar Franco, the vice president, called for a “national union” government to go through the last two years of his term. PSDB answered the call, but the workers party decided against being part of the government.

Fernando Henrique Cardoso, a sociologist who was one of the leaders of PSDB was chosen to lead the Foreign Relations Ministry, but a few months later got nominated to the Economy. At the time, Brazil lived under hyperinflation of close to 1000% a year, and several stabilization plans had been attempted. Economy Ministers did not last very much in office at the time. FHC gathered a team of economists and sponsored their stabilization plan, which turned out to be highly successful: the Plano Real (“Real Plan”). In addition to introducing a new currency, something that was becoming pretty common to Brazilians by then, it also attacked the structural causes of inflation.

Lula was counting on the failure of the Plano Real when he ran against FHC in 1994, but the plan succeeded, giving FHC two terms as president. During those two terms, FHC introduced several institutional changes that made Brazil a saner country. In addition to the hyperinflation, Brazil had lived a debt crises for decades and was still in default. FHC’s team renegotiated the debts, reopened lines of credit, but most importantly, introduced reforms that made the Brazilian finances and financial system credible.

The problem was not even that Brazil had a fiscal déficit, it just did not have any control whatsoever of money supply and budget. Banks, regardless of whether they were private or public, had very little regulation and took advantage of the hyperinflation to hide monstrous holes in their balances. When inflation was gone and regulation became more strict, those became apparent, and it was pretty clear that the system would collapse if nothing was done.

Some people like to say that FHC was a president who ruled for the rich and didn’t care about the poor. I think the way the potential collapse of the banking system was handled is a great counter-example of that. The government passed laws that made the owners of the banks responsible for the financial problems, regardless of whether caused by mismanagement or fraud. If a bank went under, the central bank intervened and added enough money to protect the deposits, but that money was a loan that had to be repaid by the owners of the bank, and the owners’ properties were added as collateral to the loan. As a brazilian journalist once said, the people did not risk losing their deposits, the bankers did risk losing the banks, though. Today, we have a separate fund, filled with money from the banks, that does what the central bank did back then when required.

Compare that to countries where the banking system was saved with tax payer money and executives kept getting huge bonuses regardless, while owners kept their profits. It is hard to find an initiative that is more focused on the public interest against the interest of the rich people who caused the problem. This legislation, called PROER, is still in place today, and it came along with solid regulation of the banking system. It should come as no surprise that Brazil went through the financial crisis of 2008 with not a single hiccup of the banking system and no fear of bank runs. Despite having been against PROER back in the day, Lula celebrated its existence in 2008, when it was clear it was one of the reasons we would not suffer much. He even advertised it as something that should be adopted by the US and Europe.

It is also pretty common to hear that under FHC social questions were not a priority. I believe it is pretty simple to see that that was not the case both by inspecting the growth of social spending and the improvement of social indicators for the period, such as UN’s human development index. One area in which people are particularly critical of the FHC government is the investment on higher education, and they are actually quite right. Brazil has free Federal universities and those did not get a lot of priority in the 90s. However, I would argue that while it is a matter of priorities, it is not one of education versus something else, but rather of what to invest on inside education. The reality is basic education was the priority.

When FHC came to power, Brazil had a significant number of children who were not going to school at all. The goal was to make access to schools universal for young children, and that goal was reached. Every child has been going to school since the early 2000s, and that is a significant achievement which reaches the poorest. While the federal universities are attended essentially by the Brazilian elite, given the difficulty of passing the exams and the relative lack of quality of free public schools compared to private ones, which is still a reality to this day, investment on getting children to even go to school for the early years has a significant impact on the lives of the poorest.

It is important to remember that getting every child to go to school is also what gave birth to one of the most celebrated programs from the Lula era: Bolsa Família (“Family Allowance”) is a direct money transfer to poor families, particularly those who have children and has been an important contribution to lowering inequality and getting people out of extreme poverty. To get the money, the families need to ensure their children are 1) attending school and 2) getting vaccinated.

That program comes from the FHC government, in which it was created with the name Bolsa Escola (“School Allowance”), in its turn inspired by a program of the same name by governor Cristovam Buarque, from PT. What Lula did, and he deserves a lot of credit for this, was to merge a series of smaller programs with Bolsa Escola, and then expand the program to ensure it got to more and more people. Interestingly, during the announcement of the program he credited the idea of doing that to a state governor from PSDB. You can see why I think these two should be allies again.

When faced with all these arguments, people will eventually say that FHC was bad because he privatized companies and used orthodox economic policies. Well, if that is what it takes, then we’ll have to take Lula down with him, because his first term was essentially a continuation of FHC’s second term: orthodox economic policies to keep inflation down, along with privatization of several state-owned companies and banks. But Lula, whom I voted for and whose government I believe was a good one, is not my subject: Dilma is.

On Lula’s second term, Dilma gained a lot of power when other major leaders of PT went down for corruption. She became second in command and started leading several programs. A big believer in developmentalism, she started pushing for a bigger role of the state in the coordination of the productive sector, with a clear focus on growing the industrial base.

One of the initiatives she sponsored was a sizable increase on the number and size of subsidized loans given out by the national development bank (BNDES). Brazil started an unnofficial “national champions” program, where the government elected a few big companies to get a huge amount of subsidized credit.

The goal was for these selected firms to get big enough to be competitive on the global market. The criteria for the choices is completely opaque, if it even exists, and includes handing out milions in subsidized credit for Eike Batista, who became Brazil’s richest enterpreneur for a while, and lost pretty much everything when it became clear the oil would not be pumping out of his camps after all, sinking with them a huge amount of public funds invested by BNDES.

The way this policy was enacted, it is unclear how much it really costs in terms of public funds: the Brazilian treasury emits debt to capitalize, lends that money to BNDES with higher than market interest, and BNDES then lends it out to the big companies with a lower than market interest rate. Although it is obviously unsustainable, the problem does not yet show in the balance because the grace period for BNDES debt with the treasury is 2040. The fact that this has a cost and, perhaps more importantly, a huge opportunity cost is not clear because it is not part of the government budget. Why are we putting money in this rather than quadrupling Bolsa Família, which studies show generates 1,78 reais in GDP for every 1 real invested? Worse, why are we not even updating Bolsa Família enough to cover inflation?

When Dilma got elected in 2010, the first signs were pretty bad. She was already seen as someone who did not care much for the environment, and on her first month in power she made good on that promise by pushing to get the Belo Monte Dam building started as soon as possible regardless of conditionalities being satisfied. To this day there are several issues with how the building of the dam is going: the handling of the indigenous people and the small city nearby are lacking, conditionalities are not met.

Beyond Belo Monte, indigenous leaders are being assassinated, deforestation in the Amazon forest has increased by 122% in 2014 alone. Dilma’s answer to people who question her on these kinds of issues is essentially: “would you rather not have electric power?”

Her populist authoritarian nature and obsession with industry are also pretty evident when it comes to her policies in the energy area as a whole. She showed up in national tv on the eve of our independence day celebration to announce a reduction in electric tariffs, mainly for industry, but also for homes. Nobody really knew how. The following week she sent a fast-track project to Congress to automatically renew concessions of power grid operators, requiring those who accepted it to lower tariffs, instead of doing an auction, which was already necessary anyway because the concessions were up on 2015. There was no discussion with stakeholders, there was just a populist announcement and a great deal of rhethoric to paint anyone who opposed as being against the people.

And now, everything went into the crapper because that represented a breach of contract that required indemnification, and we had a pretty bad drought that made power more expensive given the need to turn on the thermal generators. Combining the costs of the thermal generation, indemnity, and financial fallout that the grid operators suffered, we are already at 105 billion reais and counting, nobody knows how high the cost will reach. Any reduction in tariffs has long been invalidated. And the fact that industry has lowered production significantly ends up being good news, we would probably be under rationing already if that was not the case.

You would expect someone who fought a dictatorship to be pretty good in terms of human and civil rights. What we see in reality is a lack of respect for those things. During the world cup, Dilma has put the army on the streets and has supported arbitrary behaviour from state polices throughout the country. They jailed a bunch of demonstrators preemptively. No shit. The would be demonstrators were kept in jail throghout the tournament under false accusations. Dilma’s Minister of Justice said several times that the case against them was solid and that the arrests were legal, but it turned out the case simply did not exist. Just this week we had a number of executions orchestrated by policemen in the state of Pará and there is zero reaction from the federal government.

In the oil industry, Dilma has enacted a policy of subsidizing gas prices by using a fixed price that used to be lower than the international prices (it is no longer the case with the fall in international prices). That would not be a problem if Brazil was selfsufficient in oild and gas, which we are not: we had to import a significant amount of both. The implicit subsidy cost Petrobrás a huge amount of cash – the more gas it sold, the bigger the losses. This lead not only to decreasing the company’s market value (it is a state-controlled, but open company), but to reducing its capacity of investment as well.

That is more problematic than it sounds because, with our current concession model, every single oil camp needs to have Petrobrás as a member of the consortium. Limiting the company’s investment capacity limits the rate at which our pre-salt oil camps can be explored and thus the speed at which we can become selfsufficient. Chicken and egg anyone?

To make things worse, Dilma has made policies that lowered taxes on car production, used to foster economic activity during the crisis in 2008-2010, essentially permanent. This lead to a significant increase in traffic and polution on Brazilian cities, while at the same time increasing the pressure on Petrobrás, which had to import more and more gas. Meanwhile, Brazilian cities suffer from a severe lack of mobility infrastructure. A recent study has shown that Brazil has spend almost twice as much subsidized money on pro-car policies than on pro-mass transit projects. Talk about good usage of public funds.

One of the only remaining good news the government was still able to mention was the constant reduction in extreme poverty. Dilma was actually ellected promising to erradicate extreme poverty and changed the government’s slogan to “A rich country is a country with no poverty” (País rico é país sem pobreza). Well, it turns out all of these policies caused inequality and extreme poverty both to stop falling as of 2013. And given the policies were actually deepened in 2014, I believe it is very likely we’ll see an increase in both when we get the data for 2014, next year.

Other than that, her policies ended up being a complete failure. Despite giving tax benefits to several sectors, investment has fallen, growth has fallen and inflation is quite high at 6,6% for the last 12 months. In terms of minorities, her government has been a severe set back, with the government going back on educational material against homophoby saying it would not do “advertisement of sexual choice”, and going back on a decree that allowed the public health system to perform abortions on the cases allowed by the law (essentially if the woman has been raped).

Looking at Dilma’s policies, I really can’t see that much of the left, honestly. So why, you might ask, has this victory been deemed a victory of the left over the right? My explanation is the aura the workers party still manages to keep over itself. There’s a notion that whatever PT does, it will still be more to the left than PSDB, which I think is just crazy.

There is also a fair amount of idealizing Dilma just because she is Lula’s protegé. People will forgive anything, provided it is the workers party doing it. Thankfully, the number of people aligned on the left that supported the candidate from PSDB this election tells me this is changing quite rapidly. Hopefully that leads to PT having to reinvent itself, and get in touch with the left again.

## November 09, 2014

### Jeremy Whiting

#### Autostart in Plasma 5

In the past few days as I got more and more annoyed that git pushes and svn updates and commits were always asking for my ssh key password I looked at what the current status of plasma autostart is. When I set up my plasma 5 environment I copied all the scripts that were in ~/.kde/Autostart to ~/.config/autostart thinking they would "just work" there, but it turns out they don't. My needs are simple I only had two .sh bash scripts in there, one for running ksshaskpass to ssh-add my ssh key. The other to launch synergys on login. Both scripts showed up in the autostart kcm, but neither was getting launched at login time.

Digging a bit in the code of kinit and klauncher I found that when it looks at ~/.config/autostart it only looks for .desktop files. It turns out this comes from an xdg spec for autostart scripts. Ok, simple enough, so I removed my synergys.sh script and in the autostart kcm created a new autostart item that launches synergys for me at login time. Works like a charm.

Next I went to look at ssh-add using ksshaskpass (there's a kf5 port of ksshaskpass in kdereview now by the way, which works pretty well and has some small improvements from the kdelibs4 version). For one thing, files that aren't .desktop files in ~/.config/autostart aren't getting launched, because klauncher doesn't even look at them. To fix this the autostart kcm was modified to not show these anymore.

The other question then is where should plain bash scripts be put that users and sysadmins used to put into ~/.kde/Autostart ? One solution is to put them into ~/.config/plasma-workspace/env/ . This is what I did here with ssh-add.sh and that works fine. The only thing to note though is that these are sourced, not executed, and they are sourced before plasma has started, so they can set environment variables and such if needed. This may not be ideal for scripts that need to talk to running services, so we may need to add something to ksmserver to launch scripts from ~/.config/ksm-autostart or something like that. There's a bug about it here https://bugs.kde.org/show_bug.cgi?id=338242 already, so don't file new bugs about this issue.

## October 31, 2014

### Philip Withnall

#### Introduction to ICE and libnice

As part of the series of tea time talks we do within Collabora, I recently got to refresh my knowledge of STUN, TURN and ICE (the protocols for NAT traversal) and give an introductory talk on how they all fit together within the context of libnice.

Since the talk might be useful (and perhaps even interesting) to a wider audience, I’ve made it available: slides, handout and source (git). It’s under CC-BY-SA 4.0. Please leave comments if anything is unclear, incorrect, or could do with more in-depth coverage!

#### Recent improvements in libnice

For the past several months, Olivier Crête and I have been working on a project using libnice at Collabora, which is now coming to a close. Through the project we’ve managed to add a number of large, new features to libnice, and implement hundreds (no exaggeration) of cleanups and bug fixes. All of this work was done upstream, and is available in libnice 0.1.8, released recently! GLib has also gained a number of networking fixes, API additions and documentation improvements.

tl;dr: libnice now has a GIOStream implementation, support for scatter–gather send and receive, and more mature pseudo-TCP support — so its API should be much nicer to integrate; GLib has gained a number of fixes.

Firstly, what is libnice? It’s a GLib implementation of ICE, the standard protocol for NAT traversal. Briefly, NAT traversal is needed when two hosts want to communicate peer-to-peer in a network where there is at least one NAT translator between them, meaning that at least one of the hosts cannot directly address the other until a mapping is created in the NAT translator. This is a very common situation (due to the shortage of IPv4 addresses, and the consequence that most home routers act as NAT translators) and affects virtually all peer-to-peer communications. It’s well covered in the literature, and the rest of this post will assume a basic understanding of NAT and ICE, a topic about which I recently gave a talk.

Conceptually, libnice exists just to create a reliable (TCP-like) or unreliable (UDP-like) socket which connects your host with a remote one in a manner that traverses any intervening NATs. At its core, it is effectively an implementation of send(), recv(), and some ancillary functions to negotiate the ICE stream at startup time.

The biggest change is the addition of nice_agent_get_io_stream(), and the GIOStream subclass it returns. This allows reliable ICE streams to be used via GIOStream, with all the API sugar which comes with GIO streams — for example, g_output_stream_splice(). Unreliable (UDP-like) ICE streams can’t be used this way because they’re not technically streams.

Highly related, the original receive API has been augmented with scatter–gather support in the form of a recvmmsg()-like API: nice_agent_recv_messages(). Along with appropriate improvements to libnice’s underlying socket implementations (the most obscure of which are still to be plumbed in), this allows performance improvements by batching messages, reducing the number of system calls needed for communication. Furthermore (perhaps more importantly) it reduces memory copies when assembling and parsing packets, by allowing the packets to be split across multiple non-contiguous buffers. This is a well-studied and long-known performance technique in networking, and it’s nice that libnice now supports it.

So, if you have an ICE connection (stream 1 on agent, with 2 components) exchanging packets with 20B headers and variable-length payloads, instead of:

nice_agent_attach_recv (agent, 1, 1, main_context, recv_cb, NULL);
nice_agent_attach_recv (agent, 1, 2, main_context, recv_cb, NULL);

…

static void
recv_cb (NiceAgent *agent, guint stream_id, guint component_id,
guint len, const gchar *buf, gpointer user_data)
{
if (stream_id != 1 ||
(component_id != 1 && component_id != 2)) {
g_assert_not_reached ();
}

if (component_id == 1)
parse_component1_data (buf + 20, len - 20);
else
parse_component2_data (buf + 20, len - 20);
}
}

…

static void
send_to_component (guint component_id,
const gchar *data_buf, gsize data_len)
{
gsize len = 20 + data_len;
guint8 *buf = malloc (len);

memcpy (buf + 20, data, data_len);

if (nice_agent_send (agent, 1, component_id,
len, buf) != len) {
/* Handle the error */
}
}

you can now do:

/* Only set up 1 NiceInputMessage as an illustration. */

static guint8 buf1_1[20];  /* header */
static guint8 buf1_2[1024];  /* payload size limit */
static GInputVector buffers1[2] = {
{ &buf1_1, sizeof (buf1_1) },  /* header */
{ &buf1_2, sizeof (buf1_2) },  /* payload */
};
static NiceInputMessage messages[1] = {
buffers1, G_N_ELEMENTS (buffers1),
NULL, 0
};
GError *error = NULL;

n_messages = nice_agent_recv_messages (agent, 1, 1, &messages,
G_N_ELEMENTS (messages),
NULL, &error);
if (n_messages == 0 || error != NULL) {
/* Handle the EOS or error. */
if (error != NULL)
g_error ("Error: %s", error->message);
return;
}

/* Component 2 can be handled similarly and code paths combined. */
for (i = 0; i &lt; n_messages; i++) {
NiceInputMessage *message = &messages[i];

parse_component1_data (message->buffers[1].buffer,
message->buffers[1].size);
}
}

…

static void
send_to_component (guint component_id, const gchar *data_buf,
gsize data_len)
{
GError *error = NULL;
GOutputVector vec[2] = {
{ data_buf, data_len },
};
NiceOutputMessage message = { vec, G_N_ELEMENTS (vec) };

if (nice_agent_send_messages_nonblocking (agent, 1, component_id,
&message, 1, NULL,
&error) != 1) {
/* Handle the error */
g_error ("Error: %s", error->message);
}
}

libnice has also gained non-blocking variants of its I/O functions. Previously, one had to explicitly attach a libnice stream to a GMainContext to start receiving packets. Packets would be delivered individually via a callback function (set with nice_agent_attach_recv()), which was inefficient and made for awkward control flow. Now, the non-blocking I/O functions can be used with a custom GSource from g_pollable_input_stream_create_source() to allow for more flexible reception of packets using the more standard GLib pattern of attaching a GSource to the GMainContext and in its callback, calling g_pollable_input_stream_read_nonblocking() until all pending packets have been read. libnice’s internal timers (used for retransmit timeouts, etc.) are automatically added to the GMainContext passed into nice_agent_new() at construction time, which you must run all the time as before.

GIOStream *stream = nice_agent_get_io_stream (agent, 1, 1);
GInputStream *istream;
GPollableInputStream *pollable_istream;

istream = g_io_stream_get_input_stream (stream);
pollable_istream = G_POLLABLE_INPUT_STREAM ();

source = g_pollable_input_stream_create_source (pollable_istream, NULL);
g_source_attach (main_context, source);

static gboolean
{
GPollableInputStream *pollable_istream = user_data;
GError *error = NULL;
guint8 buf[1024];  /* whatever the maximum packet size is */

/* Read packets until the queue is empty. */
while ( (len = g_pollable_input_stream_read_nonblocking (pollable_istream,
buf, sizeof (buf),
NULL,
&error) ) > 0) {
/* Do something with the received packet. */
}

if (error != NULL) {
/* Handle the error. */
}
}

libnice also gained much-improved support for restarting individual streams using ICE restarts with the addition of nice_agent_restart_stream(), switching TURN relays with nice_agent_forget_relays(), plus a number of bug fixes.

Finally, FIN/ACK support has been added to libnice’s pseudo-TCP implementation. The code was originally based on Google’s libjingle pseudo-TCP, establishing a reliable connection over UDP by encapsulating TCP-like packets within UDP. This implemented the basics of TCP, but left things like the closing FIN/ACK handshake to higher-level protocols. Fine for Google, but not for our use case, so we added support for that. Furthermore, we needed to layer TLS over a pseudo-TCP connection using GTlsConnection, which required implementing half-duplex close support and fixing a few nasty leaks in GTlsConnection.

Thanks to the libnice community for testing out the changes, and thanks to the GLib developers for patiently reviewing the stream of tiny documentation fixes and several larger GLib patches! All of the libnice API changes are shown on the handy upstream-tracker.org tool.

## October 28, 2014

### Jeremy Whiting

#### Accessibility is alive (QtSpeech progress, Jovie's deprecation)

For some time I've been considering what to do about Jovie which was previously known as ktts (KDE Text To Speech). Since before the first KDE Frameworks release actually, since kdelibs used to host a dbus interface definition for the KSpeech dbus interface that ktts and then Jovie implemented. I have a qt5 frameworks branch of Jovie, but it didn't make much sense to port it, since a lot of it is or could become part of the upcoming QtSpeech module. So Jovie has no official qt5 port and wont be getting one either.

What will Okular, KNotify, and other applications that want to speak to users do instead? The answer is QtSpeech. QtSpeech is a project started by Frederik Gladhorn to bring speech api's to all the platforms that Qt supports. It is still in its infancy, but is quickly improving. A few weeks ago when I built my kf5 stack with kdesrc-build I noticed that kdepim(libs?) was depending on it and it hasn't been released yet, so I got motivated to send some improvements to qt-project. Frederik and Laurent Montel have been pushing fixes and improving it also. It is as easy if not easier to use than the KSpeech dbus api (and doesn't require dbus either) and can be used to speak text on linux/unix, osx, windows, and android platforms so far. If you are an expert on any of these platforms please send patches to implement api on these platforms in their backends, the more eyes on this project the faster we can get it solidified and released.

You may be asking but what about feature X in Jovie that I will miss desperately. Yes there are a few things that QtSpeech will not do that Jovie did. These will either need to be done in individual applications or we can create a small framework to add these features (or possibly add them to QtSpeech itself if they make sense there). The features I'm thinking of are:

1. Filtering - Changing ": Hey QtSpeech is really coming along now" to "jpwhiting says 'Hey QtSpeech is really coming along now'" for KNotifications and the like. This could likely be implemented easily in knotify itself and exposed in the notifications configuration dialog.
2. Voice switching - Changing which voice to use based on the text, or the application it is coming from or anything else. This might make sense in QtSpeech itself, but time will tell if it's a wanted/needed feature.
3. User configuration - Jovie had a decent (not ideal, but it was functional) ui to set some voice preferences, such as which voice you wanted to use, which pitch, volume, speed, gender, etc. This will become the only part of Jovie that will get ported, which is a KDE Control Module for speech-dispatcher settings. This may also change over time, as speech-dispatcher itself possibly grows a ui for it's settings.

All in all, progress is being made. I expect QtSpeech to be ready for release with Qt 5.5, but we'll see what happens.

### Jonny Lamb

#### Sciopero

Public transport strikes in Rome are so frequent that it’s hard to remember when they are. I wrote a Gnome Shell extension to help remind me when there’s one either coming up or in progress. Find it on extensions.gnome.org. It gets its data from another little service I just made.

A Roma gli scioperi dei mezzi pubblici sono così frequenti che spesso è facile dimenticarsi quando ci sono. Ho scritto un’estensione per Gnome Shell per avvisare quando c’è o si avvicina uno sciopero dell’Atac. La puoi trovare su extensions.gnome.org. Funziona grazie ad un altro piccolo servizio che ho creato.

## October 18, 2014

### Jeremy Whiting

#### Upcoming KDE Applications 14.12 release prep

Hello,

In preparing for the upcoming releases of KDE Applications 14.12 (2014 Month 12) I realized the other day that we have an interesting situation.  For Qt4 based applications there's libkdeedu which contains the kvtml parsing and manipulating code and also a handful of .kvtml files that KAnagram and KHangMan use to get their word lists. KAnagram has been ported to Qt5 and KDE Frameworks for some time now, and will have it's first Qt 5 based release at the end of this year. It uses libkeduvocdocument which was ported to Qt 5 also at about the same time this year. libkeduvocdocument uses Qt 5 and KDE Frameworks 5, and also ships the same handful of kvtml files that libkdeedu ships. (libkdeedu has been split for the Qt 5 based releases) KHangMan hasn't yet been ported to frameworks and Qt5 or at least the port isn't stable yet, so it will depend on libkdeedu still, as will KWordQuiz and Parley from what I understand. So we have two libraries that ship the same files, makes them not coinstallable. So we'll be moving the kvtml files out of the libraries and into kdeedu-data soonish to solve this problem. The moral of this story is to look around, see what will be released using Qt5 in the upcoming release, and what will be using Qt4 still. https://community.kde.org/Frameworks/Application-release-status-December-2014 may help also. If you maintain an application and haven't put your application on that page under the Qt4 or Qt5 tables yet, please do, the more we coordinate, the better this release will be.

Thanks, and keep up the good work all.

## October 11, 2014

### Jeremy Whiting

#### Simple Elegance

I just noticed a couple of features today and yesterday in plasma next and kwin that I appreciate and wanted to thank whoever thought of adding them. Both are simple but very handy to have. I'm talking about the little X buttons on both the wallpaper configuration dialog and the kwin present windows effect. I don't use either of these features very often, but yesterday when I was testing knewstuff with the wallpaper config it was very handy to be able to delete the installed wallpaper from the wallpaper selection dialog. Then just now it was very handy to be able to close extra windows I had open that I no longer need when I was in the present windows effect looking at what I need to be doing next. Makes it very simple to clean up a workspace.

Just throwing this out there, thanks whoever added these simple nice features.

## October 07, 2014

### Jeremy Whiting

#### Plasma Next Improvements and KApplication -> QApplication gotchas

tldr, if you port from KApplication to QApplication, remove the %i from the Exec lines of your .desktop files.

Hey all, so I've been running plasma-next on my main development machine for a month or two now and have definitely enjoyed the speed at which improvements come. Just in time for plasma 5.1 release timezone support was added to the digital clock, so you can choose which timezones you want to see, use your mouse scrollwheel to change which one is visible, and so on. Also because our translators are awesome we got a last minute change in after the string freeze with their permission to show all the configured timezones in the clock's tooltip (similar but not identical to how it worked in kdelibs4 based plasma times). I enjoy the new Breeze theme and icons, and the alternatives system for switching between different k-menus, different task managers (I'm currently enjoying the icons only one) etc. is so handy.

In other news something that has been bothering me for a while is that okular just didn't want to get launched from any visual launcher. Clicking on a pdf in dolphin acted like it was launching, but no Okular ui would appear. Clicking view book on pdf books in Calibre would do likewise. So I spent a couple of days adding debug messages to kinit and klauncher trying to figure out what was going on. Kate launches just fine, so I tried copying the kate.desktop to okularApplication_pdf.desktop and replaced kate with okular etc. and that worked fine also.

So today I asked Albert if he had any ideas and got thinking it had to be something in the .desktop files themselves. So I uncommented another qDebug line in klauncher that said exactly what it was asking kinit to start and found it using "/usr/local/bin/okular blah.pdf --icon okular. So I tried the same from a terminal and found that okular's binary failed to launch because it doesn't understand the --icon parameter. A bit of digging found that KApplication handled that argument, while QApplication doesn't and the frameworks port of okular like a good example ported from KApplication to QApplication already.
Klauncher puts --icon blah in when you have %i in the Exec line of the .desktop file. So if you port from KApplication to QApplication, be sure to remove the %i from the .desktop files of your application also.

## October 06, 2014

### Alban Crequy

#### Improving the security of D-Bus

In the last months, I have been working on improving the security of D-Bus, mainly to make it more resistant to denial of service attacks. This work was sponsored by Collabora.

Eight security issues were discovered, fixed and attributed a CVE. They were found by looking at the source code (in D-Bus and Linux' af_unix implementation), checking existing issues in the D-Bus bugzilla and a bit of luck.

### Security issues fixed in D-Bus

• CVE-2014-3477(Bug #78979): dbus-daemon sent an AccessDenied error to the service instead of a client when the client is prohibited from accessing the service, which allowed local users to cause a denial of service (initialization failure and exit) or possibly conduct a side-channel attack via a D-Bus message to an inactive service.
• CVE-2014-3532(Bug #80163): when running on Linux 2.6.37-rc4 or later, local users could cause a denial of service (system-bus disconnect of other services or applications) by sending a message containing a file descriptor, then exceeding the maximum recursion depth before the initial message is forwarded.
• CVE-2014-3533(Bug #80469): dbus-daemon allowed local users to cause a denial of service (disconnect) via a certain sequence of crafted messages that caused the dbus-daemon to forward a message containing an invalid file descriptor.
• CVE-2014-3635(Bug #83622): an off-by-one error in dbus-daemon allowed remote attackers to cause a denial of service (dbus-daemon crash) or possibly execute arbitrary code by sending one more file descriptor than the limit, which triggered a heap-based buffer overflow or an assertion failure.
• CVE-2014-3636(Bug #82820): a denial-of-service vulnerability in dbus-daemon allowed local attackers to prevent new connections to dbus-daemon, or disconnect existing clients, by exhausting descriptor limits.
• CVE-2014-3637(Bug #80559): malicious local users could create D-Bus connections to dbus-daemon which could not be terminated by killing the participating processes, resulting in a denial-of-service vulnerability.
• CVE-2014-3638(Bug #81053): dbus-daemon suffered from a denial-of-service vulnerability in the code which tracks which messages expect a reply, allowing local attackers to reduce the performance of dbus-daemon.
• CVE-2014-3639(Bug #80919): dbus-daemon did not properly reject malicious connections from local users, resulting in a denial-of-service vulnerability.

### Other fixes

In addition to fixing specific bugs, I also explored ideas to restrict the number of D-Bus connections a process or a cgroup could create. After discussions with upstream, those ideas were not retained upstream. But while working on cgroups, my patch for parsing /proc/pid/cgroup was accepted in Linux 3.17.

### Identify bogus D-Bus match rules

D-Bus security issues are not all in dbus-daemon: they could be in applications misusing D-Bus. One common mistake done by applications is to receive a D-Bus signal and handle it without checking it was really sent by the expected sender. It seems impossible to check the code of all applications potentially using D-Bus in order to see if such a mistake is done. Instead of looking the code of random applications, my approach was to add a new method GetAllMatchRulesin dbus-daemon to retrieve all match rules and look for suspicious patterns. For example, a match rule for NameOwnerChanged signals that does not filter on the sender of such signals is suspicious and it worth checking the source code of the applications to see if it is legitimate. With this method, I was able to fix bugs in Bluez, ConnMan, Pacrunner, Ofonoand Avahi.

GetAllMatchRulesis released in dbus 1.9.0 and it is now possible to try it without recompiling D-Busto enable the feature. I have used a script to tell me which processes register suspicious match rules. I would like if there was a way to do that in a graphical interface. It's not ready yet, but I started a patch in D-Feet.

## October 01, 2014

### Philip Withnall

#### Dynamic relocs, runtime overflows and -fPIC

While merrily compiling something a little while ago, my linker threw me this gem of an error message (using GNU gold):

error: libmumble.a(libmumble.o): requires dynamic R_X86_64_PC32 reloc against 'g_strdup' which may overflow at runtime; recompile with -fPIC

or, if you’re using GNU ld (the two linkers have different error messages for the same problem):

error: mumble.o: relocation R_X86_64_PC32 against symbol g_strdup' can not be used when making a shared object; recompile with -fPIC

I recompiled everything with -fPIC, and magically the problem went away. But I didn’t understand why. I finally got a bit of time to investigate, so here we go.

tl;dr: This is caused by linking a shared library (which requires position-independent code, PIC) to a static library (which has not been compiled with PIC). You need to either link the shared library against a shared version of the static code (such as is produced automatically by libtool), or re-compile the static library with PIC enabled (-fPIC or -fpic).

To understand this, we need a brief introduction to the different types of linking, and how static objects and libraries differ from shared (or dynamic) objects and libraries. Let’s run with a minimal working example: two C files, shared.c and static.c. static.c is compiled to a static archive, libstatic.a (without position-independent code, PIC), and shared.c is compiled to a shared object, libshared.so, which links against libstatic.a.

What is a static object? It’s one where all symbol references are resolved at compile time. What’s a dynamic object? One where symbol references can be resolved at runtime. This means that dynamic objects have to have relocations performed as they’re loaded, which incurs a load-time penalty, but allows for shared libraries and symbol interpositing.

It is these relocations which cause the problem hinted at by the error message above. Each relocation is effectively a note to the runtime loader instructing it to replace a symbol reference in the dynamic object being loaded, with an address calculated at load time.

There are various types of relocations, defined by the platform ABI, as they are specific to the processor’s instruction set. For a more in-depth account of them, see Relocations, Relocations by Michael Guyver. In this case, the R_X86_64_PC32 relocation was chosen by the compiler, which is defined by the AMD64 ABI (Table 4.10). What does that mean? Each relocation type is essentially a mathematical function to define the address of a relocated symbol, given the information in various symbol, section and relocation tables in the dynamic object. The ABI defines R_X86_64_PC32 as . Less succinctly, it is the offset of the referenced symbol, plus a constant adjustment (the addend) minus the offset of the relocation. This is all explained brilliantly by Michael Guyver on his blog.

So, with our example, we get the error:

$make libshared.so cc -Wall -c -o shared.o shared.c cc -Wall -c -o static.o static.c ar rcs libstatic.a static.o cc -shared -o libshared.so shared.o libstatic.a /usr/bin/ld: error: shared.o: requires dynamic R_X86_64_PC32 reloc against 'my_static_function' which may overflow at runtime; recompile with -fPIC collect2: error: ld returned 1 exit status make: *** [libshared.so] Error 1 If we look at the disassembly of the shared object: $ objdump -d shared.o

shared.o:     file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <my_shared_function>:
0:    55                       push   %rbp
1:    48 89 e5                 mov    %rsp,%rbp
4:    e8 00 00 00 00           callq  9 <my_shared_function+0x9>
9:    5d                       pop    %rbp
a:    c3                       retq

we can see at offset 4 that the callq instruction (calling my_static_function()) leaves 4 bytes for the address of the function to call (actually, callq is instruction-pointer-relative, so the 4 bytes are for the offset of the function from the RIP register).

As the code in libstatic.a is not PIC, it has to be loaded at a fixed offset in a process’ address space. The shared library, libshared.so, must be capable of being loaded anywhere in an address space. This would be fine if the callq instruction could take an absolute address to call, as the linker could substitute in the absolute address of my_static_function() (as is done on 32-bit systems). However, it cannot – it only has 4 bytes of operand to play with, rather than the 8 needed for a 64-bit address – so linking has to fail. And that’s why we get an error which talks about overflow.

What happens if libstatic.a is compiled with PIC enabled? Not a whole lot changes, actually. The disassembly of libstatic.a remains unchanged. shared.o gains a global object table (GOT) section and its relocation for the my_static_function() call changes from R_X86_64_PC32 to R_X86_64_PLT32 — a procedure linkage table (PLT) relocation using the GOT. We can see that in action in the disassembly of the successfully-linked libshared.so (with irrelevant bits omitted):

$objdump --disassemble libshared.so libshared.so: file format elf64-x86-64 Disassembly of section .plt: 00000000000005f0 <my_static_function@plt>: 5f0: ff 25 fa 13 00 00 jmpq *0x13fa(%rip) # 19f0 <_GLOBAL_OFFSET_TABLE_+0x28> 5f6: 68 02 00 00 00 pushq$0x2
5fb:    e9 c0 ff ff ff           jmpq   5c0 <_init+0x20>

Disassembly of section .text:

00000000000006e8 <my_shared_function>:
6e8:    55                       push   %rbp
6e9:    48 89 e5                 mov    %rsp,%rbp
6ec:    e8 ff fe ff ff           callq  5f0 <my_static_function@plt>
6f1:    5d                       pop    %rbp
6f2:    c3                       retq
6f3:    90                       nop

00000000000006f4 <my_static_function>:
6f4:    55                       push   %rbp
6f5:    48 89 e5                 mov    %rsp,%rbp
6f8:    5d                       pop    %rbp
6f9:    c3                       retq
6fa:    66 90                    xchg   %ax,%ax

Firstly, the callq instruction in my_shared_function() has acquired a non-zero operand. This is a constant offset from the instruction pointer at that instruction which references the entry for my_static_function() in the PLT, which we can see as my_static_function@plt in the .plt section. Rather than being the code for the my_static_function(), this is actually a ‘trampoline’ which loads the address of my_static_function() from the GOT, then jumps to it. The GOT is set up by the runtime loader, and allows for the address of my_static_function() to be changed; for example when relocating it, or when interpositing a different version using LD_PRELOAD. By default, the GOT entry for my_static_function() will point to the implementation in the .text section, as linked in from libstatic.a.

This trampolining through a PLT and GOT is the standard solution for producing position independent code, and demonstrates three things:

• Exported functions incur a runtime cost (in the PLT) on every call. This can be eliminated for private symbols (or internal calls to public symbols, with -Bsymbolic), but not (easily) for public ones, as explained by Ian Lance Taylor. This cost is only three instructions; as they change control flow, they could be relatively expensive, but are probably also catered specifically for in modern superscalar 64-bit processors, as the majority of the code they execute will do indirect function calls this way. So the cost can be safely ignored for all but rather specific use cases.
• Position independent code is easy to achieve, and the indirection it requires brings other benefits like the ever-useful LD_PRELOAD, used by developer tools everywhere.
• Marking internal functions as static is important, because ELF exports functions by default, so internal function calls end up being indirected through the PLT if you omit the static modifier. (Though note that none of the functions here could have been marked as such, as they were all in different compilation units.)

So in summary:

• The “requires dynamic R_X86_64_PC32 reloc against ‘mumble’ which may overflow at runtime; recompile with -fPIC” error is caused by attempting to link a shared library against a static object.
• One solution is to compile a position-independent version of the static object. libtool does this automatically, so why aren’t you using libtool?
• Another (highly related) solution is to link against a shared version of the static object.
• This isn’t an issue on 32-bit systems because PIC is possible by default on those systems, since instruction operands are wide enough to contain absolute symbol addresses .
• Compiling with position independent code introduces a procedure linkage table (PLT) and global offset table (GOT) for each object file, which are very hard to eliminate if you want to avoid the (small) function call overhead they introduce.
• So you should avoid PIC if compiling for constrained targets like embedded devices.
• But use it otherwise (e.g. on desktop systems) for the flexibility (the use of shared libraries!) and security (address space layout randomisation) it affords.

Source code for the example here is available on gitorious in the public domain.

## September 17, 2014

### Philip Withnall

#### Long live gnome-common? Macro deprecation

gnome-common is shrinking, as we’ve decided to push as much of it as possible upstream. We have too many layers in our build systems, and adding an arbitrary dependency on gnome-common to pull in some macros once at configure time is not helpful — there are many cases where someone new has tried to build a module and failed with some weird autotools error about an undefined macro, purely because they didn’t have gnome-common installed.

So, for starters:

What does this mean for you, a module maintainer? Nothing, if you don’t want it to. gnome-common now contains copies of the autoconf-archive macros, and has compatibility wrappers for them.

In the long term, you should consider porting your build system to use the new, upstreamed macros. That means, for each macro:

2. Adding the macro to EXTRA_DIST in Makefile.am.
3. Ensuring you have ACLOCAL_AMFLAGS = -I m4 \${ACLOCAL_FLAGS} in your top-level Makefile.am.
4. Updating the macro invocation in configure.ac; just copy out the shim from gnome-common.m4 and tidy everything up.

Here’s an example change for GNOME_CODE_COVERAGE ? AX_CODE_COVERAGE in libgdata.

It seems from the comments that there’s more discussion to be had about the best way to implement this. So hold off on these changes for the moment!

This is the beginning of a (probably) long road to deprecating a lot of gnome-common. Macros like GNOME_COMPILE_WARNINGS and GNOME_COMMON_INIT are next in the firing line — they do nothing GNOME-specific, and should be part of a wider set of reusable free software build macros, like the autoconf-archive. gnome-common’s support for legacy documentation systems (DocBook, anyone?) is also getting deprecated next cycle.

Comments? Get in touch with me or David King (amigadave). This work is the (long overdue) result of a bit of discussion we had at the Berlin DX hackfest in May.

## September 08, 2014

### Xavier Claessens

#### OpenGlucose: Again

Made progress this weekend on OpenGlucose. The GUI is still ugly but it has the info I want.

Important things on my wish-list, when I have time:

1. Handling the units. My only device is mg/dl but other countries uses mmol/L. Since I’m living in Canada where they use mmol/L I should grab a new device with those units, so I’ll be able to compare logs and see how to know in which unit the device is configured.
2. Make printable report, I’ve heard doctors like that. OTOH, I shouldn’t encourage using unofficial app for medical purpose.
3. Support other FreeStyle devices. I’m pretty sure they all have the same kind of format so most of the parser should be reusable, I hope. I should be able to get a spare FreeStyle Freedom Lite in a few weeks.
4. Publish an ubuntu package on a PPA.
5. CSV export.
6. Ideas?

I’m curious, does someone else own an InsuLinx and tried OpenGlucose yet? Or even tried to support another device?

## September 04, 2014

### Nick Richards

#### First as tragedy

Nine years ago I drew some diagrams attempting to reverse engineer the coffee, milk and foam proportions available in the beverages served at Eat. I'm not going to pretend that this was particularly original work but when walking past one of their branches the other day I noticed some uncannily familiar design.

Actually pretty decent communication, well done. Avoid the actual drinks there though.

## September 01, 2014

### Marco Barisione

#### A web browser for the Raspberry Pi

As I previously mentioned, Collabora has been working with the Raspberry Pi Foundation on various projects including a web browser optimised for the Raspberry Pi.
Since the first beta release we have made huge improvements; now the browser is more responsive, it’s faster, and videos work much better (the first beta could play 640×360 videos at 0.5fps, now we can play 25fps 1280×720 videos smoothly). Some web sites are still a bit slow (if they are heavy on the JavaScript side), but there’s not much we can do for web sites that, even on my laptop with an Intel Core i7, use 100% of one of the cores for more than ten seconds!

The browser is based on Gnome Web (Epiphany) using WebKit 1 (i.e. the non-multi-process version of WebKit).

Our main achievements are:

• Progressive tiled rendering for smoother scrolling (as mobile browsers do)
• Startup is three times faster
• Avoid useless image format conversions
• Hardware decoding of videos (through gst-omx)
• Hardware scaling of videos (again, through gst-omx)
• Reduction of the number of memory copies to play videos
• Faster fullscreen playback using dispmanx directly (a bit buggy at the moment, we are working on it)
• Memory and CPU friendly tab management
• JavaScript JIT fixes for ARMv6
• Disk image cache (decoded images are kept in memory mapped files in a cache, saving CPU)
• Memory pressure handler support

No video displayed here? Watch the video on Youtube.
The Raspberry Pi web browser (mp4 video file)

To install the browser, just update your Raspbian and install the “epiphany-browser” package:

sudo apt-get update
sudo apt-get install epiphany-browser
`

Thanks to all the people at Collabora that, at some point or another, helped on this project: Julien Isorce, Emanuele Aina, ChangSeok Oh, Tomeu Vizoso, Pekka Paalanen, André Moreira Magalhaes, Derek Foreman, Gustavo Noronha, Danilo Cesar, Emilio Pozuelo Monfort and Jonny Lamb (I hope I haven’t forgotten anybody!).
Also thanks to the Raspberry Pi Foundation, and in particular to Eben Upton, for their commitment to making browsing on the Pi better, and to Ben Avison for his work on optimising pixman and libav for ARMv6.

Update: people have reported a few bugs since the release, in particular a problem with Raspbian configured to use 24-bit or 32-bit mode for graphics. We should be able to fix this in a week or so.
Another problem is that Vimeo videos stopped working. This seems to be due to a change made by Vimeo that broke playback also on other browsers and on Android.

## August 27, 2014

### Nicolas Dufresne

#### GStreamer gains V4L2 Mem-2-Mem support

Two years since my last post, that seems a long time, but I was busy becoming a GStreamer developer. This story started late in 2009. A team at Samsung (and many core Linux contributors) started adding new type of drivers to the Linux Media Infrastructure API (also known as Video4Linux 2). They introduced video decoding, encoding and video post-processing support through a class of drivers called memory-to-memory.

At the end of 2012, my employer, Collabora was chosen to implement a proof of concept, enabling hardware decoding support to the Cotton Candy, a USB stick size computer based on Samsung Exynos 4412 and built by FXI. The new element has been developed by Sebastien Dröge and was called mfcdec. All this being demonstration code, it never got close to being useful in production..

At the end of 2013, we got contracted again, to bring the demonstration code toward production code. At this point, we took the decision that we where no longer going to build an Exynos specific decoder, but instead re-use the existing GStreamer V4L2 support and do it the “right” way.

It took nearly three months, but with the help of my colleague Julien Isorce, we managed to upstream and ship hardware decoding support for the Cotton Candy. The new element is called v4l2videoNdec, where videoN is that name of the driver node (to allow having multiple decoder at the same time). The element was well suited for static pipeline and embedded applications, but not as flexible as software decoders for desktop.

At the beginning of 2014, we started a new project with Endless Mobile. This time, the goal was to do hardware accelerated decoding also on an Exynos 4412 platform, but in a desktop environment base on Gnome Shell. Two main issues had to be addressed. The buffer pool in GstV4l2 did not track it’s memory, and the color format produced by this decoder could not be color converter using GLES2 shader (not enough coordinate precision). We had to implement a custom memory allocator and rewrite most of the v4l2 buffer pool code. To handle the color format, we had to implement an element that wraps hardware video converter in order to obtain video frames in a format that can be uploaded to GLES2.

As of today, all this effort has landed into GStreamer and is now part of 1.4 GStreamer release. Some of my colleagues went even further by demonstrating during SIGGRAH the benefits of using V4L2 decoder when combining DMABUF and Wayland. Other team, including Pengutronix on Freescale CUDA and STE have started testing against this new promising decoder which finally brings a standard and low level way of decoding medias on Linux.

## August 26, 2014

### Xavier Claessens

#### OpenGlucose: continued

I started working on the UI to display the results:

It is made using a GtkApplicationWindow containing a WebkitWebView, the content is made with HTML/CSS/JS with jquery and the chart is made using jqplot.

To make testing easier, I also added a dummy device that has random data, it can be enabled by setting OPENGLUCOSE_DUMMY_DEVICE=1 in your env.

A lot more work is needed, but that’s a start.

## August 17, 2014

### Xavier Claessens

#### New project: OpenGlucose

Hello there,

I recently got diagnosed with a diabetes type 1. Like all diabetics, I got a glucometer device that comes with a windows/mac closed-source application. That’s clearly not acceptable for a freedom lover! So here is my new challenge: reverse-engineer the USB protocol of my Abbott FreeStyle InsuLinx device, and write an open source Linux application for it.

And here it is: https://github.com/xclaesse/OpenGlucose

So far it only fetch the bare minimum information from the device and print them in the terminal. More GUI/features will come later.

If you’re a geek diabetic, your help is welcome!

## August 15, 2014

### Olivier Crête

#### GNOME.Asia Summit 2014

Everyone has been blogging about GUADEC, but I’d like to talk about my other favorite conference of the year, which is GNOME.Asia. This year, it was in Beijing, a mightily interesting place. Giant megapolis, with grandiose architecture, but at the same time, surprisingly easy to navigate with its efficient metro system and affordable taxis. But the air quality is as bad as they say, at least during the incredibly hot summer days where we visited.

The conference itself was great, this year, co-hosted with FUDCon’s asian edition, it was interesting to see a crowd that’s really different from those who attend GUADEC. Many more people involved in evangelising, deploying and using GNOME as opposed to just developing it, so it allows me to get a different perspective.

On a related note, I was happy to see a healthy delegation from Asia at GUADEC this year!

## August 13, 2014

### The problem

These days there's quite good support for CPU scaling in the mainline kernel, and many ARM SoCs are making use of it already. But in modern hardware with lots of very fast external memory, running the memory bus at its maximum frequency drastically reduces the amount of time that the device can run when on battery.

A problem that many teams are finding when trying to upstream their power management code is that there's currently no way for several clock consumers to influence the frequency of the memory bus. There has been a few tries to upstream the solutions currently in vendor trees, but so far no acceptable solution has been found.

I'm helping to upstream some of the stuff in the ChromeOS tree, and this issue is currently blocking very interesting work from reaching mainline.

### The past

In the vendor tree for Tegra this is addressed by creating virtual clocks that are child of the clock that wants to be influenced. Depending on the type of the virtual clock, setting its rate will influence the rate of its parent clock by setting a floor or ceiling value.

In Qualcomm's vendor tree for the Snapdragon family of SoCs, the concept of a voter clock is introduced. Drivers can vote on the rate of a given clock by "voting" through a child clock, so not that different to how Tegra does it.

Both approaches have the critical disadvantage of adding clk instances for things that aren't real clocks, thus making the API considerably more confusing for relatively little gain.

Both vendor trees have additional API for registering bandwidth needs: tegra_isomgr and msm_bus_scale. They bear quite some resemblance with each other and with pm_qos_interface, but both are tightly tied to specificities of their platforms.

The discussion was brought back to life a couple of months ago when a patch was posted for allowing the tegra-drm driver to set the frequency rate of the external memory controller based on the amount of bandwidth that was needed by the display controller for refreshing the display. Of course, that patch was rejected because there are other components that need to have a say in the frequency rate of the memory bus.

But in that discussion some kind of plan took form and I have been working on making something from it that can be merged upstream.

### A possible future

There's so far two main additions to existing frameworks, with the rationale being explained further below:
• Add per-user floor and ceiling constraints to the Common Clock Framework, so drivers can set maximum and minimum frequency rates that the clock should respect. Patchset here.
• Add a PM_QOS_MEMORY_BANDWIDTH class to pm_qos, for drivers to register their expected bandwidth needs. Patchset here.
The idea is for the following agents to be able to influence the current frequency of the memory bus:
• Thermal: a cooling device would call clk_set_ceiling_rate to cap the memory bus to a frequency based on the current temperature.
• Power: a battery driver would set a ceiling in the same way, based on the remaining capacity.
• Devfreq: a devfreq driver wrapping a power management unit such as the ACTMON on Tegra or the PPMU on Exynos would set a floor frequency based on the current load stats.
• Cpufreq: a cpufreq driver would set a floor frequency based on the current CPU frequency.
• Devices that can anticipate how much memory bandwidth will need (such as the display controller, the camera, multimedia codecs, an ISP, USB, etc) would register their requirements in the PM_QOS_MEMORY_BANDWIDTH class. The EMC driver would be listening for notifications and setting a floor frequency based on the aggregated bandwidth that is needed.
The impression so far is that this approach matches the needs of the Tegra and Exynos SoCs, and people working on Rockchip upstreaming are evaluating it. Others working on other SoCs are very welcome to look at it and comment, so the result is also useful to them and they can improve their power management in mainline without having to refactor things later.

## August 08, 2014

### Frederic Plourde

#### Gecko on Wayland

At Collabora, we’re always on the lookout for cool opportunities involving Wayland and we noticed recently that Mozilla had started to show some interest in porting Firefox to Wayland. In short, the Wayland display server is becoming very popular for being lightweight, versatile yet powerful and is designed to be a replacement for X11. Chrome and Webkit already got Wayland ports and we think that Firefox should have that too.

Some months ago, we wrote a simple proof-of-concept basically starting from actual Gecko’s GTK3 paths and stripping all the MOZ_X11 ifdefs out of the way. We did a bunch of quick hacks fixing broken stuff but rather easily and quickly (couple days), we got Firefox to run on Weston (Wayland official reference compositor). Ok, because of hard X11 dependencies, keyboard input was broken and decorations suffered a little, but that’s a very good start! Take a look at the below screenshot :)

## August 07, 2014

### Frederic Plourde

#### Firefox/Gecko : Getting rid of Xlib surfaces

Over the past few months, working at Collabora, I have helped Mozilla get rid of Xlib surfaces for content on Linux platform. This task was the primary problem keeping Mozilla from turning OpenGL layers on by default on Linux, which is one of their long-term goals. I’ll briefly explain this long-term goal and will thereafter give details about how I got rid of Xlib surfaces.

LONG-TERM GOAL – Enabling Skia layers by default on Linux

My work integrated into a wider, long-term goal that Mozilla currently has : To enable Skia layers by default on Linux (Bug 1038800). And for a glimpse into how Mozilla initially made Skia layers work on linux, see bug 740200. At the time of writing this article, Skia layers are still not enabled by default because there are some open bugs about failing Skia reftests and OMTC (off-main-thread compositing) not being fully stable on linux at the moment (Bug 722012). Why is OMTC needed to get Skia layers on by default on linux ?  Simply because by design, users that choose OpenGL layers are being grandfathered OMTC on Linux… and since the MTC (main-thread compositing) path has been dropped lately, we must tackle the OMTC bugs before we can dream about turning Skia layers on by default on Linux.

For a more detailed explanation of issues and design considerations pertaining turning Skia layers on by default on Linux, see this wiki page.

MY TASK – Getting rig of Xlib surfaces for content

Xlib surfaces for content rendering have been used extensively for a long time now, but when OpenGL got attention as a means to accelerate layers, we quickly ran into interoperability issues between XRender and Texture_From_Pixmap OpenGL extension… issues that were assumed insurmountable after initial analysis. Also, and I quote Roc here, “We [had] lots of problems with X fallbacks, crappy X servers, pixmap usage, weird performance problems in certain setups, etc. In particular we [seemed] to be more sensitive to Xrender implementation quality that say Opera or Webkit/GTK+.” (Bug 496204)

So for all those reasons, someone had to get rid of Xlib surfaces, and that someone was… me ;)

The Problem

So problem was to get rid of Xlib surfaces (gfxXlibSurface) for content under Linux/GTK platform and implicitly, of course, replace them with Image surfaces (gfxImageSurface) so they become regular memory buffers in which we can render with GL/gles and from which we can composite using GPU. Now, it’s pretty easy to force creation of Image surfaces (instead of Xlib ones) for just all content layers in gecko gfx/layers framework, just force gfxPlatformGTK::CreateOffscreenSurfaces(…) to create gfxImageSurfaces in any case.

Problem is, naively doing so gives rise to a series of perf. regressions and sub-optimal paths being taken, for example, to copy image buffers around when passing them across process boundaries, or unnecessary copying when compositing under X11 with Xrender support. So the real work was to fix everything after having pulled the gfxXlibSurface plug ;)

The Solution

First glitch on the way was that GTK2 theme rendering, per design, *had* to happen on Xlib surfaces. We didn’t have much choice as to narrow down our efforts to the GTK3 branch alone. What’s nice with GTK3 on that front is that it makes integral use of cairo, thus letting theme rendering happen on any type of cairo_surface_t. For more detail on that decision, read this.

Upfront, we noticed that the already implemented GL compositor was properly managing and buffering image layer contents, which is a good thing, but on the way, we saw that the ‘basic’ compositor did not. So we started streamlining basic compositor under OMTC for GTK3.

The core of the solution here was about implementing server-side buffering of layer contents that were using image backends. Since targetted platform was Linux/GTK3 and since Xrender support is rather frequent, the most intuitive thing to do was to subclass BasicCompositor into a new X11BasicCompositor and make it use a new specialized DataTextureSource (that we called X11DataTextureSourceBasic) that basically buffers upcoming layer content in ::Update() to an gfxXlibSurface that we keep alive for the TextureSource lifetime (unless surface changes size and/or format).

Performance results were satisfying. For 64 bit systems, we had around 75% boost in tp5o_shutdown_paint, 6% perf gain for ‘cart’, 14% for ‘tresize’, 33% for tscrollx and 12% perf gain on tcanvasmark.

To see the code that we checked-in to solve this, look at those 2 patches :

https://hg.mozilla.org/mozilla-central/rev/a500c62330d4

https://hg.mozilla.org/mozilla-central/rev/6e532c9826e7

Cheers !

## July 25, 2014

### Pekka Paalanen

#### Wayland protocol design: object lifespan

Now that we have a few years of experience with the Wayland protocol, I thought I would put some of my observations in writing. This, what will hopefully become a series rather than just one post, considers how to design Wayland protocol extensions the right way.

This first post considers protocol object lifespan and the related races between the compositor/server and the client. I assume that the reader is already aware of the Wayland protocol basics. If not, I suggest reading Chapter 4. Wayland Protocol and Model of Operation.

### How protocol objects are created

On a new Wayland connection, the only object that exists is the wl_display which is a specially constructed object. You always have it, and there is no wire protocol for creating it.

The only thing the client can create next is a wl_registry through the wl_display. Registry is the root of the whole interface (class) hierarchy. Wl_registry advertises the global objects by numerical name, and using wl_registry.bind request to bind to a global is the first normal way to create a protocol object.

Binding is slightly special still, as the protocol specification in XML for wl_registry uses the new_id argument type, but does not specify the interface (class) for the new object. In the wire protocol, this special argument gets turned into three arguments: interface name (string), interface version (uint32_t), and the new object ID (uint32_t). This is unique in the Wayland core protocol.

The usual way to create a new protocol object is for the client to send a request that has a new_id type of argument. The protocol specification (XML) defines what the interface is, so there is no need to communicate the interface type over the wire. All that is needed on the wire is the new object ID. Almost all object creation happens this way.

Although rare, also the server may create protocol objects for the client. This happens by having a new_id type of argument in an event. Every time the client receives this event, it receives a new protocol object.

As all requests and events are always part of some interface (like a member of a class), this creates an interface hierarchy. For example, wl_compositor objects are created from wl_registry, and wl_surface objects are created from wl_compositor.

Object creation never fails. Once the request or event is sent, the new objects it creates exists, period. This keeps the protocol asynchronous, as there is no need to reply or check that the creation succeeded.

### How protocol objects are destroyed

There are two ways to destroy a protocol object. By far the most common one is to have a request in the interface that is specified to be a destructor. Most often this request is called "destroy". When the client code calls the function wl_foobar_destroy(), the request is sent to the server and the client side proxy (struct wl_proxy) for the object gets destroyed. The server then handles the destructor request at some point in the future.

The other way is to destroy the object by an event. In that case, no destructor must be defined in the interface's protocol specification, and the event must be clearly documented to be destructive as there is no automation nor safeties for this. This is for cases where the server decides when an object dies, and requires extreme care in protocol design to work right in all cases. When a client receives such an event, all it can do is destroy the proxy. The (in)famous example of an interface like this is wl_callback.

### Enter the boogeyman: races

It is very important that both the client and the server agree on which protocol objects exist. If the client sends a request on, or references as an argument, an object that does not exist in the server's opinion, the server raises a protocol error, and disconnects the client. Obviously this should never happen, nor should it happen that the server sends an event to an object that the client destroyed.

Wayland being a completely asynchronous protocol, we have no implicit guarantees. The server may send an event at the same time as the client destroys the object, and now the event targets an object the client does not know about anymore. Rather than the client shooting itself dead (that's the server's job), we have a trick in libwayland-client: it silently ignores events to destroyed objects, until the server confirms that the object is truly gone.

This works very well for interfaces where the destructor is a request. If the client first sends the destructor request and then sends another request on the destroyed object, it just shot its own head off - no race needed.

Things get tricky for the other case, destructor events. The server may send the destructor event at the same time the client is sending a request on the same object. When the server finally gets the request, the object is already gone, and the client gets taken behind the shed and shot. Therefore pretty much the only safe way to use destructor events is if the interface does not define any requests at all. Ever, not even in future extensions. Furthermore, objects with that interface should not be used as arguments anywhere, or you may hit the race. That is why destructor events are difficult to use right.

### The boogeyman's brother

There is yet another nasty race with events that create objects, i.e. server-created objects. If the client is destroying the (parent) object at the same time as the server is sending an event on that object, creating a new (child) object, the server cannot know if the client actually handled the event or not. If the client ignored the event, it will never tell the server to destroy that new object, and you leak in the server.

You could try to make your way out of that pitfall by writing in your protocol specification, that when the (parent) object is destroyed, all the child objects will be destroyed implicitly. But then the client must not send the destructor request for the child objects after it has destroyed the parent, because otherwise the server sees requests on objects it does not know about, and kicks you in the groin, hard. If the child interface defines a destructor, the client cannot destroy its proxies after destroying the parent object. If the child interface does not define a destructor, you can never free the server-side resources until the parent gets destroyed.

The client could destroy all the child objects with a defined destructor in one go, and then immediately destroy the parent object. I am not sure if that works, but it might. If it does not, you have to specify a whole tear-down protocol sequence. The client tells the server it wants to destroy the parent object, the server acks and guarantees it no longer sends any events on it, then the client actually destroys the parent object. Hey, you have a round-trip and just turned a beautiful asynchronous protocol into synchronous, congratulations!

### Concluding with recommendations

Here are my recommendations when designing Wayland protocol extensions:
• Always make sure there is a guaranteed way to destroy all objects. This may sound obvious, but we have fixed several cases in the Wayland core protocol where there was no way to destroy a created protocol object such, that all resources on both server and client side could be freed. And there are still some cases not fixed.
• Always define a destructor request. If you have any doubt whether your new interface needs a destructor request, just put it there. It is more awkward to add later than normal requests. If you do not have one, the client cannot tell the server to free those protocol object resources.
• Do not use destructor events. They are hard to design right, and extending the interface later will be a bitch. The client cannot tell the server to free the resources, so objects with destructor events should be short-lived, and the destruction must be guaranteed.
• Do not use server-side created objects without a serious thought. Designing the destruction sequence such that it never leaks nor explodes is tricky.

## July 10, 2014

### Jeremy Whiting

#### Plasma Next is pretty darn stable

Today I wanted to share some of my experiences with using Plasma Next for the past couple of weeks. Since I had been working on some frameworks development (just a small bit here and there), I thought I'd try running Plasma Next a couple of weeks ago to see how things were coming along and to be able to work on and test some things I helped with back in KDE 4.0 days.

I have to say I'm very impressed with the stability.  I hit two issues since then, and one of the issues has been fixed. The issue I hit that has already been fixed was a crash in yakuake or konsole when closing a tab that caused the whole application to crash. I looked into the Konsole codebase with Eike Hein's guidance, but Argonel ultimately found the best patch to fix the problem.
The second issue I hit with Plasma Next has to do with disconnecting and reconnecting an external monitor. I don't do that very often at all, but when I tried last weekend I got a variety of issues. For example sometimes when disconnecting Plasma (or maybe KSmServer?) crashes and I am taken back to the sddm login screen. Other times when connecting my external screen my panel ends up floating on the external monitor but nothing on it is clickable.

I just realized this post probably sounds like a rant or complaining about Plasma Next, but that's not what I intended at all. The main point I wanted to get across is that I haven't used Plasma more than once in the past 2 weeks since Plasma Next is stable enough for my usage.

Now for the obligatory desktop screenshot:

So good job to all the people that have worked on this new iteration of the Plasma Desktop.

P.S. One other minor thing I miss from plasma is the ability to show multiple timezone's times in the clock's tooltip. I'll see if I can get that fixed though. :)

## July 09, 2014

### Jeremy Whiting

#### Qt5/KDE Frameworks porting steps

As I said in my last post I would elaborate about how porting of libkeduvocdocument (name pending currently) from Qt4 and kdelibs4 to Qt5 and KDE Frameworks happened.

Commits can be seen here but it went like this:
1. Change CMakeLists.txt to look for frameworks and Qt5 packages.
2. Try to build, fix any errors. All while checking the Porting Notes.
3. Port away from deprecated methods.
4. Port away from kdelibs4support.

I forget which part of the above involved each of these, but this is what was changed:
Ported from KUrl to QUrl.
Ported from KStandardDirs to QStandardPaths
Ported from KGlobal::locale() to QLocale
Ported away from other deprecated methods and classes.

So rinse and repeat until it's in a state where you are happy with it.
Note that step 4 above isn't strictly necessary, and is similar to porting Qt4 applications away from Qt3Support (Some kde4 applicationss never were ported away from Qt3Support sadly...) Yes KMouth, I'm looking at you.

## July 07, 2014

### Jeremy Whiting

#### Libkeduvocdocument Qt5/KDE Frameworks port

Hello all. Yes I'm still alive. Yes I'm still doing KDE stuff as I find time or make time. I'll report in the next few posts about what is happening and where we are going.
One of the things that happened recently was the port of libkeduvocdocument to Qt5 and frameworks. Vishesh started the effort, and I completed it with some review by Aleix Pol. It was decided as documented here that since libkdeedu only contains libkeduvocdocument it should be split up. Upon further investigation we realized that the other parts of libkdeedu are not used anymore. Besides the icons subfolder that are still looking for a home, the rest of the git repo is only libkeduvocdocument related, so we decided to just rename the git repository for the frameworks and going forward release. So the libkdeedu git repository holds the kde sc 4 codebase, while the libkeduvocdocument git repo holds the qt5 and frameworks based code. Both contain the history so all history is preserved.

I'll write next time about the steps taken to port the library to Qt5 and KDE Frameworks.