domenica 17 novembre 2013

It's just an init system

The recent debate for the choice of the default init system in Debian has tickled my curiosity to know more about the current situation of init systems and at the same time has led me to form a certain opinion on this matter, which I'm going to share here. It should be noted that I'm currently employed by Canonical (I take the opportunity to underline that the opinions expressed here are just my personal ones), and while this might have had some influence on the forming of my opinion, in all honesty I claim to not have noticed any of that — though indeed I'll leave the verdict to you, if you'll be patient enough to read the whole post.

From the little knowledge I have of these init systems, I'll start by saying that I like Upstart (and I'm a satisfied user of it), as well as systemd and OpenRC. I find the design of Upstart to be elegant, modular, and the implementation could be set as an example for students of what good code is. On the other hand, when in 2010 I first read about systemd from Lennart's blog, I was almost blown away: it seemed to me that every detail had been considered to make the booting phase as efficient as possible, and I was looking forward to its development. And I should state very clearly that — despite not having yet tried it — I now believe that all promises were met, and that systemd is an excellent init system. As for OpenRC, I know really little about it, other than it's out there, it works, it's maintained and is in progress of being ported to FreeBSD.

That said, my wish is that Debian goes with anything but systemd. And that of course is not limited to Debian, but I wish the same for any distribution which is pondering a choice for the init system. Why so? As I said, systemd is an excellent init system, it's used in several distributions, and I would be very happy if other init systems took inspiration from it for all the good stuff it has to offer. My concern is better expressed by quoting Lennart himself:
I believe ultimately this really boils down to this: the Linux userspace plumbing layer is nowadays developed to a big part in the systemd source tree. Ignoring this means you constantly have to work with half-baked systems, where you combine outdated components which do not belong to together into something that in many facets might work but is hardly integrated or consistent. Or to put this another way: you are at a road fork: either you take the path where you use the stuff that the folks doing most of the Linux  core OS development work on (regardless if they work at Red Hat, Intel, Suse, Samsung or wherever else) or you use the stuff Canonical is working on (which in case it isn't obvious is well... "limited").
This text, turned into the opposite perspective, excellently summarizes the danger that comes with going with systemd: entering a road which hopefully continues in green and flowery fields, but from which it will be very very difficult to escape, should you want to. The more distributions go with systemd, the more the Linux world will be tied to it. The code of udev was merged into the systemd project last year, and Kay Sievers (the udev maintainer) assured that "the udev built from the systemd source tree will stay compatible with non-systemd init systems for a long time". How long? It seems just logical that the more distributions use systemd, the less effort will be spent to make udev usable outside on non-systemd systems. And, as Lennart wrote, it's not just udev. Already now logind requires systemd, and tomorrow it could be cgroups, kdbus, Wayland and what not. But if it's true that I like systemd, why am I complaining? How could this be a bad thing? We should simply all switch to the solution that currently is the best one, which in the opinion of many is systemd, and forget worrying, fighting and (more concretely) spending time in maintaining other init systems (and related plumbing stuff).

Well, the reason is that we don't know where the currently flowery road will lead us to. Could it lead to some bad place? If you think of a future where nearly everyone will be walking on this road, well, the bad case is really unlikely: there is a solid company behind the project (Red Hat), and possibly the greatest developers that have been worked on Linux. The fact that almost everyone will be using systemd should simply make it stronger and better. Just look at the Linux kernel.
However... we would still be locked on a road. This is what the text I quoted above says, after all: if you get out of the road, you lose logind (with GNOME as of now, and possible other DEs will follow), kdbus, udev, maybe even Wayland. More precisely, you don't lose them, but you'll have to come up with a system which offers all the equivalent required functionality. It's not impossible, but it would demand a great amount of work. And here I guess that even the most patient of my readers will have asked this question for the third time: "but why the hell would I want to jump off this road?".
Because you still can, as of now. You can write a better udev. You can write a better logind. You can write a better init system, a new one, you can even start from scratch, as a hobby project. You can write it in Go. You can innovate, you can come up with some revolutionary idea (as in some ways was systemd when it was conceived). Once all the Linux plumbing layer is tied together, changing stuff will be much more difficult — especially if you are a single individual with great ideas but limited time to implement them. Contributing changes upstream is not always easy, and it becomes more difficult as the complexity of the project grows. Not anyone can afford develop a new feature in a branch, and keep it in there until it's ready to be merged, rebasing it and solving merge conflicts every other day, for months. And it's likely that the more revolutionary your idea is, the deeper and more disruptive your changes will be, to the point that keeping them in a branch would be impractical. This is why I wish that the components which form the software stack remain separate: because you might want (or need) to replace them. You don't like Upstart? Fine, use OpenRC. OpenRC sucks? Fine, use something else. But in the future you won't have this choice with systemd, and probably with any other component of the Linux plumbing layer, once it's become part of a single entity.

I liked it when systemd came out. And I want such a change to be able to happen again.

Etichette: ,

domenica 3 novembre 2013

Mappero: public source code, CLA, Qt5 port

Mappero has always been distributed under a GPL licence. However, since when I started selling Mappero Geotagger (which is built from the same source), I decided not to publish the source code in a public repository, but only to provide it to those who made an explicit request to obtain it.

I spent some time reconsidering the matter, and I've finally decided to let the source code live in a public repository. I also setup a mailing list for it. And indeed I welcome code contributions, however there's a small catch: a CLA. While Mappero is distributed under the GPLv3 licence, I request that all contributors send me an e-mail in which they give me the right to re-licence their contribution under any licence published by the Free Software Foundation.

Since I believe that the busiest time for my involvement with speculo has passed, I expect to be able to spend some more time developing Mappero. The qt5 port is more or less working, but most of the cool features are missing, so it's little more than a map viewer at the moment (Mappero Geotagger, however, is fully working under Qt5!).

Here you can see Mappero running on an Ubuntu Touch powered Nexus 4. Pinch zooming and GPS are not yet working, but I promise they'll be there in less than a week. Also I found a nasty bug which can cause the application to crash when downloading map tiles, and I'll fix it ASAP (I'm mentioning it just so that I won't be flooded with identical bug reports now :-) ).

Etichette: , , , , ,

sabato 2 novembre 2013

speculo, or shared memory made easy

The last few months I've been dedicating most of my (very little) free time to a new project: speculo, a library which implements IPC (Inter-Process Communication) on top of shared memory. Since developers appreciate conciseness and minimalism, here's a description of speculo in a few bullet points:
  • written in C
  • POSIX (tested in Linux and FreeBSD)
  • small (~850 LOC)
  • well commented (~400 lines)
  • good test coverage
  • zerocopy
  • lockless
  • one writer, many readers
  • data is written and read in chunks of arbitrary size
  • a data chunk becomes visible to the readers as soon as the writer commits it
  • data chunks can have an expiration time
  • data chunks can be obsoleted by a newer copy
  • garbage collector
  • no file descriptors are permanently kept open
  • no change notification
But here I probably need to write some paragraphs to explain a couple of points.  Except for a few memory addresses which hold the state of the memory area and which are atomically updated and guarded with memory barriers, all the data written to the shared memory object is immutable. This in particular guarantees that readers have a consistent access to the data, which will not change under their eyes. Data chunks are only appended, which means that the shared memory object can only grow. However, data chunks can be marked as expired (if they have an expiration time associated with them) or obsoleted (if a new chunk is said to replace their contents), which means that not all of the data which is written in the SHM object is actually valid. The readers' functions know this, and skip over the invalid data.
At some point the garbage collector will kick in, when the conditions specified by the writer are met or when the SHM area is completely full. All the chunks which are still valid will be copied over to a new SHM object, and then the SHM objects will be atomically switched. Readers will be able to complete their ongoing reads, and transparently move on to the new SHM object as they request to read a new data chunk.

The above means that speculo can be used in at least two ways:
  • to implement a data stream: the writer fills the shared memory area with short-lived data chunks
  • sharing changeable blocks of data
The first use-case doesn't need much of an explanation; it's usual message passing between a writer and some readers. The second use-case means having a writer expose one or more blocks of data to the readers; if the data gets updated, readers can re-read it and obtain the latest version.

On the minus side, speculo doesn't implement any mechanism of change notification: when the writer writes a new data chunk, or updates an existing one, readers won't be notified of it (though they will get the new data, if they access the shared memory area). This was done intentionally, because the scope of speculo is just to make using shared memory a little easier, and to integrate with existing IPC mechanisms, such as sockets or D-Bus (or even new IPC mechanisms).

The project is in such a state where the declared functionality is working, though I wouldn't swear about its complete reliability (more tests are needed). The current API is not fixed in stone, and on the contrary I'm thinking of a couple of changes to apply. Also, after seeing the implementation of shm_open in Linux and FreeBSD, I'm seriously considering dropping the usage of it in favour of plain open, which would then allow for per-user namespaces. So, I think this is a good time to join the project, to shape it the way it can be useful to the largest number of people.

Last but not least, the link to the project page: https://gitlab.com/mardy/speculo
There is not much documentation in the wiki yet; I recommend checking out the code and running make doc to build the HTML documentation (requires doxygen). To run the tests, use make check (requires pkg-config and check).
A mailing list is up here.

Etichette: , , , ,

domenica 20 ottobre 2013

Experimento culinari - le revelation

Circa vinti dies retro, io vos presentava le resultato de un experimento culinari. Iste resultato esseva partial, perque io de facto non completava le experimento; lo que io obteneva esseva quasi edibile, ma io non esseva secur que le procedura esseva correcte e decideva de repeter lo.
Ben, ante dicer plus, io revela qual es le ingrediente mysteriose que io usava: glandes de robore (quercus, it: quercia, en: oak). Il ha plure arbores de robore in le proximitate de mi casa, e quando io vadeva a promenar me e io videva iste enorme quantitate de glandes que se accumulava al terra sub le arbores, io decideva de investigar si illos esseva in alicun modo edibile. Io jam sapeva que altere animales como scuriolos e porcos pote mangiar los, ma io nunquam habeva audite que illos es bon pro humanos. Alora, io discoperiva alicun cosas interessante, gratias a internet.
Le prime cosa que io legeva, es que iste glandes es edibile e de facto multo nutriente, ma quasi tote le varietates es multo (e a vices, terribilemente!) amar, a causa del tanninos que illos contine. Il es totevia possibile eliminar le grande parte del tanninos e render los plus appetibile, per lavar los longemente in aqua currente. Le nativos de America los mangiava post haber los tenite in le aqua de un torrente pro un anno, e on dice que illes apprendeva iste methodo per observar le comportamento de alicun animales, que poneva le glandes inter le roccas del torrentes. Isto, on dice, es le technica melior; ma si on non vole attender un integre anno, on pote usar methodos alternative, como le bollition.
Ma lassa me vader con ordine, assi que io presenta tote le procedura, e adde alicun photos.
Glandes in aqua; illos que remane al superficie non es bon
Le prime cosa a facer es eliminar le glandes que non es bon. Isto es multo facile: on submerge tote glandes in aqua, e on reguarda qual glandes remane al superficie: istos non es bon, e debe esser eliminate.
Postea, on debe essiccar le glandes, assi que il sia plus facile remover le cortice e extraher le nucleo. Pro facer isto, io lassava le glandes per tres dies sur le pavimento del camera del sauna, que esseva legiermente rescaldate. Ma un qualcunque fonte de calor es bon, si illo non es troppo calide. Poner los al sol es un optime solution (infortunatemente in Finlandia le sol de octobre non es bastante calide).

Le nucleos del glandes
Le procedura de remotion del cortice es lente e manual; io non ha trovate un maniera rapide de facer lo. On debe haber patientia.
Quando tote le nucleos ha essite extrahite, on debe remover le tanninos. On pone le glandes in aqua bollente, e on los lassa bollir pro plure minutas (circa dece), usque on vide que le aqua es devenite scur (marron). Alora on debe cambiar le aqua, e repeter le operation. On lo face pro tres o quatro vices, e al fin on poterea vider que le aqua non deveni tanto scur (isto indica que le tanninos ha essite eliminate). Io ha legite que il es importante que le aqua es jam in bollition quando on pone le glandes in illo; alteremente, si illo es initialmente frigide, on dice que le tanninos va esser bloccate in le glandes.

Post le bollition, le glandes debe esser essiccate de novo
Le bollition rende le glandes molle, e illos se rumpe multo facilemente. Alora, il es un bon idea poner los in un blender (tritator) e reducer los a granulos, como de farina. Isto debe esser essiccate, al sol o con un altere fonte de calor non troppo forte.
Mi "farina" de glandes, preste pro esser addite al torta
Le resultato final es un sorta de farina, que un pote usar pro facer pan, tortas o altere cosas. Io seligeva de cocinar le dulce que plus me es familiar, le ciambella:

Le resultato de mi enorme labor: un ciambella!
Le ingredientes esseva:
  • 3 ovos
  • 280 gr de sucro
  • 300 gr de farina
  • 80 gr de farina de glandes
  • 16 gr de levatura (it: lievito, en: yeast)
  • 150 ml de lacte
  • 100 gr de butiro
On initia per miscer le ovos con le sucro. On continua a miscer, e on adde le butiro liquide (on debe rescaldar lo multo dolcemente, usque illo deveni liquide) e le farina. Postea, le lacte, le levatura e le farina del glandes. On pone toto in le forno, a 180 grados, e on lo cocina pro circa 40 minutas.

Benque Yulia non esseva multo supportative de mi effortios de laborar le glandes (al contrario, illa esseva quasi hostil!), post haber mangiate le ciambella illa ha cambiate opinion, e dice que io debe facer lo ancora. Isto es un bon signo, nonne?

Etichette: ,