Home Home > 2010 > 12
Sign up | Login

Archive for December, 2010

How we use our power

December 16th, 2010 by

I had a side project the last two weeks: Make the build service more fun to use.

No matter how much fun you have creating packages, if they don’t build, there is little point in using a Service that has Build in its name, no? So one of the major goals of the service is actually to help those that want to build packages as good as possible. But there is a problem:

Let me quote from the landing page of build.opensuse.org: “The openSUSE Build Service hosts 16,414 projects, with 107,691 packages, in 26,259 repositories and is used by 25,967 confirmed users.”. That are quite some high numbers – especially in the relation to the ~25 servers we have for actually building.

If you look at the build statistics of the last month (and this is just i586, x86_64 has around the same), you notice that there is not much purple in the “Busy workers / Idle workers” graphic:

OBS load

Every 2nd weekend or so we have some pause where the servers actually idle around, the rest of the time they are usually under full load and it’s not exceptional that we have over 20,000 build jobs for said 25 servers at the same time. So if Sue comes wants to build her packages at that time, she competes with quite some other packages and she gets frustrated to still see “scheduled” when she goes away. So we use some algorithms in the so called dispatcher to distribute the power to the right packages.

Over years of its existence the dispatcher used the algorithm I would dub “Randomness with exceptions” – it would if the job’s filename matches a regexp and if so, preferred it, otherwise picked a random job. Such algorithms create some fairness if you have 28.000 users all active at the same time, because there is usually not a really good balance between those.

But with 2.0 this changed: we got load and priorities. A script of mine parses the logs of download.opensuse.org and counts how many users are for the repositories. From that I calculate priorities, so that repositories of interest to people get more build power than others. E.g. KDE:Release:45 for 11.3 is downloaded 65 more often than for 11.1, so the 11.3 packages should see more attention. For that the build service calculates how many workers were busy for the repositories and then allows a factor to lower that load while picking the next repository to build for. This is much more complex than “pick a random one”, but it lead to faster return times for those projects that see actual downloads (and new projects as they have no load registered). To give some fun for those actually working on their packages, we also lower the registered load if we see commits.

But there was one problem left: with so many projects registered you also have quite some that aren’t interesting at all. They are not downloaded and very often not even their maintainers care, e.g. for some testing subproject they created in 2009 and then forgot about it. But those repositories build against the often changing openSUSE:Factory and see rebuilds because of that. Those repositories had a very low load because they have few packages, so they are often preferred over projects that have a lot of packages.

To free up more power to recent changes, we now experimented with various ways. It turned out to be useful to look at the relation between since last source change in the project and time since the jobs are “scheduled”. From this we calculate a staleness penalty – when the job is freshly scheduled, it’s basically one chance for a worker for 2 months since the last commit to the project. But this chance rises quickly, the penalty gets smaller the longer the job is in scheduled. Even those projects that have no source commits are valid “customers” and deserve to be rebuild against the latest gcc from openSUSE:Factory.

So what does this mean to you as user of the OpenSUSE Build Service?

  • Don’t add repositories unless you really plan to use it. I know that clicking the checkbox to also add all SLE versions is easily done, but remember this is build power and disk space on various mirrors you’ll be using
  • If you really care for your project, it’s a good idea to fix build failures from time to time. You’ll get more build power in return
  • If you plan to do a larger update of some package in your project and want to test the resulting packages building against it, it’s a good idea to disable all other repositories while you do so. The fewer build jobs you create, the lower is your load, the higher are your chances to get more build power
  • Make your repository popular by telling the world about it. More users means more build power
  • And last but not least: don’t get frustrated, remember there are almost 26000 other users

Verein Computerspende Hamburg and openSUSE

December 15th, 2010 by

This article of the German news magazine Der Spiegel made me aware of the Verein Computerspende Hamburg. A German verein is a non profit association, there are lots in Germany for sports, culture and these kind of stuff.

The Verein Computerspende Hamburg does something very useful: It takes used computer parts that other people do not longer need and refurbish them to working computers again. These computers are handed to people who happen to be in a difficult situation in life. They have to live from Hartz IV which is a kind German social security, following the unemployment benefit which ends after a short period. Hartz IV means very, very little money the people have to make their lives from, too little to buy a new computer.

On the other hand, an increasing amount of the job offerings are posted on the web, so people currently unemployed basically have to search the web, not speaking about preparing the resumes to apply for a job. So how would one do that without a computer?

Computing is not only hardware and well known operating systems and applications are far away from free, so it’s obvious that our great project is a perfect partner here: Not that we only offer a user friendly, easy to install, secure, feature rich and last but not least completely free Linux Distribution, we as a community are also able to help if problems come up, regardless if somebody has money or not and what kind of problem it might be. This is another very concrete example where free software and the FOSS communities help your neighbor, or you – as it is very easy to get into a difficult situation in life.

We sent 500 openSUSE 11.3 DVDs to Hamburg with our warm invitation and welcome to all new users to show up and join our community. So be aware of new kids on the block :-)

And if you are in Hamburg or around and want to help, I am sure Verein Computerspende can make use of the help of more Geekos, such as installation, first hand user support and probably much more. And if you know of a similar interested initiative, let me know, I am sure I will find more DVDs ;-)

Board elections…

December 14th, 2010 by

I’ve decided to remain a spectator during this Board elections to understand the process and I like the applies for the board. I can say I’m happy to see ‘mrdocs’ applying for the board, as he was one of the persons which helped me a lot with openSUSE Build Service in my early days. He always had a cool advice or a couple of minutes to make my life easier.

Except for ‘mrdocs’ I really don’t know much about the other candidates, so I’ve decided to take a small note and expose my frustration for not seeing Sirko Kemter running for the Board. I would support Sirko on running to the board for several reasons:

1. He’s the face of openSUSE Artwork team. He’s been doing awesome screencasts and tutorials… I guess in a way, related to Artwork, which I find to be an area which is underdeveloped in openSUSE, he can bring lots of new synergies… and a new visual identity to openSUSE.

2. His degree of commitment to the local German Community. I’ve seen some posts of him and some of the initiatives he does. I loved the openMovie night at KommKino in Nürnberg during openSUSE Conference for which he was the mastermind behind it. So he’s really a community guy.

3. His a vertical and direct person, which speaks his mind. This is something I love on people. I don’t know Sirko from talking on the back of people, I know him from being straight to point and direct. This is a very positive thing…

4.  Gnokii plays well the role of ‘bad cop’. That’s always good to have around. Sometimes people need to ear more harsh things to get back into reality. Gnokii does this role good.

5. He’s one of the most objective people I’ve seen around… many times bringing people to reality, thus speeding up people on taking actions and drop the philosophical issues.

I really see him as a community guy, always around, helping whenever he cans… He’s doing a great job and he should be empowered to run to the Board.

For all the other candidates, which many I don’t know… good luck… with so much cool people running for the Board, I can only see a smiling future ahead. I won’t vote for the Board elections for some reasons, being the major… I won’t to split votes amongst so many cool people, therefore, I’m remaining on neutral grounds wishing all the best of luck. Whatever is the outcome, I’m sure openSUSE will win!

Have a lot of fun!

PS: I hope to see a public debate with all candidates on IRC opened to questions and doubts from the openSUSE community members and users! Please make it happen!

Last Indicator builds!

December 13th, 2010 by

Finally… the last of the indicators builds and works properly… fun thing… works out of the box with Banshee!

Two pics… one from the indicator-sound and another from the messaging indicator with empathy and evolution support.

Outcome of the Christmas dinner…

December 12th, 2010 by

Yesterday we’ve finally met to discuss what we can do together to solidify the Portuguese Community locally. The dinner took place at the selected place a bit later than the established hour.

What we’ve decided:

* ENOS: the ‘Encontro Nacional openSUSE’ (openSUSE Community National Encounter) event is going to remain in our workflow. We’ve decided to fully support a proposal for this years ENOS edition, this time in Lisbon Metropolitan area.

* openSUSE IBERIA: Unanimous convergence to move forward the ‘openSUSE IBERIA’ initiative and make it a priority in our work flow. We also discussed a proposal for a bi-yearly event to be hold in the Iberian Peninsula (taking place in Portugal or Spain, but addressed to everyone). We have also established that this events are to be hold in Universities Campus.

* Battlefield: For us, in Portugal there is only one battlefield, Universities Campus. This is where we are going to be place all efforts.

* openSUSE Event: A generic event was also discussed to take place in Universidade Fernando Pessoa (Porto), to be organized by João Martins (which is applying for Portuguese Ambassador alongside with me within the next weeks). We hope to involve more Universities and local corporate tissue on this event to take place in 2011 on an unknown date (May/June).

* Portuguese Forums (forums.opensuse.org): Two moderators appointed to join the Forums team with whom they will represent our community segment and develop efforts for a sustained growth of the Portuguese Community. We’ve also discussed possible interfaces with the Brazilian and Spanish Communities and how we can benefit from their expertise and aquired know-how on past experiences.

* Ambassadors: Myself and João Martins are submitting a proposal for Ambassadors in Portugal soon. Before we do such, I’m expecting to contact Javier Llorente and the representatives of the Spanish openSUSE Community to discuss some joint action ventures and establish a more formal protocol of cooperation extending the traditional Ambassador workflow.

* Communication: We’ve known the obvious for a long time… we have no communication channels in Portugal. We have no peer points with the national LUG’s, we have pretty much nothing besides a strong will and commitment to make things happen. Our biggest goal is to support directly the existing LUG’s and represent ourselves and create at least a community interface that allows us to deploy communication with other communities in Portugal and promote ourselves. Since there is nothing, we’ll need to take the lead and set the first stone.

All of this and some more was discussed during our Christmas Dinner, while being served the best Calzone Pizza in Portugal, accompanied by the traditional Super Bock beer in a cozy and warm Christmas environment.

New day… new indicator…

December 11th, 2010 by

Ayatana’s battery indicator is known as ‘battery-status’… it’s also currently packaged and semi-tested. This indicator required a tiny patch to use cpufreq-set. It does appear working to me, but I need some more debugging for it.

There’s also a small testing repository for the indicators on my home project on OBS for Factory only, which can be found here. So if you are a openSUSE Factory user… feel free to check it out and if you find any issues, please report them to me.

Currently I’ve only uploaded the Session Menu, ME Menu and Battery Indicators. I need to check a couple of things with the messages indicator before uploading it. As said before, feedback is welcomed.

Another concern that poped out is the Summary’s and information on the spec’s which doesn’t seem consistent… I’m the one to blame, so that’s also for the next days in my TODO list… to provide this packages with a uniform way so that during updates they somehow seem related. Indicators need to be searchable by the term ‘Indicator’ or ‘Ayatana’.

New Indicators… the saga continues…

December 10th, 2010 by

As the indicator saga continues… some new indicators were built… Indicator-messages is a small indicator that pics up messages and stuff from applications to display them on a nice all-around indicator. It works fine with evolution (plugin also built and doesn’t require patching for what I’ve seen).

This indicator support more applications, though they require some patching. As being discussed on the opensuse-GNOME mailing list as as proposed by Vincent Untz, this indicators are moving to GNOME:Apps and there _might_ an effort to support some features in applications like empathy. Keep in mind this is not a priority, but if it’s possible to support some more feature, it might happen.

Additionally, I’ve also tried some other indicators… indicator-application for what I can tell is an indicator that removes the menu’s from the GTK applications and places them on the panel (MAC style). I’ve built this indicator, but I have no means to test yet because further patching is required at least in GTK and eventually on glib. I don’t think there’s actually any need to have this indicator, specially when it will bring additional efforts on GTK maintenance, and this feature is somehow supported by software like gnome-globalmenu (which I’ve also packaged for myself and works dearly with supported apps).

I’ve done some hammering on indicator-sound as well… Ubuntu’s sound widget… I’m stuck with some error and I have 2 possibilities… downgrade to the last version of the previous branch which should built, or work the current branch (more dependency demanding). I’m working on it… it will happen one way or another during the next days. I guess this one is actually one of the most wanted :)

There are few other indicators… for time and date, calendar and so on… Those will be last ones. After indicator-sound is done, I’m starting to clean up the packaging and make sure it’s compliant to the GNOME team methodology and submit them to GNOME:Apps and the support libraries that are required as dependencies forwarded to projects where they might be useful (need to check out with the people which ones those will be).

A special thanks to Ken Vandine from Canonical for some guidance with libindicate. I’ve seen around some ‘hate’ waves towards Canonical, and I would like to say that so far in the few times I’ve interacted with them, either on bug submission or asking help, they have been awesome and caring. It’s really something I would like to point… they are pretty cool, and their devs are very helpful (but hey… so are our devs ;) ).

Also a very special thanks to Vincent Untz, Dimstar and mrdocs which have been a great help and from whom I’ve learned already a lot.

So picking up the useful stuff… and placing it up together in a single Indicator:

Happy Birthday Scribus

December 9th, 2010 by

I rarely blog and even this one is merely  a link to another one, but: http://rants.scribus.net/2010/12/08/happy-birthday-scribus/ is worth a look. So, where is the connection to openSUSE ? Well, way back when, SuSE 9.0 was the first distro to really promote Scribus. :-)

You can have the latest Scribus rpms for many distros, thanks to the awesome Build Service.

Enjoy!

indicator-{session,network}

December 9th, 2010 by

I’ve manage to hammer in 2 more indicators.

Indicator-session provides a quick and easy method to manage your GNOME session. The options are pretty intuitive and it works ok so far. A simple screenshot.

Then we have also indicator-network which was a bit more problematic to build and required a couple of very ugly quick fixes. Though I’ve built a package for it and it’s dependencies (including The Connection Manager). My concerns about it:

* It’s a piece of software under heavy development;
* The user configuration interface is very fuzzy and provides cool clear text passwords for LAN;
* nm-applet is far superior;
* Requires ‘connection manager’ – Connection Manager is a pretty cool application, and it’s compat mode with Network Manager makes it’s integration pretty easy… Though ‘connection manager’ seemed to be an outstanding piece of software, I’m not going through all the trouble to package and integrate it in openSUSE (integration should be pretty fast, just a init script, since it allows compat mode with Network Manager).

I personally don’t like this network indicator… Though I’ve packaged it,  I’ve decided not to share it because of 4 points above. I really don’t see anything positive about having it for openSUSE users, it would most likely turn their GNOME experience into something a bit fuzzy.

Either way… that’s how it looks…

Indicator ME has landed!

December 7th, 2010 by

The first package I pushed to openSUSE was Notify OSD (Canonicals Notification Agent/Bubble) with a small patch for extended features (skinning/theming) maintained by Roman Sukochev. This patch is popular and wasn’t merged with Notify OSD because it ‘breaks’ the spec of Notify OSD that states it shouldn’t be skinnable/themable on Ubuntu.

Now I’ve packaged a lot of dependencies… cracked and patched GTK with a patch taken from Ubuntu, spended hours of mind crushing frustration…. but finally it has landed! The first ‘Indicator’ has landed… Often known as “Ubuntu Me Menu”, under openSUSE it will be known as ‘indicator-me’.

This screenshot was taken on openSUSE Factory 11.4 Milestone 4. It’s a clean install with Sonar and the only tweak is the Faenza-Dark icon theme, which I also packaged and pushed to openSUSE:Contrib.

Top right corner… Indicator ME on openSUSE. It works… I won’t backport it and will only maintain it from 11.4 forward. I’m doing this for personal reasons, mainly to become an alternative on a localized openSUSE Spin that will be named ‘openSUSE IBERIA’, which was already taken into openSUSE-Project by Javier Llorente.

The main reason to bring this to openSUSE and into IBERIA is tightly related to the fact that most people on the Iberian Peninsula use Ubuntu, so we find the best approach possible to offer something as close as Ubuntu and hopefully even a better offer when it comes to localization. We don’t want to make of openSUSE a ‘clone’ of Ubuntu, instead we want to make the openSUSE experience more close to what people already use.

I’m also going to post a small pic of the same software, this time with Canonical’s Radiance Theme, which introduces a small ‘tab’ like visual on the bar. This theme is awesome… regardless of what people think of Ubuntu and Canonical… This is the kind of polishing that makes Canonical successful, not just the ‘marketing’ many point and even despise.

Indicator ME as landed… soon others will follow… My plans are to submit this pieces of software to GNOME:Apps, and not to push as default, but instead offer them at the distance of a ‘zypper in package’ to the openSUSE users out there. And at the same time, since the libraries behind it are also packaged… be more attractive to developers who might want to use them. I see no loss neither an identity crisis on openSUSE by providing more packages and alternative software to improve users Desktop experience…

Future work (after indicators):
* Synapse Launcher;
* Unity (depending on some wild dependencies and some progress in understanding cmake).