Home Home > 2009 > 05
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Archive for May, 2009

Qemu network speed

May 7th, 2009 by

I’m working on a solution to automatically test drive factory installations every day and one thing that bothered me majorly was the download speed I had with kvm. I couldn’t get faster than around 300K/s, which makes a factory installation take around 5 hours. Not really practical for my purpose, and it was quite frustrating taking that the host gets almost 3MB/s out of download.opensuse.org (I get various german mirrors). So I asked around and Jan suggested the setup of Gladiac, but it didn’t want to fly for me. So I continued experimenting and went to Luwdig for some firewall advise. And his suggestion was much simpler: using pcnet. So my kvm call looks like this now:


qemu-kvm -net user,hostname=factory-pc -net nic,model=pcnet -drive ....

Now I have about the full performance – and I’m ten steps closer to automating bootchart generation of daily factory 🙂

I got Network!

May 6th, 2009 by

I tried an installing using the USB images on the Acer Aspire One I’m using to test these things (and to test preload on SSD systems).

And I wasn’t able to connect to wifi, because the NetworkManager didn’t want to give out wireless networks to the applets. The reason was in the logs: it thought the radio kill switch was on. But fortunately Helmut gave me the golden tipp:

echo "blacklist acer-wmi" > /etc/modprobe.d/coolo.conf
rmmod acer-wmi
rcnetwork restart

It’s one of these things that frustrate you, because they are very easy to work around if you find the right spot in the internet. And that’s the reason I create this blog entry: to allow the next with the problem to fix his wlan. Happy surfing and thank Helmut not me 🙂

BTW: it’s fixed for newer kernels, but 2.6.30 is too fresh for me.

Profiled Live CDs

May 6th, 2009 by

As said earlier, the Factory Live CDs are using clicfs for compression and write access. This has one performance problem: booting on ext3 accesses many different blocks (4096 bytes each) that are on many different places in the file system. Squashfs has a structure that makes it a better fit to livecds as it compresses metadata independent from file data. This doesn’t give you a huge difference, but it’s there. Especially with CDs that are very slow to seek from track to track. Booting the gnome 64bit live cd loads about 177MB (out of >2200MB in total on the CD) when booting into an autologin session. This comes with around 1700 seeks – and one seek can be as slow as of half a second (even though most don’t take that long).

But what’s worse: as clicfs (and squashfs) create larger blocks for gaining better compression (128KB for both), you will in effect decompress way more data than you need – and while the decompression used isn’t a big problem for nowadays computers (and you don’t necessarly expect the live cd to fly on yesterday’s computers), it adds quite a lot to the boot time of the live cd.

As I talked a lot with real file system developers about the problem, a pretty simple idea came up: reshuffle the blocks in a way that they are just in the right order – both within the larger compression blocks as on CD to avoid seeks as much as possible. That’s the reason clicfs and mkclicfs both take a -l option. If you mount the container with -l, it will output things like this to the logfile:


access 0+11
access 189+4
access 32957+4
access 65536+4
access 98493+4
access 131072+4
access 164029+4
access 196608+4
access 229565+4
access 262144+4
access 295101+4
access 327680+4
access 360448+4
access 393216+4
access 425984+4
access 458752+4
access 491520+4
access 524288+4
access 557056+4
access 589824+4

That’s actually the first couple of accesses on booting said live cd. It’s resize2fs that’s reading 16K here and there on the file system – accessing 20 compression blocks, decompressing 132K for each of these accesses. But the good news: if you pass this log file to mkclicfs -l, it will put all these blocks as first blocks while creating the container. Of course you need to make sure both clicfs and mkclicfs are talking about the same ext3 file system, otherwise it will be nonsense data.

So what I do is abusing the build service in a sense. I let kiwi create a live cd with fast compression as first step – fast compression because I’m only interested in the ext3 file system. Then I download and boot these live cds in kvm and grab the clicfs log. These I upload back into the project and as third step I have a package that will depend on the live cds from step 1 and the logs from step 2 to generate the real live cds with good compression.

This makes up for the initial disadvantage in performance – it’s actually the opposite, the factory live booted 12% faster than the 11.1.

Rebooting icecream

May 6th, 2009 by

Our icecream scheduler is moved to another room, so I thought I share the statistics
(just telnet <scheduler> 8766)

200-ICECC 0.8.0-make-it-cool: 5261077s uptime, 61 hosts, 0 jobs in queue (53255934 total).

So this is still the pre-0.9 code and it served in 60 days over 10 jobs per second on average.

openSUSE-GNOME BugDay!

May 5th, 2009 by

Roll up those sleeves and mark your calendars, because here comes another BugDay!

During Community Week (http://en.opensuse.org/CommunityWeek), I’ll be hosting another openSUSE-GNOME BugDay on Fri, 15 MAY 2009. We’ll start promptly at 1000 CDT and will continue until 1600 CDT. I will be around very early in the day to start prep for the meeting should you have any questions.

We’ll conduct business in #opensuse-gnome on Freenode (irc.freenode.net). I will establish a Gobby session as I’ve done in the past, and we’ll work off of that.

Can’t wait to see you there!

OpenOffice_org 3.1 beta7 available

May 5th, 2009 by

I’m happy to announce that OpenOffice.org 3.1 beta7 packages are available in the Build Service OpenOffice:org:UNSTABLE project. They include many upstream and Go-oo fixes. Please, look for more details about the openSUSE OOo build on the wiki page.

The packages are beta versions and might include even serious bugs. Therefore they are not intended for data-critical usage. A good practice is to archive any important data before an use, …

We kindly ask any interested beta testers to try the package and report bugs.

This build includes an alpha version of OpenXML export filters (DOCX, XLSX, PPTX). We also added many fixes to the OpenXML import filters. They should be able to import and export simple documents now.  These changes affected also the import and export filters of the old binary file formats (DOC, XLS, PPT). So we kindly ask anyone to try this functionality and report bugs especially in the import and export of the old binary file formats.

Other information and plans:

I would like to provide rc1 build the following week. I am sorry to say but we are delayed after the upstream release by two weeks. We did too many extra changes last weeks and they need more testing before we mark the Go-oo based release as the final.

GSoC introduction – openSUSE@ARM

May 4th, 2009 by

Hi openSUSE community!

I’m glad my proposal was accepted and today I want to introduce myself and my GSoC project.

/me , thats Jan-Simon Möller and I’m just finishing my Diploma in electrical engineering at the Leibniz Universität Hannover. I’m coordinator of the openSUSE Weekly Newsletter and contribute also to the hamradio repository, the iFolder project and the openSUSE Build Service. See also my “People of openSUSE” interview.

My Project in short:  openSUSE@ARM
My aim during GSoC 2009 is to port first the base to the ARM platform. Then KIWI needs also some attention when it comes to imaging and after that the tools, Kernel and X11.

I’ll heavily use the capabilities of the openSUSE Build Service, which is now ready for ARM.

During the last few days, I’ve done many little preparations to get it all flying when GSoC coding period starts.

Stay tuned !

Factory USB Images

May 4th, 2009 by

If you’re interested in USB images, I’m publishing factory USB images built in the build service. They are completely fresh and see no testing at all, so if you find a problem, send me a patch 🙂

They are compressed .bz2, so the download is roughly the same as a CD ISO, but they are actually .raw images. So you can deploy them on a USB stick and carry around your personal linux hard drive. But you will need something > 3G. Installing them is pretty easy.

Put your USB stick in your computer, then check /dev/disk/by-id/usb* for the name of your stick. Mine is named /dev/disk/by-id/usb-Kingston_DataTraveler_II+_5B751D8C1994-0:0. You can double check by looking if it points to the same sdX that you see last in dmesg. Like this:

sd 29:0:0:0: [sde] Assuming drive cache: write through
sde: sde2
sd 29:0:0:0: [sde] Attached SCSI removable disk


desdemona:~ # ls -l /dev/disk/by-id/usb-Kingston_DataTraveler_II+_5B751D8C1994-0:0
lrwxrwxrwx 1 root root 9 4. Mai 15:47 /dev/disk/by-id/usb-Kingston_DataTraveler_II+_5B751D8C1994-0:0 -> ../../sde

Bot are sde – fine. What partitions are on it, doesn’t matter – all data will be erased by this.

Now the command (for gnome.x86_64 – there are 3 other choices):

wget -O - http://download.opensuse.org/repositories/openSUSE:/Factory:/Live/images/openSUSE-11.2-usb-gnome.x86_64-2.8.0-Build18.1.raw.bz2 | bzcat | dd of=/dev/disk/by-id/usb-Kingston_DataTraveler_II+_5B751D8C1994-0:0 bs=4M

Depending on the speed of your USB stick, this can take a while, but the good thing is that this command doesn’t require any temporary space. My stick is a very fast one and it takes around 10 minutes to download and “burn”. After that, either reboot or put the stick in the computer you like to boot. On first boot, it will expand to the size of your USB stick, creating another partition.

Note that this images come without live installer, but you can of course zypper in yast2-live-installer and xdg-su -c /sbin/yast2 live-installer.

Suppressing KeyboardInterrupt traceback in Python

May 4th, 2009 by

If you have a running program in Python and press Ctrl+C, you’ll get a traceback like this:

$ scout java foo
^CTraceback (most recent call last):
  File "/usr/bin/scout", line 11, in 
    ret = scout.ScoutCore.run()
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 945, in run
    result = module.ScoutModule().main(clp.module_args)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 873, in main
    return self.do_query(args.query, repos, args.inversesearch)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 890, in do_query
    result.add_rows(self._query(repo, query, inversesearch))
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 896, in _query
    r = db.query(self._sql, '%%%s%%' % term)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 485, in query
    if len(row) == 1:           #(2)
KeyboardInterrupt

It is useful suppress it, because user knows he breaks the program and this output should be considered as a bug. Possible solution is wrap a main function by one big try: except KeyboardInterrupt:

try:
  main() # the main function
except KeyboardInterrupt:
  pass # KeyboardInterrupt supressed

But it makes a new level of indentation which should be uncomfortable – especially in Python. Or when you have multiple entry-points, or just don’t well structured program (which is common when you write your private helper script :)), you maybe prefer another solution.

Python has a sys.excepthook, which is called for traceback printing, so we could define our own and suppress unnecessary output here. And it would be nice suppress only one exception and handle other ones using existing function. And this function make it:

def suppress_keyboard_interrupt_message():
    old_excepthook = sys.excepthook

    def new_hook(type, value, traceback):
        if type != exceptions.KeyboardInterrupt:
            old_excepthook(type, value, traceback)
        else:
            pass

    sys.excepthook = new_hook

Function suppress_keyboard_interrupt_message (it is really nice name, don’t it ;-)) stores an existing hook and register an inner function new_hook as a new one. Advantage is that old_excepthook exists only in a scope of this function, so you don’t need use global variables for it.

Update: typos fixed

Modified Versions out

May 2nd, 2009 by

The following Packages are modified and released in hamradio Repository:

* libatlas3
* necpp
* qantenna