Home Home > Base-system
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Archive for the ‘Base System’ Category

-fomit-all-instructions

May 27th, 2009 by

Now that milestone2 is almost done, we get into the next round and we decided (basically during coffee break) to try something and change the compile flags for our i586 distribution (which is still the major one for openSUSE). If there aren’t major problems coming up with it, the next milestone will be compiled with -fomit-frame-pointer -mtune=generic instead of the old -march=i586 -mtune=i586.

It will feel around 20% faster, I’m sure 🙂

On unrelated news: I’m away for a very long Pentecost weekend. If there are issues with Factory, I won’t be able to fix them. See you in June!

NetworkManager and keyring

May 15th, 2009 by

Tonight I stumbled upon the solution to a irritating little problem I have had for a long time. I use autologin and every time I log in I get prompted for my keyring password in order to access the wireless network. I have googled for this problem numerous times in the past without any luck. All the suggested solutions had to do with Ubuntu and a tool called libpam-keyring. This does not seem to work the same on openSuSE as on Ubuntu and did not help me much. Then I found this post. Towards the bottom of the thread is the instructions that have been evading me for so long. Hope this helps someone else.

LiveCD Performance (clicfs vs. SquashFS)

May 15th, 2009 by

When Coolo looked into how to get rid of (Another) UnionFS for Live CDs and came up with the DoenerFS (now clicfs) idea, I remembered that my friend Arnd has workded on fake write support for cramfs. So I took his code and ported it to SquashFS to see how that goes. My expectation was that it might be faster than Coolo’s clicfs using FUSE. Here are some results using openSUSE-KDE4-LiveCD-i586-i686-Build0098 booting into runlevel 3:

  • clicfs: 637MB ISO Image booting in 1:28 min (0:24 min from RAM)
  • squashfs-rw: 751M ISO Image booting in 1:50 min (0:28 min from RAM)

The difference in the sizes of the ISO images are due to the fact that clicfs is using LZMA compression while SquashFS is still using the in-kernel GZIP implementation. Surprisingly the clicfs image isn’t only smaller but is also faster booting on real media and from RAM (using KVM). So even if we ignore the fact that clicfs is optimized for limiting the number of seeks on disk the SquashFS implementation is still slower. It would be interesting to see if it is just the LZMA compression that is making the difference or something completely different.

The patches for the SquashFS fake write support are here: http://git.infradead.org/users/jblunck/linux-2.6.git?a=shortlog;h=refs/heads/squashfs-rw-v1.

Today in Factory: A smart gdb

May 13th, 2009 by

I have no idea who came up with this, but I just fell over a gdb that teached me to use zypper while
debugging a problem I’m having:

Missing separate debuginfo for /lib64/ld-linux-x86-64.so.2
Try: zypper install -C "debuginfo(build-id)=41d4f203d690d7db47fbcc38c3f47a2cdcc6848f"

And so I did, and it installed glibc-debuginfo. Smart!

Qemu network speed

May 7th, 2009 by

I’m working on a solution to automatically test drive factory installations every day and one thing that bothered me majorly was the download speed I had with kvm. I couldn’t get faster than around 300K/s, which makes a factory installation take around 5 hours. Not really practical for my purpose, and it was quite frustrating taking that the host gets almost 3MB/s out of download.opensuse.org (I get various german mirrors). So I asked around and Jan suggested the setup of Gladiac, but it didn’t want to fly for me. So I continued experimenting and went to Luwdig for some firewall advise. And his suggestion was much simpler: using pcnet. So my kvm call looks like this now:


qemu-kvm -net user,hostname=factory-pc -net nic,model=pcnet -drive ....

Now I have about the full performance – and I’m ten steps closer to automating bootchart generation of daily factory 🙂

I got Network!

May 6th, 2009 by

I tried an installing using the USB images on the Acer Aspire One I’m using to test these things (and to test preload on SSD systems).

And I wasn’t able to connect to wifi, because the NetworkManager didn’t want to give out wireless networks to the applets. The reason was in the logs: it thought the radio kill switch was on. But fortunately Helmut gave me the golden tipp:

echo "blacklist acer-wmi" > /etc/modprobe.d/coolo.conf
rmmod acer-wmi
rcnetwork restart

It’s one of these things that frustrate you, because they are very easy to work around if you find the right spot in the internet. And that’s the reason I create this blog entry: to allow the next with the problem to fix his wlan. Happy surfing and thank Helmut not me 🙂

BTW: it’s fixed for newer kernels, but 2.6.30 is too fresh for me.

GSoC introduction – openSUSE@ARM

May 4th, 2009 by

Hi openSUSE community!

I’m glad my proposal was accepted and today I want to introduce myself and my GSoC project.

/me , thats Jan-Simon Möller and I’m just finishing my Diploma in electrical engineering at the Leibniz Universität Hannover. I’m coordinator of the openSUSE Weekly Newsletter and contribute also to the hamradio repository, the iFolder project and the openSUSE Build Service. See also my “People of openSUSE” interview.

My Project in short:  openSUSE@ARM
My aim during GSoC 2009 is to port first the base to the ARM platform. Then KIWI needs also some attention when it comes to imaging and after that the tools, Kernel and X11.

I’ll heavily use the capabilities of the openSUSE Build Service, which is now ready for ARM.

During the last few days, I’ve done many little preparations to get it all flying when GSoC coding period starts.

Stay tuned !

What’s behind “lzma compressed livecds”

April 28th, 2009 by

There are various ways to build a live cd and since 11.2 Milestone 1 there is a new one: clicfs. I’ll try to explain:

The challenge with a live cd is the size of the CD and what you put on it, 700MB is not enough for a typical desktop experience. So most (if not all) distributions use compression to squeeze about 2GB on a CD. There are various compression file systems, most famous is squashfs – which is since 4.0 even in the kernel mainline (>= 2.6.29) and uses gzip compression, even though there exist patches to make it use lzma. These lzma patches are not very often refreshed and not officially supported by the squashfs authors. This might actually change soon as kernel 2.6.30 has lzma decompression built in – but it’s not there yet.. There is also cloop, which I think is exclusive to Knoppix. It’s also gzip compressed and has a different semantic than squashfs. More to that later.

But however you compress, it comes with a catch: your compression will create a read only file system. But you need to write to it, not to all places, but in various (/var/run, /var/tmp, /tmp/, often in /etc, surely in /home). For a long time, the live cds existent created one large tmpfs and symlinked all the places and files that needed writing. But this is pretty unflexible and also takes more memory than really necessary. The new solution was first unionfs, which has a long history with many ups and downs and then later aufs. Aufs is Another unionfs and is the reason for one of the downs of unionfs. With a union you can generate a file system that is actually a map of two: the read only part and a read write part. But neither of them has good short term prospect of getting in the kernel mainline (aufs is trying hard at the moment, but still my personal guess is: 11.2 will pass without it being in mainline).

Some distributions have no problem with using large kernel patches, but for openSUSE it became more and more of a problem. Every kernel update broke it and the way aufs works is not easy to adopt to kernel changes – which is the reason the aufs author is pushing into mainline. Let’s wish him luck.

So I looked around and tried unionfs-fuse and deltafs. Both are fuse file systems avoiding the need of large kernel patches to create a compressed read write file system. Both had their issues, deltafs is just a prototype at the moment and unionfs-fuse is still pretty young too (unionfs-fuse fixed most of my issues meanwhile, so possibly it will come back to me 🙂

Then I looked closer at what Fedora(10) does: they do it completely different than most distributions I checked so far. They use a huge ext3 file system with their read only content, compress it in squashfs and then use a device-mapper snapshot on top of it to make it read write into a device-mapper copy-on-write file. Very interesting, in theory very slow – not slower than 11.1 in practise. So kiwi supports that mode since v3.29 (called “dmsquash”). But as the 11.2 kernel OOPSed with such loopback mounted file systems (fixed meanwhile) I kept on looking for alternatives. And as unionfs-fuse was so close and has also support for cow, I thought if I can’t do my own fuse filesystem that combines the best of all. Actually I thought out loud while eating doener with Michael and Jan and so I hacked doenerfs as prototype in just an afternoon.

The idea is very simple: Instead of using device-mapper with squashfs snapshot and copy-on-write file I put the ext3 file system in another file system just made to compress it: doenerfs. That filesystem uses xz libraries. So you can mount the compressed image and then mount the ext3 file system loop and get a read write file system. Of course it comes with a catch: the writes don’t go anywhere but in the memory of the fuse driver. The more you write, the bigger the process gets. But in the end it doesn’t matter what uses your RAM, tmpfs or fuse mounts.

Meanwhile I extended the code quite a bit in adding some possibly unique features and renamed it to clicfs: Compressed Loop Image Container. The 11.2 Milestone 1’s Live CDs use it’s first version.

As the file system is explicitly made to compress another file system, I only need to support one file with a fixed name and with a known size and all that – a lot of complexity of other file systems is gone. So I could spend most time developing the interesting parts 😉

The fuse driver has only 3 options and each of them marks a feature:

  • -m adds sparse blocks at the end of file. The actual ext3 file system has 0 blocks free, the sparse blocks are appended during boot of the livecd. After that the live cd will resize the file system to match the gained room. And only blocks actually written to will end up in memory. Sparse blocks are only marked as not yet existent. Current live cds hard code 470MB, but it’s just some /proc/meminfo left to do and then it will have free space depending on RAM amount. So far the free space was limited by either what the ext3 image left free (fedora) or how much is in the tmpfs (aufs solution)
  • -l logfile will create a log file with the access pattern. You can pass this log to mkclicfs to group the ext3 blocks in order in the compressed image. I use this while building the livecds to avoid seeks on the CD.
  • -c cowfile puts the blocks that are written to in an extra file instead of RAM. This makes it possible to store them on an usb stick and load them at later boots. This is the newest and least tested feature.

The sources are at git.opensuse.org.

What’s New in 11.2: Install Debuginfo Package by build-id

April 27th, 2009 by

With the help of a unique identifier that is stored in every binary executable matching the executable, a coredump and the corresponding debuginfo together becomes really easy. You don’t need to know the package name and the version-release string to download and install the correct debuginfo package. This is achieved by extending the linker, some additional tools and the package management itself.

(more…)

ARM support in openSUSE Buildservice – fixed

April 27th, 2009 by

The issue caused by the OBS worker update on arm builds is fixed by a new qemu.

This new qemu version also has fixed the Fedora 10 @ ARM build problem.

So we have the following working ARM target distros available for ARM: Fedora 10, Debian 5.0 and Ubuntu 9.04.

Have fun.