Home Home
Sign up | Login

bmwiedemann

coder, tester, poet

Author Archive

New Raspberry Pi Image

September 7th, 2013 by

update: new image with kernel-3.6 and minimal X11/icewm http://www.zq1.de/~bernhard/linux/opensuse/raspberrypi-opensuse-20130911x.img.xz (103MB)

We got a new armv6 based image for the Raspberry Pi.
This one is only 82MB compressed, so pretty minimalistic.
http://www.zq1.de/~bernhard/linux/opensuse/raspberrypi-opensuse-20130907.img.xz

The exciting new thing is that this was created using an alternative image building automatism which I wrote from scratch in three hours this morning.
The scripts can be found at
https://build.opensuse.org/package/show/devel:ARM:Factory:Contrib:RaspberryPi/altimagebuild
and are also embedded within the image under /home/abuild/rpmbuild/SOURCES/

This means that everyone can now easily build his own images the way he likes and even branch and do submit requests for changes that are useful for others.
The way to use this is simple.
If you have 6GB RAM, you can speed things up with export OSC_BUILD_ROOT=/dev/shm/arm before you do
osc co devel:ARM:Factory:Contrib:RaspberryPi altimagebuild
cd devel:ARM:Factory:Contrib:RaspberryPi/altimagebuild
bash -x main.sh

This pseudo-package does not easily build within OBS or osc alone because it needs root permissions for some of the steps (chroot, mknod, mount), which could only be workarounded with User-Mode-Linux or patching osc.
The build consists of three steps that can be seen in main.sh:

  1. First, osc build is used to pull in required packages and setup the armv6 rootfs.
  2. Second, mkrootfs.sh modifies a copy of the rootfs under .root to contain all required configs
  3. And finally, mkimage.sh takes the .root dir and creates a .img from it that can be booted

This can build an image from scatch in three minutes. And my Raspberry Pi booted successfully with it within 55 seconds.

There are some remaining open issues:

  • the repo key is initially untrusted
  • still uses old 3.1 kernel – solved
  • build scripts have no error handling

Compared to the old image, this one has some advantages:

  • It is easier to resize because the root partition is the last one
  • Compressed image is much smaller
  • Reproducible image build, so easy to customize
  • It is armv6 with floating point support, so could be faster
  • We have over 5200 successfully built packages from openSUSE:Factory:ARM
    so for example you can install a minimalistic graphical environment with zypper install xauth twm xorg-x11-server xinit and start it with startx

So if you wanted to play with openSUSE on RPi, you can do so right now and have a lot of fun.

GPIO on Raspberry Pi

March 31st, 2013 by

We have these working openSUSE Factory images for the Raspberry Pi, which is an ARM-based mini-computer, and since I want to encourage my kid to do more with computers than playing games (even if they are open-source), I looked into how GPIOs worked.
For that, you need to find the pin allocation – e.g.
in the elinux GPIO description or
http://www.raspberrypi.org/archives/1417
has a video which has it explained at 03:00

For my test, I wired together pin11 and pin12, which are GPIO17 and GPIO18.
I wanted GPIO17 to receive what is sent by GPIO18.
This is how this looked for me: foto of GPIO17 and 18 wired together

Using it directly from the shell is simple:
echo 17 > /sys/class/gpio/export
echo 18 > /sys/class/gpio/export
echo out > /sys/class/gpio/gpio18/direction
echo 1 > /sys/class/gpio/gpio18/value
watch -n 1 head /sys/class/gpio/gpio*/value

If the wiring and configuration was right, the “watch” will show gpio17/value to become 1 too.
You can then also pull the wire (or insert a physical switch) and see gpio17/value dropping to 0 again, when it is no more receiving the current from the other pin.

If you managed to get this working, you reached level one of hardware-hackery.

OpenStack on openSUSE

January 11th, 2013 by

Do you want to play with cloud software on your own machines?
Some people have been working to package the current OpenStack version “Folsom” for openSUSE (tested on 12.2) and add scripts to configure it into a working state.
You need 2GB of RAM and 3+ GB of free disk space under /var/lib/
Then you do

wget https://raw.github.com/SUSE-Cloud/automation/master/scripts/jenkins/qa_openstack.sh
export cloudsource=openstackfolsom ; bash -x qa_openstack.sh

This is a script we use for continous integration testing, but it is as useful to setup a simple environment for development, testing or demoing.
Folsom packages are still rather rough and might see some change over the coming weeks.

If you want the older stable version, you can use the above snippet with cloudsource=openstackessex
however, there are some known bugs in that old version and backports are really hard.

Soon there will be Grizzly packages upcoming. More is to come…

P.S. To interact with your cloud, you need credentials, which are automatically sourced from /etc/bash.bashrc.local (it is admin:openstack) and then you use commands like
nova list and glance image-list
but there is also a web-interface that allows you to do most actions in a browser – even VNC, if you use KVM instead of the default lxc.

Making different openSUSE liveCDs

December 29th, 2012 by

In my last post I explored the various liveCD creation methods out there, and I really wanted to try one of the others for openSUSE.
Thus I did so today in less than two hours.
I used Debian’s liveCD as basis and replaced the userspace with an openSUSE-11.4-GNOME-liveCD one (later ones likely do not work as systemd is not compatible with old 2.6.32 kernels).
And it worked like a charm. If you want to try it yourself, you need openSUSE and an empty directory with 5GB free space. Then you do as root:

zypper -n in clicfs squashfs cdrkit-cdrtools-compat
wget -O Makefile http://lsmod.de/bootcd/Makefile.aufslive.11.4
make

This will take a while to download the two isos and then at least another 3 minutes for the processing.
If that seems too hard for you, you can just download the finished iso and try it with qemu-kvm -m 1000 -cdrom xxx.iso

Do not let the debian logo in the bootloader confuse you. Just press enter there.
When running in KVM from RAM, this boots up in 18 seconds, while the original iso took 33 (measured from pressing enter in bootloader to the time the CPU load goes down). However, with physical media the difference will be less pronounced. Some of the difference comes from the faster gzip decompression. Unfortunately debian’s kernel does not support squashfs-xz, so I could not try that.

I hope in the future, we will have aufs patches in our normal openSUSE kernels and add an aufs-live mode to kiwi. That would help with the problems we hit with clicfs when memory runs out (and it can not be freed by deleting files either).

LiveCDs

December 27th, 2012 by

As few of you might know, I made my own SUSE-based LiveCDs a while ago, using (like Knoppix) cloop compression with iso9660 and my own kernel code for file-based overlay to make it writeable. You might be amazed at how fast it runs in KVM. However, the kernel part has bit-rotten and there are other techniques out there today, so I took a look around at how others do their LiveCDs.

But first some broader overview. To make a LiveCD, the biggest problem is that CDs are not writeable (and even modern Flash devices do not want to be written too much). Embedded devices using flash had the same problem. Various approaches have been used in the past to solve this:

  • adapt all software to write into ram-disks e.g. by having symlinks (hard to create and maintain)
  • load all software into RAM (only for small distributions)
  • use file-based overlaying such as unionfs or aufs to have software write into RAM (lsof, pwd, and hardlinks can be tricky)
  • use block-based overlaying (problem: can not easily free disk space again)

Also compression is used to fit more onto a CD. And interestingly, this usually also speeds up booting because it is faster to read 10MB off a CD and decompress it into the original 30MB than to read 30MB from such a slow medium.

Now, to the distributions.

  • openSUSE has the classic DVD installs that use special installation-images and run in RAM and then there are the real LiveCDs that are created by our kiwi tool, use block-based overlaying and LZMA compression of a ext3 by means of our FUSE-based clicfs.
  • All of the other distributions use squashfs for compression. Mageia employs dracut for initrd and unionfs for file-based overlaying
  • Debian uses aufs for file-based overlaying
  • Ubuntu uses overlayfs for file-based overlaying
  • Fedora uses an ext4 filesystem image contained in a squashfs with dm-snapshot for block-based overlaying, thus being most similar to openSUSE

I also spent some time benchmarking (on my AMD A10-5800K) the various technologies with a simple script using Debians uncompressed rootfs of 495132 KiB as data.
squashfs supports three different compression methods: lzo, gzip and xz (aka LZMA).

  • squashfs-lzo: size:220992 compression:11.1MB/s decompression:134.4MB/s
  • squashfs-gzip: size:203328 compression:15.5MB/s decompression:88.9MB/s
  • squashfs-xz: size:176064 compression:6.5MB/s decompression:22.5MB/s
  • cloop(gzip): size:213348 compression:16.2MB/s decompression:49.6MB/s
  • clicfs(xz): size:185300 compression:16.7MB/s decompression:18.2MB/s

This has some surprises: even when using the same compression method, sizes can differ by 5% and speed can differ even more.

If you want to compare numbers on your system, memory throughput is also interesting:
# dd if=/dev/zero of=/dev/null bs=1M count=100000
104857600000 bytes (105 GB) copied, 12.4499 s, 8.4 GB/s

Overall, clicfs is performing OK, considering that it already takes care of the overlaying, but for my own LiveCD I would prefer Debian’s method and I am wondering how it would work.

How to peek into remote isos

July 17th, 2012 by

When people want to provide a collection of files, they sometimes choose to do so by providing a .iso image file. But if you only want to look what files are in there or only need a few files, e.g. kernel and initrd for PXE-booting, you still had to download the whole thing to loop-mount it.

But you don’t have to anymore. Because modern web servers support delivering only parts of a file (using the “Range” header field), that allowed me to implement curlwwwfs that mounts remote HTTP directories into your local filesystem. And then you can use fuseiso on top to access the actual content within the .iso. All without root access.

This is how it works:

First you have to install the required packages (replace 12.2 with your version of openSUSE (or if you use a different Linux distribution, do git clone git://github.com/bmwiedemann/curlwwwfs and “make install” in there)):

zypper ar http://download.opensuse.org/repositories/home:/bmwiedemann/openSUSE_12.2/home:bmwiedemann.repo

zypper in curlwwwfs fuseiso

# Then you start it:

mkdir mnthttp mntiso

curlwwwfs http://zq1.de/bootcd mnthttp &

ls -la mnthttp/

fuseiso mnthttp/bmwinux-8.2-040808.iso mntiso

cat mntiso/isolinux/isolinux.cfg

# and later you clean it up with

fusermount -u mntiso

fusermount -u mnthttp

openSUSE ARM image

January 21st, 2012 by

When I wrote this week, how I ran openSUSE on my genesi smarttop some people asked for a ready-to-use image. After spending less than 8 hours fiddling with u-boot-scripts, partition tables, tuning ext3 and initrds, it was done… and is now so easy:

wget http://www.zq1.de/efika.img.xz # 83MB
xz -cd efika.img.xz | dd of=/dev/sdX bs=1M

with sdX being the device name of your SD-card (e.g. “mmcblk0” on the smarttop itself) with at least 1GB (actually 1024000000 bytes) of free space.

When inserted at boot, it should just boot up within 23 seconds and let you login as root with password “linux” on SSH, serial and with a USB-keyboard on HDMI. I spent some effort on putting as few packages as possible into it. Still, you have zypper to install packages and nano to edit files.

There is still a known hangup when you try to reboot. Workaround is: init 2 ; sleep 12 ; killall rsyslogd ; umount /boot/ ; mount -o remount,ro / ; reboot

As it still uses the original linux-2.6.31 kernel, it has another bug that also happens with pre-installed Ubuntu: sometimes (in ~40% of cases), boot stopps early, before graphics is initialized, when the last line on serial is “console handover: boot [ttymxc0] -> real [tty1]”. Try turning it off and on again.

This should allow you to have a whole lot of fun…

running openSUSE on ARM

January 19th, 2012 by

This week I finally got my genesi efika MX box. By default it has on old Ubuntu version installed on its internal IDE-attached 8GB SSD. It features 512 MB RAM and a 800 MHz ARMv7 CPU.

Using a HDMI-cable and an HDMI-DVI-Adaptor I got it connected to a monitor, plugged in a USB keyboard+mouse and it pretty much worked out of the box with WLAN,Ethernet,X11 (except for a bug that causes it to force you to change PW on every console login). How boring.

Having read about recent progress with openSUSE on ARM I wanted the excitement of running it on this box.

Michal’s image and script (now in alpha) was very helpful to get me started within 15 minutes.

If you have any (e.g. x86) openSUSE system running, there is another easy way to create a working ARM chroot-environment:

zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_12.1/openSUSE:Tools:Unstable.repo
zypper install qemu osc
osc co openSUSE:Factory:ARM bash
cd openSUSE:Factory:ARM/bash
edit bash.spec # add lines with your packages like BuildRequires: zypper,vim
osc build –no-verify –clean standard armv7l

If it worked well, /var/tmp/build-root/ should contain a chroot environment. E.g. you can run

file /var/tmp/build-root/bin/bash
/var/tmp/build-root/bin/bash: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, BuildID[sha1]=0xed9ca12f44c8591560d780cf807b6b6cf8ca8873, stripped

I partitioned my SD-card into two partitions. The first one for /boot with ext2 (needs only 150MB) and the second one for / to contain the rootfs. Be sure to have barrier=0 in your fstab for all ext[34] partitions so that writing to SD will not be as slow. The default U-Boot configuration first checks on the first partition of an SD-card for boot.scr which is a uImage-formatted version of a U-boot script. I copied all of Ubuntu’s /boot and /lib/modules/, slightly adapted their boot.script file to have root=/dev/mmcblk0p2 and uImage/uInitrd-2.6.31.14.26-efikamx as kernel/initrd, dropped “quiet splash” and added “console=ttymxc0,115200” to see more of the boot and ran a line from another helpful site:

mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n “my boot script” -d boot.script boot.scr
echo mxc0:S12345:respawn:/sbin/agetty -L 115200 ttymxc0 vt102 >> /etc/inittab # for serial console

However, this failed to boot. Using the serial debug console, I could see U-boot trying to load the boot.scr but it was thinking it was zero bytes for some strange reason. Re-creating my /boot partition as a raw copy of /dev/sda1 with my adaptions ontop finally gave me an SD-card that just boots openSUSE Factory on ARM with framebuffer console on HDMI/DVI.

Find more ARM-related info on our openSUSE ARM Portal

Where our bugzilla needs improvement

April 15th, 2011 by

update 2012-02-07: Success: after bnc#732504 you can now open Advanced Search and find RESOLVED/DUPLICATE bugs by default. When bugs are finally solved and fixes released, those can then be moved to CLOSED state to no more appear in search results.

 

bugzilla.novell.com aka bnc is the central tool to track bugs of openSUSE.

It has a guided bug submission form that helps & encourages reporters to search for existing reports on an issue.

However, the integrated search function only shows open bugs. In principle that is nice, but bugs marked as duplicates of open bugs do not count as open. Also it is common that developers mark a bug as RESOLVED/FIXED as soon as they uploaded a patch into the OBS devel-project. Such a patch then needs some days until it gets into Factory or the Update repos where normal users would benefit from the fix. During that time, there will still be people with all regular updates installed hitting the bug and not finding it on bugzilla because it is no more marked open. Of course, one could also search for RESOLVED bugs, but this brings up a huge list of issues that are long solved, but never were marked CLOSED (e.g. openSUSE-11.3 has 453 CLOSED vs 2727 RESOLVED).

This wastes time of reporters writing a duplicate bug-report and wastes time of developers having to figure out that it really is a dup. This is probably why other projects recommend setting bugs to ASSIGNED/FIXED until the patch is released. Unfortunately bnc lacks this state.

Also bugzilla has an UNCONFIRMED state for new bugs that need triaging to get into the NEW state (named “Reproducible but not assigned” in other projects). Such a state is unavailable on bnc, so that people can not tell apart bugs that have been forgotten from bugs that are known to exist, but just wait for a developer to fix.

There was also little integration between OBS and bnc. I have therefore written scripts to update bugzilla entries with links to submit requests on OBS mentioning e.g. bnc#685133 (<=see link for how it looks like). But it still could be made a lot more useful.

Overall, there is some room for improvement to make people’s lives easier, and allow us having a lot of fun…

openQA is testing for you

January 20th, 2011 by

You might have read the announcement of the factory-tested repo (or not, because it did not go on planet.o.o). There have been many additions to the testing during the last eight weeks.

openQA.o.o is a machine running automated software tests all the time to decide if factory-tested can be updated. If you think, this was boring, you might find out that it is not, since every testrun generates a video of approx 4 minutes that can be conveniently viewed in firefox.

You can view those test results on the openQA page.

And those are the updates:

  • I have added testing of development repos, so that it is now possible to detect bugs before they even get into Factory (most of those tests have “devel” at the end of their name). In fact, this has already found two such bugs.
  • The test system’s hardware was upgraded with a sponsored SSD and the software adapted to allow running multiple tests in parallel (using make -j3)
  • I have updated the web-interface to use the nice Bento theme
  • And I found yet another cool use-case for automated testing: it allows to bisect bugs. I did a binary search in old results for a live-installer bug which made it really easy to find the causing commit. However, seeing how long such a bug went unnoticed, made me think, that it would be nice if more people would check the results for anomalies, since not all of those can easily be auto-detected. As this just needs a firefox and little time, this is pretty easy to do. Much easier than to actually setup a test-system in any case.
  • An IRC notification bot was added, keeping #opensuse-openqa updated with new test results
  • Then, I am looking for additional automatic test-scripts, that also work standalone on an openSUSE install. It is especially important to test high-impact things. Things that break often or block many users if they break. And it would be nice to have a small result range such as OK|fail|unknown or it should at least be mappable to such.
  • Jürgen Weigert has packaged sikuli, so it is easy to create GUI tests with it, that could then be included in the test-suite.

Then there are still some things on the ToDo list…

  • Hermes integration to get email alerts and RDF feeds for newsreaders
  • Have more test variants auto-scheduled (-live and -dup)
  • Find a way to have Add-On-Repos with custom priority (otherwise Factory packages might be used, as happened for kernel-default-2.6.37-rc7) – using linuxrc’s driverupdate feature could be one way.

your input is highly appreciated