Home Home > Server > Virtualization
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Archive for the ‘Virtualization’ Category

OBS 2.1: ACL Feature and Status

August 15th, 2010 by

One and a half year is now gone since I posted about my work for ARM support in the OBS and the work for a port of openSUSE to ARM. Lots of things had happened in the meantime that are related, from my limited view most notably Nokia and Intel joining Moblin and Maemo to MeeGo (MeeGo is currently working on a number of Atom and ARM based devices), chosing to use OBS as build system and last but not least myself joining The Linuxfoundation (you will be not surprised to hear that I work at LF on OBS). In the meantime there had also been a major new OBS release 1.8/2.0 with a bunch of new features.

Interesting is the fact that we adapted the cross build system for OBS to MeeGo, first developed for use in Maemo and openSUSE @ ARM. An improved version for the standard MeeGo releases, and for the MeeGo weekly snapshots is used in the MeeGo OBS System to build all ARM releases of MeeGo (the cross toolchain will later get part of the MeeGo SDK @ ARM), thanks to Jan-Simon Möller (In the openSUSE ML, the issue of reactivating openSUSE:Factory ARM builds were brought up. So it might be a good variant to backport Jan-Simons new solution back into openSUSE @ ARM for that purpose). All the MeeGo related OBS installations will move sooner or later to OBS 2.1.

But now to the most recent work, Access Control support. A preview was shipped with OBS 1.8. Now an own OBS version, 2.1, will be dedicated to the introduction of this single new feature into the OBS mainline: Access Control (or abbrevated ACL for Access Control Lists). ACL means that there is control by the user on a per project or per package basis to protect information, source and binaries from the read access of other users in an OBS system and to hide projects or packages.

What is the intended audience of ACL? ACL is intended for installations of OBS that require protection of projects or packages during work. This can be but is not limited to commercial installations of OBS, or semi public installations of OBS.

How does ACL work? ACL sits on top of two features introduced with OBS 2.0: Role and Permission Management as well as freely definable user groups. ACL uses 4 specifically defined permissions (‘source_access’ for read access to sources, ‘private_view’ for viewing package and project information, ‘download_binaries’ for read access to binaries and ‘access’ permission to protect and hide everything and all from read access and viewing) on a user or group in the Role and Permission management. Also, the preexisting roles “maintainer”, “reader” and “downloader” had been modified with specific predifined permissions (which can at any time changed with the role and permission editor dynamically). And last but not least 4 new flags (namely ‘sourceaccess’ to signal a project/package has read protected source code, ‘binarydownload’ to signal it has read protected packages, ‘privacy’ to signal information/logfiles or status cannot be read and ‘access’ to hide and protect a project or package completely in all possible OBS API calls) had been added to the project and package descriptions to signal that some information is only readable by specific users or groups, or that information is hidden.

How do I use ACL? There are 4 steps to use ACL (a part of them a optional and can only be performed by the Administator of an OBS instance). Step one is to assign the listed permissions to a role, user or group (this step can be done only by the admin, and is not needed for the predefined roles “maintainer”, “reader” and “downloader”). Step two is to add a group for special users to projects which are intended to be run with ACL (this operations can only be performed by the admin). Step three is to protect a project with appropriate protection flags at project creation by adding them to the project meta. Step four is to add other users or groups with one of the new predefined roles that has ACL permissions added to the project meta.

What information can be protected by ACL? The protected information is grouped into 4 categories. Category 1 (flag ‘sourceaccess’) is source code. Category 2 (flag ‘binarydownload’) is binary packages or logfiles or builds. Category 3 (flag ‘privacy’) is project or package information like build status. Category 4 (flag ‘access’) is all viewable or accessable information to any project or package (full blocking of all access and information).

Example of a project configuration using ACL:

<user userid="MartinMohring" role="maintainer" />
<!-- grant user full write and read access -->

<group groupid="MeeGo-Reviewer" role="maintainer" />
<!-- grant group full write and read access -->

<group groupid="MeeGo-Developers" role="reader" />
<!-- grant group full source read access -->

<group groupid="MeeGo-BetaTesters" role="downloader" />
<!-- grant group access to packages/images -->

  <sourceaccess>
    <disable/>
  </sourceaccess>
  <!-- disable read access - unless granted explictely.
          This flag will not accept arch or repository arguments. -->

  <binarydownload>
    <disable/>
  </binarydownload>
  <!-- disable access - unless granted explictely -
          to packages/image and logfiles -->

  <access>
    <disable/>
  </access>
  <!-- disable access - unless granted explictely-,
          project will not visible or found via search,
          nor will any source or binary or logfile be accessable.
          This flag will not accept arch or repository arguments. -->

  <privacy>
    <enable/>
  </privacy>
  <!-- project will not visible.
          This flag will not accept arch or repository arguments. -->

What is the current status of the ACL implementation? The current status is that the complete API of the OBS git master had been instrumented with ACL code, critical portions of the API controllers had been code inspected and a big portion of these API calls now have a testcase in the OBS testsuite. Work is ongoing to make ACL as secure as possible. A code drop of current git master is under test in some bigger OBS systems, most notably the openSUSE Buildsystem. You can find snapshots of this codebase as usual in the OBS project openSUSE:Tools:Unstable. Adrian Schröter updates these “Alpha Snapshots” relatively often, on a 1-2 weekly basis, and runs the testsuite on git master daily. Thanks to Jan-Simon Möller for putting in many of the testcases into the testsuite for the ACL checks. On OBS Testing in general, read also Development and Test.

What is next? Code is tested and debugged against granting unwanted access due to some concepts inside OBS that are “working against ACL”, like project or package links, aggregates or kiwi imaging. We will inform you interested user of course about beta releases and an official 2.1 release.

Stay tuned.

automated openSUSE testing

May 25th, 2010 by

Testing is an important task. But testing daily openSUSE-Factory snapshots would mean testing the same things every day. This would be pretty tiresome to people.
And there is a lot of software to test, including software unknown to most testers or new versions of known software, so how should the tester know if the results were the intended results?
My answer is: leave as much as possible to computers. Computers do not get tired. Computers do not stop testing something after a dozen identical results. Computers do not forget.

The following assumes that you have read my text on making openSUSE install videos.

So far, I am rather satisfied with my automated installations.
At the end of those, I added some basic application testing, which already showed in MS7
openSUSE-KDE-LiveCD-x86_64-Build0625a.ogv dated 2010-05-21 16:08
an issue filed 28 hours later at bnc bug 608087

Only that it currently still needs a human to look at the results.
I was thinking to improve upon that by scanning (rectangular) parts of the screenshot for known good or bad images. If either is found, the test could be automatically marked as passed or failed.
On unknown images, a human would still need to decide which part of the image is relevant and if it is good or bad. This decision can then be used to avoid human interaction (hard work) in further runs of that test.
If we push this further, it could be similar to nagios for network monitoring. Telling when something breaks and telling when something is back working. It could have an overview page about automated test status, giving totals e.g. “50 working, 10 unknown, 3 failing”. With links to more details.

The advantage in adding the application tests after the install test is that the system starts out in a clean, reproducible way. One disadvantage I see is that a newly failing test could prevent following tests to work.

I have also been working to enable others to run my isotovideo script. For that I have cleaned up my code so that it no more contains paths from my system. The other thing is that I documented how to get it working at http://www3.zq1.de/bernhard/git/autoinst/INSTALL

MS7 installation videos:

openSUSE-KDE-LiveCD-x86_64-Build0625c.ogv
openSUSE-KDE-LiveCD-i686-Build0625a.ogv
openSUSE-GNOME-LiveCD-x86_64-Build0625b.ogv
openSUSE-NET-x86_64-Build0623b.ogv
openSUSE-NET-i586-Build0623b.ogv
openSUSE-DVD-Build0625-x86_64b.ogv
openSUSE-DVD-Build0625-i586b.ogv

Into the Cloud

March 29th, 2010 by

Setting up your own Cloud infrastructure on SUSE has just become a lot easier. You can now use Kiwi and a mostly pre-configured set-up to build your own Cloud node images. Once these images are built setting up your Cloud can be accomplished in a few minutes.

Checkout the Cloud Recipe in the Kiwi Cookbook.

Happy Hacking

Xen para-virtualized openSUSE 11.2

February 10th, 2010 by

I had to install Xen para-virtualized openSUSE 11.2 (PVM), lucky for me 11.2 DVD iso has broken xen kernel so it does not install, the live CDs do not have any xen kernel at all so they are not useful either. After reading up all the posts on the bugzilla and forums about the subject, found the way to get it done, here is a howto for anyone else who is looking for the solution.

1. Set up http installation source

Install web-server pattern from yast to install apache2 in Dom0

Edit apache configuration /etc/apache2/default-server.conf to follow symlinks, it should look something like this:

#Options none
Options Indexes FollowSymLinks

Mount the DVD iso and copy all the files to your webserver root.

mount openSUSE-11.2-DVD-i586.iso /mnt -o loop
mkdir -p /srv/www/htdocs/suse-11.2
cp -ar /mnt/* /srv/www/htdocs/suse-11.2/
cd /srv/www/htdocs/suse-11.2/boot/i386/
rm *-xen*
wget http://download.opensuse.org/distribution/11.2/repo/oss/boot/i386/initrd-xen
wget http://download.opensuse.org/distribution/11.2/repo/oss/boot/i386/vmlinuz-xen
ln -s initrd-xen initrd-xenpae
ln -s vmlinuz-xen vmlinuz-xenpae
cd /srv/www/htdocs/
wget http://download.opensuse.org/update/11.2/rpm/i586/kernel-xen-2.6.31.12-0.1.1.i586.rpm (or the latest)
mv kernel-xen-2.6.31.12-0.1.1.i586.rpm kx.rpm
rcapache2 start

2. Start installation via “yast2 vm-install”, select para-virtualization and as installation source use http://dom0IP/suse-11.2/
3. The hack part:
When the install is about 80%  or “Stop” the reboot after the completion of first stage install, switch to tty2 by using “Sendkey > ctrl+alt+F2”.

chroot /mnt
rpm -Uvh http://dom0IP/kx.rpm –force

Reboot and let the install run its course, at the end of it there should be working domU

Edit: If you have fast internet connection available during install, add the update repository as “Addon product”, with that in place the above hack will not be necessary.

Edit2: If the image still don’t boot, mount the disk image to edit grub’s menu.lst:

mount -o loop,offset=32256 /full/path/to/image/disk0 /mnt
cd /mnt
ln -s vmlinuz-….-xen vmlinuz-xen
ln -s initrd-…..-xen.img initrd-xen
vi /mnt/grub/menu.lst #to look like below:

title XEN
root (hd0,0)
kernel /vmlinuz-xen root=…..
initrd /initrd-xen

First post / First whishes

December 20th, 2009 by

Dear Santa-Claus, can you provide us a real full kvm solutions for next year.
So I didn’t have to fight with Esx .

Hybrid Live Systems

August 5th, 2009 by

When talking about live systems on USB sticks people reported many problems with bootloaders like grub to boot the stick. Even though this is most often a problem with the stick hardware or the PC BIOS it’s an annoying situation which should have a better solution. There are also many people who wants to use the stick as a data container in combination with a live system to work with

With the ISO hybrid technology and the integration into kiwi there is a way to create such a stick very easily. A hybrid ISO is an iso filesystem which contains a MBR and thus it’s seen as a disk to the PC BIOS. As it’s an ISO the isolinux bootloader is used instead of grub which works better on many systems. Additionally the hybrid ISO can be used as a live system on CD/DVD as well as on a USB stick

What’s required to use this

  • kiwi v3.68 or later
  • syslinux-3.82-2.1 or later

How do I setup a hybrid ISO in kiwi

In order to activate the creation of a hybrid iso in kiwi you only have to add the hybrid=”true” attribute as part of your iso image type in config.xml:


<type boot="isoboot/suse-11.2" flags="clic" hybrid="true">iso

You can use the suse-11.2-JeOS from the kiwi-templates package as example image description for your hybrid testing. The generated .iso file can be dumped via a simple dd call onto the USB stick. The same file also can be used to be burned on a CD/DVD


dd if=LimeJeOS-openSUSE-11.2.i686-1.11.2.iso of=/dev/... bs=32k

After that the stick can be tested. By default all attempts to write data will go into the RAM of the system. As a stick allows storing data persistently you can create a write partition on the stick using fdisk:


fdisk /dev/...

kiwi will prevent using a vfat partition for the operating system. So make sure you create a 0x83 (linux) type partition and not a vfat partition for the write support. If you additionally create a vfat partition you can use it as a container for any kind of data independently from the live system. We choose vfat here to stay compatible with Windows systems.

Known bugs

  • when using the clicfs filesystem (flags=”clic”) the persistently write feature into a single partition will fail because clicfs currently can’t deal with raw block specials as cow device. Will be fixed as soon as possible

Have fun 🙂

Fresh & Fruity

July 28th, 2009 by

Available today as part of the SUSE Appliance Program is SUSE Studio 1.0 based on the  image creator technology called kiwi. When creating an appliance with SUSE Studio you also have the possibility to export the appliance description to your local computer and use the kiwi backend directly to understand more about image creation and deployment

A professional linux distribution should be able to work as an appliance which is an ll-in-one solution including the application and the operating system. A basic appliance to start with is the JeOS – Just Enough Operating System. kiwi provides these as examples in the kiwi-templates package. To create your first SUSE 11.1 appliance just type:

kiwi --build suse-11.1-JeOS -d /destination/path

The primary image type of a JeOS template is a virtual disk which you can run in a virtual machine like QEmu, KVM, Vmware, VirtualBox, etc… To do this with qemu just call:

qemu /destination/path/LimeJeOS-openSUSE-11.1.i686-1.11.1.vmdk

and here you go with your first appliance. You want to know more about kiwi, just take a look at the wiki here:

KIWI Cookbook

or read the full system documentation as PDF here:

KIWI System Documentation

Remember to have fun 🙂

ARM support in openSUSE Buildservice – fixed

April 27th, 2009 by

The issue caused by the OBS worker update on arm builds is fixed by a new qemu.

This new qemu version also has fixed the Fedora 10 @ ARM build problem.

So we have the following working ARM target distros available for ARM: Fedora 10, Debian 5.0 and Ubuntu 9.04.

Have fun.

ARM support for openSUSE Buildservice and openSUSE – Status update

April 26th, 2009 by

Its a while since I posted the status about the ongoing work for ARM support in the OBS and for an openSUSE port. It all started with my participation in the OBS development as an external contributor. Then, on Hackweek 2008, we had the idea to enforce a new solution other than the traditional methods of compiling code either natively or via a cross compiler on a host system. The idea was to give build scripts as much of the target enviroment as they need to just work without changes in the packaging definition – in order not to change thousands of package descriptions which define a linux distribution.

A lot happened in the meantime. And I can now report some significant progess in bringing the joys of OBS and openSUSE also to all the ARM users:

  • I held a talk about cross build in OBS on FOSDEM 2009 – documenting the solution
  • ARM support is in the source tree for OBS and the publicly available packages
  • ARM support is activated in the public OBS
  • OBS 1.6 release is currently in beta – this release is the dedicated version for ARM
  • The Linux foundation will bring the joy of OBS to an even wider audience
  • Some preparations have been done for porting Base:build to ARM – we can mix cross compilers an native emulated code now
  • A Summer of Code project will be done to accelerate the development of an openSUSE @ ARM port
  • To accelerate the openSUSE @ ARM development itself, we want to involve more people of the community. We have an IRC Channel #opensuse-arm for OBS and openSUSE @ ARM – i invite you to visit us there. We will also find a solution to bring the needed changes into the openSUSE Factory codebase so regular build for openSUSE can take place once the base system is working. I will inform you once we have a working base system that can be used to port many other packages. The soon starting Summer of Code Project “Porting openSUSE to ARM platform” is intended as the starting point here.

    The next steps are to bring in all the useful applications into OBS, so you have the wide range of applications that is already available for x86 or powerpc then also on ARM. You will see interesting things happening during the next time here. To support this, more and more of the tested ARM targets will be made available also on the public OBS. I will follow up with status updates.

    ARM support in openSUSE Buildservice – currently broken

    April 26th, 2009 by

    With this message I want to make you aware that the ARM builds inside OBS are currently broken. This is due to an update of the buildservice worker code on Friday. This update removes the limit of 2 GB for the build results from the buildservice. Also, the performance of the buildservice backend code has been improved for high loads with lots of new events.

    We are now faced with an incompatibility of the underlying QEMU emulator with this new code to extract the build results in the combination of XEN and QEMU user mode. You can in fact see in your build logs for ARM error messages like:

    … saving built packages
    /usr/src/packages/DEBS/dsme-tools_0.6mer3_armel.deb
    Unsupported ioctl: cmd=0x0002 (0)
    FIGETBSZ: Function not implemented
    Unsupported ioctl: cmd=0x80041272 (4)

    We are working on a solution already. A new QEMU with this and another issues fixed is already under test and has been dropped to openSUSE:Tools:Devel/qemu-svn. I will inform you when we have this fixed in the public build service.