Home Home > Distribution > Factory
Sign up | Login

Archive for the ‘Factory’ Category

Some news from the trenches

February 7th, 2014 by

As you might know, we are focusing our development efforts in two fronts, namely openQA and staging projects. As we just started we don’t have fireworks for you (yet) but we did some solid ground work that we are going to build upon.

Working on openQA

We are organizing our daily job in openQA into highly focused sprints of two weeks. The focus of the first sprint was clear: cleanup the current codebase to empower future development and lower the entry barrier for casual contributors, which can be translated as “cleaning our own mess”. We created some tasks in progress, grouped in a version with a surprising and catchy name: Sprint 01.

Got my mojo workin’

Up to now, openQA web interface was written using just a bunch of custom CGI scripts and some configuration directives in Apache. We missed a convenient way to access to all the bell and whistles of modern web development and some tools to make the code more readable, reliable and easier to extend and test. In short, we missed a proper web development framework. We evaluated the most popular Perl-based alternatives and we finally decided to go for Mojolicious for several reasons:

  • It provides all the functionality we demand from a web framework while being lightweight and unbloated.
  • It’s stated as a “real time framework” which, buzzwords apart, means that is designed from the ground to fully support Comet (long-polling), EventSource and WebSockets. Very handy technologies for implementing some features in openQA.
  • It really “feels” very close to Sinatra, which makes Ruby on Rails developers feel like at home. And we have quite some Rails developers hanging around, don’t we? Just think about OBS, software.o.o, WebYast, progress.o.o, OSEM, the TSP app…
  • Mojolicious motto is “web development can be fun again”. Who could resist to that?

We’ve now reached the end of the sprint and we already have something that looks exactly the same than what we had before, but using Mojolicious internally. We are very happy with the framework and we are pretty confident that future development of openQA will be easier and faster than ever. OpenQA has mojo!

The database layer in openQA

Another part that we worked on during first sprint was the database layer. The user interface part of openQA use a SQLite database to store the jobs and workers registered in the system. The connection between the code and the database was expressed directly in SQL using a simple API.

We have replaced this layer with another equivalent that uses an ORM (Object-relational mapping) in Perl (DBIx::Class). Every data model in openQA is now a true object that can be created, copied and moved between the different layers of the application. Quite handy.

To make sure we don’t forget anything, we created a bunch of tests covering the whole functionality of the original code, running this test suite after each step of the migration. In this way we have achieved two goals: we now have a simple way to share and update information through the whole system and we can migrate very easily to a different database engine (something that we plan to do in the future).

What to do with staging

Over the time, coolo had accumulated quite some scripts that helped him with Factory. Most of them are actually related to something we are doing right now: the staging projects. So in the end we basically migrated all relevant ones to github and one by one we are merging their functionality to staging plugin. We also experimented with test frameworks that we could use to test the plugin itself, selected few and we even have a first test! The final plan is to have the whole plugin functionality covered with a proper test suite, so we will know when something breaks. Currently, there is a lot of mess in our repo and the plugin itself need big cleanup, but we are working on it.

Contributions are welcome

If you want to help but wonder where to start, we identified tasks that are good to dive into the topic and named them “easy hacks”: mostly self contained tasks we expect to have little effort but we lack the time to do right now. Just jump over the list for openQA or staging projects.

For grabbing the code related to staging project, you only need to clone the already mentioned repository. The openQA code is spread in several repositories (one, two, three and four), but setting up your own instance to play and hack is a piece of cake using the packages available in OBS (built automatically for every git push).

If you simply want to see what we are doing in more detail, take a look at progress.o.o, we have both openQA and Staging projects there.

We are having a lot of fun, and we encourage you to join us!

Trying to add some light

February 3rd, 2014 by

Lately there was some confusion regarding our communication. We, at the openSUSE Team@SUSE are deeply aware that our communication needs to be improved. So in the hope to make everything clear again, here is the summary to clear up what is really going on and what was not happening.

Long story short:

  • There WILL be openSUSE 13.2 in November 2014
  • 13.2 WILL have security and maintenance support provided by SUSE
  • We WILL have coolo as release manager for 13.2
  • SUSE is NOT decreasing manpower put into openSUSE
  • Everybody from the community is welcome and encouraged to be involved with, and if they want to, take over some parts of the release process and we will support you the best we can in doing that

Now for the long story.

Our team and only our team – openSUSE@SUSE – is going to work on improving ‘tooling’ side of the openSUSE project until August. These changes will benefit openSUSE by making it easier to produce better releases in the future.

Nothing changes for the rest of SUSE. SUSE is not abandoning openSUSE. The rest of SUSE will still do the same things they were doing until now and continue to keep openSUSE awesome. This includes Maintenance, Security, Infrastructure, and many other teams besides the openSUSE Team at SUSE who actively support the openSUSE project.

What is our plan?

Our plan is to make sure that future openSUSE releases are easier for everyone to produce. As we grow we could keep putting in more and more full-time release managers (if we find them somewhere), but this approach is probably unsustainable and, more importantly, goes against our desire to empower the community to do more as part of openSUSE.

Therefore, we decided to improve our tools to ensure that making a release is much more straightforward and reliable and we can reduce and distribute the workload needed for integration and release. To make this happen we need time and everyone from the team to work on adapting the tooling side. We also would welcome volunteers to help us with tools and with the following release(s).

With the release date now set in November (mirroring roadmap for 13.1), first milestone should be released in May. That is a perfect oportunity to go to openSUSE Conference in Croatia where we can meet up, gather volunteers to help and discuss how to work. Remember that openSUSE Travel Support is in place to sponsor everyone who needs financial help to get to the event.

Hopefully now we cleared things up a little and we are really sorry again for our poor communication – We’re going to work on it.

Your truely confused openSUSE Team

Discussing about the future of openSUSE

December 11th, 2013 by

This week, the openSUSE team blog is written by Agustin, talking about the proposals the team has done for openSUSE development.

A few months ago the openSUSE Team started a journey that achieved an important milestone last Tuesday, Nov 26th 2013. We have worked on creating a picture of relevant areas of the project in 2016 together with some of the actions we think should be taken during the following months to achieve it. To stop working and raise your head once in a while to analyze what is around you and setting a direction is a very good exercise.

The process we followed

The first step was working on data mining. After many hours of analysis, we identified some clear trends that helped us to establish a solid starting point to begin to work with. Once that phase was over (this is an ongoing process, in fact), we worked for a few weeks/months in trying to define that future picture interviewing several dozens of people. We refined that first attempt through several iterations, including many of those who participated in the original round and others who didn’t. Susanne Oberhauser-Hirschoff was the person who drove that process with Agustin.

We soon realized that discussing high level ideas in a community used to “Get shit done” was going to be easier if we complement them with some more down to earth proposals, specially in technical aspects. We cannot forget that, after all, openSUSE is a technical (and very pragmatic) focused community.

So, in parallel with the already mentioned refinement of the big picture, we started discussing within the team the actions needed to take to make the big picture a reality, the openSUSE development version a.k.a Enhanced/New Factory. After many hours of (sometimes never ending) discussions, we agreed on the ideas we are currently being published, together with the motivations behind them.

Another aspect we tried to bring to the discussion has been a strong dose of realism, trying to ensure that whatever we came up to was compatible with the nature of the project. We have also put focus on making sure that the initial proposal is achievable. So as part of community, we understand very well we cannot succeed alone. We need to work with you. So we just opened with the community a process analogous to what we went through within the team. It might be different in form but similar in principles and goals.

What are we going through these days?

These days the proposals are being discussed in different mailing lists. We are collecting feedback, discussing it, summarizing it, adapting the proposal to it … trying to reach agreements before defining what to do next.

What the proposal looks like?

We divided the proposal in a series of smaller proposals we are publishing in the project mailing list, where the general community topics in openSUSE are discussed, and/or factory ML, where the more technical discussions take place.

  1. openSUSE 2016: taking a picture of openSUSE today
    This mail summarizes the analysis phase we went through. We have tried to provide a simple picture of openSUSE today so the following articles can be justified to some extend.
  2. openSUSE 2016 picture
    This text summarizes the proposed picture for the end of 2016 (in three years). The goal is to set a direction for openSUSE

  3. openSUSE Development Workflow

  4. O Factory – Where art Thou?
    Stephan Kulow summarizes the Action Plan for the first aspect pointed in the previous picture: the new Development process (Factory).

The following articles describe in more detail some relevant (also new) elements pointed in the previous article, since they are new or modify the current process significantly. Some of the articles are in the queue to be published.

  1. One of the options for staging projects
    In this mail Michal Hrusecky provides some details and examples on how the new staging projects might work in the future.
  2. openQA in the new proposal
    This text, written by Ludwig Nussel, explains the principles that should drive the inclusion of openQA in the Factory development process, according with the proposed workflow.
  3. Karma for all
    This mail, written by Ancor González, summarizes our ideas to include a social feature in the process to help achieving Factory goals.
  4. Policies, or why it’s good to know how to change things
    The new process needs to be adaptive. Antonio Larrosa proposed a way, taking what other projects do in this regard as reference.

There might be an eighth article describing some smaller, still relevant, ideas. After publishing the “content”, we will release one last article providing a information about how to achieve these ideas, describing also our compromise in terms of effort and pointing out the challenges we perceive in the plan from the execution point of view.

We would like to invite you to the debate if you haven’t raised your opinions yet.

New Raspberry Pi Image

September 7th, 2013 by

update: new image with kernel-3.6 and minimal X11/icewm http://www.zq1.de/~bernhard/linux/opensuse/raspberrypi-opensuse-20130911x.img.xz (103MB)

We got a new armv6 based image for the Raspberry Pi.
This one is only 82MB compressed, so pretty minimalistic.

http://www.zq1.de/~bernhard/linux/opensuse/raspberrypi-opensuse-20130907.img.xz

The exciting new thing is that this was created using an alternative image building automatism which I wrote from scratch in three hours this morning.
The scripts can be found at

https://build.opensuse.org/package/show/devel:ARM:Factory:Contrib:RaspberryPi/altimagebuild

and are also embedded within the image under /home/abuild/rpmbuild/SOURCES/

This means that everyone can now easily build his own images the way he likes and even branch and do submit requests for changes that are useful for others.
The way to use this is simple.
If you have 6GB RAM, you can speed things up with export OSC_BUILD_ROOT=/dev/shm/arm before you do
osc co devel:ARM:Factory:Contrib:RaspberryPi altimagebuild
cd devel:ARM:Factory:Contrib:RaspberryPi/altimagebuild
bash -x main.sh

This pseudo-package does not easily build within OBS or osc alone because it needs root permissions for some of the steps (chroot, mknod, mount), which could only be workarounded with User-Mode-Linux or patching osc.
The build consists of three steps that can be seen in main.sh:

  1. First, osc build is used to pull in required packages and setup the armv6 rootfs.
  2. Second, mkrootfs.sh modifies a copy of the rootfs under .root to contain all required configs
  3. And finally, mkimage.sh takes the .root dir and creates a .img from it that can be booted

This can build an image from scatch in three minutes. And my Raspberry Pi booted successfully with it within 55 seconds.

There are some remaining open issues:

  • the repo key is initially untrusted
  • still uses old 3.1 kernel – solved
  • build scripts have no error handling

Compared to the old image, this one has some advantages:

  • It is easier to resize because the root partition is the last one
  • Compressed image is much smaller
  • Reproducible image build, so easy to customize
  • It is armv6 with floating point support, so could be faster
  • We have over 5200 successfully built packages from openSUSE:Factory:ARM
    so for example you can install a minimalistic graphical environment with zypper install xauth twm xorg-x11-server xinit and start it with startx

So if you wanted to play with openSUSE on RPi, you can do so right now and have a lot of fun.

Keeping Factory in shape

June 13th, 2013 by

Michal Hrušecký has been helping out on maintaining Factory in shape and shares his experiences.

Factory is development version of openSUSE and it is where the next openSUSE is taking form. Hundreds of packagers send packages into Factory to be integrated as a part of the new release and many more use Factory for testing or for their daily work. Thus it is really important to keep Factory rolling and usable. Everybody knows that Coolo is the Factory master and he does everything to make next openSUSE be the best ever. But keeping factory in shape is really complicated and stressing task. There are dozens of request everyday and each one of them can potentially break something. So Factory can always use a pair of extra hands and for some time I have been one of them. I’d like to give you some insight in what we do, working on keeping Factory building and working. (more…)

openQA in openSUSE

June 6th, 2013 by

factory-testedToday, we’ve got for you an introduction of the teams’ work on openQA by Alberto Planas Domínguez.

The last 12.3 release was important for the openSUSE team for a number of reasons. One reason is that we wanted to integrate QA (Quality Assurance) into the release process in an early stage. You might remember that this release had UEFI and Secure Boot support coming and everybody had read the scary reports about badly broken machines that can only be fixed replacing the firmware. Obviously openSUSE can’t allow such things to happen to our user base, so we wanted to do more testing. (more…)

AMD fglrx, fgrlx-legacy : news, cleanup & important informations

March 3rd, 2013 by

Status of fgrlx & fglrx-legacy regarding next coming 12.3

fglrx

fglrx (Catalyst 13.1) drivers has been refreshed and published for 12.3 with the RC2 build.
During the first few days after the release, and fresh new build will be made with the final version, and first updates.

fglrx-legacy

fglrx-legacy (Catalyst 13.1) will never support (actually) xorg 1.13 which is the version that come in openSUSE 12.3. Even if it can handle kernel 3.8
So the previous build has been removed from the server. To insure end-users no trouble or hassle trying to get it working.
If you still have a radeon from hd2xxx to hd4xxx you’re welcomed to use the free radeon. It made progress and could eventually be as efficient as the proprietary drivers.
The bonus you get, you can report bug, and they will be fixable.

Status of mirrors

One year ago I announced the move to the new host for the package mirror. During that time, I’ve kept a redirection active, and also a symlink from ati to amd-fglrx

This time is over now, so please update your repositories

Repositories available

FLGRX
http://geeko.ioda.net/mirror/amd-fglrx/ add openSUSE_(you version) from 11.2 to 12.3
FLGRX-LEGACY
http://geeko.ioda.net/mirror/amd-fglrx-legacy/ add openSUSE_(you version) from 11.2 to 12.2

Spread the word

If I already updated the en.opensuse.org wiki page (even if the reviewing process is stuck actually), I need your help to spread the word, to reach any end-users that need those informations.

Notice about tumbleweed, evergreen

I saw several users, trying to use the one-click-installer with tumbleweed. Sorry this can’t work due to the lack of perfect recognition of tumbleweed. etc/SuSE-release is 12.2 actually.

So if you use tumbleweed you just have to install the tumbleweed repository (again one for fglrx, one for fgrlx-legacy depending on your gpu)

But beware, tumbleweed is a moving target, and the proprio drivers could stop working at any update, kernel or xorg

Evergreen : some users successfully use fglrx-legacy 13.1 with the kernel 3.0.58

OpenStack on openSUSE

January 11th, 2013 by

Do you want to play with cloud software on your own machines?
Some people have been working to package the current OpenStack version “Folsom” for openSUSE (tested on 12.2) and add scripts to configure it into a working state.
You need 2GB of RAM and 3+ GB of free disk space under /var/lib/
Then you do

wget https://raw.github.com/SUSE-Cloud/automation/master/scripts/jenkins/qa_openstack.sh
export cloudsource=openstackfolsom ; bash -x qa_openstack.sh

This is a script we use for continous integration testing, but it is as useful to setup a simple environment for development, testing or demoing.
Folsom packages are still rather rough and might see some change over the coming weeks.

If you want the older stable version, you can use the above snippet with cloudsource=openstackessex
however, there are some known bugs in that old version and backports are really hard.

Soon there will be Grizzly packages upcoming. More is to come…

P.S. To interact with your cloud, you need credentials, which are automatically sourced from /etc/bash.bashrc.local (it is admin:openstack) and then you use commands like
nova list and glance image-list
but there is also a web-interface that allows you to do most actions in a browser – even VNC, if you use KVM instead of the default lxc.

Making different openSUSE liveCDs

December 29th, 2012 by

In my last post I explored the various liveCD creation methods out there, and I really wanted to try one of the others for openSUSE.
Thus I did so today in less than two hours.
I used Debian’s liveCD as basis and replaced the userspace with an openSUSE-11.4-GNOME-liveCD one (later ones likely do not work as systemd is not compatible with old 2.6.32 kernels).
And it worked like a charm. If you want to try it yourself, you need openSUSE and an empty directory with 5GB free space. Then you do as root:

zypper -n in clicfs squashfs cdrkit-cdrtools-compat
wget -O Makefile http://lsmod.de/bootcd/Makefile.aufslive.11.4
make

This will take a while to download the two isos and then at least another 3 minutes for the processing.
If that seems too hard for you, you can just download the finished iso and try it with qemu-kvm -m 1000 -cdrom xxx.iso

Do not let the debian logo in the bootloader confuse you. Just press enter there.
When running in KVM from RAM, this boots up in 18 seconds, while the original iso took 33 (measured from pressing enter in bootloader to the time the CPU load goes down). However, with physical media the difference will be less pronounced. Some of the difference comes from the faster gzip decompression. Unfortunately debian’s kernel does not support squashfs-xz, so I could not try that.

I hope in the future, we will have aufs patches in our normal openSUSE kernels and add an aufs-live mode to kiwi. That would help with the problems we hit with clicfs when memory runs out (and it can not be freed by deleting files either).

openSUSE ARM image

January 21st, 2012 by

When I wrote this week, how I ran openSUSE on my genesi smarttop some people asked for a ready-to-use image. After spending less than 8 hours fiddling with u-boot-scripts, partition tables, tuning ext3 and initrds, it was done… and is now so easy:

wget http://www.zq1.de/efika.img.xz # 83MB
xz -cd efika.img.xz | dd of=/dev/sdX bs=1M

with sdX being the device name of your SD-card (e.g. “mmcblk0″ on the smarttop itself) with at least 1GB (actually 1024000000 bytes) of free space.

When inserted at boot, it should just boot up within 23 seconds and let you login as root with password “linux” on SSH, serial and with a USB-keyboard on HDMI. I spent some effort on putting as few packages as possible into it. Still, you have zypper to install packages and nano to edit files.

There is still a known hangup when you try to reboot. Workaround is: init 2 ; sleep 12 ; killall rsyslogd ; umount /boot/ ; mount -o remount,ro / ; reboot

As it still uses the original linux-2.6.31 kernel, it has another bug that also happens with pre-installed Ubuntu: sometimes (in ~40% of cases), boot stopps early, before graphics is initialized, when the last line on serial is “console handover: boot [ttymxc0] -> real [tty1]“. Try turning it off and on again.

This should allow you to have a whole lot of fun…