Home Home > Infrastructure-2
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Archive for the ‘Infrastructure’ Category

First fruits – update on openQA and Staging work

February 19th, 2014 by

In our previous summary, we talked about some basic research and some ground work to build on. This time we have some first exciting results!

openQA

Last week we rearranged the repository a little bit, creating a new branch called "devel" where all the exciting (and not so exciting) changes are taking place. Our little Factory 😉

The main difference between this branch as master is that, as you could read in the previous blog, the devel branch is openQA build on Mojolicious, a nice web development framework. And having a proper web framework is starting to show its benefits: we have openID login available! Unfortunately the current openSUSE openID provider is a little bit weird, so it doesn’t play well with our tool yet, but some others are working and openSUSE accounts will be the next step. Having working user accounts is necessary to be able to start defining permissions and to make openQA truly multiuser. And to be able to deploy the new version on a public server!

The other main focus of this week has been internal polishing. We have revamped the database layout and changed the way in which the different openQA components communicate with each other. The openQA functionality is spread out over several parts: the workers are responsible of actually executing the tests in virtual machines reporting the result after every execution; some client utilities are used to load new ISO images and similar tasks and, finally, we have the one openQA Server to rule them all. Until now, the communication between the server and the other components was done using JSON-RPC (a lightweight alternative to XML-RPC). We have dropped JSON-RPC in favor of a REST-like API with just JSON over plain HTTP. This change allowed us to implement exactly the same functionality in a way that is simpler, perfectly supported by any web framework, natively spoken by browsers and easier to authenticate (using, for example, plain HTTP authentication or openID). This is also the first step to future integration with other services (think OBS, as the ultimate goal is to use openQA to test staging projects).

But, who tests the tester? openQA is growing and changing quite fast so we have continued with the task of creating a good testing infrastructure to tests openQA itself to make sure that all our changes do not result in breakage. We only have a few tests right now, but we now have a solid foundation to write more and more tests.

Staging and package manipulation

In the last blog post we told you we were investigating a code test suite to test the abilities of a osc plugin we are writing. osc is the commandline tool for handling the Open Build Service, and this plugin is meant to help with the administration of staging projects. We’ve been thinking about how to move forward with the testing part as we want to make sure the functionality works as advertised. More important, we write tests to make sure that our additions and changes do not break existing functionality. We have started merging functionality from various scripts handling staging thingies and rings we had into this plugin. This is partially done so we can do basic staging project tasks! We can take a request (be it submit or delete) and put it to test in a staging project. We can also move packages between staging projects and we have a simple YAML in project descriptions to indicate what packages and what requests are there.

We're all green!
Coolo already started using the plugin for some tasks, so you can see pretty titles and metadata in staging projects descriptions. Not impressive enough? Let me provide you a good headline:

Thanks to staging projects, no single regression have been introduced this year in the core of Factory

You can enjoy a more detailed description in this announcement written by coolo to the Factory mailing list and have some visual joy in the included screenshot (genuine pr0n for QA engineers).

Last but not least, we also did some cleanup of the sources in the repo and of course we added more tests (as functionality grows). And there has been work on other parts of the plugin, like taking rings into account.

Result

We already have some useful functionality which we are going to expand and build on. Not that much to show yet, but we are progressing towards our goal. You can follow our progress either in the way of tasks (openQA and staging projects) or just follow our commit history on github (again for both openQA and staging plugin).

We are very much looking forward to feedback and thoughts – these changes aim to make Factory development easier and thus we need to hear from all you!

The openSUSE TSP application

June 20th, 2013 by

Introduction blog of the TSP
Today, Ancor Gonzalez Sosa writes about the Travel Support Program Application he developed with the openSUSE Team.

Traveling to an event to represent your project, sharing experiences with other people with common interests and showing them what you are passionate about is absolutely awesome – but it can get expensive. This is why openSUSE introduced a Travel Support Program last year.

The openSUSE Travel Support Program

The goal of the Travel Support Program is to support contributors representing openSUSE at events by reimbursing up to 80% of the travel and/or hotel costs. In turn the contributors make a worthy contribution at the event and report back to the openSUSE community about what they did.

We’re not alone in doing this, having drawn inspiration from GNOME’s Conference Travel Subsidy Program, the KDE e.V. Travel Cost Reimbursement initiative and the Travel Policy from The Document Foundation.

vanilla entering reimbursement request_crop

entering reimbursement request

The program is sponsored by SUSE, but the Travel Committee independently manages the money and decides who is supported and how. This is a lot of work: decisions involve the event itself, the contributor asking for support, other Geekos in the area, the costs and of course the entire budget. The team also has to plan the priority of events with the marketing team and communicate about the status of the requests and reimbursement.

And the Free Software world was lacking a proper tool to manage all this… until now!

The brand new TSP application

We developed a new web tool to make the life of the TSP team and the community easier and do this in an open and generic way so other projects could benefit as well. We’ve started using it already for the upcoming openSUSE Conference 2013 and you can see it in action here. It even offers a pretty diagram explaining the TSP process! Of course, the complete source code of the project can be found on Github.

Development

For a more detailed explanation of the goals of the project you can refer to the ‘about the TSP application‘ page in our projects management tool. In that page you will find ‘the 6 Ws’ of the new application: who, what, when, where, why and how (yes, we know that ‘how’ does not start with ‘W’, but we didn’t invented the 6Ws term).

During development we honored the motto “release early, release often” and worked following agile development principles. We begun by collecting ideas and requirements of the TSP team and the people handling the payments on the SUSE side. After developing a first prototype, it was presented in a video conference. You can find the minutes of this meeting in the project’s wiki.

Once the feedback was in and new goals were set, the prototype was deployed on a provisional server in order for the Travel Committee to test it. Using this test-drive installation, the application was improved in an interactive way. Every two weeks (sometimes a bit longer), a new version was installed. Izabel tested the new version providing very useful feedback used to plan the new milestone in our projects management tool and so on. This cycle is still in motion: new version, feedback, planning, new version…

bento style request status

bento themed request status

Awesome Rails goodies

While working on the TSP application we have developed some features that can be interesting for other Ruby on Rails programmers working within the openSUSE infrastructure, like the team behind OSEM. The TSP application includes a Devise backend for the openSUSE authentication infrastructure, a Bento theme for Bootstrap (written in pure Less) and integration with openSUSE Connect through its REST API. We plan to release all these features as individual components to allow reuse in other openSUSE developments.

Present and future, sharing with others

We plan on continuing maintenance of the application and as with most free software projects, it’s hard to predict in which direction the tool is going to evolve. Conference volunteers in charge of the visa invitation letters and the team in charge of merchandising shipping already made some interesting suggestions so it will not be a surprise if we end up developing a full event management tool. Not for registration and scheduling individual conferences -the oSC’13 guys are already doing a great job developing OSEM– but for the administrative tasks and planning behind the attending various events that communities like openSUSE do.

Concluding

So, if you are an openSUSE contributor and you might need sponsorship for traveling in the future, bookmark the TSP page! If you are a Ruby on Rails developer, just Fork it on Githubℱ and meet us at oSC’13 to talk about future collaboration. And if you are in charge of a travel support program for another open source project or are thinking about the possibility of starting one, you can run it yourself and we’d be happy to help you in case of trouble. You can find me (Ancor Gonzalez Sosa) as ancorgs in the openSUSE-Project channel on Freenode.

And always remember: have a lot of fun!

openQA in openSUSE

June 6th, 2013 by

factory-testedToday, we’ve got for you an introduction of the teams’ work on openQA by Alberto Planas DomĂ­nguez.

The last 12.3 release was important for the openSUSE team for a number of reasons. One reason is that we wanted to integrate QA (Quality Assurance) into the release process in an early stage. You might remember that this release had UEFI and Secure Boot support coming and everybody had read the scary reports about badly broken machines that can only be fixed replacing the firmware. Obviously openSUSE can’t allow such things to happen to our user base, so we wanted to do more testing. (more…)

openFATE News

September 21st, 2011 by

We just added 2 new goodies to our feature tracking tool openFATE:

Print views

You can now get a decent view of a feature that is adapted for printing. Either click on “Print preview” in your browsers menu, or on “Print” in the feature export box on the right side of your feature.

Adding inline images and screenshots

To add a screenshot or any other image included in your feature, just add a relation with type “url” that points to your image in the net. You can for example upload it to paste.opensuse.org or any other image hoster.

1-2-3 Cloud

June 20th, 2011 by

Towards the end of last year there was an article in openSUSE news “announcing” the cloud efforts in the openSUSE project and on OBS. Well, cloud is still all the rage (see Jos’ contribution to openSUSE News issue 180) and people just cannot stop talking about cloud computing.

Using openSUSE as a host for your cloud infrastructure is also making great progress. We have 3 cloud projects in OBS and hopefully these cover your favorite cloud infrastructure code, Virtualization:Cloud:Eucalyptus, Virtualization:Cloud:OpenNebula, and Virtualization:Cloud:OpenStack. The projects provide repositories for Eucalyptus, OpenNebula, and OpenStack, respectively.

We attempt to make it relatively easy to get a cloud up and running. In this process OpenNebula and OpenStack have progressed the most. Eucalyptus is working, but due to an issue with Eucalyptus and openSSL 1.0 and later (the version in openSUSE) automation has to wait until these issues are resolved.

For OpenNebula we now have a KIWI example that shows how one can get a cloud setup from scratch in less than 2 hours, including the image build. The example contains a firstboot workflow for the head node, and self configuration of cloud nodes.

For OpenStack SUSE Gallery images are in the works and will be published in the near future.

All repositories provide packages you can install on running openSUSE systems. If you are interested in using openSUSE as the underlying OS for your cloud or if you want to contribute to the cloud projects, subscribe to the cloud mailing list opensuse-cloud@opensuse.org

Hermes Work

December 1st, 2010 by

Not every day is a sunshine day, also not in software development. This is my credo about the last few days which I spent debugging Hermes a bit, motivated by a kind bug report saying basically that the digest mails suck. Well, I had to kind of agree on that, so I revisited that topic.

Do you remember what Hermes is? We use Hermes in the openSUSE infrastructure to handle notifications. Since we do not want to send people emails they do not explicitly agree that they want it (otherwise it would be spamming, right?), we invented a system that recognizes all kinds of events that happen in the openSUSE world, than check if a certain user wants to know about it and finally send it to these users. The benefit the user of the system is that he can pick from a huge variety of events and control if and how he gets informed about. Hermes does not only serve users with email but also maintains RSS feeds, it Twitters and does even more. And as another bonus, it can collect similar events for you and later send a digest with a collection. That way, you for example can get a mail with a list of failed package builds in OBS each hour instead a mail every fife seconds for each and every failing package.

But back to my debugging fun: I was mainly fixing the appearance of the digest messages: They now in the subject tell you how many events are digested and how frequently the digest comes, such as hourly, minutely etc. In the mail body, you now find a numbered “table of contents” of the mail and the individual events nicely listed. So much more useful.

Unfortunately it wasn’t the most time efficient debugging session I ever had, I stumbled over some things that weren’t optimal now in an environment where Hermes processes between 40,000 and 70,000 events a day for more than 25,000 users. Some of the problems are ugly to identify. I got lost a bit which is not good for the overall mood, so I decided to cry at Susanne, one of our colleagues. She asked me quite a few questions and than she left home for dinner. Ten minutes later I could nail the bug.

So this is my strong suggestion: If in debugging trouble, talk to your friends. Tell about the problem, share your misfortune. A few question can guide you to the right path which you did not see before. Not new? Well, yes, of course we knew that already from other topics in live, talking helps 😉

The other suggestion I wanted to make: Check Hermes digests! Go to the Hermes Subscription Page and change one of your subscriptions to digest mode, will be fun. Let me know what you think.