Home Home
Sign up | Login

Author Archive

Quick tip: Novell Bugzilla and Klipper

December 22nd, 2009 by

Klipper from KDE has a set of predefined actions triggered by content of the clipboard. So when you have it enabled, it show up the popup menu with all available browsers when you copy the URL. In my job I open many bugs and the bug numbers have the well defined format bnc#number. So why don’t define automatic action triggered by this regular expression?

Because Klipper expects only oneliner as an action, I wrote the short bash script obug.sh which expects one argument with a bugzilla string.

#!/bin/bash

[ -z "${1}" ] && {
  echo "bug number required"
  exit 1
}

regex='^(bnc|bug)?#[0-9]+$'
[[ "${1}" =~ ${regex} ]] || {
  echo "${1} did not match ${regex}"
  exit 2
}

firefox https://bugzilla.novell.com/show_bug.cgi?id=${1#*#}

It’s obvious how it works – it checks if the argument exists and if matches the pattern and then opens a firefox (it’d be a xdg-open, but I use firefox for Novell Bugzilla). There should be only one strange expression – ${1#*#}. It is a Shell Parameter Expansion. This expression removes all characters from the beginning of the string including first #, so it convert bnc#1234 to 1234.

Then I defined a new action in Klipper triggered by ^(bnc|bug)?#[0-9]+$, which calls obug.sh %s, where %s is the content of clipboard.
Klipper config

And after that all I selected the bnc#1234 by mouse and then the following popup appeared.

One disadvantage of this method is that Klipper opens popup window only near its status icon in systemtray. It would be more useful to show it near current mouse position.

New pm-utils for openSUSE

November 25th, 2009 by

The current SUSE version of pm-utils is pretty old. Rpm -q said somethink like 0.99.4.20071229. And it also contains a hacked support for s2ram, which is nowadays in upstream version. There has been also a bnc#378883 – Need an updated pm-utils I started a work on this week.

A new pm-utils package for openSUSE is available in home:mvyskocil:branches:Base:System. The HIBERNATE_METHOD is no longer supported, because upstream version contains something better – modules. There are three methods how to run software suspend on Linux

  1. kernel – plain echo something > /proc/something
  2. suspend – tool contains a lot of quirks needed on some HW
  3. tuxonice – kernel and userspace support for hibernate, not in upstream kernel, nor in openSUSE

Because there are too many ways in current Linux world, pm-utils simply support all by specific modules stored in /usr/lib/pm-utils/modules.d, which implements appropriate functions for suspend/hibernate and hybrid. The SUSE default is uswsusp module calls s2ram/s2disk/s2both from software suspend project, because it should be considered as a safe default.

If you want to use different module, you can add a config file somewhere to
/etc/pm/conf.d/
and set the value of SLEEP_MODULE.

# The default sleep/wake system to use. Valid values are:
# kernel The built-in kernel suspend/resume support.
# Use this if nothing else is supported on your system.
# uswsusp If your system has support for the userspace
# suspend programs (s2ram/s2disk/s2both), then use this.
# tuxonice If your system has support for tuxonice, use this.
#
# The system defaults to "kernel" if this is commented out.
SLEEP_MODULE="uswsusp"

You can type more methods, which will be called, so SLEEP_MODULE=”kernel uswsusp” will use kernel and if it fail, or not available, it call uswsusp. Please note that config files are read in C sort order, so names matters.

So please install new pm-utils and test it and tell me if you found any regression (please inform me about a regressions only, I cannot fix generic suspend problems).

Smarter osc commit

August 3rd, 2009 by

Some hours ago I have worked on fix of eclipse build. But two parallel builds of eclipse (there are eclipse.spec and eclipse-archdep.spec) eats almost all resources of my computer, which was unusable. Fortunately vim needs a little of CPU time, so I decided to improve smarter osc commit, I implemented on Friday.

If you work with osc and packaging, you probably forgot to add a new patch, or source and commits sources with missing file, so build in OBS fails. Our internal script, which we used before BuildService has a check, which warns about files not specified in Patch or Source in a spec file, so it was a great feature for forgetful developers (as myself, for example). The problem is how to get a list of sources and patches for a current spec file? The internal script uses some “magic” shell code which calls a rpmbuild, because it’s rather impossible to parse a specfile correctly. Fortunately there is a simplest approach.

In build service package directory each file has a state – added, modified, removed, …. So my idea is simple – check state of all files in the directory and if there is any file with ‘?’ (file exists, but not in metadata) or ‘!’ (file is in metadata, but not in directory) state, ask user what to do. The initial implementation allows to continue or abort and was not user friendly as it could be. So during rebuild of (two) eclipse I wrote a better implementation.

Lets have a package (for example called xdoclet) and we want commit our changes to OBS. So

$ osc status
?    xdoclet-modules-objectweb-4.6.tar.bz2
?    xdoclet-src-1.2.3.tar.bz2
!    xdoclet-modules-objectweb-4.6.tgz
!    xdoclet-src-1.2.3.tgz
M    xdoclet.changes
M    xdoclet.spec

You see, that we used bznew to repack tgz files to tar.bz2, which is recommended in openSUSE. Then we commit our changes

osc commit
File `xdoclet-modules-objectweb-4.6.tar.bz2' is not in package meta. Would you like skip/remove/edit file lists/commit/abort? (s/r/e/c/A)

I suppose that all options are clear – skip will skip check for this file, remove will remove it, commit forced commit and abort breaks it. But I don’t like this message, so if you have some better proposal, please contact me.

The most important command is edit file list. I thought how to easily add and remove files using this smarter commit. The final implementation has been inspired by one of the coolest git feature – interactive rebase.

So after selecting e, the list of files is opened in your EDITOR

  1 leave   xdoclet-AbstractProgramElementTagsHandler.patch
  2 leave   xdoclet-WebLogicSubTask.patch
  3 leave   xdoclet-XDocletModulesEjbMessages.patch
  4 leave   xdoclet-ant.not-required.patch
  5 leave   xdoclet-build_docs_xml.patch
  6 leave   xdoclet-build_xml.patch
  7 leave   xdoclet-component-info.xml
  8 leave   xdoclet-maven-plugin-project.patch
  9 leave   xdoclet-maven-plugin-template.patch
 10 leave ? xdoclet-modules-objectweb-4.6.tar.bz2
 11 remove ! xdoclet-modules-objectweb-4.6.tgz
 12 leave   xdoclet-project_xml.patch
 13 leave ? xdoclet-src-1.2.3.tar.bz2
 14 remove ! xdoclet-src-1.2.3.tgz
 15 leave M xdoclet.changes
 16 leave M xdoclet.spec
 17
 18 # Edit a filelist for package %s
 19 # Commands:
 20 # l, leave = leave a file as is
 21 # r, remove = remove a file
 22 # a, add   = add a file
 23 #
 24 # If you remove file from a list, it will be unchanged
 25 # If you remove all, commit will be aborted

and a manipulation is obvious. Just replace a command listed in a first column by another one, then save a file and instructions will be processed and package will be commited.

BTW: Of course this feature can be suppressed by new -f/–force option.

Hackweek IV: Novell Bugzilla access from command line

July 29th, 2009 by

During a last Hackweek I decided to work on some tool which will helps users with submitting a new bug. My idea is to create something like reportbug from Debian, but targeted to SUSE. So the first step was find the way how to communicate with a bugzilla (especially to bugzilla.novell.com – bnc). Fortunately I found several libraries, which should be used for it:

perl-SUSE-BugzillaClient is written by Thomas Schmidt especially for bnc, so it is able to login using iChain. It also provides few functions for bugzilla communication, but unfortunately only for querying. Pybugz is written by Alastair Tse for Gentoo development in Python. But it is not able to login through iChain and I did not extended it for bnc, because it uses a plain GET/POST communication with parsing of HTML using regexes.

The reason why I have used the python-bugzilla written by Will Woods from RedHat is simple. It uses XMLRPC API, which bugzilla provides. And it’s also written with a flexibility in mind, so writing of another type of Bugzilla was easy. Just some subclassing and reimplementing of few methods and voila, the NovellBugzilla type is here. Because it uses iChain, it was necessary to reimplement login/logout methods – I spend a lot of time to understand how it works. Osc uses a HTTP authentication, which should be supported too, but even if it works with api.opensuse.org, it did not work with bnc. So I used a Firefox and LiveHTTP headers extension to track a communication and implemented the login according it.

The useful advantage of Novell bugzilla is that I extended a method readconfig, which reads a config file. Novell Bugzilla can read the username/password from ~/.oscrc, so you don’t need to have a same password in another config file, if you use osc.

So on the end those few lines of Python code

import bugzilla

bnc = bugzilla.NovellBugzilla(url="https://bugzilla.novell.com", user="mvyskocil", password="XXXXX")
kwargs = {
    'product': 'openSUSE 11.2',
    'severity': 'Normal',
    'cc': [],
    'rep_platform': 'x86-64',
    'component': 'Java',
    'summary': '[Java:packages/ant]: Cannot find a Java virtual machine',
    'version': 'unspecified',
    'assigned_to': 'mvyskocil@novell.com',
    'op_sys': 'Linux',
    'description': 'This is a testing bug report\n'
    }
bug = bnc.createbug(**kwargs)
print bug.bug_id

Have created this bug#525549. Even if Python is readable for regular programmers, the **kwargs should be expressed. It maps the dictionary to named function arguments (called keyword arguments – kwargs – in a Python world), so both calls in following snippet are equivalent.

d = {'name' : 'value')
call(**d)
call(name='value')

The python-bugzilla comes also with a simple command line tool called bugzilla, so this tool should be used also from shell scripts. It is sometimes little bit raw. For example listing of all products returns an output as a Python dictionary

bugzilla --bztype NovellBugzilla --user mvyskocil --password xxxxxx info -p
...
{'description': 'openSUSE 11.1', 'internals': {'disallownew': 0, 'classification_id': 7340, 'name': 'openSUSE 11.1',
'votestoconfirm': 0, 'milestone_required': 0, 'id': 651, 'votesperuser': 20, 'maxvotesperbug': 5, 'defaultmilestone': '---',
'milestoneurl': '', 'description': 'openSUSE 11.1'}, 'name': 'openSUSE 11.1', 'id': 651}
{'description': 'openSUSE 11.2', 'internals': {'disallownew': 0, 'classification_id': 7340, 'name': 'openSUSE 11.2',
'votestoconfirm': 0, 'milestone_required': 0, 'id': 755, 'votesperuser': 20, 'maxvotesperbug': 5, 'defaultmilestone': '---',
'milestoneurl': '', 'description': 'openSUSE 11.2'}, 'name': 'openSUSE 11.2', 'id': 755}
...

which might be hard to parse in other language. But it provides a lot of functionality, so for other informations about usage, please use man bugzilla.

Netbeans 6.5 is going to Factory

June 4th, 2009 by

As you might know, the netbeans package was a part of openSUSE, but for some historical reason, it was a monolithic package in non-oss repository. Last weeks I have worked on import 29 new packages to Factory, which allows us to build netbeans from source and include it to regular free repository. Fortunately thanks for hard work of guys from jpackage project it was easy to adapt their spec files for SUSE and push them into the Factory.

Majority of those packages was reviewed and included into Factory during last weeks, but five (including netbeans) itself are currently in review process, so they would be available later. Following graph shows netbeans and it’s build dependencies. Green color means – package is in Factory, yellow means package is in review process.

Netbeans and build requires

In a meanwhile the netbeans for openSUSE Factory and 11.1 is available in Java:packages BuildService repository, so feel free to install and test it.

Suppressing KeyboardInterrupt traceback in Python

May 4th, 2009 by

If you have a running program in Python and press Ctrl+C, you’ll get a traceback like this:

$ scout java foo
^CTraceback (most recent call last):
  File "/usr/bin/scout", line 11, in
    ret = scout.ScoutCore.run()
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 945, in run
    result = module.ScoutModule().main(clp.module_args)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 873, in main
    return self.do_query(args.query, repos, args.inversesearch)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 890, in do_query
    result.add_rows(self._query(repo, query, inversesearch))
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 896, in _query
    r = db.query(self._sql, '%%%s%%' % term)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 485, in query
    if len(row) == 1:           #(2)
KeyboardInterrupt

It is useful suppress it, because user knows he breaks the program and this output should be considered as a bug. Possible solution is wrap a main function by one big try: except KeyboardInterrupt:

try:
  main() # the main function
except KeyboardInterrupt:
  pass # KeyboardInterrupt supressed

But it makes a new level of indentation which should be uncomfortable – especially in Python. Or when you have multiple entry-points, or just don’t well structured program (which is common when you write your private helper script :) ), you maybe prefer another solution.

Python has a sys.excepthook, which is called for traceback printing, so we could define our own and suppress unnecessary output here. And it would be nice suppress only one exception and handle other ones using existing function. And this function make it:

def suppress_keyboard_interrupt_message():
    old_excepthook = sys.excepthook

    def new_hook(type, value, traceback):
        if type != exceptions.KeyboardInterrupt:
            old_excepthook(type, value, traceback)
        else:
            pass

    sys.excepthook = new_hook

Function suppress_keyboard_interrupt_message (it is really nice name, don’t it ;-) ) stores an existing hook and register an inner function new_hook as a new one. Advantage is that old_excepthook exists only in a scope of this function, so you don’t need use global variables for it.

Update: typos fixed

Ruby style method injection in Python

April 29th, 2009 by

Ruby has a nice, but very dangerous feature called open classes. That means you can extend any class definition if you want it.

This example shows it

#!/usr/bin/ruby
class Foo
    def do_foo
        return "foo"
    end
end

aFoo = Foo.new()

puts aFoo.do_foo()

class Foo
    def do_bar
        return "bar"
    end
end

puts aFoo.do_bar()

That means you dynamically extend the definition of Foo class, so the last line prints “bar”.

Python behavior is different, because it does not allows this extending. Python just redefines a class Foo, so new instance of Foo will have a do_bar method only. But this not affected an existing ones, like Ruby does.

class Foo:

    def do_foo(self):
        return "foo"

aFoo1 = Foo()

class Bar(Foo):

    def do_nothing(self):
        pass

print issubclass(Bar, Foo)
print isinstance(aFoo1, Foo)

class Foo:

    def do_bar(self):
        return "bar"

aFoo2 = Foo()
aBar  = Bar()

print issubclass(Bar, Foo)
print isinstance(aFoo1, Foo)
print isinstance(aFoo2, Foo)

print dir(aFoo1)
print dir(aFoo2)
print dir(aBar)

But what about method injection? My solution is based on ideas of Recipe 81732: Dynamically added methods to a class.

import types

def funcToMethod(func, instance, method_name=None):
    cls = instance.__class__ if type(instance) != types.TypeType else instance
    setattr(cls, \
            method_name or func.__name__, \
            types.MethodType(func, instance, cls))

And that’s all. The funcToMethod bounds func to instances’s class and allows under method_name (default name is a same as a function one). So lets do some testing.

class Foo(object):

    def list(self):

        return [meth for meth in dir(self) if meth[:3] == 'do_' and type(getattr(self, meth)) == types.MethodType]

    def do_bar(self):

        return "bar"

def do_foo(inst):

    return "foo"

class Bar(Foo):

    def do_nothing(self):
        pass

aBar = Bar()

aFoo1 = Foo()
print aFoo1.list()
# calling it with class instead of instance is also possible and is equivalent
#funcToMethod(do_foo, Foo, "do_foo")
funcToMethod(do_foo, aFoo1, "do_foo")

aFoo2 = Foo()
print aFoo1.list()
print aFoo2.list()

print aFoo1.do_foo()
print aFoo2.do_foo()

print aBar.list()
print aBar.do_foo()

When you run the code, you will see that funcToMethod adds a new method to Foo class and this changes both existing and new instances as Ruby does too. And subclassing is not a problem, they are be affected too.

How to track changes in packages: osc vc

April 21st, 2009 by

As you may know, SUSE was originally based on Slackware. And some some relics from those early days if SUSE survives to present. The biggest example visible for end-users was packaging of desktop environment to /opt. Gnome was switched few years ago and KDE3 packages still remains in it, because packagers decided to focus on KDE4, so only those packages are comfortable with FHS and are installed into /usr.

With opening of SUSE development towards community via BuildService’s collaboration, or Contrib, the another relict of Slackware days was raised – which I mentioned in my previous post – a .changes file.

This file is used in SUSE for tracing of package changes and rpm %changelog is created from this file during build. As it has a consistent format, we used an internal command called vc for add a new entry to it and this tool generates a proper header of changes file. So after my last patch, implementation of osc vc was a logical (but not straightforward) job.

After some discussions with Ludwig Nussel and on opensuse-buildservice mailing lists, I implemented and committed an osc vc command. This is based on buildvc script written by Ludwig and is available in build.rpm (from version 2009.04.17). It has the same behavior as an old vc command.

Basic usage of this command is simple:

user@host:some:project/package/ $ osc vc

Command will find a changes file, open it in EDITOR (default is vim) and fill a header. You can also use it in non-interactive mode using -m MESSAGE. You can also specify a file to edit, or edit a file in other directory than current, … – see osc vc –help

The main difference between buildvc and osc variant is in e-mail address handling. The osc implementation has more sources of it, so it try to

  1. use content of mailaddr variable (same as buildvc)
  2. read a configuration from ~/.oscrc
  3. read an email from user’s metadata (see osc meta user your_login)

This is because many users want to use a different e-mail for changelogs than iChain one, so osc allows configure an email for each instance of BuildService. Appropriate part of ~/.oscrc looks like

[https://api.opensuse.org/]
user = login
pass = password
email = user@defined.email

Better commit message handling for osc

April 16th, 2009 by

If you are familiar with SUSE packages, you probably know about .changes files, which is used for tracking changes. This file has a defined structure and is converted to common %changelog section in spec file during build of package. It worked well with many internal tools and everyone was happy in those old good times ;-) .

Nowadays, we had a BuildService and osc, which has it owns commit log. I have aimed to keep the content of changes and osc changelog synchronized. But manual copying of it is not very reliable approach. So I wrote a small patch (and Adrian gives me a write access, so it is a r7099), which checks changes in it and add them to commit message as a template, so it will be easier to keep them synchronized.

How it works:

vi package.spec # do some changes
vc -m "some changes done" # vc is an internal command
osc commit

and osc will open an editor with content like this

  1 - some changes done
  2 --This line, and those below, will be ignored--
  3
  4 M    package.changes
  5
  6 Diff for working copy: .
  7 Index: package.changes
  8 ===================================================================
  9
 10 --- package.changes     (revision 19)
 11 +++ package.changes     (working copy)
 12 @@ -1,3 +1,8 @@
 13 +-------------------------------------------------------------------
 14 +Thu Apr 16 16:00:58 CEST 2009 - mvyskocil@suse.cz
 15 +
 16 +- some changes done
 17 +
 18  -------------------------------------------------------------------
 19  Thu Apr 16 15:37:52 CEST 2009 - mvyskocil@suse.cz

Now you can type :w (save) for commit, or :q! (quit without saving) for abort. The behavior of -m argument is unchanged.

100 packages in Contrib

January 26th, 2009 by

Last month we started a new project called Contrib. It’s a shiny new community repository for openSUSE. In opposite of specialized repository (eg. Security:), Contrib is universal. It doesn’t matters if your package is a desktop application, or a network tool. Every type of package is welcome.

Today, we celebrated a package number 100 (gparted)! Thanks to all involved folks!

One hundred packages doesn’t look like a big repository, but consider we are active about a month and half and this is an important milestone for us. The bigger repository should be more attractive for end users, and a package maintainers too.

Users wanted
Contrib release cycle is same as Factory, but we want to help users to use it now. So Contrib is also available for 11.1. So just add the repository and start to use it! It help us!

zypper ar http://download.opensuse.org/repositories/openSUSE:/Factory:/Contrib/openSUSE_11.1/ openSUSE_Factory_Contrib

Package/rs wanted
If you maintain some interesting package in your home: project (or elsewhere), please follow instructions – New packages to Contrib and add your package to Contrib, so many users of openSUSE would use it!