Home Home > Tag > Python
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Posts Tagged ‘Python’

osc plugin – changes

March 23rd, 2011 by

OSC is a powerful tool for packaging experts, exposing all the latest and greats features of the Build Service. It’s written on python and easy in studying and using. However there are situations when its functionality is not enough; sometimes we need something special. In this case to us will help plug-in mechanism, which in osc is realised very simply.
Plugin can use all of the features, which already implemented osc, as well as provide an output in a convenient format for you. For example, if I want to check changes in kdelibs4 between openSUSE:11.3 and openSUSE:11.4, I can do something like this:

> osc rdiff openSUSE:11.3 kdelibs4 openSUSE:Factory kdelibs4

After that I will receive a detailed output about all changes. Yes, that’s great… but not always it’s convenient. For example, in this case output will contain more than 2000 strings, and I need time to find, say, a *.changes file if I want quickly to understand that has been changed. In case if I want to transfer output to processing to another program (as often happens in practice), I have to shape this data. Unfortunately osc is not as intelligent and can’t show changes from one file (from *.changes, for example) only…

Hello world

Let’s me show how we can create a very simple osc-plugin. In the derectory /var/lib/osc-plugins/ we create a new file tell_me_something.py with such content:

@cmdln.alias('say')
def do_say_something(self, subcmd, opts, *args):
    if sys.argv[2] == "something":
        print "openSUSE rulezzz"
    else:
        print sys.argv[2]

At start, osc will check this directory and will register all found there plugins. In that case, if in the plugin’s content there are errors, osc will report about it immediately. If now we run

> osc help

we will see in the list our function say_something and the key to start it – say. Let’s test:

> osc say "hello"
hello
> osc say GNU/Linux
GNU/Linux
> osc say something
openSUSE rulezzz
>

As you can see, it’s very easy – just python and nothing else. Let’s go back to the output of the function rdiff(), which we mentioned at the beginning.

show me changes

In output of rdiff() nothing wrong, but I would like to immediately get information about what exactly has been done and, for example, which bugzilla-reports (related to this package) have been closed, etc. All what I need, are in rdiff’s output. It means that all what I have to do is just to shape this output.

In 40 minutes of hacking I got such output:

> osc changes kdelibs4 openSUSE:11.3 openSUSE:11.4
PACKAGE: kdelibs4
BUGZILLA_NOVELL: 668185, 670426, 644236, 596021
BUGZILLA_OTHER: 246652, 170806, 149991, 221989, 252280, 253387, 253294, 193364, 253414
CHANGES:

- work around random error on first startup, bnc#668185,
  kubuntu has a similiar patch applied
- call update-mime-database in pre/post install scripts
- don't show synthetic volume label when none is really available,
  allow kio_sysinfo to fall back to device path (bnc#670426)
- update to KDE Platform 4.6.0
  * Plasma applets can be written in QML
  * Plasma data engines can be written in Javascript
  * Plasma data engines can use generic cache for offline mode
  * udev, udisks, upower replace HAL in Solid
  * For more details, see http://kde.org/announcements/4.6
- add patch from 4.6 branch to fix plasma crash on exit
- Add dependencies on udisks and upower for 11.3 and up for Solid
- update to 4.5.95
  * KDE 4.6 RC2
  * no upstream changelog available.
- update to 4.5.90
  * KDE 4.6 RC1
  * no upstream changelog available.
- For 11.2 and 11.3 only : Will build now with polkit-qt-1
  v 0.99.1, which is an official requirement of KDE 4.6
- update to 4.5.85
  * KDE 4.6 Beta2
  * Final Beta before RC, various fixes from Beta1
  * no upstream changelog available.
- For 11.2 and 11.3 only : Added patch to revert changes that
  requires a higher version of polkit-qt-1
- update to 4.5.80
  * KDE 4.6 Beta1
  * no upstream changelog available.
-  Closing the shell via CTRL+D crashes [bko#246652]
- fix build with gcc 4.6
- tighten qt4 dependencies
- update to 4.5.3
  * see http://kde.org/announcements/changelogs/changelog4_5_2to4_5_3.php for details
- update branch diff for various bugs in 4.5:
  * Crash on configure toolbars (bko#170806)
  * KCookieJar can't read cookies from another port (bko#149991)
  * Fix oversized number input widgets (bko#221989)
  * CSS conformance issue (bko#252280)
  * Fix helper protocols such as mailto: and telnet:
  * Plasma crash on comic applet switch (bko#253387)
  * HTTPS urls in KMail do not open properly in browser (bko#253294)
  * Mailto: links in FireFox started by kmailservice fail (bnc#644236)
  * Crash in directory listings when toggling
    show hidden files flag (bko#193364)
- Upstream patch added for kmail issue (bko#253414)
- update to 4.5.2
  * see http://kde.org/announcements/changelogs/changelog4_5_1to4_5_2.php for details
- build apidocs separately to reduce build time
- BuildRequire utempter-devel
- update to 4.5.1
  * see http://kde.org/announcements/changelogs/changelog4_5_0to4_5_1.php for details
- new package: kdelibs4-apidocs (bnc#596021)
- update to 4.5.0
  * KDE 4.5.0 final (version bump over RC3)
- update to 4.4.95
  * KDE 4.5 RC3 (not announced)
  * critical fixes for 4.5.0 release
- Add libsoprano-devel Require to libkde4-devel
- update to 4.4.93svn1149349
- update to 4.4.5
  * bugfixes over 4.4.4
  * see http://kde.org/announcements/changelogs/changelog4_4_4to4_4_5.php for details

and between 11.4 and factory:

> osc changes kdelibs4 openSUSE:11.4 openSUSE:Factory
PACKAGE: kdelibs4
CHANGES:

- update to 4.6.1
  * Bugfixes over KDE 4.6.0
  *  see http://kde.org/announcements/changelogs/changelog4_6_0to4_6_1.php for details
- remove upstreamed patches

aha… and what’s about vim between 11.2 and 11.3?

> osc changes vim openSUSE:11.2 openSUSE:11.3
PACKAGE: vim
BUGZILLA_NOVELL: 598903
CHANGES:

- Add screen control sequences to inputrc (bnc#598903)
- Use the icon from the tarball instead of our custom icon. It
  looks much better.
- Drop gvim.png from the source package.
- build data subpackage as noarch
- updated patches to apply with fuzz=0

Now we can see exactly what was has been done and which bugzilla-reports was fixed/closed. Yes, we have bnc# and bko# reports: reports from bugzilla.novell.com and bugs.kde.org (it cut be also bugs.kernel.org). Second group will be always different (KDE/Gnome/Mozilla/Kernel…).

Source code of plugin is here, and I hope this post will be useful for you if you’ve never written a plugin before.

Tags: , ,
Posted in Build Service, Packaging | Comments Off on osc plugin – changes

Suppressing KeyboardInterrupt traceback in Python

May 4th, 2009 by

If you have a running program in Python and press Ctrl+C, you’ll get a traceback like this:

$ scout java foo
^CTraceback (most recent call last):
  File "/usr/bin/scout", line 11, in 
    ret = scout.ScoutCore.run()
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 945, in run
    result = module.ScoutModule().main(clp.module_args)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 873, in main
    return self.do_query(args.query, repos, args.inversesearch)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 890, in do_query
    result.add_rows(self._query(repo, query, inversesearch))
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 896, in _query
    r = db.query(self._sql, '%%%s%%' % term)
  File "/usr/lib64/python2.6/site-packages/scout/__init__.py", line 485, in query
    if len(row) == 1:           #(2)
KeyboardInterrupt

It is useful suppress it, because user knows he breaks the program and this output should be considered as a bug. Possible solution is wrap a main function by one big try: except KeyboardInterrupt:

try:
  main() # the main function
except KeyboardInterrupt:
  pass # KeyboardInterrupt supressed

But it makes a new level of indentation which should be uncomfortable – especially in Python. Or when you have multiple entry-points, or just don’t well structured program (which is common when you write your private helper script :)), you maybe prefer another solution.

Python has a sys.excepthook, which is called for traceback printing, so we could define our own and suppress unnecessary output here. And it would be nice suppress only one exception and handle other ones using existing function. And this function make it:

def suppress_keyboard_interrupt_message():
    old_excepthook = sys.excepthook

    def new_hook(type, value, traceback):
        if type != exceptions.KeyboardInterrupt:
            old_excepthook(type, value, traceback)
        else:
            pass

    sys.excepthook = new_hook

Function suppress_keyboard_interrupt_message (it is really nice name, don’t it ;-)) stores an existing hook and register an inner function new_hook as a new one. Advantage is that old_excepthook exists only in a scope of this function, so you don’t need use global variables for it.

Update: typos fixed

Tags:
Posted in Miscellaneous | Comments Off on Suppressing KeyboardInterrupt traceback in Python

python-TurboGears2 alive and kickin’ on the build service

March 6th, 2009 by
Screenshot of TurboGears 2

Screenshot of TurboGears 2

It’s been quite a while since I last posted an update on the state of python-turbogears in the openSUSE build service. The TurboGears project currently has three branches open (which seems to cause more confusion that actually help people) – the 1.0 branch, the 1.1 branch and the 2.0 branch. Whereas newcomers are advised to install from the 1.0 branch and to develop web applications based on that branch, the truth is that most of the active development goes on in the 2.0 branch. TurboGears 2.0 is at beta7 stage right now – features are frozen and a release data is in sight for the big 2.0.

Without getting into the major differences between 1.0 and 2.0, many people on the TurboGears Google Group have expressed the opinion that the 2.0 branch, despite its beta suffix, is exceptionally stable and well suited to production deployment. One of the major obstacles facing people was how to actually install it. Because it requires some cutting edge packages which might not be standard fare for many distributions, it is recommended to install TurboGears2 (beta) in a virtual python environment using the virtualenv package. Once the new virtual environment is ready and activated, a simple easy_install will automatically install the TurboGears2 package and its requirements. This is well documented.

I got a bit fed up with the virtual environment setup – it worked fairly well, but I was constantly having to set up completely new virtual environments just to install another TurboGears web application. There is an option when using virtualenv, to make the entire virtual environment “mobile” (meaning that you can pack virtual environment plus web app off to another computer easily), but it wasn’t working for me.

Enter python-TurboGears2 on the openSUSE build service. It took me a couple of days to get all the dependencies in the correct versions and installing properly, but it looks as if it works now. As TurboGears2 will be in a more or less constant state of flux until the final 2.0 release, there will probably be quite a lot of work to do to keep the packages up to date, but it’s worth it for the simple zypper in python-TurboGears2 🙂 What I want to do now is to try creating a SUSE Studio based virtual appliance – that way practically any web app could be created and setup easily – and with a bit of elbow grease you could probably use apparmor etc to make it rock solid.

PS: it would be well cool if people would test the package

Tags: , , ,
Posted in Build Service, Packaging | Comments Off on python-TurboGears2 alive and kickin’ on the build service

Updated python-turbogears to 1.0.7

November 17th, 2008 by

Last weekend I spent some time on getting python-turbogears to build for Factory. In the process, I came across a spec file error which was causing not only python-turbogears to fail on Factory, but also dozens of other python packages. The problem was the spec file line:

%{__python} setup.py install –prefix=%{_prefix} –root=$RPM_BUILD_ROOT –record-rpm=INSTALLED_FILES

Once I fixed this for python-turbogears and it’s dependent packages (the solution is to replace –record with –record-rpm), I was able to get version 1.0.6 building for Factory. There were a few other issues which needed to be resolved (some deprecated def as() functions in python-peak and python-ruledispatch were causing syntax errors on python 2.6 – because ‘as’ is a reserved keyword), but finally I got it sorted out.

Once python 1.0.6 was successfully building on Factory, I decided to update the package to the most recent stable version of the 1.0.x package line from http://www.turbogears.org – 1.0.7. I had to rewrite the infamous cherrypy2 patch for 1.0.7 and also had to update the python-elixir package (the older 0.5 Elixir was incompatible with python-sqlalchemy >= 5) but now it builds properly.
You can now download and install python-turbogears using:

zypper in python-turbogears

(if you have the devel:languages:python repository, of course).

Thanks to Peter Poeml for his help packaging and fixing bugs.

Tags: , , ,
Posted in Packaging | Comments Off on Updated python-turbogears to 1.0.7

How to search more efficiently in Bugzilla with pybugz

June 19th, 2008 by

If you just want to search for bugs in Bugzilla, it’s (a bit?) painful: start the browser, type in the URL, insert your login and password and try to find out where to go. There is an easier way to do: pybugz for commandline lovers!

Thanks to Peter Poeml, get this very useful Python script from here. After you have installed it you need only two steps to configure it:

  1. Create a file ~/.bzuser and insert your Bugzilla login.

  2. Login into Bugzilla and insert your password. This creates the file ~/.bugz_cookie:

    $ /usr/bin/bugz-login

The script knows several subcommands, its interface is similar to CVS or Subversion. You can search, get, post, modify, attach and download an attachment, all with this utility. For example, if you want all bugs about “XML”, regardless of the product or component, you just type:

$ bugz search xml

That gives the following output:

 * Using https://bugzilla.novell.com/
 * Searching for 'XML'
 [ deleted a lot of lines ]

Maybe you want to narrow your search for KDE and specific products? No problem, here is an example:

$ bugz search KDE --product="openSUSE 11.0"
 * Using https://bugzilla.novell.com/
 * Searching for 'KDE' with the following options:
 * product = ['openSUSE 11.0']
113512 kde-maintainers Firefox in KDE - Only Uses GNOME Programs
170055 dmueller Firefox sets desktop background for Gnome under KDE
176179 kde-maintainers User can't edit properties for default notifications under KDE Storage Media and entries disepeared !
203548 sbrabec workrave-kde is an empty applet by default
[... and many more ...]

Of course, if you know the bug number you can retrieve it with:

$ bugz get 378240

and it will list all the details of the bug. Very useful! I haven’t tried the other subcommands yet, but I think they are also very convenient.

There are many more things to discover. So, when was your last time searching for bugs? 🙂

Tags: , ,
Posted in Accessibility, Quality Assurance, Usability | Comments Off on How to search more efficiently in Bugzilla with pybugz

Query your XML with xpathgrep.py

June 9th, 2008 by

Maybe you know this problem: You have a couple of XML files and you need a specific information. Probably everybody would think of grep or similar tools first. But maybe your query is a bit more complicated than just a simple piece of text. What do do?

Recently I’ve found a very useful command line utility, which is probably not very known. It’s named xpathgrep.py and you can get it from the lxml repository (you need lxml too). Let’s assume we have the following DocBook file:

File db.xml
<?xml version="1.0"?>
<book>
  <title>My Cooking Book</title>
  <chapter>
    <title>Ingredients</title>
    <para>...</para>
  </chapter>
  <chapter id="howtocook">
    <title>How to cook</title>
    <para>...</para>
  </chapter>
</book>

Now, if I want to get all the titles I have to use a XPath (which is a path description language for XML, similar to Unix/Linux paths, but more powerful). To get all title elements all I have to do is to write //title, regardless of the level:

$ xpathgrep.py //title db.xml

and I get this:

<title>My Cooking Book</title>

<title>Ingredients</title>

<title>How to cook</title>

Nice, isn’t it? Probably you say: “But, hey, I can get this with grep too!” Yes, but if you want just all chapter titles, you have a problem with grep. With XPath and xpathgrep.py I only modify my XPath expression a bit:

$ xpathgrep.py //chapter/title db.xml

Now this reduces the above output just to the wanted chapter titles. And I can extent my query just for all chapters that doesn’t have an id attribute:

$ xpathgrep.py '//chapter[not(@id)]/title' db.xml

(You need the apostroph because of the shell.) The tool outputs this:

<title>Ingredients</title>

That’s nice, isn’t it? There are a lot of more to discover. A few hours ago I send a small patch to the lxml-devel mailinglist to support namespaces. Hopefully, it will be accepted. 🙂

Tags: , ,
Posted in Miscellaneous | Comments Off on Query your XML with xpathgrep.py