Home Home > 2010 > 05
Sign up | Login

Archive for May, 2010

Guest Blog: Rares Aioanei – Weekly Review of PostgreSQL

May 29th, 2010 by

Hello everyone and welcome to the first edition of the PostgreSQL news, served OpenSUSE-style! Let’s start off with news and bits from the mailing lists, then, as we move on, we take a look at news from the Planet and around the Web.

-Pavel (he didn’t write down his last name), a student for Google Summer of Code 2010, proposed in hackers@ the idea of snapshot materialized views in PostgreSQL with questions as to how this should be properly implemented. A quick look at the mailing-list archives will give you a better insight.

-Jeff Davis posted a patch related to small exclusion constraints patch affecting /src/backend/executor/execUtils.c, explaining the patch thusly : “Currently, the check for exclusion constraints performs a sanity check that’s slightly too strict — it assumes that a tuple will conflict with itself. That is not always the case: the operator might be “<>”, in which case it’s perfectly valid for the search for conflicts to not find
itself. This patch simply removes that sanity check, and leaves a comment in place.”

-Marcus Gsteiger on testers@ reported a succesful migration from 8.4 to 9.0beta1 on CentOS on a HP Proliant server. While this is nothing particularily interesting, this report, like others of the kind, shows that 9.0 is becoming more stable and will become a succesful release.

-Robert Haas, in a mail to hackers@ titled “mapping object names to role IDs”had a proposal which I think it will be best if I quote here entirely, since it’s not that easy (for me at least) to summarise : “Suppose you have an object name as a CString and you want to convert it to an OID.  The method of doing this varies widely depending on the object type:

oid = get_database_oid(name);
oid = get_tablespace_oid(name);
oid = GetForeignDataWrapperOidByName(name, true);
oid = GetForeignServerOidByName(name, true);
oid = LookupFuncNameTypeNames(name, args, true);
oid = LookupAggNameTypeNames(name, args, true);
oid = LookupFuncNameTypeNames(name, args, true);
oid = LookupOperNameTypeNames(name, args, true);
oid = GetSysCacheOid1(LANGNAME, CStringGetDatum(name));
oid = GetSysCacheOid1(NAMESPACENAME, CStringGetDatum(name));
oid = GetSysCacheOid1(AMNAME, CStringGetDatum(name));
oid = get_roleid(name);
oid = GetConstraintByName(reloid, name);
oid = FindConversionByName(list_make1(name));
oid = TSDictionaryGetDictid(list_make1(name), true);
oid = TSTemplateGetTmplid(list_make1(name), true);
oid = TSConfigGetCfgid(list_make1(name), true);

If you’d like to throw an error if the object doesn’t exist, then you can change the “true” parameter in the above examples to false, where it exists, or you can use get_roleid_checked for roles.  For the types where a direct syscache lookup is used, you get to write the check yourself.  For constraints, GetConstraintByName throws an error anyway; there’s no equivalent that doesn’t.  Some other object types – like rules and casts – have even more complex logic that is sometimes duplicated in multiple places; and others (like tablespaces) that have convenience functions still manage to duplicate the logic anyway. Long story short, this is kind of a mess.

Looking at the different cases, there seem to be three main cases:

1. Objects that just have one name, like databases and procedural languages.
2. Objects that have possibly-qualified names, like conversions and
text search dictionaries.
3. Objects that have both names and argument, like functions and operators.

There’s enough complexity about the stuff in category #3 that I’m inclined to leave it alone, but try to make the other stuff more standardized.  What I would propose is that we create a new source file somewhere (maybe utils/cache), move all of the other functions of this type there, give them standardized names, and provide them all with an argument that specifies whether an error is to be thrown if the object doesn’t exist.  Begin bikeshedding here.  I suggest that the names be get_<object-type>_oid and that they take two parameters, either a List * for possibly-qualified names (a little wonky, but it’s what we do now) or a char * for unqualified names, and a boolean indicating whether to throw an error if the object isn’t found.  Thus:

Oid get_<object-type>_oid(List *qualname, bool missingok);
-or-
Oid get_<object-type>_oid(char *name, bool missingok);

Thus get_database_oid and get_tablespace_oid would remain unchanged except for taking a second argument, get_roleid and get_roleid_checked would merge, and all of the others would change to match that style.

Thoughts?”

-May 23rd brought another release of the PostgreSQL Weekly News, from which I will try to extract the most important/interesting bits :
-Cybercluster 2.0 has been released : http://www.cybertec.at/en/cybercluster-2-0-synchronous-postgresql-replication       -Security updates for various releases of PostgreSQL have been issued[1], as weel as RPM’s[2] :
[1] – http://www.postgresql.org/docs/current/static/release.html
[2] – http://yum.pgrpms.org/

-Postgres-XC 0.9.1 has been released : http://postgres-xc.sourceforge.net/

-Of course, PostgreSQL will be present at lots of this year’s events, like OSCON or South East Linux Fest; the complete list is also available in this Weekly News issue.
-As every week, there is a quite comprehensive list of source commits to the source tree this week; if you think you wan them listed here, say so in the comments section.

-Pavel Stehule proposed a small-time patch fixing a double free of allocated memory, Jeff Davis also wrote a 6-liner regarding btree_gist support for searching on “not equals”, Josh Berkus had an “Idea for getting rid of VACUUM FREEZE on cold pages”, which although very interesting, is too long, plus the replies, to summarize here. The archives will help you yet again :) .

-Fujii Masao announced he is designing the “synchronous” replication feature based on SR for 9.1, explaining the general idea and asking for thoughts/comments, proposing several sync levels;
apparently, the majority agreed on one (#4).In order not to explain without example, here is a quote from the OP outlining the idea :
“The log-shipping replication has some synch levels as follows. The transaction commit on the master
#1 doesn’t wait for replication (already suppored in 9.0)
#2 waits for WAL to be received by the standby
#3 waits for WAL to be received and flushed by the standby
#4 waits for WAL to be received, flushed and replayed by the standby..etc?

Which should we include in 9.1?”

-Mark Wong announced in general@ the PDXPUG day at OSCON 2010 : “Thanks to the generosity of O’Reilly, we will be having a full day of free PostgreSQL sessions on Sunday, July 18 at the Oregon Convention Center.  Location details and schedule information can be found on the wiki at: http://wiki.postgresql.org/wiki/PDXPUGDay2010 ” .

-Stephen Frost posted a small patch to psql, adding ‘S’ as an optional parameter to \da, while Tom Lane announced his fellow hackers the intention to wrap up 9.0 beta2 on June the 3rd in order to release to the public on the 7th.

-In the Planet PostgreSQL news, we have Dave Page reporting back from PGCon 2010, held in Ottawa, Canada, and of course other Planet members wrote about it : Greg Sabino Mullane, Andrew Dunstan or Selena Deckelmann.

-Selena also posted on the Planet an announcement asking for participation from reviewers (interested, anyone?) since it’s preparation time for the first commitfest for 9.1 .

-Greg Sabino Mullane announced that PostgreSQL finally switched to git as a canonical VCS; this announcement was kind of unnoticed because of posts from PGCon, so if you’re curious and/or want to see pictures, Planet PostgreSQL is the way to look.

-Peter Eisentraut has posted very interesting ideas meant to make RDBMS developers in general and of course Postgres developers in particular; to get a overall idea, here it is : “My problem is that getting database code from the editor to the database server in a safe manner is pretty difficult. [...] My answer to that problem is an old friend: package management. Package managers such as dpkg and rpm are pretty well-established solutions and have shown over the years that managing software deployments can be easy and occasionally even fun.” Of course, there;s much more to this article and it’s a recommended read, in my oppinion, so point your browser or RSS reader to the Planet! :)

-Peter also has a feed called ‘Visual Explain Reloaded’, referring to the EXPLAIN command (you can check it out here : http://www.postgresql.org/docs/9.0/static/sql-explain.html ) and it’s possibilities to output in different markup languages – JSON, XML, YAML. http://2.bp.blogspot.com/_dgdplFJMdoQ/S_In33KeqYI/AAAAAAAAADU/N_nYFZ0Uack/s1600/veung-dotty.png shows you the resulting JSON output of ‘ regression=> \a\t
regression=> EXPLAIN (FORMAT JSON) SELECT * FROM tenk1 t1, tenk2 t2 WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2 \g |veung ‘ . Pretty nifty, if you ask me.

-Andrew Dunstan announces the release of the PostgreSQL Buildfarm client, version 4.0, which according to the author, has two new features : “The SCM code is substantially rearranged into a separate OO module, with subclasses supporting CVS and Git. New config options support these changes, while old style config parameters for CVS are still supported. Support for running the buildfarm from Git is a requirement before we can move the core community repo, and this meets that requirement (a little later than planned, but as promised almost exactly one year ago). The requirement to specify a port for each branch to build with is gone. If the config parameter ‘base_port’ is specified the code will pick a unique port for the branch, a short number above that setting. The means that the config file does
not need to be changed when a new stable postgres release is made. Again, old-style configs continue to be supported.”

-The same Dave Page posted on the Planet a very interesting comparison between VoltDB and, you guessed it, Postgres – and at the same time it’s a comparison between “traditional” DBMS’s and new-style, lockless DBM systems. Selena Deckelmann tells some stories of new and interesting ideas coming from PgCon 2010 – a worth read by far.

Guest Blog: Rares Aioanei – Weekly Kernel Review

May 29th, 2010 by

Howdy y’all, and welcome to this weeks’s kernel news – OpenSUSE style!

-We begin our week by noticing a patch proposed by Larry Finger on opensuse-kernel@ that fixes a typo and a reference to websites for instructions; after a correction by Jiri Benc, the patch was committed to the master branch.

-On vger.kernel.org, the week(end) starts with a git pull request from Ryusuke Konishi working on the nilfs2 updates for 2.6.35, Steven Rostedt with perf, of course,  OpenSUSE’s Greg with TTY patches for .35 and the same Greg with a series of 38 patches for the driver-core tree also targetting 2.6.35.

-Other tree(s) updates include i2c for .35 by Jean Delvare, omap by Tony Lindgren, UDF tree updates for -rc1 by Jan Kara, also of OpenSUSE fame, block tree patches by Jens Axboe, staging by Greh Kroah Hartman and md updates by Neil Brown (where unspecified, the version the patches are referring to is 2.6.35).

-James Morris announces about the Linux Security Summit 2010, taking place in Boston at the beginning of August – the 9th.

-Jan Kara also presented fixes for filesystem-related trees, namely ext2, ext3 and quota, targetting -rc1; Al Viro posted fixes for the vfs tree, quite a bunch of them, Grant Likely posted OF code cleanups and Jesse Barnes posted PCI changes, and Pekka J. Enbeg has some minor SLAB fixes for -rc0.

-Sage Weil added an important number of bugfixes and code cleanups for the ceph tree targetting-rc1, Frederic Weisbecker added modifications to the bkl/ioctl branch, also for -rc1, and in filesystem-related changes we have Eric Van Hensbergen with fixes for 9p and Ogawa Hirofumi adding code for the fatfs tree; the perf tree received 3 updates from Arnaldo Carvalho de Melo and Boaz Harrosh updated the exofs tree.

-Speaking of updates and fixes, we also have Paul Mundt with sh and genesis fixes for -rc1, David Miller with IDE fixes described by himself as “nothing really interesting”, Thomas Gleixner with timer fixes for -rc1 and Ingo Molnar with updates to the lockup-detector tree.

-Again, updates and then more updates…:) We have Greg Ungerer with m68k fixes, described a follows : “Biggest change is the QSPI driver platform support. Also platform support for the SMC91x driver on some ColdFire platforms. A couple of bug fixes, and some cleanups as well.”, then Grant Likely with spi driver changes, fixes for GFS2 added by Steven Whitehouse, regulator fixes targeted at .35 by Liam Girdwood and others, such as the usual suspect Frederic Weisbecker (perf), Steven Rostedt (tracing), Ingo Molnar also with perf changes and Roland Dreier patching for the infiniband tree.

-As noted by some online magazines such as LWN ( http://lwn.net/Articles/389444/rss ) , there are some stable kernel updates; the first one is announced by OpenSUSE’s Greg Kroah-Hartman related to 2.6.32.14 with 34 patches, 2.6.27.47 with 25+24 patches and 2.6.33.9 with 39 patches.

-Stefan Richter updated the firewire (IEEE 1394) tree for the post .34 kernels, another “usual suspect”,  John W. Linville, posted a pull request for wireless dated 25.05.2010, plus David Miller with updates for networking, Phillip Lougher – squashfs and Dave Airlie with DRM fixes.

-Takashi Iwai updated the sound tree for Linus for -rc1, while Jiri Kosina of OpenSUSE pushed updates for HID and Richard Purdie of IBM posted updates for his trees related to LED and backlight; also,  we have Theodore Ts’O updating ext4 with minor changes, including some quota fixes and other minor bugfixes. Trond Myklebust came ups with NFS bugfixes, so did Arnaldo Carvalho de Melo with the perf tree; Miklos
Szeredi added splice() support to the fuse tree and Michael S. Tsirkin posted some code cleanups for vhost-net.

-Jeffrey Merkey announced ndiswrapper 2.6.34 with various fixes; since the language of the announcement is not quite suitable for this news article, as it contains some strong words :) , I recommend you check it out on vger.kernel.org .

-Krzysztof Hałasa posted some fixes post-.34 related to the arm architecture, Eric Paris has some cleanups and improvements for the notification tree, of course James Bottomley worked again on some patches for the SCSI tree and Dmitry Torokhov posted input updates for -rc0. In other fixes/updates/cleanups news, we have Martin Schwidefsky with updates for s390 server architecture, SFI from Len Brown, Chris Mason of Oracle with btrfs (for -rc*), Jean Delware for the hwmon tree and H. Peter Anvin with x86 fixes .

-Speaking of arm fixes, Daniel Walker has announced some simple fixes for MSM targeted towards -rc1 and Roland Dreier also has some simple patchset with some cleanups for infiniband; again, a list of some other fixes follows : Samuel Ortiz with a bunch of new drivers and MFD fixes, libata updates by Jeff Garzik et al., as announced :
” 1) Finish Tejun’s SFF<->BMDMA separation patchset, as mentioned in initial 2.6.35 push.  These had been submitted prior to merge window open, but had to wait to resolve some patch/merge conflicts.

2) Disable Asynchronous Notification (AN) by default.  Proper use is vague, and behavior of firmwares in the field do not match each other.

3) add ‘dump_id’ debugging output helper, to give us better bug reports.”, Richard Purdie posted simple LED fixes regarding some issues on PPC (-rc1), Thomas Gleixner has a few timer fixes for -rc1 and David Miller has , again, several fixes for the networking tree. Seems like this week is a week for small fixes and “few patches”.

-This is all for this week, have a beutiful weekend.

Novell Hackweek Five

May 28th, 2010 by

Hackweek Five LogoI am really looking forward to the next Hackweek that we have in Novell – it will be in the week from 7-11 of June 2010.
In that week, Novell allows a whole lot of people to spend the full work time (and more ;-) to work on whatever free software they want. That is really a huge thing, because we’re talking about hundrets of engineers.
What everybody is working on is as said not at all prescribed, except that it should benefit the idea of free software. There is a list maintained of ideas which people have for Hackweek Five in order to find somebody joining the team or to pick the idea up at all.
The good thing now is that of course openFATE is used to maintain this list and thus it is open for the openSUSE Community to also add ideas, comment or vote on whats already there. This is of course no guarantee that the idea is going to be picked up but still. So everybody who thinks she has an idea that will inspire someone on Hackweek Five, feel free to add it to openFATE and talk about.
Of course it is also possible and appreciated to work on Hackweek projects also as non Novell employee :-)
Get in touch – it will be exciting!

How to set up a Production Server for your Rails App

May 28th, 2010 by

Hi folks,

as you know it’s exciting to create a new rails application for several tasks. It’s fast, easy and everything is predefined. But what do you do if your application is (nearly) done? The next logical step is to set up a production web server – for me, this step always was a difficult issue.  Mongrel/WEBrick was started via ‘ruby script/server’ and your application was reachable on your localhost, mostly on port 3000. You’re the only user who interacts with it – no problem (as long as your application is in development).

“The Web, however is an extremely concurrent environment. Production web servers, such as Apache, Lighttpd, and Zeus, can work on several requests – even tens or hundreds of requests – at the same time. A single-process, single-threaded Ruby-based web server can’t possibly keep up.” (quoted from ‘Agile Web Development with Rails’)

Therefore I want to show briefly how to set up a front-end server with an existing Rails application using an Apache server and the RubyGem ‘Passenger‘. Do the following as root.

1. Install Passenger (One-click Install) (assumed that Ruby itself and all needed Gems are installed)

2. Install Apache and it’s dependencies:

$> zypper in apache2

3. Add the Passenger module to your Apache server:

$> a2enmod passenger

4. Create a virtual host on your Apache server. Create ‘/etc/apache2/vhosts.d/myapp.conf’ and insert:

<VirtualHost *:80>
ServerName www.myapp.com
DocumentRoot /srv/rails/myapp/public
RailsEnv development
<Directory /srv/rails/myapp/public>
Allow from all
Options -MultiViews
</Directory>
</VirtualHost>

Be sure that the path to your application is correct and do not forget the public directory! As you can see this virtual host receives all requests on port 80 (http). The line ‘RailsEnv development‘ specifies the ‘RAILS_ENV’ variable (in this case ‘development’, the default value is ‘production’).  Normally you want ‘production’ for your production server!

5. Activate your virtual host in ‘/etc/apache2/listen.conf’. Just enable the line (remove the leading hash mark)::

#NameVirtualHost *:80

6. Now you can start your Apache server:

$> rcapache2 start

Important: when you want to check the log file be aware of the mode Passenger runs the Rails application (‘production.log’/'development.log’). By default, the log file is written by the user who owns the ‘environment.rb’ – check the log file’s write permissions (See also: User switching).

Have a look at the documentation! There you find a lot of configuration options which you should think of.

That’s it. Sure, there are many many other ways to get such a server running and this was just scratching the surface but should be a good point to start. Thanks to Thomas Schmidt for a good introduction into the topic.

Phusion Passenger
Passenger Apache Documentation

KDE SC 4.5 Beta 1 available for openSUSE

May 28th, 2010 by

Since yesterday the KDE SC 4.5 Beta 1 packages are available through the openSUSE OBS. They are available in the KDE:KDE4:UNSTABLE:Desktop repository. As announced, this Beta contains the KDEPIM packages from KDE SC 4.4 due to the delay in the conversion to Akonadi. However if you feel really brave, then also the KDEPIM 4.5 packages are available in a subproject (KDE:KDE4:UNSTABLE:Desktop:kdepim45).

The kdepim45 packages will receive daily updates from svn, while the KDE SC 4.5 Desktop will receive weekly updates. Once openSUSE 11.3 has been released, KDE SC 4.5 will be moved to the KDE:KDE4:Factory:Desktop repository.

Enjoy!

openSUSE at FLISoL Chile

May 28th, 2010 by

As you already know, the Festival Latinoamericano de Instalacion de Software Libre event will be held this Saturday here in Chile.

In La Serena we are starting around 9 am at Universidad de La Serena campus Isabel Bongard, with plenty of fun talks and room for installations, I am more than ready to install our awesome green! =). By the way, I will be creating an ISO with SUSE Studio so everyone can grab it on USB devices.

Some talk topics we are presenting: KVM, Free Knowledge, Linux beyond the Desktop, and Contributing to openSUSE without having to be a developer.

I would like to thank Andreas and everyone at Novell who has been supporting the Ambassadors Program, shipping DVDs, Geekos, T-shirts that the audience really appreciates and do make a difference.

OpenOffice_org 3.2.1 rc2 available for openSUSE

May 27th, 2010 by

I’m happy to announce OpenOffice.org 3.2.1 rc2 packages for openSUSE. They are available in the Build Service OpenOffice:org:UNSTABLE project, are based on the upstream 3.2.1-rc2 sources and include many Go-oo fixes and improvements. Please, look for more details about the openSUSE OOo build on the wiki page.

The packages are release candidates and have not passed full QA cycle yet. They might include even serious bugs. Therefore they are not intended for data-critical usage. A good practice is to archive any important data before an use, …

As usual, we kindly ask any interested beta testers to try the package and report bugs. See also the list of known bugs.

Other information and plans:

We will provide rc3 within next couple of days because upstream released newer sources yesterday. They include only a rather cosmetic fix i#111766 but …

I hope that rc3 would pass all tests the following week, I would be able to put it into the OpenOffice:org:STABLE project.

openSUSE Schools! in Nicaragua!

May 25th, 2010 by

Hi!

A few years ago the LUG opensuse-nicaragua has the project named “Escuelita opeSUSE” this project was to provide basic courses for the people opensuse.

And the last week, we finally  made it!!!

The course was in UNICIT(Iberoamerican University of Technologies and Cience), were 2 intensive days of linux linux and linux jejeje, the pensum of course was:

1.      Installation of openSUSE

2.       Basic Configurations

3.       KDE

4.       YaST

5.       Shell

6.       Software and repositories

And Thanks to the marketing group for the materials!! Second fase of Escuelita openSUSE is comming soon!!!

automated openSUSE testing

May 25th, 2010 by

Testing is an important task. But testing daily openSUSE-Factory snapshots would mean testing the same things every day. This would be pretty tiresome to people.
And there is a lot of software to test, including software unknown to most testers or new versions of known software, so how should the tester know if the results were the intended results?
My answer is: leave as much as possible to computers. Computers do not get tired. Computers do not stop testing something after a dozen identical results. Computers do not forget.

The following assumes that you have read my text on making openSUSE install videos.

So far, I am rather satisfied with my automated installations.
At the end of those, I added some basic application testing, which already showed in MS7
openSUSE-KDE-LiveCD-x86_64-Build0625a.ogv dated 2010-05-21 16:08
an issue filed 28 hours later at bnc bug 608087

Only that it currently still needs a human to look at the results.
I was thinking to improve upon that by scanning (rectangular) parts of the screenshot for known good or bad images. If either is found, the test could be automatically marked as passed or failed.
On unknown images, a human would still need to decide which part of the image is relevant and if it is good or bad. This decision can then be used to avoid human interaction (hard work) in further runs of that test.
If we push this further, it could be similar to nagios for network monitoring. Telling when something breaks and telling when something is back working. It could have an overview page about automated test status, giving totals e.g. “50 working, 10 unknown, 3 failing”. With links to more details.

The advantage in adding the application tests after the install test is that the system starts out in a clean, reproducible way. One disadvantage I see is that a newly failing test could prevent following tests to work.

I have also been working to enable others to run my isotovideo script. For that I have cleaned up my code so that it no more contains paths from my system. The other thing is that I documented how to get it working at http://www3.zq1.de/bernhard/git/autoinst/INSTALL

MS7 installation videos:

openSUSE-KDE-LiveCD-x86_64-Build0625c.ogv
openSUSE-KDE-LiveCD-i686-Build0625a.ogv
openSUSE-GNOME-LiveCD-x86_64-Build0625b.ogv
openSUSE-NET-x86_64-Build0623b.ogv
openSUSE-NET-i586-Build0623b.ogv
openSUSE-DVD-Build0625-x86_64b.ogv
openSUSE-DVD-Build0625-i586b.ogv

Red hot Geeko

May 25th, 2010 by

Another gecko picture, this one was taken at Saputara.

From Paragliding