Hi all, and welcome! Let’s see what’s new in the PostgreSQL world this week…

-Benoit Carpentier announced the release of Benetl v. 3.5; Benetl is a free ETL tool for files using PostgreSQL. You can find out more/download from www.benetl.net .

-Darren Duncan announced version 0.130.0 of the specification of the Muldis D language for ORDBs : “The largest change since last month’s spec version 0.129.1 is the elimination of the “$” sigil that was used semi-gratuitously to mark data-entities (variables, parameters, attributes, named expression nodes).  These are now regularly formatted as barewords instead, like most languages and SQL, but unlike Perl.

Another significant change is renaming a syntax shorthand from “>foo” to “=>foo” for clarity; it means “foo => foo” and has nothing to do with “greater-than”.

A few other improvements were made to the concrete grammars as well.”

-Robert Haas started a thread on hackers@ titled “deprecating =>, take two”. Here are some excerpts : “By consensus, we have removed the new-to-9.0 operator text[] => text[] and renamed the hstore => text[] operator.  (The current name is “%”, but there is some discussion of “%>”, some yet other name, or getting rid of it altogether; please comment on that thread if you wish to weigh in.)  This means that the only remaining => operator in CVS is the text => text operator which constructs a single-element hstore, which has been around since 8.2.  In lieu of providing a substitute operator, Tom Lane proposed that we simply encourage people to use the hstore(text, text) function which does the same thing:

http://archives.postgresql.org/pgsql-hackers/2010-06/msg00711.php

Per that email, and subsequent concurrence, here is a series of patches which does the following:

1. In CVS HEAD, document the hstore(text, text) function and adjust CREATE OPERATOR to throw a warning when => is used as an operator name, using the wording previously suggested by Tom.
2. In the back branches, add an hstore(text, text) function.  These branches already have a tconvert(text, text) function which does the same thing, but the consensus seemed to be that we do not want to go  back to the name tconvert() for this functionality, and that back-patching the new name was preferable.
3. In 8.4 and 8.3, also add hstore(text, text) to the documentation. 8.2 appears to have no contrib documentation.

Barring vigorous objections, I will apply these tomorrow so that we can consider deprecating => as an operator name in 9.1, for better compliance with the SQL standard.

http://archives.postgresql.org/pgsql-hackers/2010-05/msg01501.php”

The general consensus was that it’s a good idea, but for all the comments and proposals I recommend you read the whole thread.

-Josh Berkus announced the first draft of the PSQL 9.0 release announcement ( http://wiki.postgresql.org/wiki/90ReleaseDraft ) also asking for feedback from the release team; people stepped in with suggestions and you might wanna check the link above for details.

-Moving on to PostgreSQL Planet, we delve into the Weekly News, which isscarce this week, so except the usual local news and the list of submitted and rejected patches, there really is nothing more. If the aforementioned topics interest you, check out http://www.postgresql.org/community/weeklynews/pwn20100620 .

-Andrew Gierth writes about range aggregation with window functions, describing the problem as ” Assume you have a table of ranges expressed as “start” (s) and “end” (e) columns; we’ll assume that these are half-open intervals, i.e. each row represents the range [s,e). We also assume that the constraint (e > s) is enforced. The problem is to aggregate all overlapping ranges and produce a result with one row for each
disjoint collection of ranges.” His solution and explanations are to be found here : http://blog.rhodiumtoad.org.uk/2010/06/21/range-aggregation-with-window-functions/ .

-Leo Hsu and Regina Obe write about NOT IN NULL and the mathematical and even philosophical implications of NULL : ” I know a lot has been said about this beautiful value we affectionately call  NULL, which is neither here nor there and that manages to catch many of us off guard with its casual neither here nor thereness. Database analysts who are really just back seat mathematicians in disguise like to philosophize about the unknown and pat themselves on the back when they feel they have mastered the unknown better than any one else. Of course database spatial analysts, the worst kind of back seat mathematicians, like to talk not only about NULL but about EMPTY and compare notes with their brethren and write dissertations about what to do about something that is neither here nor there but is more known
than the unknown, but not quite as known as the empty string.” Read more here: http://www.postgresonline.com/journal/index.php?/archives/166-NOT-IN-NULL-Uniqueness-trickery.html#extended .

-Andrew Dunstan wrote a concentrated version of a thread in hackers@ regarding the use of enums : ” Some people don’t seem to get enumeration types. They think of them as C programmers tend to – as  symbolic names for integer values.

But quiche eaters like me have quite a different point of view. Languages like Ada have had first class  enumeration types for a long time (Ada is nearly 30 years old now). PostgreSQL’s enum types are more of this kind, an ordered set of labels. Some people naïvely expect that underneath they will be stored as  their ordinal position in the label set. In fact, for a technical reason, they are not. Rather, they are stored as globally (within the database) unique Oids. This is a bit counter-intuitive to some people, but really, it’s just an implementation detail.

The biggest thing that bugs people about PostgreSQL’s enums is that you can’t extend them, i.e. you can’t add more labels to the list. There is a workaround involving creating a new type, but it involves rewriting the tables that use them, which is unpleasant. Recently I have given some thought to that. I came up with scheme for a new enum type that would have been extensible. But as often happens, Tom Lane came up with a better idea. I’ve been working on fleshing that out, especially testing the possible performance impact, and I hope we can have something in 9.1. Being able to add labels to an enum set without table rewriting would make them much more usable.

Okay getting to the point, one of our clients asked us about a peculiar problem they had with a query, and the strange results they were getting. We admit this still manages to catch us off guard every once in a while.”

Well, that’s about it, folks. Take care and have a nice weekend.

Hello everyone and welcome to the first edition of the PostgreSQL news, served OpenSUSE-style! Let’s start off with news and bits from the mailing lists, then, as we move on, we take a look at news from the Planet and around the Web.

-Pavel (he didn’t write down his last name), a student for Google Summer of Code 2010, proposed in hackers@ the idea of snapshot materialized views in PostgreSQL with questions as to how this should be properly implemented. A quick look at the mailing-list archives will give you a better insight.

-Jeff Davis posted a patch related to small exclusion constraints patch affecting /src/backend/executor/execUtils.c, explaining the patch thusly : “Currently, the check for exclusion constraints performs a sanity check that’s slightly too strict — it assumes that a tuple will conflict with itself. That is not always the case: the operator might be “<>”, in which case it’s perfectly valid for the search for conflicts to not find
itself. This patch simply removes that sanity check, and leaves a comment in place.”

-Marcus Gsteiger on testers@ reported a succesful migration from 8.4 to 9.0beta1 on CentOS on a HP Proliant server. While this is nothing particularily interesting, this report, like others of the kind, shows that 9.0 is becoming more stable and will become a succesful release.

-Robert Haas, in a mail to hackers@ titled “mapping object names to role IDs”had a proposal which I think it will be best if I quote here entirely, since it’s not that easy (for me at least) to summarise : “Suppose you have an object name as a CString and you want to convert it to an OID.  The method of doing this varies widely depending on the object type:

oid = get_database_oid(name);
oid = get_tablespace_oid(name);
oid = GetForeignDataWrapperOidByName(name, true);
oid = GetForeignServerOidByName(name, true);
oid = LookupFuncNameTypeNames(name, args, true);
oid = LookupAggNameTypeNames(name, args, true);
oid = LookupFuncNameTypeNames(name, args, true);
oid = LookupOperNameTypeNames(name, args, true);
oid = GetSysCacheOid1(LANGNAME, CStringGetDatum(name));
oid = GetSysCacheOid1(NAMESPACENAME, CStringGetDatum(name));
oid = GetSysCacheOid1(AMNAME, CStringGetDatum(name));
oid = get_roleid(name);
oid = GetConstraintByName(reloid, name);
oid = FindConversionByName(list_make1(name));
oid = TSDictionaryGetDictid(list_make1(name), true);
oid = TSTemplateGetTmplid(list_make1(name), true);
oid = TSConfigGetCfgid(list_make1(name), true);

If you’d like to throw an error if the object doesn’t exist, then you can change the “true” parameter in the above examples to false, where it exists, or you can use get_roleid_checked for roles.  For the types where a direct syscache lookup is used, you get to write the check yourself.  For constraints, GetConstraintByName throws an error anyway; there’s no equivalent that doesn’t.  Some other object types – like rules and casts – have even more complex logic that is sometimes duplicated in multiple places; and others (like tablespaces) that have convenience functions still manage to duplicate the logic anyway. Long story short, this is kind of a mess.

Looking at the different cases, there seem to be three main cases:

1. Objects that just have one name, like databases and procedural languages.
2. Objects that have possibly-qualified names, like conversions and
text search dictionaries.
3. Objects that have both names and argument, like functions and operators.

There’s enough complexity about the stuff in category #3 that I’m inclined to leave it alone, but try to make the other stuff more standardized.  What I would propose is that we create a new source file somewhere (maybe utils/cache), move all of the other functions of this type there, give them standardized names, and provide them all with an argument that specifies whether an error is to be thrown if the object doesn’t exist.  Begin bikeshedding here.  I suggest that the names be get_<object-type>_oid and that they take two parameters, either a List * for possibly-qualified names (a little wonky, but it’s what we do now) or a char * for unqualified names, and a boolean indicating whether to throw an error if the object isn’t found.  Thus:

Oid get_<object-type>_oid(List *qualname, bool missingok);
-or-
Oid get_<object-type>_oid(char *name, bool missingok);

Thus get_database_oid and get_tablespace_oid would remain unchanged except for taking a second argument, get_roleid and get_roleid_checked would merge, and all of the others would change to match that style.

Thoughts?”

-May 23rd brought another release of the PostgreSQL Weekly News, from which I will try to extract the most important/interesting bits :
-Cybercluster 2.0 has been released : http://www.cybertec.at/en/cybercluster-2-0-synchronous-postgresql-replication       -Security updates for various releases of PostgreSQL have been issued[1], as weel as RPM’s[2] :
[1] – http://www.postgresql.org/docs/current/static/release.html
[2] – http://yum.pgrpms.org/

-Postgres-XC 0.9.1 has been released : http://postgres-xc.sourceforge.net/

-Of course, PostgreSQL will be present at lots of this year’s events, like OSCON or South East Linux Fest; the complete list is also available in this Weekly News issue.
-As every week, there is a quite comprehensive list of source commits to the source tree this week; if you think you wan them listed here, say so in the comments section.

-Pavel Stehule proposed a small-time patch fixing a double free of allocated memory, Jeff Davis also wrote a 6-liner regarding btree_gist support for searching on “not equals”, Josh Berkus had an “Idea for getting rid of VACUUM FREEZE on cold pages”, which although very interesting, is too long, plus the replies, to summarize here. The archives will help you yet again 🙂 .

-Fujii Masao announced he is designing the “synchronous” replication feature based on SR for 9.1, explaining the general idea and asking for thoughts/comments, proposing several sync levels;
apparently, the majority agreed on one (#4).In order not to explain without example, here is a quote from the OP outlining the idea :
“The log-shipping replication has some synch levels as follows. The transaction commit on the master
#1 doesn’t wait for replication (already suppored in 9.0)
#2 waits for WAL to be received by the standby
#3 waits for WAL to be received and flushed by the standby
#4 waits for WAL to be received, flushed and replayed by the standby..etc?

Which should we include in 9.1?”

-Mark Wong announced in general@ the PDXPUG day at OSCON 2010 : “Thanks to the generosity of O’Reilly, we will be having a full day of free PostgreSQL sessions on Sunday, July 18 at the Oregon Convention Center.  Location details and schedule information can be found on the wiki at: http://wiki.postgresql.org/wiki/PDXPUGDay2010 ” .

-Stephen Frost posted a small patch to psql, adding ‘S’ as an optional parameter to \da, while Tom Lane announced his fellow hackers the intention to wrap up 9.0 beta2 on June the 3rd in order to release to the public on the 7th.

-In the Planet PostgreSQL news, we have Dave Page reporting back from PGCon 2010, held in Ottawa, Canada, and of course other Planet members wrote about it : Greg Sabino Mullane, Andrew Dunstan or Selena Deckelmann.

-Selena also posted on the Planet an announcement asking for participation from reviewers (interested, anyone?) since it’s preparation time for the first commitfest for 9.1 .

-Greg Sabino Mullane announced that PostgreSQL finally switched to git as a canonical VCS; this announcement was kind of unnoticed because of posts from PGCon, so if you’re curious and/or want to see pictures, Planet PostgreSQL is the way to look.

-Peter Eisentraut has posted very interesting ideas meant to make RDBMS developers in general and of course Postgres developers in particular; to get a overall idea, here it is : “My problem is that getting database code from the editor to the database server in a safe manner is pretty difficult. […] My answer to that problem is an old friend: package management. Package managers such as dpkg and rpm are pretty well-established solutions and have shown over the years that managing software deployments can be easy and occasionally even fun.” Of course, there;s much more to this article and it’s a recommended read, in my oppinion, so point your browser or RSS reader to the Planet! 🙂

-Peter also has a feed called ‘Visual Explain Reloaded’, referring to the EXPLAIN command (you can check it out here : http://www.postgresql.org/docs/9.0/static/sql-explain.html ) and it’s possibilities to output in different markup languages – JSON, XML, YAML. http://2.bp.blogspot.com/_dgdplFJMdoQ/S_In33KeqYI/AAAAAAAAADU/N_nYFZ0Uack/s1600/veung-dotty.png shows you the resulting JSON output of ‘ regression=> \a\t
regression=> EXPLAIN (FORMAT JSON) SELECT * FROM tenk1 t1, tenk2 t2 WHERE t1.unique1 < 100 AND t1.unique2 = t2.unique2 \g |veung ‘ . Pretty nifty, if you ask me.

-Andrew Dunstan announces the release of the PostgreSQL Buildfarm client, version 4.0, which according to the author, has two new features : “The SCM code is substantially rearranged into a separate OO module, with subclasses supporting CVS and Git. New config options support these changes, while old style config parameters for CVS are still supported. Support for running the buildfarm from Git is a requirement before we can move the core community repo, and this meets that requirement (a little later than planned, but as promised almost exactly one year ago). The requirement to specify a port for each branch to build with is gone. If the config parameter ‘base_port’ is specified the code will pick a unique port for the branch, a short number above that setting. The means that the config file does
not need to be changed when a new stable postgres release is made. Again, old-style configs continue to be supported.”

-The same Dave Page posted on the Planet a very interesting comparison between VoltDB and, you guessed it, Postgres – and at the same time it’s a comparison between “traditional” DBMS’s and new-style, lockless DBM systems. Selena Deckelmann tells some stories of new and interesting ideas coming from PgCon 2010 – a worth read by far.