If you are new to the zypper-upgraderepo plugin, give a look to the previous article to better understand the mission and the basic usage.

Repository check

The first important change is inherent the way to check a valid repository:

• the HTTP request sent is HEAD instead of GET in order to have a more lightweight answer from the server, being the HTML page not included in the packet;
• the request point directly to the repodata.xml file instead of the folder, that because some server security setting could hide the directory listing and send back a 404 error although the repository works properly.

Check just a few repos

Most of the times we want to check the whole repository’s list at once, but sometimes we want to check few of them to see whether or not they are finally available or ready to be upgraded without looping through the whole list again and again. That’s where the –only-repo switch followed by a list of comma-separated numbers comes in help.

–only-repo switch

All repo by default

The disabled repositories now are shown by default and a new column highlights which of them are enabled or not, keeping their number in sync with the zypper lr output. To see only the enabled ones just use the switch –only-enabled.

Table view

Report view

Beside the table view, the switch –report introduce a new pleasant view using just two columns and spanning the right cell to more rows in order to improve the number of info and the reading quality.

Report view

Other changes

The procedure which tries to discover an alternative URL now dives back and forth the directory list in order to explore the whole tree of folders wherever the access is allowed by the server itself. The side effect is a general improvement also in repo downgrade.

The output in upgrade mode is now verbose and shows a table similar to the checking one, giving details about the changed URLs in the details column.

The server timeout error is now handled through the switch –timeout which allows tweaking the time before to consider an error any late answer from the server itself.

Final thoughts and what’s next

This plugin is practically completed, achieving all the goals needed for its main purpose no other relevant changes are scheduled, so I started thinking of other projects to work in my spare time.

Among them, there is one I am interested in: bring the power of openSUSE software search page to the command line.

However, there are some problems:

• This website doesn’t implement any web API so will be a lot of scraping job;
• There are missing packages while switching from the global research (selecting All distribution) to the specific distribution;
• Packages from Packman are not included.

I have already got some ideas to solve them and did lay down several lines of code, so let’s see what happens!

This tool started as a personal project when a day I was in the need to upgrade my distro quicker than using a traditional ISO image, Zypper was the right tool but I got a little stuck when I had to handle repositories: some of them were not yet upgraded, others changed slightly in the URL path.

Who knows how to Bash the problem is not exactly a nightmare, and so I did until I needed to make a step further.

The result is zypper-upgraderepo Ruby gem which can be integrated as a zypper plugin just installing the zypper-upgraderepo-plugin package.

Installing zypper-upgraderepo-plugin

Installing zypper-upgraderepo-plugin is as easy as:

1. Adding my repo:
   sudo zypper ar https://download.opensuse.org/repositories/home:/FabioMux/openSUSE_Leap_42.3/home:FabioMux.repo
2. Install the package:
   sudo zypper in zypper-upgraderepo-plugin

How to use it

Sometime we want to know the status of current repositories, the command zypper ref does a similar job but it is primarily intended to update the repository’s data and that slow down a bit the whole process.
Instead we can type:

$ zypper upgraderepo --check-current


To know whether or not all the available repositories are upgrade-ready:

$ zypper upgraderepo --check-next


As you can see from the example above all the enabled repositories are ready to upgrade except for the OSS repo which has a slightly different URL.

# The URL used in the openSUSE Leap 42.3
http://download.opensuse.org/distribution/leap/42.3/repo/oss/suse/
# The suggested one for openSUSE Leap 15.0
http://download.opensuse.org/distribution/leap/15.0/repo/oss/


Let’s try again overriding the URL without make any real change:

$ zypper upgraderepo --check-next \
--override-url 8,http://download.opensuse.org/distribution/leap/15.0/repo/oss/


Once everything is ok, and after performed a backup including all the repositories, it’s time to upgrade all the repository at once:

$ sudo zypper upgraderepo --upgrade \
--override-url 8,http://download.opensuse.org/distribution/leap/15.0/repo/oss/


Conclusions

That’s all with the basic commands, more information is available in the wiki page of zypper-upgraderepo gem where all the commands are intended with the only use of the gem, but installing the plugin they are also available as zypper subcommands like shown above, also a man page is available as

$ zypper help upgraderepo


Completion for package-names just made its way into git (thus soon will appear in Factory aka Tumbleweed) after ~6 weeks of back and forth exploring different approaches.

And it is super-fast

If you do not want to wait, you can use OneClickInstallCLI http://multiymp.zq1.de/zypp:Head/zypper
with allowing some vendor changes for libzypp and libsolv

Now the thinking part is about the speed. For the tab-completion I needed a list of installed packages and of course we have that in our RPM database (using berkeley DB as a backend). However querying the list with rpm -qa already took over a second on a modern and fast system. On my poor netbook with a cold cache, it took 25 seconds (5 secs on second try with hot cache)… And the point is that you probably do not want to wait 5 seconds for your tab-completion to react.

So to avoid this problem, I used caching via make to produce a better format (plain text). This is then post-processed with sed in a fraction of a second – a speedup factor somewhere between 15 and 150. This makes a big difference.

In the end, I still wonder why plain text is so much faster than a DB. I guess, one reason is that the DB is optimized for retrieval of single values – e.g. rpm -q bash – this is very fast (but even there an egrep “^bash-[^-]+-[^-]+$” is more than twice as fast).

I still want to optimize zypper for better speed, so that a search might some day return in under 2 seconds. One idea for that is to not parse all those config+repo files every time, but only when they change. It could use mmaped files under /var/cache/zypp* as memory to store the binary representations. Though it might become complicated, if dynamic structures such as linked lists are involved.

The future will be interesting…

The build id is a unique identifier stored in the .note.gnu.build-id note section of the executable file and loaded into the process image during run-time. Different means to compute the unique identifier are supported although the default setting is to use a 20 byte SHA1 hash of the unstripped executable (see ld linker documentation for further information about the –build-id option).

To be able to read the build id from a core dump the kernel must include the ELF header pages in file-backed private memory areas (see documentation on /proc/<pid>/coredump_filter). This is the default setting on recent openSUSE kernel versions.

Different tools can be used to print out the build-id. eu-readelf (from the elfutils package) prints the contents of the note sections given the -n option in a human readable fashion. pbuildid (from the ptools package) is prints the build-id from executables and from core files including the build-id of all loaded shared objects during the crash.

Now, zypper can be asked to install the debuginfo package that provides the debuginfo for the given build-id.

# zypper install -C "debuginfo(build-id)=b75bab63c9a25eb13264bb95f1fef190e157f865"
Loading repository data...
Reading installed packages...
Resolving package dependencies...

The following NEW package is going to be installed:
  bash-debuginfo


Overall download size: 605.0 K. After the operation, additional 2.1 M will be used.
Continue? [YES/no]: yes
Retrieving package bash-debuginfo-3.2-148.2.x86_64 (1/1), 605.0 K (2.1 M unpacked)
Retrieving: bash-debuginfo-3.2-148.2.x86_64.rpm [done]
Installing: bash-debuginfo-3.2-148.2 [done]
# 

If you are really lazy you can call the following script to do the job for you:

#!/bin/bash                                                                     
#                                                                               
# Sample script how to install debuginfo packages by build-id                   
#                                                                               

IDS=''
for f in "$*"; do
    case "$(file -L $f)" in
        *ELF*)
            IDS+="$(pbuildid $f 2>/dev/null | awk '{print $NF}') "
            ;;
        *)  
            ;;
    esac
done

echo "Install Debuginfo for following build-ids: $IDS"

CMDLINE=""
for i in $IDS; do
    CMDLINE+="debuginfo(build-id)=$i "
done

echo $CMDLINE | xargs zypper install -C

How todo this is little tricky, at first you must enable caching downloaded files (I do it only for remote connection):

zypper mr –keep-packages –remote

So now you cache all downloaded files and now try testing run of dup. Trick is that all packages download for that test is cached.

zypper dup –dry-run

Now if you have slow connection I reccomend also disable autorefresh for all repositories, because if repository is refreshed before dup, you can easily find that some packages is newer than package in cache and you must download it.

zypper mr –all –no-refresh

Now is everything prepared for zypper dup, which use files from cache. Cache can take quite lot of disk space, so after dup you can clean it.

zypper clean

And thats all. This features work from OpenSuse 11 and you can also use this trick for zypper update or zypper install.