Home Home > Systems-management > Software-management
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Archive for the ‘Software Management’ Category

« Older Entries
Newer Entries »

Highlights of YaST development sprint 31

February 20th, 2017 by

As we announced in the previous report, our 31th Scrum sprint was slightly shorter than the usual ones. But you would never say so looking to this blog post. We have a lot of things to talk you about!

Teaching the installer self-update to be a better citizen

As you may know, YaST includes a nice feature which allows the installer to update itself in order to solve bugs in the installation media. This mechanism has been included in SUSE Linux Enterprise 12 SP2, although it’s not enabled by default (you need to pass an extra option selfupdate=1 to make it work).

So after getting some feedback, we’re working toward fixing some usability problems. The first of them is that, in some situations, the self-update mechanism is too intrusive.

Consider the following scenario: you’re installing a system behind a firewall which prevents the machine to connect to the outside network. As the SUSE Customer Center will be unreachable, YaST complains about not being able to get the list of repositories for the self-update. And, after that, you get another complain because the fallback repository is not accessible. Two error messages and 2 timeouts.

And the situation could be even worse if you don’t have access to a DNS server (add another error message).

So after some discussion we’ve decided to show such errors only if the user has specified SMT or a custom self-update repository (which is also possible). In any other case, the error is logged and the self-update is skipped completely.

You can find further information in our Self-Update Use Cases and Error Handling document.

During upcoming sprints, we’ll keep working on making the self-update feature great!

Configuring workers during CaaSP installation

While CaaSP release approaches, our team is still working hard to satisfy the new requirements. Thankfully, YaST is a pretty flexible tool and it allows us to change a lot of things.

For CaaSP installation, YaST features a one-dialog installation screen. During this sprint, configuration of the Worker role has been implemented, including validation of the entered URL and writing the value to the installed system. You can check the animated screenshot for details.

The CaaSP worker configuration

New desktop selection screen in openSUSE installer

The world of Linux desktop environments change relatively quick, with new options popping-up and some projects being abandoned. Thanks to the openSUSE’s community of packagers we have a lot of these new desktop environments available on the openSUSE distributions. But the status of those packages for openSUSE is also subject to changes: some desktop environments are poorly maintained while others have a strong and active group of packagers and maintainer behind.

The YaST Team does not have enough overview and time to watch all these desktop environment and evaluate which one is ready or not for being in the installer’s desktop selection screen. So the openSUSE Release Team decided to replace this dialog with something a bit more generic but still useful for newcomers.

They asked the YaST Team to come up with a new dialog featuring the two openSUSE main desktops (KDE Plasma and GNOME) and allowing the easy selection of other environments without reworking the dialog in the future. The goal of that new dialog was to replace the existing one you can see in the following screenshot.

The old desktop selection screen

We decided the new dialog should rely on patterns for several reasons. The main one is that the set of patterns is under the close control of the openSUSE community, which looks more closely than us to the desktop environments and their integration into the distribution. Moreover, each pattern specifies its own icon and description that can be somehow re-used by the installer.

We also took the opportunity to merge this almost empty and outdated dialog with the new one.

The old additional repositories screen

Addons are no longer produced for openSUSE, so only the second checkbox made any sense nowadays. Moreover, the functionality of that second checkbox directly influence the available selection of patterns, so it made more sense to merge everything in a single screen that keeping an extra step in the installation just to accommodate a checkbox.

So we sent a proposal for the new dialog to the opensuse-factory mailing list and, after implementing many of the ideas discussed there (like better wording or using a button instead of a checkbox for the online repositories), this is the new dialog that replaces the two ones mentioned above.

The new desktop selection dialog

Selecting “custom” will take the users to the already existing patterns selection screen. Just in case you don’t remember how that screen looks like, you can check this image.

The patterns selection screen

If these screenshots are not enough to make your mind about the change, you can check the following animation, in which KDE Plasma is initially chosen to be changed at a later point (going back in the workflow) to LXQt.

Desktop selection animation

It will take some time before the changes hit the Tumbleweed installer, since they obviously have a non-trivial impact on the openQA tests, that will need some adaptation.

We would like to thank everybody that contributed to this new feature by providing feedback and suggestions through the mailing list. Once again, the openSUSE community has proved to be simply awesome!

Storage reimplementation: improved handling of logical partitions

Our reimplementation of the storage layer keeps getting improvements here and there. With the base x86 case working, we are now implementing the tricky parts, like the partitions renumbering that takes place when dealing with logical partitions in a MBR style partition table.

With GPT partition tables or with primary partitions in a MBR partition table, the partition gets a number (like sda1) when it’s created and keeps that number for its whole lifetime. But logical partition gets constantly renumbered when other partitions are created and destroyed. We need to know in advance what device name every partition will get in order to configure everything beforehand and only commit the changes to the disk when the user clicks in the “install” (during the installation process) or “commit” (if running the expert partitioner).

Now, libstorage-ng is able to simulate in memory the re-numbering process, so we can calculate all the settings related to partitioning (like the configuration of the bootloader) before really touching the disk.

Making kdump work in CaaSP

When you enable the Kernel Crash Dump (kdump), you probably expect that you will get a kind of core dump, like you do for regular processes. What you might not expect is an automatic reboot. That is a nice bonus. If you are tired of waiting for an actual kernel crash, you can test your kdump setup by triggering a crash yourself. Just run this as root:

echo c > /proc/sysrq-trigger

The way kdump works is by allocating some extra memory at boot time and putting a second kernel+initrd there. When the main kernel realizes it is crashing, it transfers control (by kexec) to the other one, which has only one purpose, to dump the memory of the crashed kernel.

In the upcoming Containers as a Service Platform, kdump was not working because the root filesystem is read-only there and we were not able to create the kdump initrd. We fixed it by creating it just after the RPMs are installed, when the FS is still read-write.

Fixes for Snapper in CaaSP

Kdump was not the only component affected by the fact of having a read-only filesystem in CaaSP. Snapper was also running into problems when trying to execute the rollback and cleanup operations. Now the problems are fixed. While working on that we did enough interesting findings to deserve a separate blog entry. So you can expect a new entry in the Snapper.io blog soon.

Snapshot-related improvements in the expert partitioner

While we work on the new storage system, we are still taking care and improving the current one that will continue to be shipped with SLE12-SP3, SUSE CaaSP and openSUSE Leap 42.3. During this sprint we introduced a couple of usability improvements in the expert partitioner related to Btrfs subvolumes.

First of all, we moved the “Enable Snapshots” checkbox that was pretty much hidden in the “Subvolume Handling” dialog to the place where it really belongs – below the selector of the file-system type.

New location for enabling snapshots

In addition, we added the warning you can see in the screenshot below for users enabling snapshots in a potentially problematic setup.

Warning for snapshots in a small partition

Bring back the beta warning on CaaSP

And talking about warnings, we improved the CaaSP installation dialog to show the Alpha/Beta product warning at the beginning. After changing the CaaSP installation to use the all-in-one dialog described in our previous post, this feature was lost as it is part of the initial EULA dialog. Now it is back and the users should now properly see the current product status.

The CaaSP alpha/beta warning pop-up

Storage reimplementation: encrypted LVM proposal

Back to our storage layer reimplementation, the new system is now able to propose a setup with encrypted LVM. Since some sprints ago, we were already able to propose a partition-based and a LVM setup. That means the new proposal is now feature-pair with the old one, with the only exceptions of Btrfs sub-volumes (that shouldn’t be a big challenge) and s/390 storage (that is still not properly managed by the underlying libstorage-ng).

The bright part is that the new code is written with re-usability in mind, which means implementing an encrypted partition-based proposal (with no LVM) would be trivial. The new code is 100% covered by automatic unit tests and scores to the top in all the automatic quality checkers we have run (like Rubycritics, CodeClimate, and Rubocop).

So now we are prepared for whatever the future brings, having lost no single feature in the way.

Storage reimplementation: prototype if the UI for the proposal settings

The next challenge is to make the power of that new storage proposal available to the users through the user interface. In the previous post we presented the document describing the general ideas we wanted to discuss with UX experts.

It’s our pleasure to announce we met the experts and we very easily reached an agreement on how the user interaction should be. During this sprint we already implemented a prototype of the future proposal settings wizard that is, as usual, included in our testing ISO.

Now that we have solid foundations, it’s very likely that the following sprint will result in the fully working wizard, with almost-final wording and design.

Support for CaaSP added to the AutoYaST integration tests

While fixing a problem with AutoYaST and CaaSP, we decided to extend our AutoYaST integration testing tool to support CaaSP. But, as a side effect, we also made some additional improvements that were long overdue (like improving the procedure to download ISOs, reducing timeouts, etc.).

Now, our internal Jenkins instance takes care of running AutoYaST integration tests for the development version of CaaSP (as it already does for SLE 12 SP2).

Services manager moved to first auto-installation stage

AutoYaST allows to define a set of services to be enabled/disabled in the installed system, applying this configuration during the 2nd stage (after the first reboot).

Unfortunately, this approach won’t work for CaaSP because, in that case, the 2nd stage won’t be available. For that reason, during this sprint, we have adapted AutoYaST to write services configuration during 1st stage.

As usual, not only CaaSP, but other future (open)SUSE versions will benefit of this change.

Faster YaST installation when the release notes cannot be downloaded

Sometimes a very small simple change in a program makes a very noticeable impact in its everyday use. That’s the case of a fix implemented during this sprint. YaST usually re-tries to download the distribution release notes several times if the first attempt was unsuccessful. Although that’s correct from a general point of view, there are cases in which retrying makes no sense and only delays the installation. We have now added failed DNS resolution to that list of cases, which should result in a noticeable faster installation in many scenarios.

Moreover, in the case of AutoYaST we now skip downloading the release notes completely. Downloading them don’t make us much sense in the kind of unattended scenarios handled by AutoYaST and skipping this step and all the possible associated problems result in a more robust process.

Better continuous integration for Libyui

In the previous sprint we switched to Docker at Travis so we could build and test our packages in the native openSUSE system instead of the default Ubuntu. This sprint we did the same change also for the Libyui library which implements the UI part of YaST.

Originally we could not build the Libyui packages at Travis as either the required build dependencies were missing or were present at a very old unusable version. With this switch we can easily use the latest packages from openSUSE Tumbleweed and thus enable Travis builds for all Libyui packages.

See you after Hack Week!

As already mentioned, this week is Hack Week 15! So our Scrum process will be on hold for some days. That doesn’t necessarily mean the YaST development will stall. Since there are quite some YaST-related projects in the Hack Week page, you can expect YaST to simply go in unexpected directions. 🙂

And remember Hack Week is not a SUSE internal initiative, we are open to collaboration from anybody within or outside the company. So jump in and have fun with us!

Posted in Desktop, Distribution, Factory, GNOME, Hackweek, KDE, Packaging, Programming, Software Management, Systems Management, XFCE, YaST | 5 Comments »

Highlights of YaST development sprint 29

December 22nd, 2016 by

It’s Christmas time and since (open)SUSE users have been nice, the YaST team brings some gifts for them. This is the result of the last development sprint of 2016.

As you may have noticed, in the latest sprints we have been focusing more and more in making SUSE CASP possible. That’s even more obvious in this last sprint of the year. For those that have not been following this blog recently, it’s probably worth to remember that SUSE CASP will be a Kubernetes based Container As a Service Platform.

But our daily work goes beyond CASP, so let’s take a look to all the highlights.

More improvements in the management of DHCLIENT_SET_HOSTNAME

In the previous report we presented the changes introduced in yast2-network to make the configuration parameter DHCLIENT_SET_HOSTNAME configurable in a per-interface basis.

One of the great things about working in an agile an iterative way, presenting and evaluating the result every three weeks, is that it allows us to detect room for improvements in our work. In this case we noticed some discrepancy in the expectations of Linuxrc and yast2-network and also some room for improvement in the code documentation and in the help texts.

Thus, we used this sprint to refine the work done in the previous one and tackle those problems down.

Improved error message

Ensure installation of needed packages

Another example of iterative development. We already presented in the report of the 26th development sprint a new mechanism to detect when the user had deselected during installation some package that was previously pre-selected by YaST in order to install the bootloader. Since the new functionality proved to work nicely, we decided to extend it to cover other parts of the system beyond the bootloader.

The software proposal now contains an error message including a list of missing packages or patterns, in case the user deselects some needed items.

Warning about missing packages

After clicking the Install button the installation is blocked, the user must resolve the problem either by selecting the packages back or by adjusting the respective YaST configuration (e.g. do not install any bootloader and disable the firewall).

Blocking an incomplete installation

Rethinking the expert partitioner

May we insist one more time on the topic of using Scrum to organize our work in an iterative way? 😉 As our usual readers should already know, we structure the work into minimal units that produce a valuable outcome called PBIs in Scrum jargon. That valuable outcome doesn’t always have to be a piece of software, an implemented feature or a fixed bug. Sometimes a document adds value to YaST, specially if it can be used as base to collaborate with people outside the team.

Our readers also know that we are putting a lot of effort in rewriting the whole storage layer of YaST. That also implies rewriting the most powerful tool known by humanity to define partitions, volumes, RAIDs and similar stuff – the YaST expert partitioner.

It would be great if we could use the opportunity to make it both more powerful and more usable. You can take the first part for granted, but we are not so sure about our UI design skills. That’s why we wanted to have a base to discuss possible changes and alternative approaches with UX (user experience) experts. And we decided that it was worth to invest some time to create a document collecting the state of the art and some ideas for the future and to send that to SUSE experts in UX and to anybody with some interest in the topic.

Here you can find that fine piece of documentation. Take a look to that document if you want to peek into YaST developers’ mind. That’s the kind of stuff we discuss when we are about to start rewriting something… specially something that need to serve hundreds of different use cases.

And of course we would like to know your ideas or thoughts. We usually discuss this stuff at the public #yast IRC channel and at the yast-devel mailing list. But if you prefer so, you can simply open an issue at the repository hosting the document. Whatever works for you.

Rethinking yast2-network

But that was not the only documentation PBI finished during this sprint. Inspired by the first fruits of the storage layer reimplementation, we decided yast2-network also deserves a reincarnation.

As we did in the past with yast2-storage and libstorage, the first step is to collect as much information as possible about what can be currently done with the module and how it behaves in several situations, specially in tricky or complex scenarios. The outcome was three documents, one about the behavior during installation (installation.md), a second one about AutoYaST (autoinstallation.md) and another collecting general features (features.md).

CASP: merged dialogs for root password and keyboard layout

CASP is a product targeted to a quite specific use case with simplicity as a main priority. The installation process has been streamlined to a minimal set of dialogs to configure just the very basic stuff. Among other removed things, there is no step to configure the system language. That can be a problem when entering the root password (the only user that will be created during installation), since the language settings screen is normally also used to select the keyboard layout.

The implemented solution is shown in the screenshot below. As you can see, the keyboard layout and root passwords selections are merged into a single step. As a bonus, we made both widgets more reusable, opening the possibility to place the root password widget or the keyboard layout selection anywhere.

Keyboard layout and root password screen

Storage reimplementation: handling GPT disks in the installation proposal

After several sprints reporting small steps forward, in the 27th sprint we were happy to announce that our testing ISO for the new storage stack was fully installable under certain circumstances. As we reported, it worked in UEFI or legacy systems with the only requirement of having a pre-existing MBR partition table in the disk.

Now we can say it also works with GPT partition tables and even with systems with a mixture of both technologies.

Making the GPT scenario work was much harder that it sounds due to several factors, like the strange way in which parted handles partition types in GPT or some peculiarities in the way the space is distributed in such partition tables.

But now our test ISO can install a fully functional system in the four combinations of MBR/GPT partition table and UEFI/Legacy boot, as it can be seen in the next image.

Storage proposal in several scenarios

The storage reimplementation gets its own openQA instance

But there are better ways than screenshots to prove that something is working, even to prove it keeps working after future modifications. And in (open)SUSE we have one of the best tools for that – openQA.

We have always considered having the new stack tested in openQA as the first big milestone in its development (and we are finally there!) but we are aware that openQA.opensuse.org is already quite busy testing a huge combination of products, architectures and scenarios… even testing releases of openQA itself. Fortunately openQA is free software and can be installed anywhere so we created our own instance of openQA to test YaST stuff, specially the new storage layer.

So far, that instance is hosted in the internal SUSE network, which is enough for us to get continuous feedback about the changes we introduce. In addition to installing the new instance and configuring it to continuously grab and check the latest testing ISO, we had to introduce several changes in the ISO itself with the goal of keeping our tests as aligned as possible with the tests performed in the current Tumbleweed version by openQA.opensuse.org.

For example, we made sure the ISO was properly signed to avoid the need to always pass the insecure=1 boot argument. We also included several packages that were missing in order to make sure the ISO included all the software checked during the so-called MinimalX test and to make sure it shared the look and feel with a regular Tumbleweed, since many openQA checks are screenshot-based.

From now on, we can back every new feature with the corresponding integration tests, something crucial to ensure the quality of a piece of software meant to handle storage hardware.

Making Snapper work without DBus

As you may know, some YaST team members are also the main developers and maintainers of Snapper, the ultimate file-system snapshot tool for GNU/Linux systems.

Normally the snapper command line tool uses DBus to connect to snapperd which does most of the actual work. This allows non-root users to work with snapper.

There are however situations when using DBus is not possible and not being able to work in those situations was limiting Snapper’s usefulness. Now with the latest version all snapper commands support the –no-dbus option. This evolution is worth a blog post by itself… and, of course, we have it. To know all the details check this post at Snapper’s blog.

CASP (and beyond): improved roles

Do you remember the system roles feature introduced during development sprint 16 and improved in subsequent sprints? In case you don’t, let us remind you that system roles allow to define many settings of the installation just by choosing one of the offered roles. That’s only possible, of course, in products making use of that feature, like SLES.

For CASP we will have 3 different roles, as shown in the following screenshot.

CASP system roles

The main difference between these three roles is the selection of patterns to be installed. But apart from that, the Worker role will offer an extra step during installation allowing the user to specify the address of the so-called Administration Dashboard.

Configuration screen for the Worker role

That relatively small detail implied the development of a full new feature in the installer – the ability of a given role to define it’s own specific configuration, including the dialog to interact with the user. As expected from any other installation dialog, you can go back and forward without loosing the entered information. If the user goes back and selects a different role, then this additional dialog is not run again.

That new feature is, of course, not specific to CASP and could eventually be used in other products and roles. Just as a crazy example, openSUSE could decide to introduce a role called “NTP server”, running the YaST NTP server configuration right after the user selecting the role.

Other CASP related features

As already said, we have been focusing quite a lot on introducing features that are needed for CASP. It’s worth mentioning, in case it’s still unclear, that CASP will NOT ship its own adapted version of YaST. All the features introduced in the installer are in fact configurable and available for all other products as well. There is only one YaST codebase to rule them all.

Let’s briefly describe some of the introduced CASP-specific (at least for the time being) features.

CASP always uses Btrfs as filesystem for the root partition. At the end of the installation, the root btrfs subvolume will become read-only. All the other subvolumes will stay as read-write, as shown in this screenshot taken right after rebooting at the end of the installation process.

CASP subvolumes

It makes no sense to update from any existing product to CASP. Thus, CASP media should not show an “update” option when booting, even if it’s still possible for advanced users to pass the UPDATE boot parameter. Since we needed to modify the installation-images package, we took the opportunity to make the “update” option and other settings configurable in a per product basis and we unified SLES and openSUSE packages, so now they share a single branch in the source code repository.

CASP is targeted to big deployments extended all over the world. To make possible the synchronization of geographically distributed nodes, the UTC timezone is enforced in every CASP installation. Thus, we implemented support for products to enforce a given timezone in the installer. Take into account this is different from a default timezone.

Last but not least, it has already been mentioned that the CASP installation workflow will have very few steps. That also affects the screen displaying the installations settings summary. In comparison to a regular SLES, some options must disappear because they are not configurable and some other sections must be added because they are not longer presented as a separate previous step. So far, this is the appearance of the installation settings screen in the current CASP prototype.

Installation settings in CASP prototype

…and a surprise about the blog

We also prepared a Christmas gift related to the blog. The technical aspects are solved, but we are ironing out the administrative details. So you will have to wait until the next sprint report to see it in full glory. But, as the Spanish proverb says, “good things are worth waiting for”.

See you next year

That’s enough to report from our December sprint, we don’t want to bore you with every small bug fix. And talking about things that are worth waiting for, our next report will very likely be published at the beginning of February 2017.

That’s because we will put our Scrum process on hold during the Christmas session. We will restart it on the second week of the year, after the visit of the Three Wise Men. In several countries, it’s a tradition that the Three Kings bring gifts to the kids that have been nice, so let’s expect they bring us some new members for the team!

Posted in Distribution, Factory, Programming, Quality Assurance, Software Management, Systems Management, Usability, YaST | 3 Comments »

Highlights of YaST development sprint 19

May 18th, 2016 by

Here we are after another Scrum sprint with our usual report about the activity in YaST development.

Trusted boot

YaST bootloader module got a new option, Trusted Boot (FATE#316553). It installs TrustedGRUB2 instead of the regular GRUB2. Trusted Boot means measuring the integrity of the boot process, with the help from the hardware (a TPM, Trusted Platform Module, chip).

It enables some interesting things which we unfortunately haven’t provided out of the box. We give you a bootloader which measures the boot integrity and places the results in Platform Configuration Registers (PCRs).

First you need to make sure Trusted Boot is enabled in the BIOS setup (the setting is named Security / Security Chip on Thinkpads, for example). Then you can enable the new YaST Bootloader option that will install TrustedGRUB2.

Trusted boot in YaST Bootloader

In the description of this pull request you can find a more detailed explanation including some commands and hexadecimal dumps to check the result. Geek pr0n!

SSH keys importing… and a glance at a YaST Developer’s life

When looking at any software project, it’s common to find some feature or piece of code that is there due to the so-called “historical reasons”. YaST2 code-base has been around since 1999, adapting to changes and new requirements in a (almost literally) daily basis since then. That leads to a new level of heritage – the “prehistoric reasons”. Working in the YaST Team implies coding, debugging, testing… and archaeological research.

We got a bug report about the installer “stealing” some SSH host keys (but not all of them) from previously installed systems. It was actually the effect of a little-known YaST feature that can look surprising (not to say weird) at first sight. Ten years ago, somebody decided that when installing SUSE in a networked environment, where people use SSH to log in, it was better to import SSH keys from a previously installed Linux than to get that “ssh host key changed” for everybody who tries to connect. The rational was that forcing everybody to change the ~/.ssh/known_hosts file often could become a security breach, since people could get used to ignore the security warnings. Welcome to the world of historical reasons. 🙂 Moreover, it was decided that the operation should be performed without showing any information to the users, in order to not confuse them.

More or less at the same time (we are still talking about 2006), it was decided to introduce importing of users from an existing system, this time with user interaction. The YaST developers decided that it would be fine to share some mechanisms in the implementation of both features. Another step into the historical reasons void.

Fast forward to the present. After several fate entries, bug reports and redesigns over the years, we decided to make the importing of SSH host keys more visible and usable, to make both functionalities (SSH import and users import) more independent and more clean and to take the first step to clean up the insanity introduced through the years (see fate#320873 for details).

The installer does not longer silently just import the SSH host keys from the most recent Linux installation on the disk (remember, you might have several distributions installed), it has now become a part of the installation proposal dialog:

SSH host keys proposal

And from there you can change it:

SSH host keys selection

Notice that one of the options is “none”, so copying of previous keys is not longer enforced. In addition, now is also possible to import the rest of the SSH configuration in addition to the keys.

Disabling local repositories

When installing from a local media, like a CD/DVD or an USB stick, those sources were still enabled after the installation. It potentially could cause problems during software installation, upgrade, migration, etc. because an old or obsolete installation source is still there. On the other hand, if the source is physically removed (for instance, ejecting the CD/DVD), zypper will complain about that source not being available.

Fortunately, this behavior will change in future SUSE/openSUSE versions. Now, at the end of the installation, YaST will check every local source disabling those whose products are available also through a remote repository (so they’re not needed at all).

Smart bonding with NPAR and SR-IOV interfaces

Support for bonding interfaces with NPar or SR-IOV capabilities has also been improved during this sprint. Too many weird words in one sentence? Don’t worry, we actually love to explain things.

Bonding is a way to combine multiple network connections to increase the throughput and bandwidth and to provide redundancy. On the other hand, NPAR and SR-IOV are technologies that provide the capability to create multiple virtual devices from the same physical or ethernet port.

Until this sprint, YaST offered no way to know whether two interfaces with these capabilities were sharing the same physical port. As a result, users could bond them without realizing that they were not getting the desired effect in terms of redundancy.

Information about the physical port ID has been added to all the relevant dialogs for all devices supporting it, like it’s shown in the following screenshots.

Physical ports

More physical ports

Additionally, the user will be alerted when trying to bond devices sharing the same physical port.

Bonding warning

Last but not least, following the Boy Scout Rule (also known as opportunistic refactoring), we took the opportunity to fix some small quirks in YaST Network, like the counter-intuitive sorting of devices in some lists.

Paying our debts: much better cleanup rules in Snapper

Somebody said once “a promise made is a debt unpaid“. In the previous post we promised you all an article on Snapper.io detailing the new space-aware cleanup introduced in Snapper. Here you are!

That’s all folks

Enough for a blog post. That’s of course not all we did during the last sprint (to the contrary, it’s just around 20% of the finished work according to the story points count) but, you know, we need to go back to hacking!

Posted in Distribution, Factory, Programming, Software Management, Systems Management, YaST | Comments Off on Highlights of YaST development sprint 19

Highlights of YaST development sprint 18

May 2nd, 2016 by

The wait is over, the report of the latest Scrum sprint of the YaST Team is here! In the previous post we promised that after this sprint we would have much more to show… and now we do. This sprint was quite productive, so let’s go straight to the most interesting bits.

More improvements in the self-update

The YaST self-update feature mentioned in the two previous blog posts (sprint 16 and sprint 17) has also received a couple of improvements.

At this gist (with screenshots) you can see some of the fixes done when going back and forward in the work-flow after an installer self-update.

But even more important are the improvements done in AutoYaST to properly handle the new feature. If you want to use your own self-update repository you have to specify it on the boot command line. This is easy in a single installation, but if you want to install many machines using AutoYaST then writing the possibly long URL at each installation again sounds annoying.

In this sprint we improved handling of the custom update URL so that it can be read from the AutoYaST XML profile. Now you need to write the custom URL only once into the profile and it will be used automatically in every AutoYaST installation.

There is not much to show in the UI, the log proves that the self update was really loaded.

AutoYaST self-update

You can find more details about specifying the URL in the profile and evaluating the self-update repository URL at the documentation.

Goodbye perl-Bootloader

For quite some time, we have had the plan to stop using Perl-Bootloader as library for yast2-Bootloader and finally we managed to make it happen. There were many reasons for this change.

From the user point of view, the most visible reasons were speed and size. Getting rid of Perl-Bootloader means that we not longer perform hardware probing twice (one for Perl-Bootloader and another one for grub2-config), making kernel updates faster. In addition, we are not only simplifying and reducing the size of yast2-bootloader itself. Dropping the Perl libraries and other associated dependencies, makes possible to have a smaller minimal system, something quite relevant in various environments and scenarios.

From the developer point of view, the main reason was to unify the used programming languages and also reuse existing solution for file reading and parsing. Now yast2-bootloader uses an Augeas file parser and serializer, which does not only allow us to have less code to maintain, but also offers smarter editing and better handling of the comments in the configuration files.

As nice side effects, this change also leaded to the removal of a lot of work-arounds, the simplification of the installation work-flow for bootloader and a very visible improvement in the source code quality. In addition, we improved the test coverage of the whole module to make sure we didn’t break things too much. But all those would be bold statements if we wouldn’t have some geeky numbers to prove then, so here they are.

Of course, after such a big rewrite is not unlikely that some new bugs will pop up, but we really believe the improvements are worth the price.

Storage reimplementation: calculation of partition location

For the storage-ng project we made a big step towards a modern system. We do not use geometries and cylinders for disks any longer. Instead we only use sectors.

The advantage for the user is much better control over the size of new partitions. Since so far the size had to be a multiple of the cylinder size, often 7.84 MiB, it was not possible to create very small partitions, e.g. for the BIOS boot partition. Also the size of partitions was usually not the exact number entered in YaST, e.g. 509.88 MiB instead of 512 MiB.

Now the size of partitions is as accurate as possible with the hardware, so usually a multiple of 512 B or 4 KiB.

Like in the past we also takes care of optimal partition alignment to avoid performance loss due to read-modify-write cycles.

Snapper: much better cleanup rules

Snapper is a great tool, but the current default configuration usually causes it to be a little bit too greedy with disk space. During this sprint a new set of cleanup rules was implemented, allowing a much more reasonable usage of the disk.

The feature is already developed, submitted to Factory and even integrated into the next version of SUSE Enterprise that is right now being cooked. But, as you all know, the testing work-flow of openSUSE Tumbleweed (staging projects, openQA and so on) can sometimes cause some delay until the new feature appears in the distribution. As soon as the new feature reaches Tumbleweed, a blog post explaining the new functionality will be published at snapper.io. Stay tuned!

Improved password protection for bootloader

The password protection widget in YaST2-Bootloader had not been changed since the GRUB1 times, but GRUB2 allows more fine tunning of the password settings. In fact, it contains a whole user management system with multiple users that can have different permissions and different passwords. For YaST we like to keep things simple, so we support only password for “root” user. That was confusing for some people, so we decided to improve the user experience by adjusting some labels… and fixing some typos in the process. 😉

Two pictures are worth a thousand words, so here you can see how the dialog looked before the change

BEFORE: boot password dialog

and below the new appearance, which adds more explanation to the password specification. Of course, the help text (not displayed in the screenshots) was also improved to reflect the changes.

AFTER: boot password dialog

Handling of default product patterns

The YaST installer implements several possibilities how the default patterns for the installed product should be specified. One of them is using Recommends RPM dependencies for the product package. (The other possibilities include control.xml file or the content file on the medium.)

The installer by default installs the recommended packages so the default patterns are automatically installed in the initial installation.

However, this approach makes troubles during system upgrade. If you remove some packages or patterns which are installed by default and then you do a system upgrade the installer will (silently) add the removed packages or patterns back as the Recommends dependencies will pull them in again.

The solution is to provide another mechanism for selecting the default patterns. The new implementation allows using a specific Provides RPM package tag which specifies the default pattern name for the product.

The new tag is defaultpattern(pattern) where “pattern” is the name of the default pattern. YaST looks for these specific tags and collects the pattern names for all installed or updated products. This step is skipped during package based system upgrade so this should avoid adding unnecessary packages in system upgrade.

Obviously it depends on the products themselves to switch from the RPM dependencies to this new style. So far it is supported only by the SLES12 SP2 High Availability addon.

See more details at https://github.com/yast/yast-packager/wiki/Selecting-the-default-product-patterns.

Storage reimplementation: another step closer to the perfect booting layout

A couple of reports ago we made public our intention to squeeze some booting experts brains in order to create Ruby classes capable of always proposing a sensible partitioning schema. The delicious result is finally at the repository, accompanied with a rather complete document explaining the logic behind the code.

If reading Ruby code is not your thing but you are geeky enough, you can also review the output generated by the RSpec unit tests, which shows how the new partitioning proposal will behave in all scenarios from the booting requirements point of view.

Storage reimplementation: other improvements in the new proposal

Apart from the already mentioned improvements in setting the boot-related partitions, the brand new partitioning proposal gained the ability of shrinking Windows partitions when needed (in a sensible and gentle way) and reusing partitions that can be shared with other installed systems, like swap of PReP. Even when sharing a swap partition is not possible and the proposal needs to delete an existing swap partitions to create a different one, it will take care of reusing the old label and UUID, so other systems in the same computer can still find a suitable swap for them even after changing the partitioning layout. Because we all can be better citizens…

In addition, we took the first steps to develop a smarter way of proposing a solid partitioning schema when the free space is split over several different disks or disk chunks. Is still a work in progress, but you can trust that in the future (open)SUSE will do a nice job when looking for a home for itself in your sparse hard disk space.

Prevent wrong usage of LDL DASD disks in S/390 mainframes

The management of storage devices in a Linux system running in a S/390 mainframe is a tricky topic full of corner cases. Depending on the type of disk, its format and other factors, some apparently simple operations (like creating partitions) are not possible. If you are interested in the subject and want to learn a lot of weird acronyms, we would recommend this article.

We realized that the expert partitioner allowed the user to specify some operations that were actually not supported, which resulted in the installation being aborted at a subsequent step. To avoid that situation, we improved the expert partitioner to detect those unsupported S/390 configurations and alert the user, like it’s shown in the screenshot below.

LDL DADS popup

AutoYaST: moving network device renaming to first stage of installation

We refactored the network module to unify the handling of device naming. Now AutoYaST assigns the naming udev rules in the 1st stage of the installation already. (yast2-network-3.1.147, autoyast2-3.1.121)

New toys for the team

We got a new server to run integration tests via openQA and AutoYaST.

The AutoYaST tests now used to take 8 hours but now finish in 1.5 hours.

Collaboration with other (open)SUSE Teams

Even though we call ourselves “the YaST team” we are happy to share the project with other teams in the company and people in the community. During this sprint, we had a chance to review code for two significant features.

The authentication client module, dealing with LDAP, Kerberos, Active Directory, NSS, PAM, and SSSD, got a big upgrade (yast2-auth-client-3.3.7).

Since a couple of months, Tumbleweed has had a package for firewalld (see FATE#318356 ). Work is underway to make YaST aware of it but it has not been merged yet. If you are interested you will have to find it in the git repo.

In closing

Definitely, this was a very productive sprint and, as usual, this report is just a sample of the total work delivered by the team. To be precise and to please number lovers, the features and fixes covered by this report represent 85 Scrum story points out of a total of 124 delivered ones. Hopefully enough to keep you entertained until the next report in about three weeks. See you then!

Posted in Distribution, Factory, Programming, Software Management, Systems Management, YaST | 1 Comment »

How kiwi can help to cleanup your system

September 25th, 2012 by

After some iterations of updating the system with zypper dup or yast and some years of service with the system you will find out that there is a lot of dust which is obsolete or has been forgotten. Recently I had the problem that I need to move my 32bit system to a 64bit system and thus the way to go was to migrate the running system into an image description, build a 64bit image from it and install that on the 64bit machine. But the most important part was to cleanup the running system and find out what it really contains. The report kiwi migrate created here was helpful and so I think it might be helpful for others too. Just call:

kiwi –migrate mySystem

It will end up with some data below /tmp/mySystem which of course can be removed at any time without any risk. Most interesting is the html report generated which you can view with any browser. So far kiwi collects the following information:

  • kernel version and kernel specific kmp packages
  • hardware dependent packages
  • installed gem packages
  • repository checkouts
  • rpm packages installed multiple times
  • rpm packages which could not be found according to the current repo setup including version and repo information
  • tree of modified files, packaged but changed
  • tree of custom files, those which doesn’t belong to any package or other part of the bullet list

basically the use case of kiwi migrate is to migrate the running system into an appliance description but I’m not there yet. There is still room for improvement but I think it still can help to cleanup the system and to see what is installed on the system and not managed by a package manager

I have tested this since openSUSE 11.4

Remember to have fun 🙂

Posted in Base System, Distribution, Packaging, Software Management, Systems Management | Comments Off on How kiwi can help to cleanup your system

Updates for openSUSE Edu Li-f-e

September 15th, 2012 by

With the release of openSUSE Edu Li-f-e 12.2, we also have new KDE waiting in the official update repository. To resolve a couple of conflicts you will need to add Education:update repository before running yast2 online_update.

Follow these steps:

- Add Education:update repository by using this 1-click install, remain subscribed to the suggested repositories or via command line:
  zypper ar obs://Education:update/openSUSE_12.2 Education:update then run yast2 online-update.
 - select replacement for kioaudiocd
 - select deinstallation of yast2-qt-branding-life
 - Proceed with the update

Posted in Derivative, Distribution, Education, Software Management | Comments Off on Updates for openSUSE Edu Li-f-e

Systems Management Zeitgeist

September 14th, 2010 by

Dear Lizards,
This recent release from IT World on the best Linux distributions out there caught my eye last weekend, as it declares “The package’s administration utility, YaST, is widely acknowledged as one of the best” in its entry on openSUSE and SLE (the documentation also drew praise, distinguishing itself as “some of the best printed documentation you’ll find for any distro“), and reminded me I wanted to share some of the positive feedback I collected during our 11.x development and after final release.  Ready? Here we go.

Some of the initial ‘Net commentary was all centered on performance and memory footprint, from Snorp’s “I don’t think it’s possible to overstate just how much of an improvement it really is” to Duncan’s benchmarks providing interesting numerical comparisons like  “Yum uses about 9 times more memory” (and takes several times longer).  This was refreshing given that at the same time Yum’s less-than-nimble footprint was drawing some interesting comments from Zed and Zbr.

Eventually, the improvements rolled over to the press, with Jason Perlow proclaiming 11 RC1 the Mercedes-Benz to Ubuntu’s Wolkswagen. Jason had plenty of praise in his review, but I am singling out “the most beautiful installer program I have ever seen” and “quite impressed with how fast the package repository management works” since this is the Systems Management team’s ticker-tape parade, after all.  Our then Community Ambassador Zonker followed up with his Package Keeper piece on the special that Linux Pro Magazine issued for the 11.0 release, focusing on package management as “one of the most impressive advances” in the release (link sadly missing as article still paywalled).   Linux Format retorted with “One of our favorite features of SUSE is the one-click install system” and “faster than any other package manager we’ve seen, and on top of that it looks great, too” in their What SUSE Does Best review (no link, as LXF requires subscription).

Finally, with the release of our Enterprise distribution, the commentary rolled over to our corporate customers, as I previously reported when one customer I like to track personally as particularly representative reported a 300% speed improvement in rolling updates to production.

Afterwards, we have moved up live distro upgrade (more famously known as zypper dup) to fully supported status, quickly receiving loud praise from a Linux Journal editor with clearly too many Debian-using friends.  We do relate to his plight, in a tongue-in-cheek manner, and are happy to help.  Indeed, other distributions have started adopting Zypper as well, with Ark leading the way.

So what is next for us? Well, with Btrfs around the corner, integrating snapshot and rollback into the update system stands clearly out from the crowd: an undo button to painlessly bring back the system to where it was before your last upgrade. Stay tuned!

The package’s administration utility, YaST, is widely acknowledged as one of the best,The package’s administration utility, YaST, is widely acknowledged as one of the best,

Posted in Boosters, Documentation, Software Management, Systems Management, YaST | Comments Off on Systems Management Zeitgeist

Software search trick

August 4th, 2010 by

Do you use software.opensuse.org and get an error message “search limit reached” when searching for a generic term like “perl” or “kde”? Here is the solution:
The software search now also supports matching for exact package names, just put your search string into double quotes! See for example perl or kde4

Tags:
Posted in Boosters, Build Service, Software Management | Comments Off on Software search trick

Road to 11.3 : when pattern are not your friend, pre selection can be a trap

June 10th, 2010 by

So it’s time to take some hours to test our future version.

Today I start a fresh M7/Factory install : booting from pxe. The test case is build quickly a minimal server text mode.

Just uncheck the auto configuration, we are after all linux admin. Choose your partition keyboard, language (en recommanded for server) etc … normal.

Just before starting the install check software :  click on installation resume . You will discover that base-system-pattern would like to install a kernel-desktop, wtf why we want a server !

So there’s a new ticket about that : https://bugzilla.novell.com/show_bug.cgi?id=613216

I express the fact that it would be nice to have a new pattern selected when we choose minimal install server text mode.

And you what about your opinion about pre-selection or having a base-system-server pattern … Please comment, & vote on bugzilla

A pattern guru wanted to build a patch for that.

Posted in Architectures, Base System, Factory, Kernel, openSUSE 11.2, Quality Assurance, Software Management, YaST | 3 Comments »

News from the Zypper Revolution

April 5th, 2010 by

Maybe revolution is a bit strong, but at this hour in the night I can probably be excused for using a bit of hyperbole – besides, nothing great in the world has ever been accomplished without passion as Hegel would have it, so I am on solid ground here.

I have been going over customer feedback from Novell’s Brainshare conference for my internal “Systems Management Zeitgeist” report, and there are a couple of points I just had to share with you all as they are plain simply inspirational.

Our update stack is, well, zippy.  Like greased lightning, according to this happy SUSE Linux Enterprise customer:

Zypper updates a Linux system across major versions in 5 minutes, full Oracle server update done in 15 minutes

We of course appreciate speed in of itself, as a technical achievement powered by enhancements like libsat and DeltaRPM, and Community users share this point of view with us.  But Enterprise users have a different and equally valid point of view: administrator time is costly, and while many management consoles exist, industry data shows that tools do help, but not nearly enough: administrators are still involved, personally, in most Systems Management tasks —  I could quote analyst data, but not at this wee hour, so just trust me on this point.

This one medium-sized customer actually took the time to calculate what the time savings meant to his business:

The faster update stack is resulting in 56,000 dollars in [operational budget] savings

It is not everyday a customer gives you a precise dollar number in describing what a technology’s impact is on his expenses — So I just had to share it with you all, it is such a nice commentary on our effort’s tangible impact.

I can hear some of you wonder why I blogged this on my Lizard’s Community account, rather than on Novell’s Corporate site, since I am talking up Enterprise distro data and as the Systems Management guy I really have either option.  Good question!  I could say it is because I was not in the mood to dig up analyst quotes, and this setting allows me to be more cavalier and just waltz over those references, but there is a more important reason, read on.

We in the Systems Management team happen to think sleep is for the weak, and have been cooking up our next scheme for improvement — but we need your help to get there.

As the keenest observers among you have long ago noticed, with the 11.2 release we declared “zypper dup” a supported migration path, and received some accolades for it already.  But we all know that live distro upgrade migration across major version changes is a big endeavor, and we would like to solicit your help in improving it: if you have the time and inclination to test zypper dup and provide a properly filed bug report of any kinks you might discover, we would be delighted to use your feedback to improve the 11.3 implementation of this process.

Just a word of caution: comments to this entry, or bugs filed without sufficient data to be analyzed, are not going to further the result we all seek.  If you report something, make sure enough data to reproduce the issue is included, and that you are able to provide additional data upon request of the developer handling your report: if we cannot reproduce a problem, we cannot fix it.

Thanks in advance to those among you joining us in this effort!

Posted in Base System, Distribution, openSUSE 11.2, Software Management, Systems Management | 2 Comments »

« Older Entries
Newer Entries »