An Actionable Partitioner Overview Screen

Until now, the Partitioner landing screen has been useful to have a big picture of the devices in your system and as a shortcut to jump directly to the device page just with a double click over it. But, do you know what? From yast-storage-ng 4.2.74 on you can work directly with devices from that screen similar as you already do in the more specific pages, through the contextual actions added below the devices list. That means, for example, no more jumps to Hard Disks just to add a new partition nor resize an existing one. Enjoy

More details: PR 1024

Numeric Sorting in Tables

We have improved the sorting for tables in libyui, the UI library of YaST. So far columns were sorted directly by the text displayed, e.g. the device name or the size in the expert partitioner. For some use-cases this resulted in unexpected ordering, e.g. partitions of a disk were ordered “/dev/sda1”, “/dev/sda10”, “/dev/sda2”, and sizes were ordered “1 GiB”, “2 TiB” and “4 GiB”.

Now it is possible to provide a sort-key for every table entry which is then used instead of the displayed text. This allows the expected ordering and is already implemented for the tables in the expert partitioner as the two pictures below show.

Improving the NFS Module

YaST offers a specific module to configure your NFS shares. Similar to every YaST module, you can run it by executing yast2 nfs in your terminal, or by launching it from the YaST Control Center. But there is another cool way to use the YaST NFS module: opening the Expert Partitioner!

The Expert Partitioner offers a NFS section on the left menu tree where you can do everything that the NFS module provides. Thanks to that, you can configure your NFS shares at the same time you format your partitions!

But that integration needed some improvements after we migrated YaST to the new Storage Stack (a.k.a. storage-ng). Moreover, some bugs were detected when using the NFS module for mounting and unmounting shares, see for example bsc#1006815 and bsc#1151426.

All those bugs were fixed, ww the NFS module behaves as expected in both cases, when running in standalone mode and inside the Expert Partitioner. Note that now the current status of the existing shares is preserved. That is, an unmounted share will continue unmounted after editing it. Unmounted entries are indicated with an asterisk in the list of shares, similar to what the Expert Partitioner does for the rest of unmounted devices. All these improvements will be available for SUSE Linux Enterprise SP1, openSUSE Leap 15.1 and openSUSE Tumbleweed.

Installation Progress Improvements

We got some bug reports about how installation progress reporting works and while we were touching it, we also added a few smaller improvements to the code.

The first change is that nowadays installing from multiple discs almost never happens but still there was always a “Medium 1” column which did not make much sense. So we removed the column and if there is a multi-media source, it will be appended to the name if needed.

The second visible change is a new Unicode character ⌛ (hourglass) during the initial phase of RPM installation until the remaining time can be estimated.

The third change is that now the maximum time is always capped at 2 hours, so even if there are multiple sources and some of them took more then two hours, it always show just “>2:00:00” and even in total it is capped, so it can no longer show something like “>6:00:00”.

The fourth one is that now you can read the release notes without disturbances. Previously you would get switched to the package log tab after each package finished its installation. Now it will redraw only when you go back from the release notes screen.

The fifth one is a fix for showing the remaining packages, where it is shown only for the active source and not for all. So now it shows remaining packages for all repositories.

And last but not least we do a bunch of refactoring, code quality improvements and also adding automatic unit tests to reduce regressions in the future.

Tumbleweed before and now:

SLE before and now:

and new ncurses:

Doomsday Preparations: Retracted Packages

If a maintenance update is released for any of our supported products, it may happen that after its release we realize that it introduces new problems, so we have to unpublish (retract) it.

So far, our maintenance team always managed to find other solutions, but sooner or later it will happen that it takes too long to realize that an update was broken, so users will install it.

For that purpose we introduced a new status retracted for patches and packages. We hope that we will never need it, but if we do, we need it in a hurry — until a better, fixed version of those packages is released.

We added new filters “Retracted Packages” and “Retracted Installed Packages” to the package selection, and the affected versions are colored in red and get a [RETRACTED] marker in the “Versions” tab:

Those lists should always be empty. Also, retracted versions will never automatically be installed. If package versions are retracted, but are already installed, the “Retracted Installed Packages” view will be opened automatically when starting the package selection to make you aware of them. Then you can choose to manually downgrade to a previous version or to wait until a fixed version is available.

In general, don’t worry: We never needed this so far, and we hope that we will never need it. Still, we take precautions for the worst case.

More details: PR 82

Qt Package Selection Gets Faster

This came as a byproduct of the previous item: While working on the new filter views for retracted packages, we found that it could take a long time (10-20 seconds) when switching away from the “All Packages” view, so we started digging deeper to find out why.

We found it strange that clearing the package list on the right side of that dialog was so slow; considerably slower than filling it with all packages. After some investigations, we found that in the course of all those changes for all those Qt versions (since Qt 3.x in mid-2006) some internal housekeeping for those list items was now no longer necessary because later Qt versions took over more and more of those responsibilities, and our own housekeeping now got in the way of that and was considerably slowing it down.

Once we found the cause, the fix was easy: We threw out our own housekeeping code and are now relying on what the Qt widget does, and hey presto, clearing that list now happens instantly instead of taking 10-20 seconds.

More details: PR 82 (“Other Fixes”)

The code for that is partially in the Windows launcher. The Windows side actually prompts for the user name:
https://github.com/microsoft/WSL-DistroLauncher/blob/master/DistroLauncher/DistroLauncher.cpp#L44

and passes it to ‘adduser’:
https://github.com/microsoft/WSL-DistroLauncher/blob/1f8551f7e2ea22bba2e6fb02f01e7a5f7fb757f3/DistroLauncher/DistributionInfo.cpp#L14

That seems to be a Debian specific tool that also prompts for a password. We don’t have it in openSUSE. When done, the Windows part actually calls into the Linux environment again with ‘id -u’ to get the uid of the added user:
https://github.com/microsoft/WSL-DistroLauncher/blob/1f8551f7e2ea22bba2e6fb02f01e7a5f7fb757f3/DistroLauncher/DistributionInfo.cpp#L44

So in order to also prompt for the password we’d have to write a wrapper like the Debian one or implement another prompt in the launcher. Implementing such a prompt in Windows code seems boring to me. When writing a wrapper, I’d do something dialog based to make it look more fancy. There’s already jeos-firstboot that does something similar already and more. But then the WSL image doesn’t have to be really minimal, which means we have YaST!

So even though WSL doesn’t really boot as it has no systemd it would be still possible to run the YaST firstboot wizard on first start. What modules it launches is configurable via xml file. So leaving out hardware/VM specific things like network configuration it works pretty well:

For the launcher to know the name of the created user a small YaST module was needed to write the name into /run/wsl_firstboot_uid. The launcher fetches it from there.

Using the YaST firstboot wizard also allows to use e.g. the existing registration dialogs on SLE or add other useful configuration steps. One feature I have in mind would be for example is the role selection screen to offer some pre defined package selections for WSL use cases.

Tumbleweed and Leap appx files to test this are available from download.opensuse.org. Keep in mind that one needs to import the certificates used by OBS for signing first.

As usual, we have been working in a wide range of topics which include:

• Improving support for multi-device file systems in the expert partitioner.
• Fixing networking, secure boot and kdump problems in AutoYaST.
• Stop waiting for chrony during initial boot when it does not make sense.
• Preparing to support the split of configuration files between /usr/etc and /etc.
• Using /etc/sysctl.d to write YaST related settings instead of the /etc/sysctl.conf main file.

Expert Partitioner and Multi-Device File Systems

So far, the Expert Partitioner was assuming that Btrfs was the only file system built on top multiple devices. But that is not completely true because some file systems are able to use an external journal to accomplish a better performance. For example, Ext3/4 and XFS can be configured to use separate devices for data and the journaling stuff.

We received a bug report caused by this misunderstanding about multi-device file systems. The Expert Partitioner was labeling as “Part of Btrfs” a device used as an external journal of an Ext4 file system. So we have improved this during the last sprint, and now external journal devices are correctly indicated in the Type column of the Expert Partitioner, as shown in the screenshot below.

Moreover, the file system information now indicates which device is being used for the external journal.

And finally, we have also limited the usage of such devices belonging to a multi-device Btrfs. Now, you will get an error message if you try to edit one of those devices. In the future, we will extend this feature to make possible to modify file systems using an external journal from the Expert Partitioner.

AutoYaST Getting Some Love

During this sprint, we have given AutoYaST some attention in different areas: networking, bootloader and kdump.

About the networking area, we have finished s390 support in the new network layer, fixing some old limitations in devices activation and udev handling. Apart from that, we have fixed several bugs and improved the documentation a lot, as we found it to be rather incomplete.

Another important change was adding support to disable secure boot for UEFI through AutoYaST. Of course, we updated the documentation too and, during the process, we added some elements that were missing and removed others that are not needed anymore.

Finally, we fixed a tricky problem when trying to get kdump to work on a minimal system. After some debugging, we found out that AutoYaST adds too late kdump to the list of packages to install. This issue has been fixed and now it should work like a charm.

As you may have seen, apart from writing code, we try to contribute to the documentation so our users have a good source of information. If you are curious, apart from the documents for released SUSE and openSUSE versions, you can check the latest builds (including the AutoYaST handbook). Kudos to our documentation team for such an awesome work!

Avoiding chrony-wait time out

Recently, some openSUSE users reported a really annoying issue in Tumbleweed. When time synchronization is enabled through YaST, the system might get stuck during the booting process if no network connection is available.

The problem is that, apart from the chrony service, YaST was enabling the chrony-wait service too. This service is used to ensure that the time is properly set before continuing with other services that can be affected by a time shift. But without a network connection, chrony-wait will wait for around 10 minutes. Unacceptable.

The discussion about the proper fix for this bug is still open, but for the time being, we have applied a workaround in YaST to enable chrony-wait only for those products that require precise time, like openSUSE Kubic. In the rest of cases, systems will boot faster even without network, although some service might be affected by a time shift.

Splitting Configuration Files between /etc and /usr/etc

As Linux users, we are all used to check for system-wide settings under /etc, which contains a mix of vendor and host-specific configuration values. This approach has worked rather well in the past, not without some hiccups, but when things like transactional updates come into play, the situation gets messy.

In order to solve those problems, the plan is to split configuration files between /etc and /usr/etc. The former would contain vendor settings while the latter would define host-specific values. Of course, such a move have a lot of implications.

So during this sprint we tried to identify potential problems for YaST and to propose solutions to tackle them in the future. If you are interested in the technical details, you can read our conclusions.

Writing Sysctl Changes to /etc/sysctl.d

In order to be able to cope with the /etc and /usr/etc split, YaST needs to stop writing to files like /etc/sysctl.conf and use an specific file under .d directories (like /etc/sysctl.d).

So as part of the aforementioned research, we adapted several modules (yast2-network, yast2-tune, yast2-security and yast2-vpn) to behave this way regarding /etc/sysctl.conf. From now on, YaST specific settings will be written to /etc/sysctl.d/30-yast.conf instead of /etc/sysctl.conf. Moreover, if YaST founds any of those settings in the general .conf file, it will move them to the new place.

What’s next?

Sprint 87 is already running. Apart from fixing some bugs that were introduced during the network refactoring, we plan to work on other storage-related stuff like resizing support for LUKS2 or some small snapper problems. We will give your more details in our next sprint report.

Stay tunned!

So it is time for another sprint report. During the last two weeks, we have been basically busy squashing bugs and trying to get the network module as feature-complete as possible. But, after all, we have had also some time to improve our infrastructure and organize for the future.

YaST2 Network Refactoring Status

Although we have been working hard, we have not said a word about the yast2-network refactoring progress since the end of July, when we merged part of the changes into yast2-network 4.2.9 and pushed it to Tumbleweed. That version included quite a lot of internal changes related to the user interface and a few bits of the new data model, especially regarding routing and DNS handling.

However, things have changed a lot since then, so we would like you to give you an overview of the current situation. Probably, the most remarkable achievement is that the development version is able to read and write the configuration using the new data model. OK, it is not perfect and does not cover all the use cases, but we are heading in the right direction.

In the screencast below you can see it in action, reading and writing the configuration of an interface. The demo includes handling aliases too, which is done way better than the currently released versions.

Moreover, we had brought back support for many types of devices (VLAN, InfiniBand, qeth, TAP, TUN, etc.), improved the WiFi set-up workflow and reimplemented the support for renaming devices.

Now, during the current sprint, we are focused on taking this new implementation to a usable state so we can release the current work as soon as possible and get some feedback from you.

Finally, if you like numbers, we can give you a few. Since our last update, we have merged 34 pull requests and have increased the unit test coverage from 44% in openSUSE Leap 15.0/SUSE Linux Enterprise SP1 to around 64%. The new version is composed of 31.702 (physical) lines of code scattered through 231 files (around 137 lines per file) vs 22.542 in 70 files of the old one (more than 300 lines per file). And these numbers will get better as we continue to replace the old code

Missing Packages in Leap

It turned out that some YaST packages were not updated in Leap 15.1. The problem is that, normally, the YaST packages are submitted to the SLE15 product and they are automatically mirrored to the Leap 15 distribution via the build service bots. So we do not need to specially handle the package updates for Leap.

However, there are few packages which are not included in the SUSE Linux Enteprise product line, but are included in openSUSE Leap. Obviously these packages cannot be updated automatically from SUSE Linux Enterprise because they are not present there. In this case Leap contained the old package versions from the initial 15.0 release.

In order to fix this issue, we manually submitted the latest packages to the Leap 15.2 distribution. To avoid this problem in the future we asked the Leap maintainers to add the Leap specific packages to a check list so they are verified before the next release. Of course, if you see any outdated YaST package in Leap you can still open a bug report.

Just for reference, the affected packages are: yast2-alternatives, yast2-slp-server, yast2-docker and skelcd-control-openSUSE (the content is only present on the installation medium, it’s not released as an RPM).

Let’s use all disks!

As you may remember, three sprints ago we added some extra configuration options to make the storage guided proposal able to deal with the SUSE Manager approach. We even wrote a dedicated blog post about it!

Despite offering the new options in the Guided Setup, we tried to keep the default initial behavior of the installer consistent with other (open)SUSE products. So the installer initially tried to install the whole system in a single disk, unless that was impossible or it was told by the user to expand on several disks.

But the SUSE Manager folks found that to be contrary to the new ideas introduced in their Guided Setup. According to their feedback, in this case remaining consistent with other (open)SUSE product was not reducing the confusion, but rather increasing it. SUSE Manager should try from the very beginning to expand the product as much as possible among all available disks.

For that reason, during this sprint we introduced the first improvement (a.k.a. another configuration option), so now it is possible to tell whether the initial proposal should try to use multiple disks as first try.

Bootloader and Small MBR Gaps

We received a bug report because a system was not able to boot after installation. In this case, the user decided to use Btrfs and placed the root file system in a logical partition. In theory, this scenario should work but, unfortunately, the MBR gap was too small to embed the Grub2 bootloader code.

At first sight, this problem could be solved by asking YaST to install the bootloader into the logical partition and the generic boot code in the MBR. But this will only work if you set the logical partition as the active one. Sadly, some BIOSes could insist on having a primary partition as the active one.

But don’t worry, we have good news. Grub2 maintainers took care of this problem. In case the MBR gap is too small, Grub2 will automatically fall-back to the Btrfs partition. That’s all. And what does it mean for YaST? Well, thanks to this fix, YaST will simply work out of the box and your system will be bootable again. But not so fast! You still have to wait a little bit more to have these Grub2 improvements available in a Tumbleweed installer.

Handling Empty Comment Lines in NTP Configuration

AutoYaST supports defining an specific NTP configuration to be applied during the installation and it relies in Augeas to read/write the ntp.conf file. But it seems that Augeas has some problems when it tries to write comments with empty lines, as you can see in bug 1142026. The solution was to adapt YaST to filter out empty comment lines before saving the configuration file, working around the Augeas problem.

Error Resizing Some Partitions

Typically, an MS-DOS partition table reserves its first MiB for the MBR gap, so the partitions normally start after that point. But it is possible, especially in partitions for old Windows systems, that it starts before that first MiB. In that case, if we try to resize that partition (e.g., by using the Expert Partitioner), YaST crashes due to an error when calculating the resize information. Fortunately, this problem is gone now, and you will be able to resize this kind of partitions as well.

Side Effects of Keyboard Layouts Unification

During the sprint 81, the openSUSE and SUSE Linux Enterprise console keyboard layouts were unified after some minor changes. One of those changes was to stop using the, in appearance, useless keymaps symlinks for Arabic and Cambodian. But they were there for a reason: are being used by YaST to correctly adapt the keyboard in the X11 environment. Just visit the pull request if you prefer to scare yourself want to dive in more technical details.

Fortunately for the users of those keyboards, we realized about this problem before the upcoming SLE-15-SP2 was released And, it’s fixed.

House Keeping Tasks

As part of our development duties for this sprint, we invested quite some time in reviewing and updating our continuous integration (CI) set up. Apart from using Travis CI for pull requests, we rely on Jenkins to run the tests and submit the code to the appropriate projects in the Open Build Service instances.

Then, when the development of a new version starts or when the product is about to be released, we need to adjust the configuration. Just in case you are wondering, we do not do this work by hand anymore and we use Salt and Jenkins Job Builder to handle this configuration.

Closing Thoughts

During the next sprint (actually, the current one) we are working in three different areas, apart from squashing bugs: improving encryption support in the storage layer, adding some features to the installer (repo-less installer, support for reading product licenses from a tarball, etc.) and, of course, refactoring the network code. Obviously, we will give you all sort of details in our next sprint report.

Although there is much more work behind this sprint, we will have a look at these changes in this report:

• The first bits of the YaST Network refactoring have been submitted to factory.
• AutoYaST gets support for specifying NFS shares in a sane way.
• The Partitioner’s tree behaviour has been improved.
• Libyui has been updated to support Qt 5.13 (do not miss the screenshots in this report).
• The installer does not offer enabling autologin for text-only system roles.

But that’s not all. We have some pretty exciting news regarding the LibYUI Testing Framework which, hopefully, will reduce the hassle of maintaining YaST openQA tests in the future. If you are interested, we published a separate post a few days ago with all the details about this topic.

Submitting First Bits of the YaST Network Refactoring to Factory

yast2-network 4.2.2 is the first version which includes some code from the ongoing refactoring effort. As we announced in our last report, we have just started and there is a long road ahead of us, but you can see some results already in the routes management area.

Additionally to cleaning up and improving the codebase quality, we are fixing some bugs and introducing some minor enhancements along the process. For instance, version 4.2.2 permits to have multiple default routes and does not drop extra options for them.

If you want to know more details about the process, you might be interested in the kickstart meeting notes that we added to our repository a few days ago.

Auto-installation onto NFS Shares: the Sane Way

During the previous sprint, AutoYaST was powered again with the capability of installing onto an NFS share. That feature was available in SUSE Linux Enterprise 12 (and openSUSE 42.x) but, for several reasons, we did overlook it when the new storage stack was re-implemented for (open)SUSE 15.

The problem with this feature in older versions, apart of not being documented at all, is that it requires to use some hacks and a non-validating AutoYaST profile. For that reason, we have redesigned how NFS drives are described in the AutoYaST profile. With the new format, we have a drive section per each NFS share, for example,

<partitioning config:type="list">
  <drive>
    <device>192.168.1.1:/exports/root_fs</device>
    <type config:type="symbol">CT_NFS</type>
    <use>all</use>
    <partitions config:type="list">
      <partition>
        <mount>/</mount>
        <fstopt>nolock</fstopt>
      </partition>
    </partitions>
  </drive>
</partitioning>

Although the old format is still supported to keep backward compatibility, we encourage you to use the new one for now on. And of course, this feature will be appropriately documented in the official (open)SUSE 15 documentation.

No Longer Offering Autologin for Text-Only System Roles

Autologin is a feature that most modern display managers (KDM, GDM, SDDM, LightDM) offer as a convenience function for home users. Users reported that YaST also provided this for the “Server (text only)” system role where that does not make sense, so we fixed that.

Yet Another Qt Version

This is not actually a big deal; there is another version of the Qt libs which we use for the graphical version of YaST. Now it’s Qt 5.13, and we needed to adapt some functions that have become obsolete in the meantime. Fortunately, Qt always provides drop-in replacements for things they obsolete, so this was mostly a pretty mechanical task to go through the warnings about functions or classes that are now obsolete and replace them with the new counterpart.

But how long have we been doing that? Quite a while, actually; it all started with Qt 2.x in late 1999 for SuSE Linux 6.3. That was the first YaST2. It already came in both a Qt version with a graphical user interface and in a text-only (NCurses) version.

Back then we were more advanced than the KDE of that time: KDE 1.x was still using Qt 1.x, and the first version of YaST2 already used Qt 2.x. We kept upgrading throughout all the Qt 2.x versions and in SuSE Linux 9.0 we jumped to Qt3, which looked like this.

And as you know, that’s not the end of the story. We have kept updating through all the Qt 3.x, Qt 4.x and now Qt 5.x versions, as you can see in the following screenshot of the upcoming openSUSE Leap 15.1. It’s been a while: almost 20 years of happy Qt usage, always up-to-date.

Improving the Partitioner UX

When it comes to UX, the tree of our new and shiny partitioner needed some love. We fixed an annoying issue which caused all the branches to be expanded after every change on any device. Besides that, we also changed the initial view. See the screenshot below in which sections -levels with icons like “Hard Disks” or “RAID”- are initially expanded and items representing devices are collapsed, not displaying, for example, the full list of partitions until the user decides to expand those.

However, the bad news is that these changes will not be available in (open)SUSE 15.1, although you can give it a try as soon as they land in Tumbleweed.

Let’s Talk about Multi-device Btrfs

The improved behavior of the navigation tree is not the only enhancement in the Partitioner we are preparing for the future (where “future” means Tumbleweed and the 15.2 releases of both SLE and openSUSE Leap). We also want to make possible to use the YaST Partitioner to define Btrfs file-systems that spread over several disks and partitions. That’s a pretty unique feature of Btrfs that combines at file-system level certain characteristics usually offered by software RAID and LVM.

But that also implies a quite different way of organizing the storage devices, which is a challenge to represent in the already complex user interface of the Expert Partitioner. So, first of all, we have put together this document describing all the struggles of representing advanced Btrfs features in the Partitioner, together with some possible solutions for the short and mid terms.

Please, feel free to provide feedback about the proposed solutions and to offer new ones. Help us to shape the future of the YaST interface!

Conclusions

As we mentioned at the beginning of this report, we owe you a blog post about the Libyui Testing Framework status. But as you wait for it, you might want to register for the openSUSE Conference or check the schedule.

Welcome back to our linguistic blog disguised as a YaST team sprint report.

• Installation and Upgrade: an online offline upgrade; selecting add-ons easily; better selection of license translations.
• Replaced components: even more traces of xinetd removed; AutoYaST export for firewalld.
• Storage and Partitioner: format options dialog; empty views removed; installation summary; blocker errors; autoinstall by-partlabel; more precise booting setup; mounted /home or haunted /home?

Installation and Upgrade

Offline Upgrade Using Bootable Media via SCC

The last few weeks we spent quite some time implementing the offline migration from the old SUSE Linux Enterprise products (versions 11 and 12) to the upcoming version 15.

Note: the offline migration term actually means that your production system is not booted and running, it is not about the network status. At offline migration a different system is booted (usually from a DVD). See more details in the official SUSE documentation.

We implemented and tested the upgrade from SLE 12 SP3. The offline migration workflow is similar to the online migration as implemented in SLE 12 release. The only difference is that you boot the SLE 15 medium, select Upgrade in the boot menu and then select the disk partition to upgrade. The rest should be the same as in the online migration.

The migration from SLE 11 is a bit more complex as it is registered against the older Novell Customer Center and requires some additional changes in the installer. This is work in progress.

And the last note: for testing the offline migration to SLE 15, your system needs to be registered using the Beta registration keys. For regular SLE 12 and 11 registrations, the migration to SLE 15 is blocked. It will be unblocked after the SLE15 has been officially released.

Made the addon Linuxrc Option Work Well with the Packages DVD

In SLE 15 there are numerous modules and extensions, such as the Live Patching module, or the Web Scripting module. On physical DVDs, we are putting all of them on a single Packages DVD.

When you are installing SLES and choose to add such a module during the installation from the DVD, you will be presented with a screen to select from all the modules found on the DVD.

You can also automate this step by passing the addon=dvd:/// option at the installation boot prompt. (See the Linuxrc reference). Formerly this worked only with single-product media. Starting now, the addon option will work also with multi-product media such as the Packages DVD.

Improve Licenses Translations Support

Some months ago, YaST started to use libzypp to get product licenses, instead of using the old SUSE Tags approach. However, until this sprint, this feature was somehow incomplete.

On one hand, the complete list of supported languages was shown, no matter whether a translation was available for a given language or not. On the other hand, the "Licenses Translation" button was missing (it is still used in single product media).

Now both problems are solved and, as soon as new translations are included in the installation media, they should be handled gracefully in the installer.

Replaced Components

Finalize Xinetd Conversion to Systemd Sockets

This sprint we finished our change from xinetd to systemd sockets for starting services on demand. To finalize it there is basically two main tasks.

The first one was dropping the YaST module for xinetd. That required a conversion of yast2-ftp-server that used this module and also adding a note to AutoYaST that xinetd configuration is no longer supported, so if you have it in your AutoYaST profile, it won’t be applied. The FTP server part was harder because, as mentioned in the last report, one of the two backends does not support systemd sockets, but we found that this backend is a bit ancient and support for us was quite painful. The result is that we dropped pure-ftp and kept only vsftpd backend, which makes the code much simplier and our life better (the final diff-stat is +1100/-3700). And then we converted the usage to systemd sockets. Then we could proceed to dropping yast2-inetd because there was no dependency anymore.

The second task was xinetd usage directly, with an API for starting on demand. It is not used too often and in the end the biggest parts were dropping xinetd usage for VNC based installation and yast2-inst-server which is now converted to use systemd sockets. And here we can give you a nice trick we discovered during the implementation: If a systemd service has a parameter (often the case with services started by a systemd socket) you can stop all of them with a wildcard, e.g. systemctl stop vnc@*.

So here is a happy end – after this sprint there is no xinetd usage and we can support only one tool for starting on demand, allowing us to focus on improving other parts.

Added support for exporting firewalld AutoYaST configuration

In the previous blog entry we announced AutoYaST support for configuring firewalld but cloning the firewall configuration was not supported yet and also the AutoYaST summary concerning the Firewall module needed some love and that is basically what we have implemented during this sprint.

It should be noted that editing of the AutoYaST Firewall configuration is not allowed since the firewall configuration is now done with the firewalld graphical configuration tool (firewall-config)

Storage and Partitioner

Added the Format Options Dialog to the Partitioner

One of the missing things in the new partitioner was the format options dialog letting you tune the file system a bit when it is created.

The options itself are more or less the same as in the old partitioner. This feature is intended more for the experts. As an average user you will rarely find a need to fine-tune file system parameters. But in case you do, the dialog is there to aid you (remember the help button).

Here is a screenshot of the ext4 options:

Removed the Empty Views in the Partitioner

In the process of rewriting the YaST storage stack, we also rewrote the partitioner. Some views in the partitioner were taken over from the old one, but not filled with any functionality so far – they only showed empty pages. We now removed those empty pages:

• Crypt Files
• Device Mapper
• Unused Devices
• Mount Graph
• Settings (this one will be back soon with content)

This is what it looks like now:

Compared to the previous version with the empty views:

See also Bug #1078849.

Installation Summary in the Partitioner

One of the sections that survived the Partitioner sifting mentioned above was the Installation Summary. During this sprint we re-implemented this useful view that shows the changes that would be performed in the system, including packages to install in order for the system to work with the chosen technologies. One image is worth a thousand words.

Of course, the information in both lists is updated with every change done in the Partitioner and, as usual, everything works as a charm also in text mode. Including the possibility of collapsing or extending the (usually lengthy) list of operations on subvolumes.

Blocker Errors in Partitioner

A few sprints ago we implemented warnings in the partitioner that inform you if something looks like probably not working, but with expert knowledge it can be made working. Now we add also blocking errors where we are sure it won’t work. It is just a first draft so it will be adapted as needed and as problems appear. Some checks are already moved from the bootloader to the partitioner, so you can fix the partitioning quickly. But enough words, check out the screenshots, where the first one is an error which prevents continuing and the second one is just a warning.

Extending the AutoYaST <device> Element

When defining a <drive> section in an AutoYaST profile, the <device> element should determine to which disk you want to apply that partitioning schema. Usually, it is a device kernel name, like /dev/sda, or a link which resolves to a disk (for instance, /dev/disk/by-id/*, /dev/disk/by-path, etc.).

However, AutoYaST supports specifying other names, like by-partlabel, by-label, etc. Those links won’t resolve directly to a disk, but the storage layer will be able to find out which disk they belong to.

Although SLE 12 supported this behaviour, it was missing from SLE 15 until this sprint.

Fine-tuning of the Requirements to Boot a System

In some situations, an extra partition or a given disk layout is needed to boot an (open)SUSE operating system. Like the separate /boot partition needed in some corner-case legacy scenarios, the ESP partition that must be mounted at /boot/efi in EFI systems or the PReP partition needed by some PowerPC systems. The partitioning proposal performed by the installer tries to ensure those booting requirements are met (so does AutoYaST in some cases) and our beloved Partitioner also includes some checks to warn you user if you forget to create or mount any of those partitions.

But in some situations, the installer was suggesting partitions with a suboptimal size or even partitions that were not strictly needed. On the other hand, the Partitioner was sometimes being too picky, warning about situations that were not such a big problem. So during this sprint we refined our list of booting requirements, updating the corresponding documentation, relaxing the partitioner checks and fine-tuning the proposal outcome. Specifically the PowerPC requirements were revamped based on the input from several experts and bug reports from manufacturers of some systems.

So no matter if you usually trust the installer proposal, if you like to handcraft stuff with the Partitioner or if you install using AutoYaST, the experience should be more smooth now, which fewer (if any) ugly surprises at boot time.

Mount Options Revisited: When the Old Demons Come back to Haunt You

Recently, we reintroduced per-filesystem-type defaults for mount options. We thought this would be a great chance to clean up old code that had become messy over time; we thought we could provide a clean, well-structured and easy-to-understand way to do this.

Then people began to test some more scenarios, and we found out the hard way that in some situations, those mount options depend on the mount point for various reasons: Some quirks of underlying kernel modules like the VFAT filesystem or the way systemd handles mounting the root filesystem during booting made this necessary.

So we had to bite the bullet and reintroduce some of that old code which was kind of messy; for example, for ext4 or ext3 root filesystems, we no longer specify any data=... mount options because this might make remounting the root filesystem read/write at boot time fail; in another case, a mkdir -p /boot/efi/EFI (when installing the boot loader) on a VFAT partition failed despite VFAT technically being case-insensitive (omitting iocharset=utf8 fixed this).

Lesson learned: Some messy old code is messy for a reason. Trying to streamline it may break some scenarios.

Conclusion

The answers to the initial questions are

1. Yes.
2. 说不定.
3. Most of the time.

As the SLE release cycle is shifting from the "all features are mandatory" phase to the "all bugs are top priority" phase, expect less of feature news and more bugfix news in the next report, due in two weeks.

Chrony support in AutoYaST

As part of our effort to support Chrony as the default NTP service for (open)SUSE, we have revamped how AutoYaST handles the configuration of such a service. The first noticeable change is that we have redesigned the schema which, instead of containing low level configuration options, is now composed of a set of high level ones that are applied on top of the default settings.

And here is how the new (and nicer) configuration looks like:

<ntp-client>
  <ntp_policy>auto</ntp_policy>
  <ntp_servers config:type="list">
    <ntp_server>
      <iburst config:type="boolean">false</iburst>
      <address>cz.pool.ntp.org</address>
      <offline config:type="boolean">true</offline>
    </ntp_server>
  </ntp_servers>
  <ntp_sync>15</ntp_sync>
</ntp-client>

Updating the Remote Administration Capabilities

During this sprint, the remote administration client has been deeply modified. To begin with, as xinetd is being replaced by systemd sockets, we have dropped that dependency (adjusting the code accordingly).

Additionally the VNC handling have been improved too. Until now, YaST offered the possibility to connect through a web browser using a Java applet. Now YaST allows the user to enable/disable this feature (check the screenshot below to see how it looks now). It is worth to mention that Michal Srb has replaced the old viewer with novnc, a JavaScript based one. Thanks a lot for that, Michal!

And last but not least, we have seized the occasion to do some code cleaning, reimplementing some dialogs using the Common Widget Manipulation object oriented API.

Modifying AutoYaST Profile During Installation

AutoYaST offers a cool feature that allows the profile to be modified during the initial stages of the installation using an user script. So you can run a script which adjusts the profile and AutoYaST will read it again. If you are interested in such a feature, you could find more information in the official documentation.

On the other hand, in our previous report, we mentioned that AutoYaST was able again to use multipath devices using the new storage stack. But we didn’t count that it was possible to modify the profile on runtime so the initialization happened too early.

Now the bug is fixed so you can again adjust any storage setting using the aforementioned feature.

Properly Handling Selected Modules

As you may know, some time ago we added a support for the multi-product media (DVDs which contain more than one repository/product in separate subdirectories). This time we fixed some issues regarding this functionality.

Originally after selecting several products only one of them was actually selected to install and only one product was displayed in the installation proposal. Fortunately, those issues have been addressed now.

Unified Look & Feel for Multi-Product Selection Dialog

For the multi-product DVD media we used this selection dialog:

The functionality is very similar to the on-line product selection dialog displayed after registering the system:

As you can see the look & feel is quite different, but from the usability point of view the dialogs should look the same regardless whether the products are added from a DVD medium or from an on-line source (a registration server).

This sprint we improved the DVD media dialog to better match the registration dialog:

.

The dialog is still not exactly the same, but now it looks more similar so users should feel more familiar with it. There is also displayed an additional note about not handling the product dependencies automatically. This note was already present in the help text but that is hidden by default. As we got quite a lot of bug reports about this issue we decided to make this fact more prominent.

(Note: The dialogs actually cannot look the very same as the DVD media currently lack some information like the dependencies, beta status, detailed descriptions…).

Dropping SYSTEMCTL_OPTIONS Variable Support

We have been using the environment variable called SYSTEMCTL_OPTIONS (which is SUSE-specific) in our systemd services to prevent locks in dependencies. As this hack will not be necessary for upcoming the (open)SUSE 15 version, it will be dropped from systemd and, therefore, we already removed it from our systemd services.

Unifying Disklabel Handling in AutoYaST

When specifying an AutoYaST partitioning schema, you can select which partition type you want to use for each device (MSDOS, GPT, DASD, etc.). In the past, AutoYaST implemented its own logic to select which one to use in case that it was not specified by the user. After this sprint, AutoYaST relies on the new storage stack in order to decide which option fits better when the user does not specify one.

Bonus: Automatically Checking the Defined Systemd States

Some time ago we had serious issues with service management in YaST (see bug#1012047 and bug#1017166). The problem was caused by introducing a new systemd service state which was not expected by YaST. We fixed the problem by correctly handling the new state.

But the main problem was that we (as the YaST developers) were not notified about this major system change and we found this change later after we got bug reports in Bugzilla. To avoid this problem again in the future we decided to implement a script which would regularly check the defined systemd states notifying us if a unknown state was detected.

To implement the regular check we use the Travis cron job feature which allows running continuous integration builds not only after a change is pushed into the repository but also in regular intervals, even when there is no change in the code.

Alternatively you can use any CI service, but we chose Travis because it is easy to use and we already use Docker for normal CI jobs which allows us to run the latest systemd from openSUSE Tumbleweed in an easy way.

In this case we could possibly run the script in OBS when building the yast2-services-manager package, but that would need adding systemd to BuildRequires which does not sound as a good idea…

So if you also need to run some check scripts regularly you can see more details in this pull request.

Conclusion

2017 was an exciting year with a lot of interesting stuff: a new storage layer, multi-product installation medium, integration of new components (firewalld, chrony, etc.). And it looks like 2018 will not be different and we will have a lot of fun.

Thanks for your support and happy new year!

• You’re still carrying around your precious autoyast config files on an unencrypted usb stick?
• You have a customized installation disk that could reveal lots of personal details?
• You use ad blockers, private browser tabs, or even tor but still carry around your install or rescue disk unencrypted for everyone to see?
• You have your personal files and an openSUSE installation tree on the same partition just because you are lazy and can’t be bothered to tidy things up?
• A simple Linux install stick is just not geekish enough for you?

Not any longer!

mksusecd can now (well, once this pull request has been merged) create fully encrypted installation media (both UEFI and legacy BIOS bootable).

Everything (but the plain grub) is on a LUKS-encrypted partition. If you’re creating a customized boot image and add sensitive data via --boot or add an add-on repo or autoyast config or some secret driver update – this is all safe now!

You can get the latest mksusecd-1.54 already here to try it out! (Or visit software.opensuse.org and look for (at least) version 1.54 under ‘Show other versions’.

It’s as easy as

mksusecd --create crypto.img --crypto --password=xxx some_tumbleweed.iso

And then dd the image to your usb stick.

But if your Tumbleweed or SLE/Leap 15 install media are a bit old (well, as of now they are) check the ‘Crypto notes’ in mksusecd --help first! – You will need to add two extra options.

This is how the first screen looks then

Hostname configuration during installation

And let’s start with one of those stories that illustrate the complexity hidden below the user-oriented YaST surface. During installation is very common to assign a hostname to the machine being installed to identify it clearly and unequivocally in the future.

Usually it is a fixed hostname (stored in /etc/hostname) but in some circumstances is preferable to set it dynamically by DHCP. Since some time ago (as you read in a previous report and is shown in the image below) YaST allows to set the hostname selecting a concrete interface or with a system-wide variable named DHCLIENT_SET_HOSTNAME which is defined in /etc/sysconfig/network/dhcp. The value to be set for such variable during installation can be optionally read from the distribution control file. Last but not least, as you already know, Linuxrc can also be used to enforce a particular network configuration.

Most users have a simple setup that works flawlessly, but we recently got a bug report about a wrong network configuration after installing the system if the hostname configuration was set via Linuxrc. After some research we found that the value of DHCLIENT_SET_HOSTNAME coming from the control file was overwriting the Linuxrc configuration at the end of the installation. Now it’s fixed and the global variable will be set by the linuxrc sethostname option if provided or loaded from the control file if not. And all that happens now at the very beginning of the installation to give the user to chance to modify it and to ensure the user’s choice is respected at the end.

Take into account that with multiple DHCP interfaces the resulting value for DHCLIENT_SET_HOSTNAME is not fully predictable. Hence, in that scenario we recommend to explicitly select the interface which is expected/allowed to modify the hostname.

Extending the installation process via RPM packages

As we have mentioned (a couple of times) during latest reports, we are implementing multi-product support for the installer. It means that SUSE will ship several products on a single installation media.

One interesting feature is that products, modules and extensions can define its own installation roles. For instance, if you select the desktop extension, you will be able to select GNOME as system role.

During this sprint, we have improved roles definitions handling, displaying a different list of roles depending on which product was selected.

As a side effect, we added support for sorting roles assigning them a display order.

Getting Release Notes from the Installation Repository

As part of our effort to drop SUSE tags from the installation media, we improved the way in which release notes are handled during installation.

Release notes are downloaded from openSUSE or SUSE websites in order to show always the latest version. Of course, the installation media includes a copy of them, which may be outdated, to be used when there is no network connection.

From now on, instead using some additional files, this offline copy of release notes will be retrieved from the release-notes package which lives in the packages repository. So we do not need to ship additional files containing release notes in the installation media anymore.

Moreover, although the old approach worked just fine in almost all cases, there was an uncovered scenario. Let’s consider a system which have access to an updated packages repository but is not connected to Internet. That could be the case, for instance, if you are using SUSE Subscription Management Tool (SMT). With the new approach, the installer will get release notes from that repository instead of displaying the (potentially outdated) ones included in the installation media.

Additionally, we refactored and clean-up a lot of old code, improving also test coverage.

Storage reimplementation: bringing more features back

We are also working hard to make sure the brand new yast2-storage-ng includes all the features from yast2-storage, in addition to the new ones. That means that, after this 44th sprint, SLE15 is already able to perform the following operations using the new module.

• Creating MD software RAID devices in the expert partitioner. This feature is specially relevant for many openQA tests that rely on it.
• Displaying the compact description of the partitioning proposal in the one-click-installer screen used by SUSE CaaSP and openSUSE Kubic
• Importing users and SSH system keys from a previous (open)SUSE installation.

Rethinking LVM thin provisioning

When trying to create a thin-pool using all free space the metadata has to be accounted for. In contrast to linear LVs the metadata for thin-pool uses space of the VG. For instance, if there are 2048 GiB free in the VG, the metadata for a maximal size thin-pool is about 128 MiB and the pool can be about 2047.9 GiB big.

Additionally LVM creates a spare metadata with the same size. This spare metadata is shared between all pools and thus has the size of the biggest pool metadata. The spare metadata can be deleted manually and all pool metadata can also be resized.

When starting with an empty VG it is relative easy to account for the metadata. But how to handle this with an already existing volume group? Also take into account a volume group containing e.g. RAID LVs or cache pools (which also have metadata).

We finally decided that, during probing, YaST will check how much free space the VG has and then it will calculate “reserved” value for the volume group:

reserved = total size - used by LVs the library handles - free

So when calculating available space for a normal or thin pool, it will take the “reserved” into account:

max size = total size - reserved - used by LVs the library handles

The only drawback is that the maximal size for the pool can be smaller than actually possible since e.g. the spare metadata might be shared with an already existing thin pool.

More to come

The 45th sprint has already started and you can expect more and more work in the installer for SLE15 and openSUSE Leap 15 and more news regarding the revamped storage stack. Meanwhile, don’t forget to have a lot of fun!

Anyway, long story short spending some face time with other contributors and working out the kinks that existed in the pipeline worked wonders.  Rather than sending a small patch here and there the main implementation for openSUSE and SLE, lots of code, were accepted shortly after the cloud-init summit and over the last couple of days another couple of patches took us another step forward.

There are a few more loose ends that need work but with 17 patches removed from the package build, currently building in Cloud:Tools:Next in OBS we’ve made major progress.

Well, I for one am happy about this, and those that want to install from source can do so and have openSUSE and SLE support working from the upstream sources and not just from the packages included with openSUSE and SLE.

Thanks to Canonical for organizing the summit to get everyone together and thanks to Google for hosting the summit.

Oh and before I forget, getting the changes accepted was not the only major step forward, openSUSE Leap 42.3 will, in the not too distant future, like in the next couple of days, be integrated into cloud-init testing using containers the lxd project builds, go figure who knew these even existed.