Home Home > Distribution > Factory
Sign up | Login

Archive for the ‘Factory’ Category

Highlights of YaST Development Sprint 91

December 18th, 2019 by

The last two weeks of the year, and also the first one of the new year, are vacation season in many parts of the world and YaSTland is not an exception. But before we enter hibernation mode, let’s take a look to the most important features and bugfixes we implemented in the last sprint of 2019. That includes:

  • bringing back to life some sections of the Software Manager,
  • implementing system upgrade with the new SLE media types,
  • making the installation in Raspberry Pi and IBM Z System even better,
  • improving usability of encryption,
  • reducing the footprint of the Snapper plugin for ZYpp,
  • as always, many other small improvements and fixes.

Restored Some Package Views: Recommended, Suggested, etc.

Let’s start with a redemption story. Some time ago we implemented feature fate#326485 which requested dropping the “Package Groups” view from the package manager UI. That was quite an easy task.

However, a few weeks later we got a bug report that the lists of recommended, suggested, etc… packages couldn’t be displayed anymore. It turned out that, in the Qt package manager front-end, the removed “Package Groups” view not only used to display the static group data from the packages but it also contained some special computed package lists like orphaned, suggested or recommended packages. So these lists were lost as a collateral damage of removing the “Package Groups” view.

The ncurses package manager was not affected by the same problem because, in that front-end, those views are grouped in a separate “Package Classification” section. So the task for this sprint was to somehow revive the lists in Qt and make them again available to the users.

We partly reverted the Package Groups removal and restored displaying those special package groups. To make it consistent we also use the “Package Classification” name for the view, like in the ncurses package manager.

The new Package Classification view in Qt

On the other hand, the ncurses front-end was missing some lists like the “Multiversion Packages” and “All Packages”. To take consistency another step further, we added these missing lists and did some small cleanup and fixes so now both the Qt and the ncurses package managers should offer the same functionality and should look similar.

The revamped Package Classification view in ncurses

User-friendly Encryption Device Names

And talking about bug reports that trigger some usability revamp, some users had pointed that, when the system is booting and prompts for the password of an encrypted device, it’s not always that easy to identify which exact device it is referring to:

Booting password prompt before the change

The root of the problem is that when YaST creates an encryption device (during the installation by means of the storage proposal, or manually with the Expert Partitioner), the device mapper name for the new encrypted device is generated from the udev id of the underlying device (e.g., cr_ccw-0XAF5E-part2).

We decided to improve the encryption naming generation in YaST for Tumbleweed and future releases of Leap and SLE. From now on, the name will be based on the mount point of the device. For example, if an encrypted device is going to be mounted at root, its device mapper name would be cr_root. In general, when the encrypted device is mounted, the device mapper name would be cr_mount_point (e.g., cr_home_linux for an encrypted device mounted at /home/linux).

Booting password prompt after the change

Note that udev-based names might still be used for some scenarios. For example, when the device is not mounted or for an indirectly used encrypted device (e.g., an encrypted LVM Physical Volume).

And related to the identification of encryption devices, we have also added more information about the device when the encryption devices are activated during the installation process. Providing the password for the correct device was very difficult because the user needed to know the UUID of the encryption device. Now on, the activation popup also informs about the kernel name of the underlying device, making much easier to identify it.

New password prompt during installation

Because names matter… which leads us to the next topic.

How does it Feel to Run a Mainframe?

As you may know, (open)SUSE runs in a vast range of hardware, including powerful mainframes like the IBM Z family. One of the strengths of our beloved distributions is that, despite the differences in hardware and scope, the installation and usage experience is very similar in all the supported systems.

Consistency and ease of use are good, but when you drive a luxury car you want to see the brand’s badge on top of the hood. So in future versions of the installer, the model of the machine will be displayed when installing in an IBM Z system. See the right-top corner of the following screenshot.

IBM Z Model in the installer

The text-based installer also has been modified to include the same banner in a similar place.

IBM Z Model in the text-mode installer

But in the same way that (open)SUSE enables you to install and use Linux in a mainframe “just like in any other computer”, we also target to do the same in the other extreme of the hardware spectrum.

Better Support for Raspberry Pi in the Partitioning Proposal

One year ago we announced that openSUSE Leap 15.1 and SLE-15-SP1 would be the first Linux distributions that could be installed in Raspberry Pi devices following the standard installation procedure, instead of deploying a Raspberry-specific pre-built image. The only prerequisite was the existence in the target SD card (or disk) of a partition containing the Raspberry Pi boot code.

But we are now able to go one step further for SLE-15-SP2 (and Leap 15.2). Thanks to the technologies included in those upcoming releases, (open)SUSE will not longer need a separate partition with the boot code in all cases. Now the installer can make a reasonable installation proposal in all situations, even if the target storage device doesn’t contain a booting partition in advance. See, for example, what the installer suggests by default for installing a fully standard SLE-15-SP2 Beta1 in a 32 GiB SD card that contained initially a GPT partition table (tip: GPT partition tables cannot be used to boot in a Raspberry Pi device… and the installer knows it).

Installer proposal for a Raspberry Pi

With that, the installation of the standard SLE-15-SP2 Beta1 (the aarch64 version, of course) in a Raspberry Pi 3 or 4 is as easy as “next”, “next”, “next”… with the only exception of a couple of packages that must be manually selected for installation (raspberrypi-* and u-boot-rpi3). Hopefully, future beta images of both SLE and openSUSE Leap 15.2 will select those packages automatically when installing in a Pi, which will make the (open)SUSE experience in those devices basically identical to any other computer.

SLE Upgrade with the New Media Types

And talking about the standard installation images of the upcoming SLE-15-SP2, we explained in our previous blog post that those versions of SUSE Linux Enterprise (SLE) and all its associated products will be distributed in two new kinds of media – Full and Online. The Full Media contains many repositories and the system can be installed without network connectivity. The Online Media is similar to the openSUSE’s net installer, it contains no repository and it must download everything from the network. The big difference with openSUSE is that SLE systems need to be registered in order to have access to remote repositories.

But apart from installation, those two new media types can also be used to upgrade an existing system… at least after all the improvements implemented during the latest sprint.

In the case of the Online Media, if the system is registered the upgrade process will switch all repositories to point to their corresponding versions at the SUSE Customer Center (SCC) and will get the new software from there. If the system is not registered, the upgrade process is cancelled and the user is advised to either register the system or use the Full Media.

The Full Media can be used to upgrade any system, registered or not, but the process is different in each case. For a non-registered system, the repositories will be switched to the ones included in the media and the system will be upgraded from there. For registered systems the process is the same that with the Online Media, so the software will be fetched from the remote repositories at the SUSE Customer Center.

Last but not least, we also made sure the process with both medias works with an AutoYaST upgrade (yes, you can also use AutoYaST to perform an unattended upgrade, in addition to the better known unattended installation). For a registered system, we simplified the procedure as much as possible and it only needs access to SCC and an empty AutoYaST profile. For non-registered systems it is a little bit more complex because the profile must specify which repositories from the media should be used for the upgrade. But other than that, the process works quite smooth.

And, of course, we used the opportunity to improve the unit test coverage of the code and to improve the documentation, including the profiles we used for testing.

The Snapper Plugin for ZYpp Becomes More Compact and Future-proof

Snapper lets you make filesystem snapshots. It has a companion, snapper-zypp-plugin, a plugin for ZYpp that makes snapshots automatically during commits. See the “zypp” descriptions in this listing:


# snapper list

  # | Type   |Pre # | Date                     | User | Used Space | Cleanup  | Description  | Userdata     
----+--------+------+--------------------------+------+------------+----------+--------------+-------------
  0 | single |      |                          | root |            |          | current      |              
[...]
824 | pre    |      | Tue Dec 17 10:00:27 2019 | root |  16.00 KiB | number   | zypp(zypper) | important=no
826 | post   |  824 | Tue Dec 17 10:02:19 2019 | root |  16.00 KiB | number   |              | important=no
827 | single |      | Tue Dec 17 11:00:01 2019 | root |  16.00 KiB | timeline | timeline     |             
828 | single |      | Tue Dec 17 11:00:01 2019 | root |  16.00 KiB | timeline | timeline     |             

To make our enterprise products supportable for a looong time, we have rewritten this plugin to C++, starting with snapper-0.8.7. (The original Python implementation is not dead, it is resting in old Git commits.)

As a result, Python regular expressions are no longer supported in the /etc/snapper/zypp.conf file. POSIX extended regular expressions work instead, which should work sufficiently well for the purpose of package name matching. Shell patterns continue working unchanged.

Happy new year!

During the following three weeks, the YaST team will interrupt the usual sprint-based development pace. That also means, almost for sure, that we will not publish any blog post about the development of YaST until mid January of 2020. So we want to take this opportunity to wish you a happy new year full of joy and Free Software.

See you soon and make sure to start the year with a lot of fun!

Highlights of YaST Development Sprint 90

December 5th, 2019 by

As usual, during this sprint we have been working on a wide range of topics. The release of the next (open)SUSE versions is approaching and we need to pay attention to important changes like the new installation media or the /usr/etc and /etc split.

Although we have been working on more stuff, we would like to highlight these topics:

  • Support for the new SLE installation media.
  • Proper handling of shadow suite settings.
  • Mount points handling improvements.
  • Help others to keep the Live Installation working.
  • Proper configuration of console fonts.
  • Better calculation of minimum and maximum sizes while resizing ext2/3/4 filesystems.
  • Small fixes in the network module.

The New Online and Full SLE Installation Media

The upcoming Service Pack 2 of SUSE Linux Enterprise products will be released on two media types: Online and Full.

On the one hand, the Online medium does not contain any repository at all. They will be added from a registration server (SCC/SMT/RMT) after registering the selected base product. The Online medium is very small and contains only the files needed for booting the system and running the installer. On the other hand, the Full medium includes several repositories containing base products and several add-ons, which can help to save some bandwidth.

Obviously, as the installer is the same for both media types, we need to adapt it to make it work properly in all scenarios. This is an interesting challenge because the code is located in many YaST packages and at different places. Keep also in mind that the same installer needs to also work with the openSUSE Leap 15.2 product. That makes another set of scenarios which we need to support (or at least not to break).

The basic support is already there and we are now fine-tuning the details and corner cases, improving the user experience and so on.

Proper Handling of Shadow Suite Settings

A few weeks ago, we anticipated that (open)SUSE would split system’s configuration between /usr/etc and /etc directories. The former will contain vendor settings and the latter will define host-specific settings.

One of the first packages to be changed was shadow, which stores now its default configuration in /usr/etc/login.defs. The problem is that YaST was not adapted in time and it was still trying to read settings only from /etc/login.defs

During this sprint, we took the opportunity to fix this behavior and, what is more, to define a strategy to adapt the handling of other files in the future. In this case, YaST will take into account the settings from /usr/etc directory and it will write its changes to a dedicated /etc/login.defs.d/70-yast.conf file.

Missing Console Font Settings

The YaST team got a nice present this year (long before Christmas) thanks to Joaquín, who made an awesome contribution to the YaST project by refactoring the keyboard management module. Thanks a lot, Joaquín!

We owe all of you a blog entry explaining the details but, for the time being, let’s say that now the module plays nicely with systemd.

After merging those changes, our QA team detected that the console font settings were not being applied correctly. Did you ever think about the importance of having the right font in the console? The problem was that the SCR agent responsible for writing the configuration file for the virtual consoles was removed. Fortunately, bringing back the deleted agent was enough to fix the problem, so your console will work fine again.

Helping the Live Installation to Survive

Years ago, the YaST Team stopped supporting installation from the openSUSE live versions due to maintainability reasons. That has not stopped others from trying to keep the possibility open. Instead of fixing the old LiveInstallation mode of the installer, they have been adapting the live versions of openSUSE to include the regular installer and to be able to work with it.

Sometimes that reveals hidden bugs in the installer that nobody had noticed because they do not really affect the supported standard installation procedures. In this case, YaST was not always marking for installation in the target system all the packages needed by the storage stack. For example, the user could have decided to use Btrfs and still the installer would not automatically select to install the corresponding btrfsprogs package.

It happened because YaST was checking which packages were already installed and skipping them. That check makes sense when YaST is running in an already installed system and is harmless when performed in the standard installation media. But it was tricky in the live media. Now the check is skipped where it does not make sense and the live installation works reasonably well again.

A More Robust YaST Bootloader

In order to perform any operation, the bootloader module of YaST first needs to inspect the disk layout of the system to determine which devices allocate the more relevant mount points like /boot or the root filesystem. The usage of Btrfs, with all its exclusive features like subvolumes and snapshots, has expanded the possibilities about how a Linux system can look in that regard. Sometimes, that meant YaST Bootloader was not able to clearly identify the root file system and it just crashed.

"Missing '/' mount point" error

Fortunately, those scenarios are reduced now to the very minimum thanks to all the adaptations and fixes introduced during this sprint regarding mount points detection. But there is still a possibility in extreme cases like unfinished rollback procedures or very unusual subvolumes organization.

So, in addition to the mentioned improvements in yast2-storage-ng, we have also instructed yast2-bootloader to better deal with those unusual Btrfs scenarios, so it will find its way to the root file system, even if it’s tricky. That means the “missing ‘/’ mount point” errors should be gone for good.

But in case we overlooked something and there is still an open door to reach the same situation again in the future, we also have improved YaST to display an explanation and quit instead of crashing. Although we have done our best to ensure this blog entry will be the only chance for our users to see this new error pop-up.

YaST2 Bootloader: root file sytem not found

Improving the Detection of Mount Points

As mentioned above, improving the detection of mount points helped to prevent some problems that were affecting yast2-bootloader. However, that is not the only module that benefits from such changes.

When you run some clients like the Expert Partitioner, they automatically use the libstorage-ng library to discover all your storage devices. During that phase, libstorage-ng tries to find the mount points for all the file systems by inspecting /etc/fstab and /proc/mounts files. Normally, a file system is mounted only once, either at boot time or manually by the user. For the first case, both files /etc/fstab and /proc/mounts would contain an entry for the file system, for example:

$ cat /etc/fstab
/dev/sda1  /  ext4  defaults  0  0

$ cat /proc/mounts
/dev/sda1 / ext4 rw,relatime 0 0

In the example above, libstorage-ng associates the / mount point to the file system which is placed on the partition /dev/sda1. But, what happens when the user bind-mounts a directory? In such a situation, /proc/mounts would contain two entries for the same device:

$ mound /tmp/foo /mnt -o bind
$ cat /proc/mounts
/dev/sda1 / ext4 rw,relatime 0 0
/dev/sda1 /mnt ext4 rw,relatime 0 0

In the Expert Partitioner, that file system will appear as mounted at /mnt instead of /. So it will look like if your system did not have the root file system after all!

This issue was solved by improving the heuristic for associating mount points to the devices. Now, the /etc/fstab mount point is assigned to the device if that mount point also appears in the /proc/mounts file. That means, if a device is included in the /etc/fstab and the device is still mounted at that location, the /etc/fstab mount point takes precedence.

As a bonus, and also related to mount points handling, now the Expert Partitioner is able to detect the situation where, after performing a snapshot-based rollback, the system has not been rebooted. As a result, it will display a nice and informative message to the user.

System not rebooted after snapshot rollback

Improved Calculation of Minimum and Maximum Sizes for ext2/3/4

If you want to resize a filesystem using YaST, it needs to find out the minimum and maximum sizes for the given filesystem. Until now, the estimation for ext2/3/4 was based on the statvfs system call and it did not work well at all.

Recently, we have improved YaST to use the value reported by resize2fs as the minimum size which is more precise. Additionally, YaST checks now the block size and whether the 64bit feature is on to calculate the maximum size.

Polishing the Network Module

As part of our recent network module refactorization, we have improved the workflow of wireless devices configuration, among other UI changes. Usually, these changes are controversial and, as a consequence, we received a few bug reports about some missing steps that are actually not needed anymore. However, checking those bugs allowed us to find some small UI glitches, like a problem with the Authentication Mode widget.

Moreover, we have used this sprint to drop the support for some deprecated device types, like Token Ring or FDDI. Below you can see how bad the device type selection looks now. But fear not! We are aware and we will give it some love during the next sprint.

Network Device Type Selection

Conclusions

The last sprint of the year is already in progress. This time, we are still polishing our storage and network stacks, improving the migration procedure, and fixing several miscelaneous issues. We will give you all the details in two weeks through our next sprint report. Until then, have a lot of fun!

Highlights of YaST Development Sprint 87

October 23rd, 2019 by

It’s time for another YaST team report! Let’s see what’s on the menu today.

  • More news and improvements in the storage area, specially regarding encryption support.
  • Some polishing of the behavior of YaST Network.
  • New widgets in libYUI.
  • A look into systemd timers and how we are using them to replace cron.
  • And a new cool tool for developers who have to deal with complex object-oriented code!

So let’s go for it all.

Performance Improvements in Encrypted Devices

As you may know, we have recently extended YaST to support additional encryption mechanisms like volatile encryption for swap devices or pervasive encryption for data volumes. You can find more details in our blog post titled "Advanced Encryption Options Land in the YaST Partitioner".

Those encryption mechanisms offer the possibility of adjusting the sector size of the encryption layer according to the sector size of the disk. That can result in a performance boost with storage devices based on 4k blocks. To get the best of your systems, we have instructed YaST to set the sector size to 4096 bytes whenever is possible, which should improve the performance of the encrypted devices created with the recently implemented methods.

Additionally, we took the time to improve the codebase related to encryption, based on the lessons we learned while implementing volatile and pervasive encryption. We also performed some additional tests and we found a problem that we are already fixing in the sprint that has just started.

Other improvements related to encryption

One of those lessons we have learnt recently is that resizing a device encrypted with a LUKS2 encryption layer works slightly different to the traditional LUKS1 case. With LUKS2 the password must be provided in the moment of resizing, even if the device is already open and active. So we changed how libstorage-ng handles the passwords provided by the user to make it possible to resize LUKS2 devices in several situations, although there are still some cases in which it will not be possible to use the YaST Partitioner to resize a LUKS2 device.

As a side effect of the new passwords management, now the process that analyzes the storage devices at the beginning of the installation should be more pleasant in scenarios like the one described in the report of bug#1129496, where there are many encrypted devices but the user doesn’t want to activate them all.

And talking about improvements based on our users’ feedback, we have also adapted the names of the new methods for encrypting swap with volatile keys, as suggested in the comments of our already mentioned previous blog post. We also took the opportunity to improve the corresponding warning messages and help texts.

New name and description for encryption with volatile keys

Network and Dependencies Between Devices

Similar to encryption, the network backend is another area that needed some final adjustments after the big implementation done in the previous sprints. In particular, we wanted to improve the management of devices that depend on other network devices, like VLANs (virtual LANs) or bridges.

Historically, YaST has simply kept the name of the device as a dependency, even if such device does not exist any longer. That leaded to inconsistent states. Now the dependencies are updated dynamically. If the user renames a device, then it’s automatically renamed in all its dependencies. If the user deletes a device that is needed by any other one, YaST will immediately ask the user whether to modify (in the case of bonding and bridges) or to remove (in the case of VLANs) those dependent devices.

New libYUI Widget: ItemSelector

Now that we mention the user experience, it’s fair to note that it has been quite a while since we created the last new widget for libYUI, our YaST UI toolkit. But we identified a need for a widget that lets the user select one or many from a number of items with not only a short title, but also a descriptive text for each one (and optionally an icon), and that can scroll if there are more items than fit on the screen.

So say hello to the new SingleItemSelector.

SingleItemSelector in graphical mode

As you would expect from any libYUI widget, there is also a text-based (ncurses) alternative.

SingleItemSelector in text mode

Please, note the screenshots above are just short usage examples. We are NOT planning to bring back the desktop selection screen. On the other hand, now we have the opportunity to make a prettier screen to select the computer role. Stay tuned for more news about that.

There is also an alternative version of the new widget that allows to select several items. The unsurprisingly named MultiItemSelector.

MultiItemSelector in graphical mode

Which, of course, also comes with an ncurses version.

MultiItemSelector in text mode

In the near future, we are planning to use that for selecting products and add-on modules. But this kind of widgets will find other uses as well.

Fun with Systemd Timers

And talking about the close future, many of you may know there is a plan coming together to replace the usage of cron with systemd timers as the default mechanism for (open)SUSE packages to execute periodic tasks.

In our case, we decided to start the change with yast2-ntp-client, which offers the possibility to synchronize the system time once in a while. So let’s take a look to how systemd timers work and how we used them to replace cron.

When defining a service in systemd it is possible to specify a type for that service to define how it behaves. When started, a service of type oneshot will simply execute some action and then finish. Those services can be combined with the timers, which invoke any service according to monotonous clock with a given cadence. To make that cadence configurable by the user, the YaST module overrides the default timer with another one located at /etc/systemd/system.

As a note for anyone else migrating to systemd timers, our first though was to use the EnvironmentFile directive instead of overriding the timer. But that seems to not be possible for timers.

One clear advantage of using a systemd service to implement this is the possibility of specifying dependencies and relations with other services. In our case, that allows us to specify that one time synchronization cannot be used if the chrony daemon is running, since they would both conflict. So the new system is slightly more complex than a one-liner cron script, but it’s also more descriptive and solid.

And another tip for anyone dealing with one-shot services and systemd timers, you can use systemd-cat to catch the output of any script and redirect it to the systemd journal.

Everybody Loves Diagrams

But apart from tips for sysadmins and packagers, we also have some content for our fellow developers. You know YaST is a huge project that tries to manage all kind of inter-related pieces. Often, the average YaST developer needs to jump into some complex module. Code documentation can help to know your way around YaST internals that you don’t work with every day. To generate such documentation, we use the YARD tool, and its output is for example here, for yast-network. Still, for large modules with many small classes, this is not enough to get a good overview.

Enter yard-medoosa, a plugin for YARD that automatically creates UML class diagrams, clickable to get you to the classes textual documentation.

The yast2-network medoosa

It is still a prototype but it has proven useful for navigating a certain large pull request. We hope to soon tell you about an improved version.

More Solid Device Names in fstab and crypttab

Back to topics related to storage management, you surely know there are several ways to specify a device to be mounted in the /etc/fstab file or a device to be activated in the /etc/crypttab. Apart from using directly the name of the device (like /dev/sda1) or any of its alternative names based on udev, you can also use the UUID or the label of the file-system or of the LUKS device.

By default, YaST will use the udev path in s390 systems and the UUID in any other architecture. Although that’s something that can be configured modifying the /etc/sysconfig/storage file or simply using this screen of the Partitioner, which makes possible to change how the installation (both the Guided Setup and the Expert Partitioner) writes the resulting fstab and crypttab files.

Changing the way devices are referenced

But, what happens when the default option (like the udev path) is not a valid option for some particular device? So far, YaST simply used the device name (e.g. /dev/sda1) as an immediate fallback. That happened at the very end of the process, when already writing the changes to disk.

We have improved that for Tumbleweed, for SLE-15-SP1 (which implies Leap 15.1) and for the upcoming versions of (open)SUSE. Now, if the default value is not suitable for a particular device because the corresponding udev path does not exists, because using a given name is incompatible with the chosen encryption method, or for any other reason, YaST will fall back to the most reasonable and stable alternative. And it will do it from the very beginning of the process, being immediately visible in the Partitioner.

Stay Tuned for More… and Stay Communicative

As usual, when we publish our sprint report we are already working on the next development sprint. So in approximately two weeks you will have more news about our work, this time likely with a strong focus in AutoYaST.

Don’t forget to keep providing us feedback. As commented above, it’s very valuable for us and we really use it as an input to plan subsequent development sprints.

Advanced Encryption Options Land in the YaST Partitioner

October 9th, 2019 by

Welcome to a new sneak peek on the YaST improvements you will enjoy in SLE-15-SP2 and openSUSE Leap 15.2… or much earlier if you, as most YaST developers, are a happy user of openSUSE Tumbleweed.

In our report of the 84th sprint we mentioned some changes regarding the encryption capabilities of the YaST Partitioner, like displaying the concrete encryption technology or the possibility to keep an existing encryption layer.

Keeping the previous encryption layer

And the report of sprint 85 contained a promise about a separate blog post detailing the new possibilities we have been adding when it comes to create encrypted devices.

So here we go! But let’s start with a small disclaimer. Although some of the new options are available for all (open)SUSE users, it’s fair to say that this time the main beneficiaries are the users of s390 systems, which may enjoy up to four new ways of encrypting their devices.

Good Things don’t Need to Change

As you may know, so far the YaST Partitioner offered an “Encrypt Device” checkbox when creating or editing a block device. If such box is marked, the Partitioner asks for an encryption password and creates a LUKS virtual device on top of the device being encrypted.

LUKS (Linux Unified Key Setup) is the standard for Linux hard disk encryption. By providing a standard on-disk-format, it facilitates compatibility among distributions. LUKS stores all necessary setup information in the partition header, enabling to transport or migrate data seamlessly. So far, there are two format specifications for such header: LUKS1 and LUKS2. YaST uses LUKS1 because is established, solid and well-known, being fully compatible with the (open)SUSE installation process and perfectly supported by all the system tools and by most bootloaders, like Grub2.

You should not fix what is not broken. Thus, in most cases, the screen for encrypting a device has not changed at all and it still works exactly in the same way under the hood.

Editing an encrypted device

But using an alternative approach may be useful for some use cases, and we wanted to offer an option in the Partitioner for those who look for something else. So in some special cases that screen will include a new selector to choose the encryption method. Let’s analyze all those new methods.

Volatile Swap Encryption with a Random Key

When a swap device has been marked to be encrypted, the user will be able to choose between “Regular LUKS1” and “Volatile Encryption with Random Key”. Both options will be there for swap devices on all hardware architectures. The first option simply uses the classical approach described above.

Selecting the encryption method

The second one allows to configure the system in a way in which the swap device is re-encrypted on every boot with a new randomly generated password.

Encrypt swap with a random password

Some advanced users may point that configuring such a random encryption for swap was already possible in versions of openSUSE prior to Leap 15.0. But the procedure to do so was obscure to say the least. The encryption with a random password was achieved by simply leaving blank the “Enter a Password” field in the encryption step. The exact implications were not explained anywhere in the interface and the help text didn’t mention all the risks.

Now the same configuration can be achieved with a more explicit interface, relevant information is provided as you can see in the screenshot below and some extra internal controls are in place to try to limit the potential harm.

Help text for the random swap method

With this approach, the key used to encrypt the content of the swap is generated on every boot using /dev/urandom which is extremely secure. But you can always go a bit further…

Swap Encryption with Volatile Protected AES Keys

One of the nice things about having a mainframe computer (and believe us there are MANY nice things) is the extra security measures implemented at all levels. In the case of IBM Z or IBM LinuxONE that translates into the so-called pervasive encryption. Pervasive encryption is an infrastructure for end-to-end data protection. It includes data encryption with protected and secure keys.

In s390 systems offering that technology, the swap can be encrypted on every boot using a volatile protected AES key, which offers an extra level of security compared to regular encryption using data from /dev/urandom. This document explains how to setup such system by hand. But now you can just use YaST and configure everything with a single click, as shown in the following screenshot.

Encrypting swap with volatile protected keys

The good thing about this method is that you can use it even if your s390 system does not have a CCA cryptographic coprocessor. Oh, wait… you may not know what a cryptographic coprocessor is. Don’t worry, just keep reading.

Pervasive Encryption for Data Volumes

Have you ever wondered how James Bond would protect his information from the most twisted and resourceful villains? We don’t know for sure (or, at least, we are supposed to not disclosure that information), but we would bet he has an s390 system with at least one Crypto Express cryptographic coprocessor configured in CCA mode (shortly referred as a CCA coprocessor).

Those dedicated pieces of hardware, when properly combined with CPU with CPACF support, make sure the information at-rest in any storage device can only be read in the very same system where that information was encrypted. They even have a physical mechanism to destroy all the keys when the hardware is removed from the machine, like the self-destruction mechanisms in the spy movies!

As documented here, the process to enjoy the full power of pervasive encryption for data volumes in Linux can be slightly complex… unless you have the YaST Partitioner at hand!

Pervasive volume encryption

As you can see in the screenshot above, the process with YaST is as simple as choosing “Pervasive Volume Encryption” instead of the classic LUKS1 that YaST uses regularly for non-swap volumes. If YaST finds in the system a secure AES key already associated to the volume being encrypted, it will use that key and the resulting encryption device will have the DeviceMapper name specified for that key. If such secure keys don’t exist, YaST will automatically register a new one for each volume.

Help for pervasive volume encryption

Pervasive encryption can be used on any volume of the system, even the root partition.

I want it all!

So far we have seen you can use protected AES keys for randomly encrypting swap and registered secure keys for protecting data volumes. But what if you want your swap to be randomly encrypted with a volatile secure AES key? After all, you have already invested time and money installing those great CCA coprocessors, let’s use them also for the random swap encryption!

If your hardware supports it, when encrypting the swap you will see a “Volatile Encryption with Secure Key” option, in addition to the other four possibilities commented above. As easy as it gets!

All possible methods for encrypting a swap

More Booting Checks in non-s390 Systems

As described in the help for pervasive volume encryption showed above, that encryption method uses LUKS2 under the hood. So we took the opportunity to improve the Partitioner checks about encryption and booting. Now, in any architecture that is not s390 the following warning will be displayed if the expert partitioner is used to place the root directory in a LUKS2 device without a separate plain /boot.

New check for booting

As mentioned, that doesn’t apply to s390 mainframes. The usage of zipl makes possible to boot Linux in those systems as long as the kernel supports the encryption technology, independently of the Grub2 (lack of) capabilities.

What’s next?

We are still working to smooth off the rough edges of the new encryption methods offered by YaST and to add AutoYaST support for them. You may have noticed that most of the improvements currently implemented will only directly benefit s390 systems… even just a subset of those. But at the current stage, we already have built the foundation for a new era of encryption support in YaSTland.

We are thinking about adding more encryption methods that could be useful for all (open)SUSE users, with general support for LUKS2 being an obvious candidate. But that’s not something we will see in the short term because there are many details to iron up first in those technologies to make then fit nicely into the current installation process.

But hey, meanwhile you can play with all the other new toys!

Highlights of YaST Development Sprint 84

September 16th, 2019 by

The YaST Team finished yet another development sprint last week and we want to take the opportunity to let you all glance over the engine room to see what’s going on.

Today we will confess an uncomfortable truth about how we manage the Qt user interface, will show you how we organize our work (or at least, how we try to keep the administrative part of that under control) and will give you a sneak peak on some upcoming YaST features and improvements.

Let’s go for it!

There Be Dragons: YaST Qt UI Event Handling

The YaST Qt UI is different in the way that it uses Qt. Normal Qt applications are centered around a short main program that after initializing widgets passes control to the Qt event loop. Not so YaST: it is primarily an interpreter for the scripts (today in Ruby, in former times in YCP) that are executed for the business logic. Those scripts, among other things, also create and destroy widget trees. But the control flow is in the script, not in a Qt event loop. So YaST uses different execution threads to handle both sides: graphic’s system events for Qt widgets and the control flow from the scripts.

This was always quite nonstandard, and we always needed to do weird things to make it happen. We always kind of misused Qt to hammer it into shape for that. And we always feared that it might break with the next Qt release, and that we might have a hard time to make it work again.

This time has now come with bug#1139967, but we were lucky enough to find a Qt call to bring it back to life; a QEventLoop::wakeUp() call fixed it. We don’t quite know (yet) why, though. Any hint, anyone?.

Should we even tell you that? Well, we think you deserve to know. After all, it worked well for about 20 years (!)… and now it’s working again.

The Refactoring of YaST Network Keeps Going

What is still not working that fine is the revamped network management. We have been working on it during the latest sprint, but it will take at least another one before it’s stable enough to be submitted to openSUSE Tumbleweed.

On which parts have we be working during the this sprint? Glad you asked! 😉 We are cleaning the current mess in wireless configuration. Soon that part will be more intuitive and consistent with other types of network. We are also making sure we propose meaningful defaults for the new cards added to the network configuration (all types of cards, not only wireless). The functionality to configure udev in order to enjoy stable names for the network interfaces has also received some love. The new version is more stable and flexible. And last but not least, we are improving the device activation in s390 systems for it to be more straightforward in the code and more clear in the user interface.

If everything goes as planned, by the end of the next sprint we will be ready to submit the improved YaST Network to Tumbleweed. That will be the right time for a dedicated blog post with screenshots and further explanations of all the changes.

Enhancing the Partitioner Experience with Encrypted Devices

And talking about ongoing work, we are currently working to broaden the set of technologies and use-cases the Partitioner supports when it comes to data encryption. As with the network area, the big headlines in that regard will have to wait for future blog posts. But if you look close enough to the user interface of the Partitioner available in Tumbleweed you can start spotting some small changes that anticipate the upcoming new features.

The first change is very subtle. When visualizing the details of an encrypted device, next to the previously existing “Encrypted: Yes” text you will now be able to see the concrete type of encryption. For all devices encrypted using YaST, that type will always be LUKS1 since that’s the only format that YaST has supported… so far. 😉

Partitioner: show the type of encryption

Some other small changes are visible when editing an encrypted device. If such a device was not originally encrypted in the system, nothing changes apart from minor adjustments in the labels. The user simply sees a form with an empty field to enter the password.

Encrypting a plain device

When editing for a second time a device that was already marked for encryption during the current execution of the Partitioner, the form is already prefilled with the password entered before. In the past, the previous encryption layer was ditched (so it’s password and other arguments were forgotten) and the user had to define the encryption again from scratch. That will become even more relevant soon, when the form for encryption becomes more than just a password field. Stay tuned for news in that regard.

Editing an encrypted device

Moreover, when editing a device that is already encrypted in the system, an option is offered to just use the existing encryption layer instead of replacing it with a (likely more limited) encryption created by the Partitioner.

Keeping the previous encryption layer

Apart from opening the door for more powerful and relevant changes in the area of encryption, these changes represent an important usability improvement by themselves.

Tidying up the YaST Team’s Trello Board

As you can see in this report and in all the previous ones, the YaST Team works constantly on many different areas like installation, network management, storage technologies… you name it. We use Trello to organize all that work. For each bug in Bugzilla or feature in Jira there is a corresponding Trello card. As it happens, sometimes when a bug is closed its Trello card is forgotten to be updated.

A check with ytrello revealed that, out of a total of 900-something cards, about 500 were outdated and could be closed. More than the half! Why so many?

We found that quite a number of these cards were open cards in closed (archived) lists. So when you archive a list, don’t forget to archive its cards before. We have just learned that Trello does not do this automatically. That’s exactly why there’s a menu item Archive All Cards in This List... besides Archive This List in the Trello user interface.

Back to work!

This has been a summer of promises on our side. We told you we are working to improve our user interface library (libYUI), revamping the code to manage the network configuration, extending the support for encryption… which means we have a lot work to be finished.

So let us go back to work while you do your part – having a lot of fun!

openSUSE OBS git mirror

September 13th, 2019 by

There was some discussion about our OBS and how in contrast Gentoo, VoidLinux or Fedora used git to track packages.

So I made an experimental openSUSE:Factory git mirror to see how well it goes and how using it feels.
The repo currently needs around 1GB but will slowly grow over time. I did not want to spend effort to import all history.

Binary files are replaced by cryptographically secure symlinks into IPFS
and I am currently providing files up to 9MB there.

If you can not run ipfs, you can still get these files through any of the public gateways like this:
curl https://ipfs.io$(readlink packages/a/aubio/aubio-0.4.9.tar.bz2) > OUTPUT

So some benefits are already obvious.
It is now much easier to find and download our patches.
Downloading and seaching all of openSUSE is now much faster.
And it works even on Thursdays (when our maintenance window often causes OBS downtimes).

It is meant to be a read-only mirror, so there is no point in opening pull-requests on github.

I hope, you enjoy it and have a lot of fun…

Highlights of YaST Development Sprint 83

August 30th, 2019 by

The summer is almost gone but, looking back, it has been pretty productive from the YaST perspective. We have fixed a lot of bugs, introduced quite interesting features to the storage layer and the network module refactoring continues to progress (more or less) as planned.

So it is time for another sprint report. During the last two weeks, we have been basically busy squashing bugs and trying to get the network module as feature-complete as possible. But, after all, we have had also some time to improve our infrastructure and organize for the future.

YaST2 Network Refactoring Status

Although we have been working hard, we have not said a word about the yast2-network refactoring progress since the end of July, when we merged part of the changes into yast2-network 4.2.9 and pushed it to Tumbleweed. That version included quite a lot of internal changes related to the user interface and a few bits of the new data model, especially regarding routing and DNS handling.

However, things have changed a lot since then, so we would like you to give you an overview of the current situation. Probably, the most remarkable achievement is that the development version is able to read and write the configuration using the new data model. OK, it is not perfect and does not cover all the use cases, but we are heading in the right direction.

In the screencast below you can see it in action, reading and writing the configuration of an interface. The demo includes handling aliases too, which is done way better than the currently released versions.

YaST2 Network New Data Model in Action

Moreover, we had brought back support for many types of devices (VLAN, InfiniBand, qeth, TAP, TUN, etc.), improved the WiFi set-up workflow and reimplemented the support for renaming devices.

Now, during the current sprint, we are focused on taking this new implementation to a usable state so we can release the current work as soon as possible and get some feedback from you.

Finally, if you like numbers, we can give you a few. Since our last update, we have merged 34 pull requests and have increased the unit test coverage from 44% in openSUSE Leap 15.0/SUSE Linux Enterprise SP1 to around 64%. The new version is composed of 31.702 (physical) lines of code scattered through 231 files (around 137 lines per file) vs 22.542 in 70 files of the old one (more than 300 lines per file). And these numbers will get better as we continue to replace the old code 🙂

Missing Packages in Leap

It turned out that some YaST packages were not updated in Leap 15.1. The problem is that, normally, the YaST packages are submitted to the SLE15 product and they are automatically mirrored to the Leap 15 distribution via the build service bots. So we do not need to specially handle the package updates for Leap.

However, there are few packages which are not included in the SUSE Linux Enteprise product line, but are included in openSUSE Leap. Obviously these packages cannot be updated automatically from SUSE Linux Enterprise because they are not present there. In this case Leap contained the old package versions from the initial 15.0 release.

In order to fix this issue, we manually submitted the latest packages to the Leap 15.2 distribution. To avoid this problem in the future we asked the Leap maintainers to add the Leap specific packages to a check list so they are verified before the next release. Of course, if you see any outdated YaST package in Leap you can still open a bug report. 😉

Just for reference, the affected packages are: yast2-alternatives, yast2-slp-server, yast2-docker and skelcd-control-openSUSE (the content is only present on the installation medium, it’s not released as an RPM).

Let’s use all disks!

As you may remember, three sprints ago we added some extra configuration options to make the storage guided proposal able to deal with the SUSE Manager approach. We even wrote a dedicated blog post about it!

Despite offering the new options in the Guided Setup, we tried to keep the default initial behavior of the installer consistent with other (open)SUSE products. So the installer initially tried to install the whole system in a single disk, unless that was impossible or it was told by the user to expand on several disks.

But the SUSE Manager folks found that to be contrary to the new ideas introduced in their Guided Setup. According to their feedback, in this case remaining consistent with other (open)SUSE product was not reducing the confusion, but rather increasing it. SUSE Manager should try from the very beginning to expand the product as much as possible among all available disks.

For that reason, during this sprint we introduced the first improvement (a.k.a. another configuration option), so now it is possible to tell whether the initial proposal should try to use multiple disks as first try.

Bootloader and Small MBR Gaps

We received a bug report because a system was not able to boot after installation. In this case, the user decided to use Btrfs and placed the root file system in a logical partition. In theory, this scenario should work but, unfortunately, the MBR gap was too small to embed the Grub2 bootloader code.

At first sight, this problem could be solved by asking YaST to install the bootloader into the logical partition and the generic boot code in the MBR. But this will only work if you set the logical partition as the active one. Sadly, some BIOSes could insist on having a primary partition as the active one.

But don’t worry, we have good news. Grub2 maintainers took care of this problem. In case the MBR gap is too small, Grub2 will automatically fall-back to the Btrfs partition. That’s all. And what does it mean for YaST? Well, thanks to this fix, YaST will simply work out of the box and your system will be bootable again. But not so fast! You still have to wait a little bit more to have these Grub2 improvements available in a Tumbleweed installer.

Handling Empty Comment Lines in NTP Configuration

AutoYaST supports defining an specific NTP configuration to be applied during the installation and it relies in Augeas to read/write the ntp.conf file. But it seems that Augeas has some problems when it tries to write comments with empty lines, as you can see in bug 1142026. The solution was to adapt YaST to filter out empty comment lines before saving the configuration file, working around the Augeas problem.

Error Resizing Some Partitions

Typically, an MS-DOS partition table reserves its first MiB for the MBR gap, so the partitions normally start after that point. But it is possible, especially in partitions for old Windows systems, that it starts before that first MiB. In that case, if we try to resize that partition (e.g., by using the Expert Partitioner), YaST crashes due to an error when calculating the resize information. Fortunately, this problem is gone now, and you will be able to resize this kind of partitions as well.

Side Effects of Keyboard Layouts Unification

During the sprint 81, the openSUSE and SUSE Linux Enterprise console keyboard layouts were unified after some minor changes. One of those changes was to stop using the, in appearance, useless keymaps symlinks for Arabic and Cambodian. But they were there for a reason: are being used by YaST to correctly adapt the keyboard in the X11 environment. Just visit the pull request if you prefer to scare yourself want to dive in more technical details.

Fortunately for the users of those keyboards, we realized about this problem before the upcoming SLE-15-SP2 was released 🙂 And, it’s fixed.

House Keeping Tasks

As part of our development duties for this sprint, we invested quite some time in reviewing and updating our continuous integration (CI) set up. Apart from using Travis CI for pull requests, we rely on Jenkins to run the tests and submit the code to the appropriate projects in the Open Build Service instances.

Then, when the development of a new version starts or when the product is about to be released, we need to adjust the configuration. Just in case you are wondering, we do not do this work by hand anymore and we use Salt and Jenkins Job Builder to handle this configuration.

Closing Thoughts

During the next sprint (actually, the current one) we are working in three different areas, apart from squashing bugs: improving encryption support in the storage layer, adding some features to the installer (repo-less installer, support for reading product licenses from a tarball, etc.) and, of course, refactoring the network code. Obviously, we will give you all sort of details in our next sprint report.

Highlights of YaST Development Sprint 82

August 14th, 2019 by

July and August are very sunny months in Europe… and chameleons like sun. That’s why most YaST developers run away from their keyboards during this period to enjoy vacations. Of course, that has an impact in the development speed of YaST and, as a consequence, in the length of the YaST Team blog posts.

But don’t worry much, we still have enough information to keep you entertained for a few minutes if you want to dive with us into our summer activities that includes:

  • Enhancing the development documentation
  • Extending AutoYaST capabilities regarding Bcache
  • Lots of small fixes and improvements

AutoYaST and Bcache – Broader Powers

Bcache technology made its debut in YaST several sprints ago. You can use the Expert Partitioner to create your Bcache devices and improve the performance of your slow disks. We even published a dedicated blog post with all details about it.

Apart of the Expert Partitioner, AutoYaST was also extended to support Bcache devices. And this time, we are pleased to announce that … we have fixed our first Bcache bug!

Actually, there were two different bugs in the AutoYaST side. First, the auto-installation failed when you tried to create a Bcache device without a caching set. On the other hand, it was not possible to create a Bcache with an LVM Logical Volume as backing device. Both bugs are gone, and now AutoYaST supports those scenarios perfectly.

Configuring Bcache and LVM with AutoYaST

But Bcache is a quite young technology and it is not free of bugs. In fact, it fails when the backing device is an LVM Logical Volume and you try to set the cache mode. We have already reported a bug to the Bcache crew and (as you can see in the bug report) a patch is already been tested.

Enhancing Our Development Documentation

This sprint we also touched our development documentation, specifically we documented our process for creating the maintenance branches for the released products. The new branching documentation describes not only how to actually create the branches but also how to adapt all the infrastructure around (like Jenkins or Travis) which requires special knowledge.

We will see how much the documentation is helpful next time when somebody has to do the branching process for the next release. 😉

Working for a better world YaST

We do our best to write code free of bugs… but some bugs are smarter than us and they manage to survive and reproduce. Fortunately we used this sprint to do some hunting.

Those are only some examples of the kind of bugs we have fought during this sprint. But checking bug reports has also made us think in the future…

LibYUI in 21st Century

We fixed a bug related to how the focus was managed in text mode after changing any setting via the hyperlinks available in the installation summary.

Installation summary in text mode

The implemented solution is actually not perfect, it’s just the better we can do with our set of widgets. And that was yet another example of such problem – LibYUI is an awesome library that allows us to create interfaces that work in both graphic and text modes, but it has basically not evolved for more than a decade… and it’s time to fix that!

So we have been discussing how to organize our time in the close future to leave some room for innovation and renovation regarding LibYUI and the YaST UI in general. Stay tuned for more news.

August is still not over

The YaST Team will keep working during the rest of the summer sharpening our Linux Swiss army knife. But half of the team is still on vacation or starting their vacation now. So most likely our next report will be here in two weeks and it will also be a light read.

Meanwhile, don’t forget to have a lot of fun!

Highlights of YaST Development Sprint 80

July 19th, 2019 by

After our (really long) sprint report, which featured information from 3 different sprints together, we are back to our regular schedule of publishing a blog post after each sprint.

In a nutshell, these are the highlights from the previous one:

  • A new version of yast2-network will be submitted to Tumbleweed shortly, including quite some internal changes as part of the refactoring effort.
  • The support for offline installation of SUSE Linux Enterprise products has been improved to handle modules and extensions dependencies automatically, among other goodies.
  • The partitioning proposal has been adapted to support SUSE Manager special needs.
  • The guided partitioning ignores now the adjust_by_ram parameter in IBM z Systems, where it is basically useless.
  • Some (open)SUSE 15 features have been backported to 5th Service Pack of SUSE Linux Enterprise.

And last but not least, we have some words to say about the feedback we get from you (that we really appreciate!) and the future of YaST.

Shipping Another Round of Network Refactored Code

One of the problems we wanted to avoid while refactoring yast2-network is diverging too much from the version shipped in Tumbleweed. As we mentioned in our last report, we have done quite some work, especially when it comes to the internals of the user interface, and codebases are starting to look quite different.

For that reason, we decided to merge the new code into the master branch so it can be included in Tumbleweed shortly. We are talking about a pull request which contains more than 340 commits, adds 9000 lines, removes 5000 and modifies 197 files. So, what could possibly go wrong? To be honest we did quite some testing but there is a chance that you can find some issues. In that case, please, open a bug report and we will happily fix it.

About the technical details, we put most of our efforts into drawing a clear line between the user interface and the business logic. In parallel, we are still working on the new data model which enables us to read/write the network configuration using different backends, although at this time we are only supporting sysconfig. We have made some progress during this sprint (you can check the pull request if you are interested), but we will not merge this code into our master branch yet. You can find more details in the updated documentation.

After reading this status update about the yast2-network refactoring, you might be wondering about our plans. Basically, we would like to finish the support to read/write network interfaces configuration and start using it through all the code, killing some rather old and buggy modules like LanItems.

Offline Media Support

For SUSE Linux Enterprise 15 products, there are two installation media:

  • The installer media, which contains only basic packages for a minimal system.
  • The so-called Packages DVD, which contains several modules and extensions, like the Development Tools Module or the Server Application Module.

The main use case of the Packages DVD is allowing to install the systems without an Internet connection. However, YaST support for such a scenario is pretty simplistic. For instance, dependencies between modules are not evaluated and the user has to select them manually. This approach is error-prone and user-unfriendly.

Fortunately, these shortcomings will be addressed in SLE 15 SP2. YaST is now able to solve modules and extensions dependencies, so the user does not have to worry about them anymore. For instance, if you select the Web and Scripting Module, the Basesystem Module and Server Application Module will be automatically selected.

At first, solving dependencies through the 25 repositories which are included in the Packages DVD might be time-consuming, but the current implementation takes advantage of libsolv to reduce it to about 2 seconds in our testing machine.

Improved Extensions and Modules Selection

Additionally, other improvements have been included, like displaying additional information about each module/extension or filtering out base products from the list.

SUSE Manager and the Partitioning Guided Setup

As you know, YaST is a very flexible Linux installer. The creators of each (open)SUSE product or role can define the particular sequence of steps presented to the user and configure many of the options on each of those step. But that was still not enough for the maintainers of SUSE Manager, the purpose-specific distribution to manage software-defined infrastructures.

SUSE Manager follows a pretty specific approach regarding the management of the storage devices. So we had to add some extra configuration options to our partitioning guided proposal to fulfill their needs. We felt the topic deserved its own separate blog post. So follow the link and enjoy the love story of SUSE Manager and the Partitioning Guided Setup.

Taking care of IBM z Systems

For many reasons, IBM z System architecture (s390) is one of those special cases where YaST flexibility is crucial. One of the many options that YaST offers to products and roles creators is called adjust_by_ram, which instructs YaST to make sure that a given volume size is, at least, as big as the RAM of the system. Obviously, this option is especially useful for the swap volume when we want to be able to suspend the system to disk.

However, on the 64-bits s390 architecture, resuming from a swap device is not supported, which renders this option useless. For that reason, YaST will take this situation into account, and it will not enlarge the swap volume bye default for s390 machines, even if adjust_by_ram was set in the control file. Of course, the user will be able to enlarge the volume by activating the corresponding checkbox in the Guided Setup options.

Guided Partitioning Filesystem Options

Backporting Stuff to SLE 12 SP5

Although most of the development effort is focused on SLE 15 SP2 and openSUSE Leap 15.2, we do not forget about SLE 12. SUSE is working on the 5th Service Pack of this release and the YaST team is no exception.

During this sprint, SLE 12 got two interesting features that were backported from SLE 15. The first one is the ability to specify kernel mitigations settings, as you already can do in Tumbleweed. And the second one is the support for PMEM devices, that you can see in the screenshot below.

YaST2 Partitioner PMEM Support

Recently, there was an interesting discussion about the openSUSE installer on Reddit, continued as a thread on our yast-devel mailing list.

This is just a quick heads-up to confirm that we hear you. Some of the issues discussed there were already known to be sore points, some others may need more clarification on why and how things are as they are.

We are taking your feedback and criticism seriously. But we also have to consider all the requirements, and we try to avoid making unrealistic promises. So please bear with us until we had some further discussions first about what we can do and in what time frame.

Conclusions

In addition to what we have described in this report, we have been working on fixing bugs and making other small improvements here and there. And sprint 81th will not be different: part of the team will keep working on yast2-network while the rest squashes as many bugs as possible.

SUSE Manager and the Partitioning Guided Setup

July 16th, 2019 by

Apart from our usual development sprint reports, we (the YaST Team) sometimes publish separate blog posts to summarize a new feature or to present an idea we are working on. Lately, several of those posts have been focused on new features of the YaST Partitioner, like the support for Bcache or the new Btrfs capabilities. But today it’s the turn of another part of yast2-storage-ng: the partitioning proposal, also known as the Guided Setup.

As you may know, YaST is an universal installer used to configure all the (open)SUSE and derivative products. Moreover, the installer options and steps can be refined even further by each of the system roles available for each product. The goal of this blog post is to present some ideas aimed to add new possibilities in the area of the storage guided proposal for those who configure the installer for a certain product or system role. With that we hope to ease the life for the creators of SUSE Manager, the SUSE’s purpose-specific distribution to manage software-defined infrastructures.

Although many of the presented capabilities will land soon in openSUSE Tumbleweed they will not be used by default. Not only because they are not targeted to the openSUSE use-case, but also because so far this is just a prototype. That means all texts are subject to change and most screens will get some adaptations before being used in a final product… or maybe they will even be completely revamped.

One Guided Proposal to Rule them All

Although the Expert Partitioner can be used to tweak the storage configuration of any SUSE or openSUSE distribution during installation, the installer always tries to offer a reasonable proposal about it. Moreover, the “Guided Setup” button in the “Suggested Partitioning” screen leads to a wizard that can be used to configure some aspects of such a proposal, as shown in the following diagram (some actions have been blurred just to emphasize the fact that the concrete list of actions will change after each execution of the wizard).

Default Guided Setup wizard

The exact behavior of the Guided Setup is different in every product and, potentially, in every system role. Many things can be adjusted by the creators of the product or the role, like the partitions and LVM volumes to be proposed, the options to be offered in the wizard, the default value for every option and much more. But all those possibilities were still not enough in the case of SUSE Manager and its unique approach to organize the storage devices.

The Strange Case of SUSE Manager

First of all, the SUSE Manager documentation suggests to allocate each of several data directories (/var/spacewalk, /var/lib/pgsql, /var/cache and /srv) in its own dedicated disk when installing in a production environment. For such setup to make sense, it’s absolutely crucial to choose the right disk for every data directory taking into account both the size and the speed of the disks.

The documentation also suggests to use LVM in production environments. In order to achieve a clear separation of disks when using LVM, the recommended approach is to set up a separate LVM volume group for each relevant data directory instead of allocating all the logical volumes in the usual single shared “system” group.

So, although it may look overkill when installing SUSE Manager just for evaluation purposes, the preferred setup for a final deployment of the product spreads over up to five disks – one containing an LVM volume group with the usual logical volumes of any Linux system (like the root system and the swap space) and each of the other disks containing additional LVM groups, each one dedicated to a particular data directory.

Last but not least, the SUSE Manager guided setup should never offer the possibility of keeping the preexisting partitions in any of the disks. So the usual questions “Choose what to do with existing Linux/Windows/other partitions” (see the image above) should not even be displayed to the user. The answer is always “remove even if not needed”. Period. 😉

Breaking Down the Problem into Smaller Pieces

We didn’t want to implement a completely different guided proposal for SUSE Manager. Instead, we wanted to merge the main ideas behind its approach into the current configurable system, so other products and roles could use them. We identified three different features that we turned into the corresponding optional configuration settings at disposal of anyone defining a new system role. All the new settings are independent of each other and can be combined in any way to provide a fully customized user experience.

First Piece: Explicit Selection of Disks per Volume

First of all, it was necessary to support letting the user explicitly choose a disk for every partition or LVM volume, unlike the default guided setup which automatically finds the best disk to allocate every partition given the requirements and a set of “candidate disks”. To enable that, now the product or role can choose between two values for the new allocate_volume_mode setting. A value of auto (which is the default to keep backwards compatibility) will result in the already known wizard with up to 4 steps.

  • Select the candidate disks
  • Decide what to do with existing partitions
  • Configure the schema (LVM and/or encryption)
  • Configure each file system

As always, the steps in which there is nothing for the user to decide are skipped so the wizard is usually shorter than four steps.

No surprises so far. But allocate_volume_mode can also be set to device, which will result in the alternative wizard displayed in the following image.

New possible Guided Setup wizard

As you can see, there is no initial step to select the set of disks to be used by the system to automatically allocate the needed partitions. Instead, the following screen allows to explicitly assign a disk to every partition or LVM volume group.

New step to assign volumes and partitions to disks

Second Piece: Enforcing a Behavior about Previous Partitions

No matter which allocate mode is configured (auto or device), there is always one step in which the user is asked what to do with the preexisting partitions in the affected disks. So far, the product defined the default answer for those questions, but the user always had the opportunity to change that default option.

Now, the creator of the product or the system role can disable the setting called delete_resize_configurable which is enabled by default in order to prevent the user from modifying the default behavior. The wizard will include no questions about what to do with existing Windows/Linux/other partitions. In most cases, that will imply a whole step of the wizard to be simply skipped.

Third Piece: Separate Volume Groups for some Directories

The most important setting configured by every system role is the list of so-called volumes. That list includes all the file systems (both mandatory and optional ones) that the guided setup should create as separate partitions or LVM logical volumes. Now it’s possible to specify that a volume could be created in its very own separate LVM volume group using the new attribute separate_vg_name. If any of the volumes defined for the current product and role contains such attribute, the screen for selecting the schema will contain an extra checkbox below the usual LVM-related one.

New checkbox for directories into their own separate LVM

Putting the Pieces Together for SUSE Manager

With all the above, we expanded the toolbox for anyone wanting to configure the (open)SUSE installation experience. Which means now we can fulfill the requirements of SUSE Manager maintainers by just adding separate_vg_name to some volumes, setting delete_resize_configurable to false and adjusting the allocate_volume_mode. With all that, the new SUSE Manager workflow for the guided setup will look like this.

First of all, the user will be able to specify the creation of separate LVM volume groups as suggested in the product documentation.

SUSE Manager setup - first screen

Then a second screen to select which separate file systems should be created and to fine-tune the options for every one of them, if any.

SUSE Manager setup - second screen

And finally a last step to assign the correct disk for every partition or separate volume group, depending on the selections on previous screens. With this step the user can optimize the performance by distributing the disks as explained in the SUSE Manager documentation, allocating the areas that need intensive processing to the faster disks and the greedy directories to the bigger devices.

SUSE Manager setup - third screen

As usual, the list of actions will reflect the selections of the user creating as many LVM structures as requested.

SUSE Manager setup - result

Beyond SUSE Manager

As already mentioned, all the guided proposal features can be combined within a given product in any way. For example, one product could adopt the approach of creating separate LVM volume groups while still sticking to the traditional auto allocate mode. Or a given system role could enforce to never delete any existing partition without allowing the user to change that.

But beyond the “Guided Setup” button, the availability of two different allocate modes brings back one idea that has been floating around since the introduction of Storage-ng – adding a section “Wizards” to the Expert Partitioner. That would allow to combine some manual steps with the execution of any of the two available allocate modes of the guided proposal… or with any other workflow that can be implemented in the future.

As always, we are looking forward any feedback about the new features or the guided partitioning proposal in general. And stay tuned for more news!