After adding many new features and introducing quite some important changes for the upcoming (open)SUSE releases, the YaST team is now going into bug squashing mode. We want to offer our users another rock solid release, so we are focusing on polishing our work.

However, when it comes to the storage layer, the story is slightly different. We are bringing back all those feature that you already know and love from the old Expert Partitioner but in a better shape.

So let’s have a look at the most relevant changes:

• We have been working on improving the system upgrade process in order to offer a smooth transition.
• The Expert Partitioner got back the settings dialog and the ability to clone disks.
• Regarding the new storage layer, subvolumes, resizing and support for unformatted DASD devices have been improved.
• YaPI is now deprecated.

In addition to that, we have fixed some issues like package installation from NFS/Samba/CIFS repositories, a problem preserving the DHCP client-id and a bunch of translations issues.

Offline Upgrade from SLE11

In the previous post we announced support for the offline migration from SLE12 to SLE15 via SCC. In this sprint we focused on upgrading from SLE11 products to SLE15.

The main difference (from the registration point of view) between SLE11 and SLE12/SLE15 is that SLE11 uses the old Novell Customer Center (NCC) for registration while the newer products use the SUSE Customer Center (SCC).

However, the SCC server knows which systems are registered in NCC (there is some kind of automatic synchronization between them) so it is possible to only use the SCC server during migration without any interaction with the old NCC. That makes the transition transparent for our users.

Besides some small adjustments, the changes in YaST were mainly related to handling the different configuration files. As expected, the most difficult part was testing, so we have added some notes below about the process.

In the end we could migrate a SLES11 system to SLES15 via SCC, performing a manual upgrade and an automated one with AutoYaST too. There were some issues related to package dependencies but these are not related to YaST or SCC, so probably some RPM packages will need to be fixed.

SLE11 Migration Notes

When migrating from a SLE11 system, there are some specific issues you should be aware:

• The SLE11 systems registered using the internal SUSE registration keys (belonging to the Novell Inc organization) are synchronized every 24 hours. That means it might take up to 24 hours after registering a new SLE11 system to make it upgradable to SLE15 via SCC. If the registration has not been synchronized from NCC yet you will see the “Invalid credentials” error during migration.
• For customers there is a trick: logging into the SCC Web UI will cause the registrations to be syncrhonized in few minutes. But this does not work for the SUSE keys.
• Migration from SLE12 is not affected by this issue because SLE12 already uses SCC and the data synchronization from NCC is not involved in the migration process.

Generic Migration Notes

These notes apply when upgrading from both SLE11 or SLE12:

• Because SLE15 is in Beta stage, your registration key needs to be entitled for the Beta products. Otherwise you will see “No migration found” error in YaST. Either join the SUSE Beta program or just wait until the SLE15 product is released and the migration is enabled.
• Once the system is migrated, the process cannot be repeated. But if you rollback the migrated system (using a backup or a snapshot), running SUSEConnect --rollback will restore your registration status.

Configuring the Expert Partitioner

The Expert Partitioner features again a “Settings” section where users can adjust some settings to influence how the partitioner works. It is a work in progress, but for the time being the user can configure how the devices should be mounted (path, UUID, label, etc.).

But these settings are not only limited to the Expert Partitioner scope but to the whole storage layer. You could, for instance, launch again the Guided Setup after adjusting them and the new values will be taken into account.

Moreover, the partitioner is now able to perform some additional checks over mount options. For example, when trying to mount a device by label, the user will be warned if that label is not valid (for instance, because there is another file system using that label).

Cloning Partitions Layout

Cloning partitions layout was one of that kind of surprising features the old Partitioner offered. Imagine you have a disk with some partitions and you would like to have the same partition layout in another disk (or even in several disks). This is very common, for example, when you are creating a MD RAID setup, where it is useful to have the disks partitioned in the same way. To avoid the annoying work of creating all disk partitions manually, the new Expert Partitioner allows again to clone a disk layout by using the Expert button.

When you are cloning a disk, you can select all the devices where you want to clone the partition scheme of the current disk. Only suitable devices for cloning will be offered, that is, you will not see in the list disks without enough size or different topology.

After selecting in which disks you want to clone, a confirmation message will be presented and you will be warned about all devices that will be deleted before performing the disk cloning. In case you accept, you will have exactly the same partitions in all selected devices, including gaps between partitions.

Temporary Mount and Unmount File Systems for Resize

Unfortunately most file systems support resizing only while the file system is mounted or while it is unmounted. Whereas doing a temporary mount is easy, a temporary unmount is often not possible since file systems are usually in use.

libstorage-ng now inserts unmount and mount actions when needed for a specific file system. It also provides function to immediately mount or unmount a file system, so that the UI can give feedback to the user if unmounting failed.

Taking Btrfs Subvolume Hierarchy Right

btrfs subvolumes are rather delicate beasts and great care has to be taken to do things right.

For one, subvolumes are organized like a directory hierarchy. And you cannot create new subvolumes just anywhere but only in free ‘leaf’ positions. That is, neither another subvolume nor a directory with this name must exist.

In other words, if (for example) a subvolume foo/bar/xxx already exists you can no longer create a subvolume foo/bar.

Let’s see what happens:

# you must mount the btrfs file system first
~ mount /dev/sdb5 /mnt

~ btrfs subvolume list -tap /mnt
ID      gen     parent  top level       path
--      ---     ------  ---------       ----
257     8       5       5               foo
258     8       257     257             /foo/bar/xxx

~ btrfs subvolume create /mnt/foo/bar
ERROR: target path already exists: /mnt/foo/bar

The reason for that error is that there is already a directory called `foo/bar`. At this point, there are two options:

• Delete the subvolume foo/bar/xxx and the directory foo/bar if you do not mind to loose the data in that subvolume
• Or rename temporarily and move it back later

Let’s move the existing subvolume out of the way (no fancy command, just using mv) and then create the new one:

~ mv /mnt/foo/bar /mnt/foo/old_bar
~ btrfs subvolume create /mnt/foo/bar
Create subvolume '/mnt/foo/bar'
~ mv /mnt/foo/old_bar/xxx /mnt/foo/bar/xxx
~ rmdir /mnt/foo/old_bar

~ btrfs subvolume list -tap /mnt
ID      gen     parent  top level       path
--      ---     ------  ---------       ----
257     9       5       5               foo
258     8       259     259             /foo/bar/xxx
259     9       257     257             /foo/bar

That’s rather a complicated stunt and you even might have valuable data in the old `foo/bar` directory that could get lost in the process. So YaST refuses to do this.

Instead, it recognizes the situation and shows a hint to the user:

Of course it’s all different if you are about to format the btrfs filesystem anyway. Then the YaST partitioner will simply create the subvolumes in the correct order and everything will be fine.

Better handling of unformatted DASD and other unavailable devices

Direct-access storage devices (DASD) are the most common storage devices in s390 mainframes. When compared to common hard disks, they are special in several ways. One of them is that they need to be formatted at low-level (to not be confused with the usual meaning of “format” related to creating a file system) in order to be used by the operating system.

We got a bug report for the pre-release of SLE15 because the installer was crashing when trying to use an unformatted DASD as part of the automatic partitioning proposal (i.e. the Guided Setup). So we instructed YaST to ignore such devices, not only in the automatic proposal but also in the Partitioner. It makes very little sense to list devices in the Partitioner that cannot be manipulated in any way. That’s consistent with the SLE15 approach for other “untouchable” devices, like the hard disks that are part of a BIOS-defined RAID or the individual connections of a multipath device.

In other words, the “Hard Disks” section of the Partitioner only shows disks that can be indeed used as disks (e.g. can be partitioned) and, unlike previous SLE15 pre-releases, now unformatted DASDs are not longer in that list. As always, the user can still go a couple of steps back in the installation process and perform a low-level format the DASD in order to make them appear.

YaPI is deprecated and should not be used anymore

Dropping some old stuff is part of our roadmap too. For instance during this sprint, we have marked YaPI as deprecated.

YaPI was designed to expose functionalities from YaST but, to be honest, it is not much used these days. Due to the complexity to maintain several interfaces every time we develop a new feature, we have decided to drop YaPI. Although the code has not been completely removed, we consider it as deprecated and it should not be used anymore.

Installing Packages from NFS or Samba/CIFS

We got a bug report that when installing a system from an NFS repository, the installed system will be unable to access that repository later.

The problem was that for accessing the NFS repositories (and Samba/CIFS as well) you need additional packages which can mount such file systems. And these packages are not included in the minimal installation.

That means you cannot access the repository later which is a quite unfortunate situation. You get into the chicken/egg problem – you need to install a package to access the repository but to access the repository you need to have the package already installed…

During debugging it turned out that the problem was caused by using a dropped value from the `/etc/install.inf` file. In the past, linuxrc wrote the type of the installation repository there, but that’s not written anymore.

The fix was to evaluate all used repositories and check if any of them is located on an NFS or Samba/CIFS system. As a side effect, it fixed an issue when installing from DVD but using an additional repository located on NFS (or Samba/CIFS).

Preserving the DHCP client-id

Since SLE15, wicked will use the RFC 4361 client-id. This change needed some adaptation in the installer to copy the created identity (duid + daid) to the installed system so, after rebooting, it can get the same IP (when it is possible) that was used during the installation.

Fixing Translation Issues

Currently we are getting quite a lot of bugs about missing translations in YaST. It turned out that some bugs were caused by a missing textdomain statement in the YaST code. In such situations, no translatable texts are extracted from those files and, obviously, translators cannot translate them.

The YaST script printed a warning in that case, but it can be easily overlooked. The reason for a warning is that the check is not perfect and it reports many false positives (it complains about missing textdomain calls that are not required).

Fixing the script would be too difficult so we took other approach: we added the harmless textdomain statement also to files which strictly did not require it. Then we could change the warning to error and stop the script with a failure.

Now the more strict check is enabled in the continuous integration (Travis) so we should get a failure earlier, before the change actually hits the build service, preventing bug reports later.

We need your help!

Our QA department is doing a pretty good job when it comes to detect problems. But we would love to get feedback from you too. So, if possible, have a look at (open)SUSE beta versions and report any issue you find.

Thanks!

Sprint 50 is finished and we are ready to share with you all the highlights of this last sprint!

We are close to finishing features for the new SUSE Linux Enterprise 15, so this time we came up with a lot of information about the new storage, partitioner, firewall enhancements, and roles, which now is also present for the Desktop version of SUSE Linux Enterprise. Besides that, we fought another issue regarding memory consumption in Tumbleweed installation and once again we were able to solve it. So, let’s take a look into details!

Filesystem type specific default mount options in /etc/fstab are back

We reimplemented another feature that was still missing with the rewrite of the YaST storage stack: Reasonable mount options in /etc/fstab that may be different for each filesystem type. For example, for an Ext4 filesystem we used to set options data=ordered,acl,user_xattr by default. Of course, you can still change those options in the partitioner if the defaults don’t work well for you.

For most filesystem types, this was straightforward: Just filling a table with fixed default options. But for some others, most notably the legacy Microsoft filesystem types FAT/VFAT, this involved some not-so-trivial heuristics to figure out locale data so we could provide reasonable values for iocharset=... and codepage=...; those are necessary to recode filenames with non-ASCII international characters between whatever a MS Windows system might use and Linux; otherwise, you might not even see those files when mounted on a Linux system.

Resurrected sections in the Expert Partitioner

We keep bringing the functionality from the old Partitioner back to the Storage-ng reimplementation. During this sprint, it was the turn of the NFS and Device Graph sections.

The NFS case is a little bit special from the technical point of view since it’s the only section not managed directly by the Expert Partitioner. Instead, an embedded version of yast2-nfs-client is executed when visiting that section. The same approach has been followed in this reimplementation, which means NFS is the only section in which the old Partitioner and its Storage-ng clone will offer an absolutely identical experience and behavior (including both features and bugs). Just a heads-up: NFSv4 support is still not fully functional in libstorage-ng, so the check-boxes about the NFS version may be ignored until that support reaches your distribution.

On the other hand, the “Device Graph” section was, as most of the new Partitioner, rewritten from scratch. In addition to the already known graph showing how the system is going to look after applying the changes, it includes now a representation of the current system in a separate tab. Both graphs are interactive, double-click on any node takes the user to the corresponding section of the Partitioner.

As you can see in the previous screenshot, the usual button to export the graph to Graphviz format is not alone anymore. Its new friend allows exporting the graph to the very detailed XML format used by the YaST Team to reproduce test scenarios.

Of course, the “Device Graph” section is not available if using the text-based ncurses interface, as you can see comparing the left part of the following screenshot with the previous one.

Partitioner: LVM thin provisioning

Our new Expert Partitioner was already able to manage LVM volume groups and logical volumes since several weeks ago. But now it also recovers its great ability to work with LVM thin pools and volumes. LVM thin provisioning is a powerful technology, very useful in cases where you need to administrate storage resources for a large group of users. Basically, thin provisioning allows you to provide more storage space than it is actually available in the system. You can increase the real hardware storage only when it is really required. For more information about LVM thin provisioning, you can find a great guide in this link.

With the Expert Partitioner, you can create a thin pool over a LVM volume group in a similar way you create a normal logical volume. You only have to add a new logical volume and check the “Thin pool” option in the first dialog. Once the volume group contains at least a thin pool, you will be able to create thin volumes. Once again, the process is exactly the same. Add a new logical volume and select “Thin volume” option and in which pool to create it. The rest of steps are exactly the same as for creating a normal logical volume.

Apart from creating new thin pools and volumes, options for editing, resizing and deleting are now also available for all kind of logical volumes: normal volumes, thin pools and thin volumes. In the case of resizing a thin pool, you will be warned when the resulting pool is overcommitted. Take a look at the following screenshot.

Adapting partitions and logical volumes sizes in AutoYaST

In older versions, when the proposed partitioning in a profile and it did not fit in the disk, AutoYaST tried its best to reduce the biggest partition. A typical use case was to set the root (/) partition to a huge value, so the profile could be used in systems with different disk sizes (as AutoYaST will take care of reducing that partition to make it fit). To be honest, the best solution in that scenario would be to set the size to max, so AutoYaST will do what’s expected.

However, if for any reason the wanted layout does not fit, AutoYaST is now smarter about what to do: instead of blindly reducing the biggest one, it will try to reduce all partitions in a (kind of) proportional way, informing the user about the new sizes.

Small behavior change in AutoYaST installation

Up to now, not installable packages have been ignored silently by AutoYaST, which has been defined in the AutoYaST configuration file. From now on, the user will be informed that these packages cannot be installed.

System roles for SUSE Linux Enterprise Desktop 15

SUSE Linux Enterprise uses system roles to define, during the installation process, the packages to install on the system, so it can have the necessary software to play its “role”. This feature was already available for SUSE Linux Enterprise Server and now it is also available for SUSE Linux Enterprise Desktop. In order to choose a specific role for your system, you need to select a module or extension which contains this role during the installation process. There are four available roles for Desktop:

• GNOME Desktop (Wayland): available when Desktop Productivity (on SLED) or Workstation Extension are selected.
• GNOME Desktop (X11): available when Desktop Productivity (on SLED) or Workstation Extension are selected.
• GNOME Desktop (Basic): available when the Desktop Application module is selected.
• IceWM Desktop (Minimal): available when Basesystem module is selected.

Firewalld enhancements

During the last sprints, our team has been working hard in the integration of firewalld with AutoYaST and with the CWM library for opening ports / services in our different modules (http-server, squid, dns-server etc..). Here are some changes we did during this last sprint:

YaST firewall module is discontinued

As we now adopted firewalld in our distribution, there is no more reasons to have a YaST module for the firewall. Therefore, as you may see here, we discontinued the YaST Firewall module and we recommend to use firewall-config to configure your firewall via a user interface or firewall-cmd for the command line.

AutoYaST

A new AutoYaST schema has been defined for firewalld configuration although the features supported are very limited.

We can configure properties like the default zone, the service state, the type of packages to be logged and zones specific configuration like interfaces, services, ports, protocols, and sources.

For example, a SuSEFirewall2 based profile defining some interfaces, services, and ports in the EXT zone like this one:

<firewall> 
  <enable_firewall config:type="boolean">true</enable_firewall>
  <start_firewall config:type="boolean">true</start_firewall>
  <FW_DEV_EXT>eth0</FW_DEV_EXT>
  <FW_SERVICES_EXT_TCP>443 80 8080</FW_SERVICES_EXT_TCP>
  <FW_SERVICES_EXT_UDP>21 22</FW_SERVICES_EXT_UDP>
  <FW_CONFIGURATIONS_EXT>dhcp dhcpv6 samba vnc-server</FW_CONFIGURATIONS_INT>
</firewall>

will be translated to something like this:

<firewall>
  <enable_firewall config:type="boolean">true</enable_firewall>
  <start_firewall config:type="boolean">true</start_firewall>
  <default_zone>public</default_zone>
  <zones config:type="list">
    <zone>
      <name>public</name>
      <interfaces config:type="list">
        <interface>eth0</interface>
      </interfaces>
      <services config:type="list">
        <service>dhcp</service>
        <service>dhcpv6</service>
        <service>samba</service>
        <service>vnc-server</service>
      </services>
      <ports config:type="list">
        <port>21/udp</port>
        <port>22/udp</port>
        <port>80/tcp</port>
        <port>443/tcp</port>
        <port>8080/tcp</port>
      </ports>
    </zone>
  </zones>
</firewall>

SuSEFirewall2 based profiles will continue working although limited to the configuration currently supported by YaST.

During the autoinstallation, an error will be shown if the profile has some property not supported, however, we will able to continue with the installation and also a warning will be shown suggesting the use of the new schema even when all the properties are translatable.

Opening ports / services in zones through interfaces selection

In YaST, CWMFirewallInterfaces module provides a set of widget definitions for opening services in zones through a selection of interfaces (each interface belongs to a ZONE).

The module has been adapted to work properly with the new firewalld API.

If a firewalld service is not defined (probably because the package shipping the service definition has not been adapted yet), then the CWMFirewallInterfaces widget will show a list of missing services suggesting to deploy them to be able to configure the firewall.

Move from Xinetd to Systemd sockets

As requested by our friends from packages, now the future of starting service on demand is systemd sockets instead of xinetd. There are multiple reasons for that, but for us the most important one is that we can concentrate in one thing and do it properly.

It’s hard to support different ways to start services on demand and especially to handle such a situation when a user can set it up with sockets. Our old approach to make it happen was to set up the start of services on demand with xinetd. However, this approach is really hard to debug. Besides this problem, we also would like to unify our approach with the one suggested by packagers people.

So what have we done in this last sprint? The goal was to do some research, to create a proof of concept on one specific module and to find potential obstacles when unifying approaches to start services on demand. We found out that the conversion from xinetd to systemd sockets is quite easy, but also cannot be done automatically. Basically, what we need to do is to use systemctl call (which we already support) to activate the socket that is provided by the package instead of our old approach, which is to write a file into /etc/xinetd.d/ and to reload xinetd. Such a change will make our life much easier than previously.

The hardest part is that xinetd configuration file also often contains the service configuration that YaST can and would like to modify. This is no longer possible with systemd sockets and it is also the reason that we cannot automate this conversion.

So how will we proceed with the conversion of the YaST modules? We first checked all modules that use xinetd and we verified that, for the majority, the conversion is straight-forward. YaST also has a module for inetd configuration, which no longer makes sense, so the plan is to drop it in the near future and to allow socket activation in services manager module.

A problematic module is ftp-server, which supports two backends: vsftp and pure-ftp. This is problematic because pure-ftp does not support systemd-sockets, only xinetd. We plan to discuss how to handle it, as we are also not happy that we support two backends, because it makes the life of our users harder when they just want to quickly configure a ftp server. For SLE users we already support only vstftp, so if we agree on it, we will probably also support only this server in YaST. But right now nothing is set in stone.

Memory eating strikes again

Once again we have to mention our battle with memory consumption. This time the problem happened within the NET iso for Tumbleweed, which should need no more than 1 GiB of memory during the installation process.

In the beginning of our research, we had no suspicious code to look at, we just knew that it could be related to the fact that the NET iso was using the whole repository of Tumbleweed (over 60k packages) while the DVD iso uses only a subset of packages.

So which technique did we use to find the problematic part of the code? We changed logging in order to append the memory status of the whole process to each log line. By using this approach we quickly identified the problematic part of the code. This problematic code was responsible only for logging, but as it was searching for all packages and loading all packages properties, it was consuming a huge amount of data and causing this memory issue.

As a hotfix, we removed this logging from packages and kept it only for patterns and products, which is more important for us and has low memory consumption. We also looked into all code that searches through all packages properties and we reduced it as much as we could. Finally, we forced the Ruby garbage collector to run before the processes of disk preparation and rpm installation, reducing the chances that the memory consumption of the installation process goes up.

Once again we had a happy end and we are now able to install the NET iso with 1 GiB of memory when using the graphical installer.

Concluding

As we’re getting closer to finish SLES 15, we’re getting busier and busier with features and bugs to finish. Therefore, YaST team is already working hard on Sprint 51 and we’re looking forward to come back to you in two weeks with much more cool stuffs that we’re doing. Meanwhile, have fun and stay tuned!

On the night of December 5th, teams of three are sprinting through the streets of Czech towns, visiting homes: Mikuláš (the developer), čert (the scrum master), and anděl (the product owner). If you’ve been a bad kid, you will get pieces of coal or brown potatoes. Good kids, on the other hand, will get new YaST features:

• Performance: adapting to jemalloc, a better Ruby memory allocator
• Replaced components: s/ntpd/chrony/g s/SUSEFirewall2/firewalld/g
• Storage: resizing partitions, resizing RAID
• Software modules and extensions: preselecting them, resolving dependencies automatically, creating your own installation ISO with mksusecd
• Security related: tarballs for reproducible builds, SSH keys for root
• AutoYaST: asking for a disk

Performance

Using Jemalloc Memory Allocator in Ruby

The MRI Ruby (the original C implementation) interpreter allows using the jemalloc library for allocating the memory instead of the usual glibc default. The advantage is better memory usage (30-40% less used memory was reported here) and it has some extra debugging features.

But after enabling this feature in Ruby in openSUSE Factory and SLE15 many YaST packages failed to build. It turned out to be an issue caused by dynamically loading the library by YaST. A workaround was to directly link the YaST against libjemalloc.

Later we found out that some packages still do not build. This time the problem was caused by a warning printed by the library on the error output. That happened only when running the old YaST test suite which still uses an embedded Ruby interpreter (normal YaST usage or the new unit tests use full Ruby which does not have this problem). The fix was just to ignore that warning when checking the error output in the tests.

If you want to check whether your Ruby uses the jemalloc feature then run this command:

ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']"

If the output includes the -ljemalloc option then your Ruby is using the jemalloc library.

Replaced Components

Switching NTP from ntpd to chrony

It was decided that it is a good time now to switch the Network Time Protocol (NTP) implementation from ntpd to chrony as it is more secure, nicer and better. So also YaST has to adapt and modify its NTP client module to work with chrony. The module also shows its age and it is a bit hairy, so we took the opportunity to do some cleaning.

The most visible change is the UI on the running system. Please note that the UI is not final yet and it is mostly just a proof of concept. But the goals are clear: The first goal is to be really client only, as previously NTP also allowed to configure an NTP server, making the UI complex and confusing for the majority of users that just want to select a server to sync against. The second goal is to focus on the majority of users who want a simple UI to configure a common NTP client, leaving fine tuning for experts who modify configuration files directly.

In the installer, chrony is now called instead of ntpdate. This change is invisible to users and it simply works the same as previously.

We also decided to drop the command-line interface (CLI) of the NTP client module as chrony itself comes which a nice CLI chronyc. Instead of duplicating the work our CLI just suggests to use chronyc.

Please also note that changes are not finished yet and more will come. One of such changes is AutoYaST support, which is currently just skipped. But let us show you some pictures to get an idea how it will look soon in Tumbleweed and explain our decisions along the way:

The first image is from the main screen. It shows three main areas:

1. The way the time is synchronized. It allows a pure manual run, synchronizing in a given interval or using the daemon that runs always.
2. Configuration of sources for synchronization servers. It can be dynamic or static. For static, only the ones in the servers table (see point 3) are used. For dynamic, it also adds NTP servers from the network (like ones from DHCP). So when the user uses only public servers, the static configuration works well. But when internal NTP servers are used and they can change, then using the dynamic option makes sense.
3. A list of synchronization servers. This specifies a statically defined list of NTP servers to sync against. You can use the familiar Add/Edit/Delete buttons.

The second image is from the Edit dialog. It was also simplified to the most common options. The Test button allows a quick check if a server is reachable and there are no typos in its address. The Quick Initial Sync option is useful to synchronize time quickly when connected to network. And Starting Offline is a nice chrony feature that allows to define the server as offline during the start-up and only when connected to the network to mark it as online and synchronize. It greatly helps with boot time and is a good option for notebooks where fast boot is important and the network is managed by Network Manager.

Firewalld will be the default firewall in SLE15 (Installer adapted)

Firewalld has some nice advantages over SuSEFirewall2: it can be dynamically managed, permits more zones, has a separation between runtime and permanent configuration options, it is well maintained, and it is already supported in some applications or libraries.

These are some of the main reasons why it has been decided to replace completely SuSEFirewall2 with firewalld.

In a past sprint report we announced that yast-firewall was adapted to support firewalld, but that was mainly an adaptation of the code to supporting both backends.

During this sprint we have completely replaced SuSEFirewall2 in the installer using a new firewalld API and although the UI seems to not have changed at all, the firewall dialogs have been completely reimplemented using CWM (our object-oriented API to YaST widgets).

During next sprints we will continue adapting the firewalld API and YaST modules to complete the replacement. The next target will be supporting firewalld in AutoYaST.

Storage

Expert Partitioner: Resizing

The Expert Partitioner has regained the ability to resize partitions. A Resize button is now available in the Hard Disks view as well as in the view of either a particular disk or a specific partition. When you try to resize a partition, a new dialog is shown with three options: maximum size, minimum size and custom size.

When the Custom Size option is selected, you have to specify manually the new partition size, but do not worry, the Expert Partitioner will prevent you from entering wrong sizes. Also, it will automatically align the partitions for a better performance.

Improvements in RAID management

As with partitions, it is now also possible to resize software RAIDs, although in the case of RAIDs resizing is a completely different operation performed by adding or removing devices to it. The partitioner will ensure you have selected the minimum amount of devices necessary for the current RAID type. Take into account that resizing a Software RAID will be only possible for new RAIDs being created, that is, resize action is not allowed when the RAID exists on disk.

As you can see in that screenshot, buttons for sorting the devices within a RAID have also been added at the right of the device list during this sprint. Those buttons are available both when creating a new RAID and when resizing it.

Last but not least, now it is also possible to work with BIOS RAIDs in the same way you do with regular disks. You will find all BIOS RAIDs in the Hard Disks section of the Expert Partitioner, where you could add or remove partition to the RAID device.

Software Modules and Extensions

Preselecting Recommended Modules or Extensions

The SLE15 products were split into several repositories and the installation medium contains only the minimal set of packages required to boot the system. If you want to have something more then you have to either register the system or use an additional medium.

To make that easier for users the registration module now pre-selects the recommended modules or extensions. However, if you for some reason really need a minimal system you can still deselect the pre-selected items.

The list of recommended addons is maintained at the server side (SCC) so it is possible to change that without updating the YaST installer.

Adding Modules and Extensions in AutoYaST

The list of available modules and extensions for SLE15 is already quite long as the base SLE15 products have been split into several modules. That makes adding the modules from the registration server more complicated compared to SLE12.

Originally the AutoYaST registered the addons in the XML profile exactly in the specified order. And that made troubles because SCC requires the dependent modules to be registered first. Writing the modules in the correct registration order in the profile was not easy for SLE15.

To make it easier AutoYaST now automatically reorders the modules according to their dependencies and even registers the dependent modules which are missing in the profile. Obviously that works well only if the missing module does not require a registration key.

mksusecd and linuxrc Supporting the New Media Layout

Starting with SLE15 the installation media have a slightly different layout. In short: the content file that was holding information about the product and a list of checksums is gone and has been replaced by .treeinfo and CHECKSUMS. Also, the package repository is now a repomd repository.

Also, SLE15 has been split into several so-called modules. The installation medium itself contains only a minimal set of packages. You have to register during the installation process to access other modules or use a separate medium containing them.

To save you from carrying several USB sticks around, mksusecd lets you build your own customized installation medium.

First, check what is on the media. The installation iso has just the base product:

mksusecd --list-repos SLE-15-Installer.iso
Repositories:
  SLES15 [15-0]

and you need an extra image for the modules:

# mksusecd --list-repos SLE-15-Packages.iso 
Repositories:
  Basesystem-Module [15-0]
  Desktop-Productivity-Module [15-0]
  Desktop-Applications-Module [15-0]
  Development-Tools-Module [15-0]
  HA-Module [15-0]
  HPC-Module [15-0]
  Legacy-Module [15-0]
  Public-Cloud-Module [15-0]
  SAP-Applications-Module [15-0]
  Server-Applications-Module [15-0]

Then pick the modules you want to add to the installation medium:

mksusecd --create new.iso \
  --include-repos Basesystem-Module,Legacy-Module \
  SLE-15-Installer.iso SLE-15-Packages.iso
Repositories:
  SLES15 [15-0]
  Basesystem-Module [15-0]
  Legacy-Module [15-0]
assuming repo-md sources
El-Torito legacy bootable (x86_64)
El-Torito UEFI bootable (x86_64)
building: 100%
calculating sha256...

Now use new.iso to start an installation.

Security / Development

Creating Reproducible Tarballs

We automatically submit the YaST packages from the Git repositories to the OBS projects using Jenkins. We use two Jenkins instances, the public Jenkins submits to openSUSE Factory/Tumbleweed, the internal Jenkins submits the same packages to SLE15.

The OBS compares the SLE15 packages with the openSUSE version to make sure the same packages are built in both projects. But this check compares only the file checksums, not the archive content.

The problem was that the internal and external Jenkins created a tarball with different checksum although the files inside were exactly same. If you run simple tar cfjv archive.tar.bz2 files you will find out that the tarball checksum is different for each run. So how to create a tarball in a reproducible way?

To build a reproducible tarball we use these tar options:

• --owner=root --group=root – to make sure the file owner and the group is always the same, it does not matter who builds the tarball or who owns the files on the system
• --mtime=<timestamp> – to have constant file time stamps, it does not matter when you downloaded the source files. Having fixed time stamps (like 0) is not a good idea as you do not know how the files are old actually, but using the current time makes the tarball not reproducible. So we use the date of the last commit in the Git repository (git show -s --format=%ci), it is stable across multiple runs and still describes how old the files are.
• --format=gnu – use the GNU tar format, the default POSIX format contains some archive time stamps
• --null --files-from - – this tells tar to read the file names from standard input, we use this options together with find ... -print0 | LC_ALL=C sort -z to sort the files to always have the very same file order.

Note: it also depends which compression method you use for compressing the tarball. For example gzip by default also adds some time stamps to the archive, bzip2 does not. If you want to use gzip then use the -n option.

See the tarball Rake task for the implementation details.

AutoYaST supports root SSH Authorized Keys deployment

From a security point of view, it is recommended to use SSH for logging into a remote machine instead of user and password in plain text format. But having to memorize the user and password specially if you try to use a strong non predictable password is also hard and hard to maintain.

For that reason SSH provides authentication based on public key cryptography and AutoYaST supports the deployment of SSH Authorized keys for normal users since SLE-12-SP2 but it was not working for the root user.

We have fixed this issue in SLES-12-SP3 allowing to deploy the SSH authorized keys also for the root user and exporting them into the cloned system profile.

AutoYaST keeps improving

The storage-ng based AutoYaST version has received a couple of improvements like, for instance, an improved resizing handling (supporting logical volumes) or multipath support. Additionally, we’ve squashed some bugs, like setting size=max for LVs (bsc#1070131) or some problems when not defining any partition (bsc#1070790 and bsc#1065061).

But even more interesting is the fact that we have brought back a hidden (and undocumented) feature. Did you know that if you set the <device/> element to ask, AutoYaST will ask you about which device to use for installation? Cool, right? This feature will continue working in (open)SUSE 15 and now it is documented!

Conclusion

My colleagues are telling me that practicing Scrum may be getting just ever so slightly on my nerves because I have several bugs in my description of the St. Nicholas tradition. Surely they are wrong. Stay tuned for the next report from a new and improved Scrum team: Ježíšek, Santa, und das Christkind.