openSUSE Lizards

Archive for the ‘Factory’ Category

Highlights of YaST Development Sprint 57

May 31st, 2018 by Yast Team

Three weeks from our last update on this blog. Time flies when you are busy! As you know, openSUSE Leap 15.0 was released in the meantime, which also means the active development of SLE15 is coming to an end… so time to look a little bit further into the future.

That’s why we had a face-to-face workshop with the whole YaST Team at the beautiful city of Prague during several days right before joining the openSUSE Conference 2018.

But we have done much more in three weeks than attending workshops and conferences. Apart from last-minute fixes, here you have a list of some interesting changes we have done in YaST in this period. Take into account that some of these changes didn’t make it into Leap 15.0, although all will be available in SLES15 and are probably already integrated into openSUSE Tumbleweed.

Fine tuning installer behavior in small disks

As you may know, the default installation of SLE and both openSUSE distribution enables Btrfs snapshots in the root partition alongside separate partitions for /home and swap. That means a default installation needs quite some space. In SLE12 and openSUSE Leap 42.X, if such disk space was not there the installer silently tries to disable the separate /home and even the snapshots in order to be able to create an initial proposal.

That behavior has become configurable for each product and role with Storage-ng and during the last sprint there was some controversy about what the configuration should be, both for openSUSE and the SLE family. It may look like a minor problem, but it becomes very relevant in virtualization environment (where virtual disks smaller than 10 GiB are not uncommon) or certain architectures with special storage devices like s390 and ARM.

The final decision was to never disable snapshots automatically in the case of openSUSE, so the user will be forced to manually go through the Guided Setup and explicitly disable snapshots to install in a small disk. In the SLE case, it was decided to keep the traditional behavior (automatically disabling snapshots if really needed) but making the situation more visible by adding a previous sentence to explain how the initial proposal was calculated.

So the installation in a normal disk would look like this.

While the installation in a very small disk displays some information similar to the following screen (the wording was slightly improved after taking the screenshots).

The explanatory text preceding the list of actions will be available in all products based on SLE15, but will not be there for Leap 15.0, since the modification to the installer was not ready on time for the deadline and, moreover, would have been impossible to get the translations on time.

By the way, if you are interested in a more in-depth explanation on how the partitioning proposal adapts to all kind of situations like small disks and other scenarios, don’t hesitate to check Iván’s presentation at openSUSE Conference 2018 detailing its internals.

More parameter passing for s390

And talking about uncommon scenarios and the s390 architecture, you may remember that in the latest sprint we improved the handling of the persistent network device names kernel parameter for such systems. Shortly after, we found out a similar improvement was needed also for the FIPS parameter.

FIPS is a military encryption standard in USA. If the installation is started using the corresponding parameter, YaST will enforce strong encryption and will install an specific FIPS pattern. Moreover, after the recent fix, a system installed in hardened mode s390 will continue operating in this mode after the installation.

Fun with MD RAIDs

As SLE15 comes closer, future users start testing the system with more exotic and complex hardware setups. Same applies to openSUSE Leap 15.0 right after the official release. As a result of all that testing, we found several scenarios in which Storage-ng got confused about MD RAIDs defined by some specific hardware or manually by the user before starting the installation.

By default, the old storage didn’t handle partitions within software RAIDs and it didn’t handle software RAIDs directly on top of full disks (with no partitions in the physical disks). For the first version of Storage-ng present in Leap 15.0 and SLE15, we tried to implement the same behavior with the intention to rethink the whole thing and open new possibilities in the close future. Check more about the present and future of Storage-ng in Ancor’s talk at openSUSE Conference 2018.

Unfortunately, while trying to replicate the old storage behavior with software-defined MD RAIDs, we overlooked some heuristic that was hidden in the old implementation to recognize some special setups in which a given RAID device currently detected as regular software-defined RAIDs should be treated like hardware RAIDs. That’s the case of Software RAID Virtual Disks defined on a S130/S140 controller on DellEMC PowerEdge Servers through the BIOS Interface. We also found that some users used to produce a similar situation by manually creating software MD RAIDs and creating partitions within them before starting the installation.

With the preparation of SLE15 already in the final stages and with openSUSE Leap 15 already out, it was too late to introduce drastic changes in how MD RAIDs are detected and used. To mitigate the problem while limiting the potential breakage, we reintroduced an ancient installer parameter. Now, when we run the installer using LIBSTORAGE_MDPART=1, all existing software-defined RAIDs will be considered as BIOS RAIDs.

The new parameter is not available in Leap 15.0 (we added it too late) and will hopefully not be necessary anymore in future versions of SLE and openSUSE, since the short term plan is to redesign everything about how MD RAIDs are handled during installation.

And even more fun with MD RAIDs

Another example of RAID that looks like defined by software but is indeed assembled by BIOS is the Intel RSTe technology. In this case, the usage of LIBSTORAGE_MDPART is not needed, but still we found the bootloader installation to be broken because YaST was once again getting confused by the mixed RAID setup.

Fortunately it was possible to fix the issue and verify the solution in only two days, despite the YaST Team not having direct access to the hardware, thanks to the outstanding help of the user reporting the bug. Connecting users and developers directly always produces great results… and that’s one of the reasons open source rocks so much!

Improved error reporting for wrong bootloader in AutoYaST

That was not the only improvement in the bootloader handling done during this sprint. We also invested some time improving the user experience in AutoYAST, since the error message displayed when using an EFI variant not supported in the system architecture was far from being useful or even informative.

So alongside a more clear message, AutoYaST will now list all the possible values supported on the given architecture to better guide the user.

Setting the default subvolume name in AutoYaST

AutoYaST also received improvements in other areas, like making use of the new possibilities offered by Storage-ng. The new storage layer allows the user to set different default subvolumes (or none at all) for every Btrfs file system. As shown in the example below, a prefix name can be specified for each partition using the subvolumes_prefix.

<partition>
  <mount>/</mount>
  <filesystem config:type="symbol">btrfs</filesystem>
  <size>max</size>
  <subvolumes_prefix>@</subvolumes_prefix>
</partition>

To omit the subvolume prefix, set the subvolumes_prefix tag:

<partition>
  <mount>/</mount>
  <filesystem config:type="symbol">btrfs</filesystem>
  <size>max</size>
  <subvolumes_prefix><![CDATA[]]></subvolumes_prefix>
</partition>

As a consequence of the new behaviour, the old btrfs_set_default_subvolume_name tag is not needed and, therefore, it is not supported in Leap 15.0 and SLE15.

Skipping Btrfs subvolume creation

And more changes in AutoYaST that arrived just in time for SLE15 and openSUSE Leap 15.0. Recently, we have introduced a new flag in AutoYaST partition sections to skip the creation of Btrfs subvolumes because, due to a known limitation of our XML parser, it is not possible to specify an empty list.

So from now on, setting create_subvolumes to false will prevent AutoYaST from creating any Btrfs subvolumes in a given partition.

<partition>
  <mount>/</mount>
  <filesystem config:type="symbol">btrfs</filesystem>
  <size>max</size>
  <create_subvolumes config:type="boolean">false</create_subvolumes>
</partition>

Keep it rolling!

As usual, the content of this post is just a small part of everything we did during the sprint. There were also many other fixes and improvements, from auto-repairing wrong partition tables (with different sizes than the underlying disk) during installation to better interaction with other components like udisk or mdadm auto-assembling and many other things in between.

But it’s time to go back to work and start implementing all the new ideas that emerged from the YaST Team Workshop and the openSUSE Conference. See you in the next report!

Posted in Distribution, Events, Factory, Miscellaneous, Programming, Systems Management, YaST | Comments Off on Highlights of YaST Development Sprint 57

Highlights of YaST Development Sprint 56

May 8th, 2018 by Yast Team

LEAP/SLE 15 is getting more stable and closer to be released, but to keep this process flowing, our team of bug killers is having a lot of work to do!

This last sprint we had several fixes for really special scenarios. The kind of problems that you can find once ~~most of~~ things are working fine! So let’s take a look at some of these cases and how we’re working to stabilize this upcoming release.

Keep predictable network devices settings on S390x

A not well-known feature in YaST is that many specific boot parameters for installation are also used on the target system. However, this approach has one exception: the S390 mainframe. In this case, many installation specific parameters should not go to the target system and, therefore, we ignore all installation parameter for this system.
In the last sprint, we worked on a bug, which reported that at least systemd predictable names for network devices should be also used for S390 systems, otherwise the configuration done during the installation won’t be valid in a running system as network names will differ. So, from SLE15 we start to keep the settings of predictable network names for S390 systems.

Fun with console configuration of GRUB2

Another report that helped us to learn about other not well-known features was the one reporting that bootloader module does not support multiple console outputs of GRUB2. After digging into some code, we found out that YaST bootloader takes only native terminal, gfxterm or serial console in consideration, but not a mix of them.
Once we looked at the manual of GRUB2, we learned that it supports some funny outputs such as morse code, PC beeper or simple data protocol using system speaker. Of course, YaST2 bootloader does not support all these options and when it gets one of them, it is treated as an unexpected value and bootloader fails.

As we are really close to Leap/SLE 15 release, we want to avoid big changes in the system. So we decided to handle this issue by showing a popup, which informs that the configuration contains an unexpected value and asks if the whole proposed configuration should be proposed again or if YaST should quit and let the user edit it manually.

If you are curious about how it looks like:

For Leap 15.1 / SLE 15 SP1 we plan to extend the values that we support to provide a nicer experience for the user.

Bootloader configuration during upgrade

Another reported issue in bootloader was also solved this last sprint. When upgrading from Leap 42/SLE 12 to Leap/SLE 15, if the user clicks on booting on the proposal, the system crashes. The reason is that usually on openSUSE 13.2/SLE 11 it needs to repropose bootloader during the upgrade. This is no longer required for the latest upgrade and, therefore, YaST does not expect that the user will click on it. We would like to remove this option completely, but YaST still support upgrade from SLE11 to SLE15, so we still need it there. In the end, the solution is to show a popup informing the user that the modification of bootloader is not supported during upgrade.

In short, take a look at the screenshot:

Fixing kdump on Xen

We got a report about kdump breaking when it is used in Xen. To explain the problem, we need to go back to how we configure a parameter and the reasons we implemented it in this way: In current versions (before this fix) when a user wants to use kdump, we configure the crashkernel kernel parameter for all targets, for the common kernel, Xen PV0 domain and also Xen guests. This approach worked very well in the past because traditional xenlinux ignores crashkernel for PV0 and just pass it to Xen VMs. However, the current pvops implementation of Xen no longer ignores this parameter and consequently, it results in breaking the Xen virtualization. The solution for this issue was pretty simple: YaST stopped to propose using crashkernel for Xen PV0 and everything works again. This is a perfect example to show how to understand the issue, sometimes, takes much more time than to fix it!

Improving the upgrade from SLE11/12 to SLE15

We are still improving the migration from SLE11 or SLE12 to SLE15 and this sprint we were focused on the automated registration upgrade using AutoYaST. If you are not familiar with the autoupgrade feature you can find more details in the documentation.

We already supported manual registration upgrade from the old products, but the automated way was still not adapted to the new SUSE Customer Center (SCC) API and it did not work correctly. This sprint we have adapted the registration upgrade code to correctly work in the interactive mode and also in the automatic upgrade.

The code was adapted to skip the user interaction and do everything manually when running in the autoupgrade mode. The only problematic part was how to handle multiple migration targets, which in the interactive upgrade we ask the user to choose one. To have a simple solution we decided to take the first migration, later (SP1) we might allow configuring this as well. But as now there is only one migration possible anyway, this looks like a good enough solution.

Falling back to the guided proposal

During this sprint, we were informed that AutoYaST was unable to display a proper error message when no partitioning section was specified and there was not enough disk space. The bug was rather easy to solve, but we wanted to take the opportunity to highlight how AutoYaST works when the partitioning section is missing from the profile.

In the past, AutoYaST implemented its own logic, different from the one used during a normal installation. Fortunately, as part of the adaptation to the new storage layer, AutoYaST relies now on the same code than the regular installation in order to propose a partitioning layout when the partitioning section is not specified. What is more, you can override some values which are defined in the product’s control file by setting them in the general/storage section from the AutoYaST profile.

Leap /SLE 15 is closed, but Tumbleweed is still rolling

We are really close to release Leap/SLE 15 and we are more focused on minimal changes that fix only critical stuff. On the other hand, Tumbleweed users are looking always for the latest and greatest features. In order to satisfy both groups, YaST separated Leap 15.1/SLE 15 and Tumbleweed in two different git branches. In this way, we can easily start adding new features, bug fixes and other improvements for Tumbleweed while we keep SLE15 stabilized. Besides that, there is another planned Service Pack for SLE12, and once we start to work at it, we’ll also create a new separated branch to include these changes. We also adapted all related infrastructure around the branches, such as CI, docker testing images, among others.

This way, we are able to allow you to enjoy the stable Leap 15 or the latest and hottest Tumbleweed.

Conclusion

We’re now working on our last sprint before the openSUSE Conference 2018. In two weeks we’ll come back with the highlights of Sprint 57 and until there we hope that you have already everything planned to enjoy the conference that will occur in Prague this year (we hope that you enjoy the city too). We’re looking forward to seeing you there!

Posted in Distribution, Factory, Localization, Systems Management, YaST | Comments Off on Highlights of YaST Development Sprint 56

Highlights of YaST Development Sprint 55

April 24th, 2018 by Yast Team

Time flies. We are almost in May and the openSUSE Conference ’18 is around the corner. So after booking your flights (if you need to) and your acommodation, you might want to know what happened in the YaST world during the Development Sprint 55th.

The YaST team is currently polishing the upcoming release, introducing some improvements and fixes. There are no breaking changes but still we have a lot of things to blog about.

Updating NFS Version Handling

Once upon a time, back in 2008 to be precise, the Large Hadron Collider (LHC) was finally ready, Raúl Castro replaced Fidel as President of Cuba, the TV show Phineas and Ferb was previewed… and yast2-nfs-client added support to configure NFSv4 mounts. Back then, the proper way of doing that was using “nfs4” as type for mounting the NFS share, i.e. writing “nfs4” in the vfstype column of the /etc/fstab file. Some time later, NFS4.1 (also known as pNFS) came out, and a new mount option “minorversion=1” was added. Very soon it was clear that such solution was not scaling and was not the way to go.

So at some point “nfs4” was deprecated as acceptable value for vfstype and “minorversion” was ditched in favor of “nfsvers”. Since the old deprecated way of doing things was still working, yast2-nfs-client was never updated to reflect this. But starting with the upcoming Leap 15 and SLE 15, some things will change in NFSland (in fact, the change landed in openSUSE Tumbleweed some time ago already). The type “nfs4” will be considered identical to “nfs” and “minorversion” will be completely ignored, so your old NFS mounts may not work as you expect them to do it. Time to refresh yast2-nfs-client!

During this sprint, yast2-nfs-client was not only fixed internally to produce valid entries in /etc/fstab, it also got a slightly revamped form to create and edit NFS mounts that should be less confusing than the old one and also more explanatory about how NFS versioning really works when defining a mount.

To ensure our users don’t get fooled by old entries that seems to be enforcing a particular NFS version (because they use “nfs4” as mount type, for example), but are in fact not doing it due to the new behavior in SLE 15 and openSUSE 15, yast2-nfs-client is now able to detect such circumstance, mark such entries in the list and offer a safe migration path to users.

As you can infer from the screenshot above, all these improvements are available when yast2-nfs-client runs standalone, as well as when it runs embedded within the YaST Partitioner. Enjoy!

Fixing Broken Translations

Recently we got some bug reports about YaST crashing at some points when running in some specific locales. It turned out that the problem was caused by broken translations.

A lot of translated texts contain placeholders like %s, %{text} or %1. These tags are replaced by the real values by YaST. But that requires that the translated text contains the same tags. If they are missing the value will not be included and, what is even worst, if they are invalid the Ruby interpreter throws an exception which means YaST aborts making our users unhappy. And that’s really bad, right?

Unfortunately the Ruby gettext does not support format tags and the GNU gettext does not support Ruby at all. As a quick solution we wrote a script which checks whether all tags are included in the translated text and reports broken translations.

The script found about 160 broken translations. The most common problems were usually just typos (s% instead of %s, {%foo} instead of %{foo}, or extra space in %␣1). But some cases were not that trivial. Translators by mistake also used the Unicode ٪ instead of the ASCII % or even translated the tags, which must stay untouched (%{مساعدة}).

Some translations were obviously wrong or even contained the original English texts – we removed them. In some cases the tags were wrong but we were not sure whether the whole translation is valid. In that case instead of fixing the tags we removed the translated text completely. It is better to ask the translators for translating again than have a completely invalid translation.

In the future we plan to improve these checks, so the tags are properly handled by Ruby and/or GNU gettext directly and we do not need a separate script for that.

Installing Over VNC Using the Browser

You are surely accustomed to remote administration using SSH. And, as you may know, the (open)SUSE installation can be done over SSH too. But, additionally, YaST also have support for installing over VNC.

When using VNC for installation, you can choose between using a native VNC viewer or a web browser based one. The cool thing about the second option, is that you can follow the installation just pointing your browser to http://IP-ADDRESS:5801.

Until now, YaST was using a Java applet based implementation, which is no longer supported in browsers. But during this sprint, we have completed the switch to a JavaScript based solution.

Unfortunately, that has resulted in losing an encryption layer: the HTTP connection on port 5801 is unencrypted, but the typical VNC port (5901) continues to be encrypted.

Asking Once About Equivalent Licenses

After splitting SUSE Linux Enterprise in several modules, it was pretty common that the user had to accept a couple of equivalent licenses during the installation process. Given that the content for those licenses was pretty much the same, it was quite confusing. Actually, we got a bug report about the installation process being stuck asking the user to accept the license over and over (it was just the same license being shown for different modules).

In order to make our users happy, YaST is now able to decide whether two licenses are the same and, in that case, it will only ask once for acceptance. For the time being, YaST applies a hash function to license contents and compare the result, but most likely this mechanism will be refined in the future.

License Confirmation in CaaSP 3.0

And talking about licenses, another small change about how they are handled was introduced in CaaSP 3.0. As you know, CaaSP features a One Dialog Installer and there was no room for the license to be shown. Now, before proceeding with the installation, YaST will show the license in the confirmation screen if needed.

Improving the `addon` Boot Option Handling

Back in February, we improved the addon boot option to handle the SUSE Linux Packages DVD properly. However, during testing, we found out that if you are using a system which only has one DVD drive, the installation DVD will be automatically used as an addon.

In order to fix this conflict, if the installation media and the Packages DVD are going to use the same drive, YaST will ask the user to change the DVD before using it as an addon.

Additionally, we improved the documentation of the addon boot option adding new examples to clarify how the dvd:/// URLs are handled.

Echoes of Winter: White Text on a White Background

These days we fixed a bug that only allowed clairvoyant users to finish the installation of openSUSE Kubic.

The bug is pretty unremarkable but may we draw your attention to the related CSS styling engine? It powers the high-contrast color mode that you can select with F3 or with Y2STYLE:

and if you press Ctrl-Alt-Shift-S (for style) you can change the styling on the fly, as in this example of changing the background color:

Conclusions

openSUSE Leap 15.0 release is approaching and, as usual, we need help from our dear users to give testing versions a try and report bugs. Thanks in advance!

Posted in Distribution, Factory, Localization, Systems Management, YaST | Comments Off on Highlights of YaST Development Sprint 55

Highlights of YaST Development Sprint 54

April 11th, 2018 by Yast Team

We were in the middle of ~~rewriting~~ no, ~~refactoring~~ recompiling all of YaST into Visual Basic when we found that it was April 2nd already and had to scratch the entire project. Next year for sure. So you are left with a report of enterprise grade stabilization and we hope that your servers will be very bored running our software.

Installation and Upgrade

Clearer Description of Migration Targets

Life goes through various roads and it is same for SLE life. SLE15 is now split into multiple modules and during the upgrade it can be quite complex to pick the desired upgrade target. We have to react to this issue as customers start complaining that the upgrade overview starts to be hard to understand and we should improve it. So we did it and now you can check the changes on the attached screenshots. We modified the overview label from listing all products to just a summary with the details displayed below as it was before. Be aware that in the future and for some products or extensions/modules more migration targets will be possible.

Old screenshot:

and the new one (for a slightly different system, so it is not an exact match for the previous screenshot):

Importing the SMT Server SSL Certificate at Upgrade

We are still improving and fixing bugs in the migration from the SLE11 or SLE12 products to the new SLE15 line. One issue we fixed this sprint was importing the SSL server certificate from the old system at upgrade.

For registration you can use a local SMT server (Subscription Management Tool) instead of the usual SCC server (SUSE Customer Center).

The SMT servers usually use a self-signed SSL certificate to save some money for buying a real certificate signed by a well-known certificate authority. This self-signed certificate is imported to the system by YaST during the initial registration so the registration process and the repositories from the server can be properly accessed.

But during the offline upgrade to SLE15 the old system is not running, the installer runs from the installation medium. In that case we need to import the SSL certificate from the old system to the installer so it can properly access the registration server and do the upgrade.

The certificate import is quite easy, we just need to be careful as SLE11 uses a different (old) path for storing the imported certificates than in SLE12 or SLE15.

As the result you should be now able not only to upgrade the systems registered against the SCC server but also the systems registered against your local SMT server.

Many System Roles

Various products that we’re able to install have grown so many groups of presets, called System Roles, that they no longer fit on the screen. We applied some dark gray magic to make them fit in a scrollable box, at the expense of losing the keyboard shortcuts, sorry.

Storage

Better Message for Multipath (and other) Problems

While scanning the storage hardware, or at a later stage while manipulating it, there is always a chance of finding problems in the system that make it very hard to continue with the installation or the execution of YaST. In that case, previous versions of storage-ng used to show you a pop-up message with some technical details about what went wrong (for example, the command that failed and its output) and with options to abort YaST or continue despite the error.

But we found that for some situations we could do better in trying to understand what went wrong and explain it to you, instead of directly showing those raw technical details. One clear example is finding the same LVM physical volume twice, something that should never happen. Apart from double vision problems (libstorage-ng doesn’t drink alcohol), the most likely cause is that a multipath system is not being correctly detected and thus every one of the connections to the disk is being detected as a different disk, duplicating the content in the eyes of YaST.

Now such a circumstance is detected and explained to you, advising to use LIBSTORAGE_MULTIPATH_AUTOSTART (see linuxrc documentation) or the corresponding entry in the AutoYaST profile if it has not been used. By the way, during this sprint we also instructed storage-ng about LIBSTORAGE_MULTIPATH_AUTOSTART, since it used to ignore that ancient libstorage modifier.

The technical details are still available under the "Details" button, as you can see below. They are simply not displayed at first sight, which should make the whole experience less daunting for less-experienced users. That change applies to all the severe errors found during the three critical phases of storage-ng: hardware activation, system probing, and commit (when the partitions and other devices are created).

Of course, the new pop-up messages have full support for AutoYaST. The most appropriate default option (continue or abort) is automatically selected depending on which one of the mentioned phases is being executed and, if AutoYaST is configured to display pop-ups, the usual countdown is displayed before doing such selection. See below the new generic error (for a different, unidentified problem) in action in AutoYaST.

AutoYaST is now Able to Reuse Encrypted Devices

As you may know, AutoYaST is quite flexible when it comes to partitioning, so we are still writing the final bits of the adaptation with the new storage layer. And this time, we were working on teaching AutoYaST how to reuse encrypted devices properly.

However, the implementation was not that straightforward, as the hardware probing occurs even before the partitioning section of the profile has been analyzed. And, in some scenarios, it is not clear which key should be used to unlock a device (for instance, this can happen when more than one encryption key is defined). To solve this problem, AutoYaST will try all defined keys on all encrypted devices until a working key is found.

Of course, this behavior is properly documented now in the AutoYaST handbook.

Miscellaneous

Fixed AutoYaST profiles validation issues.

In our previous blog entry we already mentioned that there are significant changes between SLE12 and SLE15 profiles which have been documented in this appendix.

It is very common to adapt the profiles by hand which is error-prone and sometimes it is also hard to identify where the errors are just running an installation and looking deeply into the logs. That is why profiles validation using xmllint or jing is recommended (more info here).

During this sprint we have fixed some errors with the cloned profiles after installation which were not validating.

Translation Issues

We are receiving quite a lot of bugs regarding the translations. The usual problem is that some text is not translated at all and the original English text is displayed. This sprint we fixed several issues in this area, two of them are worth sharing in the blog.

The XSL File Format

The first problem was reported for missing translations in the role descriptions in the SLES4SAP product. The SLES4SAP installation basically behaves like the standard SLES installation just with changed few defaults. To avoid the duplication and make the SLES4SAP maintenance easier we simply take the original SLES XML control file, which describes the installer behavior and the defaults, and change just few values using a XSL transformation into the resulting SLES4SAP installer control file.

It turned out that the roles with missing translations were located in that XSL file. And unfortunately YaST did not support extracting the translatable strings from XSL files. However, we support translations in XML files and because a XSL file is actually a valid XML file we could easily extend the translation support in YaST to also cover the XSL files. So now the SLES4SAP roles are correctly translated.

Missing `textdomain` Call

We fixed several bugs with missing translations which were caused by missing textdomain call in the code. This call defines which POT file should be loaded and searched for the translations. If the YaST code does not use this call then obviously no text can be translated as YaST does not know which POT file should be used and it silently used the original untranslated text.

That means it was quite difficult to find why some text was not translated. And because that was quite common bug we had a nice idea to improve the situation by logging a warning into the YaST log with the exact message which could not be translated. And more importantly the log now also contains the location of the code which was trying to use the translation wrongly. See the pull request for more details.

With the openQA team we discussed also the possibility to add a new check to openQA which would scan the YaST log for this particular warning and report a problem. Which means we should not overlook this quite important warning in the future.

Posted in Factory, Systems Management, YaST | 1 Comment »

Highlights of YaST Development Sprint 53

March 23rd, 2018 by Yast Team

As the release dates for SUSE Linux Enterprise 15 and openSUSE Leap 15 approach, we keep adapting YaST to make easier for our users to take advantage of all the new features that these rock-solid operating systems will bring.

During the last two weeks that has implied, apart from regular bug fixing that we usually don’t cover here, working on AutoYaST, improving Storage-ng and polishing several aspects related to modules and extensions, like their registration and licenses.

Let’s start with the rewritten Partitioner that is part of yast2-storage-ng.

Partitioner: more flexibility with the partition id

Setting the right partition id (also known as partition type) for each partition is an important part of the system setup that is often overlooked. Our Partitioner has always displayed in a prominent place the widget allowing to set that id, suggesting always the best value based on the selected role and the chosen file system type. But in many cases, that was more than a simple suggestion. In the old Partitioner (and in the new one until this sprint) the value of the partition id field (Linux, swap, Linux LVM, etc.) could only be manually edited in case the user had selected to not format the partition. When the option “Format device” was selected, the automatically chosen value could not be changed.

In SLE15 and openSUSE Leap 15 (and quite soon in openSUSE Tumbleweed), it will be possible to modify the id, no matter if the partition is going to be formatted or not. Of course, the logic to propose the best option every time the user selects a file system type is still there, but now it can be always overridden if the user wish. That change resulted in a small rearrangement of the widgets in that screen, as you can see below (remember we are trying to be very conservative with the UI changes in the Partitioner).

Partitioner: better support for DASD

In our previous report we explained some of the aspects in which the Direct-access storage devices (DASD) used in s390 mainframes are different from regular hard disks. But as you can imagine, there are more differences… and we know our readers love to learn new stuff while enjoying our reports. 😉

In short, there are two possible kinds of DASDs devices: Extended Count Key Data (ECKD) and Fixed Block Architecture (FBA). As explained in the previous report, the ECKD devices need to be formatted at low-level in order to be used by the operating system and, moreover, there are two possible low-formats for them: Compatible Disk Layout (CDL) and Linux Disk Layout (LDL).

And now the fun – ECKD devices formatted as LDL do NOT have a partition table. FBA devices can potentially have one, but it’s also often skipped. To manage those DASDs without partition table, the Linux kernel simulates an implicit single partition taking the whole disk. Of course, working with such implicit partitions implies some restrictions, and we have introduced several controls to make sure things stay under control in the storage-ng Partitioner. For example, an error message is now shown if the user tries to remove an implicit partition.

For curious readers, there is more information about DASD available in this link.

Partitioner: can’t resize a partition… but why?

In SLE15 and openSUSE Leap 15 we will report very detailed reasons why a partition or a file system cannot be resized, as you can see in this screenshot.

This used to be just a very simplistic message “Device cannot be resized”. But there may be many reasons for that, and sometimes different restrictions might contradict each other: While some type of file system only lets you grow, not shrink (e.g. XFS), the partition that the file system is on might not be able to grow, for example because there is another partition right next to it. We want to minimize user frustration that might happen when we only report the first reason, and when the user somehow managed to fix that problem, show another one that can’t be fixed.

As usual, this feature will be available in Tumbleweed in a matter of days.

Handling registration rollback in SLE15 Migration

Of course, the Partitioner was not the only YaST area to get attention during this sprint. Several aspects related to products, modules and extensions were also worked, with all the implications they have about registration, migration and licenses.

For the offline migration to SLE15 we reused some parts from the online migration which handles service pack upgrade. But it tuned out that the reused part was not correctly integrated into the installer and in some corner cases (registration errors) it did not behave correctly.

Moreover if the upgrade failed early then the system still contained a SLE12 installation but was registered as a SLE15 system on the SCC server. After booting the original SLE12 system the access to the online repositories was broken.

This sprint we fixed that so in case of registration error or when going back the original registration is restored. Now you can go back and choose a different system to upgrade and it will work as expected.

Additionally we fixed some small issues with custom repositories (add-on or driver updates) used at upgrade.

More fun with hiding/showing beta versions in SLE15

Usual readers of our blog already know that SUSE is taking extensions and modules to a whole new level in SLE15, making them a cornerstone of the system installation and upgrade process. As already explained in previous posts, that implies more complex dependencies between extensions and modules. All those mechanisms usually work nice… except a small problem we found out with beta versions.

If a given extension was in beta phase and some of its dependencies were also in beta, if the “Hide Beta Versions” checkbox was unchecked the system was displaying only the extension selected by the user, but not the auto-selected dependent beta extensions. Our SLE testers found that quite confusing. So to make everyone’s life easier, we fixed the behavior as shown in the following screenshot.

A look into the future: analyzing how we display licenses

Currently there are many different ways to handle and display licenses. That can happen during the installation or upgrade process, while adding additional products to an installed system and, last but not least, while using YaST2 Firstboot to perform additional installation steps on the first system execution.

Additional there are 3 different locations from which these licenses come from. They can be provided by the SUSE Customer Center, be provided by libzypp or come from a repository using a legacy approach.

To simplify and unify all that in a close future, the first step was researching all those possibilities and how they are handled in (Auto)YaST. The result of such research can be found in this document hosted on Github.

AutoYaST product selection and installer update improvements

As you probably already know, starting with SLE-15, all products are distributed using one medium and you need to choose explicitly which product to install. Of course, if the medium only contains one product that would not be needed.

In AutoYaST profile the product is selected using the /software/products/product XML node:

<software>
  <products config:type="list">
    <product>SLED</product>
  </products>
</software>

Due to a bug, the cloned system exported the product short_name instead of the name, resulting in an internal error reported by the installer update and a later error during the auto-installation which aborted it because no product was selected.

So, during this sprint we have made improvements for both scenarios.

The installer update will not rely in the product selection at all (the installer is the same for all the products) but will use the self_update_id from the control file and the version and architecture from the first product available on the media. The installer update documentation has been also updated according the last changes and it is probably the best place for knowing more about its behavior.
The wrong product selection error reported was not very useful and it was decided to provide more information about the list of available products from the media. Just see the image below with the latest implementation:

Document main differences in AutoYaST profiles between SLE12 and SLE15

The need to select a product is not the only relevant change affecting AutoYaST profiles for SLE15. There are many other significant changes in SLE15 compared to SLE12. Like the new modules concept, replacing SuSEfirewall2 with firewalld, replacing ntp with Chrony… Users wanting to reuse existing SLE12 profiles with SLE-15, will probably need to adjust them.

We have created this summary describing some of the most important changes in order to help with the conversion.

That document is just a preliminary and temporary work that is currently being reviewed and improved by the awesome documentation team at SUSE. Very soon (probably already done at the time you are reading this) the content will be merged and a new section titled “Main differences between SLES 12 and 15 profiles” will be available in the current guide for AutoYaST. Have we ever mentioned how much the doc team rocks? So please, use that last link as final reference instead of our temporary summary.

Cron config for NTP client

It is possible to setup the YaST-ntp-client module to sync the system clock at regular intervals. If that feature is used, YaST writes the needed configuration to a cron.d config file. We were still using “novell” as part of the name of such file, which was reported as a bug. It turned to be a good opportunity to take a look to a module that, as you can guess from that bug, we don’t update very often. 😉

First of all, we made sure that newly written files will have a more up-to-date name. Straightforward and easy.

The second part was to provide an upgrade path if the file already existed. We integrated that with the existing ntp to chrony conversion. That means the existing configuration is updated when a new version of the yast2-ntp-client package is installed, so the user does not need to run the module again to start using chrony with an existing configuration.

Last but not least, the third part was to adapt the package to be a better citizen in the RPM world, marking that file as ghost file in RPM spec. Now this command can recognize that yast2-ntp-client is responsible for that configuration file.

  rpm -qf <file>

Two months… and counting

Only two months of countdown until the release date of openSUSE Leap 15! That means a lot of hard work ahead of us, so stay tuned for more updates.

Posted in Distribution, Factory, Miscellaneous, Programming, Systems Management, YaST | Comments Off on Highlights of YaST Development Sprint 53

Highlights of YaST Development Sprint 49

January 25th, 2018 by Yast Team

Time goes by and the YaST wheel keeps rolling. So let’s take a look to what have moved since our previous development report.

More flexible NET installation ISOs

Network installation media for Tumbleweed or Leap only work properly with the exact repository they have been built for – which for Tumbleweed may mean they could be outdated after just one day.

You would then run into this message:

To improve the situation the installer can now offer to download matching boot files (kernel and initrd, to be precise) from the repository if it detects this situation:

Of course, you can say ‘No’ here – but then you’re back to the red dialog. 😀

Technically, what’s done is to download a new kernel/initrd pair from the repository and restart the installation process with them (using kexec). So be prepared for a slight déjà vu.

This feature is controlled by the kexec boot option.

Storage-ng lands into Tumbleweed: handle with care

But that’s not the only news we have about openSUSE Tumbleweed. Our usual readers already know about Storage-ng, our effort to rewrite the whole YaST storage stack from scratch. And they also know it’s still a work in progress. But since there were too many valuable changes blocked by the adoption of Storage-ng, it was decided it was time to push the red button. So we are glad to announce the Storage-ng era has started with its inclusion in the first official (open)SUSE product – starting with snapshot 20180117, libstorage-ng has replaced libstorage and, thus, yast2-storage-ng has replaced yast2-storage.

They say forewarned is forearmed, so an article was published in advance in news.opensuse.org to set the expectations and to provide and overview of the current status. We would like to encourage all openSUSE Tumbleweed users to (re)visit the article to get a better picture of the situation.

Alignment of partitions in the expert partitioner

An important part of that work in progress is the re-implementation of the Expert Partitioner with Storage-ng technologies. As mentioned many times in previous posts, this is mainly a 1:1 clone, with the same functionality presented in exactly the same way than the classic YaST partitioner. But some times we take the opportunity to introduce some improvement here and there, as we did this week with a topic that can have a very noticeable impact in the system performance: partitions alignment.

Although many people is not aware of it, the partitions in a system must be properly aligned to avoid the performance drop caused by excessive read-modify-write cycles. For details please refer to the great article at Wikipedia explaining the topic, especially the sections titled “4 KB sector alignment” and “SSD page partition alignment”. Moreover, leaving performance considerations aside, some partition tables require alignment to simply work, like DASD partition tables which need alignment to tracks (usually 12 sectors).

The new expert partitioner takes all that into consideration when creating and resizing partitions, ensuring always the required alignment (like the DASD tracks) and encouraging the optional performance-related one, avoiding undesired gaps between partitions in the process.

Above you can see the dialog for choosing the size for a new partition that, unsurprisingly, looks very much like the same dialog in the pre-storage-ng Expert Partitioner. If a size is specified by the user in that dialog (any of the two first options in the form), the start and end of the partition will be aligned to ensure optimal performance and to minimize gaps. That may result in a slightly smaller partition (with the difference being usually less than 1MiB). If a custom region is specified, the start and end will be honored as closely as possible, with no performance optimizations (although mandatory alignment, like DASD tracks, still will take place). This third option is the best to create very small partitions.

The same considerations for optimal alignment will also be taken into account while resizing an existing partition and calculating the minimal and maximal sizes suggested by the partitioner during that process.

Sanity checks for the storage setup

The possibility of bypassing the performance optimizations in the Expert Partitioner is just one example of the (potentially unleashed) power that tool provides. As a consequence of that flexibility, sometimes the user can overlook some important setup configurations or even make mistakes. To help with that, the Expert Partitioner recovered this week its ability to check the entered storage setup.

Once the user has set partitions, LVM volumes, file systems, mount points, etc. and decides to proceed, the Partitioner will validate that setup to ensure it fulfills all necessary requirements for booting and running the system. When some issue is detected, a popup message is presented to show what the problem is, offering the option to ignore the warning and move forward.

Two kind of checks are carried out to ensure the partitioning setup validity. First, the presence of needed partitions for booting is checked. Booting requirements depends on the current architecture (x86, PowerPC, AArch, etc.) and other technical details like the partition table type (GPT vs MS-DOS). Then, the mandatory volumes for the current product are checked. The mandatory volumes are defined in the revamped partitioning section of the control file. Typically, only a volume for root and another for swap used to be mandatory, but now this is totally configurable by anyone defining the product (SLE, Leap, Tumbleweed, your own custom openSUSE derivative…).

As a bonus, all the sanity checks are now centralized (they used to be scattered around the YaST source code) and it’s easier to add new ones (you will miss some old checks at this moment) and to use them from other parts of YaST (like the bootloader module or AutoYaST).

More improvements in the Expert Partitioner

The new warnings and the alignment improvements commented above are not the only news on the evolution of the Expert Partitioner clone this week. Resizing of LVM devices has also been brought back to life, both for volume groups and logical volumes. In the case of logical volumes, the functionality is not much different, at least in the surface, from the partition resizing that was already present and that you can see in the screenshot of the alignment section.

On the other hand, in the context of the Partitioner, resizing a volume group actually means adding or removing physical volumes. Actions that are now possible again, including the corresponding checks. For example, a physical volume cannot be removed if it already exists on disk (that could destroy your data) or if the resulting size of the volume group is not enough to cover all its logical volumes.

Apart from the mentioned functionality, there has also been improvements in how the Expert Partitioner presents the information. For example, now the “type” column shows the correct label and icon for each device instead of that useless TODO label. Moreover, similar TODO marks were replaced by proper data in the device overview tab.

Minimize changes between the SLE15 “Installer” and “Packages” DVDs

The SUSE Enterprise Server 15 (SLES15) product can be installed from a bootable “Installer” DVD medium which contains the installer and a subset of packages needed for a very minimal system. The other packages are available either from a registration server (after registering the SLES product) or via a separate “Packages” DVD medium.

Due to the structure of those DVDs (with some packages being in present in both) the SLES installer was asking the user to change the medium several times during the installation process. Ideally the installer should use all packages from the “Packages” medium without changing the media.

In addition, there is yet another requirement for preferring the packages from the installation DVD to the packages available via a remote repository. Downloading a package from the internet is usually much slower than the DVD and can be problematic in network connections with a download limit or with a price based on the bandwidth usage.

Now the installer properly adjust the priority of all the repositories to achieve the desired behavior. To avoid possible side effects we decided to change the repository priority only when more than one repository is used and all repositories are local (e.g. DVD, hard disk, USB flash disk…). That means in some less common cases (2 DVDs + a remote repository) you will still need to change the medium but this is a safer solution.

Add On products in AutoYaST

For those using SLE Add On products, we have improved the error message if an Add On Product cannot be added during an AutoYaST installation. The user can see now which wrongly configured Add On Product has produced the error.

This will be specially useful with the upcoming SLE15, in which the concepts of Add Ons and Modules will become more relevant than ever.

Fixed a crash when shutting down the YaST user interface

And now it’s time for the corresponding dose of technical insights for those who enjoy that part of our reports.

When UI::OpenDialog() and UI::CloseDialog() calls didn’t match when shutting down the UI (user interface YaST component), you’d get a segmentation fault with a core dump. Well, you did want to shut down YaST, but probably not like that. This is now fixed.

After tracking this down, it was surprisingly simple to reproduce: Just use the YaST version of the trivial “Hello, World” program and comment out the UI::CloseDialog() call.

This was a case of providing additional error reporting causing more problems than the original error: leaving dialogs open while terminating the program is an error, of course. But fixing this little problem by cleaning up the remaining dialogs lead to handling widgets after some of the underlying infrastructure (in this case the QApplication) was already destroyed, so all the QWidgets were also destroyed (because the QApplication takes care of that), but YaST’s generic UI layer was still unaware of that fact and tried to destroy them again.

This is now fixed by properly cleaning up the widget tree in YaST’s generic UI layer first which will also clean up the associated QWidgets so there is nothing left to clean up for the QApplication.

This might also fix a number of similar segfaults in other situations where the YaST Ruby engine would need to shut down because of other problems, e.g. when there is an unhandled Ruby exception.

Surprisingly enough, this must have been a very old (10+ years?) bug, but it never became quite obvious, or at least nobody was ever annoyed enough to try to track it down.

If you want even more details, check the conversation in the bug report.

More to come

The end of this sprint caught up with a lot of almost finished stuff. But following the Scrum principle of “nothing is done until it fits the Definition of Done”, we don’t blog about such stuff. Fortunately, that means the next report will likely be quite juicy. So, see you again in a couple of weeks!

Posted in Distribution, Factory, Programming, Software Management, Systems Management, YaST | 1 Comment »

Encrypted installation media

November 17th, 2017 by Steffen Winterfeldt

Hackweek project: create encrypted installation media

You’re still carrying around your precious autoyast config files on an unencrypted usb stick?
You have a customized installation disk that could reveal lots of personal details?
You use ad blockers, private browser tabs, or even tor but still carry around your install or rescue disk unencrypted for everyone to see?
You have your personal files and an openSUSE installation tree on the same partition just because you are lazy and can’t be bothered to tidy things up?
A simple Linux install stick is just not geekish enough for you?

Not any longer!

mksusecd can now (well, once this pull request has been merged) create fully encrypted installation media (both UEFI and legacy BIOS bootable).

Everything (but the plain grub) is on a LUKS-encrypted partition. If you’re creating a customized boot image and add sensitive data via --boot or add an add-on repo or autoyast config or some secret driver update – this is all safe now!

You can get the latest mksusecd-1.54 already here to try it out! (Or visit software.opensuse.org and look for (at least) version 1.54 under ‘Show other versions’.

It’s as easy as

mksusecd --create crypto.img --crypto --password=xxx some_tumbleweed.iso

And then dd the image to your usb stick.

But if your Tumbleweed or SLE/Leap 15 install media are a bit old (well, as of now they are) check the ‘Crypto notes’ in mksusecd --help first! – You will need to add two extra options.

This is how the first screen looks then

Tags: booting, Hackweek, openSUSE, Security
Posted in Distribution, Factory, Hackweek, Security, Uncategorized | Comments Off on Encrypted installation media

Highlights of YaST Development Sprint 46

November 10th, 2017 by Yast Team

It’s Hack Week time at SUSE! But before we dive into all kind of ~~crazy~~innovative experiments, let’s take a look to what we achieved during the latest development sprint.

User-friendly error messages in AutoYaST

During recent weeks, the AutoYaST version for the upcoming SLE 15 family has received quite some love regarding the integration with the new storage layer, from fixing bugs to adding some missing (and even some new) features. So let’s have a look at what we have done so far.

First of all, a new error reporting mechanism will debut in the upcoming AutoYaST version. Until now, when a problem occurred during partitioning, you got a message like “Error while configuring partitions. Try again.“. It does not help at all, right? At that point, you were on your own to find out the problem.

Now AutoYaST is able to identify and report different problems to the user in a convenient way. What is more, in many situations it is even able to point to the offending section of the AutoYaST profile.

The error reporting mechanism can distinguish between two different kind of issues: warnings and errors. When a warning is detected, a message is shown to the user but the installation will not be stopped (it honors the settings in the <reporting> section). Errors, on the other hand, will block the installation entirely.

Please, bear in mind that this error reporting mechanism is only available for the <partitioning> section. Maybe it could be extended in the future to cover other parts of the auto-installation process.

Bringing back skip lists to AutoYaST partitioning

When defining a partitioning schema, you can let AutoYaST decide which device should be used for installation. Thanks to that, you can use the same profile to install machines with, for instance, different storage devices kernel names (like /dev/sda and /dev/vda).

Needless to say that, in such a situation, we might want to influence the decision process. For example, we would like to avoid considering USB devices for installation. AutoYaST offers a feature known as skip lists which allow the user to filter out devices using properties like name, driver, size, etc.

Unfortunately, skip lists support in SLE 15 Beta1 is rather limited. But these days we have extended yast2-storage-ng to offer additional hardware information and now AutoYaST is able to use it to filter devices.

As a side effect, the ayast_probe client has been fixed to show (again) which keys you can use in your skip lists.

More on AutoYaST

Apart of adding or bringing back features, we have fixed several bugs. You can check the recent entries in the yast2-storage-ng changes file if you are interested in the details.

We know that a few features are still missing and more bugs should be addressed sooner or later, but hopefully AutoYaST must work in most use cases.

SLE15 media based upgrade for unregistered system

This sprint we also continued implementing the upgrade from SUSE Linux Enterprise (SLE) 12 products to the version 15. Particularly we solved the upgrade of unregistered systems.

In that case you need the “SLE15 Installer” medium and additionally also the “SLE 15 Packages” medium. The installer medium contains only the minimal packages for installing just a very minimal system. The rest is available either via the registration server or via the extra medium. Obviously for unregistered systems only the second option makes sense.

In this sprint we were focused on making all pieces to work together. You can see the result in the following screencast.

Upgrading an unregistered system

Fixed an installer crash in systems with 512MB RAM

We got a bug report that the beta version of the upcoming SUSE Linux Enterprise Server 15 was sometimes crashing during installation on a system with 512MB RAM. That’s bad, the 512MB is a required limit which should be enough to install a minimal system in text mode.

At first we thought that the crash was caused by insufficient memory, but the reported memory usage was OK, there was still enough free memory.

It turned out that the problem was in the pkg-bindings which tried to evaluate undefined callback function. The fix was quite simple, however, the question was why that happened only in systems with 512MB RAM and not when there was more memory.

Later we found out that the difference was caused by the extra inst-sys cleanup (mentioned in the Sprint 22 report) which YaST runs when there is low memory. In that case YaST removed the libzypp raw repository metadata cache. The assumption was that when the data is already parsed and cached in the binary solv cache the original files are not needed anymore. However, libzypp still might use some raw files later.

So we changed the inst-sys cleanup algorithm to remove only the files which we know are not needed later and keep the rest untouched.

Expert partitioner: the some boys are back in town

Several features have been brought back to the expert partitioner during this sprint.

Allow to create and delete logical volumes.
Allow to delete MD RAIDs.
Allow to work with multipath devices.

Now you can create logical volumes using the expert partitioner. When you go to the LVM overview or visit a specific volume group, a button for adding a logical volume is available. Clicking on it, you will be taken through a wizard for the creation of a logical volume. Note that although the logical volume type can be selected in the first wizard step, only normal volumes can be created. Thin logical volumes and thin pools will come soon. And apart of creating logical volumes, now there is also a button for deleting them.

Delete action has been also implemented for MD RAIDs. For that reason, you have a delete button in the RAID overview and also when you access to a specific MD RAID. And of course, you will be asked for confirmation before removing the device.

Additionally, another important feature recovered during this sprint is the possibility to work with multipath devices. Now, multipaths are listed together with other disks in the tree view of the expert partitioner, allowing you to manage them as regular disks. For example, you can create or remove partitions over them. Moreover, when a multipath device is selected, a new tab is showed to list the so-called “wires” that belong to the multipath.

Improving the product upgrade workflow

Although the possibility to offer an upgrade option from openSUSE Leap to SLE is on both SUSE and openSUSE radars for the future, the reality is that it has been, and still is, an unsupported scenario.

But with previous versions of SUSE Linux Enterprise, you could take a SLES DVD, boot it in the Upgrade mode, and select to upgrade an openSUSE partition. YaST would let you proceed several screens before telling you that it actually will not let you upgrade from openSUSE to SLES.

Starting with recent SLE15 pre-releases, the incompatible products are filtered out in the partition selector already (overridable with a Show All Partitions checkbox), letting you know earlier whether you will be able to upgrade your system to the new SLES.

Fix of a registration issue during installation process

In SLE 15 Installer, there is a product selection dialog at the very beginning of the installation. After that, you can register the selected product but you cannot change the product later as unregistering the product and registering another one is not supported. Our awesome QA squad found out that when the installation was aborted and then started again from Linuxrc without rebooting, the installer thought that the product had been already selected and did not offer any product for installation. A little fix made it work again – now we always execute the following SUSEConnect command at the start of the installer.

SUSEConnect --cleanup

That removes all traces of previous registration attempts from the Installer. This also means that you might still want to unregister directly at SUSE Customer Center if needed.

Improving help texts in the registration process

As you have seen so far, we have been working hard to polish the registration experience in many aspects and scenarios. That also includes a better communication with the user. Thus, the help text in the registration module has been improved to also include the description of the check box states. This is especially important for the “auto selected” state which is specific to this dialog and is not used anywhere else.

The help texts in YaST use an HTML subset which allows also including images. In this case we included the check box images directly from the UI stylesheet. But in the text mode we have to use text replacement instead of the images. That means the help text content must be created dynamically depending on the current UI.

Here you can see examples of both interfaces.

Twisting the storage proposal: this time for real

In our report of sprint 42 (to be precise, in the section titled “Twisting the storage proposal”) we presented our plans to make the software proposal more customizable in a per-product basis and the draft document of the new format for control.xml that would allow release managers to define the installer behavior in that regard.

Now this goes further than a mere specification and the new format is actually being used to define the partitioning proposal of both the KVM/Xen role of SUSE Linux Enterprise 15 and the upcoming SLE15-based CaaSP.

In the following screenshot you can see the corresponding step of the guided setup for the mentioned KVM/Xen role, in which the classical controls for the /home and Swap partitions have been replaced by more goal-specific volumes defined in the section of the control file describing the role.

And, as you can see below, the installer now honors those settings to propose a reasonable partition layout.

The new format and the corresponding implementation of both the logic and the UI are flexible enough to empower the release managers to define all kind of products and to make possible for everybody to create a more customized derivative of openSUSE without renouncing to the power of the automatic proposal. See another example below (not corresponding to any product or derivative planned in the short term) with more possibilities and note how the wording was automatically adapted to talk about LVM volumes instead of partitions, based on the user choice in a previous step.

Replacing ntpd with Chrony in yast2 ntp client

Chrony will replace the classical ntpd as default NTP client starting with SLE15 and openSUSE Leap 15. That will offer several advantages to system administrators and other users, as can be seen in this comparison. In order to make this replacement possible, we started a research to find out what is supported in Chrony and how to allow our users to configure it through YaST.

The research phase is now complete and we have already a plan to proceed with the adaptation of the existing yast2-ntp-client module. Also a few bits of code, which allows to set the NTP service during installation, are now in a feature branch (so not yet in Tumbleweed).

The next step will be a huge improvement (and simplification) of the YaST module, which will go further than adapting a list of options. In the screenshot below you can see the not yet finished prototype in action.

Configuring the keyboard in the installer via systemd

Originally the keyboard configuration was written directly by YaST in the corresponding Systemd-related configuration files. But we got a bug report that YaST should not touch the config directly and rather call the localectl tool for changing it. (See the details in the localectl man page).

However, this works only in the installed system, it does not work in the system installation as it needs a running Systemd that is not available during the installation process. Changing the setting for a not running system must be done using the systemd-firstboot command.

But this did not supported modifying the keyboard settings. Fortunately one of the SUSE developer helped us and implemented this feature to Systemd (pull request). Currently the feature is available in (open)SUSE packages but later it will be available in the upstream release for others.

Another related change was that YaST not only set the console keyboard but also constructed the keyboard settings for X11 (GUI). But this is actually a duplicated functionality, localectl itself includes this feature. So we have removed it from YaST and let the localectl tool to set both keyboard setting automatically.

And now for something completely different

Hack Week 0x10 (that is, the 16th edition) is starting just right now. Which means most developers of the YaST team will spend a week working on topics that may or may not have a direct and visible impact in our beloved users in the short term. But hey, maybe we will build a robot or a space rocket!

After that week, we will restart our Scrum activity. So if nothing goes wrong, you will have another update about the YaST development in approximately four weeks. Meanwhile, join us at Hack Week and let’s have a lot of fun together!

Posted in Distribution, Factory, Hackweek, Programming, Systems Management, YaST | Comments Off on Highlights of YaST Development Sprint 46

Highlights of YaST development sprint 41

August 24th, 2017 by Yast Team

We all know that everything slows down in summertime and software development is not an exception. But heat is not enough to stop the YaST team from turning the Scrum wheel and delivering the corresponding sprint reports. Let’s take a look to what we have been doing the last two weeks.

The storage reimplementation gets on the launchpad

As already anticipated in the previous report, one of the goals of this sprint was to merge the new storage stack into the codebase of SUSE Linux Enterprise 15 and openSUSE Leap 15. That implies submitting everything to Factory first and making sure the result looks harmless and good enough there. Thanks to the awesome openSUSE tools and processes, that kind of experiments can be isolated in a dedicated staging project allowing us to reach useful conclusions without risking the stability and features of Tumbleweed.

So we submitted two new source packages libstorage-ng and yast2-storage-ng to Factory, together with new versions of all the affected packages (already adapted to use the new system, instead of the old yast2-storage) and a modified version of the list of packages to be used during installation.

Everything was mixed and cooked in the Staging:E project and… guess what! We got brand new Factory ISOs with storage-ng, successfully building and verified to work by openQA, as you can see in this screenshot of the Staging Dashboard.

Yes, we know there are two failing tests in that dashboard, but that was fully expected since those tests use the expert partitioner to configure an installation of openSUSE on top of a MD RAID system and the reimplemented partitioner still lacks some controls to configure MD RAID arrays.

The new stack will live in Factory:Staging:E (or any other staging project the Tumbleweed crew decides) for quite some time, until it’s feature-pair with the old storage layer and, thus, can progress further in its travel to Tumbleweed. But Factory was just the first stop, the ultimate goal of this sprint was getting into the preliminary versions of the next SLE and openSUSE Leap.

That second integration is taking a little bit longer because it has coincided on time with other important changes in the installer and the base system… and the fact that August is the typical European vacation period is not exactly helping to iron all the details out. But since the new storage system works for Factory, we are certain it will do it for SUSE Linux and Leap.

As readers familiar with the Tumbleweed development process may have noticed already, having all those packages in Staging:E implies that newer versions of them will only reach Tumbleweed all at once, when yast2-storage-ng is considered mature enough for that. Somehow, that will block us from delivering new features for the packages you see in the list in the mentioned image of the dashboard. But don’t worry, if something serious happens and a critical update is needed we will not let our beloved Tumbleweed users down.

But there is much more happening in YaSTland beyond the storage reimplementation. Let’s take a look to the improvements in other areas.

Installation without Grub packages

Sometimes, users have already Linux installed in their system and they do not want to install Grub in MBR again with a new Linux distribution since the installed Linux can manage the bootloader. For this case, the user may decide to not install grub packages at all in the system. However, until now the user was obligated to install this package otherwise an error message would appear, as the image below shows.

For some specific scenarios, as you may find here, even other packages are required, and when the user decided for not installing the bootloader, these packages were still required for the installation.

We changed this behavior in Tumbleweed and SLE 15, and now the users will be able to install the system without the packages that are not required, in case they decide to manage bootloader through another operational system.

But that’s not the only improvement introduced in the bootloader management during this sprint.

Improve how YaST finds disk to install Grub in MBR

In Leap 42.3 and SLE 12.3, we found out that, in some very specific cases (check the bug report for more details), YaST was not finding the correct disk to install Grub in MBR. When it happened, an error message appeared at the end of the installation, showing that Grub could not be installed in /dev/btrfs disk.

We improved our approach to finding the correct MBR device, by adding a specific search for the disk where the partition /boot or / (in case /boot does not exist) is located.

Such a change will be released as maintenance update and self-update, and it affects only Leap 42.3 and SLE 12.3, since SLE 15 will use the new storage layer, which does not need this double check for the correct disk.

And talking about the new storage system…

Remove support for ReiserFS

The support of new installations with ReiserFS was removed from YaST in SUSE Linux Enterprise 12 and openSUSE Leap 42 but upgrades were still supported.

With SUSE Linux Enterprise 15 and Leap 15 the support of ReiserFS will be completely removed from YaST and the installer will block the upgrade of systems formatted with ReiserFS.

If some of the entries in the /etc/fstab file of the system to be upgraded is using ReiserFS, the installer will suggest to convert them to another filesystem type before migrating the system to SUSE Linux Enterprise 15 or openSUSE Leap 15.

A similar blocking error will be reported for ReiserFS root partitions.

Another Ruby 2.4 fix

This may be interesting for Ruby developers in general. We got a bug report about crashing YaST which in the end turned out to be caused by upgrade to Ruby 2.4. The tricky part was that YaST crashed randomly and it was difficult to reproduce the problem.

It turned out that the crash happened when Ruby wanted to print a warning on the error output, which in some situations failed. We did not fix the race condition, as it likely would be too difficult to debug the Ruby internals, but we at least fixed the code to not produce the warnings anymore.

So if you are a Ruby developer take this free advise from your YaST fellows – if your code crashes randomly with Ruby 2.4 then check for the Ruby warnings first.

A heads-up about network devices names

Two sprints ago we told you about the new possibility of configuring the network with AutoYaST already in the first stage, avoiding an extra restart of the system in most cases.

During this sprint we spent some time trying to test old AutoYaST profiles (with complex network configurations) with the upcoming version of SUSE Linux Enterprise Server, using our suite of automatic AutoYaST Integration Tests. But we found some issues caused by the current architecture of our test suite that may be of interest for some of our readers.

Let’s see some technical background first.

Tumbleweed has been using ‘predictable network interface names’ for some time now and it fits most regular use cases. Inspired or following the scheme idea introduced by ‘biosdevname’, Predictable Network Interface Names was adopted in systemd/udev v197 trying to solve an historical problem with the non deterministic classic naming scheme for network interfaces (eth0, eth1, eth2 …)

Basically it will assign fixed names based on firmware, topology, and location information making them stable between system reboots, hardware additions or removals and also between kernel or drivers updates.

For the upcoming SLE15, we are giving predictable network interface names a try (they are disabled in SLE12 and openSUSE Leap 42.x). For us that turned to be a problem because our AutoYaST testsuite dynamically creates new virtual machines on every system reboot (instead of really rebooting the virtual machine created in the previous step). So from the point of view of the operating system being tested, all the network devices are replaced by new ones in every reboot and that drives the network settings nuts.

That was only our case (arguably “our fault”), but there might be other situations in which going back to the old naming scheme (with names like ‘eth0’) would be more convenient than adapting the preexisting AutoYaST profiles to the new one. In such cases you still can use the old scheme (not fully predictable but very well known by Linux veterans) by just booting the SLE15 installation with this parameters.

biosdevname=0 net.ifnames=0

More to come

In addition to everything reported in this post, we have been working hard to get some new cool features to the upcoming SLE15 and to get the storage reimplementation full-featured enough to substitute the old one in all possible situations.

So, although it would still be summertime (in Europe), stay tuned for more news in two weeks.

Posted in Distribution, Factory, Programming, Systems Management, YaST | 3 Comments »

Developing with OpenSSL 1.1.x on openSUSE Leap

August 18th, 2017 by AndreasStieger

The SUSE OpenSSL maintainers are hard at work to migrate openSUSE Tumbleweed and SUSE Linux Enterprise 15 to use OpenSSL 1.1 by default. Some work remains, see boo#1042629.

Here is how you can use OpenSSL 1.1 in parallel to the 1.0 version on openSUSE Leap for development and run-time:
(more…)

Posted in Factory, Programming, Security | 1 Comment »

Advertisement
Tags
11.3 11.4 12.1 12.2 12.3 13.1 13.2 amd ARM ATI Beta buildservice Build Service C-Language cloud Collaboration Community conference Education event Events Factory fglrx fun GNOME gsoc Hackweek KDE Kernel Kraft Linux LXDE obs openSUSE Package PostgreSQL radeon raspberry Raspberry Pi rpm Ruby Tumbleweed XML xorg YaST
Lizards
- Adrian Schröter (12)
- Agustin Chavarria (6)
- Alessandro de Oliveira Faria (13)
- Alex Barrios (12)
- Alexander Naumov (10)
- Alexander Orlovskyy (3)
- Alin M Elena (5)
- Andrea Florio (27)
- Andreas Jaeger (70)
- Andreas Stieger (12)
- Andrew Wafaa (31)
- Arvin Schnell (9)
- Atri Bhattacharya (3)
- Bernhard Wiedemann (31)
- Bonnie Kurniawan (1)
- Bruno Friedmann (98)
- Calumma Brevicorne (29)
- Carl Fletcher (1)
- Christopher Hobbs (17)
- Ciaran Farrell (3)
- Stephan Kulow (17)
- craig gardner (2)
- Stephan Barth (2)
- Thomas Schmidt (2)
- Dinar Valeev (1)
- Dirk Mueller (2)
- Dmitry Serpokryl (7)
- Efstathios Iosifidis (21)
- Fabio Mucciante (5)
- Federico Lucifredi (9)
- Greg Freemyer (1)
- Holger Sickenberg (2)
- Hubert Mantel (1)
- Ilya Chernykh (5)
- Ismail Donmez (1)
- J. Daniel Schmidt (2)
- James Tremblay (7)
- Jan Blunck (4)
- Jan Loeser (3)
- Jan Madsen (1)
- Jan-Christoph Bornschlegel (3)
- Jan-Simon Möller (20)
- Javier Llorente (12)
- Jigish Gohil (85)
- Jiri Srain (1)
- Jiří Suchomel (3)
- Johan Kotze (5)
- José Oramas M. (6)
- Josef Reidinger (16)
- Juergen Weigert (1)
- Julio Vannini (9)
- Dinar Valeev (5)
- Kevin "Yeaux" Dupuy (11)
- Klaas Freitag (55)
- Lars Vogdt (11)
- Ludwig Nussel (13)
- M. Edwin Zakaria (4)
- Marcus Hüwe (39)
- Marcus Meissner (2)
- Marcus Moeller (3)
- Marcus Schaefer (4)
- Martin Lasarsch (8)
- Martin Mohring (11)
- Masim "Vavai" Sugianto (20)
- Michael Andres (1)
- Michael Löffler (7)
- Michal Marek (7)
- Michal Vyskocil (12)
- Miguel Angel Barajas Hernandez (2)
- P Linnell (2)
- Nelson Marques (55)
- Nenad Latinović (1)
- Nikanth Karthikesan (2)
- Przemyslaw Bojczuk (1)
- Peter Pöml (4)
- Petr Gajdos (2)
- Petr Mladek (60)
- Petr Uzel (5)
- Ray Wang (1)
- Raymond Wooninck (1)
- Ricardo Chung (7)
- Ricardo Varas Santana (7)
- Richard Bos (11)
- Robert Schweikert (16)
- Rossana Motta (1)
- Rupert Horstkötter (10)
- Sascha Manns (66)
- saydul akram (3)
- Sebastian Siebert (6)
- Shawn Dunn (2)
- Stanislav Visnovsky (7)
- Stefan Haas (1)
- Stefan Hundhammer (5)
- Stefan Schubert (7)
- Steffen Winterfeldt (8)
- Suresh Jayaraman (3)
- Susanne Oberhauser (3)
- Thomas Göttlicher (6)
- Thomas Schraitle (26)
- Togan Muftuoglu (3)
- Tuukka Pasanen (36)
- Will Stephenson (22)
- YaST Team (90)
Archives
- March 2020 (1)
- February 2020 (2)
- January 2020 (1)
- December 2019 (3)
- November 2019 (2)
- October 2019 (4)
- September 2019 (3)
- August 2019 (3)
- July 2019 (4)
- June 2019 (2)
- April 2019 (4)
- March 2019 (3)
- February 2019 (5)
- January 2019 (1)
- December 2018 (2)
- November 2018 (2)
- October 2018 (3)
- September 2018 (1)
- August 2018 (3)
- July 2018 (2)
- May 2018 (2)
- April 2018 (2)
- March 2018 (2)
- February 2018 (2)
- January 2018 (2)
- December 2017 (1)
- November 2017 (2)
- October 2017 (2)
- September 2017 (3)
- August 2017 (4)
- July 2017 (4)
- June 2017 (2)
- May 2017 (4)
- April 2017 (2)
- March 2017 (3)
- February 2017 (3)
- January 2017 (2)
- December 2016 (5)
- November 2016 (3)
- October 2016 (6)
- September 2016 (2)
- August 2016 (3)
- July 2016 (4)
- June 2016 (2)
- May 2016 (2)
- April 2016 (1)
- March 2016 (2)
- February 2016 (4)
- January 2016 (4)
- December 2015 (6)
- November 2015 (2)
- October 2015 (3)
- September 2015 (2)
- August 2015 (2)
- July 2015 (2)
- June 2015 (3)
- May 2015 (12)
- April 2015 (7)
- March 2015 (6)
- February 2015 (6)
- January 2015 (7)
- December 2014 (5)
- November 2014 (3)
- October 2014 (5)
- September 2014 (3)
- August 2014 (5)
- July 2014 (5)
- June 2014 (7)
- May 2014 (9)
- April 2014 (2)
- March 2014 (9)
- February 2014 (9)
- January 2014 (10)
- December 2013 (9)
- November 2013 (10)
- October 2013 (10)
- September 2013 (6)
- August 2013 (7)
- July 2013 (3)
- June 2013 (7)
- May 2013 (4)
- April 2013 (4)
- March 2013 (7)
- February 2013 (6)
- January 2013 (3)
- December 2012 (3)
- October 2012 (6)
- September 2012 (6)
- August 2012 (5)
- July 2012 (12)
- June 2012 (6)
- May 2012 (4)
- April 2012 (4)
- March 2012 (5)
- February 2012 (2)
- January 2012 (5)
- December 2011 (10)
- November 2011 (6)
- October 2011 (5)
- September 2011 (9)
- August 2011 (12)
- July 2011 (14)
- June 2011 (11)
- May 2011 (18)
- April 2011 (15)
- March 2011 (26)
- February 2011 (16)
- January 2011 (23)
- December 2010 (27)
- November 2010 (18)
- October 2010 (21)
- September 2010 (16)
- August 2010 (21)
- July 2010 (20)
- June 2010 (33)
- May 2010 (29)
- April 2010 (24)
- March 2010 (29)
- February 2010 (22)
- January 2010 (20)
- December 2009 (15)
- November 2009 (21)
- October 2009 (17)
- September 2009 (22)
- August 2009 (28)
- July 2009 (36)
- June 2009 (38)
- May 2009 (40)
- April 2009 (30)
- March 2009 (20)
- February 2009 (21)
- January 2009 (27)
- December 2008 (23)
- November 2008 (12)
- October 2008 (23)
- September 2008 (40)
- August 2008 (24)
- July 2008 (12)
- June 2008 (28)
- May 2008 (26)
- April 2008 (1)

Archive for the ‘Factory’ Category

Fine tuning installer behavior in small disks

More parameter passing for s390

Fun with MD RAIDs

And even more fun with MD RAIDs

Improved error reporting for wrong bootloader in AutoYaST

Setting the default subvolume name in AutoYaST

Skipping Btrfs subvolume creation

Keep it rolling!

Keep predictable network devices settings on S390x

Fun with console configuration of GRUB2

Bootloader configuration during upgrade

Fixing kdump on Xen

Improving the upgrade from SLE11/12 to SLE15

Falling back to the guided proposal

Leap /SLE 15 is closed, but Tumbleweed is still rolling

Conclusion

Updating NFS Version Handling

Fixing Broken Translations

Installing Over VNC Using the Browser

Asking Once About Equivalent Licenses

License Confirmation in CaaSP 3.0

Improving the addon Boot Option Handling

Echoes of Winter: White Text on a White Background

Conclusions

Installation and Upgrade

Clearer Description of Migration Targets

Importing the SMT Server SSL Certificate at Upgrade

Many System Roles

Storage

Better Message for Multipath (and other) Problems

AutoYaST is now Able to Reuse Encrypted Devices

Miscellaneous

Fixed AutoYaST profiles validation issues.

Translation Issues

The XSL File Format

Missing textdomain Call

Partitioner: more flexibility with the partition id

Partitioner: better support for DASD

Partitioner: can’t resize a partition… but why?

Handling registration rollback in SLE15 Migration

More fun with hiding/showing beta versions in SLE15

A look into the future: analyzing how we display licenses

AutoYaST product selection and installer update improvements

Document main differences in AutoYaST profiles between SLE12 and SLE15

Cron config for NTP client

Two months… and counting

More flexible NET installation ISOs

Storage-ng lands into Tumbleweed: handle with care

Alignment of partitions in the expert partitioner

Sanity checks for the storage setup

More improvements in the Expert Partitioner

Minimize changes between the SLE15 “Installer” and “Packages” DVDs

Add On products in AutoYaST

Fixed a crash when shutting down the YaST user interface

More to come

Hackweek project: create encrypted installation media

User-friendly error messages in AutoYaST

Bringing back skip lists to AutoYaST partitioning

More on AutoYaST

SLE15 media based upgrade for unregistered system

Fixed an installer crash in systems with 512MB RAM

Expert partitioner: the some boys are back in town

Improving the product upgrade workflow

Fix of a registration issue during installation process

Improving help texts in the registration process

Twisting the storage proposal: this time for real

Replacing ntpd with Chrony in yast2 ntp client

Configuring the keyboard in the installer via systemd

And now for something completely different

The storage reimplementation gets on the launchpad

Installation without Grub packages

Improve how YaST finds disk to install Grub in MBR

Remove support for ReiserFS

Another Ruby 2.4 fix

A heads-up about network devices names

More to come

Advertisement

Tags

Lizards

Improving the `addon` Boot Option Handling

Missing `textdomain` Call