The SUSE OpenSSL maintainers are hard at work to migrate openSUSE Tumbleweed and SUSE Linux Enterprise 15 to use OpenSSL 1.1 by default. Some work remains, see boo#1042629.
Here is how you can use OpenSSL 1.1 in parallel to the 1.0 version on openSUSE Leap for development and run-time:
(more…)
Doubtlessly, these are pretty exciting times for the YaST team. The merge of the new storage layer into the main codebase is around the corner and we are working on other features that will debut on the next open(SUSE) major release. So let’s summarize what happened during the last sprint.
As you may already know, the YaST team has invested a lot of time and effort preparing our storage layer for the future and we have started to merge the new layer into the main code base during the current sprint. But that’s something for our next report, right? By now, we will just focus on the stuff that got added and fixed during the last two weeks.
libstorage-ng, the low level library in which our new storage layer relies on, got support for BIOS RAID (handled in Linux via MD devices). Now, YaST could take advantage of such a feature to allow the installation of open(SUSE) systems on this kind of devices, including the bootloader.
The new Expert Partitioner is getting a lot of attention these days and, during sprint 40, it got initial support for Btrfs subvolumes management.
Now, when you select the BtrFS section in the general menu placed on the left, all BtrFS filesystems are presented allowing you to edit its subvolumes through a dialog which contains the list of subvolumes that belongs to the filesystem. Apart from the usual stuff, like adding and deleting subvolumes, it is also possible to set the noCoW property when you are creating a new one.
However, some features are still missing. For instance the partitioner will not prevent you to create a subvolume which is shadowed by an already existing mount point. Consider the current implementation as the first step towards a really cool Btrfs subvolume handling.
During installation, YaST uses a mechanism known as SUSE tags as source of information for media handling. For instance, a /content file contains information about the product, languages, etc. Additionally, information like release notes or the slide-show texts are stored in the installation media.
Some time ago, SUSE decided to drop SUSE tags and use RPM metadata and packages to store all that information. To make it possible, the installation media would use REPOMD repositories.
Obviously, YaST needs some adaptation. As a first step, support for the /content has been dropped, cleaning up some old and even unused code.
In the upcoming sprints, YaST will be adapted to retrieve licensing, release notes, etc. from RPM repositories and packages, which is also an opportunity to do some refactoring and to improve test coverage.
Nowadays YaST supports having add-on products on the same media than the base product. The problem is that the EULA for those products is displayed too early, even before AutoYaST had been initialized at all.
To solve this issue, now the EULA acceptance of included add-ons is performed at the same time than other add-ons which are not included in the installation media. As a side effect, now the user needs to define those add-ons on the AutoYaST profile in order to handle the EULA acceptance.
As developers, we enjoy working on new features and, of course, we are committed to fix critical bugs as fast as possible. But there are many small (and annoying) bugs out there that deserve our attention. Additionally, there are several bug reports that are no longer valid (the bug was fixed, it is not reproducible, it is a duplicate, the affected product is not supported anymore, etc.). In order to reduce the list of open issues, the team decided some sprints ago to reserve one day to do some bug squashing.
Among the bugs we closed during this sprint, we would like to highlight a usability problem in YaST services manager. Bear in mind that, along with the graphical interface, YaST ships a text based one which is supposed to fit in good old-fashioned 80×24 terminals. That’s an interesting constraint when you are designing interfaces for YaST.
Needlessly to say that, from time to time, we get a bug report about some element that just do not fit. In this case, YaST services manager had a problem when the service name was too long as you can see in the screenshot below.
Now, if there is not enough space, the name will be truncated and the rest of the information will be shown in an proper way.
As you may have noticed, a lot of interesting things are currently happening in the YaST world and more cool stuff is about to come. So you should not miss our next sprint report.
By now, enjoy openSUSE 42.3 (you already upgraded your system, right?) and see you in two weeks.
openSUSE 42.3 is out! Do you need some reading material while you wait for the download of the new release to finish? Don’t worry, we have the solution right here – another YaST team report. 😉
Obviously, we stopped adding new features to SLE12-SP3 and Leap 42.3 some days ago, because everything needed to be tested properly before the release. So now we are mainly looking into the future. And one of the plans for that future regarding SUSE Linux Enterprise (SLE) is offering several products packed in a single installation medium.
SUSE offers several mission-specific products based on SLE and, so far, every product needs to be installed from its own medium (usually a DVD or a virtual image). So if you use SLE Server, SLE Desktop and SLE Server for SAP, you have to have three DVDs which is a bit cumbersome.
After some discussions about the technical implementation details, we created a first prototype of the installer with an extra dialog that allows to select one of the detected products and continues the installation from there according to the installation workflow of the chosen product. It is still a proof of concept, but we can at least share screenshots showing how it looks for the time being.
So far, there are no plans to use the new feature in openSUSE, mainly because the project does not deliver separate mission-specific products in the same way than SUSE does with SUSE Linux Enterprise.
In the Sprint 36 report we presented the rewrite of the YaST Partitioner and we have been informing about its evolution in subsequent reports. We told you back then that we decided to split it in a separate yast2-partitioner package. But time has proved that decision to have too many drawbacks so we decided to bring the Partitioner back home, to yast2-storage-ng. As part of the process, we got rid of an old previous prototype of the Partitioner that was still lying in the yast2-storage-ng repository and some code that was there just to support that old prototype.
You may be asking why is all that relevant. It is because that means the repository (and thus the package) is finally approaching the final structure it will have when released into Tumbleweed. And that implies that all the systems we use to automatically measure the quality and reliability of our repositories are now providing trustworthy results… as trustworthy as automatic quality evaluations can be.
And according to those tools:
yast2-storage-ng is covered with automated unit tests in addition to openQA (this number is expected to raise in close future as we polish the new Partitioner),If you wonder about the numbers for the old codebase we want to replace, its code quality is 0.94 and only a 31% of it is covered by unit tests. A perfect example of legacy code.
As reported in previous posts, the new storage stack can already process AutoYaST profiles including partitions, LVM and MD arrays, but some details are still missing. The first of those details we wanted to address was the definition and creation of subvolumes in a Btrfs file-system.
Now it works according to the official documentation – both syntaxes for the <subvolumes> section are supported, it never creates subvolumes that would be shadowed by any other mounted file-system and it uses the list in control.xml as fallback for the root partition if Btrfs is used but subvolumes are not specified.
All that, as usual in the re-implemented stack, with fully tested and documented code.
In the previous report we showed you how the support for Multipath I/O looked at the library level, which usually means just geeky graphs. During this sprint, we have taught the installer to use that new library feature, so we now have real screenshots to show!
During the installation, now multipath hardware is detected and the user is asked for activation.
If the user agrees, the installer will never use the individual disk devices to propose a partitioning layout and it will not offer them as an option during the guided setup. The installer always works on the final (compound) multipath device, proposing the correct names for the partitions and so on (which, being a devicemapper device, follows a
different pattern when compared to raw devices).
The resulting system is still not fully bootable because yast2-bootloader has still not been adapted to this scenario. Very likely, something for the upcoming sprint, so stay tuned.
The world changes every day and we are always adapting YaST to remain shiny. The 2.4 version of Ruby is about to land in openSUSE Tumbleweed and is expected to be the default Ruby for SUSE Linux Enterprise 15 and openSUSE Leap 15. We found that some of the YaST packages were not fully ready for this new Ruby, so it was time for some tweaking.
After dealing with quite some details too technical and boring for this blog (but feel free to ask if you want the gory bits), YaST is shining again in Factory, which means we are no longer blocking the adoption of Ruby 2.4 in Tumbleweed.
As our team keeps always developing new features, solving bugs, and receiving feedback, we always evaluate our priorities and product. Sometimes, during this evaluation, we see that some YaST modules do not bring enough value or do not shine enough as part of our standard package.
After some evaluation, we come to announce that the modules Add-on Creator and Product Creator will no longer be part of YaST. These packages use Kiwi as backend and we have high competition on UI sides – SUSE Studio and Open Build Service. So it no longer makes sense to have these packages and we recommend for users of these modules to use one of the alternatives or Kiwi directly if you already have an XML definition file for Kiwi.
As Service Request Numbers can be now composed of 11 or 12 digits, instead of only 11 digits as before, we had to adapt YaST to handle this change. YaST module Support can now accept 11 or 12 digit service request numbers. We implemented such a change for all products dating back from SLE 10 SP3 until the most recent SUSE Linux versions. Updates with this change will be soon released.
The YaST installation used to have two stages, separated by a reboot. Starting with SLE 12 and openSUSE Leap 42.1 we have eliminated the
second stage. But it was still needed for AutoYaST, controlled by the setting
<profile><general><mode><second_stage>true | false</...>
We have fixed those parts of the networking setup and now you can explicitly set AutoYaST to not use a second stage anymore.
An issue that we have found out is that GDM has problems with the system when different users have the same UIDs. If it happens, GDM does not start properly. As a solution, we decided that either UIDs will be defined in the AutoYaST configuration file for ALL users or this tag will not be used at all for ALL users since a mix of both can result into duplicate UIDs.
We hope you liked our report as much as we loved to build all of that. We’ll continue YaSTing so we can reach you again in two weeks with much more cool stuff to show.
By now, enjoy your openSUSE 42.3 and all the cool features that came with it!
Browsers like Chrome, Chromium and Mozilla Firefox have moved to running tabs in separate threads and processes, to increase performance and responsiveness and to reduce the effects of crashes in one tab.
Occasionally, this exhausts the default limit on the amount of processes and threads that a user can have running.
Determine the maximum number of processes and threads in a user session:
$ ulimit -u
1200
The SUSE defaults are configured in /etc/security/limits.conf:
# harden against fork-bombs
* hard nproc 1700
* soft nproc 1200
root hard nproc 3000
root soft nproc 1850
In the above, * the catch-all for all users.
To raise the limit for a particular user, you can either edit /etc/security/limits.conf or create a new file /etc/security/limits.d/nproc.conf. Here is an example for /etc/security/limits.d/nproc.conf raising the limit for the user jdoe to 8k/16k threads and processes:
jdoe soft nproc 8192
jdoe hard nproc 16384
If you want to do that for a whole group, use the @ prefix:
@powerusers soft nproc 8192
@powerusers hard nproc 16384
In either case, this change is effective only for the next shell or login session.
Here we go again with a new report from the YaST trenches. This time with the storage reimplementation as the clear star of the show.
As we have announced in our previous sprints and as you probably already know, the YaST team is working hard to rewrite the whole storage stack on time for SLE15 and openSUSE Leap 15. As part of this reimplementation we have designed a brand new storage proposal that automagically offers the user the best combination of partitions and LVM volumes based on the current configuration of the system and the user preferences.
When we are working with very small disks or with special technologies like DASD (which doesn’t accept more than three partitions by device), the storage proposal might not be able to generate a valid initial proposal honoring the initial requirements of the product (e.g. creating a separate home partition and enabling btrfs snapshots for the root partition in the openSUSE Leap case). Now the proposal is not limited to fail when it is not possible to satisfy the default product requirements. Before giving up, the new system looks for alternatives, like discarding the separate home partition or disabling snapshots. Moreover, now the proposal is able to automatically adjust the size requirements not only for root, but also for swap and home. And, of course, the guided setup continues there for fine tuning the proposal settings.
As our usual readers also know, we recently introduced a more fair desktop selection screen for openSUSE, both Leap 42.3 and Tumbleweed. We used part of the latest sprint to implement some feedback we gathered about the wording and behavior of that dialog.
That feedback gathering included some discussions on how to make user experience nicer after selecting one of the user interfaces available through the “custom” option. As a result, the awesome openSUSE crew created a new mechanism for selecting the default window manager on each graphical login, so YaST can delegate the details to the maintainers of those alternative interfaces.
How everything works now? Glad you asked. 🙂
If the user select KDE or GNOME in the YaST dialog, /etc/sysconfig/windowmanager is configured to point to that desktop by default. If the “custom” option is selected, then YaST does not enforce any interface in that file and the new mechanism comes into play. It relies in the default.desktop file, which defines the default window manager and can be managed by the common update-alternatives workflow. Meaning it can be easily tweaked by the package maintainers and by the users, specially since YaST includes a nice module for managing alternatives.
Although, as explained above, we keep improving the storage automatic proposal to support more and more situations, we cannot ignore that flexibility and adaptability have always been two of the flagships of (open)SUSE. And one of the most prominent examples is the YaST Expert Partitioner.
As detailed in our report of the sprint 36, we have been rewriting that powerful Swiss Army knife using the new storage stack but keeping the same user interface and functionality. So far, the new implementation was only able to display information about the existing partitions, LVM systems and MD RAIDs. But now we added many options to create, edit and delete partitions.
It is still a work in progress because the number of possibilities offered by the YaST Partitioner is sometimes overwhelming and implementing them all takes time, but it’s progressing nicely.
Beside the improvements in the Partitioner itself, we also worked on its integration in the installation workflow. Now the Expert Partitioner can be used to refine the schema automatically proposed by the installer. As a bonus, the behavior of the “abort” and “finish” buttons has been improved in relation to the Expert Partitioner currently available in (open)SUSE, which historically shows a usability inconsistency there compared to the rest of the installation process.
In both SLE and openSUSE Leap, the online updates can be installed either manually or automatically at some regular intervals. For the automatic installation we provide the yast2-online-update-configuration package which provides a cron job script and an YaST module for configuring it (hint: it is not installed by default, you might give it a try, maybe it is something “new” for you).
You can configure how often the patches should be installed or filter the patch categories in the YaST module, but we got a bug report saying that when multiple patch categories are selected only one is actually used.
It turned out to be a trivial mistake in the cron job and we fixed it for SLE12-SP2 and Leap 42.2, as well as for the upcoming SLE12-SP3 and Leap 42.3. So if you use this module and want to use the category filter then it’s recommended to upgrade the package.
And going back to our new storage stack, we keep working to integrate it better with other parts of YaST, specially AutoYaST. During this sprint we polished some rough edges and we added support for MD RAID. With all that, now is possible to automatically setup a system based on an AutoYaST profile containing any combination of partitions, LVM systems and MD arrays, including encryption at any of the levels.
But the relation of AutoYaST with the system works in both directions. Apart from being able to install a system based on an AutoYaST profile, it also offers the interface to export the current system configuration, including the storage layout, to a profile in order to reproduce the system later. During this sprint we also ported that logic to rely on the new storage layer.
It was harder than it sounds, due to the need of keeping the backwards-compatibility with several behaviors that have been introduced along the AutoYaST history to adapt to several specific situations. On the bright side, the new code is easier to follow, includes behavior-driven automated tests (RSpec) and contains information about the rationale of each decision… which in some cases required some archaeology.
Another part of YaST we are constantly working to integrate better with the new storage stack is yast2-bootloader. Although the new storage system was already able to correctly setup a valid disk layout for most situations and architectures (each one with it’s own requirements for booting), our bootloader module is still not fully compatible with the new system in all those scenarios.
During this sprint, we adapted it to ensure all the combinations suggested by the new storage stack (partitions, LVM, encryption and so on) are correctly covered by YaST2-Bootloader. As a result, we can already say that our testing ISO images are fully installable in any x86_64, PowerPC and S/390 system by just clicking “next” a few times. And we have automated integration tests (a separate openQA instance) to ensure the resulting system boots just fine.
We know some of our readers enjoy our more technical posts and like to lurk into the kitchen to see how we deal with all kind of surprises maintaining a complex tool like YaST. Today’s chapter in that regard started with a bug report about the bootloader being installed in the wrong device name (/dev/vda instead of /dev/vdb) in an emulated PowerPC machine in openQA.
After a lot of investigation and with help from our PowerPC expert, we found the culprit, that turned out to be an emulator quirk. The next couple of paragraphs may be interesting or daunting, depending how much virtualization and PowerPC jargon you know. Be warned.
On POWER, the PReP partition containing the bootloader has no unique identifier other than the serial number of the disk on which it created. QEMU virtualization does not provide any disk serial number when the user does not explicitly specify one. This means that the PReP partition in QEMU installation does not have any unique identification and the partition name may change when a disk is added or removed from the virtual machine or the storage configuration otherwise changes. This may lead to system errors related to bootloader installation and updates.
It is recommended to assign a unique serial number to each disk in a QEMU virtual machine on POWER when it is expected the storage configuration of the virtual machine may change. Otherwise, there is nothing YaST or any other tool running within the virtual machine can do to avoid the problems. So this time we only did the investigation part, with the fix coming from the openQA side, which was improved to explicitly set serial numbers when needed.
As exposed above, our StorageNG testing image already works in all the supported architectures and in scenarios combining plain partitions, LVM, RAID and encryption in any way. But there are even more situations and technologies currently supported by YaST that we need to incorporate into the new stack.
First of all, we used this sprint to add Multipath I/O support to the new libstorage. Now it can also be combined with all the other mentioned technologies (like having an LVM system on top of an encrypted multipath device), although the storage proposal still needs to be adapted to play nicely with preexisting multipath setups.
As usual with the library stuff, the best “screenshot” we have to offer is one of its geeky autogenerated graphs.
Another scenario that goes beyond the most regular use cases is installing the system into a network storage device, instead of a local hard disk. Now, the new storage system can report whether a root filesystem via a network is used. When that happens, YaST sets the network interfaces with the start mode nfsroot, which is used to avoid interface shutdown and, therefore, the unavailability of the system.
Once again, we omitted the boring parts about bugfixing (with yast2-ntp-client being the star in that regard) and similar stuff. We hope you enjoyed the report and we hope to reach you again in two weeks. Meanwhile…
Have a lot of fun testing the Leap 42.3 pre-releases and reporting bugs!
Since we announced in the latest report, we decided to shorten our development sprints from three to only two weeks. As a natural consequence, this is the first report of a series of shorter ones. But shorter doesn’t have to mean less juicy! Keep reading and enjoy.
We got a bug report about YaST not responding when a very long package changelog was displaying in the package manager. It turned out that some packages have a huge change log history with several thousands entries (almost 5000 for the kernel-default package). That produces a very long table which takes long time to parse and display in the UI.
The solution is to limit the maximum number of displayed items in the UI. You cannot easily read that very long text anyway, for such a long text you would need some search functionality which the YaST UI does not provide.
Finding the limit, that magic number, was not easy as we want to be backward compatible and display as much as possible but still avoid that pathological cases with a huge list.
In the end we decided to go for a limit of 512 change entries. The vast majority of packages have way fewer entries, so you should not notice this clipping functionality. When the limit is reached a command to display the full log is displayed at the end so you can easily see the missing part when needed. (Hint: the widget supports the usual copy&paste functionality, you can copy (with Ctrl+C) the displayed command and paste it into a terminal window directly.)
In the previous report we told you we are rewriting the YaST partitioner to use the new storage stack and modern reusable CWM widgets under the hood while retaining exactly the same behavior and look&feel on the surface. We are reimplementing one feature at a time, and this sprint it was the MD RAID’s turn.
Now the partitioner displays current RAIDs, with details about them and the devices used to construct each RAID. If a picture is worth a thousand words, here you have 3,000 words:
Our brand new containers-oriented solutions, SUSE CaaSP and its openSUSE counterpart Kubic, need to have /var/lib/docker as a separate partition for several reasons. It was a separate Btrfs subvolume already, but that wasn't quite separate enough. 🙂
The problem was just that the automated storage proposal in the old YaST storage subsystem is not prepared for anything like this; it can deal with root, swap and optionally a separate home partition (or volume if using LVM). That's it. Extending the current syntax of the control.xml file to be able to specify arbitrary partitions and adapting the code of the old proposal to understand and handle this new specification was unfortunately almost impossible. Sadly, we have to admit the old code is hardly maintainable these days and not flexible enough to accommodate this kind of changes in a safe way.
This is one (out of many) reasons why we are currently in the process of rewriting that entire YaST storage subsystem, as you already know from many previous posts in this blog. But, as you also know, the new system will debut in SLES15 and openSUSE Leap 15.0, too late for the current SUSE CaaSP and openSUSE Kubic.
We decided we'd introduce a hack based in the old proposal, well knowing that hacks accumulated in code can develop their own life just like Dust Puppy. But we plan to kill that hack immediately once StorageNG arrives.
So the hack was to simply use the logic for the separate home partition and repurpose it, keeping all the respective parameters in control.xml and introducing yet another one called home_path where the actual mount point for that partition can be specified — in this case /var/lib/docker. The tradeoff is that there can be no separate /home anymore in parallel to that.
That "feature" should not be used outside of that very specific use case in CaaSP and we even considered the possibility of not documenting it too publicly to avoid misuse. But we are living the open source spirit and whatever we do, we do in public. Even if it's the (quite embarrassing) fact that we consider changes to certain parts of our own code too risky. But again, that's why we are pushing the storage layer reimplementation (Storage-NG) so hard: we want to regain control over that part.
And talking about the new storage layer and our previous posts, you already know we are working hard to integrate it with AutoYaST. For the time being, custom partitioning layouts are supported, but only using plain partitions and LVM. Other features, like RAID or Btrfs subvolumes support, are still missing.
A nice thing about the new code is that it relies as much as possible on the new storage layer. On older versions, AutoYaST implemented some logic on its own and that caused some unexpected troubles. Fortunately, that is not the case anymore, and the new code looks way easier to extend and maintain.
/etc/vconsole.conf anymoreWhen configuring the system keyboard, YaST used to write the keyboard map configuration directly in the /etc/vconsole.conf file. However, this approach is no longer appropriated since it may cause undesired effects to other tools. Now YaST uses the Systemd tool localectl to set the keyboard map in /etc/vconsole.conf, instead of writing in it directly. Another step to make YaST a good citizen of the Systemd world.
If you use MD RAID arrays you probably know there are two ways of identifying them – by number or assigning a meaningful name to them. Named arrays use device names like /dev/md/<name> instead of /dev/md<number>.
During this sprint, we taught the new libstorage how to create and manage named RAID array, in addition to the already supported numbered ones. If the user decides so, the MD RAID name is used in fstab instead of the UUID (which was the only option with the old libstorage). We also made sure to improve the behavior in several scenarios, so we consider some bugs of the old storage to be fixed (or obsoleted, if you prefer) with storage-ng.
Linux also supports names of the form /dev/md_<name>. The new library is also able to handle this format, but the feature is intentionally disabled and documented to be unsupported because other parts of the system could not be 100% verified to work in that scenario. And we take quality assurance very seriously before labeling a feature as “supported”.
If you are not happy enough with the screenshots showing the RAID support in the partitioner, here you have more pictures. But, as usual with stuff implemented in the library, here “pictures” doesn’t mean screenshots, but fancy automatically generated graphs.
Of course there are a lot of other things we did during the last two weeks, although some of them were considered not interesting enough for this report or were not finished on time. We are already working in finishing the unfinished stuff and implementing new exciting improvements. And the bright side is that you only have to wait two weeks to know more… so stay tuned.
We are still digesting all the great content and conversations from openSUSE Conference 2017, but the development machine never stops, so here we are with the report of our post-conference sprint.
You have been reading for months about the new stack for managing storage devices and the new features and improvements it will bring to the installation. But so far there was no way to view and fine-tune the details of those devices. During this sprint we have implemented a first prototype of the new version of the YaST2 Expert Partitioner, that awesome tool you can invoke with yast2 storage.
To make the transition easier and to be able to submit it to Tumbleweed as soon as possible (hopefully in a couple of months, together with the rest of the new stack) we decided to postpone any UI redesign. So this first incarnation of the new expert partitioner looks and behaves exactly like the one available in current versions of (open)SUSE.
To try it out (on a scratch machine!), add a repository and remove the current storage library, as described in yast-storage-ng: Trying on Running System and then run zypper install yast2-partitioner. As you may have noticed, we split the partitioner in a separate package, unlike the current version that was part of the basic yast2-storage.
The new expert partitioner will only give you a read-only view of things similar to the following screenshots, not being able to modify anything yet.
As you can see in your own system or in the screenshots, the following items are already functional
The other kinds of devices that you can see in the navigation tree are so far only stubs.
You may feel a bit underwhelmed by this, and that’s OK, because most of the effort that we spent on this is actually hidden in a set of nice UI classes which we use to reconstruct the legacy procedural UI code. So the new expert partitioner not only relies on the revamped storage stack, but also on a powerful and reusable set of shiny UI components. If you ever need to code a user interface for YaST, the next section is for you.
This section may be a little bit too developer-oriented, so feel free to skip it if you don’t care about the YaST implementation details. If, to the contrary, you want to have a glance at the new YaST widgets, go ahead.
Before diving into the new widgets, let us introduce what CWM is. It stands for Common Widget Manipulation and it is an old procedural YaST module which puts together a widget, its help and its callbacks. These callbacks are used to initialize, validate and store the content of the widget. This organization allows easier re-usability of widgets, which are then put together into a dialog. We also made an object-oriented version of CWM, which uses the old one under the hood, but is based on classes. So the contents and callbacks all live in their own class which is then used in dialogs. It is already used e.g. in the bootloader module.
As part of the Expert Partitioner rewrite, we created new types of reusable widgets, like Table or Tree, that are now available for its usage in any YaST module.
We also realized that it would be cool to be able to construct full dialogs out of smaller “bricks”, because the partitioner dialogs usually have rather complex structures in which some parts are shared by several dialogs. For this purpose we added new kinds of widgets – a Page which represents a part of a dialog that contains other widgets, and a Pager which allows switching of pages. So far there are two different pagers. The first one is Tabs which shows a set of tabs and allows switching among them and the second one is TreePager which allows switching pages according the item selected in a tree.
As you can see in the screenshots from the Expert Partitioner, there is a tree on the left side, which decides which page is shown on the right side. That right side sometimes contains a set of tabs, which decides what is displayed for every single tab.
Building blocks for the win!
A new option to allocate memory high during enable of Kdump was already implemented in YaST interface but unavailable through command-line. From the next Service Pack (i.e. SLES 12 SP3, Leap 42.3, and Tumbleweed), the user will be able also to use this option in command-line and scripts. In order to do that you can just use the command yast2 kdump enable alloc_mem=low,high, where low sets Kdump Low Memory and high sets Kdump High Memory.
For current users of Kdump command line, the old command to enable kdump yast2 kdump enable alloc_mem=$mem will still work as before, keeping its compatibility.
During installation, when YaST detects in the system a particular filesystem or technology for which the installer would need additional packages to deal with, it alerts the user and tries to install those packages. A very visible case are the ntfs-3g and ntfsprogs packages, installed when a MS Windows partition is found in the system.
But, what happens if those packages are simply not available for installation? That’s the case of SLE12-SP3, which doesn’t include ntfs-3g. Should the installer block the installation of SLE12-SP3 alongside an existing MS Windows just because of that?
Fortunately we have solved that problem for the upcoming SLE12-SP3… and also created the code infrastructure to avoid similar problems in the future. Now we have a separate list for packages that would be nice to have installed in order to deal with a particular technology but that are not 100% mandatory to the point of blocking the installation process if they are not available. So we don’t bother the user about things that cannot be solved anyway.
But apart from looking into the future, we keep taking care of the existing YaST modules and its supported scenarios. During this sprint, we also addressed some issues related to YaST Remote, when using the command line.
The command yast2 remote list was installing required packages for YaST Remote and also restarting the display manager. However, as this command is expected to be a read-only operation, it shouldn’t change anything in the system. Such a problem was solved and now this command just lists the status of remote options.
Another issue was in the command yast2 remote allow=yes, which was opening a pop-up interface to alert the user about the changes in the system. Such a pop-up was impeding the use of this command in scripts. Therefore, we removed it when executing YaST Remote in command-line and, instead, we now just show a warning message on the console.
Both fixes were submitted as a maintenance update to all the supported versions of SLE and openSUSE and will reach our user as soon as they pass all the extra security checks performed by the respective maintenance teams. Of course, both fixes will also be included in future releases.
The Expert Partitioner was not the only thing we did related to the new storage stack during this sprint. We also tried to improve how the information is presented to the user everywhere.
Having a huge amount of information at a glance might be useful in certain cases… as long as that amount can be handled by a human brain! Since we don’t expect all our users to be androids, we decided to improve our storage actions summary. Now is much easier to understand what is going to happen in the disks after pressing the confirmation button.
They say a picture is worth a thousand words. So let’s compare the ultra-detailed list offered before this sprint…
…with the new digested one.
As you can see, the new summary carries the essential information in a clear and legible way. Delete actions are highlighted in bold and, moreover, the set of actions related to btrfs subvolumes are grouped in a collapsible list.
Integration of AutoYaST with the new storage has also received our attention during this sprint. Now, the summary dialog in AutoYaST shows the list of storage actions in the new compact way. Currently it is not possible to edit partitions from this AutoYaST dialog, but stay tuned for more information in upcoming sprints.
You already know how powerful can AutoYaST be in terms of automating complex installations based on flexible profiles, even trying its best if the profile contains parts that are challenging to implement in the target system.
One of those adjustments that AutoYaST can perform is reducing the size of some of the partitions specified in the provided profile if the target disk is not big enough, to make sure the installation doesn’t get blocked just by some missing space.
The mechanism works very well but that kind of automatic adjustments can be unexpected and can produce undesired results. That’s why we have added the following warning message.
Of course, this new warning uses the usual AutoYaST reporting mechanisms, so even if the users are not in front of the screen (something very common when performing an unattended installation) they will be notified about the special circumstance.
And now, another dose of technical content for those of you that love to lurk into the kitchen.
In the report of the sprint 30 we already described how we adopted Docker to power up our continuous integration process in the master branch of our repositories (the one in which we develop Tumbleweed and upcoming products). As also reported, we adopted the same solution for Libyui in the next sprint. And now it was the turn the branches of YaST that we use to maintain already released version of our products. Not a trivial task taking into account the many repositories YaST is divided in and the many products we provide maintenance for.
If you want to refresh your memory about the whole topic of using Docker for the continuous integration infrastructure, here you can watch the talk Ladislav offered about the topic a few days ago in the openSUSE Conference 2017.
If you don’t have a S/390 mainframe laying around, maybe you are not familiar with the concept of DASD (direct-access storage devices). DASDs are used in mainframe basically as regular disks… just that they are not.
DASDs are special disks in various aspects – they have a different partition table type allowing only three partitions with a restricted set of partition ids, they must be managed by a different set of partitioning tools, they have their own specific alignment logic and requirements…
But thanks to YaST and libstorage, in (open)SUSE you don’t have to care about most of those details. The expert partitioner and the installer allow you to treat DASDs almost as regular disks.
During this sprint we adjusted the new libstorage, i.e. the library C++ based layer of the stack, to be able to deal with DASD. As usual with new features implemented in the library, the only “screenshot” we have to show is one of the graphs generated by the library. Enjoy.
We want to have a shorter and more agile feedback loop regarding our development efforts. To achieve that, we have decided to shorten our Scrum sprints from the current three weeks to just two. So you will have more news from us in half a month.
But a feedback loop works in both ways, so we also expect to have more news from you. 🙂 See you soon!
I work to improve the openSUSE Tumbleweed (GNU/)Linux distribution. Specifically I make sure that all packages can be built twice on different hosts and still produce identical results, which has multiple benefits. This generates a lot of patches in a single week.
OBS
Sometimes it is enough to adjust the .spec file – that is a small text file usually specific to us. Then it is straight-forward
And OBS will even auto-clean the branch when the submit-request is accepted. And it has a ‘tasks’ page to see and track SRs in various stages. For the spec_add_patch to work, you need to do once
ln -s /usr/lib/build/spec_add_patch /usr/local/bin/
When you want to contribute patches upstream, so that other distributions benefit from your improvements as well, then you first need to find out, where they collaborate. A good starting point is the URL field in the .spec file, but a google search for ‘contribute $PROJECT’ often is better.
github
Then there are those many projects hosted on github, where it is also pretty low effort, because I already have the account and it even remains signed in. But some repos on github are only read-only mirrors.
sourceforge
The older brother of github. They integrate various ways of contributing. The easiest one is to open a Ticket (Patch or Bug) and attach the .patch you want them to merge with a good description. While many developers do not have the time and energy to debug every bug you file, applying patches is much easier, so gets your issue fixed with a higher chance.
devel Mailinglist
Some projects collaborate mainly through their development MLs, then I need to
project bugtracker
Like https://bugzilla.gnome.org/ https://bugs.python.org/ https://bugs.ruby-lang.org/ https://bz.apache.org/bugzilla/
So as you can see there is a wide range of ways. And most of them have some initial effort that you would only have once… But then I only contribute once per project, so I always pay that.
Thus, please make it easy for people to contribute one simple fix.
When installing openSUSE or SUSE Linux Enterprise, YaST is able to configure encrypted LVM using LUKS for full disk encryption. The default configuration is aes-xts-plain64 using a 256 bit master key. However, due to how the XTC mode splits the key into two halves, this reduces the effective key size used for AES to 128 Bits.
In order to use a 512 bit key for 256 effective AES, one needs to perform manual formatting prior to installation:
cryptsetup LuksFormat --key-size 512 /dev/sda1
However the installer suffers from boo#1030299 which prevents it from writing an entry to /etc/crypttab in this instance. This results in a system that is unable to boot after installation.
The work-around is as follows: Boot into the rescue system, open the crypto device and enter the installed system as a chroot:
cryptsetup luksOpen /dev/sda1 crypto
mount /dev/mapper/system-root /mnt
for X in proc dev sys; do mount -bind /$ /mnt/$X; done
chroot /mnt
(This example assumes /dev/sda1 to be the crypto device, and an LVM VG named system with a LV named root, and no separate /boot.)
Then in the chroot, edit /etc/crypttab to have the following line:
crypto /dev/sda1 none none
See man crypttab for additional settings and options. To finalize, regenerate the initrd and reboot
mkinitrd
exit
reboot
A future rewrite of the YaST storage abstraction layer is planned which should address this issue.
openSUSE Conference 2017 is coming! And as we flight there (literally, one third of the YaST team is in a plane right now typing this), we wanted to inform our beloved readers on what we did in the previous three weeks.
So here is our report, brought to you by airmail!
Leaving openSUSE Tumbleweed aside, The YaST team is currently working to deliver SLE12-SP3, openSUSE Leap 42.3, SLE15, openSUSE Leap 15, SUSE CaaSP 1.0 and Kubic (more about Kubic later). Three of them are already in beta phase, which means they are being extensively tested by several parties and in many scenarios, hardware platforms and possible configurations. That amount of manual testing always result in several bug being discovered, no matter how much we try to have some automated tests for the most common cases.
Many of the bugs our testers are finding are related to internationalization and localization, mainly texts in the UI that are always displayed in English, despite the system been configured (or being installed) in a different language.
But, of course, other kind of bugs are also being found. For example, our hardware detection component (hwinfo) was not able to deal with some new machines, making the installation experience everything but pleasant.
As a result, a significant amount of the YaST team manpower during this sprint was targeted to squash those annoying bugs. Which doesn’t mean we didn’t have time for some interesting new features and improvements.
Once again, our new storage system comes with news. Now it’s able to detect and unlock preexisting encrypted devices during the hard disks probing step, raising you a new pop-pup dialog to ask for the corresponding device password. After unlocking the devices, all your installed systems will be accessible for upgrade and, moreover, the LVM volumes allocated over encrypted devices will be activated.
The new storage stack is expected to debut in SLE15 (and, thus, openSUSE Leap 15), but the functionality can already be tested, for both the installation and upgrade processes, with the StorageNG test ISOs.
But the happiest news coming from the new storage stack during this sprint is it’s marriage with AutoYaST. The new automatic partitioning proposal (that is, the “Guided Setup”) is now integrated with AutoYaST.
Thanks to the new software architecture, AutoYaST users will be able to override every single partitioning setting from the control file.
<profile xmlns="http://www.suse.com/1.0/yast2ns" xmlns:config="http://www.suse.com/1.0/configns">
<general>
<storage>
<!-- Override settings from control file -->
<try_separate_home config:type="boolean">false</try_separate_home>
<proposal_lvm config:type="boolean">true</proposal_lvm>
</storage>
</general>
</profile>
So you can easily switch on/off LVM, use a separate partition for /home, enable/disable snapshots, enable/disable Windows resizing, etc. All that, still relying on the automated storage proposal to iron the details up. Something that is not possible with the current version of AutoYaST without being forced to define explicitly every partition and LVM volume.
But the simplest way to use the new libstorage proposal is to not define any setting at all in the AutoYaST configuration file. In that case, the partitioning proposal code will do the complete job, installing a new system with the default options.
Of course, before integrating the new storage stack into the upcoming SLE15, the AutoYaST support have to go one step further. Apart from using and configuring the proposal, it must be possible to define a completely custom setup including partitions, LVM volumes, software RAID devices and so on through the corresponding <partitioning> section of the AutoYaST profile. So we used this sprint to sketch a plan to make that possible in the following months, analyzing all the scenarios and configurations supported by AutoYaST and looking for the best way to support them using the existing yast2-storage-ng infrastructure. The outcome of that effort is this detailed document and a list of tasks (PBIs in Scrum jargon) for the upcoming sprints. So be prepared for more news in this regard.
So far the user had to ensure that snapshots created by rollbacks got deleted to avoid filling up the storage. This process has now been automated. During a rollback, Snapper now sets the cleanup algorithm to “number” for the snapshot corresponding to the previous default subvolume and for the backup snapshot of the previous default subvolume. This enhanced behavior will be available in SLE12-SP3 and openSUSE Leap 42.3. For more information take a look at the more detailed post in the Snapper blog.
For several sprints already we have been presenting features targeted to SUSE CaaSP, the Kubernets-powered solution for managing containers. Many of those features and custom configurations live in a package called yast2-caasp, originally targeted to this great upcoming product built on top of the SLE12-SP2 codebase.
But now the package is also available for Tumbleweed-based systems by request of the Kubic project. Kubic will be the openSUSE alter ego of SUSE CaaSP, that is, a Container as a Service Platform based on openSUSE and Kubernetes. As with any other YaST component, the exact same source code will shared by the SUSE product and its openSUSE brother.
Humans make mistakes, but when the mistakes are made entering some option in
the installation command line, it usually means that a reboot of the machine is be needed to fix them.
That was the case for the registration URL (regurl) option. In the provided address was malformed the installation just stopped. During this sprint we have added an early check of that URL which allows the user to reenter it and continue with the installation. Something that obviously improves the user experience.
In case of an autoinstallation (AutoYaST), the error is reported and the steps to get installer updates and to register the system are skipped.
There is still room for more improvements, allowing the user to also modify the URL in other scenarios. For example, for an URL with a valid format but that points to an unreachable server. But in those cases is not so straightforward to identify the culprit of the problem. It would make no sense to annoy the user with a recurring pop-up to change the registration URL if the root of the issue is not the URL but a incorrect network configuration.
As mentioned at the begining of this post, we recently got quite some bug reports about missing translations. Although some of them were really caused by bugs in the YaST code, others were a consequence of a buggy Ruby rxgettext script which collects the translatable strings from the Ruby source code. The bug is known by the Ruby-GetText developers, but it’s unclear when (or whether) it will be fixed.
The problem is that the tool cannot collect the translatable strings from interpolations. For example it cannot find the “foo” string from this string literal: "#{_("foo")}". As a result, that string is missing in the resulting POT file and cannot be translated by the SUSE or openSUSE localization teams.
As a workaround, we fixed the YaST code to not use the translations inside interpolations. We also documented the possible problems when mixing translations a interpolations and their solution.
And talking about new developer oriented documentation…
YaST runs with the administrator privileges (root) and therefore we have to be aware of the possible security issues in the code. During this sprint we published a document with a short summary of security tips for YaST developers.
If you are programming an YaST module you should definitely read it, but it might be interesting also for other programmers as many mentioned issues are generic, not tight only to YaST.
The document is available online here.
That’s all for this sprint report. We have many more things in the oven, be we didn’t manage to finish them during the sprint, so they will have to wait for the next report. Meanwhile we hope to see many of you at the openSUSE Conference 2017. There will be a whole workshop about modern YaST development, a summary with the more relevant news in the last year of YaST development, talks about the new superb yast2-configuration-management module, about our continuous delivery infrastructure and about how we use Docker to deliver YaST… And, of course, also many other interesting content like the awesome presentation from Thorsten Kukuk about the brand new openSUSE Kubic we mentioned earlier. And even more important, a lot of fun!
For those of you that cannot attend to the conference, see you again in this little corner of the internet in three weeks!
Adrian Schröter (12)
Agustin Chavarria (6)
Alessandro de Oliveira Faria (13)
Alex Barrios (12)
Alexander Naumov (10)
Alexander Orlovskyy (3)
Alin M Elena (5)
Andrea Florio (27)
Andreas Jaeger (70)
Andreas Stieger (12)
Andrew Wafaa (31)
Arvin Schnell (9)
Atri Bhattacharya (3)
Bernhard Wiedemann (31)
Bonnie Kurniawan (1)
Bruno Friedmann (98)
Calumma Brevicorne (29)
Carl Fletcher (1)
Christopher Hobbs (17)
Ciaran Farrell (3)
Stephan Kulow (17)
craig gardner (2)
Stephan Barth (2)
Thomas Schmidt (2)
Dinar Valeev (1)
Dirk Mueller (2)
Dmitry Serpokryl (7)
Efstathios Iosifidis (21)
Fabio Mucciante (5)
Federico Lucifredi (9)
Greg Freemyer (1)
Holger Sickenberg (2)
Hubert Mantel (1)
Ilya Chernykh (5)
Ismail Donmez (1)
J. Daniel Schmidt (2)
James Tremblay (7)
Jan Blunck (4)
Jan Loeser (3)
Jan Madsen (1)
Jan-Christoph Bornschlegel (3)
Jan-Simon Möller (20)
Javier Llorente (12)
Jigish Gohil (85)
Jiri Srain (1)
Jiří Suchomel (3)
Johan Kotze (5)
José Oramas M. (6)
Josef Reidinger (16)
Juergen Weigert (1)
Julio Vannini (9)
Dinar Valeev (5)
Kevin "Yeaux" Dupuy (11)
Klaas Freitag (55)
Lars Vogdt (11)
Ludwig Nussel (13)
M. Edwin Zakaria (4)
Marcus Hüwe (39)
Marcus Meissner (2)
Marcus Moeller (3)
Marcus Schaefer (4)
Martin Lasarsch (8)
Martin Mohring (11)
Masim "Vavai" Sugianto (20)
Michael Andres (1)
Michael Löffler (7)
Michal Marek (7)
Michal Vyskocil (12)
Miguel Angel Barajas Hernandez (2)
P Linnell (2)
Nelson Marques (55)
Nenad Latinović (1)
Nikanth Karthikesan (2)
Przemyslaw Bojczuk (1)
Peter Pöml (4)
Petr Gajdos (2)
Petr Mladek (60)
Petr Uzel (5)
Ray Wang (1)
Raymond Wooninck (1)
Ricardo Chung (7)
Ricardo Varas Santana (7)
Richard Bos (11)
Robert Schweikert (16)
Rossana Motta (1)
Rupert Horstkötter (10)
Sascha Manns (66)
saydul akram (3)
Sebastian Siebert (6)
Shawn Dunn (2)
Stanislav Visnovsky (7)
Stefan Haas (1)
Stefan Hundhammer (5)
Stefan Schubert (7)
Steffen Winterfeldt (8)
Suresh Jayaraman (3)
Susanne Oberhauser (3)
Thomas Göttlicher (6)
Thomas Schraitle (26)
Togan Muftuoglu (3)
Tuukka Pasanen (36)
Will Stephenson (22)
YaST Team (90)