Home Home > Usability
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Archive for the ‘Usability’ Category

Highlights of YaST Development Sprint 69 & 70

January 31st, 2019 by

Almost two months has passed since our last sprint report but, except during the Christmas break, the team has been quite busy working on some features and bugfixes for the upcoming (open)SUSE releases.

But a post describing all that we have done would be quite long :), so let’s try to highlight a few of them.

  • YaST got a security audit and, although no real security problems were found, we were asked to introduce some improvements.
  • Now it is possible to run the installer through PXE Boot without any local repository. Pretty specific but cool stuff!
  • We are in the process of revamping SUSE Manager Salt Formulas support in the YaST2 Configuration Management module. Do not be fooled by the name, it is not limited to SUSE Manager.
  • YaST icons are now included in the package were they are used. We hope it will make things easier for icon designers.
  • The Firewall module got support for creating firewalld custom zones.
  • Performance when reading huge /etc/hosts files has been greatly improved.
  • CD/DVD sources are always disabled after installation.

YaST Security Hardening

Our SUSE security team did a security audit for YaST. The good news is that there were no real security problems that you should be concerned about. Still, we did some hardening to make the code even more secure.

This might have caused some breakages in Factory / Tumbleweed because many places in the code were touched. We apologize for any inconveniences that might have caused; but we are sure you prefer YaST to be more secure.

Most changes were centered around calling external commands, which YaST does a lot. Since YaST is running with root permissions in most cases, we want to make sure that this is as secure as possible. If you find any problems with it, please write bug reports.

What exactly we did and how we did it is summarized here: YaST Security Audit Fixes: Lessons Learned and Reminder

Installing via PXE Boot without any Installation Repository

In data centers and other big-scale enterprise environments, administrators rarely install new software via removable media such as DVDs. Instead, administrators rely on PXE (Preboot eXecution Environment) booting to image servers.

Installing Linux Enterprise in such environments typically requires two auxiliary servers in the local network:

  • The DHCP/TFTP server providing the minimal system used by PXE to execute the installer.
  • A server making the SLE DVD repository accessible in the local network via FTP, HTTP or any similar protocol.

Very often, the second one is more a requisite imposed by the installer than something really useful. In most cases, the system been installed will be registered in the SUSE Customer Center (or any of its proxy technologies like SMT or RMT) and will get all the software from there. Thus, we decided to save the administrators the extra steps of downloading the SLE ISO image and setting up an install server to serve the content of that ISO, for cases in which that was really not needed.

But the repositories are not only used to get the software been installed in the final system. As explained often in this blog, we have a single installer for all the products and flavors of SUSE and openSUSE, as different as the installation process looks for all of them. That generic installer uses the information in the installation repository to get its own configuration. That includes the available products (and its corresponding system roles), the steps and options to present to the user, the desired partitioning setup and many other aspects. Without that information, the installer is basically a musician without his score.

Starting with SLE-15-SP1, it will be possible to use the boot parameter NOREPO=1 to tell the installer to not expect (and more important, to not require) any local repository in the DVD or in the local network. In that case, the installer will be able to proceed up to the registration screen and get the information for the upcoming steps of the installation from the registration server. In the openSUSE case (where registration makes no sense), it will be able to reach the screen that allows to add more repositories.

Another step (and certainly not an easy one) to improve the installation experience for our users. Data center administrators, enjoy! 🙂

Revamping SUSE Manager Salt Formulas Support

Back in 2017, the YaST Configuration Management module got support to handle SUSE Manager Salt Formulas as part of a Hack Week project. If you do not know what this feature is about, you might be interested in checking the Forms are the Formula for Success presentation or the Hack Week project follow-up post.

Since then, the forms specification has evolved quite a lot and YaST support was basically outdated. So on November 2018 we started to work in order to bring the missing pieces to the YaST module. Basically, we rewrote the forms support and, although there are still rough edges, we are pretty close to release a new version with up-to-date support for this powerful feature.

Screenshot of how the dhcpd formula looks like

Managing Custom Zones Definitions in YaST Firewall

The new YaST UI for configuring firewalld was announced in the report of the sprint #63 (four months ago… time flies!) and, since then, we have continued improving it.

firewalld ships with some predefined zones. Although it covers most users needs, in addition it allows the user to define custom zones. During the last sprint we have added support in the new UI and also in AutoYaST to manage custom zones.

YaST2 Firewall custom zones definition dialog

During the development process some problems detected in the AutoYaST configuration were addressed too.

Updated YaST Branding and Icon Handling

In the past the YaST icons were included in the yast2-branding-openSUSE (openSUSE) and yast2-theme-SLE (SUSE Linux Enterprise) packages. The standard YaST icons were included in these packages, the standard YaST modules did not include any icons.

However, the disadvantage for the icon designer was that it was not clear which icons were really used.
If you wanted to update the icon theme you could potentially do a lot of useless work because some icons were not used anymore.

Now the icons are included in the respective YaST package, if the package is dropped the icon is dropped as well.

The package manager UI includes compiled-in fallback icons. That means if the branding package is broken or the icon files are accidentally deleted from disk then it will be still usable for emergency recovery.

The branding still works, the vendor can still provide specific icons which will override the included ones. So it is still possible to have a different look in the openSUSE and SLE products.

YaST2 Control Center new branding Screenshot

A big thank you goes to Stasiek Michalski and Noah Davis from the community who did the changes in the YaST code, designed the new icons and did a lot of cleanup!

Improving Performance when Loading Huge /etc/hosts Files

It might happen that you need to maintain a huge /etc/hosts file, especially when dealing with ads blockers. Such file with thousands of lines took an incredible amount of time to get loaded into YaST2. On some configurations it could even happen that loading a /etc/hosts with around 10.000 lines freezes the system completely. After some refactoring in YaST2 Host module, the performance has been significantly improved and loading a file with 10.000 lines now takes approximately 30s on the same configuration where it crashed before.

Disabling CD/DVD Repositories After Installation

If you install your system from a CD/DVD source it usually happens that this repository was not available for whole live of the system. In some use cases this was only uncomfortable because of some warnings but, in other cases, it caused serious complications, for instance, when trying to do a migration.

In the past, under some circumstances, those repositories were already disabled. But, from now on, they will be disabled always in order to avoid unwanted side effects.

Closing Thoughts

That’s all for the first report of 2019. In case you are wondering, the plan is to stick to the plan of publishing a report after each sprint, so expect the next one in about two weeks.

However, we recently had to migrate from the so called GitHub Services (now deprecated) to GitHub web hooks, so you might get an extra blog post about that very soon.

Stay tuned!

Highlights of YaST Development Sprint 64

October 9th, 2018 by

Another two weeks of development, another report from the YaST team. During this sprint, we have been working to improve the usage and installation experience in many areas, including but not limited to the following.

  • Improvements in several areas of the Partitioner.
  • More informative Snapper.
  • Better integration of the new Firewall UI with AutoYaST.
  • Improvements in roles management and in the roles description.
  • Better support in YaST Firstboot for devices with no hardware clock, like Raspberry Pi.

Let’s dive into the details

Changes in the Partitioner UI to Unleash the Storage-ng Power

We have explained already in several previous posts how we were struggling to come up with a set of changes to the user interface of the Partitioner that would allow to expose all the new functionality brought by storage-ng, while still being familiar to our users and fitting in a text console with 80 columns and 24 lines.

We finally implemented the interface described in this gist, which fits into a 80×24 text console and allows all kind of operations. Check that document for more info about the behavior and its rationale.

But what does “all kind of operations” mean? For example, it means it’s possible to start with three empty disks and end up creating this complex setup using only the Partitioner.

Complex storage setup

  • In that example, /dev/md0 is an MD RAID defined on top of two partitions and formatted as “/”. Nothing impressive here so far.
  • /dev/md1 is an MD RAID defined on top of a combination of full disks and partitions. Using disks as base for a RAID was not possible in the old Partitioner.
  • Even more, /dev/md1 contains partitions like /dev/md1p1 and /dev/md1p2, another thing that the old Partitioner didn’t allow to configure.
  • /dev/volgroup0 is an LVM VG based on one of those MD partitions, allowing to combine the best of the MD and LVM technologies in a new way.
  • Last but not least, /dev/sdc is a disk formatted to host a file-system directly, with no partitions in between (also a new possibility).

The general approach of the new UI is described in the linked document. But since an image is worth a thousand words (and an animation is probably worth two thousands), let’s see how some part of the process to create the complex setup described about would look in a text console.

This is how you can now directly format a disk with no partitions.

Formatting a disk

Playing with the partitions of a disk is also a good way to get a feeling on how the buttons are now organized and how they dynamically change based on which row is selected in each table. Click on the following image to animate it and see those views in action.

Playing with partitions

And for a full experience of completely new stuff. Click on the image below to see an animation showing the whole process of creating an MD RAID on top of two full disks and then creating partitions within the resulting RAID.

Creating a partitioned RAID

But although the text mode is the limiting factor to design a YaST UI, many users install their systems and use the Partitioner in graphical mode. For those wondering how the reorganized buttons look in that case, here are some screenshots of the installation process of the upcoming SLE-15-SP1 (static screenshots this time, we already had enough animations for one post).

Managing RAIDs with the new Partitioner UI

Managing Partitions with the new Partitioner UI

Displaying Bcache Devices Consistently in the Device Graphs

Surely most Partitioner users have recognized the style of the visual representation used above for the complex example setup. As you know, the Partitioner offers similar representations in the “Device Graphs” section, both for the original layout of the system and for the target one.

After adding support for Bcache to the Partitioner we detected a small but annoying problem in those graphs. The caching devices were using their UUID as labels, which had two drawbacks.

  • It was too long.
  • It’s not known in advance for “planned” cache sets (i.e. sets that will be created after going forward in the Partitioner), which resulted in boxes with no labels

So know we use a fixed “bcache cache” label for all cache sets, which looks like this.

New label for cache sets in the Device Graph

As opposed to the old way with empty boxes.

Lack of labels in the old Device Graphs

Adding and removing Bcache devices

And since we mention the Bcache support in the Partitioner, it’s worth noticing that the implementation continues moving forward at good pace. During this sprint we implemented a first version of the operations to add a new Bcache device and to delete it.

When adding a new device, the only options that can currently be defined is which devices to use to construct it. But the next sprint has started and you can expect more options to be supported in the near future.

Creating a new Bcache device

When the Bcache device is created, then it can be formatted, mounted or partitioned with the same level of flexibility than other devices in the system. So soon (after the usual integration and automated testing phases) Tumbleweed users will be able to use the YaST Partitioner to test this exciting technology.

Of course the operation to delete a Bcache device offers the usual checks and information available in other parts of the Partitioner, like shown in the following screenshot.

Deleting a bcache device

Both screenshots are taken with an updated version of the installer of the upcoming SLE-15-SP1, since this functionality will be available in such distribution and, of course, also in openSUSE Leap 15.1.

Snapper: Show Used Space for each Snapshot

As those following our blog already know, the YaST Team is also somehow responsible for the development and maintenance of Snapper, the ultimate file-system snapshot tool for Linux. And Snapper has also received some usability improvements during this sprint.

For systems with btrfs and quota enabled, the output of “snapper list” now shows the used space for each snapshot. The used space in this case is the exclusive space of the btrfs quota group corresponding to the snapshot.

# snapper --iso list
Type   | # | Pre # | Date                | User | Used Space | Cleanup | Description      | Userdata     
-------+---+-------+---------------------+------+------------+---------+------------------+--------------
single | 0 |       |                     | root |            |         | current          |              
single | 1 |       | 2018-10-04 21:00:11 | root | 15.77 MiB  |         | first filesystem |              
single | 2 |       | 2018-10-04 21:19:47 | root | 13.78 MiB  | number  | after install    | important=yes

For more details about this change, its advantages and limitations, check the new post at the Snapper blog.

Simplified Role Selection

The role selection dialog in SLE-15 is always displayed in the installation workflow. However, it does not make much sense to display it if there is only one role to select. When you do not register the system and do not use any additional installation repository then in the default SLES-15 installation you can see only the minimal system role.

Selecting one out of one roles

In such case you cannot actually change anything as the only role is pre-selected by default and the only thing which you can do is to press the Next button.

Therefore we improved in for SLE15-SP1, if there is only one role to select then the role is selected automatically and the dialog is skipped.

In addition to that, many of the role descriptions have been adapted and simplified to, hopefully, be more clear.

YaST Firstboot in devices with no hardware clock

SLE and openSUSE can be installed on a great variety of devices, including some system that doesn’t include a hardware Real Time Clock, like the popular Raspberry Pi. That means the usual mechanism to establish the current date and time (using the hwclock command) fails in such devices. That general problem was detected during the usage of YaST Firstboot to configure new devices.

So now YaST detects situations in which there is no Real Time Clock and uses the date as an alternative to set the date and time. This fix, already submitted to openSUSE Tumbleweed, will be available in all upcoming versions of SLE (like SLE-12-SP4 and SLE-15-SP1) and openSUSE Leap.

Better integration of the new Firewall UI with AutoYaST

On the previous report we anticipated the new UI we are building for configuring Firewalld from YaST. During this sprint we have been focusing on some aspects that need to be finished before we can release this new functionality.

Now this UI can be invoked from the AutoYaST module in YaST, meaning it can be used to import and then fine tune the current configuration of the system so it can be exported to an AutoYaST profile.

And since we are already in animation mood, check how the new UI can be used to define an AutoYaST profile.

Using the Firewalld UI from the AutoYaST module

Very soon the whole functionality will be ready for prime time and we will release it together with a separate blog post to explain all the details.

Stay tuned

We are already working on the next sprint, with special focus on AutoYaST, on Snapper and on improving the installation experience in several scenarios. As mentioned above, it’s likely that you will get more news from us (about the new Firewalld support) even before that sprint is finished.

But if you can’t wait for more news, don’t hesitate to visit us on our Irc #yast channel on Freenode. Otherwise, see you here again soon.

Highlights of YaST development sprint 40

August 10th, 2017 by

Doubtlessly, these are pretty exciting times for the YaST team. The merge of the new storage layer into the main codebase is around the corner and we are working on other features that will debut on the next open(SUSE) major release. So let’s summarize what happened during the last sprint.

New storage layer is coming

As you may already know, the YaST team has invested a lot of time and effort preparing our storage layer for the future and we have started to merge the new layer into the main code base during the current sprint. But that’s something for our next report, right? By now, we will just focus on the stuff that got added and fixed during the last two weeks.

Storage reimplementation: BIOS RAID support

libstorage-ng, the low level library in which our new storage layer relies on, got support for BIOS RAID (handled in Linux via MD devices). Now, YaST could take advantage of such a feature to allow the installation of open(SUSE) systems on this kind of devices, including the bootloader.

BIOS RAID support

Storage reimplementation: managing BtrFS subvolumes in new Expert Partitioner

The new Expert Partitioner is getting a lot of attention these days and, during sprint 40, it got initial support for Btrfs subvolumes management.

Btrfs subvolumes list

Now, when you select the BtrFS section in the general menu placed on the left, all BtrFS filesystems are presented allowing you to edit its subvolumes through a dialog which contains the list of subvolumes that belongs to the filesystem. Apart from the usual stuff, like adding and deleting subvolumes, it is also possible to set the noCoW property when you are creating a new one.

Adding/Removing Btrfs subvolumes

However, some features are still missing. For instance the partitioner will not prevent you to create a subvolume which is shadowed by an already existing mount point. Consider the current implementation as the first step towards a really cool Btrfs subvolume handling.

Dropping SUSE tags support

During installation, YaST uses a mechanism known as SUSE tags as source of information for media handling. For instance, a /content file contains information about the product, languages, etc. Additionally, information like release notes or the slide-show texts are stored in the installation media.

Some time ago, SUSE decided to drop SUSE tags and use RPM metadata and packages to store all that information. To make it possible, the installation media would use REPOMD repositories.

Obviously, YaST needs some adaptation. As a first step, support for the /content has been dropped, cleaning up some old and even unused code.

In the upcoming sprints, YaST will be adapted to retrieve licensing, release notes, etc. from RPM repositories and packages, which is also an opportunity to do some refactoring and to improve test coverage.

AutoYaST support for add-on products on same installation media

Nowadays YaST supports having add-on products on the same media than the base product. The problem is that the EULA for those products is displayed too early, even before AutoYaST had been initialized at all.

To solve this issue, now the EULA acceptance of included add-ons is performed at the same time than other add-ons which are not included in the installation media. As a side effect, now the user needs to define those add-ons on the AutoYaST profile in order to handle the EULA acceptance.

Bug squashing and 80×24 terminals

As developers, we enjoy working on new features and, of course, we are committed to fix critical bugs as fast as possible. But there are many small (and annoying) bugs out there that deserve our attention. Additionally, there are several bug reports that are no longer valid (the bug was fixed, it is not reproducible, it is a duplicate, the affected product is not supported anymore, etc.). In order to reduce the list of open issues, the team decided some sprints ago to reserve one day to do some bug squashing.

Among the bugs we closed during this sprint, we would like to highlight a usability problem in YaST services manager. Bear in mind that, along with the graphical interface, YaST ships a text based one which is supposed to fit in good old-fashioned 80×24 terminals. That’s an interesting constraint when you are designing interfaces for YaST.

Needlessly to say that, from time to time, we get a bug report about some element that just do not fit. In this case, YaST services manager had a problem when the service name was too long as you can see in the screenshot below.

Too long service name

Now, if there is not enough space, the name will be truncated and the rest of the information will be shown in an proper way.

Truncating a too long service name

Do not miss the next report!

As you may have noticed, a lot of interesting things are currently happening in the YaST world and more cool stuff is about to come. So you should not miss our next sprint report.

By now, enjoy openSUSE 42.3 (you already upgraded your system, right?) and see you in two weeks.

Highlights of YaST development sprint 32

March 22nd, 2017 by

To make sure you didn’t missed us too much, in our latest blog post we summarized all the YaST-related projects worked during Hack Week 15. But after all the fun, it was time for… more fun! So let’s take a look to what the team has delivered on this first sprint after Hack Week 15.

Storage reimplementation: encrypted proposal without LVM

One of the known limitations of the current installer is that it’s only able to automatically propose an encrypted schema if LVM is used. For historical reasons, if you want to encrypt your root and/or home partitions but not to use LVM, you would need to use the expert partitioner… and hope for the best from the bootloader proposal.

But the new storage stack is here (well, almost here) to make all the old limitations vanish. With our testing ISO it’s already possible to set encryption with just one click for both partition-based and LVM-based proposals. The best possible partition schema is correctly created and everything is encrypted as the user would expect. We even have continuous tests in our internal openQA instance for it.

The part of the installer managing the bootloader installation is still not adapted, which means the resulting system would need some manual fixing of Grub before being able to boot… but that’s something for an upcoming sprint (likely the very next one).

Improved add-ons listing for SLE12-SP1

The dialog in SLES-12-SP1 for selecting the add-ons after registering the system was originally designed just for a small list of add-ons. Unfortunately (or fortunately, depending on how you look at it), the number of add-ons grew over the time and it exceeded the original limit for the text mode UI.

The equivalent screen in SLE-12-SP2 is not affected by the problem because it uses a different layout with scrollable list. But the SP1 dialog looks like this.

Broken add-ons list in SP1

If you look carefully at the screenshot you will see that the Web and Scripting Module is missing in the list and the Back, Next and Abort buttons at the bottom are also not displayed.

The fix decreased the size of the Details widget and allowed displaying more items in each column. Now there is even free space for three more add-ons.

Fixed addons list in SP1

Moreover the dialog is now dynamic and checks the current size of the screen. If there is enough free space then the list is displayed in one column so the labels are not truncated and the Details widget size is increased back to the original size.

Add-ons list in SP1 with enough space

Storage reimplementation: Btrfs subvolumes

The management of subvolumes is one of those features that make Btrfs rather unique and that need special handling when compared to more traditional file systems. That was indeed one of the several reasons to rewrite libstorage – Btrfs subvolumes never fully fitted the philosophy and data structures on the old (current) libstorage and yast2-storage.

In this sprint we introduced support for subvolumes in libstorage-ng from the ground up, taking into consideration all the specificities, use cases and scenarios found in the past. And, hopefully, in a way that is also prepared for whatever the future brings.

The new functionality is already working and tested and it’s included in the latest versions of libstorage-ng, but is still not used in the proposal or any other part yast2-storage. You will have to wait another sprint to see more visible results. At least if “more visible” means screenshots. Meanwhile, if you like images you can always enjoy the graphs generated from the internal structures managed by libstorage-ng.

Internal subvolumes representation in libstorage-ng

Storage reimplementation: system upgrade

The new storage stack has been able to install an openSUSE system for quite some time already. While we keep improving that area, the next challenge was to make the upgrade from a previous openSUSE version also possible using our testing ISO.

That implies scanning the hard disks looking for previous installations, allowing the user to select one, mounting the corresponding partitions or LVM volumes, performing the update of every package and doing some final tasks like updating the bootloader configuration.

Following the iterative approach encouraged by Scrum, we focused in the first three steps, which is something that a user (or openQA, for that matter) can test and verify. So now we are able to detect and list pre-existing systems and start the upgrade process on the selected one. And we have automated tests in openQA to ensure it works across all the combinations of partition-based vs LVM-based layout and UUID-based vs name-based fstab file.

Add-ons can define new system roles

YaST is pretty customizable when it comes to adapt/modify the installation workflow. Among other things, add-ons are allowed to adapt the workflow (adding/removing steps), define new proposals, etc. And starting now, they can also define new system roles.

Let’s see an example of adding a new mail server role:

<update>
  <system_roles>
    <insert_system_roles config:type="list">
      <insert_system_role>
        <system_roles config:type="list">
          <system_role>
            <id>mail_role</id>
            <software>
              <default_patterns>base Minimal mail_server</default_patterns>
            </software>
          </system_role>
        </system_roles>
      </insert_system_role>
    </insert_system_roles>
  </system_roles>
</update>

<!-- Don't forget to add the texts -->
<texts>
  <mail_role>
    <label>Mail Server</label>
  </mail_role>
  <mail_role_description>
    <label>• Software needed to set up a mail server
• No production ready yet!</label>
  </mail_role_description>
</texts>

And now let’s see how it looks:

A role added by an addon

Which leads us to the next section…

The list of roles becomes responsive in text mode

A really nice thing about YaST is that it’s able to run in textmode, so you don’t need a graphical interface to install or configure your system. As a consequence, YaST developers need to keep certain limitations in mind when working in the user interface.

Now that add-ons can add new system roles, we noticed a potential problem in the dialog selection screen: we eventually will get out of space if more than one system role is added. So we decided to improve how system roles are displayed to make them fit in a 80×25 mode (that is, only 25 lines of text). Let’s see the changes with some examples.

This is how the screen looks by default, with a reasonably small set of roles.

Default system roles list

If the system detects there is no space to present all the information in such a convenient way, it removes all the spaces so at least the information is all there, even if it looks a little bit packed.

Roles list with no extra space

If even that is not enough, the extra descriptions are omitted, which gives us way more room.

Compact roles list

If roles don’t fit even without the descriptions, the introductory text will be also omitted which means we can present up to eighteen (yes, 18!) roles in the screen.

Storage reimplementation: guided setup mock-up

As explained in several previous reports, we are collaborating closely with SUSE UX experts to design the revamped interfaces of the installer’s partitioning proposal and the expert partitioner. We already showed you the document we used as a base to discuss the partitioning proposal, including the conclusions, and the first very simple prototype of the so-called Guided Setup.

During this sprint, that collaborative effort focused on defining exactly how every step of that wizard should work and look like. The goal was to get some interface mock-ups to be used as starting point for the upcoming sprint. More than ever, a picture (well, four of them) is worth a thousand words.

First step of the guided partitioning setup mock-up

Second step of the guided partitioning setup mock-up

Third step of the guided partitioning setup mock-up

Fourth step of the guided partitioning setup mock-up

Prevent the installation of CaaSP if Btrfs snapshots are not possible

CaaSP is a single purpose system, and having snapshots enabled is essential. So there’s now a check in place that will simply prevent you from going on with the installation if snapshots are disabled (for example, if the disk is too small).

Blocked CaaSP installation

Storage reimplementation: better handling of /etc/fstab and /etc/cryptab

For the new storage stack, we refactored the classes to handle /etc/fstab. While this would normally not be anything to write much about, we included intelligent handling for existing comments based on this standalone GitHub project.

This means that existing comment blocks at the start and at the end of the file remain untouched, and comments before any content entry remain attached to that entry; i.e. when that entry is moved around in the file (e.g. because of mount dependencies), that comment is moving along with the entry it belongs to. While this is not 100% fool proof, it is much better than the usual strategy to simply discard such comments when the file is rewritten.

Quite some adaptations and bugfixes for CaaSP

As you already know from previous reports and other sources, a considerable part of SUSE’s development firepower is focused on building the upcoming CaaSP. As part of that heavy development process, the YaST team invested a significant part of the sprint adapting YaST for CaaSP and fixing bugs introduced by previous adaptations. A large collection of changes here and there that are hard to summarize here but that help CaaSP to be a couple of steps closer to the final goal.

Keep it rolling!

We have already planned our next sprint which will hopefully bring more features to the new storage stack, CaaSP-related improvements, a surprise about AutoYaST and more stuff. And, of course, it will be followed by its corresponding report.

So see you in three weeks. Stay tuned and have a lot of fun!

Highlights of YaST development sprint 30

February 3rd, 2017 by

This is our first post in 2017 and looks like we must start apologizing. In our previous post we promised news about this blog, but the administrative part slowed us down and the surprise is still not ready. On the bright side, we have quite some news about YaST. So let’s go for it!

One-click system installation for CaaSP

As you may know, SUSE has been working on making containers easier, with SUSE Container as a Service Platform. We have referred to it in several previous posts using the CASP acronym, although nowadays the correct one is CaaSP (maybe we could sell the new “a” as a shiny feature 😉 ).

Part of this upcoming product is also an interactive installation option using the good old YaST. CaaSP uses a limited subset of the SLE possibilities and we wanted to make the installation simpler to reflect that. So we reduced the number of dialogs you have to click through… to one!

One-click installation in CaaSP

As you can see, it is at the expense of stuffing the screen with more widgets than usual. Still, the only part where you must make a decision is the root password.

We expect that most of the CaaSP installations will actually not use this, because they will be done automatically with AutoYaST. But still this should be useful when you are only getting started.

Refining the read-only installation proposals

It was possible to make a proposal “read-only” for some time already. However, its black-and-white logic was not sufficient for some use cases. So, it was redesigned and you can mark a proposal hard read-only or soft read-only. The difference is that users will never be able to change hard read-only proposal. However their will be able to modify a soft read-only proposal if the proposal reports an error. It can be useful e.g. for error recovery in software proposal. It has been implemented originally for CaaSP, but it will be available for SLE-12-SP3 and Leap 42.3 too.

Installing directly from a repomd repository

When you install (open)SUSE you have up to now needed a specially prepared install repository. In addition to the repository with the RPM packages, it needed a bunch of specially prepared files containing the installation system and our beloved YaST installer.

That’s all gone now!

You can now point the installer to any plain repomd repository. For this to work you have to point the installer to the repomd repository and to the installation system (they can be completely separate now).

For example:

install=http://download.opensuse.org/tumbleweed/repo/oss instsys=disk:/boot/x86_64/root

In that example, we install Tumbleweed from the openSUSE website and use the installer from some local media (maybe the NET iso).

To make things even easier there is now a regular package (tftpboot-installation-openSUSE) that contains the installation system and some sample config files.

Check out this linuxrc documentation for technical details.

Storage reimplementation: removing stones from the installation path

In our latest post, we presented the dedicated openQA instance contantly testing the new storage layer implementation. It still doesn’t run exactly the same tests than openQA.opensuse.org because not all technologies and operations are supported yet in the new yast2-storage. But now we are a couple of steps closer to run the full-blown tests also in our dedicated instance.

During this sprint, the partitioning proposal gained the ability to deal with disks not containing a partition table in advance (it always proposes to create a GPT one in that scenario) and the software selection proposal learned how to use the new storage API, so it can properly inspect the system and the associated error pop-ups are gone from the installation workflow.

More power to the system roles

We keep extending the capabilities of the system roles, now with the ability to specify some systemd services to enable. As the roles can define which software gets installed in the system, it made sense for them to also be able to specify the desired status for the services included in that software

For example, it would be possible for a given product (let’s say a customized openSUSE) to define a “static web server” role. Choosing that role during installation would result in a system with a HTTP server already installed and enabled, so the user just need to copy the files to be served into the right directory.

Expert partitioner is now less restrictive with encryption

Setting up an encrypted LVM was always pretty easy when using the automated storage proposal – simply select “encrypted LVM” at the proposal settings and you are done.

But doing that manually was almost impossible: The expert partitioner wouldn’t allow any of the system mount points (“/”, “/usr”, “/var”, …) on any encrypted partition, and it also wouldn’t allow to encrypt, but not format a partition of type “LVM” for use as an LVM physical volume.

Both restrictions are now lifted; you can now create an LVM physical volume with encryption, or you can do the encryption layer on the logical volume if you prefer. And you can create an encrypted plain partition with a filesystem directly on it without LVM.

Over the years, Grub2 learned how to do that, so you don’t even need a /boot partition anymore. For the time being, you’ll need to enter the encryption password twice, though: once at the Grub2 prompt and once later at the graphical console so systemd can mount those filesystems. Our base system developers are working on a secure solution to avoid that.

Migrate Travis CI to Docker

That’s actually not a change in YaST itself, but in its development infrastructure. Still, we believe it would be interesting for the average reader of our blog.

So far we used Travis CI for building and testing the commits and pull requests at GitHub. But the problem was that by default Travis runs Ubuntu 12.04 or 14.04 at the build workers. That had several drawbacks for us, since compiling and testing YaST on Ubuntu is not trivial and the result is not always 100% equivalent to openSUSE. All this meant extra maintenance work for us.

Fortunately Travis allows using Docker containers at the workers and that allows using basically any Linux distribution. This sprint we spent some time converting the Travis configuration to use a dockerized openSUSE Tumbleweed at Travis.

From Github to Travis thanks to Docker

The work was successful, we switched all YaST modules to use this new builds and the result is already paying off at several levels, although it took us over 100 pull requests (all of them manually tweaked and reviewed) to make it happen.

The current solution is documented and we had also a short internal presentation about this change. The notes from the presentation are available here.

Improved continuous integration for Snapper

We also enabled Travis integration with Docker for Snapper. As you may know, Snapper is a portable software that has always offered packages for many Linux distributions in the filesystems:snapper OBS repository.

So we took the continuous integration one step further and enabled Travis builds on more distributions, currently we build for openSUSE Tumbleweed, openSUSE Leap 42.2, Fedora 25, Debian 8 and Ubuntu 16.10. You can see an example build here or more details in the documentation.

Example build result of Snapper at Travis

That means we can ensure that the package still builds on all these distributions even before merging a pull request!

Better integration with systemd for YaST Services

Systemd recognizes many possible states for a service beyond the classic Unix enabled/disabled and running/stopped, and that list of possible states grows with every systemd release. In the past YaST have had some issues displaying the services status.

Now the problems are fixed by delegating to systemd the conversion from the concrete state to the good old known Unix equivalent. So the user now gets more precise information about all services running on the system.

Storage reimplementation: redesigning the installation user experience

In the latest post we showed you the document we were using as a base to discuss the new expert partitioner UI with usability experts. Now it was turn for the proposal settings dialog. We collected the current state, had a very productive discussion and ended up with a proposal for a new interface. You can check the resulting document covering all that.

As mentioned, the SUSE UX experts will use that document as a starting point to design the final interface. But we want the process to be as open as everything around YaST, so feel free to provide feedback.

Reading product renaming information from libzypp

When performing an upgrade, YaST needs to know whether a product is renamed or replaced by a another one. For example, in the past, the Subscription Manager Tool (SMT) lived in its own product but it’s included in SUSE Linux Enterprise 12. So YaST needs to know that the suse-smt product was just replaced by sles.

This information is usually provided by the SUSE Customer Center (SCC). But what happens if, for example, we are performing an offline upgrade? Until now, YaST used a hard-coded fallback list.

From now on, before falling back to such a list, YaST will ask libzypp for that information so, hopefully, it will avoid some problems while upgrading extensions and it will reduce the hassle of maintaining a hard-coded list.

Storage reimplementation: Making sure to install storage-related packages

The YaST storage subsystem has been taking care about storage-related software packages for a long time. For example, when a specific filesystem type like Btrfs or XFS is used by the system, we need to make sure that necessary support packages like btrfsprogs or xfsprogs are installed.

Figuring out what features are used is now done by the new libstorage. In this sprint, we created one Ruby class that maps those features to respective packages and one class that handles package installation itself. One interesting technical aspect is how Ruby introspection capacities are used to avoid duplicating the list of defined features from the C++ part (i.e. libstorage).

Power the chameleon

Apart from all those changes in YaST, and many more we have not included in this summary, we have something else to celebrate. On February 1st the YaST Team at SUSE has grown with the addition of a new member, Iván, who will allow the project to evolve even faster and better… and that will not be the last announcement in that direction, so stay tuned.

But don’t forget you can also help YaST, and openSUSE in general, to keep moving. This week we added several ideas for Google Summer of Code projects to the openSUSE mentoring page, including one idea to contribute to YaST. Do you have a better plan for this summer?

See you in less than three weeks, since the next sprint will be slightly shorter due to Hack Week 15.

Highlights of YaST development sprint 29

December 22nd, 2016 by

It’s Christmas time and since (open)SUSE users have been nice, the YaST team brings some gifts for them. This is the result of the last development sprint of 2016.

As you may have noticed, in the latest sprints we have been focusing more and more in making SUSE CASP possible. That’s even more obvious in this last sprint of the year. For those that have not been following this blog recently, it’s probably worth to remember that SUSE CASP will be a Kubernetes based Container As a Service Platform.

But our daily work goes beyond CASP, so let’s take a look to all the highlights.

More improvements in the management of DHCLIENT_SET_HOSTNAME

In the previous report we presented the changes introduced in yast2-network to make the configuration parameter DHCLIENT_SET_HOSTNAME configurable in a per-interface basis.

One of the great things about working in an agile an iterative way, presenting and evaluating the result every three weeks, is that it allows us to detect room for improvements in our work. In this case we noticed some discrepancy in the expectations of Linuxrc and yast2-network and also some room for improvement in the code documentation and in the help texts.

Thus, we used this sprint to refine the work done in the previous one and tackle those problems down.

Improved error message

Ensure installation of needed packages

Another example of iterative development. We already presented in the report of the 26th development sprint a new mechanism to detect when the user had deselected during installation some package that was previously pre-selected by YaST in order to install the bootloader. Since the new functionality proved to work nicely, we decided to extend it to cover other parts of the system beyond the bootloader.

The software proposal now contains an error message including a list of missing packages or patterns, in case the user deselects some needed items.

Warning about missing packages

After clicking the Install button the installation is blocked, the user must resolve the problem either by selecting the packages back or by adjusting the respective YaST configuration (e.g. do not install any bootloader and disable the firewall).

Blocking an incomplete installation

Rethinking the expert partitioner

May we insist one more time on the topic of using Scrum to organize our work in an iterative way? 😉 As our usual readers should already know, we structure the work into minimal units that produce a valuable outcome called PBIs in Scrum jargon. That valuable outcome doesn’t always have to be a piece of software, an implemented feature or a fixed bug. Sometimes a document adds value to YaST, specially if it can be used as base to collaborate with people outside the team.

Our readers also know that we are putting a lot of effort in rewriting the whole storage layer of YaST. That also implies rewriting the most powerful tool known by humanity to define partitions, volumes, RAIDs and similar stuff – the YaST expert partitioner.

It would be great if we could use the opportunity to make it both more powerful and more usable. You can take the first part for granted, but we are not so sure about our UI design skills. That’s why we wanted to have a base to discuss possible changes and alternative approaches with UX (user experience) experts. And we decided that it was worth to invest some time to create a document collecting the state of the art and some ideas for the future and to send that to SUSE experts in UX and to anybody with some interest in the topic.

Here you can find that fine piece of documentation. Take a look to that document if you want to peek into YaST developers’ mind. That’s the kind of stuff we discuss when we are about to start rewriting something… specially something that need to serve hundreds of different use cases.

And of course we would like to know your ideas or thoughts. We usually discuss this stuff at the public #yast IRC channel and at the yast-devel mailing list. But if you prefer so, you can simply open an issue at the repository hosting the document. Whatever works for you.

Rethinking yast2-network

But that was not the only documentation PBI finished during this sprint. Inspired by the first fruits of the storage layer reimplementation, we decided yast2-network also deserves a reincarnation.

As we did in the past with yast2-storage and libstorage, the first step is to collect as much information as possible about what can be currently done with the module and how it behaves in several situations, specially in tricky or complex scenarios. The outcome was three documents, one about the behavior during installation (installation.md), a second one about AutoYaST (autoinstallation.md) and another collecting general features (features.md).

CASP: merged dialogs for root password and keyboard layout

CASP is a product targeted to a quite specific use case with simplicity as a main priority. The installation process has been streamlined to a minimal set of dialogs to configure just the very basic stuff. Among other removed things, there is no step to configure the system language. That can be a problem when entering the root password (the only user that will be created during installation), since the language settings screen is normally also used to select the keyboard layout.

The implemented solution is shown in the screenshot below. As you can see, the keyboard layout and root passwords selections are merged into a single step. As a bonus, we made both widgets more reusable, opening the possibility to place the root password widget or the keyboard layout selection anywhere.

Keyboard layout and root password screen

Storage reimplementation: handling GPT disks in the installation proposal

After several sprints reporting small steps forward, in the 27th sprint we were happy to announce that our testing ISO for the new storage stack was fully installable under certain circumstances. As we reported, it worked in UEFI or legacy systems with the only requirement of having a pre-existing MBR partition table in the disk.

Now we can say it also works with GPT partition tables and even with systems with a mixture of both technologies.

Making the GPT scenario work was much harder that it sounds due to several factors, like the strange way in which parted handles partition types in GPT or some peculiarities in the way the space is distributed in such partition tables.

But now our test ISO can install a fully functional system in the four combinations of MBR/GPT partition table and UEFI/Legacy boot, as it can be seen in the next image.

Storage proposal in several scenarios

The storage reimplementation gets its own openQA instance

But there are better ways than screenshots to prove that something is working, even to prove it keeps working after future modifications. And in (open)SUSE we have one of the best tools for that – openQA.

We have always considered having the new stack tested in openQA as the first big milestone in its development (and we are finally there!) but we are aware that openQA.opensuse.org is already quite busy testing a huge combination of products, architectures and scenarios… even testing releases of openQA itself. Fortunately openQA is free software and can be installed anywhere so we created our own instance of openQA to test YaST stuff, specially the new storage layer.

So far, that instance is hosted in the internal SUSE network, which is enough for us to get continuous feedback about the changes we introduce. In addition to installing the new instance and configuring it to continuously grab and check the latest testing ISO, we had to introduce several changes in the ISO itself with the goal of keeping our tests as aligned as possible with the tests performed in the current Tumbleweed version by openQA.opensuse.org.

For example, we made sure the ISO was properly signed to avoid the need to always pass the insecure=1 boot argument. We also included several packages that were missing in order to make sure the ISO included all the software checked during the so-called MinimalX test and to make sure it shared the look and feel with a regular Tumbleweed, since many openQA checks are screenshot-based.

From now on, we can back every new feature with the corresponding integration tests, something crucial to ensure the quality of a piece of software meant to handle storage hardware.

Making Snapper work without DBus

As you may know, some YaST team members are also the main developers and maintainers of Snapper, the ultimate file-system snapshot tool for GNU/Linux systems.

Normally the snapper command line tool uses DBus to connect to snapperd which does most of the actual work. This allows non-root users to work with snapper.

There are however situations when using DBus is not possible and not being able to work in those situations was limiting Snapper’s usefulness. Now with the latest version all snapper commands support the –no-dbus option. This evolution is worth a blog post by itself… and, of course, we have it. To know all the details check this post at Snapper’s blog.

CASP (and beyond): improved roles

Do you remember the system roles feature introduced during development sprint 16 and improved in subsequent sprints? In case you don’t, let us remind you that system roles allow to define many settings of the installation just by choosing one of the offered roles. That’s only possible, of course, in products making use of that feature, like SLES.

For CASP we will have 3 different roles, as shown in the following screenshot.

CASP system roles

The main difference between these three roles is the selection of patterns to be installed. But apart from that, the Worker role will offer an extra step during installation allowing the user to specify the address of the so-called Administration Dashboard.

Configuration screen for the Worker role

That relatively small detail implied the development of a full new feature in the installer – the ability of a given role to define it’s own specific configuration, including the dialog to interact with the user. As expected from any other installation dialog, you can go back and forward without loosing the entered information. If the user goes back and selects a different role, then this additional dialog is not run again.

That new feature is, of course, not specific to CASP and could eventually be used in other products and roles. Just as a crazy example, openSUSE could decide to introduce a role called “NTP server”, running the YaST NTP server configuration right after the user selecting the role.

Other CASP related features

As already said, we have been focusing quite a lot on introducing features that are needed for CASP. It’s worth mentioning, in case it’s still unclear, that CASP will NOT ship its own adapted version of YaST. All the features introduced in the installer are in fact configurable and available for all other products as well. There is only one YaST codebase to rule them all.

Let’s briefly describe some of the introduced CASP-specific (at least for the time being) features.

CASP always uses Btrfs as filesystem for the root partition. At the end of the installation, the root btrfs subvolume will become read-only. All the other subvolumes will stay as read-write, as shown in this screenshot taken right after rebooting at the end of the installation process.

CASP subvolumes

It makes no sense to update from any existing product to CASP. Thus, CASP media should not show an “update” option when booting, even if it’s still possible for advanced users to pass the UPDATE boot parameter. Since we needed to modify the installation-images package, we took the opportunity to make the “update” option and other settings configurable in a per product basis and we unified SLES and openSUSE packages, so now they share a single branch in the source code repository.

CASP is targeted to big deployments extended all over the world. To make possible the synchronization of geographically distributed nodes, the UTC timezone is enforced in every CASP installation. Thus, we implemented support for products to enforce a given timezone in the installer. Take into account this is different from a default timezone.

Last but not least, it has already been mentioned that the CASP installation workflow will have very few steps. That also affects the screen displaying the installations settings summary. In comparison to a regular SLES, some options must disappear because they are not configurable and some other sections must be added because they are not longer presented as a separate previous step. So far, this is the appearance of the installation settings screen in the current CASP prototype.

Installation settings in CASP prototype

…and a surprise about the blog

We also prepared a Christmas gift related to the blog. The technical aspects are solved, but we are ironing out the administrative details. So you will have to wait until the next sprint report to see it in full glory. But, as the Spanish proverb says, “good things are worth waiting for”.

See you next year

That’s enough to report from our December sprint, we don’t want to bore you with every small bug fix. And talking about things that are worth waiting for, our next report will very likely be published at the beginning of February 2017.

That’s because we will put our Scrum process on hold during the Christmas session. We will restart it on the second week of the year, after the visit of the Three Wise Men. In several countries, it’s a tradition that the Three Kings bring gifts to the kids that have been nice, so let’s expect they bring us some new members for the team!

Highlights of YaST development sprint 28

December 2nd, 2016 by

November is over, Santa Claus elves start to stress and the YaST team brings you one of the last reports of 2016. Let’s see what’s new in YaSTland.

Harder to ignore installation warning

The “installation settings” summary screen usually reports some non-critical errors displayed as a red text. Although the installation can proceed despite those errors, they are usually serious enough to lead to problems. That’s why we decided to introduce a change to highlight them a little bit more, making them harder to overlook.

The following screenshot shows the newly introduced confirmation dialog, presented before proceeding with installation.

Preventing users to shoot their feet

Make DHCLIENT_SET_HOSTNAME configurable on a per-interface basis

But that’s not the only usability-oriented enhancement on this sprint. We also reworked a bit the network configuration dialog.

For home users is very common to use a fixed hostname -set during installation- for our beloved linux box. But in some circumstances it’s better to set the hostname of the machine dynamically using DHCP, something YaST has always allowed to do by just ticking a checkbox that used to be in the network configuration screen. See “Change Hostname via DHCP” below.

The old network settings screen

That checkbox used to modify the system-wide variable DHCLIENT_SET_HOSTNAME, which was fine in scenarios in which only one of the network interfaces was configured via DHCP. But with several network interfaces connected to different DHCP-enabled networks, some problems arose.

During installation, if network configuration is used, Linuxrc creates the ifcfg files with DHCLIENT_SET_HOSTNAME='yes' for all of the enabled or configured interfaces and this value has precedence over the global one.

So the main problem was that YaST only allowed us to modify the global variable and setting it to ‘no’ did nothing because it was enabled for some interface.

During this sprint we have fixed that and now the user interface offers the possibility of choosing which DHCP interface will be used to decide the hostname.

The new network settings screen

Apart from choosing one of the existing interfaces, the new setting can also be set to ‘no’ or to ‘any’. In any case, YaST will always configure the system-wide options and the interface specific ones in a consistent way, so the behavior is always predictable.

But YaST is not the only way of configuring the network, so it’s always possible to have an unpredictable configuration. Fortunately, those potentially problematic scenarios will be detected by YaST and reported to the user.

Detecting dangerous scenarios in network settings

Partitioning in CASP

In the previous report we already explained how are we improving the installer to support the definition of the ultra-streamlined installation process of SUSE CASP, the new Kubernetes based member of the SUSE family.

In this sprint we introduced several additional changes to enable a different partitioning approach, more guided and automatic than ever. In a CASP node it makes no sense to use the advanced settings offered by our storage proposal, like encryption or LVM. Moreover, CASP relies on Btrfs to provide some of its cool and advanced features, like transactional updates.

As a result, although the regular SUSE and openSUSE releases will keep offering all the current possibilities in the same way than ever, in CASP the partitioning step will be skipped and the automatically calculated proposal will be simply displayed in the installation summary.

The new CASP installation summary

Clicking on the proposal will allow to re-target the installation to a different disk (or disks) in a similar way than the regular installer, but the options will be more limited. Again, no easy way to use LVM, encryption, separate home or any file system type other than Btrfs.

Selecting the partitions in CASP, no proposal settings button

The expert partitioner is still available during CASP installation, but using it will show an extra warning, since it implies a much bigger risk than using it in a regular SUSE or openSUSE system.

Expert partitioner warning in CASP

Improved debugger integration

We have improved the Ruby debugger integration in YaST. So far you could start the debugger using the y2debugger=1 boot option or by setting the Y2DEBUGGER=1 environment variable. The new feature allows starting the Ruby debugger also later when the YaST module is already running.

Simply press the Shift+Ctrl+Alt+D keyboard shortcut (D as debug) and it will start the Ruby debugger. It works during installation and also in installed system (just make sure the byebug Ruby gem is installed).

Unfortunately this new feature works only in the Qt UI, the ncurses UI is not supported (currently it does not handle the debugging keyboard shortcut at all).

After pressing the keyboard shortcut the debugger window will pop up:

New debugger integration

Storage reimplementation: it’s alive!

It took us one more sprint than originally expected, but finally we can say the testing ISO for the new storage stack is fully installable.

We fixed the UEFI + MBR partition table scenario we already had almost working in the previous sprint (turns out it was not that broken in Tumbleweed after all) and we adapted yast2-bootloader to be also able to deal with legacy (i.e. no UEFI) booting using the new storage stack.

As a nice result, our testing ISO can be used to install a perfectly functional system in both UEFI or legacy systems with the only requirement of having a pre-existing MBR partition table in the disk. It only shows a couple of error pop-ups related to the calculation of the proposal of software to be installed, but nothing that would prevent you from replacing whatever operating system you have with a new shiny openSUSE-based experiment.

This milestone opens the door to start testing the new stack with openQA, the same system that helps to guarantee the robustness of all the recent SUSE and openSUSE versions.

Storage reimplementation: preparations for the storage proposal

Now that yast2-bootloader starts to be ready to work with the new storage stack in more and more scenarios, it’s time to adapt the only component that still complains during the installation.

In order to make that task doable during the next sprint, we invested some time in this sprint analyzing the interaction between the software proposal calculator and the old storage layer. The outcome was a small document detailing what needs to be adapted in the proposal and in the new stack. The perfect input for a task in the next sprint.

Help for power-users with short memory

Our beloved YaST is packed with magic tricks below the surface. Many of them are very useful to debug installation problems or to better understand how the YaST internals work. Unfortunately developers tend to not be that good at blindly memorizing stuff and the functionality is so well hidden that most newcomers would have hard times finding it… until now.

We have added a couple of new keyboard shortcuts to show a summary of all the advanced hotkeys, so now you only have to remember one key combination instead of a dozen of them. In both text (ncurses) and graphical (Qt) mode, it will be enough to press shift+F1 to get the advanced help displayed below. Since some terminal emulators could already use that combination, ctrl+D F1 can also be used in the ncurses interface as an alternative.

Advanced Hotkeys help dialog

Contributions keep coming!

As we have already mentioned in previous sprint reports, an important part of our daily job as open source developers is helping casual (and not so casual) contributors to bring their ideas and code into YaST and related projects.

This time that (hopefully not casual) contributor was Devin Waas, who wanted to improve the installation to make the life of cloud-lovers easier.

For cloud guys out there retrieving logs of a failed installation is was a huge problem. Now, thanks to Devin, all you need is a running a rsyslog server and you’ll be able to easily access your installation logs from there.

A drawing is worth a thousand words

As a matter of fact, the newest Tumbleweed release allows us to specify the IP address of a remote server from the bootloader through the “Loghost flag”. Linuxrc will take care of setting up a UDP broadcast for dmesg contents and YaST installation logs.

This is just a first step. Devin promised further improvements of our newly implemented remote logging system. And he codes better than he draws, so stay tuned!

Storage reimplementation: LVM-based proposal

As we already mentioned in previous reports, when we started to develop the partitioning proposal we first focused in the scenario of a partition-based proposal with one or several MBR-style partition tables. That looked like the most complex scenario due to the limited number of primary partitions, the alignment problems, the overhead introduced by the EBR (extended boot record) of every logical partition and so on.

A couple of sprints ago, we got that working so we started to work on the LVM-based proposal. It took a little bit longer than expected but now we are able to generate LVM-based proposals for almost every possible scenario. The goal was to have them working in our mocked test cases. So probably the new LVM-based proposal cannot still be used to install a fully functional system, but it is backed by a full load of tests that prove we can handle many situations, from trivial to really tricky ones… and believe us, things can get quite tricky if you mix logical partitions with their EBR overhead and LVM volumes with their PE size rounding and their metadata overhead.

Bugs, bugs, bugs

In this sprint we kept the already commented approach of making the fix of low-priority and small bugs part of the Scrum process. As a result we accounted for approximately 50 deaths of those annoying creatures.

Conclusion

Looking at the report, we could say it was a quite successful sprint. But to be honest we were aiming even higher. Quite some interesting PBIs (features or bug-fixes in Scrum jargon) were almost done at the end of the sprint. But following Scrum philosophy, we never blog about almost-done stuff.

Thus, if nothing goes wrong things will be even better in the next report in three weeks. So have a lot of fun trying the new stuff and stay tuned for more!

Highlights of YaST development sprint 26

October 20th, 2016 by

We did our best to keep you entertained during this development sprint with a couple of blog posts ([1] and [2]). But now the sprint is over and it’s time for a new report.

Squashing low priority bugs

One of the main reasons to adopt Scrum was to ensure we make a good use of our development resources (i.e. developers’ time and brains) focusing on things that bring more value to our users. In the past we had the feeling that many important things were always postponed because the developers were flooded by other not so important stuff. Now that feeling is gone (to a great extent) and we have a more clear and shared view of the direction of our development efforts.

But there is always a drawback. We have accumulated quite some unsolved low-priority bugs. That was an expected consequence, but still it was starting to feel wrong. On one hand, it makes us feel uncomfortable – replying “this will have to wait” so often is not nice, even if it’s for the shake a bigger goal. On the other hand, the amount of low-priority stuff was affecting the signal/noise ratio on Bugzilla, making harder to distinguish the important stuff.

So far, dealing with those low-priority bug was something that each developer did on his own, as permitted by the time dedicated to Scrum. In sprint 26 we decided to make that effort an explicit part of the process and to devote a significant portion of the sprint to it. As a result we closed or reassigned a total of 135 bugs that were just sitting in our list.

Yes, you did read it right. One hundred and thirty five bugs.

Storage reimplementation: our testing ISO can already destroy your data

On the previous sprint we already showed a screenshot of the installer using the new storage stack to calculate a partitioning proposal. Now the installer can go one step further. As you can see on this animation, the changes are now committed to the disk, meaning the system is actually partitioned, formatted and installed.

Installation with the new storage stack

The process is interrupted after installing the software, when trying to configure and install the bootloader. That was expected because yast2-bootloader has still not been adapted to use the new stack. First of all, because we wanted to leave some fun for sprint 27. But also because we used this sprint (26) to document all the requirements of yast2-bootloader in relation to the new storage stack. Now we have in place all the ingredients to cook an installable ISO.

Automatic update of translation files

Recently openSUSE has adopted Weblate to perform and coordinate the translation of the software and the project’s web pages. The openSUSE’s Weblate instance enables everybody (from dedicated translators to casual contributors) to take part in the process and makes possible to coordinate the translations of openSUSE with the ones for SUSE Enterprise Linux, boosting collaboration between the translators of both projects.

As YaST developers, is of course our responsibility to ensure that openSUSE’s Weblate contains always the latest English strings to be translated. Making our developer’s life easier sometimes not only brings advantages for us but also for our users. Until now, after each code change we had to keep in mind to trigger the translation process for every added or changed English text. Sometimes we were not quick enough so that some English leftovers remained in our awesome YaST when being used in one of the 20 languages where the translation is normally 100% complete.

Now we finally set up a Jenkins job to automate the process of triggering the translation update after code changes. This saves the developers some work and makes the update of translations even faster.

Looking at Weblate numbers, you can see we have 20 languages that are about 100% translated, another 20 that are translated more than 75% and 37 languages which are translated less than 75%. So we still need some help to bring all languages to 100% coverage. If you are willing to contribute, why not join our team of translators? Check out the localization guide to get in contact with the coordinators of your preferred language to learn about how to contribute with translations, reviews or by any other mean.

Ensure installation of packages needed for booting

We got some reports of systems not being able to work after the installation in scenarios in which the user had customized the list of packages to install. That happened because, although the bootloader component of YaST pre-selects for installation all the needed packages, the user can override that selection and manually disable the installation of those packages.

To prevent this situation, or at least to increase awareness of it, the installer now alerts in the summary screen (the last step before proceeding to installation) if some of the required packages is missing, as you can see in the screenshot below. We still allow the users to shoot their own feet if they insist, but now we warn them very clearly.

Warning about de-selected Grub2 package

Progress in the low-vision accessibility of the installer

During this sprint, we have been working to make the (open)SUSE installer accessible to people with low-vision impairment. We already blogged about it looking for feedback.

One of the new color modes available in the installer

In a few days, some changes will land in Tumbleweed:

  • Alternative style selection (color and font-sizes). Currently, we offer four options: default (no changes), high contrast (cyan/green/white/black), white on black and cyan on black. Those styles are just a proof of concept in order to test the code changes and, most important, to get feedback from you.
  • A long-standing issue, which prevented to switch to high-contrast mode during installation (shift+F4), has been fixed.

Style selection at the beginning of installation
Although we have made some progress, it is still an ongoing effort and we hope to release more improvements during the upcoming weeks.

Recover from broken bootloader configuration

There are situations in which YaST Bootloader is not able to read the system configuration. For example, when the udev device originally used by Grub2 is no longer available. In the past that leaded to YaST crashes, requiring a manual fix of the bootloader configuration files. Now YaST correctly detects the situation and offers the option to propose a new configuration with correct devices.

YaST bootloader fixing a broken configuration

Disable autorefresh by default in local media

We have changed the default autorefresh flag for the new repositories added by YaST. In the past the autorefresh was enabled for all repositories except CD/DVD media and ISO files.

With the new defaults the autorefresh is enabled only for remote repositories (like http, ftp, nfs,…). The reason is that some local repositories might not be always available (e.g. external hard disk, USB flash drive,…) and the automatic refresh might cause ugly errors when starting the package manager.

Of course, you can still manually change the autorefresh flag after adding a repository if you need a different value.

Note: The default has been changed in Tumbleweed distribution only, Leap 42.2 or SLE-12-SP2 keep the old defaults. The zypper behavior is unchanged as well, it by default disables autorefresh for all repositories. Only repositories imported from a .repo file have autorefresh enabled. See man zypper for more details.

Tons of improvements in network bridge handling

YaST Network is Swiss Army knife for network configuration which comprehends the management of routing, bonding, bridging and many other things. But, to be honest, the management of bridges was not in the best possible shape. It had quite some usability problems and it was not 100% consistent with the way in which bridges are managed nowadays by Wicked in (open)SUSE. Until now!

This pull request with several screenshots and animations tries to summarize all the changes that have been done during this sprint. Like adapting old configuration files to the new conventions or unifying the UI to make it consistent with the one for managing bonding.

Revamped YaST interface for handling bridges

This revamp includes also quite some usability improvements:

  • “NONE” is shown instead of 0.0.0.0 for old bridge configuration.
  • The bridge master is shown in the enslaved interface.
  • The interfaces overview is updated after a bridge is modified.

Optimizing read of hosts file

It was reported that yast2-network was slow in system with a lot of entries in the /etc/hosts file. We took that as an opportunity to test the new profiler support in YaST. The profiler revealed the problem was in some slow calls to SCR, the layer traditionally used by YaST to interact with the underlying system… which sounded like another opportunity. This time an opportunity to expand the use of CFA (the component we are developing to steady replace SCR) and its Augeas parser. Since Augeas already supports parsing of /etc/hosts files, it was quite straightforward to implement that into YaST… and the result is quite impressive.

The time needed to execute the next command in a system containing a huge /etc/hosts with around 10,000 entries (quite an extreme case, we know) was reduced from 75 seconds to just 20.

yast2 lan list

As you can see in this pull request, we also improved CFA itself, greatly reducing the time needed for reading configuration files with Augeas.

That’s all… until next report

Once again, we must conclude the report telling that this was just a small summary of all the work done during the sprint and that we will be back in three weeks with the next report. Or maybe before, now that we are starting to get used to blog more often.

In any case, see you soon and don’t forget to have a lot of fun!

Improving low-vision accessibility of the installer

October 7th, 2016 by

In our latest report, we promised you would not have to wait another three weeks to hear (or read) from us. And here we are again, but not with any of the anticipated topics (build time reduction and Euruko 2016), but with a call for help in a topic that could really make a difference for (open)SUSE.

Nowadays, YaST team is trying to fix a long-standing issue in the installer: low-vision accessibility. In the past, a user could get a high-contrast mode just pressing shift+F4 during installation. Unfortunately, that feature does not work anymore and, to be honest, changing to a high-contrast palette is not enough. Other adjustments, like setting better font sizes, should be taken into account.

Another option is to use the textmode installation and set some obscure variable (Y2NCURSES_COLOR_THEME) to get the high-contrast mode. But it sounds like the opposite to user friendly.

Some days ago, the team fired up the discussion in the opensuse-factory mailing list but we would like to reach as many people as we can to gather information and feedback about this topic. Getting some affected people involved in the process would be really awesome!

For the time being we’re already working on some improvements:

  • Adding a Linuxrc option so the user can set the high-contrast mode from the very beginning.
  • Fixing shift+F4 support.
  • Improving the high-contrast mode appearance. Below you can see a screenshot of the work in progress.

First prototype of the new high contrast mode

But we would like to hear from you. You can raise your voice in the already mentioned thread at the opensuse-factory mailing list or leave a comment in the related pull request at Github. If you prefer to have a chat, we’re also available on the #yast IRC channel at Freenode… and we love to see people there. 😉

Please, join us to make YaST even better!

Live ISO Multi-boot USB revisited – live-grub-stick

December 25th, 2015 by

Earlier tool live-fat-stick uses syslinux to create multiboot USB stick/hdd on a vfat parition without having to format the stick preserving existing data and copying whole ISO so the same stick can serve as demo as well as to copy ISOs for distributing. However the disadvantages are all of them that comes from using vfat.

Grub2 has come a long way and almost all major distributions now support booting from the iso image via loopback. So here is live-grub-stick script that uses grub in place of syslinux bringing in all the advantages of using grub2.

Currently live images of openSUSE, Ubuntu, Fedora and all their clones are supported. Go ahead and fork it if you would like to add support for your distribution.