Home Home > 2017 > 05
Sign up | Login

Deprecation notice: openSUSE Lizards user blog platform is deprecated, and will remain read only for the time being. Learn more...

Archive for May, 2017

Manual encryption partition setup for stronger full disk encryption

May 26th, 2017 by

When installing openSUSE or SUSE Linux Enterprise, YaST is able to configure encrypted LVM using LUKS for full disk encryption. The default configuration is aes-xts-plain64 using a 256 bit master key. However, due to how the XTC mode splits the key into two halves, this reduces the effective key size used for AES to 128 Bits.

In order to use a 512 bit key for 256 effective AES, one needs to perform manual formatting prior to installation:
cryptsetup LuksFormat --key-size 512 /dev/sda1
However the installer suffers from boo#1030299 which prevents it from writing an entry to /etc/crypttab in this instance. This results in a system that is unable to boot after installation.

The work-around is as follows: Boot into the rescue system, open the crypto device and enter the installed system as a chroot:

cryptsetup luksOpen /dev/sda1 crypto
mount /dev/mapper/system-root /mnt
for X in proc dev sys; do mount -bind /$ /mnt/$X; done
chroot /mnt

(This example assumes /dev/sda1 to be the crypto device, and an LVM VG named system with a LV named root, and no separate /boot.)

Then in the chroot, edit /etc/crypttab to have the following line:

crypto /dev/sda1 none none

See man crypttab for additional settings and options. To finalize, regenerate the initrd and reboot

mkinitrd
exit
reboot

A future rewrite of the YaST storage abstraction layer is planned which should address this issue.

Posted in Security | Comments Off on Manual encryption partition setup for stronger full disk encryption

Highlights of YaST development sprint 35

May 25th, 2017 by

openSUSE Conference 2017 is coming! And as we flight there (literally, one third of the YaST team is in a plane right now typing this), we wanted to inform our beloved readers on what we did in the previous three weeks.

So here is our report, brought to you by airmail!

Bugfixes, bugfixes everywhere

Leaving openSUSE Tumbleweed aside, The YaST team is currently working to deliver SLE12-SP3, openSUSE Leap 42.3, SLE15, openSUSE Leap 15, SUSE CaaSP 1.0 and Kubic (more about Kubic later). Three of them are already in beta phase, which means they are being extensively tested by several parties and in many scenarios, hardware platforms and possible configurations. That amount of manual testing always result in several bug being discovered, no matter how much we try to have some automated tests for the most common cases.

Many of the bugs our testers are finding are related to internationalization and localization, mainly texts in the UI that are always displayed in English, despite the system been configured (or being installed) in a different language.

But, of course, other kind of bugs are also being found. For example, our hardware detection component (hwinfo) was not able to deal with some new machines, making the installation experience everything but pleasant.

As a result, a significant amount of the YaST team manpower during this sprint was targeted to squash those annoying bugs. Which doesn’t mean we didn’t have time for some interesting new features and improvements.

Storage reimplementation: unlock encrypted devices

Once again, our new storage system comes with news. Now it’s able to detect and unlock preexisting encrypted devices during the hard disks probing step, raising you a new pop-pup dialog to ask for the corresponding device password. After unlocking the devices, all your installed systems will be accessible for upgrade and, moreover, the LVM volumes allocated over encrypted devices will be activated.

The new storage stack is expected to debut in SLE15 (and, thus, openSUSE Leap 15), but the functionality can already be tested, for both the installation and upgrade processes, with the StorageNG test ISOs.

Luks activation in StorageNG

The storage reimplementation & AutoYaST – a love story

But the happiest news coming from the new storage stack during this sprint is it’s marriage with AutoYaST. The new automatic partitioning proposal (that is, the “Guided Setup”) is now integrated with AutoYaST.

Thanks to the new software architecture, AutoYaST users will be able to override every single partitioning setting from the control file.

<profile xmlns="http://www.suse.com/1.0/yast2ns" xmlns:config="http://www.suse.com/1.0/configns">
  <general>
    <storage>
      <!-- Override settings from control file  -->
      <try_separate_home config:type="boolean">false</try_separate_home>
      <proposal_lvm config:type="boolean">true</proposal_lvm>
    </storage>
  </general>
</profile>

So you can easily switch on/off LVM, use a separate partition for /home, enable/disable snapshots, enable/disable Windows resizing, etc. All that, still relying on the automated storage proposal to iron the details up. Something that is not possible with the current version of AutoYaST without being forced to define explicitly every partition and LVM volume.

But the simplest way to use the new libstorage proposal is to not define any setting at all in the AutoYaST configuration file. In that case, the partitioning proposal code will do the complete job, installing a new system with the default options.

Of course, before integrating the new storage stack into the upcoming SLE15, the AutoYaST support have to go one step further. Apart from using and configuring the proposal, it must be possible to define a completely custom setup including partitions, LVM volumes, software RAID devices and so on through the corresponding <partitioning> section of the AutoYaST profile. So we used this sprint to sketch a plan to make that possible in the following months, analyzing all the scenarios and configurations supported by AutoYaST and looking for the best way to support them using the existing yast2-storage-ng infrastructure. The outcome of that effort is this detailed document and a list of tasks (PBIs in Scrum jargon) for the upcoming sprints. So be prepared for more news in this regard.

Automatic Cleanup of Snapshots created by Rollback

So far the user had to ensure that snapshots created by rollbacks got deleted to avoid filling up the storage. This process has now been automated. During a rollback, Snapper now sets the cleanup algorithm to “number” for the snapshot corresponding to the previous default subvolume and for the backup snapshot of the previous default subvolume. This enhanced behavior will be available in SLE12-SP3 and openSUSE Leap 42.3. For more information take a look at the more detailed post in the Snapper blog.

Helping to bring the CaaSP fun to openSUSE

For several sprints already we have been presenting features targeted to SUSE CaaSP, the Kubernets-powered solution for managing containers. Many of those features and custom configurations live in a package called yast2-caasp, originally targeted to this great upcoming product built on top of the SLE12-SP2 codebase.

But now the package is also available for Tumbleweed-based systems by request of the Kubic project. Kubic will be the openSUSE alter ego of SUSE CaaSP, that is, a Container as a Service Platform based on openSUSE and Kubernetes. As with any other YaST component, the exact same source code will shared by the SUSE product and its openSUSE brother.

Improved UX when an invalid registration URL is provided

Humans make mistakes, but when the mistakes are made entering some option in
the installation command line, it usually means that a reboot of the machine is be needed to fix them.

That was the case for the registration URL (regurl) option. In the provided address was malformed the installation just stopped. During this sprint we have added an early check of that URL which allows the user to reenter it and continue with the installation. Something that obviously improves the user experience.

Invalid regurl handling in normal installation

In case of an autoinstallation (AutoYaST), the error is reported and the steps to get installer updates and to register the system are skipped.

Invalid regurl handling in autoinstallation

There is still room for more improvements, allowing the user to also modify the URL in other scenarios. For example, for an URL with a valid format but that points to an unreachable server. But in those cases is not so straightforward to identify the culprit of the problem. It would make no sense to annoy the user with a recurring pop-up to change the registration URL if the root of the issue is not the URL but a incorrect network configuration.

Translations and Interpolations

As mentioned at the begining of this post, we recently got quite some bug reports about missing translations. Although some of them were really caused by bugs in the YaST code, others were a consequence of a buggy Ruby rxgettext script which collects the translatable strings from the Ruby source code. The bug is known by the Ruby-GetText developers, but it’s unclear when (or whether) it will be fixed.

The problem is that the tool cannot collect the translatable strings from interpolations. For example it cannot find the “foo” string from this string literal: "#{_("foo")}". As a result, that string is missing in the resulting POT file and cannot be translated by the SUSE or openSUSE localization teams.

As a workaround, we fixed the YaST code to not use the translations inside interpolations. We also documented the possible problems when mixing translations a interpolations and their solution.

And talking about new developer oriented documentation…

Security Tips for YaST Developers

YaST runs with the administrator privileges (root) and therefore we have to be aware of the possible security issues in the code. During this sprint we published a document with a short summary of security tips for YaST developers.

If you are programming an YaST module you should definitely read it, but it might be interesting also for other programmers as many mentioned issues are generic, not tight only to YaST.

The document is available online here.

See you at the conference

That’s all for this sprint report. We have many more things in the oven, be we didn’t manage to finish them during the sprint, so they will have to wait for the next report. Meanwhile we hope to see many of you at the openSUSE Conference 2017. There will be a whole workshop about modern YaST development, a summary with the more relevant news in the last year of YaST development, talks about the new superb yast2-configuration-management module, about our continuous delivery infrastructure and about how we use Docker to deliver YaST… And, of course, also many other interesting content like the awesome presentation from Thorsten Kukuk about the brand new openSUSE Kubic we mentioned earlier. And even more important, a lot of fun!

openSUSE Conference 2017

For those of you that cannot attend to the conference, see you again in this little corner of the internet in three weeks!

Posted in Distribution, Documentation, Events, Factory, Localization, Programming, Systems Management, YaST | Comments Off on Highlights of YaST development sprint 35

CFA (Config Files API) at a glance

May 17th, 2017 by

In our latest sprint report, we promised you an extra post about the technology we have been using lately to manipulate configuration files. The wait is over! Here is the fine text by Michal Filka explaining everything you need to know to get started with CFA.

Welcome to CFA

As you should know if you follow our blog (we make sure to repeat it once in a while 😉 ), YaST was converted some time ago from a custom programming language called YCP to Ruby. However, this conversion was done on language basis. Some old design decisions and principles stayed, like the usage of SCR for accessing the underlying system.

SCR was designed together with YaST. It uses concept of “agents” for accessing configuration files. These agents contains a description of configuration file using their own format. Moreover SCR offers location transparency. You can e.g. work with a file in the execution system or in a chrooted environment. However, this piece of code is proprietary and limited by the inconsistent quality level of the agents. In addition, is written in C++, developed only within SUSE and, sadly, not very well designed. You cannot easily use just the parser or the location transparency functionality. You always have to go through the complete SCR stack when parsing an input. Similarly, when using location transparency (setting a new location), all subsequent SCR calls are influenced by this. For this and some other reasons we decided to replace the proprietary SCR with something else. That’s how we started to develop and use “Configuration files API”

Configuration files API (CFA) is a library written in ruby intended for accessing various configuration files. You can download it from rubygems.org. It is also available as a set of RPM packages for OpenSUSE 42.3 in the build service. It is structured into several layers and creates an internal abstraction of configuration file. It has been designed and developed in SUSE’s YaST team. However this time it uses (or can use) third party parsers. CFA provides location transparency for the parser on the bottom layer and unified API for application on the top one. Location transparency is achieved by a well known File interface, so you can use any piece of code which implements that interface. Implementing support for a new parser is a bit more complicated. In the worst case you may need to implement a ruby bindings. However, once you have a bindings, implementing other pieces which are needed to get things working in the CFA’s stack is simple.

Lets go through the layers in details.

Bottom layer: File access

Is responsible for the direct access to the configuration files. In the simplest case it accesses local configuration files using the Ruby’s File class, but it can be adapted to access remote, chrooted or memory files too. The developer simply needs to provide a file handler implementing the corresponding read and write methods. Handlers for files in memory and in a chrooted environment are already provided by the common CFA framework.

Middle layer: Parser

This layer parses the configuration file which was loaded by the underlying layer. It knows the structure of the file and transforms it into an abstract representation. The library typically uses external tool for parsing, like Augeas. So, if the external tools has specific requirements, they have to be satisfied to get things working. For example if Augeas is used, you need to provide a proper lens (Augeas jargon for a file format descriptor) to parse the particular configuration file.

Top layer: Configuration file model

The last layer creates a model of the configuration file – an semantically meaningful API for accessing the configuration from an application. It basically creates “an abstraction on top of another abstraction”. It means that it unifies the usage of the various tools that can be used for accessing and parsing the configuration files.

Limitations

The model described above has some handicaps, mainly affecting the developers but visible sometimes for the users as well.

Feed the beast

If you are a developer planning to use CFA to manipulate a given configuration file, you must take into account that you will need to provide the parser and satisfy all its requirements. So, you at least have to know which parser is used for each file and what is needed in order to make the parser work with the file of your interest.

This is especially important in case of nonstandard / custom configuration files. It may require some previous work to evaluate the available options.

Beat the beast

There are also limitations coming from the fact that the library and/or parsers on the second layer provides an abstraction on top of the configuration file. This abstraction transform a configuration file into a more convenient and meaningful model and establishes a relation between the file and the model.

But the problem is that with most parsers (specially with Augeas) this relation is not bijective. That means some irrelevant pieces of the configuration file are not represented in the model. For example some spaces can be left out if they are not needed from a syntactic point of view. That can lead to loss of custom padding.

Another example can be comments. You can often see that, if your file uses e.g. “#” as a comment mark, then some parsers can squash lines full of these marks (which some developers use as a kind of delimiter) to just one “#”.

As a more concrete example, some Augeas lenses do not store initial comment marks in the model. That’s specially common in lenses for files where several different comment marks are allowed. However, some lenses return the comments including their mark at the beginning, so extra handling is needed at an upper layer of CFA or at application level.

Last but not least, some parsers use the concept of default values when adding new key with not defined value. This can of course lead to some inconsistencies in configuration file’s look if not handled by an upper layer.

Practical example

As already mentioned, CFA is being used as a replacement for the old fashioned SCR in YaST. So we can really take a look to the result of that replacement in several parts, like the handling of /etc/hosts in yast2-network. Leaving aside a fact that the code is now much better readable, we gathered also some performance numbers.

The test was run using an example /etc/hosts file with 10.000 entries (believe it or not, the experiment is based in a real life use case reported by a user). The test was done using YaST’s command line interface and measured using the good old time utility. Since the command line interface doesn’t currently support entering hosts entries, only reading operations were tested.

time SCR CFA
real 1m15.735s 0m19.079s
user 1m15.076s 0m18.348s
sys 0m0.164s 0m0.244s

As you can see, this part of code is now approximately four times faster than before, so the practical results look very promising. Moreover, the CFA’s code is better designed and much better covered by automated tests.

That’s why the YaST team is investing into both sides of the same coin – CFA development and conversion of YaST’s code from SCR to CFA.

Posted in Programming, Systems Management, YaST | Comments Off on CFA (Config Files API) at a glance

Highlights of YaST development sprint 34

May 3rd, 2017 by

Here we go again! Only one week after our previous report, we already have a new bunch of (hopefully) exciting news. Let’s take a look to the outcome of our 34th Scrum sprint.

Trusted boot support for EFI

In the report of our 19th sprint, we already presented the new (at that point in time) and shiny Trusted Boot support in YaST2 Bootloader. So far, the only supported scenario was legacy boot (i.e. no UEFI) on x86_64 systems. Now, thanks to TPM2, is also possible use Trusted Boot with EFI, so we added support for it in our beloved bootloader module.

So now YaST Bootloader looks the same in non-EFI and EFI variants, no matter which underlying technology is actually used. Of course, YaST is only the tip of the iceberg, booting in Trusted Boot with EFI is possible thanks to all the tools that has recently added support for TPM2. openSUSE developers and packagers rock!

Configuring the NTP service in CaaSP

For a CaaSP cluster to work properly, it’s vitally important that all nodes have their clocks in sync. So, from now on, the installer is able to configure each node in a proper way and the administration node will act as NTP server for the worker nodes.

To achieve that, the user will be asked to specify one or several NTP servers to be used as time source during administration node installation and YaST will take care of the rest (updating the configuration and enabling the service).

If a NTP service is announced through SLP, YaST will propose automatically.

NTP configuration in YaST

For worker nodes, YaST will configure the system to keep it synchronized with the administration role.

Storage reimplementation: improvements in the guided setup

Through the previous reports, you have been able to follow the evolution of the renovated guided setup for the partitioning proposal. This sprint is not different, we have improved and adjusted that new wizard even further, making it smarter and more usable.

The new version is able to decide which steps to show depending on the current scenario. For example, in systems with only one disk the whole disk selection dialog will be skipped. The steps are also simplified by disabling widgets that are not applicable to the current situation. For example, if there is no previous Linux installation, the question about what to do with the existing Linux partitions will be disabled.

And talking about the actions to perform on preexisting partitions, that has also been improved. In the first guided setup version, these options were only displaying for illustrative purposes, but now they are 100% real and the proposal will honor their values, so the user can easily select what to do with previous Windows or Linux partitions. We even added a third option for other kind of partitions.

New settings in the storage proposal

Last but not least (regarding the guided setup), the password selection for encryption is now more usable, allowing the user to choose a not so strong password if really desired.

Allowing users to shoot their own feet

Insserv removal

YaST is a complex and large piece software. It means that time to time some pieces that used to be great and shiny become obsolete and end up being useless. The cycle of life. 😉

Some time ago, it was decided it was time for insserv to enjoy retirement and it was replaced by a stub. But there were still calls to insserv in YaST and we decided to remove them all. There were several reasons for that decision. Basicaly (open)SUSE has used systemd for couple of years already, so revisiting places where the YaST code depended on SySV was a must. As a side effect it shortened the list of YaST dependencies and, in the end, it is another small step towards smaller installation.

So good bye insserv, it was a pleasure to work with you.

Improvements in the ZFCP Controller Configuration for zKVM on S/390

When running the installer on a mainframe in a zKVM virtual machine it displays a warning about not detected ZFCP controllers:

ZFCP warning on S/390

However, when running in a zKVM virtual machine the disk is accessible via the virtio driver, not through an emulated ZFCP controller. The warning is pointless and confusing.

The fix was basically just an one-liner which skips the warning when the zKVM virtualization is detected, but the YaST module for ZFCP doesn’t usually receive to much maintenance, so we applied our boy scout rule and improved the code a bit.

The improvements include using Rubocop for clean and unified coding style, enabling code coverage to know how much the code is tested (in this case it turned out to be horribly low, just about 4%), removing unused files, etc… You can find the details in the pull request.

Storage reimplementation: improvements all around

As we reported in our report about Hack Week 15, we have been working on an alternative way to offer the power of the new storage library to the Ruby world. The new API is finally ready for prime-time and used in all the Ruby code.

We took the opportunity to greatly improve the developer documentation and to revamp the yast2-storage-ng README and status page.

We also did some extra checks and added automated testing to ensure our partitioning proposal ensure the requisites if a S/390 system using ZFCP, so mainframe users also have a peaceful transition to the new storage stack.

We also greatly improved the ability of the new library to work with alternative name schemes for the devices. Up to now, only using plain kernel device name (e.g. /dev/sda1) was fully supported. Now we can do all the operations (specially generating the /etc/fstab file) by device name, by UUID, by filesystem label, by device id and by device path.

More content already in the oven!

As you already know, at least we repeat it quite often, 😉 YaST was converted from YCP to Ruby some time ago. However, this conversion was done on language basis. Some old design decisions and principles stayed, like the usage of SCR for accessing the underlying system.

Some time ago we introduced CFA (Config Files Api Gem) as a more powerful and flexible Ruby-powered replacement for SCR. Although we have been using it for a while in several YaST modules, we felt the concepts and rationales behind its operation where not that clear.

So we have invested some time improving the documentation and writing a blog post to properly present and explain CFA to the world. We will publish it next week, so stay tuned!

Great news ahead!

The next sprint will be the first in which the whole YaST team will work together integrating the new storage stack in every single part of YaST. So we hope the next report to be full of good news about the expert partitioner, improvements in AutoYaST’s handling of disks and so on.

Of course, that would be only a small fraction of all the stuff we plan to work on. See you in three weeks with more news!

Posted in Distribution, Documentation, Factory, Hackweek, Programming, Systems Management, YaST | Comments Off on Highlights of YaST development sprint 34