Home Home > Documentation
Sign up | Login

Archive for the ‘Documentation’ Category

Highlights of YaST Development Sprint 82

August 14th, 2019 by

July and August are very sunny months in Europe… and chameleons like sun. That’s why most YaST developers run away from their keyboards during this period to enjoy vacations. Of course, that has an impact in the development speed of YaST and, as a consequence, in the length of the YaST Team blog posts.

But don’t worry much, we still have enough information to keep you entertained for a few minutes if you want to dive with us into our summer activities that includes:

  • Enhancing the development documentation
  • Extending AutoYaST capabilities regarding Bcache
  • Lots of small fixes and improvements

AutoYaST and Bcache – Broader Powers

Bcache technology made its debut in YaST several sprints ago. You can use the Expert Partitioner to create your Bcache devices and improve the performance of your slow disks. We even published a dedicated blog post with all details about it.

Apart of the Expert Partitioner, AutoYaST was also extended to support Bcache devices. And this time, we are pleased to announce that … we have fixed our first Bcache bug!

Actually, there were two different bugs in the AutoYaST side. First, the auto-installation failed when you tried to create a Bcache device without a caching set. On the other hand, it was not possible to create a Bcache with an LVM Logical Volume as backing device. Both bugs are gone, and now AutoYaST supports those scenarios perfectly.

Configuring Bcache and LVM with AutoYaST

But Bcache is a quite young technology and it is not free of bugs. In fact, it fails when the backing device is an LVM Logical Volume and you try to set the cache mode. We have already reported a bug to the Bcache crew and (as you can see in the bug report) a patch is already been tested.

Enhancing Our Development Documentation

This sprint we also touched our development documentation, specifically we documented our process for creating the maintenance branches for the released products. The new branching documentation describes not only how to actually create the branches but also how to adapt all the infrastructure around (like Jenkins or Travis) which requires special knowledge.

We will see how much the documentation is helpful next time when somebody has to do the branching process for the next release. 😉

Working for a better world YaST

We do our best to write code free of bugs… but some bugs are smarter than us and they manage to survive and reproduce. Fortunately we used this sprint to do some hunting.

Those are only some examples of the kind of bugs we have fought during this sprint. But checking bug reports has also made us think in the future…

LibYUI in 21st Century

We fixed a bug related to how the focus was managed in text mode after changing any setting via the hyperlinks available in the installation summary.

Installation summary in text mode

The implemented solution is actually not perfect, it’s just the better we can do with our set of widgets. And that was yet another example of such problem – LibYUI is an awesome library that allows us to create interfaces that work in both graphic and text modes, but it has basically not evolved for more than a decade… and it’s time to fix that!

So we have been discussing how to organize our time in the close future to leave some room for innovation and renovation regarding LibYUI and the YaST UI in general. Stay tuned for more news.

August is still not over

The YaST Team will keep working during the rest of the summer sharpening our Linux Swiss army knife. But half of the team is still on vacation or starting their vacation now. So most likely our next report will be here in two weeks and it will also be a light read.

Meanwhile, don’t forget to have a lot of fun!

Debugging jenkins

July 31st, 2019 by

We had strange near-daily outages of our internal busy jenkins for some weeks.

To get to the root cause of the issue, we enabled remote debugging with

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=ci.suse.de -Dcom.sun.management.jmxremote.password.file=/var/lib/jenkins/jmxremote.password

and attached visualvm to see what it was doing.
This showed the number of threads and memory usage in a sawtooth pattern. Every time the garbage collector ran, it dropped 500-1000 threads.

Today we noticed that every time it threw these java.lang.OutOfMemoryError: unable to create new native thread errors, the maximum number of threads was 2018… suspiciously close to 2048. Looking for the same time in journalctl showed
kernel: cgroup: fork rejected by pids controller in /system.slice/jenkins.service

So it was systemd refusing java’s request for a new thread and jenkins not handling that gracefully in all cases.
That was easily avoided with a
TasksMax=8192

Now the new peak was at 4890 live threads and jenkins served all Geekos happily ever after.

Highlights of YaST Development Sprint 75

April 10th, 2019 by

With the upcoming releases of openSUSE Leap 15.1 and SLE-15-SP1 approaching, the YaST Team at SUSE is investing a quite significant time in polishing details and fixing small (and not so small) bugs. But fortunately, that still leaves us enough time to also work in our mid term goals.

So welcome to our usual selection of selected bug-fixes (listing them all would be boring) and exciting new stuff. This edition includes:

  • A nice howto for reporting Snapper bugs
  • Tons of fixes for right-to-left languages like Arabic
  • Some adjustments and improvements in the storage area
  • A sneak peak into the future of the yast2-network code
  • Some contributor-oriented content: like our new pull request templates and revamped Docker images for testing

Snapper Bug Reporting Howto

During this sprint we fixed a bug that was causing Snapper to crash under very specific circumstances. The scenario was quite unusual so we had to request quite some information from the reporter of the bug to confirm what was happening. As a nice consequence, in addition to having now a more robust Snapper (one bug killed) you can also enjoy a new page in the openSUSE wiki listing the information you should attach to Bugzilla if you find a bug in Snapper while using (open)SUSE.

Which is also a nice excuse to remind you about the equivalent "Report a YaST bug" page.

YaST around the globe… in all directions

Many of the YaST users and of our blog readers are not native English speakers that surely appreciate the fact that YaST and (open)SUSE in general can be used in several languages. But have you ever thought about the implications of developing a multi-language software? Sure? In all of them? 😉

Human languages are so diverse as the human cultures and there are many details to take into account, from the usage of different alphabets to the various ways of dealing with genre or number (in English the words have just one form for singular and another for plural, but that can be way more complex in other languages). In today’s issue we will take a look to one of our favorite translation issues – languages that are written from right to left, like Arabic.

The installer summary in Arabic

Dealing with text that is a mixture of Latin and Arabic script is complex and sometimes we have to deal with interesting bugs. Fortunately we have our own weapon to fight those bugs. If in Star Wars they have protocol droids like C3PO, in the YaST team we have Martin Vidner, which is the closer human equivalent.

He fixed all the reported bugs and even created a tool to help debugging similar problems in the future. You can find the source code of that tool in Github. There is even a hosted instance of the tool to be used by translators or anyone who is curious.

Now, even complex interfaces like our Partitioner look correct enough in right-to-left languages, so we will not have to send mirrors to all our Arabic users.

The YaST Partitioner in Arabic

If you want to know more about this exciting but very complex problem of bidirectional texts, you can start with the following documents.

  • Martin’s great summary of the types problems found in YaST and their respective solutions.
  • Wikipedia: Bi-directional text, an overview of the concepts
  • Unicode Standard Annex #9: Unicode Bidirectional Algorithm, the gory details, 50 pages of them
  • More Arabic YaST

    On related news, we got also some reports about some problems visualizing bullet-points in Korean with the beta versions of the future SLE-15-SP1. But as we could verify, all those problems are gone now.

    SLES installer in Korean

    Storage Fixes

    Other area that has received some attention in this sprint is the storage management. Three related features needed adjustments before the upcoming (open)SUSE releases:

    • Fixed the detection of the boot disk in the Partitioner warnings.
    • The Guided Setup now works better when doing several attempts in different disks.
    • AutoYaST can now install over NFS.

    One of the last storage features that the YaST Team has developed is the support for Bcache devices in the Expert Partitioner. While our QA team was testing it, they found a bug. The Partitioner was complaining because the boot disk did not contain a partition table, which is a mandatory condition for a Legacy (non-UEFI) x86 system. But it was a bogus warning, since they had actually defined a /boot partition in another disk.

    That’s how we found that our Partitioner gets confused if there is a separate partition mounted at /boot and located in a different disk than the root file-system. The Partitioner insisted in considering the disk containing / to be the one that would be using for booting, instead of checking the structure of the disk containing /boot. Now that is fixed and the improvement will be available for the upcoming SLE 15 SP1, Leap 15.1 and, of course, openSUSE Tumbleweed.

    But that was not the only storage bug fixed just in time for the upcoming releases. Some sprints ago, the Storage Proposal algorithm for the initial proposal was modified to try installing on each of the individual disks. If the installation was not possible over a given disk, even after disabling all optional configurations (e.g., snapshots and separate /home), a new proposal is tried over the next disk and so on. The problem was that the disabled options in the previous attempt were not restored back when switching to the next disk. This caused some ugly side effects, for example, if the swap partition was disabled when trying over the first disk, then the proposal did not try to create a swap when it was performing the proposal over the next disk. But now this is also fixed and it will work as expected.

    And last but not least, AutoYaST now supports to install over a Network File System (NFS). This feature was left back when the new YaST storage stack was re-implemented for SLE 15 GA. Actually, this is a non documented feature. That’s why we overlooked that SUSE 12 was able to do it using some hacks and a non-validating AutoYaST profile. But no worries, the feature is available again and such profile will work now in any updated SLE-15 or Leap 15.0. Of course, it will also work while installing SLE-15-SP1 or openSUSE Leap 15.1 and Tumbleweed.

    Nevertheless, we are working on a better and documented way of supporting that scenario in the future, with no need to twist the specification of the AutoYaST profile. Stay tuned for more information.

    Rethinking the Location of Special Boot Partitions

    And now that the storage layer looks sane and healthy for the upcoming releases, we also took some time to think about future improvements. As you know, the storage Guided Setup always proposes to create special boot partitions as needed on each case. That can be a BIOS BOOT (for Legacy x86 systems with GPT), an ESP (for UEFI systems), PReP (for PPC systems) or zipl (for S/390 systems). Strictly speaking those partitions doesn’t have to always be in the same disk than the root partition and in some cases having it on a separate one can have some advantages (like sharing the ESP partition with another operating systems).

    But we have been reconsidering all the cases, the expectations of most users and of the majority of BIOS vendors and the known bugs in other operating systems about sharing boot partitions. We have decided to be more strict in the future about the location of those partitions. Starting today with openSUSE Tumbleweed and in the 15.2 releases of openSUSE Leap and SLE, the Guided Setup will always propose those partitions in the system disk. That is, in the disk containing /boot and the root filesystem.

    The future of YaST Network is here

    Those who follow this blog know that we invested quite some time on the last couple of years rewriting the part of YaST that was more buggy and harder to modify – the storage stack. And surely you have already noticed that since we did it we are introducing new features at a very good pace (like bcache, more powerful Partitioner, Raspberry Pi support, etc.) and fixing the reported bugs in a matter of days or even hours.

    The next in our list of YaST areas to revamp is the networking support. And we are happy to announce that we are starting to have some visible results in that. There is still a very long road ahead and we will provide more information in upcoming reports. But at least we have already a preview of a fully rewritten management of network routes. It’s still not available in openSUSE Tumbleweed. But for those who can’t wait, here you can see the first screenshot. All based in new and clean code backed by automated tests.

    New network routing dialog

    Activating Online Repositories in openSUSE Leap 15.1

    The openSUSE Tumbleweed installer asks at the beginning of the installation whether to activate and use the online repositories when a network connection is available.

    The reason is that the installation DVD does not contain all available packages because of the limited media size. Another advantage is that the installer might directly install newer packages than on the media, this avoids installing the older versions first and then upgrading them to the latest version.

    However, in some case you might not want to use the online repositories, for example if the network connection is slow or is paid.

    We got a bug report that this question was missing in the Leap 15.1. It turned out that the control.xml file which drives the installer did not contain this step. After adding few lines into the file you can now enjoy the online repositories also in Leap 15.1!

    Online repositories in Leap 15.1

    Why are we writing about this? The reason for the missing step in the Leap 15.1 was a bit surprising. Normally all YaST packages are developed in the Git master branch for both Tumbleweed and Leap. However, in this case the Leap 15.1 has been already branched and was developed separately, the changes in the master went only to the Tumbleweed. And we overlooked that small difference when adding this step.

    To avoid this in the future we added a pull request template with a reminder which informs the developers about this difference in the Git setup when opening a pull request.

    If your project also has some unusual setup then the pull request template might a good reminder for you as well.

    Building the Docker Images in OBS

    But the reminders about the correct branches and procedures is not the only news we have for YaST contributors and main developers. As you may remember, few years ago we switched to using Docker at Travis. That works well but we found some disadvantages of that initial setup.

    • You need extra account at the Docker Hub to manage the images.
    • There is no link between OBS and the Docker Hub, we cannot easily trigger image rebuild when a package is updated in OBS.
    • We only blindly triggered the rebuild every 2 hours (in some cases the rebuild is not necessary, in some cases it took too much time).
    • The Docker Hub can use the new OBS packages only after they are published by OBS.
    • The build at the Docker Hub is quite slow (~20 minutes in our case), if an image is currently being built the build is added into the queue and it will start after the previous builds are finished.

    The result is that a new package can be available in Travis several hours after merging the pull request. And even after triggering the build manually it still might take more than one hour.

    We needed a faster cycle and the solution, as usually happens, was in the openSUSE ecosystem. As you may know, the Open Build Service is able of much more than just building packages. So we decided to make use of the OBS capacity of building Docker images.

    Building both our packages and our Docker images in OBS comes with many advantages:

    • The image build is started immediately when the new packages are built, it does not wait for publishing the packages and does not wait for full rebuild (only for the needed packages).
    • No extra accounts/permissions (just use your OBS account).
    • The build in OBS is faster (6-7 minutes).
    • No need for extra Jenkins jobs periodically triggering the image rebuilds.

    This means the new packages should be available in the Docker image in about 10-15 minutes after merging a pull request (for leaf packages, changing a core package which triggers a complete YaST rebuild will of course take more time).

    If you want to learn more about this topic, take a look to the following links:

    And that was not all!

    As usual, the content of this report is just a small subset of all the work the YaST Team does in two weeks. In this sprint, most of that work went to fixing all kind of bugs in preparation for the next releases. Big bugs, small ones, hidden bugs and embarrassingly obvious ones. Hopefully, you got a fix for your reported bug. If not, you can always stay tuned for more news after the next sprint. And don’t forget to have a lot of fun!

Highlights of YaST development sprint 35

May 25th, 2017 by

openSUSE Conference 2017 is coming! And as we flight there (literally, one third of the YaST team is in a plane right now typing this), we wanted to inform our beloved readers on what we did in the previous three weeks.

So here is our report, brought to you by airmail!

Bugfixes, bugfixes everywhere

Leaving openSUSE Tumbleweed aside, The YaST team is currently working to deliver SLE12-SP3, openSUSE Leap 42.3, SLE15, openSUSE Leap 15, SUSE CaaSP 1.0 and Kubic (more about Kubic later). Three of them are already in beta phase, which means they are being extensively tested by several parties and in many scenarios, hardware platforms and possible configurations. That amount of manual testing always result in several bug being discovered, no matter how much we try to have some automated tests for the most common cases.

Many of the bugs our testers are finding are related to internationalization and localization, mainly texts in the UI that are always displayed in English, despite the system been configured (or being installed) in a different language.

But, of course, other kind of bugs are also being found. For example, our hardware detection component (hwinfo) was not able to deal with some new machines, making the installation experience everything but pleasant.

As a result, a significant amount of the YaST team manpower during this sprint was targeted to squash those annoying bugs. Which doesn’t mean we didn’t have time for some interesting new features and improvements.

Storage reimplementation: unlock encrypted devices

Once again, our new storage system comes with news. Now it’s able to detect and unlock preexisting encrypted devices during the hard disks probing step, raising you a new pop-pup dialog to ask for the corresponding device password. After unlocking the devices, all your installed systems will be accessible for upgrade and, moreover, the LVM volumes allocated over encrypted devices will be activated.

The new storage stack is expected to debut in SLE15 (and, thus, openSUSE Leap 15), but the functionality can already be tested, for both the installation and upgrade processes, with the StorageNG test ISOs.

Luks activation in StorageNG

The storage reimplementation & AutoYaST – a love story

But the happiest news coming from the new storage stack during this sprint is it’s marriage with AutoYaST. The new automatic partitioning proposal (that is, the “Guided Setup”) is now integrated with AutoYaST.

Thanks to the new software architecture, AutoYaST users will be able to override every single partitioning setting from the control file.

<profile xmlns="http://www.suse.com/1.0/yast2ns" xmlns:config="http://www.suse.com/1.0/configns">
  <general>
    <storage>
      <!-- Override settings from control file  -->
      <try_separate_home config:type="boolean">false</try_separate_home>
      <proposal_lvm config:type="boolean">true</proposal_lvm>
    </storage>
  </general>
</profile>

So you can easily switch on/off LVM, use a separate partition for /home, enable/disable snapshots, enable/disable Windows resizing, etc. All that, still relying on the automated storage proposal to iron the details up. Something that is not possible with the current version of AutoYaST without being forced to define explicitly every partition and LVM volume.

But the simplest way to use the new libstorage proposal is to not define any setting at all in the AutoYaST configuration file. In that case, the partitioning proposal code will do the complete job, installing a new system with the default options.

Of course, before integrating the new storage stack into the upcoming SLE15, the AutoYaST support have to go one step further. Apart from using and configuring the proposal, it must be possible to define a completely custom setup including partitions, LVM volumes, software RAID devices and so on through the corresponding <partitioning> section of the AutoYaST profile. So we used this sprint to sketch a plan to make that possible in the following months, analyzing all the scenarios and configurations supported by AutoYaST and looking for the best way to support them using the existing yast2-storage-ng infrastructure. The outcome of that effort is this detailed document and a list of tasks (PBIs in Scrum jargon) for the upcoming sprints. So be prepared for more news in this regard.

Automatic Cleanup of Snapshots created by Rollback

So far the user had to ensure that snapshots created by rollbacks got deleted to avoid filling up the storage. This process has now been automated. During a rollback, Snapper now sets the cleanup algorithm to “number” for the snapshot corresponding to the previous default subvolume and for the backup snapshot of the previous default subvolume. This enhanced behavior will be available in SLE12-SP3 and openSUSE Leap 42.3. For more information take a look at the more detailed post in the Snapper blog.

Helping to bring the CaaSP fun to openSUSE

For several sprints already we have been presenting features targeted to SUSE CaaSP, the Kubernets-powered solution for managing containers. Many of those features and custom configurations live in a package called yast2-caasp, originally targeted to this great upcoming product built on top of the SLE12-SP2 codebase.

But now the package is also available for Tumbleweed-based systems by request of the Kubic project. Kubic will be the openSUSE alter ego of SUSE CaaSP, that is, a Container as a Service Platform based on openSUSE and Kubernetes. As with any other YaST component, the exact same source code will shared by the SUSE product and its openSUSE brother.

Improved UX when an invalid registration URL is provided

Humans make mistakes, but when the mistakes are made entering some option in
the installation command line, it usually means that a reboot of the machine is be needed to fix them.

That was the case for the registration URL (regurl) option. In the provided address was malformed the installation just stopped. During this sprint we have added an early check of that URL which allows the user to reenter it and continue with the installation. Something that obviously improves the user experience.

Invalid regurl handling in normal installation

In case of an autoinstallation (AutoYaST), the error is reported and the steps to get installer updates and to register the system are skipped.

Invalid regurl handling in autoinstallation

There is still room for more improvements, allowing the user to also modify the URL in other scenarios. For example, for an URL with a valid format but that points to an unreachable server. But in those cases is not so straightforward to identify the culprit of the problem. It would make no sense to annoy the user with a recurring pop-up to change the registration URL if the root of the issue is not the URL but a incorrect network configuration.

Translations and Interpolations

As mentioned at the begining of this post, we recently got quite some bug reports about missing translations. Although some of them were really caused by bugs in the YaST code, others were a consequence of a buggy Ruby rxgettext script which collects the translatable strings from the Ruby source code. The bug is known by the Ruby-GetText developers, but it’s unclear when (or whether) it will be fixed.

The problem is that the tool cannot collect the translatable strings from interpolations. For example it cannot find the “foo” string from this string literal: "#{_("foo")}". As a result, that string is missing in the resulting POT file and cannot be translated by the SUSE or openSUSE localization teams.

As a workaround, we fixed the YaST code to not use the translations inside interpolations. We also documented the possible problems when mixing translations a interpolations and their solution.

And talking about new developer oriented documentation…

Security Tips for YaST Developers

YaST runs with the administrator privileges (root) and therefore we have to be aware of the possible security issues in the code. During this sprint we published a document with a short summary of security tips for YaST developers.

If you are programming an YaST module you should definitely read it, but it might be interesting also for other programmers as many mentioned issues are generic, not tight only to YaST.

The document is available online here.

See you at the conference

That’s all for this sprint report. We have many more things in the oven, be we didn’t manage to finish them during the sprint, so they will have to wait for the next report. Meanwhile we hope to see many of you at the openSUSE Conference 2017. There will be a whole workshop about modern YaST development, a summary with the more relevant news in the last year of YaST development, talks about the new superb yast2-configuration-management module, about our continuous delivery infrastructure and about how we use Docker to deliver YaST… And, of course, also many other interesting content like the awesome presentation from Thorsten Kukuk about the brand new openSUSE Kubic we mentioned earlier. And even more important, a lot of fun!

openSUSE Conference 2017

For those of you that cannot attend to the conference, see you again in this little corner of the internet in three weeks!

Highlights of YaST development sprint 34

May 3rd, 2017 by

Here we go again! Only one week after our previous report, we already have a new bunch of (hopefully) exciting news. Let’s take a look to the outcome of our 34th Scrum sprint.

Trusted boot support for EFI

In the report of our 19th sprint, we already presented the new (at that point in time) and shiny Trusted Boot support in YaST2 Bootloader. So far, the only supported scenario was legacy boot (i.e. no UEFI) on x86_64 systems. Now, thanks to TPM2, is also possible use Trusted Boot with EFI, so we added support for it in our beloved bootloader module.

So now YaST Bootloader looks the same in non-EFI and EFI variants, no matter which underlying technology is actually used. Of course, YaST is only the tip of the iceberg, booting in Trusted Boot with EFI is possible thanks to all the tools that has recently added support for TPM2. openSUSE developers and packagers rock!

Configuring the NTP service in CaaSP

For a CaaSP cluster to work properly, it’s vitally important that all nodes have their clocks in sync. So, from now on, the installer is able to configure each node in a proper way and the administration node will act as NTP server for the worker nodes.

To achieve that, the user will be asked to specify one or several NTP servers to be used as time source during administration node installation and YaST will take care of the rest (updating the configuration and enabling the service).

If a NTP service is announced through SLP, YaST will propose automatically.

NTP configuration in YaST

For worker nodes, YaST will configure the system to keep it synchronized with the administration role.

Storage reimplementation: improvements in the guided setup

Through the previous reports, you have been able to follow the evolution of the renovated guided setup for the partitioning proposal. This sprint is not different, we have improved and adjusted that new wizard even further, making it smarter and more usable.

The new version is able to decide which steps to show depending on the current scenario. For example, in systems with only one disk the whole disk selection dialog will be skipped. The steps are also simplified by disabling widgets that are not applicable to the current situation. For example, if there is no previous Linux installation, the question about what to do with the existing Linux partitions will be disabled.

And talking about the actions to perform on preexisting partitions, that has also been improved. In the first guided setup version, these options were only displaying for illustrative purposes, but now they are 100% real and the proposal will honor their values, so the user can easily select what to do with previous Windows or Linux partitions. We even added a third option for other kind of partitions.

New settings in the storage proposal

Last but not least (regarding the guided setup), the password selection for encryption is now more usable, allowing the user to choose a not so strong password if really desired.

Allowing users to shoot their own feet

Insserv removal

YaST is a complex and large piece software. It means that time to time some pieces that used to be great and shiny become obsolete and end up being useless. The cycle of life. 😉

Some time ago, it was decided it was time for insserv to enjoy retirement and it was replaced by a stub. But there were still calls to insserv in YaST and we decided to remove them all. There were several reasons for that decision. Basicaly (open)SUSE has used systemd for couple of years already, so revisiting places where the YaST code depended on SySV was a must. As a side effect it shortened the list of YaST dependencies and, in the end, it is another small step towards smaller installation.

So good bye insserv, it was a pleasure to work with you.

Improvements in the ZFCP Controller Configuration for zKVM on S/390

When running the installer on a mainframe in a zKVM virtual machine it displays a warning about not detected ZFCP controllers:

ZFCP warning on S/390

However, when running in a zKVM virtual machine the disk is accessible via the virtio driver, not through an emulated ZFCP controller. The warning is pointless and confusing.

The fix was basically just an one-liner which skips the warning when the zKVM virtualization is detected, but the YaST module for ZFCP doesn’t usually receive to much maintenance, so we applied our boy scout rule and improved the code a bit.

The improvements include using Rubocop for clean and unified coding style, enabling code coverage to know how much the code is tested (in this case it turned out to be horribly low, just about 4%), removing unused files, etc… You can find the details in the pull request.

Storage reimplementation: improvements all around

As we reported in our report about Hack Week 15, we have been working on an alternative way to offer the power of the new storage library to the Ruby world. The new API is finally ready for prime-time and used in all the Ruby code.

We took the opportunity to greatly improve the developer documentation and to revamp the yast2-storage-ng README and status page.

We also did some extra checks and added automated testing to ensure our partitioning proposal ensure the requisites if a S/390 system using ZFCP, so mainframe users also have a peaceful transition to the new storage stack.

We also greatly improved the ability of the new library to work with alternative name schemes for the devices. Up to now, only using plain kernel device name (e.g. /dev/sda1) was fully supported. Now we can do all the operations (specially generating the /etc/fstab file) by device name, by UUID, by filesystem label, by device id and by device path.

More content already in the oven!

As you already know, at least we repeat it quite often, 😉 YaST was converted from YCP to Ruby some time ago. However, this conversion was done on language basis. Some old design decisions and principles stayed, like the usage of SCR for accessing the underlying system.

Some time ago we introduced CFA (Config Files Api Gem) as a more powerful and flexible Ruby-powered replacement for SCR. Although we have been using it for a while in several YaST modules, we felt the concepts and rationales behind its operation where not that clear.

So we have invested some time improving the documentation and writing a blog post to properly present and explain CFA to the world. We will publish it next week, so stay tuned!

Great news ahead!

The next sprint will be the first in which the whole YaST team will work together integrating the new storage stack in every single part of YaST. So we hope the next report to be full of good news about the expert partitioner, improvements in AutoYaST’s handling of disks and so on.

Of course, that would be only a small fraction of all the stuff we plan to work on. See you in three weeks with more news!

How we run our OpenStack cloud

January 23rd, 2017 by

This post it to document how we setup cloud.suse.de which is one of our many internal SUSE OpenStack Cloud deployments for use by R&D.

In 2016-06 we started the deployment with SOC6 on 4 nodes. 1 controller and 3 compute nodes that also served for ceph (distributed storage) with their 2nd HDD. Since the nodes are from 2012 they only have 1gbit network and spinning disks. Thus ceph only delivers ~50 MB/s which is sufficient for many use cases.

We did not deploy that cloud with HA, even though our product supports it. The two main reasons for that are

  • that it will use up two or three nodes instead of one for controller services, which is significant if you start out with only 4 (and grow to 6)
  • that it increases the complexity of setup, operations and debugging and thus might lead to decreased availability of the cloud service

Then we have a limited supply of vlans even though technically they are just numbers between 1 and 4095, in SUSE we do allocations to be able to switch together networks from further away. So we could not use vlan mode in neutron if we want to allow software defined network (=SDN) (we did not in old.cloud.suse.de and I did not hear complaints, but now I see a lot of people using SDN)
So we went with ovs+vxlan +dvr (open vSwitch + Virtual eXtensible LAN + Distributed Virtual Router) because that allows VMs to remain reachable even when the controller node reboots.
But then I found that they cannot use DNS during that time, because distributed virtual DNS was not yet implemented. And ovs has some annoying bugs are hard to debug and fix. So I built ugly workarounds that mostly hide^Wsolve the problems from our users’ point of view.
For the next cloud deployment, I will try to use linuxbridge+vlan or linuxbridge+vxlan mode.
And the uptime is pretty good. But it could be better with proper monitoring.

Because we needed to redeploy multiple times before we got all the details right and to document the setup, we scripted most of the deployment with qa_crowbarsetup (which is part of our CI) and extra files in https://github.com/SUSE-Cloud/automation/tree/production/scripts/productioncloud. The only part not in there are the passwords.

We use proper SSL certs from our internal SUSE CA.
For that we needed to install that root CA on all involved nodes.

We use kvm, because it is the most advanced and stable of the supported hypervisors. Xen might be a possible 2nd choice. We use two custom kvm patches to fix nested virt on our G3 Opteron CPUs.

Overall we use 3 vlans. One each for admin, public/floating, sdn/storage networks.
We increased the default /24 IP ranges because we needed more IPs in the fixed and public/floating networks

For authentication, we use our internal R&D LDAP server, but since it does not have information about user’s groups, I wrote a perl script to pull that information from the Novell/innerweb LDAP server and export it as json for use by the hybrid_json assignment backend I wrote.

In addition I wrote a cloud-stats.sh to email weekly reports about utilization of the cloud and another script to tell users about which instances they still have, but might have forgotten.

On the cloud user side, we and other people use one or more of

  • Salt-cloud
  • Nova boot
  • salt-ssh
  • terraform
  • heat

to script instance setup and administration.

Overall we are now hosting 70 instance VMs on 5 compute nodes that together have cost us below 20000€

Basic Nextcloud installation on openSUSE Leap

October 28th, 2016 by

Nextcloud Logo

I see the official documentation has full tutorial for RHEL 6 or CentOS 6 and RHEL 7 or CentOS 7. The main documentation covers Ubuntu 14.04 LTS

openSUSE already has the Nextcloud client packaged in Tumbelweed and the Server is in the PHP extra repo! Personally, I prefer to install eveything from official repository, so when an update is available, I can have it without a glitch. This tutorial describes how to install Nextcloud using command line. I followed the official documentation of Ubuntu 14.04 LTS installation.

Why choose openSUSE Leap? openSUSE Leap is a brand new way of building openSUSE and is new type of hybrid Linux distribution. Leap uses source from SUSE Linux Enterprise (SLE), which gives Leap a level of stability unmatched by other Linux distributions, and combines that with community developments to give users, developers and sysadmins the best stable Linux experience available. Contributor and enterprise efforts for Leap bridge a gap between matured packages and newer packages found in openSUSE’s other distribution Tumbleweed. You can download openSUSE Leap from the site https://software.opensuse.org/.
(more…)

Highlights of YaST development sprint 25

September 28th, 2016 by

Another development sprint is over. Time flies! In our previous post we already reported about the branching of Tumbleweed and the upcoming releases and about the expected consequences: the landing of some cool features in a less conservative Tumbleweed.

We are still dedicating quite some effort to polish the upcoming stable releases (SLE12-SP2 and Leap 42.2), but in this sprint we finally found some time to play. Which is great because blogging about new features is more fun than doing it about bug fixes. 🙂

Importing Authorized Keys with AutoYaST

When logging in via SSH, public key authentication should be preferred over password authentication. Until now, the best way of setting up the required authorized_keys files in AutoYaST was using the files section.

However, that approach is tedious and error prone, as you need to make sure you set the correct owner, permissions, etc. Moreover you need to keep in sync the user definition (username and home directory) with the file definition.

AutoYaST now supports the specification of a set of public keys for each user with a pretty straightforward syntax:

<user>
  <username>suse<username>
  <authorized_keys config:type="list">
    <listentry>ssh-rsa your-public-key-1</listentry>
    <listentry>ssh-rsa your-public-key-2</listentry>
  <authorized_keys>
<user>

AutoYaST takes care of writing the files and setting the ownership and the proper permissions.

While documenting this new feature we realized the AutoYaST documentation about users management could be more detailed, which leads us to…

Improving the documentation

Usually developers love to create programs loaded with cool features but hate to write documentation. Fortunately there are people out there who enjoy writing documentation and bringing all those features to light. We have already mentioned in previous reports how grateful we are for having the SUSE documentation team polishing and publishing our documentation drafts and how open and straightforward the process is.

We updated the YaST documentation to include information about the installer self-update feature, which will debut in SUSE Linux Enterprise 12 SP2 and openSUSE Leap 42.2. As part of the same pull request and in the AutoYaST side, some additional improvements were made, including cleaning-up some duplicated information about SUSE registration.

On the other hand and as a consequence of the above mentioned new feature, the AutoYaST documentation regarding users management has been rewritten adding missing information like groups, user defaults and login settings.

All our pull requests are already merged in the doc-sle repository. At a later point in time, the SUSE documentation team will review and polish all the new content (including ours) and will publish an up-to-date version of the online documentation. If you don’t want to wait, you can easily generate an HTML or PDF version of the documentation including all the non-reviewed contributions just following the very simple instructions in the README file of the doc-sle repository.

Did we already mention we love the open source, programmer-friendly processes of the documentation team? 😉

Storage reimplementation: something you can touch

We promised news about the storage reimplementation and here they are. Our customized Tumbleweed image (labeled as NewStorage) in the storage-ng OBS repository can now perform some simple actions during installation and display the result to the user.

First of all, when proposing the timezone settings it will, as usual, check for MS Windows installations in the disk to guess if the hardware clock should be set to UTC. The news is that check is performed using the new storage stack, that offers more functionality in every sprint.

More important is that the installer will show the partitioning proposal calculated also using the new stack. As you can see in the screenshot below, the screen is way more simpler than the one you would find in a regular Tumbleweed. There are no buttons to change the settings or to run the expert partitioner yet. That doesn’t mean the functionality is not there, it’s simply that we prefer to focus first on modifying all the installer steps to use the new stack (what will enable us to use openQA) before refining every screen to add all options there.

The new partitioning proposal

Right now the system works only in disks containing a MS-DOS style partition table and will always propose a partition-based (no LVM) setup. That’s because we prefer to solve the hardest scenarios first. Using LVM and/or GPT partition tables is less challenging than the already supported scenario.

Reduce global warming by saving OBS build power

As you may know, we use the awesome Open Build Service (OBS) to generate both the YaST rpm packages and the openSUSE/SLE ISO images. Every time the source code of any component changes, OBS rebuilds that component and all the packages that depend on it.

Our beloved openSUSE and SLE release managers told us that there were several YaST packages that often triggered rebuild of other YaST packages, that triggered yet another rebuild, that triggered… you got the idea. 😉

The mentioned problem slows down the creation of new ISO images, interferes with the continuous integration process (specially visible in Tumbleweed) and wastes valuable OBS resources.

During this sprint we reduced the rebuild time of YaST by 30%. That’s for sure interesting, but knowing the details about how we did it could be even more interesting for many readers. We feared the explanation could be too complex and technical to fit into this report… which gives us just another opportunity for blogging. So expect an upcoming post including interesting technical stuff and crazy graphs like this one.

YaST dependencies graph

Some adjustment for the installer in the LiveCDs

One of the good things about working in open source is that sometimes the evolution of the projects you have created can surprise you. Quite some time ago, the YaST team dropped support for the live installer. It was simply too demanding to keep it alive while still doing our regular work (bug fixes and new features for YaST and the regular installer).

Recently the live installer was removed from Tumbleweed, the only system in which it was still available (after having been dropped in the past from stable openSUSE releases). One could have expected that somebody would decide then to step up and take the maintainership of the live installer.

But what actually happened was that Fabian Vogt decided to try a different approach we haven’t considered – adding the standard network installer to the LiveCDs images of Tumbleweed. He managed to make it work well enough and asked us for help to debug some problems. We fixed the initial problems by disabling the self-update functionality in the LiveCDs (it’s simply not designed to work on that scenario).

There are still quite some problems to be resolved to make everything work flawlessly, but if Fabian and others don’t give up, we will keep assisting them in order to bring the installation back to the LiveCDs… even in unexpected ways.

UI Designer

The YaST user interface is quite difficult to design and code. The main problem is that there is was no interactive UI designer where you could build a dialog or modify an existing one. Instead, you had to write new code or modify the existing code which creates and opens the dialog. Then, to see your changes you had to start the YaST module, go to the respective dialog and check its content. If it didn’t look like you intended, you had to close it, modify the code and start it again. And again… and again. Very annoying especially if the dialog is hidden deep in the module and you need to take several steps to get there.

During Hack Week 14, a project to improve the situation a bit was started. We already had a dialog spy which can be opened by Ctrl+Shift+Alt+Y keyboard shortcut, but that was read-only. You could only inspect the dialog tree and see the details of the selected widget but you could not change anything.

During that Hack Week project it was improved so it could edit the properties of the existing widgets, remove them or even add some new widgets. However the code was more or less a proof of concept than ready to be merged into the YaST UI and released to public. So we decided to finish it in this sprint.

As usual, it was harder than expected… but we made it and here is a short demo showing how it works and what you can do there:

The new UI designer in action

The new tool is still far from being perfect. The most obvious missing feature is that the dialog is changed in place and you cannot save or export you changes. After closing the dialog everything is lost. But it can still help to try things in the UI or make a quick prototype, specially when discussing solutions with interface designers. Hopefully we find some more time in the future to make it even better.

Storage reimplementation: encryption support

Although the partitioning proposal still does not support encryption or LVM, we implemented full LUKS (encryption) support in the underlying library (libstorage-ng). Together with the LVM support implemented in the previous sprint, that makes the new library already a valid replacement for the old libstorage in many situations and scenarios. Now it’s mainly a matter of switching from one version to another in every single YaST component, starting with the expert partitioner that we plan to start redesigning in the next sprint.

As usual, new features in the library are hard to illustrate, unless you accept the action diagrams as screenshots. In that case, here you can see the sequence of actions performed by the library when creating an encrypted home volume.

Creation of an encrypted home with libstorage-ng

Syncing keyboard layouts and console fonts in Leap and Tumbleweed

In parallel to our Scrum sprints, we have been for some time steady working, together with the openSUSE maintainers of X.Org and systemd, in redesigning how keyboard maps and console fonts are managed by YaST. Some changes were introduced in Tumbleweed some time ago but never made it to SLE (or Leap) because they needed more testing.

Recently Ludwig Nussel, the Leap’s release manager, decided that he wanted to sync 42.2 with Tumbleweed in that regard, using the new approach instead of the more conservative SLE one. Thus, we also invested quite some time coordinating again with Stefan Dirsch (X.Org) and Franck Bui (systemd) to push the changes just in time for the beta2 version of Leap 42.2… just in time to introduce bug#1000565, that was honored with its inclusion in the list of most annoying bugs in 42.2 Beta2.

The bright side is that a fix for the bug has already been provided (see bug report) and you can now finally test the new fonts and keyboard maps. Please, do so and provide feedback in order to get a properly localized Leap 42.2 release.

See you soon

As usual, this post was just a quick overview of the most interesting part of the sprint, because most people (including ourselves) don’t want to read about the boring part of the work, which is mainly fixing bugs.

The good news is that this time you will not have to wait another three weeks to read interesting stuff about YaST. As mentioned, we plan to publish a blog post about the reduction of the build time of YaST. And we will probably also publish a post about the visit of a YaST geecko to Euruko 2016.

So this time more than ever… stay tuned!

Highlights of YaST development sprint 23

August 18th, 2016 by

As already mentioned in our previous blog post, with Leap 42.2 in Alpha phase and SLE12-SP2 in Beta phase, the YaST Team is concentrating the firepower in fixing bugs in the installer. We fixed more than 40 bugs in three weeks! The dark side is that most bug fixes are not juicy enough for writing a blog post… but there is always some interesting stuff to report.

Integration of installer self-update with SCC and SMT

The installer self-update feature integrates now with SUSE Customer Center (SCC) and Subscription Management Tool (SMT) servers. Until now, there were three different mechanisms to get the URL of the installer updates repository:

  • User defined (using the `SelfUpdate` boot option).
  • Using an AutoYaST profile.
  • The default one, specified in the `control.xml` which is shipped in the media.

Now YaST2 is able to ask for the repository URL to SCC/SMT servers. The details of how the URL is determined are documented in the repository.

Fixes and enhanced usability in dialogs with timeout

As you may know, it’s possible to install (open)SUSE in an automatic, even completely unattended, basis using AutoYaST. AutoYaST can be configured to display custom configuration dialogs to the user and wait for the reply a certain amount of time before automatically selecting the default options. Until now, the only way for the user to stop that countdown was to start editing some of the fields in the dialog.

We got a bug report because that functionality was not working exactly as expected in some cases so, in addition to fixing the problem, we decided to revamp the user interface a little bit to improve usability. Now there are more user interactions that are taken into account to stop the counter, specially we added a new “stop” button displaying the remaining seconds. You can see an example of the result below.

New layout for dialogs with timeout

Following, as usual, the Boy Scout Rule we also took the opportunity to add automated tests to make that part of YaST more robust for the future.

Automatically integrating add-on repositories during installation

Sometimes you want to extend the regular installation media by adding just a few extra packages or provide a number of fixed packages along with the media.

For this purpose, the installer can automatically register an add-on repository. All you have to do is to put the repository on the installation medium and to add a file /add_on_products.xml that points to this repository.

The file looks like this:

<?xml version="1.0"?>
<add_on_products xmlns="http://www.suse.com/1.0/yast2ns"
    xmlns:config="http://www.suse.com/1.0/configns">
    <product_items config:type="list">
        <product_item>
            <name>My Add-on</name>
            <url>relurl://myaddon?alias=MyAddon</url>
            <priority config:type="integer">70</priority>
            <ask_user config:type="boolean">false</ask_user>
            <selected config:type="boolean">true</selected>
            <check_name config:type="boolean">false</check_name>
        </product_item>
    </product_items>
</add_on_products>

You can define the following elements:

  • <name> is the name of your repository
  • <url> points the the repository location; you’ll likely want to use the relurl scheme here that gives the location relative to the main installation repository
  • <priority> is the repository priority (lesser number means higher priority, the main installation repository has priority 99)
  • <ask_user>: should the user be asked about adding the repository?
  • <selected>: should the repository be automatically selected?
  • <check_name>: should the repository’s actual name be matched against the value of the <name> element?

You can of course list several repositories in this file.

If you’re too lazy to remember all this, mksusecd can do all this for you.

For example, if you have a set of new kernel packages you would like to use, do:

mksusecd --create new.iso --addon kernel-*.rpm --addon-name 'my kernel' sles12-sp2.iso

This creates a new iso based on sles12-sp2.iso that will install your new kernel packages instead.

Storage reimplementation: small steps for the code, giant leap for continuous integration

During bug squashing we managed to find some time to keep the storage stack reimplementation rolling… slow and steady. The customized Tumbleweed images (labeled as NewStorage) in the storage-ng OBS repository are already able to analyze most systems, creating a representation of the system storage devices in memory that will be used to manipulate the disks and propose a partitioning schema. Unfortunately, this representation is only visible in the YaST logs since fixing installer bugs was more urgent than representing that information in the UI.

This turned to be an important milestone, not because of the functionality itself or the value of the code (we just added a couple of lines of Ruby code), but because for the first time the dependencies in some packages were switched from libstorage to libstorage-ng. That had important implications for the code organization and for our continuous integration infrastructure, specially the Travis CI integration, which implies the generation of .deb packages. We can now say that our continuous delivery workflow (from Github to OBS, passing through Jenkins, Travis, Coveralls and Code Climate) is free of any trace of the old storage code.

In addition, we also did some good progress in LVM support in the new library, being able to recognize and manipulate in memory all kind of LVM structures.

The joy of openness: updating the SUSE Linux Enterprise documentation

An important part of our job, specially as a new release date approaches, is helping to shape the SUSE Linux Enterprise (SLE) documentation. One of the strongest points of SUSE products is the awesome SUSE’s documentation team which, as the rest of the company, have open source in their genes. Suggesting improvements and updates for the documentation is as straightforward as creating a pull request in the completely open documentation repository at Github… and anybody can do it!

The documentation team uses Docbook, but they would accept contributions in other formats (e.g. Markdown) and transform it themselves into Docbook… just because they are that cool. 🙂

Better support for ARM systems using EFI

The world is getting full of cool ARM64 devices and both SUSE and openSUSE are actively working in supporting as many of them as possible. We took another small step during this sprint improving the installer’s partitioning proposal for ARM systems that use EFI.

That’s not all, folks

As we always say, this was just the small portion of the work done that we consider exciting enough to be part of our development reports, since we don’t want to bore you with details about every single fixed bug. During this installer bug-fixing phase, this is more true than ever and the next sprint, which is already planned, will be similar to this in that regard. Nevertheless, in the next report we expect to have some interesting news about the installer self-update functionality and about the LVM support in the new storage stack. Stay tuned.

How to create an openSUSE Banana Pi M1 image with MATE Desktop

September 17th, 2015 by

I won a Banana Pi from ownCloud. So I tried to install openSUSE.

There are 3 options:

1. According to the wiki page, you can download the image they provide but there’s no kernel support for Mali400MP2 GPU (who knows if it’s fixed by now). No Mali mean no GUI. The link to image is http://download.opensuse.org/ports/armv7hl/tumbleweed/images/.

2. Download the image from http://www.lemaker.org. The GUI used is XFCE.

3. Do it the hard way, build it yourself. I would like to install MATE. I know, I could use the lemaker image.
I followed the page HowTo Build Banana Pi Image.

This post has 2 sections. The first is how to create the SD card and the next one is how to install MATE.
(more…)