Home Home > Uncategorized
Sign up | Login

Archive for the ‘Uncategorized’ Category

Highlights of YaST Development Sprint 44

October 11th, 2017 by

Here is the YaST team again with a new report from the trenches, this time with a small delay over the usual two weeks. Most of the team keeps focused on the development of the upcoming SUSE Linux Enterprise 15 products family, including openSUSE Leap 15. That means finishing and polishing the new storage stack, implementing the new rich ecosystem of products, modules, extensions and roles (one of the biggest highlights of the SLE15 family) and much more. So let’s dive into the most interesting bits coming out of the sprint.

Hostname configuration during installation

And let’s start with one of those stories that illustrate the complexity hidden below the user-oriented YaST surface. During installation is very common to assign a hostname to the machine being installed to identify it clearly and unequivocally in the future.

Usually it is a fixed hostname (stored in /etc/hostname) but in some circumstances is preferable to set it dynamically by DHCP. Since some time ago (as you read in a previous report and is shown in the image below) YaST allows to set the hostname selecting a concrete interface or with a system-wide variable named DHCLIENT_SET_HOSTNAME which is defined in /etc/sysconfig/network/dhcp. The value to be set for such variable during installation can be optionally read from the distribution control file. Last but not least, as you already know, Linuxrc can also be used to enforce a particular network configuration.

YaST DHCP configuration with several network interfaces

Most users have a simple setup that works flawlessly, but we recently got a bug report about a wrong network configuration after installing the system if the hostname configuration was set via Linuxrc. After some research we found that the value of DHCLIENT_SET_HOSTNAME coming from the control file was overwriting the Linuxrc configuration at the end of the installation. Now it’s fixed and the global variable will be set by the linuxrc sethostname option if provided or loaded from the control file if not. And all that happens now at the very beginning of the installation to give the user to chance to modify it and to ensure the user’s choice is respected at the end.

Setting hostname in Linuxrc

Take into account that with multiple DHCP interfaces the resulting value for DHCLIENT_SET_HOSTNAME is not fully predictable. Hence, in that scenario we recommend to explicitly select the interface which is expected/allowed to modify the hostname.

Extending the installation process via RPM packages

As we have mentioned (a couple of times) during latest reports, we are implementing multi-product support for the installer. It means that SUSE will ship several products on a single installation media.

One interesting feature is that products, modules and extensions can define its own installation roles. For instance, if you select the desktop extension, you will be able to select GNOME as system role.

During this sprint, we have improved roles definitions handling, displaying a different list of roles depending on which product was selected.

As a side effect, we added support for sorting roles assigning them a display order.

Getting Release Notes from the Installation Repository

As part of our effort to drop SUSE tags from the installation media, we improved the way in which release notes are handled during installation.

Release notes are downloaded from openSUSE or SUSE websites in order to show always the latest version. Of course, the installation media includes a copy of them, which may be outdated, to be used when there is no network connection.

From now on, instead using some additional files, this offline copy of release notes will be retrieved from the release-notes package which lives in the packages repository. So we do not need to ship additional files containing release notes in the installation media anymore.

Moreover, although the old approach worked just fine in almost all cases, there was an uncovered scenario. Let’s consider a system which have access to an updated packages repository but is not connected to Internet. That could be the case, for instance, if you are using SUSE Subscription Management Tool (SMT). With the new approach, the installer will get release notes from that repository instead of displaying the (potentially outdated) ones included in the installation media.

Additionally, we refactored and clean-up a lot of old code, improving also test coverage.

Storage reimplementation: bringing more features back

We are also working hard to make sure the brand new yast2-storage-ng includes all the features from yast2-storage, in addition to the new ones. That means that, after this 44th sprint, SLE15 is already able to perform the following operations using the new module.

  • Creating MD software RAID devices in the expert partitioner. This feature is specially relevant for many openQA tests that rely on it.
  • Displaying the compact description of the partitioning proposal in the one-click-installer screen used by SUSE CaaSP and openSUSE Kubic
  • Importing users and SSH system keys from a previous (open)SUSE installation.

One-click-installer view on SUSE CaaSP 4.0 (yast2-storage-ng)

Rethinking LVM thin provisioning

When trying to create a thin-pool using all free space the metadata has to be accounted for. In contrast to linear LVs the metadata for thin-pool uses space of the VG. For instance, if there are 2048 GiB free in the VG, the metadata for a maximal size thin-pool is about 128 MiB and the pool can be about 2047.9 GiB big.

Additionally LVM creates a spare metadata with the same size. This spare metadata is shared between all pools and thus has the size of the biggest pool metadata. The spare metadata can be deleted manually and all pool metadata can also be resized.

When starting with an empty VG it is relative easy to account for the metadata. But how to handle this with an already existing volume group? Also take into account a volume group containing e.g. RAID LVs or cache pools (which also have metadata).

We finally decided that, during probing, YaST will check how much free space the VG has and then it will calculate “reserved” value for the volume group:

reserved = total size - used by LVs the library handles - free

So when calculating available space for a normal or thin pool, it will take the “reserved” into account:

max size = total size - reserved - used by LVs the library handles

The only drawback is that the maximal size for the pool can be smaller than actually possible since e.g. the spare metadata might be shared with an already existing thin pool.

More to come

The 45th sprint has already started and you can expect more and more work in the installer for SLE15 and openSUSE Leap 15 and more news regarding the revamped storage stack. Meanwhile, don’t forget to have a lot of fun!

SUSE Support Lands Upstream In cloud-init

September 21st, 2017 by

Well it’s been many many years and many many releases that we’ve been carrying a large number of patches for the cloud-init package in openSUSE and SUSE Linux Enterprise. I remember the first semi serious implementation of SLES support happened when I worked with HP to get SLES into the HP Public Cloud offering, which was based on OpenStack. The offer was eventually named Helion Public Cloud and then eventually shut down. Yes, it’s been many many years and I have received many questions about when is SUSE support going to be upstreamed, and my answer was always, “when I get around to it“. Well, it finally happened, in big part thanks to the cloud-init summit which was held for the first time earlier this year. Google in Seattle was a great host and I very much appreciate that I was invited.

Anyway, long story short spending some face time with other contributors and working out the kinks that existed in the pipeline worked wonders.  Rather than sending a small patch here and there the main implementation for openSUSE and SLE, lots of code, were accepted shortly after the cloud-init summit and over the last couple of days another couple of patches took us another step forward.

There are a few more loose ends that need work but with 17 patches removed from the package build, currently building in Cloud:Tools:Next in OBS we’ve made major progress.

Well, I for one am happy about this, and those that want to install from source can do so and have openSUSE and SLE support working from the upstream sources and not just from the packages included with openSUSE and SLE.

Thanks to Canonical for organizing the summit to get everyone together and thanks to Google for hosting the summit.

Oh and before I forget, getting the changes accepted was not the only major step forward, openSUSE Leap 42.3 will, in the not too distant future, like in the next couple of days, be integrated into cloud-init testing using containers the lxd project builds, go figure who knew these even existed.

Highlights of YaST Development Sprint 43

September 21st, 2017 by

The summer is about to end (in Europe) and it is time for another YaST Development Sprint report. As usual, storage-ng has been one of the stars of the show, but new installer features for upcoming SUSE/openSUSE versions have received a lot of attention too. Also CaaSP 2.0 got some love from us during this sprint.

storage-ng: Udev mapping and ARM64 support

The new storage layer is getting better everyday. After the big amount of work that came with the re-implementation, the team is trying to unleash the power of the new design.

Udev Mapping is Back

The bootloader module supports using persistent device names provided by Udev. It is a pretty useful feature that comes in handy in many situations but it was missing in the storage-ng based version of the module.

But fear not: this feature has been re-implemented taking advantage of the much improved API of the new storage layer. And that’s not all: the team also took this opportunity to clean up some code and document the strategy for picking Udev device names in a proper way.

Do you want more details? Here you have them. Let’s start with the scenarios we support:

  • S1: Disk with the booting configuration is moved to different machine
  • S2: Disk dies and its content is loaded to new disk from a backup (because you have a backup, right?)
  • S3: Path to disk breaks and is moved to different one

Given these scenarios, let’s have a look at the strategies:

  • If the device has a filesystem with its mount_by, do not change it.
  • If the device names includes a by-label, just use it. This behaviour enables us to handle the three scenarios.
  • If there is by-uuid, then use it. It can also handle the three scenarios.
  • If there is by-id, use it. It can handle S3, but not always.
  • If there is by-path, just use it. It is the last supported Udev symlink and, at least, it will prevent the name changing during boot.
  • As fallback, just use the kernel name (for instance, /dev/sda3).

storage-ng now works also on ARM64

For quite some time, the storage-ng code has been tested (and worked) on x86_64, ppc64 and s390x architectures. Now, we have added aarch64 to match the list of supported architectures in SUSE Linux Enterprise and openSUSE Leap.

Speeding up the Service Manager Startup

As you may know, YaST includes a nice module for managing systemd services. Compared to systemadm, included in the system-ui package, there are some key differences:

  • It displays only services, not other unit types such as sockets.
  • It can enable/disable them for the next boot.
  • It works, as any YaST module, in textmode (even on 80×25 terminals).

Some time ago we got a report that the presented information was inaccurate in some corner cases. We fixed that but also made the module a lot slower at the same time. It did not take long for that to be reported.

We tested the following scenario: 286 services (SUSE Linux Enterprise 12 SP3 with nearly all software patterns installed) on a not very fast virtual machine. Normally, you should have fewer services and probably a faster system, but we wanted to fix the issue even for the worst scenario.

After analyzing the problem, we found out the root cause: too many calls of systemctl, at least 3 times per service (show, is-active, is-enabled). With a couple dozen milliseconds per call, it quickly adds up.

The fix was to combine all show calls into one and correctly interpreting the ActiveState property to eliminate all of the is-active calls. But you want to know the numbers, right? After the fix, the startup time went down from 69 to 15 seconds (bear in mind that it is a slow virtual machine).

So even if your system is not that slow or you have fewer services installed, you may benefit of a shorter startup time for this module.

Multi Repository Media

There are some hidden gems in YaST that are maybe not well known although they have been there for a long time. One of those features is support for multi repository media.

What actually is a multi repository medium? Imagine a CD or DVD with several independent repositories. The advantage is that, if you want to release several add-ons, you can put them all on a single DVD medium. Really nice stuff, isn’t it?

Up to now YaST added all repositories found on the medium automatically without any user interaction. In this sprint we have added a new dialog into the workflow which asks the user to select which repositories should be used. Of course, if there is only one repository, it does not make sense to ask and the repository is added automatically.

Selecting which add-ons should be installed

i18n support for CaaSP 2.0

On June 2017, SUSE released the first version of the promising SUSE CaaS (Container as a Service) Platform. The YaST team actively worked on this project by adding several new features to the installer, like the one-dialog installation screen.

That very first version of CaaSP was only available in English. However, version 2.0, which is around the corner, will support more languages. For the YaST team, it means that we needed to add the language selector to the
installer, as you can see in the screenshot below, and to mark every string in the yast2-caasp for translation.

YaST2 CaaSP features a language selector

Finally, if you are interested in CaaSP, maybe you would like to check out Kubic, its Tumbleweed based variant.

More bug fixes

Apart from the new and shiny stuff, the YaST team was able to fix quite some issues during this sprint. Let’s have a look at some of them.

Taking Care of Small Details

Not enough space for device name

Usability is a critical point for a project like YaST. From time to time, we receive a bug report about some usability problem that needs to be addressed and we took them very serious. In this case, the bootloader module had a problem when showing long device names in the dialog to change the order of disks. So we just needed to do a minor fix in order to ensure that there is enough space.

Device name is shown properly

Please, keep reporting usability related issue you find in order to make YaST even better.

Fixing the INSECURE mode

It sounds scary, but YaST supports an insecure mode during installation. What does it mean? YaST is pretty flexible when it comes to system installation. You, as a user, has the power to modify/tweak the installer (using a Driver Update Disk), add custom repositories, etc. And sometimes it can be useful to skip some checks.

Some time ago, libzypp introduced a new callback to inform about bad (or missing) GPG signatures. This callback was properly handled by AutoYaST but it was ignored in regular installations, so the user always got the warning about the failing signature, even when running on insecure mode.

Now the problem has been fixed and you can run the installer in insecure mode if you want to do so.

Learning about FCOE

YaST deals with a lot of moving parts and, although it can be daunting for the newcomer, it also has a bright side: we regularly learn new stuff to play with.

In this case, Martin Vidner, one of our engineers, had to deal with a fix related to the Fibre Channel over Ethernet support in YaST. But instead of blindly applying the patch that we already had, Martin decided to learn more about the topic sharing his findings in its own blog. Sure it will be a valuable resource to check in the future.

Increase the thread/process limit for Chrome and Chromium to prevent “unable to create process” errors

July 25th, 2017 by

Browsers like Chrome, Chromium and Mozilla Firefox have moved to running tabs in separate threads and processes, to increase performance and responsiveness and to reduce the effects of crashes in one tab.

Occasionally, this exhausts the default limit on the amount of processes and threads that a user can have running.

Determine the maximum number of processes and threads in a user session:

$ ulimit -u
1200

The SUSE defaults are configured in /etc/security/limits.conf:

# harden against fork-bombs
* hard nproc 1700
* soft nproc 1200
root hard nproc 3000
root soft nproc 1850

In the above, * the catch-all for all users.

To raise the limit for a particular user, you can either edit /etc/security/limits.conf or create a new file /etc/security/limits.d/nproc.conf. Here is an example for /etc/security/limits.d/nproc.conf raising the limit for the user jdoe to 8k/16k threads and processes:

jdoe soft nproc 8192
jdoe hard nproc 16384

If you want to do that for a whole group, use the @ prefix:

@powerusers soft nproc 8192
@powerusers hard nproc 16384

In either case, this change is effective only for the next shell or login session.

Fun things to do with driver updates

March 16th, 2017 by

Today: But what if I need a new kernel?

A driver update (DUD) can of course update a single driver. But if that’s not enough and you need a whole new kernel to run an installation?

There are two parts to solve:

  1. replace the kernel used during installation and
  2. get the new kernel installed

We’ll need two tools for this (both available in Tumbleweed or here: mksusecd and mkdud).

1. Replace the kernel used during installation

For this it’s important to know which kernel packages you’ll actually need. Typically it will be kernel-default and kernel-firmware. But older SUSE distributions (SLE 11 comes to mind) had the kernel packages split into kernel-default and kernel-default-base – you’ll need them both.

To make things confusing, modern SUSE distributions also have kernel-default-base – but it’s an alternative to kernel-default. In this case we don’t need it.

If unsure, check kernel-default. If it contains the actual kernel (e.g. /boot/vmlinuz) then you don’t need kernel-default-base.

On some architectures modules are also taken from xen-kmp-default. If that’s important for you, you can add this package to the kernel list as well.

In fact you can add any number of kernel packages or kmps you like.

In the past, sometimes a different kernel flavor was used. For example PowerPC had kernel-ppc64 for a while. Simply use the flavor you need.

It’s a good idea to gather all the kernel rpms into a single directory for easier use:

> mkdir k
> cp kernel-default.rpm kernel-firmware.rpm k
> cp kernel-default-base.rpm k    # only if needed
# add any kernel-related rpms you need

Then, take your SUSE installation iso and run

> mksusecd --create new.iso \
  --kernel k/* -- \
  original_dvd1.iso

Note that the --kernel option accepts a variable number of arguments, so you have to add an isolated -- to terminate the argument list properly.

The output could look like this:

> mksusecd --create new.iso \
  --kernel k/* -- \
  SLES-11-SP4-DVD-ppc64-GM-DVD1.iso
kernel version: 3.0.101-63-ppc64 --> 3.0.101-94-ppc64
CHRP bootable (ppc64)
building: 100%
calculating sha1...

The command above will actually get the list of required modules from the old installation iso. If you are missing some driver or the new kernel comes with some additional driver, the module will not be added to the new iso.

But there’s the --modules option. It will add the listed modules together with any implicitly required modules via module dependencies.

For example, let’s add the airport wifi-module to our PowerPC iso:

> mksusecd --create new.iso \
  --kernel k/* \
  --modules airport -- \
  SLES-11-SP4-DVD-ppc64-GM-DVD1.iso
kernel version: 3.0.101-63-ppc64 --> 3.0.101-94-ppc64
kernel modules added:
  airport, cfg80211, orinoco
CHRP bootable (ppc64)
building: 100%
calculating sha1...

As you can see, it automatically adds orinoco and cfg80211 as well.

2. Get the new kernel installed

This is relatively simple. A driver update can do this:

> mkdud --create foo.dud \
  --dist sle11 \
  --install repo \
  k/*

This creates a driver update for SLE 11 (which also applies to SP4) and the kernel rpms are installed via an auto-generated add-on repo (--install repo).

Now we have the driver update that installs our kernel packages. But how do we use it?

We integrate it into our iso above!

> mksusecd --create new.iso \
  --initrd foo.dud \
  --kernel k/* -- \
  SLES-11-SP4-DVD-ppc64-GM-DVD1.iso

mksusecd has an --initrd option that directly accepts driver updates and integrates them into the iso.

3. Can I have a choice?

Maybe you just want to test this new kernel or sometimes need the old one and sometimes the new one. Can you make an installation iso that lets you choose the kernel?

Oh yes! 🙂

> mksusecd --create new.iso \
  --add-entry 3.0.101-94 \
  --initrd foo.dud \
  --kernel k/* -- \
  SLES-11-SP4-DVD-ppc64-GM-DVD1.iso

This does not replace the old kernel but adds a new boot entry Installation - 3.0.101-94.

So you can install with old or the new kernel.

TOSprint or not to sprint?

February 27th, 2016 by

TOSprint Paris 2016

Report of a week of sprint

TOSprint in Paris has just ended (wiki page).

What a week!

First of all I want to warmfully thank the sponsors, especially Olivier Courtin from Oslandia for the organization, and Mozilla France for hosting us.

What is TOSprint?

Once a year a bunch of hackers from projects under OsGeo umbrella, meet in a face to face sprint.
This year it happenned in Paris with great number of participants (52).

There was globally five big groups, and if each community was running its own schedule,
there was a lot of cross echanges too.

TOSprint Mozilla

Mateusz Łoskot

Personal objectives

My main objective, except being enough luckly to be a sponsor, was to go there and be in direct contact with upstream.

This can help a lot to improve packages, and create new ones.

Moreover, as one of my openSUSE’s Application:Geo peer maintainer, Angelos Kalkas was also participating, we decided to make somes changes, and improve the situation of the repository.

(more…)

Cleanup on OBS – Bacula packages to adopt

October 25th, 2015 by

I was one of the creator and maintainers of bacula packages on obs (Story started 4 years ago).
Since then (+3 years) I’m using another tool called Bareos, and have no more interest
nor time to maintain those packages.

Since sometimes, bacula project has released two main major version. (7.0 and 7.2).
Nobody has taken care of the packages available on obs Archiving:Backup, nor have
proposed an update. [1]

Those packages have never been submitted to Factory, and proposed to any released openSUSE version.
From one of the last upstream announcement Bacula project will deliver directly prebuild packages for major distributions.

So if you’re interested to take ownership of those, raise your hands now (ask maintainer status with obs interface).
I will let a grace period of one month before sending a delete request.

Have fun!

Lizards, time to pack your stuff for openSUSE miniSummit @Scale 13x

February 17th, 2015 by

Again this year the thirteen annual Southern California Linux Expo is in the starting block.

scale_13x_onair

During 3 days you will be able to visit us at our booth (38,39,40).
Yeah 3 booths cause we co-run the KDE and Gnome booth.
The exhibition hall open Friday afternoon at 2pm.


Drew and Peter are working as daemon to get everything ready to spread, Doug have brought also quite numerous goodies there. I will do my best to inform you here or follow my G+ channel
scale_13x_hands-feets

Whatever the way you come, bring your feet there and shake hands.
On Thursday, no one has to miss our full day of openSUSE mini-summit, room Century AB.
There will be interesting talks and also a full green hallway, We’re looking forward SUSE’s team, working together in this promising adventure.

scale13x_luggage_ready
On our side Geeko is ready to cross 9.000 kilometers tomorrow.
Hey cool first time in the famous Airbus A380…
Demo laptop with Tumbleweed and KF5 is also secured.



See you all in Los Angeles

Final Election Thoughts

January 22nd, 2015 by

As the openSUSE Board election quickly draws to a close, I take a moment to remind openSUSE Members to vote.

Elections and voting are a rather remarkable thing.  We take up or values, our beliefs, our interests, and briefly toss them in the air, letting the air flow somewhat disrupt them as they gently fall like leaves.  We think for a moment: What’s important to me?  Why is this important to me?  And does it really matter who I vote for?  Can a small Board of Directors really make a difference in the larger scheme of things?  Can the elected person really represent my interests?

In American politics, there is much apathy for and little trust in elections.  There are many reasons for this (and it’s a topic for discussion and reflection saved for a very different forum).  But as much analysis is done by professionals who analyze these things, they often conclude that each individual doesn’t think that one vote will matter.  “I’m just one person in an ocean of people; how will my vote matter?”  But for us, well, no one knows better about Freedom and Openness than an opensource community.  Voting — expressing one’s confidence in another person to serve as a representative of an ideal, a promise — is a hallmark of Freedom and Openness.  If only one vote is cast among many, then the ideal of only the one is expressed.  If your vote is not cast and the others of your community are, then your voice is diminished.  When a whole community votes, the full range voices is heard and discerned.

Vote.  Our community is strengthened when your vote is cast.  Whether or not you vote for me, please vote.

I believe it matters.

The election ends on Monday, 26 Jan 2015.

Standing for Re-Election to the openSUSE Board

January 12th, 2015 by

Hi Fellow Geekos,

This post is a summary of my wish to continue to serve on your behalf on the openSUSE board.

My term has been a short one, as I was appointed to serve out the remainder of Vincent Untz’ term.

While I now work for SUSE (and it has been a fabulous experience), that does not change my view or efforts to contribute to openSUSE.  Prior to joining SUSE, as a part of the Sales Engineering team, I was elected to the board for a two year term. To avoid having too many SUSE employees, I had to step down.  A rule I completely support.

Having been an openSUSE member long before joining SUSE, I think I have a keen awareness of what the community is about and where we can improve together.

My take is we have an awesome group of contributors who want to see the community grow and prosper.  With things like Tumbleweed and OBS, among others, we have shown real innovation and technical leadership in the Linux world. We have arguably one the top distros available.  It is solid, polished and usable for a wide variety of use cases. We need to keep the openness and solid collaboration which enables everyone to participate and succeed.

My efforts within openSUSE have been mostly focused on the Open Build Server, maintaining several projects, as well as, being part of the Factory review team.

Going forward, I want to concentrate on reaching out beyond our community to build more awareness of what an awesome distro we have, along with a pretty friendly community. I see other, less compelling, distros getting more visibility than perhaps is deserved.

Along the same lines, my take is we as a community can take a more active role in bringing in new members, who might not be technical folks, but can help in the marketing and outreach. I’ve started a local SUSE group through meetup.com which is a different way to find new users and contributors.

One other reason I wish to remain on the board, is we have a solid working relationship and there is a lot of mutual respect and good collaboration.

No matter who is elected to the board, I am very pleased with the caliber of the folks running and know the community will be in good hands moving forward.

Thank you in advance for your vote!

Peter Linnell