But don’t worry much, we still have enough information to keep you entertained for a few minutes if you want to dive with us into our summer activities that includes:
Bcache technology made its debut in YaST several sprints ago. You can use the Expert Partitioner to create your Bcache devices and improve the performance of your slow disks. We even published a dedicated blog post with all details about it.
Apart of the Expert Partitioner, AutoYaST was also extended to support Bcache devices. And this time, we are pleased to announce that … we have fixed our first Bcache bug!
Actually, there were two different bugs in the AutoYaST side. First, the auto-installation failed when you tried to create a Bcache device without a caching set. On the other hand, it was not possible to create a Bcache with an LVM Logical Volume as backing device. Both bugs are gone, and now AutoYaST supports those scenarios perfectly.
But Bcache is a quite young technology and it is not free of bugs. In fact, it fails when the backing device is an LVM Logical Volume and you try to set the cache mode. We have already reported a bug to the Bcache crew and (as you can see in the bug report) a patch is already been tested.
This sprint we also touched our development documentation, specifically we documented our process for creating the maintenance branches for the released products. The new branching documentation describes not only how to actually create the branches but also how to adapt all the infrastructure around (like Jenkins or Travis) which requires special knowledge.
We will see how much the documentation is helpful next time when somebody has to do the branching process for the next release.
We do our best to write code free of bugs… but some bugs are smarter than us and they manage to survive and reproduce. Fortunately we used this sprint to do some hunting.
select_product
control file option work as it’s really expected. That is, selecting the given product in addition to displaying its license. See the pull request containing the fix for (a lot of) additional information.
undefined method
errors. See yast/yast-packager#462.
Those are only some examples of the kind of bugs we have fought during this sprint. But checking bug reports has also made us think in the future…
We fixed a bug related to how the focus was managed in text mode after changing any setting via the hyperlinks available in the installation summary.
The implemented solution is actually not perfect, it’s just the better we can do with our set of widgets. And that was yet another example of such problem – LibYUI is an awesome library that allows us to create interfaces that work in both graphic and text modes, but it has basically not evolved for more than a decade… and it’s time to fix that!
So we have been discussing how to organize our time in the close future to leave some room for innovation and renovation regarding LibYUI and the YaST UI in general. Stay tuned for more news.
The YaST Team will keep working during the rest of the summer sharpening our Linux Swiss army knife. But half of the team is still on vacation or starting their vacation now. So most likely our next report will be here in two weeks and it will also be a light read.
Meanwhile, don’t forget to have a lot of fun!
]]>To get to the root cause of the issue, we enabled remote debugging with
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=ci.suse.de -Dcom.sun.management.jmxremote.password.file=/var/lib/jenkins/jmxremote.password
and attached visualvm to see what it was doing.
This showed the number of threads and memory usage in a sawtooth pattern. Every time the garbage collector ran, it dropped 500-1000 threads.
Today we noticed that every time it threw these java.lang.OutOfMemoryError: unable to create new native thread
errors, the maximum number of threads was 2018… suspiciously close to 2048. Looking for the same time in journalctl showed
kernel: cgroup: fork rejected by pids controller in /system.slice/jenkins.service
So it was systemd refusing java’s request for a new thread and jenkins not handling that gracefully in all cases.
That was easily avoided with a
TasksMax=8192
Now the new peak was at 4890 live threads and jenkins served all Geekos happily ever after.
]]>So welcome to our usual selection of selected bug-fixes (listing them all would be boring) and exciting new stuff. This edition includes:
During this sprint we fixed a bug that was causing Snapper to crash under very specific circumstances. The scenario was quite unusual so we had to request quite some information from the reporter of the bug to confirm what was happening. As a nice consequence, in addition to having now a more robust Snapper (one bug killed) you can also enjoy a new page in the openSUSE wiki listing the information you should attach to Bugzilla if you find a bug in Snapper while using (open)SUSE.
Which is also a nice excuse to remind you about the equivalent "Report a YaST bug" page.
Many of the YaST users and of our blog readers are not native English speakers that surely appreciate the fact that YaST and (open)SUSE in general can be used in several languages. But have you ever thought about the implications of developing a multi-language software? Sure? In all of them?
Human languages are so diverse as the human cultures and there are many details to take into account, from the usage of different alphabets to the various ways of dealing with genre or number (in English the words have just one form for singular and another for plural, but that can be way more complex in other languages). In today’s issue we will take a look to one of our favorite translation issues – languages that are written from right to left, like Arabic.
Dealing with text that is a mixture of Latin and Arabic script is complex and sometimes we have to deal with interesting bugs. Fortunately we have our own weapon to fight those bugs. If in Star Wars they have protocol droids like C3PO, in the YaST team we have Martin Vidner, which is the closer human equivalent.
He fixed all the reported bugs and even created a tool to help debugging similar problems in the future. You can find the source code of that tool in Github. There is even a hosted instance of the tool to be used by translators or anyone who is curious.
Now, even complex interfaces like our Partitioner look correct enough in right-to-left languages, so we will not have to send mirrors to all our Arabic users.
If you want to know more about this exciting but very complex problem of bidirectional texts, you can start with the following documents.
On related news, we got also some reports about some problems visualizing bullet-points in Korean with the beta versions of the future SLE-15-SP1. But as we could verify, all those problems are gone now.
Other area that has received some attention in this sprint is the storage management. Three related features needed adjustments before the upcoming (open)SUSE releases:
One of the last storage features that the YaST Team has developed is the support for Bcache devices in the Expert Partitioner. While our QA team was testing it, they found a bug. The Partitioner was complaining because the boot disk did not contain a partition table, which is a mandatory condition for a Legacy (non-UEFI) x86 system. But it was a bogus warning, since they had actually defined a /boot
partition in another disk.
That’s how we found that our Partitioner gets confused if there is a separate partition mounted at /boot
and located in a different disk than the root file-system. The Partitioner insisted in considering the disk containing /
to be the one that would be using for booting, instead of checking the structure of the disk containing /boot
. Now that is fixed and the improvement will be available for the upcoming SLE 15 SP1, Leap 15.1 and, of course, openSUSE Tumbleweed.
But that was not the only storage bug fixed just in time for the upcoming releases. Some sprints ago, the Storage Proposal algorithm for the initial proposal was modified to try installing on each of the individual disks. If the installation was not possible over a given disk, even after disabling all optional configurations (e.g., snapshots and separate /home
), a new proposal is tried over the next disk and so on. The problem was that the disabled options in the previous attempt were not restored back when switching to the next disk. This caused some ugly side effects, for example, if the swap partition was disabled when trying over the first disk, then the proposal did not try to create a swap when it was performing the proposal over the next disk. But now this is also fixed and it will work as expected.
And last but not least, AutoYaST now supports to install over a Network File System (NFS). This feature was left back when the new YaST storage stack was re-implemented for SLE 15 GA. Actually, this is a non documented feature. That’s why we overlooked that SUSE 12 was able to do it using some hacks and a non-validating AutoYaST profile. But no worries, the feature is available again and such profile will work now in any updated SLE-15 or Leap 15.0. Of course, it will also work while installing SLE-15-SP1 or openSUSE Leap 15.1 and Tumbleweed.
Nevertheless, we are working on a better and documented way of supporting that scenario in the future, with no need to twist the specification of the AutoYaST profile. Stay tuned for more information.
And now that the storage layer looks sane and healthy for the upcoming releases, we also took some time to think about future improvements. As you know, the storage Guided Setup always proposes to create special boot partitions as needed on each case. That can be a BIOS BOOT (for Legacy x86 systems with GPT), an ESP (for UEFI systems), PReP (for PPC systems) or zipl (for S/390 systems). Strictly speaking those partitions doesn’t have to always be in the same disk than the root partition and in some cases having it on a separate one can have some advantages (like sharing the ESP partition with another operating systems).
But we have been reconsidering all the cases, the expectations of most users and of the majority of BIOS vendors and the known bugs in other operating systems about sharing boot partitions. We have decided to be more strict in the future about the location of those partitions. Starting today with openSUSE Tumbleweed and in the 15.2 releases of openSUSE Leap and SLE, the Guided Setup will always propose those partitions in the system disk. That is, in the disk containing /boot
and the root filesystem.
Those who follow this blog know that we invested quite some time on the last couple of years rewriting the part of YaST that was more buggy and harder to modify – the storage stack. And surely you have already noticed that since we did it we are introducing new features at a very good pace (like bcache, more powerful Partitioner, Raspberry Pi support, etc.) and fixing the reported bugs in a matter of days or even hours.
The next in our list of YaST areas to revamp is the networking support. And we are happy to announce that we are starting to have some visible results in that. There is still a very long road ahead and we will provide more information in upcoming reports. But at least we have already a preview of a fully rewritten management of network routes. It’s still not available in openSUSE Tumbleweed. But for those who can’t wait, here you can see the first screenshot. All based in new and clean code backed by automated tests.
The openSUSE Tumbleweed installer asks at the beginning of the installation whether to activate and use the online repositories when a network connection is available.
The reason is that the installation DVD does not contain all available packages because of the limited media size. Another advantage is that the installer might directly install newer packages than on the media, this avoids installing the older versions first and then upgrading them to the latest version.
However, in some case you might not want to use the online repositories, for example if the network connection is slow or is paid.
We got a bug report that this question was missing in the Leap 15.1. It turned out that the control.xml
file which drives the installer did not contain this step. After adding few lines into the file you can now enjoy the online repositories also in Leap 15.1!
Why are we writing about this? The reason for the missing step in the Leap 15.1 was a bit surprising. Normally all YaST packages are developed in the Git master
branch for both Tumbleweed and Leap. However, in this case the Leap 15.1 has been already branched and was developed separately, the changes in the master
went only to the Tumbleweed. And we overlooked that small difference when adding this step.
To avoid this in the future we added a pull request template with a reminder which informs the developers about this difference in the Git setup when opening a pull request.
If your project also has some unusual setup then the pull request template might a good reminder for you as well.
But the reminders about the correct branches and procedures is not the only news we have for YaST contributors and main developers. As you may remember, few years ago we switched to using Docker at Travis. That works well but we found some disadvantages of that initial setup.
The result is that a new package can be available in Travis several hours after merging the pull request. And even after triggering the build manually it still might take more than one hour.
We needed a faster cycle and the solution, as usually happens, was in the openSUSE ecosystem. As you may know, the Open Build Service is able of much more than just building packages. So we decided to make use of the OBS capacity of building Docker images.
Building both our packages and our Docker images in OBS comes with many advantages:
This means the new packages should be available in the Docker image in about 10-15 minutes after merging a pull request (for leaf packages, changing a core package which triggers a complete YaST rebuild will of course take more time).
If you want to learn more about this topic, take a look to the following links:
As usual, the content of this report is just a small subset of all the work the YaST Team does in two weeks. In this sprint, most of that work went to fixing all kind of bugs in preparation for the next releases. Big bugs, small ones, hidden bugs and embarrassingly obvious ones. Hopefully, you got a fix for your reported bug. If not, you can always stay tuned for more news after the next sprint. And don’t forget to have a lot of fun!
]]>So here is our report, brought to you by airmail!
Leaving openSUSE Tumbleweed aside, The YaST team is currently working to deliver SLE12-SP3, openSUSE Leap 42.3, SLE15, openSUSE Leap 15, SUSE CaaSP 1.0 and Kubic (more about Kubic later). Three of them are already in beta phase, which means they are being extensively tested by several parties and in many scenarios, hardware platforms and possible configurations. That amount of manual testing always result in several bug being discovered, no matter how much we try to have some automated tests for the most common cases.
Many of the bugs our testers are finding are related to internationalization and localization, mainly texts in the UI that are always displayed in English, despite the system been configured (or being installed) in a different language.
But, of course, other kind of bugs are also being found. For example, our hardware detection component (hwinfo
) was not able to deal with some new machines, making the installation experience everything but pleasant.
As a result, a significant amount of the YaST team manpower during this sprint was targeted to squash those annoying bugs. Which doesn’t mean we didn’t have time for some interesting new features and improvements.
Once again, our new storage system comes with news. Now it’s able to detect and unlock preexisting encrypted devices during the hard disks probing step, raising you a new pop-pup dialog to ask for the corresponding device password. After unlocking the devices, all your installed systems will be accessible for upgrade and, moreover, the LVM volumes allocated over encrypted devices will be activated.
The new storage stack is expected to debut in SLE15 (and, thus, openSUSE Leap 15), but the functionality can already be tested, for both the installation and upgrade processes, with the StorageNG test ISOs.
But the happiest news coming from the new storage stack during this sprint is it’s marriage with AutoYaST. The new automatic partitioning proposal (that is, the “Guided Setup”) is now integrated with AutoYaST.
Thanks to the new software architecture, AutoYaST users will be able to override every single partitioning setting from the control file.
<profile xmlns="http://www.suse.com/1.0/yast2ns" xmlns:config="http://www.suse.com/1.0/configns"> <general> <storage> <!-- Override settings from control file --> <try_separate_home config:type="boolean">false</try_separate_home> <proposal_lvm config:type="boolean">true</proposal_lvm> </storage> </general> </profile>
So you can easily switch on/off LVM, use a separate partition for /home
, enable/disable snapshots, enable/disable Windows resizing, etc. All that, still relying on the automated storage proposal to iron the details up. Something that is not possible with the current version of AutoYaST without being forced to define explicitly every partition and LVM volume.
But the simplest way to use the new libstorage proposal is to not define any setting at all in the AutoYaST configuration file. In that case, the partitioning proposal code will do the complete job, installing a new system with the default options.
Of course, before integrating the new storage stack into the upcoming SLE15, the AutoYaST support have to go one step further. Apart from using and configuring the proposal, it must be possible to define a completely custom setup including partitions, LVM volumes, software RAID devices and so on through the corresponding <partitioning>
section of the AutoYaST profile. So we used this sprint to sketch a plan to make that possible in the following months, analyzing all the scenarios and configurations supported by AutoYaST and looking for the best way to support them using the existing yast2-storage-ng
infrastructure. The outcome of that effort is this detailed document and a list of tasks (PBIs in Scrum jargon) for the upcoming sprints. So be prepared for more news in this regard.
So far the user had to ensure that snapshots created by rollbacks got deleted to avoid filling up the storage. This process has now been automated. During a rollback, Snapper now sets the cleanup algorithm to “number” for the snapshot corresponding to the previous default subvolume and for the backup snapshot of the previous default subvolume. This enhanced behavior will be available in SLE12-SP3 and openSUSE Leap 42.3. For more information take a look at the more detailed post in the Snapper blog.
For several sprints already we have been presenting features targeted to SUSE CaaSP, the Kubernets-powered solution for managing containers. Many of those features and custom configurations live in a package called yast2-caasp
, originally targeted to this great upcoming product built on top of the SLE12-SP2 codebase.
But now the package is also available for Tumbleweed-based systems by request of the Kubic project. Kubic will be the openSUSE alter ego of SUSE CaaSP, that is, a Container as a Service Platform based on openSUSE and Kubernetes. As with any other YaST component, the exact same source code will shared by the SUSE product and its openSUSE brother.
Humans make mistakes, but when the mistakes are made entering some option in
the installation command line, it usually means that a reboot of the machine is be needed to fix them.
That was the case for the registration URL (regurl
) option. In the provided address was malformed the installation just stopped. During this sprint we have added an early check of that URL which allows the user to reenter it and continue with the installation. Something that obviously improves the user experience.
In case of an autoinstallation (AutoYaST), the error is reported and the steps to get installer updates and to register the system are skipped.
There is still room for more improvements, allowing the user to also modify the URL in other scenarios. For example, for an URL with a valid format but that points to an unreachable server. But in those cases is not so straightforward to identify the culprit of the problem. It would make no sense to annoy the user with a recurring pop-up to change the registration URL if the root of the issue is not the URL but a incorrect network configuration.
As mentioned at the begining of this post, we recently got quite some bug reports about missing translations. Although some of them were really caused by bugs in the YaST code, others were a consequence of a buggy Ruby rxgettext
script which collects the translatable strings from the Ruby source code. The bug is known by the Ruby-GetText developers, but it’s unclear when (or whether) it will be fixed.
The problem is that the tool cannot collect the translatable strings from interpolations. For example it cannot find the “foo” string from this string literal: "#{_("foo")}"
. As a result, that string is missing in the resulting POT file and cannot be translated by the SUSE or openSUSE localization teams.
As a workaround, we fixed the YaST code to not use the translations inside interpolations. We also documented the possible problems when mixing translations a interpolations and their solution.
And talking about new developer oriented documentation…
YaST runs with the administrator privileges (root
) and therefore we have to be aware of the possible security issues in the code. During this sprint we published a document with a short summary of security tips for YaST developers.
If you are programming an YaST module you should definitely read it, but it might be interesting also for other programmers as many mentioned issues are generic, not tight only to YaST.
The document is available online here.
That’s all for this sprint report. We have many more things in the oven, be we didn’t manage to finish them during the sprint, so they will have to wait for the next report. Meanwhile we hope to see many of you at the openSUSE Conference 2017. There will be a whole workshop about modern YaST development, a summary with the more relevant news in the last year of YaST development, talks about the new superb yast2-configuration-management module, about our continuous delivery infrastructure and about how we use Docker to deliver YaST… And, of course, also many other interesting content like the awesome presentation from Thorsten Kukuk about the brand new openSUSE Kubic we mentioned earlier. And even more important, a lot of fun!
For those of you that cannot attend to the conference, see you again in this little corner of the internet in three weeks!
]]>In the report of our 19th sprint, we already presented the new (at that point in time) and shiny Trusted Boot support in YaST2 Bootloader. So far, the only supported scenario was legacy boot (i.e. no UEFI) on x86_64 systems. Now, thanks to TPM2, is also possible use Trusted Boot with EFI, so we added support for it in our beloved bootloader module.
So now YaST Bootloader looks the same in non-EFI and EFI variants, no matter which underlying technology is actually used. Of course, YaST is only the tip of the iceberg, booting in Trusted Boot with EFI is possible thanks to all the tools that has recently added support for TPM2. openSUSE developers and packagers rock!
For a CaaSP cluster to work properly, it’s vitally important that all nodes have their clocks in sync. So, from now on, the installer is able to configure each node in a proper way and the administration node will act as NTP server for the worker nodes.
To achieve that, the user will be asked to specify one or several NTP servers to be used as time source during administration node installation and YaST will take care of the rest (updating the configuration and enabling the service).
If a NTP service is announced through SLP, YaST will propose automatically.
For worker nodes, YaST will configure the system to keep it synchronized with the administration role.
Through the previous reports, you have been able to follow the evolution of the renovated guided setup for the partitioning proposal. This sprint is not different, we have improved and adjusted that new wizard even further, making it smarter and more usable.
The new version is able to decide which steps to show depending on the current scenario. For example, in systems with only one disk the whole disk selection dialog will be skipped. The steps are also simplified by disabling widgets that are not applicable to the current situation. For example, if there is no previous Linux installation, the question about what to do with the existing Linux partitions will be disabled.
And talking about the actions to perform on preexisting partitions, that has also been improved. In the first guided setup version, these options were only displaying for illustrative purposes, but now they are 100% real and the proposal will honor their values, so the user can easily select what to do with previous Windows or Linux partitions. We even added a third option for other kind of partitions.
Last but not least (regarding the guided setup), the password selection for encryption is now more usable, allowing the user to choose a not so strong password if really desired.
YaST is a complex and large piece software. It means that time to time some pieces that used to be great and shiny become obsolete and end up being useless. The cycle of life.
Some time ago, it was decided it was time for insserv
to enjoy retirement and it was replaced by a stub. But there were still calls to insserv
in YaST and we decided to remove them all. There were several reasons for that decision. Basicaly (open)SUSE has used systemd for couple of years already, so revisiting places where the YaST code depended on SySV was a must. As a side effect it shortened the list of YaST dependencies and, in the end, it is another small step towards smaller installation.
So good bye insserv
, it was a pleasure to work with you.
When running the installer on a mainframe in a zKVM virtual machine it displays a warning about not detected ZFCP controllers:
However, when running in a zKVM virtual machine the disk is accessible via the virtio driver, not through an emulated ZFCP controller. The warning is pointless and confusing.
The fix was basically just an one-liner which skips the warning when the zKVM virtualization is detected, but the YaST module for ZFCP doesn’t usually receive to much maintenance, so we applied our boy scout rule and improved the code a bit.
The improvements include using Rubocop for clean and unified coding style, enabling code coverage to know how much the code is tested (in this case it turned out to be horribly low, just about 4%), removing unused files, etc… You can find the details in the pull request.
As we reported in our report about Hack Week 15, we have been working on an alternative way to offer the power of the new storage library to the Ruby world. The new API is finally ready for prime-time and used in all the Ruby code.
We took the opportunity to greatly improve the developer documentation and to revamp the yast2-storage-ng README and status page.
We also did some extra checks and added automated testing to ensure our partitioning proposal ensure the requisites if a S/390 system using ZFCP, so mainframe users also have a peaceful transition to the new storage stack.
We also greatly improved the ability of the new library to work with alternative name schemes for the devices. Up to now, only using plain kernel device name (e.g. /dev/sda1
) was fully supported. Now we can do all the operations (specially generating the /etc/fstab
file) by device name, by UUID, by filesystem label, by device id and by device path.
As you already know, at least we repeat it quite often, YaST was converted from YCP to Ruby some time ago. However, this conversion was done on language basis. Some old design decisions and principles stayed, like the usage of SCR for accessing the underlying system.
Some time ago we introduced CFA (Config Files Api Gem) as a more powerful and flexible Ruby-powered replacement for SCR. Although we have been using it for a while in several YaST modules, we felt the concepts and rationales behind its operation where not that clear.
So we have invested some time improving the documentation and writing a blog post to properly present and explain CFA to the world. We will publish it next week, so stay tuned!
The next sprint will be the first in which the whole YaST team will work together integrating the new storage stack in every single part of YaST. So we hope the next report to be full of good news about the expert partitioner, improvements in AutoYaST’s handling of disks and so on.
Of course, that would be only a small fraction of all the stuff we plan to work on. See you in three weeks with more news!
]]>In 2016-06 we started the deployment with SOC6 on 4 nodes. 1 controller and 3 compute nodes that also served for ceph (distributed storage) with their 2nd HDD. Since the nodes are from 2012 they only have 1gbit network and spinning disks. Thus ceph only delivers ~50 MB/s which is sufficient for many use cases.
We did not deploy that cloud with HA, even though our product supports it. The two main reasons for that are
Then we have a limited supply of vlans even though technically they are just numbers between 1 and 4095, in SUSE we do allocations to be able to switch together networks from further away. So we could not use vlan mode in neutron if we want to allow software defined network (=SDN) (we did not in old.cloud.suse.de and I did not hear complaints, but now I see a lot of people using SDN)
So we went with ovs+vxlan +dvr (open vSwitch + Virtual eXtensible LAN + Distributed Virtual Router) because that allows VMs to remain reachable even when the controller node reboots.
But then I found that they cannot use DNS during that time, because distributed virtual DNS was not yet implemented. And ovs has some annoying bugs are hard to debug and fix. So I built ugly workarounds that mostly hide^Wsolve the problems from our users’ point of view.
For the next cloud deployment, I will try to use linuxbridge+vlan or linuxbridge+vxlan mode.
And the uptime is pretty good. But it could be better with proper monitoring.
Because we needed to redeploy multiple times before we got all the details right and to document the setup, we scripted most of the deployment with qa_crowbarsetup (which is part of our CI) and extra files in https://github.com/SUSE-Cloud/automation/tree/production/scripts/productioncloud. The only part not in there are the passwords.
We use proper SSL certs from our internal SUSE CA.
For that we needed to install that root CA on all involved nodes.
We use kvm, because it is the most advanced and stable of the supported hypervisors. Xen might be a possible 2nd choice. We use two custom kvm patches to fix nested virt on our G3 Opteron CPUs.
Overall we use 3 vlans. One each for admin, public/floating, sdn/storage networks.
We increased the default /24 IP ranges because we needed more IPs in the fixed and public/floating networks
For authentication, we use our internal R&D LDAP server, but since it does not have information about user’s groups, I wrote a perl script to pull that information from the Novell/innerweb LDAP server and export it as json for use by the hybrid_json assignment backend I wrote.
In addition I wrote a cloud-stats.sh to email weekly reports about utilization of the cloud and another script to tell users about which instances they still have, but might have forgotten.
On the cloud user side, we and other people use one or more of
to script instance setup and administration.
Overall we are now hosting 70 instance VMs on 5 compute nodes that together have cost us below 20000€
]]>I see the official documentation has full tutorial for RHEL 6 or CentOS 6 and RHEL 7 or CentOS 7. The main documentation covers Ubuntu 14.04 LTS
openSUSE already has the Nextcloud client packaged in Tumbelweed and the Server is in the PHP extra repo! Personally, I prefer to install eveything from official repository, so when an update is available, I can have it without a glitch. This tutorial describes how to install Nextcloud using command line. I followed the official documentation of Ubuntu 14.04 LTS installation.
Why choose openSUSE Leap? openSUSE Leap is a brand new way of building openSUSE and is new type of hybrid Linux distribution. Leap uses source from SUSE Linux Enterprise (SLE), which gives Leap a level of stability unmatched by other Linux distributions, and combines that with community developments to give users, developers and sysadmins the best stable Linux experience available. Contributor and enterprise efforts for Leap bridge a gap between matured packages and newer packages found in openSUSE’s other distribution Tumbleweed. You can download openSUSE Leap from the site https://software.opensuse.org/.
Make sure that ssh (sshd) is enabled and also the firewall either is disabled or make an exception to the apache and ssh services. You can also set a static IP (check out how).
First of all, let’s install the required and recommended modules for a typical Nextcloud installation, using Apache and MariaDB, by issuing the following commands in a terminal:
Create Database (optional since it’ll create eveything automatically)
Next step, create a database. First of all start the service.
The root password is empty by default. That means that you can press enter and you can use your root user. That’s not safe at all. So you can set a password using the command:
Where newpass is the password you want.
Now you set the root password, create the database.
mysql -u root -p
#you’ll be asked for your root passwordCREATE DATABASE nextcloudb;GRANT ALL ON nextcloudb.* TO ncuser@localhost IDENTIFIED BY ‘dbpass’;
Database user: ncuser
Database name: nextcloudb
Database user password: dbpass
You can change the above information accordingly.
PHP changes
Now you should edit the php.ini file.
nano /etc/php5/apache2/php.ini
change the values
post_max_size = 50G
upload_max_filesize = 25G
max_file_uploads = 200
max_input_time = 3600
max_execution_time = 3600
session.gc_maxlifetime = 3600
memory_limit = 512M
and finally enable the extensions.
extension=php_gd2.dll
extension=php_mbstring.dll
Apache Configuration
You should enable some modules. Some might be already enabled.
a2enmod php5
a2enmod rewrite
a2enmod headers
a2enmod env
a2enmod dir
a2enmod mime
Now start the apache service.
systemctl start apache2.service
systemctl enable apache2.service
Install Nextcloud from source code (option 1, preferable)
Before the installation, create the data folder and give the right permissions (preferably outside the server directory for security reasons). I created a directory in the /mnt directory. You can mount a USB disk, add it to fstab and save your data there. The commands are:
mkdir /mnt/nextcloud_data
chmod -R 0770 /mnt/nextcloud_data
chown wwwrun /mnt/nextcloud_data
Now download Nextcloud (find the latest version at https://nextcloud.com/install/). Then unzip and move the folder to the server directory.
wget https://download.nextcloud.com/server/releases/nextcloud-10.0.0.zip
unzip nextcloud-10.0.0.zip
cp -r nextcloud /srv/www/htdocs
chown -R wwwrun /srv/www/htdocs/nextcloud/
Make sure that everything is OK and then delete the folder nextcloud and nextcloud-10.0.0.zip from the root (user) directory.
Now open your browser to the server IP/nextcloud
Set your administrator username and password.
Your data directory is: /mnt/nextcloud_data
Regarding database, use the following.
Database user: ncuser
Database name: nextcloudb
Database user password: dbpass
Wait until it ends the installation. The page you’ll see is the following.
Install Nextcloud using the respository (option 2)
If you want to have automatic updates of your Nextcloud instance when there’s a new version, you can add the repository. There are packages available for openSUSE Leap 42.1, 42.2 and Tumbleweed (we recommend openSUSE Leap 42.1). You should be an administrator, so you can install Nextloud on your server.
1. Add the Nextcloud repository.
openSUSE_Leap_42.2
zypper ar http://download.opensuse.org/repositories/server:/php:/applications/openSUSE_Leap_42.2/ Nextcloud
openSUSE_Leap_42.1
zypper ar http://download.opensuse.org/repositories/server:/php:/applications/openSUSE_Leap_42.1/ Nextcloud
openSUSE_Leap_Tumbleweed
zypper ar http://download.opensuse.org/repositories/server:/php:/applications/openSUSE_Tumbleweed/ Nextcloud
2. Refresh your repositories
zypper refresh
3. Install Nextcloud (be careful you have to install LAMP first and change permissions of the files).
zypper install nextcloud
4. Open http://serverIP/nextcloud to install your instance (admin user account). Be careful to create another folder with the proper permissions for your data (as described).
5. Login and use Nextloud.
For more information about Nextcloud on openSUSE, check openSUSE wiki.
For any changes, check the github page.
For more configuration, you can follow the official documentation. That was the basic installation on openSUSE Leap.
]]>We are still dedicating quite some effort to polish the upcoming stable releases (SLE12-SP2 and Leap 42.2), but in this sprint we finally found some time to play. Which is great because blogging about new features is more fun than doing it about bug fixes.
When logging in via SSH, public key authentication should be preferred over password authentication. Until now, the best way of setting up the required authorized_keys files in AutoYaST was using the files section.
However, that approach is tedious and error prone, as you need to make sure you set the correct owner, permissions, etc. Moreover you need to keep in sync the user definition (username and home directory) with the file definition.
AutoYaST now supports the specification of a set of public keys for each user with a pretty straightforward syntax:
<user> <username>suse<username> <authorized_keys config:type="list"> <listentry>ssh-rsa your-public-key-1</listentry> <listentry>ssh-rsa your-public-key-2</listentry> <authorized_keys> <user>
AutoYaST takes care of writing the files and setting the ownership and the proper permissions.
While documenting this new feature we realized the AutoYaST documentation about users management could be more detailed, which leads us to…
Usually developers love to create programs loaded with cool features but hate to write documentation. Fortunately there are people out there who enjoy writing documentation and bringing all those features to light. We have already mentioned in previous reports how grateful we are for having the SUSE documentation team polishing and publishing our documentation drafts and how open and straightforward the process is.
We updated the YaST documentation to include information about the installer self-update feature, which will debut in SUSE Linux Enterprise 12 SP2 and openSUSE Leap 42.2. As part of the same pull request and in the AutoYaST side, some additional improvements were made, including cleaning-up some duplicated information about SUSE registration.
On the other hand and as a consequence of the above mentioned new feature, the AutoYaST documentation regarding users management has been rewritten adding missing information like groups, user defaults and login settings.
All our pull requests are already merged in the doc-sle
repository. At a later point in time, the SUSE documentation team will review and polish all the new content (including ours) and will publish an up-to-date version of the online documentation. If you don’t want to wait, you can easily generate an HTML or PDF version of the documentation including all the non-reviewed contributions just following the very simple instructions in the README file of the doc-sle
repository.
Did we already mention we love the open source, programmer-friendly processes of the documentation team?
We promised news about the storage reimplementation and here they are. Our customized Tumbleweed image (labeled as NewStorage) in the storage-ng OBS repository can now perform some simple actions during installation and display the result to the user.
First of all, when proposing the timezone settings it will, as usual, check for MS Windows installations in the disk to guess if the hardware clock should be set to UTC. The news is that check is performed using the new storage stack, that offers more functionality in every sprint.
More important is that the installer will show the partitioning proposal calculated also using the new stack. As you can see in the screenshot below, the screen is way more simpler than the one you would find in a regular Tumbleweed. There are no buttons to change the settings or to run the expert partitioner yet. That doesn’t mean the functionality is not there, it’s simply that we prefer to focus first on modifying all the installer steps to use the new stack (what will enable us to use openQA) before refining every screen to add all options there.
Right now the system works only in disks containing a MS-DOS style partition table and will always propose a partition-based (no LVM) setup. That’s because we prefer to solve the hardest scenarios first. Using LVM and/or GPT partition tables is less challenging than the already supported scenario.
As you may know, we use the awesome Open Build Service (OBS) to generate both the YaST rpm packages and the openSUSE/SLE ISO images. Every time the source code of any component changes, OBS rebuilds that component and all the packages that depend on it.
Our beloved openSUSE and SLE release managers told us that there were several YaST packages that often triggered rebuild of other YaST packages, that triggered yet another rebuild, that triggered… you got the idea.
The mentioned problem slows down the creation of new ISO images, interferes with the continuous integration process (specially visible in Tumbleweed) and wastes valuable OBS resources.
During this sprint we reduced the rebuild time of YaST by 30%. That’s for sure interesting, but knowing the details about how we did it could be even more interesting for many readers. We feared the explanation could be too complex and technical to fit into this report… which gives us just another opportunity for blogging. So expect an upcoming post including interesting technical stuff and crazy graphs like this one.
One of the good things about working in open source is that sometimes the evolution of the projects you have created can surprise you. Quite some time ago, the YaST team dropped support for the live installer. It was simply too demanding to keep it alive while still doing our regular work (bug fixes and new features for YaST and the regular installer).
Recently the live installer was removed from Tumbleweed, the only system in which it was still available (after having been dropped in the past from stable openSUSE releases). One could have expected that somebody would decide then to step up and take the maintainership of the live installer.
But what actually happened was that Fabian Vogt decided to try a different approach we haven’t considered – adding the standard network installer to the LiveCDs images of Tumbleweed. He managed to make it work well enough and asked us for help to debug some problems. We fixed the initial problems by disabling the self-update functionality in the LiveCDs (it’s simply not designed to work on that scenario).
There are still quite some problems to be resolved to make everything work flawlessly, but if Fabian and others don’t give up, we will keep assisting them in order to bring the installation back to the LiveCDs… even in unexpected ways.
The YaST user interface is quite difficult to design and code. The main problem is that there is was no interactive UI designer where you could build a dialog or modify an existing one. Instead, you had to write new code or modify the existing code which creates and opens the dialog. Then, to see your changes you had to start the YaST module, go to the respective dialog and check its content. If it didn’t look like you intended, you had to close it, modify the code and start it again. And again… and again. Very annoying especially if the dialog is hidden deep in the module and you need to take several steps to get there.
During Hack Week 14, a project to improve the situation a bit was started. We already had a dialog spy which can be opened by Ctrl+Shift+Alt+Y keyboard shortcut, but that was read-only. You could only inspect the dialog tree and see the details of the selected widget but you could not change anything.
During that Hack Week project it was improved so it could edit the properties of the existing widgets, remove them or even add some new widgets. However the code was more or less a proof of concept than ready to be merged into the YaST UI and released to public. So we decided to finish it in this sprint.
As usual, it was harder than expected… but we made it and here is a short demo showing how it works and what you can do there:
The new tool is still far from being perfect. The most obvious missing feature is that the dialog is changed in place and you cannot save or export you changes. After closing the dialog everything is lost. But it can still help to try things in the UI or make a quick prototype, specially when discussing solutions with interface designers. Hopefully we find some more time in the future to make it even better.
Although the partitioning proposal still does not support encryption or LVM, we implemented full LUKS (encryption) support in the underlying library (libstorage-ng). Together with the LVM support implemented in the previous sprint, that makes the new library already a valid replacement for the old libstorage in many situations and scenarios. Now it’s mainly a matter of switching from one version to another in every single YaST component, starting with the expert partitioner that we plan to start redesigning in the next sprint.
As usual, new features in the library are hard to illustrate, unless you accept the action diagrams as screenshots. In that case, here you can see the sequence of actions performed by the library when creating an encrypted home volume.
In parallel to our Scrum sprints, we have been for some time steady working, together with the openSUSE maintainers of X.Org and systemd, in redesigning how keyboard maps and console fonts are managed by YaST. Some changes were introduced in Tumbleweed some time ago but never made it to SLE (or Leap) because they needed more testing.
Recently Ludwig Nussel, the Leap’s release manager, decided that he wanted to sync 42.2 with Tumbleweed in that regard, using the new approach instead of the more conservative SLE one. Thus, we also invested quite some time coordinating again with Stefan Dirsch (X.Org) and Franck Bui (systemd) to push the changes just in time for the beta2 version of Leap 42.2… just in time to introduce bug#1000565, that was honored with its inclusion in the list of most annoying bugs in 42.2 Beta2.
The bright side is that a fix for the bug has already been provided (see bug report) and you can now finally test the new fonts and keyboard maps. Please, do so and provide feedback in order to get a properly localized Leap 42.2 release.
As usual, this post was just a quick overview of the most interesting part of the sprint, because most people (including ourselves) don’t want to read about the boring part of the work, which is mainly fixing bugs.
The good news is that this time you will not have to wait another three weeks to read interesting stuff about YaST. As mentioned, we plan to publish a blog post about the reduction of the build time of YaST. And we will probably also publish a post about the visit of a YaST geecko to Euruko 2016.
So this time more than ever… stay tuned!
]]>The installer self-update feature integrates now with SUSE Customer Center (SCC) and Subscription Management Tool (SMT) servers. Until now, there were three different mechanisms to get the URL of the installer updates repository:
Now YaST2 is able to ask for the repository URL to SCC/SMT servers. The details of how the URL is determined are documented in the repository.
As you may know, it’s possible to install (open)SUSE in an automatic, even completely unattended, basis using AutoYaST. AutoYaST can be configured to display custom configuration dialogs to the user and wait for the reply a certain amount of time before automatically selecting the default options. Until now, the only way for the user to stop that countdown was to start editing some of the fields in the dialog.
We got a bug report because that functionality was not working exactly as expected in some cases so, in addition to fixing the problem, we decided to revamp the user interface a little bit to improve usability. Now there are more user interactions that are taken into account to stop the counter, specially we added a new “stop” button displaying the remaining seconds. You can see an example of the result below.
Following, as usual, the Boy Scout Rule we also took the opportunity to add automated tests to make that part of YaST more robust for the future.
Sometimes you want to extend the regular installation media by adding just a few extra packages or provide a number of fixed packages along with the media.
For this purpose, the installer can automatically register an add-on repository. All you have to do is to put the repository on the installation medium and to add a file /add_on_products.xml
that points to this repository.
The file looks like this:
<?xml version="1.0"?> <add_on_products xmlns="http://www.suse.com/1.0/yast2ns" xmlns:config="http://www.suse.com/1.0/configns"> <product_items config:type="list"> <product_item> <name>My Add-on</name> <url>relurl://myaddon?alias=MyAddon</url> <priority config:type="integer">70</priority> <ask_user config:type="boolean">false</ask_user> <selected config:type="boolean">true</selected> <check_name config:type="boolean">false</check_name> </product_item> </product_items> </add_on_products>
You can define the following elements:
<name>
is the name of your repository<url>
points the the repository location; you’ll likely want to use the relurl
scheme here that gives the location relative to the main installation repository<priority>
is the repository priority (lesser number means higher priority, the main installation repository has priority 99)<ask_user>
: should the user be asked about adding the repository?<selected>
: should the repository be automatically selected?<check_name>
: should the repository’s actual name be matched against the value of the <name>
element?You can of course list several repositories in this file.
If you’re too lazy to remember all this, mksusecd can do all this for you.
For example, if you have a set of new kernel packages you would like to use, do:
mksusecd --create new.iso --addon kernel-*.rpm --addon-name 'my kernel' sles12-sp2.iso
This creates a new iso based on sles12-sp2.iso
that will install your new kernel packages instead.
During bug squashing we managed to find some time to keep the storage stack reimplementation rolling… slow and steady. The customized Tumbleweed images (labeled as NewStorage) in the storage-ng OBS repository are already able to analyze most systems, creating a representation of the system storage devices in memory that will be used to manipulate the disks and propose a partitioning schema. Unfortunately, this representation is only visible in the YaST logs since fixing installer bugs was more urgent than representing that information in the UI.
This turned to be an important milestone, not because of the functionality itself or the value of the code (we just added a couple of lines of Ruby code), but because for the first time the dependencies in some packages were switched from libstorage to libstorage-ng. That had important implications for the code organization and for our continuous integration infrastructure, specially the Travis CI integration, which implies the generation of .deb packages. We can now say that our continuous delivery workflow (from Github to OBS, passing through Jenkins, Travis, Coveralls and Code Climate) is free of any trace of the old storage code.
In addition, we also did some good progress in LVM support in the new library, being able to recognize and manipulate in memory all kind of LVM structures.
An important part of our job, specially as a new release date approaches, is helping to shape the SUSE Linux Enterprise (SLE) documentation. One of the strongest points of SUSE products is the awesome SUSE’s documentation team which, as the rest of the company, have open source in their genes. Suggesting improvements and updates for the documentation is as straightforward as creating a pull request in the completely open documentation repository at Github… and anybody can do it!
The documentation team uses Docbook, but they would accept contributions in other formats (e.g. Markdown) and transform it themselves into Docbook… just because they are that cool.
The world is getting full of cool ARM64 devices and both SUSE and openSUSE are actively working in supporting as many of them as possible. We took another small step during this sprint improving the installer’s partitioning proposal for ARM systems that use EFI.
As we always say, this was just the small portion of the work done that we consider exciting enough to be part of our development reports, since we don’t want to bore you with details about every single fixed bug. During this installer bug-fixing phase, this is more true than ever and the next sprint, which is already planned, will be similar to this in that regard. Nevertheless, in the next report we expect to have some interesting news about the installer self-update functionality and about the LVM support in the new storage stack. Stay tuned.
]]>There are 3 options:
1. According to the wiki page, you can download the image they provide but there’s no kernel support for Mali400MP2 GPU (who knows if it’s fixed by now). No Mali mean no GUI. The link to image is http://download.opensuse.org/ports/armv7hl/tumbleweed/images/.
2. Download the image from http://www.lemaker.org. The GUI used is XFCE.
3. Do it the hard way, build it yourself. I would like to install MATE. I know, I could use the lemaker image.
I followed the page HowTo Build Banana Pi Image.
This post has 2 sections. The first is how to create the SD card and the next one is how to install MATE.
Create the SD card.
1. Create a folder where you’re going to work (download the nessesary files).
mkdir WORKSPACE
cd WORKSPACE
2. I’ll skip the steps 1-5 from the Build it yourself page. You can download the file:
Download also the rootfs openSUSE image file.
openSUSE-Tumbleweed-ARM-JeOS.armv7-rootfs.armv7l-Current.tbz
3. Create the folder with the ROOTFS_DIR
mkdir ROOTFS_DIR
4. Decompress the file to ROOTFS_DIR
openSUSE-Tumbleweed-ARM-JeOS.armv7-rootfs.armv7l-Current.tbz
5. Now work with the file BananaPi_hwpack.tar.xz. Decompress the file.
tar xvfJ BananaPi_hwpack.tar.xz
6. Copy related files to the directory ROOTFS_DIR
cp kernel/script.bin ROOTFS_DIR/boot
cp kernel/uImage ROOTFS_DIR/boot
Create the file:
nano ROOTFS_DIR/boot/uEnv.txt
with the following content
mmcboot=fatload mmc 0 0x43000000 script.bin || fatload mmc 0 0x43000000 evb.bin; \
fatload mmc 0 0x48000000 uImage; if fatload mmc 0 0x43100000 uInitrd; \
then bootm 0x48000000 0x43100000; else bootm 0x48000000; fi
uenvcmd=run mmcboot
bootargs=console=ttyS0,115200 console=tty0 \
disp.screen0_output_mode=EDID:1280x720p60 \
hdmi.audio=EDID:0 root=/dev/mmcblk0p1
Copy the rootfs folder:
cp -r rootfs/* ROOTFS_DIR
7. Now prepare the SD. Format the sdcard (assume the sdcard mounted at /dev/sdb. You can find it with the command cat /proc/partitions)
sudo umount /dev/sdb1
sudo dd if=/dev/zero of=/dev/sdb bs=1k count=1024
sudo dd if=bootloader/u-boot-sunxi-with-spl.bin of=/dev/sdb bs=1024 seek=8
Create partition (you can do it using gparted too)
sudo fdisk /dev/sdb
* Delete partitions: o
* List partitions: p
* Create new partitions: n
* Primary partitions: p
* Partition number: 1
* Press ENTER twice to use the total size of the card
* Write the partition table: w
Format the parititon
sudo mkfs.ext4 /dev/sdb1
8. Copy ROOTFS_DIR into sdcard
mkdir mnt
sudo mount /dev/sdb1 mnt
sudo cp -a ROOTFS_DIR/* mnt
sudo sync
sudo umount mnt
Now boot the card. The default username/password are:
Username: root
Password: linux
Unfortunately ssh didn’t work. I logged in and changed few things.
First of all I edited the file sshd_conf
And found:
Port 22
PasswordAthentication yes
PermitRootlogin yes
Then I used the command
Rebooted and all set.
You can download the image from openSUSE-Tumbleweed-BananaPi-headless-20150928.tar.xz
copy it at least 2GB sd card and resize it.
Install MATE Desktop
The first thing you have to do is to update (zypper up).
The easiest way is to open YaST and go to Software Management.
Then filter by Patterns.
Click to install MATE Desktop Environment and MATE Base system.
After everything is installed, make MATE-session as default window manager
Find the line:
DEFAULT_WM = “kde-plasma”
and change it to
DEFAULT_WM = “mate-session”
Then reboot. Login and type startx
]]>