openSUSE Lizards » Josef Reidinger

Ruby on rails, Ajax and memory watching

Josef Reidinger — Thu, 14 May 2009 14:44:09 +0000

As work on webinterface for YaST is in progress we must learn new technologies suitable for web development. WebYast will be written in ruby and ruby on rails framework. Also because WebYast is new interface it could contain AJAX features for better user comfort. Today I found that ajax support in RoR is on good level and with documentation it takes few minutes to create first example which show current used memory on server. It is not connected anyhow to YaST because I want to focus on AJAX.
And here is a code. It update page every fifth second (but not refresh only update div, on bigger page it is really significant):
/views/home/index.html.erb: <%= javascript_include_tag :defaults %>

`Hello world!`



<%= periodically_call_remote(:url => { :action => 'get_averages' }, :update => 'avg',:frequency => '5') %>



Memory usage is X MB

controllers/home_controller.rb:

class HomeController < ApplicationController

    def index

    end

    def get_averages

        output = `free -m` # bash solution - | sed 's/Mem:[^0-9]+[0-9]+[^0-9]+([0-9]+).*$/1/;2q;1d'`

        output = $1 if (output =~ /.*n(.*)n.*n.*n/) #let live second line

        output = $1 if (output =~ /.*:s+S+s+(S+)s+/) #second field

        render :text  => "Usage mem is "+output+"MB RAM."

    end

end

And thats all to watch your server usage.

Edit system on iso image

Josef Reidinger — Thu, 02 Apr 2009 12:37:55 +0000

I start maintaining yast2-repair and first bug which I start solving is that repair from DVD menu doesn’t work same as from installation menu. Find where is problem and also test if fix is correct is not trivial. I describe below how to modify inst-sys on DVD or software on LiveCD.

I found useful information on our wiki – Creating modified installation system. But I don’t want install from network, as it doesn’t shown same menu as on DVD. I try use mkisofs, but it is not easy set same boot sector as have original iso image. And there I found good software – isomaster. This allows you to replace file on iso image and remember from original iso image where it has its boot sector. This iso should be easily tested in e.g. Virtual Box and I can verify, that my fix work before we release first DVD with opensuse 11.2. Just sidenote – linuxrc show warning, that your inst-sys doesn’t match checksum.

Same way is possible edit whole LiveCD. Simple mount and copy content of live system to your disc and with zypper –root you can change software in your LiveCD. Then create squashfs ( I use mksquashfs live_root openSUSE-kde-11.1-read-only.i686-2.7.0 -no-duplicates -noappend ) and with isomaster replace file openSUSE-* and you have your modified liveCD.

Zypper: Improved bash completion and practical usage

Josef Reidinger — Thu, 12 Feb 2009 09:28:43 +0000

Because no one reported any bugs to perl-Bootloader, I have some free time. I use it to improve bash completion for zypper, because I find current one have some really annoying things. In short now completion work also for zypper global options, hint also for short version of commands and help name of repo, service and lock, if you use command which takes that as argument. Next few paragraphs contain practical examples how it help improve your command line productivity, how to get it to your system and also some notes how I implement it.
At first how you can try it. It is really easy, in 11.2 it will be part of package, 11.1 contain only old bash-completion (created by Marek Stopka), so you must download this file and save it as /etc/bash_completion.d/zypper.sh. After bash restart (simple type ‘bash’ to get new shell) you can enjoy new bash completion.
First change is that automatic hints works for all commands – long or short version and also on different zypper version (it is dynamic generated), so if you are not what is exactly name of option just use tab.
zofie:/home/jreidinger # zypper in -- --auto-agree-with-licenses --from --capability --name --debug-solver --no-force-resolution --dry-run --no-recommends --force --repo --force-resolution --type
Improvement is also that it work also after first option (before it hint only first option after command. Now it also help you with global zypper option, so if you don’t yet write command you can type — and get all global options.
zofie:/home/jreidinger # zypper -- --cache-dir --plus-repo --disable-repositories --quiet --disable-system-resolvables --raw-cache-dir --help --reposd-dir --no-abbrev --root --no-cd --rug-compatible --no-gpg-checks --table-style --non-interactive --verbose --no-refresh --version --no-remote --xmlout
What I find quite useful is that now you get hint also for repository, service or lock. I have quite long repository names and it is really annoying write it. So I usually use zypper lr to get number and then use that number instead repository name, as it is less typing. Now you can get help (problem is with names with spaces, I reported it):
zofie:/home/jreidinger # zypper ref repo- repo-debug repo-non-oss repo-oss repo-source repo-update
So I hope you enjoy this improvements and I welcome any idea how improve it in comments.
Last paragraph is some notes about implementation. At first for compare you can look at original bash complete from marek stopka here. First what you can see, is that I remove static generating options help and create it dynamic. This really improve maintenance of this script. Only part which is static is special command handling like commands with repository, service or lock arguments. Second change is that now commands is recognized also if user use short option (e.g. when user type zypper mr and want hint for repository). This is quite easy change, as I only change awk output and remove comma between long and short version of command. Next change is more backward study of user line to find what command is typed. As you can see in original is study only previous argument. This causes problem when user want more options or use option and want help for argument. After my rewrite script is shorted by 20% and has better functionality and maintainability.

What changed in perl-Bootloader between 11.0 and 11.1

Josef Reidinger — Thu, 18 Dec 2008 12:04:25 +0000

I start with history. After 11.0 I became maintainer of perl-Bootloader (I never before write anything in perl, but know some other scripting language, so it is not so hard learn another one) after Alexander. Problem is that alexander doesn’t have enough time for maintainer it (he is also leader of arch team). This mean I get many unresolved bugs (around 150), because lack of resource prevents fixing it. Also I get some features to implement and some enhancement I found enough useful (some idea start in bug reports or on factory mailing list, so thanks community) to implement it. I describe what succeed and what not in rest of this blog entry.
Features can be divide to official which come through fate process ( you can read more about this process in another blog) and enhancement, which sounds reasonable for me.

Official features

First I look on official features. I think most important is automatic test suite. This is set of automatic test, which test again interface of library (it is black box testing). Before release of 11.1 this test suite contain 232 tests. I hope this increase quality of each perl-Bootloader release, because this suite catch many problems and also if new occur I add to it (this should prevent regression). Only problem with this test suite is that it doesn’t test whole kernel upgrade. It is hard to due, because kernel upgrade must fill itself hardware informations and if I want test it correctly I must simulate it with actual utilities which is used and on many types of hardware. So this part should be improved, but need some idea how test many hardware configuration (like different RAIDs (linux and bios), LVMs, multipath, different hardware architecture (like macos, efika or chrp on powerpc)) on one machine.
Another feature is consistent device names. This is most problematic feature, because it is hard to resolve many different devices given by udev (also udev is broken during some part of development). I experiment with many different sollution and at final I decide (it is after last beta) to create function which translate everything to kernel device and then compare that device to another translated device (previous solution based on filling all symlinks also work, but after device mapper problem with udev(more lower) I change it to more efficient and more reliable solution).
Next feature you can use if you have machine with chip to trusted computing (some notebooks have it). More about trusted computing you can read here. Most work did thorsten duwe (maintainer of grub) and in final I only ensure that due to security reason no splash screen is loaded (I remove message line even if user want it). If you have pc with GPT table and x86_64 processor you must for 11.0 use legacy booting. Now you can use ELILO bootloader (which have support efi) also on x86 architecture. This is also quite easy to implement, but harder to ensure it works, because get hardware with that configuration is not so easy. Only short notes for another features. Old disks which uses C/H/S is dropped and LBA is forced, kernel append during upgrade is taken from sysconfig and add support for disk remapping for windows entries.

Enhancement

Difference between feature and bug is really small and maybe some of enhancement looks more like bug and vice versa. For me interesting enhancement is check, if /boot is mounted. I use it at home quite often, because if you have separate home you needn’t mount it, but if during update also kernel is updated I often forgot mount it. First implementation is not ideal and I get many experience, that some users have really exotic entries in fstab, which of course bad match to check pattern. But after three iteration and one work-around this work quite good and I am satisfied with it. Another enhancement (really close to bug) is none bootloader. It is special bootloader settings, which ensure that nothing happen during kernel upgrade. It is quite useful if kernel update mess your configuration (which is bug, but you want prevent it until I fixed it) or take to much time (this is also bug). More important usage is for netboot, when you needn’t any bootloader, because you boot via PXE.
Important enhanced is stop kernel install, if bootloader update fail (problem is in using tee in pipe and ignore script return value). It is good, because you know that something goes wrong and reload backup. Also because usually new kernel is installed first, you still have old kernel and can boot it. Last enhancement is for anyone who want look how perl-bootloader works inside. Now you can use make doc in sources and it generates html page from comments (something like javadoc). Beside this documentation together with Jozo Uhliarik we create wiki page about interface of library and anyone who want use perl-Bootloader should use it.

Bugs

Solving bugs in perl-Bootloader is not so interesting as it looks like . You need analyze many long logs (if perl-Bootloader uses yast, then it is saved to y2log, if used by kernel upgrade then save to perl-BL-standalone log) and find what and when going wrong. You find what is wrong, but not who break it. Sometime it is bug in parsing configuration, sometime in parsing output of external commands and sometime is bug outside (like break udev). Source code (also with test suite) have 15k LOC and when I try look how many line I change (oneliner for it `find . -type f 2>/dev/null | grep -v .svn | xargs svn blame | grep jreidinger | wc -l `) I find, that I change 3k LOC. So code base is good, but need some improvement. I note some interesting bugs, which I solve.
One is longterm problem with chainload, when as root is current mounted root, but chainloader key is with right prefix (root of chainloaded partition). This on some hardware causes problems and now root is correctly set and I don’t have any negative responses (so I hope it works).
Another one is also quite long term problem when you change flavor of kernel, sometime previous kernel is uninstalled and after that new installed (normal work-flow is install new and uninstall old). Of course it switch default, which is quite annoying. Fix this is quite tricky, and trick is that after remove last image section and if it is default, I add comment about that and when new kernel is added, then set it as default. That fix works quite good, but after sent iso to factory (for boxes) I get bugs, that if you update via YaST, it doesn’t work. Problem is in some deprecated code which overwrite my comment by another one. Fix is easy, I only remove deprecated code, which cause it. So if you after install of opensuse11.1 see update of perl-Bootloader you know why. As late update is also solved problem with hanging kernel update. It take too much time because when I read logs I add useful logging lines to code (I hope it help me next time find problem faster) and when log records reach some level, it take too much time (due to array copying of records). I reduce debug logging and also improve whole performance of logging, so I hope it significant improve time to kernel update (at least it should not take minutes to upgrade bootloader configuration).
Not every bug is problem of perl-Bootloader, but you must solve it in code. Example is device mapper and udev inconsistency. Udev points to /dev/dm0, but device mapper to /dev/mapper/something. This became real problem when persistant names feature is implemented. Solution is also little tricky, udev define variables for links and one of that variable is DM_NAME for name of device and DM_PART for partition number and with this information I can construct whole /dev/mapper/name. I find this solution as working, but not much robust. If someone know better, (s-)he is welcome to write it to comment. Also many perl warnings is fixed, it usually doesn’t break anything, but it looks really unprofessional.
Now I have only 2 unresolved bugs (both is enhancement on which I work) and wait for another, so report all problems in opensuse11.1, I don’t throw it to bin

New software in build service

Josef Reidinger — Tue, 18 Nov 2008 18:33:33 +0000

I try testing xfce4 desktop and find some bugs. But I also find that some really interesting missing in opensuse build service, so I add two new applications to build service (gnome community repository…but now gaupol is available only in my personal repository, because in gnome pygtk wait for python 1.6) – osmo and gaupol.

I start with gaupol which I often use to correct subtitle timing. Yes it is subtitle editor, which support translating, previewing or converting between subtitle format. more info you can get on project website – http://home.gna.org/gaupol/. This package depends on python bindings of enchant library, so I also package that wrapper. And here is screenshot how it run on opensuse beta4 with xfce desktop:

Second is osmo. This is personal organizer for your appointments or tasks. It is nice described in article on linux.com . I only add to it that it is good integrated to xfce enviroment (see right up corner on screenshot) and I personally prefer it against orage which is xfce project. It depends only on library for synchronizing syncml and library for support icalendar Libical. And screenshot for osmo:

This is my first work with build service and It take me two hour to package this applications. So don’t wait and also prepare your own package for software which you miss in opensuse. But at first try search , if someone before you doesn’t make it and share his(her) work with you.

What to do if every kernel update break your bootloader settings

Josef Reidinger — Thu, 06 Nov 2008 08:57:00 +0000

Perl-Bootloader response for kernel post-install bootloader update script. Current target is ensure, that you have in your bootloader actually kernel and also that entries for old kernel is removed. This is problematic for some complex or manually enhanced configuration. In this case perl-Bootloader should somehow break your settings (this mean you still can boot, but your enhancement or extra sections can dismiss). This should change in future as noticed in bugzilla .

If you want maintain your configuration manually, you can simple set your bootloader to none. There is two ways how you can do it. First is set LOADER_TYPE=”none” in /etc/sysconfig/bootloader. Second is set this in yast2 bootloader. Another advantage is that this take almost no-time, so if obtain hardware configuration for update take to much time, this is workaround.

Warning, if you set bootloader type to none you must manually edit your bootloader configuration after kernel update. What can help you, is set image and initrd to symlinks which lead to actual kernel.

How survive zypper dup on system with bad internet connection

Josef Reidinger — Thu, 30 Oct 2008 08:40:37 +0000

Maybe someday you try zypper dup to actualize your distribution and in middle of process it fail, because you are disconnected or some packages is actualized before you download it (especially on factory this can happen). It is more safety download packages at first and then install from this local files.

How todo this is little tricky, at first you must enable caching downloaded files (I do it only for remote connection):

zypper mr –keep-packages –remote

So now you cache all downloaded files and now try testing run of dup. Trick is that all packages download for that test is cached.

zypper dup –dry-run

Now if you have slow connection I reccomend also disable autorefresh for all repositories, because if repository is refreshed before dup, you can easily find that some packages is newer than package in cache and you must download it.

zypper mr –all –no-refresh

Now is everything prepared for zypper dup, which use files from cache. Cache can take quite lot of disk space, so after dup you can clean it.

zypper clean

And thats all. This features work from OpenSuse 11 and you can also use this trick for zypper update or zypper install.