Tolero’s tech notes

Some little sweets

Ubuntu 7.10 (gutsy) screenshots
	�
Deskbar applet	�	File browser image rotation

Startup and shutdown screens won’t blink any more, so no more watching of a boring black screen even for a second. Desktop theme received some minor updates: some pixels changed at the mouse cursors to make them more eye pleasant. Evolution is now displaying popup at the tray in case of a message arrival. In the file manager an image thumbnails will be automatically rotated if they were taken with a camera. Buttons of the ‘windows list’ applet in Ubuntu 7.10 will stay they original size despite a change of a window title, for example in case of web browser tab change. The very impressive Deskbar applet is initially activated. Microsoft ntfs file system support should be available for reading and writing by default (by ntfs-3g driver using FUSE). Additional set of utilities for manipulation with ntfs partitions such as format, resize, undelete and etc. is accessible via ‘ntfsprogs’ package.

Compiz Fusion - Desktop Effects

Compiz Fusion in Ubuntu screenshots
	�
Compiz config settings manager	�	Resize of maximized window

Yes, it is finally happened - the Compiz Fusion is now enabled by default on all supporting platforms in Ubuntu 7.10 release! That is the most awaited feature since the first announcement of the technology. Now it is labeled as stable, and is believed to work fine just out of the box, or after a proper drivers configuration, which in my case was a simple activation of a checkbox in the ‘Restricted Drivers Manager’ tool and reboot of the computer . There is a three pre-configured levels of special effects settings: ‘No effects’, ‘Normal effects’ and ‘Extra effects’. You can select one of them at the ‘Appearance’ dialog in the ‘Preferences’ group of the ‘System’ menu. To get additional interface with a much greater tweaking possibilities, you have to install a ‘compizconfig-settings-manager’ package, which is located in the ‘universe’ repository. This will bring ‘Advanced Desktop Effects Settings’ interface at the same place in the ‘System’ menu.

Screen, Graphics and Driver Preferences

Screenshots of Gutsy display setup utility
	�
Choose of graphics card driver	�	Screen and graphics preferences

The much complained necessity of a manual video configuration through the ‘xorg.cfg’ file is no longer a mandatory. Since the Ubuntu gutsy there is a way to set most necessary properties through a user friendly interface. These options include video card driver, screen resolution, frequency and type settings. And the best thing is that no more need to restart the X server (no reboot or even logout) to test and apply changes. Moreover, there is a two kind of configuration: one is for a whole system and the second is for individual user which requires no administrative priviledges.

In addition to graphical configuration, there is a ordinance of ‘bullet proof X‘. This means that user should always get to X session, even if there is some bad luck with a driver configuration. System initially will try to set X with a settings from the ‘xorg.cfg’ file. In case of a bad luck it will try to init a reduced VESA mode in 800×600 by 256 colors or VGA 640×480 by 16 colors for the next try within a solo X configuration application mode. Read more about BulletProofX in Ubuntu gutsy.

Desktop search

Tracker search in Ubuntu gutsy screenshots
	�
Tracker search tool interface	�	Tracker ‘Open File’ dialog integration

The Tracker is a new desktop search engine is now with Ubuntu gutsy. It is installed by default instead of the Beagle. The Tracker is an indexing search engine and is written in a plain C. That should decrease a level of dependencies and increase a speed of content indexing. There is a interface named ‘Tracker Search Tool’ which represents a navigation over the indexed content, displaying additional information about each indexed entry. ‘Open file’ dialog received a search field, which also performs a search over a Tracker index, and is available in a Gtk based applications like the gedit, the OpenOffice and the Gimp.

The old issue of an unexpected system failure to start a normal graphical session when disk space is full, has finally got a workaround. User will receive a warning message and a possibility to do some space cleanup at the startup.

Printing service changes

Screenshots of new printing subsystem in Ubuntu
	�
‘Pinter added’ popup	�	Printer configuration dialog

The previous versions of Ubuntu are know for bringing nice and user friendly printer setup interface, which is much more pleasant than the original Cups one in a plain html. The distribution development team made a major step forward in Ubuntu 7.10 with a detection of a hotplugged USB printers. The system will setup a newly plugged printer automatically, and it will be ready to accept jobs.

Another great stuff is a shipped virtual PDF printer. It makes possible to generate a PDF documents from all non Gnome applications like a Firefox, Gimp, Rhythmbox and other.

Free flash player

The Ubuntu 7.10 gets gnash - a free and open source version of the flash player. I’ve started using it since the early summer of 2007, and it showed a nice ability to play youtube videos and some banners. It would also be a choice for whom a closed source Adobe Flash Player made the browser to hang. Since testing it for a several months, it has never made my browser to halt even for a second. But disappointment came since middle of the September, when youtube made some updates and gnash lost his ability to play video. Waiting with a hope for a soon update of the player…

Firefox

Firefox plugin detection in Ubuntu screenshots
	�
Firefox plugin installer	�	Firefox playing Nelson Mandela speech on youtube

A third (currently alpha) version of the Firefox browser named ‘Gran Paradiso’ is included into the ‘universe’ repository. If installed, it will settle aside of the stable application, even using own settings directory at the home folder. Thanks to a numerous efforts from the developers, browser promises to be less heavy in memory than its predecessor version, and especially if you’re running it for a long time. Web services now can be used as a handlers for different file types. For example, in the stable version of a browser, web handler can be associated for only a one file type - RSS news subscription. This feature will provide a possibility for the sites to register handlers for a types other than RSS. In addition, the new type handling configuration system will be provided. Download manager will be reworked in terms of interface and bonus functionality such as interoperability with virus and malware scanners. Also will be added a possibility to resume an incomplete downloads even if the browser has been reopened. Identity manager will ask to save a password after the user will be sure of a successful login try. Many changes are coming to ‘Bookmarks’ handling. The some of them are: ability to sync stored links to a web service like the del.icio.us; protection against bookmarks lost; the easy way to export bookmarks; performance improvements. Browser will get an ability to track individual site settings, like a font zoom. Refer to the ‘Firefox 3 Requirements Document ‘ for an ongoing progress.

A stable version a browser also got improvement. And this time it is a distribution specific support for automated plugin installation through a package manager. For example, when web page will require a flash player, Firefox window with a suitable plugins list will contain a two options: the Gnash Flash Player and the Adobe Flash Player, both of them will be installed using a package manager (the last one will additionally perform a download from an Adobe site). That is first time the ability to install a flash player came by a proper way in a comfortable manner.

Gimp 2.4

Gimp 2.4 - Background Select Tool

So long awaited version update of the Gimp is now available at the Ubuntu gutsy. The icons set has been totally changed, to became better looking with a different Gnome themes and even with a Kde and Xfce desktop environments. All types of brushes are now scalable, including bitmapped and tubed kinds. Completely rewritten selection tool brings such stuff as rounded rectangular selector, resizing of existing selections, and a simplified using of curved selection. The new ‘foreground select tool’ makes it possible to identify object edges by mapping it surrounding background area, which leads you to the perfectly selected object. Some improvements been done for a better handling of alternative file formats, such as Adobe Photoshop PSD, and Vista icons. More information can be found at the official Gimp 2.4 release notes page.

OpenOffice 2.3

OpenOffice Calc - new diagram

Almost half a year passed since 2.2 release of the OpenOffice suit. The new 2.3 version is now out and shipped with the Ubuntu gutsy. The most significant update touched a diagrams module: from the new Diagram Master to a flexible selection of the axis parameters and the data sources. Writer got an ability to export into a MediaWiki format. Calc menus and dialogs were touched for a better fill and the support of external file formats received an enhancement. Much speed improvement of the Impress, presentation building instrument.

The kernel and the basics

Ubuntu gutsy steps forward by two versions of the Linux kernel, from 2.6.20 on to 2.6.22. One of the significant improvements is the ‘tickless’ idle loop which unifies the timer handling and makes the kernel more power efficient. That is a measurable for a notebook owners because it can reduce a power consumption up to 25%. Improved ‘Virtual Machine Interface’ makes paravirtualization better (in future) when running under supporting it hypervisors like Xen and Vmware. A completely new ‘Wireless Stack’ and ‘FireWire stack’ are now in the kernel, bringing new compatibilities and drivers, and losing some old because of the portability issue. Check the ‘Linux Kernel Newbies’ articles for additional information about 2.6.21 and 2.6.22 versions.

In addition to “vanilla” updates, there is a collaboration between the Canonical and the Intel. The goals are to made a distribution much more efficient in terms of a processor usage. Intel’s PowerTop utility is used to track a glitches in a software and to help to made patches to reduce a power consumption. As the Intel’s leading software engineer Arjan Van de ven said, they already made a numerous amount of patches and expect a half of them to be in use at the upcoming Ubuntu gutsy. The test run of such patches on the ThinkPad T61 showed a reduce of a power hungry from 23.3 watts on the ’standard install’ to 15.6 on optimized, what is as much as 33%. However it is not clear, what is the ’standard install’ exactly and how resulting 33% efficiency intersects with kernel ‘tickless’ improvement. Follow to the APC comments on Intel’s San Francisco Developer Forums for further information.

Conclusion

Distribution update confuse

As always, new version of Ubuntu is better than previous. But probably this version did the most powerful step forward than any other. I will strongly recommend my friend, who is using another operating system, to settle on the Ubuntu gutsy now, because most of the major barriers for newbies are gone. During testing run of the gutsy I have met only one confusing case. It was during installation of updates, when Update manager opened a terminal emulator frame. It was unclear even to me, what should I do next: just wait, because it was just an information message or type something in. In real, I have simply to hit the ‘Enter’ button.

The next diagram contains 19 currency rates graphs starting at January 04 of 2001 and till now days. The base currency to which rates are shown is a Russian Rouble. Despite the existence of several European currencys till Dec 31 of 2001, they are not shown due to they actual worthless.

Diagram is plotted and updated automatically every day and may contain a glitches.

If you’re placing this diagram at your site, you should keep the signature placed at the bottom right corner of the image, and you have to place a link to this article with a title relative content.

I have recently wrote a post about forum and blog spam protection, which covered a several tricks on how to satisfy from bots attacks. And now, after a months of my self testing, I’m presenting a spam filtering plugin for a wordpress.

Feel free to discuss or to ask any questions here about the filter.

Open source processor emulator qemu of the 0.9′th version is out, and the new version of kqemu acceleration module is released under GPL. That is the first reason to install or to upgrade them. The second reason is current absence qemu 0.9 and kqemu packages for ubuntu edgy and dapper. And the final reason for a someone can be the kernel panic issue in the guest os runnig in qemu installed from the official ubuntu repository (happend for me only with kqemu).

You can install a packages of qemu 0.9 and kqemu for ubuntu dapper and edgy from my repository. The qemu in my repository is a packaged official binary build of qemu. And the kqemu packages are from from debian experimental repository. In addition qemu package will automaticaly initialise a recommended for qemu system parameters and will insert a kqemu modle if such is installed, after the system bootup (check /etc/init.d/qemu file), even if last is installed not from a package.

instalation

At first add my repository to /etc/apt/sources.list for edgy:

deb http://ubuntu.tolero.org/ edgy main
deb-src http://ubuntu.tolero.org/ edgy main

Or that lines if you’re running dapper (simply mepis 6.0):

deb http://ubuntu.tolero.org/ dapper main
deb-src http://ubuntu.tolero.org/ dapper main

The installation is better to do from the console package manager, rather then from the graphical one. Ensure that you have also ubuntu universe repository included to the sources.list. If so, cross your fingers and pass one by one to the console the next commands:

1. sudo aptitude update
2. sudo aptitude install kqemu-common kqemu-source
3. sudo aptitude install module-assistant
4. sudo m-a prepare
5. sudo m-a build kqemu
6. sudo m-a install kqemu
7. sudo aptitude install qemu
   

The commands from 4 to 6 will install a kernel headers, compiller and other assistant packages, will build a kqemu-modules package for your kernel version package and install it. If all ok - the all is done.

You can protect your linux hosted ssh server from the password brute force attack with pam-abl. This plugin to the ssh pam authentification module measures the amount of login tries by an specific IP address or exact login name. If tries count exceeds the allowed limit, pam-abl will block IP address or login name.

Manual installation of pam-abl is not much complicated, however requires some time to be spended with a few manipulations in console. I have builded a deb package of libpam-abl for ubuntu edgy and ubuntu dapper (mepis 6.0). It fully automates the process of installation and proper configuration process. All you have to do, to protect your ssh from brute force attack, is just to install the package. It is necessary to say, that libpam-abl is shipped with fedora linux since fedora core 4 release by default, but only debian based distributions lack that useful security package.

Important note for ubuntu (mepis) linux users: the openssh-server package bounded with distros contain a bug №405041 which makes the libpam-abl to work unproperly. My repository contains a patched version of openssh-server package, which you have to update to.

For a curious linuxers there are source packages available for both, openssh-server and libpam-abl.

how to install

At first of all add my repository to your /etc/apt/source.list configuration file.

For ubuntu 6.06 dapper (SimplyMEPIS 6.0):

deb http://ubuntu.tolero.org/ dapper main

For ubuntu 6.10 edgy:

deb http://ubuntu.tolero.org/ edgy main

And execute the next four commands:

sudo aptitude update
sudo aptitude upgrade
sudo aptitude install libpam-abl
sudo /etc/init.d/ssh restart

First command is to fetch the repository listing. Second is to upgrade the openssh-server package to patched version (openssh-client will be also updated). Third is to set up a protection plugin. And the fourth is to restart a server with a new security plugin.

That is all! Now your’s machine ssh shold be protected with pam-abl.

How to check that all is working

To check that pam-abl is installed properly just try to login onto your ssh server. Simply execute the

ssh localhost

command, and input a wrong passwords for all tryes. That failed attempt will be listed in a pam-abl statistic. You can see it by the command

sudo pam_abl

And you should see there your failed attempt. If you see only <none> indicatiors, this signs that you are running not patched version of openssh-server package, and you have to install it from my repository, and restart ssh daemon.

The pam_abl utility is a control instrument over libpam-abl plugin statistic. You can unblock any host or account only by it. Refer to it’s --help for details.

What does my libpam-abl.deb package do

Many of readers do not have a need to read this chapter. It is mostly for a curious guys.

Excepth the simple extraction of a pam-abl files in to their followed places, my package performs a configuration job.

As the first, it creates a default configuration file for a pam-abl plugin. You can find it as /etc/security/pam_abl.conf. The default configuration is aimed to block any IP address or login name (except the root account) for three failed attempts in hour, or thirty attempts in a day. If you wish, you can easily change the limits right after installation.

The second point is altering the /etc/pam.d/ssh file, to include a pam-abl.so library to participate the authentification process. Adding required line into the right place of configuration file on installation, and removing it on deinstallation of the package is fully automated.

Collected information about failed login attempts is stored under the /var/lib/abl/ path. This directory is created during installation, and removed only when package is deinstalled with the --purge flag. The same is with pam_abl.conf file. That is the common behavior for all in debian packages.

Preamble

Forum and blog spam is an absolute pest for a last years. I see a lot of their’s holders complaints, and I also see a lot of polluted content. Spammers do their dirty job by the two ways: automaticaly by the bots and manualy writing to the forums. The basic threat is made by an automated spam, and I’ll explain protection from it there.

There is a lot of standart techiques to protect from the spam. We all know them: captha, email confirmation, user interative and so on. All of them are less or more require odd user attention. And because they are visible for a spammers, they are not invulnerable. Captcha is breakable with a cheap porno traffic, email confirmation is easily automated with a scripts and free mail services, user interactive sometimes confuse users.

But there is another good method, not disturbing users, and not so obvious to bypas - behavior tracking. This is a good way to protect yor blog or forum from spam, because this protection is hidden, and spammer don’t know where exactly you are checking him.

Bot’s behavior

The main goal of spammers is to spam as much as possible. As the result - the lack of the real user behavior. Spammers are very prudent to the traffic, and do not perform a many usual steps. For example, the user browser after loading the page always loads at least a CSS file from the site, because it is required to display the page properly. But the CSS is not required to be received by the spammers. The same is with the images from the page - they also are not downloaded by the bot (but sometimes they are not downloaded by a real visitors, for example if user is on GPRS and tryes to reduce the traffic).

Most smammers recognize the site only once, and perform a spam after a some time (usualy next day). The “one recognition” is also a speedup from their side. This means that they will do not download the page containing a form every time before to do the message post. I also think that they are using a two types of software: one for a crowling across the web in search for a forum or a blog, and another for the messages posting. This can explain why do they mostly post the spam on the next day after their spider reaches a victim site.

Antispam protection

There is a several simple recommendations from me, how to track a spammer:

1. Do the check for a previously downloaded CSS or image by the poster’s IP address. The best is to check for the image receiving: site page always contain several images, and it is not easy to understand for a spammer which of them is checked by you (or maybe all of them are checked). You can display the pictures manualy by the scripts. With the thing like mod rewrite this will not be obvious for a spammer. Or you may check the access.log file of your site. If you’re hosted on the unix compatible machine, this can be easily and effectively done with the bundled tail and grep utilitys together.
2. Add the additional hidden parameter to your html form. This parameter should be unique for each minute, and it should be easy to understand for you (but not for a spammer) to which minute it does belong. Then you can drop all posts which are submitted with a ancient parameter value.
3. The spammers do not implement JavaScript enjine into their tools. It is possible to mask by composing a message html form with a JavaScript. But be careful, generate you form with the several steps. Do not simply write it by one fucntion call. If spammer is using a regular expressions instead of html parser, it is possible that he will catch a footprint of a html form, and your trick will not help.
4. Always check a Referer field in HTTP headers. Some spammers do not pass a valid Referer. Typical user browser always submit message with a valid Referer in HTTP headers to the page containing html form.
5. Check a HTTP UserAgent. There is a little amount of dummy (but very annoying) spammers, wich do not supply a relevant UserAgent. And there is another group of too much “intelligent” spammers which use a browsers UserAgents, but they are changing them on each access. I suppose they always pick them randomly from an array in a script. So, if the html form was transmitted from your site to the one UserAgent, but submitted with another, also discard a message.

Conclusion

All this tricks were used by me on a several projects, and showed a good result. For sure, if the spammer is interested in spamming exactly your resource, and he is creating a specialized software, and having a good experience, he can finally pass throug all your barricades. But he well need a several hours of bashing his head over the keyboard, tearing a hairs from all places over his body, and shouting “how? why? where?”, and maybe he will leave you away.

If you found this article useful. If you’re using your own tips and tricks to protect a resources. Feel free to submit a commet. And do not mind to share a link to this article with your friends - let’s resist together!

Due to lack of the native Ubuntu edgy and dapper packages of xfce 4.4.0, I manually rebuilded them from Ubuntu feisty reposiory. I also rebuilded (only for edgy) from feisty xubuntu-desktop package and relatives, cos it handles additional dependencys, not mentioned in current xubuntu in edgy. Thus these packages works good for me, I can not give any guaranties, and I’am not responsible for any possible harm. If you still interested in xfce upgrade, you can use my repository. Just add the following to /etc/apt/sources.list for edgy:

deb http://ubuntu.tolero.org/ edgy xfce-4-4-0
deb-src http://ubuntu.tolero.org/ edgy xfce-4-4-0

Or that lines if you’re running dapper (simply mepis 6.0):

deb http://ubuntu.tolero.org/ dapper xfce-4-4-0
deb-src http://ubuntu.tolero.org/ dapper xfce-4-4-0

And then at your favorite package manager do update and upgrade if you are already using xfce4, or update and install xubuntu-desktop if you’re setting it at first. I truely hope that everything will pass well, as in my case, but if shit happens, I’am always open for help.

I’am also recommend to install a thunar-volman package, plugin which automaticaly mounts a removable media devices, which makes a use of xubuntu desktop powered by xfce4 more comfortable.

PS: If you found this info useful don’t mind to place link to this post from your resource (blog, forum, e t. c.) with title like “xfce 4.4.0 for ubuntu edgy and dapper packages”. Let’s help others to find a solution.

Update 02-feb-2007: repository moved from the Russian servet at http://tech.tolero.org/ubuntu/ to the US server at http://ubuntu.tolero.org/. Please update your /etc/apt/sources.list if you are already subscribed.

I have just read a BBC “users OS review”. BBC have selected two users to explain theirs opinion about Windows Vista, and by one user for Linux and OS X.

After reading that article, my final thought is that vista bloated too much. That was mentioned by both writers. No one of Linux distros requires a minimum of 512 megabytes of memory just to start an empty OS. The exact vista fan’s phrase was:

Whilst 512Mb is quoted as the minimum I would say that 1Gb is needed for a wait-free session.

How is it, to have 1Gb just “for a wait-free session”? Microsoft also did, in my opinion, a funny solution for those who have a not enough (I suppose less than 1Gb) memory. As the other writer sayed:

When you’re having performance issues due to insufficient memory, you can use a USB flash drive as an additional cache of memory to boost performance.

No one of users mentioned notning interesting besides the new Windows 3D Aero interface. To be clear, both articles are telling to buy more memory and new video card, just for a new Microsoft fetish interface. The completely different articles about the Linux and the Apple OS. The basic ideas from the article about Linux, as always, is about freedom:

1. … it also gives more experienced users radical powers over their computer which are not available in other operating systems.
2. Also there is a huge variety of high quality, free software available for Linux…
3. Also it encourages programmers to be better people, working in the open…
   

I’am absolutely agree with this opinion. The same as the author of the first Vista article, I did all steeps following the Misrosift. I was faithful to it, and went through all popular OS versions: 3.11, 95, 95 osr2, 98, 2000, XP. But the real comfort came to me in 2004 only with Linux.

What is for the Mac OS X? It is a truely polished OS. It is much more stable than any Windows, and it is straight forward developed than GNU/Linux distributins with theirs tons of variants how to do the same task. But I think, it is not fearful Microsoft competitor, because it supports mostly own expensive Apple computers, and no possibility to set it up on any other cheap variant.

Why do I personally hate proprietary drivers? I have a well featured laptop Asus W3V which is equiped with ATI mobility radeon X600 video card, and Intel PRO/Wireless 2200BG network card. Both require a closed source proprietary drivers to work (fglrx, ipw2200). Both brings a headache in use. The whole pain for both is bugs.

My Asus supports ACPI as well, but I’am unable to use it freely with the proprietary drivers. The problem is with suspend function. It doesn’t work perfectly: neither ram nor hdd. The suspend to ram and suspend to disk goes well only in one way - to sleep. The way back is not well. I’am using ubuntu edgy (previously used dapper) which comes with pre-builded, easy installable, proprietary drivers packages. To be fair, I also tryed patched suspend 2 kernel, which gaved the same picture.

Resume from suspend to ram leaves the screen black, and the computer doesn’t respond to any actions (even Ctrl+Alt+F1 or Ctrl+Alt+Backspace). If the machine was suspended in a text mode, it will resume, but switch to X server will halt it. This problem came with the fglrx drivers since spring of 2006. Everything works well on the debian sarge with the old drivers. But the old drivers are not possible to setup onto the new ubuntu kernel, it is unable to compile the kernel module.

The resume from hard disk sleep causes ipw drivers failure, and the wireless network doesn’t work until reboot.

Update 29-jan-2007: there is a solution to restore a suspend to ram! It is working with suspend 2 patches and hibernate-ram script. Just tweak a /etc/hibernate/ram.conf file, and comment all lines containing a vbetool occurence. Also uncomment, or add RadeonTool yes to the end of the script. Mine script now look like this:

TryMethod ususpend-ram.conf
TryMethod sysfs-ram.conf
#EnableVbetool yes
#VbetoolPost yes
RadeonTool yes

Yesterday I finally decided to start blogging. The final grain of sense were dropped while my optimization of the Ubuntu Edgy boot up speed by switching off some unnecessary scripts from /etc/init.d/. I were inspecting each script one by one for theirs behavior. And I discovered that the script /etc/init.d/clamav-freshclam contains a brutal text (some symbols replaced with *):

# This only does something if I’ve f**ked up before
# Not entirely impossible

Really interesting for production distro. Later a did a global grep on a whole system for inclusion of the word f**k. And wow! The reasult is 966 inclusions. Detailed inspection showed that around two/third of them are in dictionarys (/etc/dictionaries-common/words, /usr/lib/firefox/dictionaries/, /usr/lib/openoffice/share/dict/), but the rest one/third is on conscience of the distro scripts (less polluted), development headers and the kernel headers (most polluted). I don’t want to know what results can be if to grep all disrto software sources and kernel sources. But I think it can be interesting “homework” for someone to grep full sources for keywords like: shit, damn, hack, how it works, and so on.