Life Pattern

Attack vectors deja vu

zImage — Mon, 11 May 2009 07:01:37 +0000

I have to keep an eye on the IT security news. You know, “security is a process not product”. Just recently, Linux kernel vulnerability CVE-2009-1337 caught my attention. This even has l33t in its name The more interesting part is, of course, not the CVE number but the attack vector used in a recent exploit. Basically, a core is dumped to the logrotate.d directory. After this, logrotate executes the malicious code included in this dump since it uses rather naive parsing to find instructions in its configuration files.

Inevitably, this reminded me of a very similar situation from few years ago. In 2006, CVE-2006-2451, which is another kernel vulnerability, allowed core to be dumped in a directory that the attacker isn’t allowed to write to. A weakness in cron.d parsing similar to that in recent versions of logrotate was used as attack vector.

Just a few weeks ago, I had another deja vu. There’s a flaw in udev versions before 1.4.1 that allows local users to gain root privileges by not checking whether a NETLINK message originates from kernel (CVE-2009-1185). It took me some time to remember why this sounded so familiar since the older case is from 2003. Back then, the zebra routing suite failed to check the NETLINK message originators (CVE-2003-0858).

Oh well, to err is human, don’t you think?

Linux kernel vmsplice root exploit

IO performance monitoring

zImage — Sat, 11 Apr 2009 12:08:21 +0000

If you can measure it, you can manage it. I’m a measurement, monitoring, analysis and statistics addict

That’s why I’ve always wanted to be able to monitor the IO load of the Linux systems I’ve worked with. While there are well established monitoring and accounting tools for the CPU usage - both system wide and per process - there were virtually none for the IO system until very recently.

Two of the more important reasons why I’d like to see better IO load monitoring are:

The mechanical drives have big latency. In general the CPU feels much better than the disks when overloaded. For example if load average 10 is caused by CPU bound processes the system feels much more responsive than the same load but caused by IO bond processes. CPU load average 10 on a server system with two processors isn’t very noticeable. At the same time IO load average of 10 on the same system with 2x 7200 rpm disk drives in RAID1 feels very sluggish.
The hard disk drives failed to keep up with the performance improvements in microprocessor technologies. Disk capacity has grown quite well, but the speed and especially access times are far behind. The IO performance is the most common bottleneck and most precious resource in today’s systems. Or at least the systems I work with

At the beginning of my Linux career, ten years ago, there was only one metric - blocks read/written. And that’s it. How busy the disk is you can guess only by looking at load average and checking how many processes are stuck in D state. I wish there are separate load average readings for CPU and IO…

At some point (linux 2.5 times?) extended statistics were added and things like queue size, utilization in % etc. became available. Much better. Still it was hard to tell who exactly is causing the load. If we speak of multi user system all you can see is multiple processes in D state. It’s unclear whether these are the ones causing the IO havoc or just victims of the already overloaded IO subsystem waiting.

In Linux 2.6.20 another step was made by adding per process IO accounting. I was very excited when I heard about this feature and eager to try it. It turned out that this per process IO accounting counts only the bytes read/written by a process. Not that better. A modern 7200 rpm SATA drive is only capable of about 90 IOPS so it could be choked with the pathetic 90 bytes per second…

Then there are the atop patches. These add per process IO occupation percentage. That sounds great but… when you have a lot of small random writes they go to the page cache first and only then are periodically flushed to the physical device. This is performance feature and is generally a (very) good thing as it allows the elevators to group writes together etc. Unfortunately, atop ends up accounting all these writes and IO utilization to pdflush and kjournald.

Ok, lets see what’s the state of the affairs in some other operating system. Everybody talks about dtrace so it’s time to check it out. Linux doesn’t have dtrace. At least yet. There is work in progress by Paul Fox. On the other hand Linux has system tap but it doesn’t look very mature to me. Anyway, there are number of operating systems that support dtrace: as it is create by Sun engineers first come Solaris and OpenSolaris. Then there is the FreeBSD port and Apple OS X. I’m familiar with FreeBSD but I wanted to check the current state of OpenSolaris kernel. On the other hand I wanted to keep the learning curve less sloppy, so I opted for Nexenta core 2 rc1. Nexenta is GNU userspace (Debian/Ubuntu) and OpenSolaris kernel.

Download, install - everything was smooth. The install defaulted to root fs on ZFS. Good! I was thinking about playing with ZFS these days anyway.

And the moment of truth:

I started dbench -S 1, run dtrace -s iotop.d and here’s the output:

  UID    PID   PPID CMD              DEVICE  MAJ MIN D   %I/O
    0      0      0 sched            cmdk0   102   0 W     17

Hm, that looks somewhat familiar. I see a pattern there. Isn’t sched the ZFS cousin of pdflush/kjournald? Oh, well it is: http://opensolaris.org/jive/thread.jspa?threadID=39545&tstart=285

No luck… dtrace’s iotop works with UFS but has problem with ZFS.

Turns out the proper IO monitoring is a very tricky business.

MySQL Usage Accounting

Noise canceling

zImage — Thu, 26 Feb 2009 21:58:08 +0000

Wearing noise canceling headphones in a noisy data centre.

Listening to industrial/ebm music.

Sounds weird.

No related posts.

Tasks of the Day

zImage — Wed, 07 Jan 2009 18:45:02 +0000

So, as I was not very happy with how my projects were advancing I had to research and implement various systems to help me move forward. After much fiddling GTD was the first breakthrough. It helped me to get my current affairs in order and gave me the peace of mind to go to the upper levels. I identified my big goals and sought to align my actions with these goals. I learned to distinguish between task’s urgency and importance. Things started to gradually get better but still not good enough. I was still missing something… and it turned out that my daily routine is weak and allows me to procrastinate Basically I didn’t have a daily routine. I used to just open the list with tasks and diligently delay the more difficult looking ones until they fall off the current day and were left off for the next day. Probably these are just my personal flaws but the good news is there are fixes. The concept of Most Important Tasks of the day saved the day.

I was doing my daily task lists for some time when I stumbled upon the term MIT on the Zen Habits blog and I adopted it (I’m not a native English speaker).

Essentially you need to pick a bunch of tasks to do for the day. These are the Most Important Tasks. Of course you may do other things as well but you should throw all of your energy at completing these MITs. Make separate list with these tasks and stick to it.

Start with the most difficult or daunting task. This is important. Direct your efforts at the task you need most energy to accomplish. Keep an eye on the other important and urgent tasks. But you have good chance to do these later if they are easier and not that scary as the MITs. Thanks to my GTD based approach my tasks already have attributes like Energy (mental and physical), Importance, Urgency, Context and Time (ETA) so it’s rather easy for me to sort and choose.

It’s important to start with the tasks that require most energy because with the advancement of the day most people’s energy levels start to drop. If you delay the MITs too much you will not have the energy to start or complete them.

For example I discovered that my ability to concentrate varies greatly throughout the day. Even though I might think that in the evening I’m at the same energy level as in the morning I can easily prove myself wrong. Just have to try to focus on something more difficult while there is some distraction like say TV. I do ignore distractions much more easily in the morning. And it’s nearly impossible for me to do the same in the evening.

Because MITs list is small it allows for better focus. Most people have tens if not hundred of tasks in their lists (or worse - in their heads). If you keep this enormous pile of tasks in front of you it easily makes you feel overwhelmed and hesitant to start working on it.

The process of choosing tasks for the MITs list is essentially a planning process. The usual disclaimer about plans apply: circumstances may force you to abandon your plan but the planning process is important. Planning forces you to do the required thinking. As the saying goes: the failure to plan is a plan for failure. By keeping the more important tasks first you have the chance to complete them even if you’ve underestimated how much time would they take.

And finally the MITs list is a commitment. You bind yourself to the course of action.

For example that’s how I do it:

First I identify the hard landscape for the day. Are there any tasks that must be done at a specific time? Any meetings? At very least there’s your lunch and it’s definitely important.

After putting my hard landscape on the calendar I’m ready to distribute other tasks between the fixed ones. I keep in mind the context, energy levels, urgency and importance. You can’t do a task if you are not in the right context. And it’s a whole lot better to do some hard work that early in the day when you are still fresh. Don’t overcommit! You can always pick some more things to do if you finish earlier but people tend to underestimate the time required to complete a task. And in my case new things pop-up every now and then during the day.

It sounds like common sense but unfortunately it took me some time to figure out and more importantly to establish the habit. I was underestimating how important it is until I forced myself to include it in my daily routine and never miss it. Almost every time I skip it I got sloppy results. Of course the GTD still applies: if you find yourself stuck somewhere or your energy level drops dramatically for some reason - you can always pick another Next Action that matches your current context or energy level.

The Internet Tablet Revolution

zImage — Wed, 31 Dec 2008 16:55:00 +0000

Since the introduction of the first Palm devices back in 1997 I was tracking the evolution of pocket computers with great interest. However, for a very long time I couldn’t find the combination of factors that would make me buy one. The first device I seriously considered was N770 from Nokia. It was breakthrough in many ways but still quite limited in it’s memory and processing speed. I kept an eye on its development and when Nokia released it’s successor N800 I was sold. I bought one and used it for about 9 months when N810 was released. I immediately upgraded. I wear my N810 wherever I go for more than a year now and I’m very satisfied.

Some of the reasons I didn’t like any of the previous generations of PDA were their small screens, the use of closed operating systems or their high price tag. So the main advantages of N810 over it’s rivals at least for me are:

Superior display than most of the PDAs. 4.1″ is very good and balanced size. Any bigger and it won’t fit in your pocket. Superb 800×480 screen resolution - you can’t go better for a display of this size. And due to the fact this display is transflective it’s readable even in bright daylight.

N810 is more mobile than the netbooks, which are gaining popularity recently. It’s smaller and lighter and it fits in your pocket or in a lather sleeve like this one. Better mobility is also aided by the very good battery life. I use mine to read while commuting, some internet browsing and I might turn on the gps once in a while. That’s about 1.5 hrs daily and I keep it in off-line suspended mode (but instant on - less than a second) for the rest of the time. Its battery lasts about 5 days between recharges with this usage pattern.

N810 runs an open source, linux based OS and application stack designed for use with touchscreens. Openness, flexibility, best of breed package management (debian’s apt). Want maemo mapper + openstreetmap, xterm + openssh, Doom or other popular application? No problem - check out http://maemo.org/downloads/OS2008/. Not strictly related but it’s browser is gecko based (that’s mozilla & firefox engine) and has flash support. Some of the greatest apps for N810 are written by the community.

N810 has built-in gps and comes with Wayfinder pre-installed. If you want turn by turn navigation you’ll have to pay a modest subscription fee to use this option. maemo mapper is a free mapping and navigation alternative. maemo mapper’s routing capabilities are not as extensive and route recalculation needs an Internet connection but it’s very good app nevertheless. Go try it. There’s also navit - a very promising navigation app in the making. This gps led us trough most of the southern Spain few months ago.

As expected for an Internet Tablet N810 has bluetooth and wifi. No device is useful without good connectivity options in this day and age. Hook it to the nearest WiFi hotspot or your 3g enabled bluetooth phone. And it’s WiFi is among the best I’ve used - very good range and compatibility with hot spots around the world.

One of the great surprises was the quality of the built-in qwerty keyboard. I didn’t expected nearly as comfortable typing. Sometimes I have this feeling that I didn’t hit the right key but when I look at the screen it’s all correct. Very good job, Nokia.

Another surprise was the amazing sound for a device this small. It actually sounds better than my ThinkPad X41. It’s last.fm player, vagalume, is killer app. Use it like a portable personalized radio.

There are built-in camera and mic as well. Combined with the pre-loaded software like jabber client with voice and video support (and gtalk), gizmo and skype it’s the ultimate communicator.

It’s light sensor is very handy. Based on the current light conditions the display brightness is adjusted to save power and make work with the tablet more comfortable.

And last but not least the price: it was first released at $450 and is currently (Dec 2008) being sold for about $335 in various on-line stores.

What’s next? There are quite a lot people that prefer their internet tablet to have some built-in long range connectivity options in addition to WiFi like GPRS, 3G or WiMAX. Nokia responded to this demand with N810 WiMAX edition. I personally am completely satisfied with 3G via bluetooth phone.

But what’s more interesting is the upcoming fourth generation internet tablet (N900?). Rumours are that it will be OMAP3 based, will have built-in HSPA connectivity, high definition camera and hardware based graphics acceleration. Recently nokia released the Maemo 5 SDK which is targeted at this future device’s developers.

I’m glad someone (Nokia) finally has taken this technological way. I hope they continue evolving the internet tablet series in the same way and spirit. Can’t wait to see what N9xx will look like…

No related posts.

Merry Christmas

zImage — Thu, 25 Dec 2008 12:04:26 +0000

perl -e '$n="\e[0;40;32m";for(1..10)
{print"\ec$n\n".$"x(15)."\e[1;35mY$n" .$"x15;for(1..8)
{print$/.$"x($a=16-$_);$c=int rand($d=2*$_);for(2..$d)
{$f=31+int rand(3);print($_-$c?"X":"\e[$f;1m*$n")}print$"x$a}
print $/.$"x(31)."\e[0m\n".$"."\tMerry Christmas".$/x2;select$q,$q,$q,0.5}'

And for those of you lacking perl

No related posts.

On Rising Earlier

zImage — Mon, 01 Dec 2008 14:54:07 +0000

As a teenager I used to go to bed late in the night and then slept till noon. Then for the year I was in the army I had to give up on this habit. By the way I’m mostly pacifist but here in Bulgaria we had a compulsory military service at that time. Later after my discharge from the army I returned to my previous sleeping habits again. I was fiddling with my new toys - learning Linux, computer networks. I was hanging a lot on the IRC (actually I was the administrator of an IRC server). I even had some problems in the office because I was often late for work. I’m not sure why was that - probably because it used to give me some false sense of having more time?

Don’t know how happened but now I prefer to wake up early and in general to synchronize my active time with the bright daylight time. I definitely get better sleep at night when it’s dark and quiet. And then I feel fresher and more focused when it’s bright and sunny outside. It’s not bright and sunny during the winter but it’s still better. Moreover when I wake up early enough I don’t have to rush for work and my entire day is much less stressful. I do my breakfast, quickly scan the news headlines, go through the bathroom and dress up at a what I’d consider fairly relaxed speed. By the way most of the year I commute to work on a bicycle. This is not only far less stressful than driving but also saves me some time cause I have to go less often to the gym. Actually my situation is a little bit more complicated cause I work in shifts. I’m also on-call some of the nights. And I do my workouts in home without going to the gym… but you get the idea

I think that one of the main reasons the early rising has so positive effect on me is that it gives me enough time to tune up to the coming day and to calibrate my internal clock as well. You know how time flies when we’re busy, but really drags when we’re waiting? By having some free and calm time in the morning, as opposed to rushing through the door ’cause you are late for work, makes a difference to your perception of time for the whole day.

I really find it useful to step back and just look through the window for few minutes every now and then, just to slow down my time perception. It has the added benefit of relaxing my eyes and gives me some rest in general. And if you think that’s too wasteful - think about the time lost in inefficient or ineffective work, useless conversations, procrastination…

In order not to return to my sleep-till-late habit I use alarm even on the days I don’t need to wake up early. On these days I put it a little bit later to allow for 08:30-09:00 hours of sleep. It serves two purposes: on one hand I don’t oversleep and on the other hand if I wake up earlier I don’t have to get out of the bed because it looks like it’s late. I could just lounge till the alarm goes off.

P.S.: There’s a very interesting wikipedia article on the role of melatonin in humans.

No related posts.

Two months without blogging

zImage — Tue, 18 Nov 2008 11:17:51 +0000

Wow, It’s been more than two months without me blogging. But what months…

We’ve been to the marvellous Spain. Three weeks was enough time to completely forget my “ordinary” life and all the incomplete things fighting for my attention. While in Spain I’ve deliberately chosen to stay away from computers and then when we returned I still kept away from computers for a while driven by the inertia. I mean Internet connected PCs - you can’t easily avoid computers cause they are everywhere these days - in the camera, in the GPS, in the car, in the phone…

We travelled around Andalusia and some of Madrid and Valencia regions. We’ve been to Valencia, Mojacar, Granada, Tarifa, Seville, Cordoba, Madrid, Segovia, Avila, Toledo to name few. Overall we traveled about 4000 kilometres. It sounds like great distance but that’s only 10% of the Earth’s circumference. And still it’s quite a lot for three weeks

Spain is a big and diverse country. It meant lots of broken stereotypes for me. It’s organized into several autonomous communities and some of them even have their own local languages. For example at the Valencia’s airport there were signs in English, Castellano and Valenciano. Different landscapes, different cultures… We’ve seen wild flamingos, deer, wild boars, great sand dunes, countless beautiful beaches, white villages, castles, the big and modern Madrid. And I can’t find words to describe how delicious the food was.

From the local news - the 6th annual OpenFest conference - was held on 1st and 2nd of November. I liked only three of the lectures: “Wordpress.com - 40 million pages a day” by Nikolay Bachiyski, “28 Months Scalability” by Slavi Nikolov and “Free Hardware” by Radoslav Kolev. In my opinion the main problem was the weak presentational skills of the other lecturers. I know it’s easy to sit aside and criticise, so I’d be better take part and make a presentation for the next year’s OpenFest.

We used the opportunity to exchange PGP keys on the key signing party kindly organised by Peter Pentchev. There was a pleasant “unofficial after-party” as well, so overall I’d call this OpenFest a success.

At work I’m currently having fun with things like DRBD, Hearbeat, OpenVZ… My impressions from DRBD so far are that it’s a very complete product. Decent documentation, predictable behaviour. I wonder whether it has something to do with it’s commercial backing? OpenVZ looks quite complete as well.

I started to use my Facebook account a little bit more because some of my friends use it and it’s rather convenient way to show your pictures And Niki invited people to his birthday party via Facebook. Niiice party

I spend some time learning the Kohana framework + jquery + CSS. I don’t use it for something specific at the moment - just trying to figure out what is it like to create websites with modern tools like these. Most of the time I’m quite busy with system administration and I started to feel too far behind the web technologies So far it looks very powerful - nothing to do with the way things were done 5-6 years ago. I’ve tried to make a collapsed tree (ul/li) in which branches are dynamically populated on expansion by ajax - it fit in less than 10 lines of Kohana/Jquery. And it work’s on each browser I’ve tried so far. Impressive!

И те така те.

Maybe I’ll write in more details on the aforementioned topics. Or maybe won’t - only time will tell :-P. If you happen to know Bulgarian you can read more about our journey through Spain at Antonia’s blog:

By the way I’m looking for a book called Getting Things Done: The ABCs of Time Management by Edwin Bliss but I’m left with the impression that this book is out of print. Any ideas where to find it? Especially if there’s PDF or other open format version because I hate to wait for the dead-tree books to arrive. Moreover they are bulky and inconvenient to carry around.

No related posts.

MySQL Usage Accounting

zImage — Fri, 05 Sep 2008 08:00:54 +0000

Those of you who use MySQL in a multi-user environment are probably disappointed by it’s lack of per user usage statistics. For example in Linux there are BSD process accounting for CPU usage and IO accounting for disk usage tracking. Most applications like apache, ftp servers and mail servers provide some sort of usage statistics as well. But MySQL is the great anonymizer.

Not anymore, thanks to the people at Percona for their microslow patch and Google for the UserTableMonitoring patch.

Latest version of the patches can be found here but at the time of this writing they are against MySQL 5.0.62 and the current MySQL version is 5.0.67. So I tweaked the patches to apply cleanly to MySQL 5.0.67 and placed them for download: microslow, userstats, microslow+userstats.

IO performance monitoring

Going DSLR

zImage — Sun, 31 Aug 2008 09:49:06 +0000

We (me and Antonia) have finally decided to go the DSLR way. We were considering the switch for some time but only recently the pros of the current DSLR technology started to outweigh the cons for us.

We are proud owners of a Canon Powershot A95 compact digital camera for about 3 years and are generally very satisfied with it. Some of the main advantages of the compact digital cameras include:

Lightweight - A95 is not the lightest one at about 330 g but still only a half of the 2005’s lightest DSLRs.
Compact - you can carry it in your pocket.
Automatic shooting modes and various scene modes for point-and-shoot simplicity.
Live preview on a tilt LCD - none of the 2005’s DSLRs provided live preview and even though the current models do provide live preview it’s not nearly as useful as A95’s is.
Much cheaper than DSLR.

In November 2007 we were on a business trip to the USA and a friend of ours asked us if we could buy a Nikon 40D for her. You know - electronics are much cheaper in the US than in Europe. So we bought it and used it for few weeks while we were staying in Boston. It turned out DSLRs got cheaper, lighter and even my wife, who don’t want to know too much technical details about photography and prefers point-and-click interfaces, liked it. And the main selling points for us were:

Excellent low light/high ISO performance. This one is quite annoying with A95 and we have lots of blurry and noisy low light pictures.
No shutter lag. It’s instant. No more “oh, could you please stand still one more second please” or shots that include only the cat’s tail.

There are more SLR advantages that come as a bonus to the two above:

High degree of manual control.
Wide range of interchangeable lenses. Some are very expensive though.
Lenses are often high quality and produce images that are sharp and have a wide dynamic range.
Precise focus.
Higher frame rate.
More control over depth of field.

And some of the biggest disadvantages that were there in 2005 have improved significantly. For example Canon EOS 450D, aka Digital Rebel XSi, weights only 475 g (body only). A DSLR with comparable specifications weighted at least 700 g back in 2005.

So we decided to buy a Canon EOS 450D aka Digital Rebel XSi. It’s the latest incarnation of the Canon’s Digital Rebel line and some of the more notable improvements in comparison to 400D are:

Larger 3.0″ LCD monitor.
Live View.
Spot metering
Improved auto-focus.
3.5 fps continuous shooting.
Larger viewfinder.
Display of the ISO sensitivity on the viewfinder status line.

We are waiting for our shiny new Digital Rebel to arrive from the US. Without any lenses so far because we are going to test first what kind of lenses will suit us best. Thanks to my brother Dave who has a rather extensive collection of Canon compatible lenses and is willing to borrow us some of them for test ride.

No related posts.