24 November 2008

alias or function? and bash 'time'

In a previous post (and a follow-up), I wrote about a number of useful and time-saving aliases that I use in the bash shell. Those posts are pretty old now. In the meantime, I've defined another interesting alias. While nothing special in itself, I've discovered that this is one task that can be done better in another way.

The new alias is this:
pig='pkg_info | grep '

I've found it to be very useful. I often want to know what version of some port is installed, or if I have a port installed at all, and this allows me to find out quickly without paging through the whole list. For example:
mybox$ pig swfdec
swfdec-0.6.8 Flash Rendering Library
swfdec-plugin-0.6.0_1 Flash rendering plugin

It's very much like the alias "psg='ps auxw | grep '" that I'd talked about previously.

But, on a box with a large amount of software installed -- which is pretty well any FreeBSD desktop now that we have modular X -- this new command is pretty slow. Let's use bash's time utility and see just how slow:
mybox$ time pig swfdec
swfdec-0.6.8 Flash Rendering Library
swfdec-plugin-0.6.0_1 Flash rendering plugin

real 0m13.245s
user 0m0.306s
sys 0m0.233s

Thirteen seconds, wow! Immediate, subsequent re-invocations should go much faster since the file information has been cached. See:
mybox$ time pig swfdec
swfdec-0.6.8 Flash Rendering Library
swfdec-plugin-0.6.0_1 Flash rendering plugin

real 0m0.416s
user 0m0.258s
sys 0m0.163s

But, the underlying problem remains: pkg_info spends a lot of processing time going through all the installed ports, and we're only interested in one (or a few). We can do better.

Knowing that the package information is found in the directory '/var/db/pkg/' and how it's structured, we can use filename expansion and a couple of other tricks to dramatically reduce these times. After some fiddling, this is what I've come up with:
pi2 ()
{
RND=$RANDOM;
cd /var/db/pkg/;
cat *${1}*/+COMMENT > /tmp/pig.$RND;
echo *${1}* | sed 's/ /\
/g' | paste - /tmp/pig.$RND;
rm /tmp/pig.$RND;
cd $OLDPWD
}

I call it 'pi2' here for the sake of clarity. Yes, that's a literal newline for the sed replacement string (might cause problems with some versions of sed :/ ).

This function dramatically reduces times. Compare these times with those above:
mybox$ time pi2 swfdec
swfdec-0.6.8 Flash Rendering Library
swfdec-plugin-0.6.0_1 Flash rendering plugin

real 0m0.015s
user 0m0.001s
sys 0m0.023s

And, these fast times stay nearly the same when the file information is not cached. Which takes away that pesky 10 plus second wait when you want to use this utility and the file information doesn't happen to be cached already.

Labels: ,

4 Comments:

At 19/4/09 13:04, Anonymous Anonymous said...

pi2.sh
real 0m0.047s
user 0m0.006s
sys 0m0.038s

pi3.sh
real 0m0.028s
user 0m0.007s
sys 0m0.021s

On a side note; I get misalignment of data when I run pi2.sh. IE:

xf86bigfontproto-1.1.2 X11 libraries and headers from X.Org
xmlcatmgr-2.2 X11 protocol headers
xorg-libraries-6.9.0_1 Abstract network code for X
xproto-7.0.10_1 The Z shell
xtrans-1.0.4
zsh-4.3.4_2

Here is the pi3.sh
#!/bin/bash

LPATH="/var/db/pkg/"


for i in `ls $LPATH | grep -i $1`
do
COMMENT=`cat $LPATH$i/+COMMENT`
echo $i $COMMENT

Sorry about the text wrapping.

 
At 19/4/09 13:17, Anonymous Anonymous said...

I forgot to mention that I really enjoy your blog kace.

PS: the above script could / should be improved to handle $1 equals 0. if $1 equals 0 then cat all +COMMENT

PSS: You always have great ideas. I would have never thought about trying to improve pkg_info; nor would I have looked into or examined the directory / data structure. You're an original thinker.

Thanks for the blog.

 
At 20/4/09 00:37, Blogger kace said...

Well, 'pi3' is definitely a great improvement. 'pi2' was done kind of quickly and has several problems.

Thank you for the feedback and thank you very much for the kind words.

 
At 22/4/09 23:52, Anonymous Anonymous said...

Well, 'pi3' is definitely a great improvement. 'pi2' was done kind of quickly and has several problems. Thanks for the compliment.

Thank you for the feedback and thank you very much for the kind words.You're welcome; please keep up the good work.

PS: my benchmark was done on a public (shell) FreeBSD server. Your script is still much faster.

There is one bug; it checks the:
/var/db/pkg/pkgdb.db/ director and doesn't find a COMMENT file.

Cheers

 

Post a Comment

<< Home