Category Archives: Open Source

Virtualisation

A while back I thought virtualisation technology was going to be the hot topic of 2006. Well, it was, in certain niche areas, but the momentum is still growing.

Shortly after VMware’s amazing IPO, XenSource, a spin-out from the Cambridge Computer Lab, have been bought by Citrix in a deal worth $500M. And not all of the money is virtual – there’s a good chunk of cash there too.

Many congratulations to my pals there, who will now definitely be buying the drinks next time we meet at the pub.

But this is also a nice challenge to those who don’t believe you can make money from Open Source…

Hex your Mac

Want to edit binary files on your Mac? Hex Fiend looks neat.

Open Source in action

I had a great time last night visiting Catalyst, who provide software and services to some pretty high-profile NZ projects, but also seem to have a lot of fun. I liked them immensely, and it’s always encouraging to see successful businesses built around Open Source.

Some of them are helping with OLPC on the side, and I was able to play with one of the laptops for the first time. I was quite impressed.

olpc-gw.jpg

MacFusion

Michael blogged recently about MacFUSE, which lets you mount SSH-accessible servers as file shares on your Mac. It’s very handy.

And now it has a prettier front-end, in the form of MacFusion.

Tick-tock

SVG – scalable vector graphics – is an XML-based standard for storing images based on lines, curves and shapes (as opposed to photo-type pictures, which are arrays of pixels).

It’s been around for some time – I think I first experimented with it in around 1999 – but there were a very limited number of programs able to create or view SVG files. That’s changing, however, and SVG is gaining ground for a variety of reasons:

  • An SVG file can include Javascript, which can modify the graphical components to create an animation
  • It turns out to be quite a good format for delivery some types of graphics to devices like mobile phones
  • Firefox supports it – which means that a very large number of people are now able to see SVG images without installing any extra software

Martin sent me a link to this simple but very pleasing SVG animation by Tavmjong Bah, which you should be able to see if you’re using Firefox or similar browsers.

It assumes you’re on a large display, though, and if not, you might like this version, which I simply scaled down using the free Inkscape application. Note that the animation still runs, that it shows your timezone, and that if you were to scale it back up you’d get the full quality of the original.

GRUB CD image and Error 15

This is a post for anyone who, like me, has been doing web searches to find out what might be the problem if your Linux machine displays GRUB Error 15 on booting. Or who has general GRUB issues to debug.

My problem was Error 15, which indicates that GRUB cannot find one of the files it needs. If you get it while setting up GRUB, it’s often fairly easy to find out what’s wrong. But if GRUB thinks it’s installed OK, and you then reboot, you can still get this message but without any further information to help you debug it.

To cut a long story short, the issue for me was that the BIOS (and GRUB while booting), saw my two hard disks in a different order from the way the kernel saw them after booting. So my assumptions that /dev/sda was the same as (hd0) was invalid.

Finding this out took a very long time, though, because, for reasons too complex to go into here, I was booting this server not from a regular CD but from an emulated CD the other side of the Atlantic.

Things became a lot faster when I found this section in the GRUB documentation which explained how to make a bootable CD ISO image with GRUB on it. To save you the trouble, here’s one:

I could mount this and use the GRUB console command line to find out what was wrong. It’s worth exploring the GRUB console, if you haven’t already. It can do things like filename completion when you press TAB, and can even display the contents of text files using, for example,

cat (hd0,0)/boot/grub/menu.lst

In my case I found that the BIOS of the machine allowed me to choose the boot order of the hard disks, and swapping them there was the easiest solution.

Hope this is useful to somebody!

Europe’s feeling foxy

The Firefox browser continues to gain popularity in Europe, with a market share of just over 23%, according to research done by a French firm and reported in The Inquirer.

We in the Royaume Uni are a little behind some of our neighbours at 15.8%, while Slovenia has adopted Firefox to a whole-hearted level of over 40%.

Virtual-Q

I’m just moving Status-Q to a new server. If you can see this, it’s working!

The Status-Q server is now actually a virtual one, based on the Xen virtualisation system. I had about 18 web domains on the old server, and I wanted a bit more flexibility in how they’re managed, so I’m gradually migrating them onto a set of virtual servers on the new machine, which will make it easier, for example, to upgrade key bits of software without endangering all the sites at once.

Please let me know if you notice any problems…

Open Source database frontend

These days, most databases use a web browser as a frontend and something like Ruby on Rails or Django to link the web to the database. But if you’re on Linux and looking for a desktop equivalent to Microsoft Access, Knoda is a good place to start.

Still not quite foxy enough

Firefox 2 is out. Can’t say I really notice much difference, but I thought 1.5 was just fine, so I’m not upset.

On the Mac, though, it still doesn’t deal with RSS feeds as nicely as Safari, so it won’t quite displace my default browser yet.

Samba and the French Cafe Technique

To the London LinuxWorld today, which seemed miniscule after the San Francisco one only a few weeks ago. I was surprised at just how small it was, given the importance of Linux to the UK, and the importance of the UK to Linux, but it was still interesting.

On the train I was listening to podcasts, as I often do now while travelling. That has been the biggest impact of the iPod for me personally: the fact that I no longer consider waiting time and travelling time as wasted time, even if I’m driving or walking and couldn’t easily read a book. I spend much more time on my iPod listening to speech than I do to music…

Anyway, one of the interviews I listened to was with Jeremy Allison, a key developer of Samba. For those who don’t know, Samba is free software which shares files and printers over a network using Microsoft’s protocols, meaning that Windows machines can communicate with Macs, Linux & Unix boxes and a variety of other devices that use Samba under the hood. It’s a very important piece of software, and I’ve been using it for about 11 years.

Of course, Microsoft don’t publish the details of their protocols. If they had their way, then Windows machines would only be able to talk to other Windows machines. So Andrew Tridgell, the creator of Samba, has to work out what they’re doing through a variety of cunning techniques. He wrote a nice article about how he does it. I particularly liked his description of what he calls ‘The French Cafe Technique’:

Imagine you wanted to learn French, and there were no books, courses etc
available to teach you. You might decide to learn by flying to France and
sitting in a French Cafe and just listening to the conversations around you.
You take copious notes on what the customers say to the waiter and what food
arrives. That way you eventually learn the words for “bread”, “coffee”
etc.

We use the same technique to learn about protocol additions that Microsoft
makes. We use a network sniffer to listen in on conversations between
Microsoft clients and servers and over time we learn the “words” for “file
size”, “datestamp” as we observe what is sent for each query.

Now one problem with the “French Cafe” technique is that you can only learn
words that the customers use. What if you want to learn other words? Say for
example you want to learn to swear in French? You would try ordering something
at the cafe, then stepping on the waiters toe or poking him in the eye when he
gives you your order. As you are being kicked out you take copious notes on
the words he uses.

The equivalent of “swear words” in a network protocol are “error packets”.
When implementing Samba we need to know how to respond to error conditions. To
work this out we write a program that deliberately accesses a file that
doesn’t exist, or uses a buffer that is too small or accesses a file we don’t
own. Then we watch what error code is returned for each condition, and take
notes.

Making out?

Warning – geeky post ahead… I’ve been doing some coding this week…

Anyone who’s done any programming, at least if it’s outside the limited confines of an integrated development environment, will have come across the make utility, which was developed nearly 30 years ago at Bell Labs.

Make lets you list which bits of a program depend on which other bits, so that when you make a change, say, to one of your source code files, you can just type ‘make’ and the bits which need to be updated as a result all get rebuilt automatically.

People use it for other things, too; many years ago I had to produce Rose’s PhD thesis through a rather complicated process which started with floppies from a dedicated Panasonic wordprocessor, ran through a C program I wrote to decode the disk format, a Perl script to convert the files to LaTeX, and then Latex itself and finally dvi2ps to get the Postscript output for printing! Each stage generated different files, updated tables of contents etc, and when she fixed a typo on her Panasonic, I needed to ensure that it was propagated through the entire pipeline and made it into print. Make was good for that.

But anyone who’s built a project of any size will also know that make is far from perfect. It really hasn’t evolved much in its thirty years and the syntax, while elegant for very small projects, becomes unintelligible for large ones.

Here’s a small segment of a Makefile I was writing last week:

define PROGRAM_template
$(1): $$($(patsubst %,%_OBJS, $(notdir $(1)))) \\
        $$($(patsubst %,%_LIBS, $(notdir $(1))))
     g++ -o  $$@ $$^ $$($(patsubst %,%_LDFLAGS, $(notdir $(1)))) $(LDFLAGS)
all:: $(1)
$(foreach obj, $($(notdir $(1)_OBJS)), \\
     $(eval $(call OBJECT_template, $(obj),$(notdir $(1)) )) )
endef

# Build every program defined in PROGS
$(foreach prog,$(PROGS),$(eval $(call PROGRAM_template, $(prog))))

Don’t bother trying to understand this. The point is that it’s pretty arcane stuff and I wasn’t trying to do anything too sophisticated here.

If you’ve written Makefiles before you probably know roughly what’s going on, but do you know exactly what’s going on? Would you have got the right number of $ signs in the right places? Could you say why $$^ is in here and not $$< ? Why I have to call things and then eval them? Then try and imagine what it’s like for somebody seeing a Makefile for the first time!

And here’s the worst bit: for this to work at all, the gap at the beginning of the ‘g++’ line must be a tab, not spaces. So simple code that looks perfectly correct may not actually work when you try to run it. It’s a nightmare.

So last week I decided that using make in the 21st century was probably ridiculous, and it was time to search for alternatives.

Continue Reading

© Copyright Quentin Stafford-Fraser