Category Archives: Open Source

GRUB, Ubuntu, and failed boots

Another geeky technical post. Ignore if it’s not your thing….

I wrote recently about the GRUB bootloader and how it can sometimes cause a remote or headless server not to come back online after, e.g. a kernel update.

This can happen in other situations: The configuration used on recent versions of Ubuntu is such that, if the system thinks that the last boot attempt failed, it stops it automatically booting into the same configuration again by cancelling the countdown timer, setting the number of seconds to -1, causing it to wait indefinitely at the boot menu until you decide on the best action to take. This is a sensible default, because a machine that goes into an infinite loop of reboots is doing nobody any good and puts a fair bit of strain on its own hardware.

Unfortunately, though, other things can trigger this behaviour. If you have a power fluctuation, for example, such that the machine restarts, gets part-way and then power-cycles again, you may find yourself with a machine that doesn’t come back online of its own accord.

On the most recent Ubuntu versions (12.04 with updates, and later) you can add a setting to /etc/default/grub:

GRUB_RECORDFAIL_TIMEOUT = 30

for a 30 second timeout after it has recorded a failure condition. You can use 0 if you don’t want it to pause and show the boot menu at all, but remember that it could then go into fairly fast repeated reboots if something really does go wrong.

On earlier versions, you’ll need to edit /etc/grub.d/00_header and find the line near the bottom, in the maketimeout()_ function, which sets

set timeout=-1

and change that to your preferred value.

In either case you’ll then need to run:

update-grub

to make your changes take effect.

Old men forget; yet all shall be forgot, but he’ll remember, with advantages…

Richard and I have been playing with flash cards as a way of learning things.

The great thing about an electronic implementation of the old ‘question on one side, answer on the other’ idea, is that it can make smart decisions about when and how frequently you should be presented with a particular card. Things you find easy to remember need only occasional repetition, while those which are new or more challenging need more regular viewing until they stick in your memory. When you see the answer, you just say whether or not you got it right, and how hard you found it.

Richard wrote a little while back about using this model to learn a reading he had been asked to give at a wedding. I’ve always liked learning poetry or bits of Shakespeare, but often find that large chunks will flow easily while there are one or two lines I always forget. Could this be the solution?

One of the popular flashcard systems out there is an Open Source one called Anki, created by Damien Elmes. It has Windows, Mac, Linux and Web clients, plus Android and iOS (though these don’t yet work on the latest version). And there are various ways you can get decks of cards in and out. The user interfaces are rather quirky, I find, and even the web sites can be confusing to navigate, but the underlying system works fine.

It’s easy to find plain-text versions online of most things I want to learn, so I wrote a little script called poem2anki which will take a text file containing lines of poetry (or prose!) and convert it into a file suitable for importing into Anki.

A question:

and the answer:

It will create these for all the lines in the poem, but you’ll quickly find you’re only tested on the ones you find difficult to answer.

You can find poem2anki here if wanted.

Remote keyboard and mouse for your RaspberryPi

Warning – technical post ahead…

My shiny new RaspberryPi came through the door this week, and the first thing I noticed was that it was the first computer I’d ever received where the postman didn’t need to ring the bell to deliver it.

The next thing I discovered, because it came sooner than expected, was that I was missing some of the key bits needed to play with it. A power supply was no problem – I have a selection of those from old phones and things, and I found a USB-to-micro-USB cable. Nor was a network connection tricky – I have about as many ethernet switches as I have rooms in the house.

A monitor was a bit more challenging – I have lots of them about, but not with HDMI inputs, and I didn’t have an HDMI-DVI adaptor. But then I remembered that I do have an HDMI monitor – a little 7″ Lilliput – which has proved to be handy on all sorts of occasions.

The next hiccup was an SD card. You need a 2GB or larger one on which to load an image of the standard Debian operating system. I had one of those, or at least I thought I did. But it turned out that my no-name generic 2GB card was in fact something like 1.999GB, so the image didn’t quite fit. And a truncated filesystem is not the best place to start. But once replaced with a Kingston one – thanks, Richard – all was well.

Now, what about a keyboard and mouse?

Well, a keyboard wasn’t a problem, but it didn’t seem to like my mouse. Actually, I think the combined power consumption probably just exceeded the capabilities of my old Blackberry power supply, which only delivers 0.5A.

If you’re just doing text-console stuff, then this isn’t an issue, because it’s easy to log into it over the network from another machine. It prints its IP address just above the login prompt, so you can connect to it using SSH, if you’re on a real computer, or using PuTTY if you’re on Windows.

But suppose you’d like to play with the graphical interface, even though you don’t have a spare keyboard and mouse handy?

Well, fortunately, the Pi uses X Windows, and one of the cunning things about X is that it’s a networked display system, so you can run programs on one machine and display them on another. You can also send keyboard and mouse events from one machine to another, so you can get your desktop machine to send mouse movements and key presses from there. On another Linux box, you can run x2x. On a Mac, there’s Digital Flapjack’s osx2x (If that link is dead, see the note at the end of the post).

These both have the effect of allowing you to move your mouse pointer off the side of the screen and onto your RaspberryPi. If you have a Windows machine, I don’t think there’s a direct equivalent. (Anyone?) So you may need to set up something like Synergy, which should also work fine, but is a different procedure from that listed below. The following requires you to make some changes to the configurations on your RaspberryPi, but not to install any new software on it.

Now, obviously, allowing other machines to interfere with your display over the network is something you normally don’t want to happen! So most machines running X have various permission controls in place, and the RaspberryPi is no exception. I’m assuming here that you’re on a network behind a firewall/router and you can be a bit more relaxed about this for the purposes of experimentation, so we’re going to turn most of them off.

Running startx

Firstly, when you log in to the Pi, you’re normally at the command prompt, and you fire up the graphical environment by typing startx. Only a user logged in at the console is allowed to do that. If you’d like to be able to start it up when you’re logged in through an ssh connection, you need to edit the file /etc/X11/Xwrapper.config, e.g. using:

sudo nano /etc/X11/Xwrapper.config

and change the line that says:

allowed_users=console

to say:

 allowed_users=anybody

Then you can type startx when you’re logged in remotely. Or startx &, if you’d like it to run in the background and give you your console back.

Allowing network connections

Secondly, the Pi isn’t normally listening for X events coming over the network, just from the local machine. You can fix that by editing /etc/X11/xinit/xserverrc and finding the line that says:

exec /usr/bin/X -nolisten tcp "$@"

Take out the ‘-nolisten tcp’ bit, so it says

exec /usr/bin/X "$@" 

Now it’s a networked display system.

Choosing who’s allowed to connect

There are various complicated, sophisticated and secure ways of enabling only very specific users or machines to connect. If you want to use those, then you need to go away and read about xauth.

I’m going to assume that on your home network you’re happy for anyone who can contact your Pi to send it stuff, so we’ll use the simplest case where we allow everything. You need to run the ‘xhost +‘ command, and you need to do it from within the X environment once it has started up.

The easiest way to do this is to create a script which is run as part of the X startup process: create a new file called, say:

/etc/X11/Xsession.d/80allow_all_connections

It only needs to contain one line:

/usr/bin/xhost +

Now, when the graphical environment starts up, it will allow X connections across the network from any machine behind your firewall. This lets other computers send keyboard and mouse events, but also do much more. The clock in the photo above, for example, is displayed on my Pi, but actually running on my Mac… however, that’s a different story.

For now, you need to configure x2x, osx2x, or whatever is sending the events to send them to ip_address:0, where ip_address is the address of your Pi. I’m using osx2x, and so I create the connection using the following:

Once that’s done, I can just move the mouse off the west (left-hand) side of my screen and it starts moving the pointer on the RaspberryPi. Keyboard events follow the pointer. (I had to click the mouse once initially to get the connection to wake up.)

Very handy, and it saves on desk space, too!

Update: Michael’s no longer actively maintaining binaries for osx2x, and the older ones you find around may not work on recent versions of OS X. So I’ve compiled a binary for Lion(10.7) and later, which you can find in a ZIP file here. Michael’s source code is on Github, and my fork, from which this is built, is here.

Small but powerful

My friend Aideen has been doing fun stuff with the TP-Link micro-routers.

These things are amazingly small – note the relative size of the USB socket – and very cheap.

She’s written it up in a nice blog post here.

 

 

After the $100 laptop, the $50 desktop?

It’s almost exactly 10 years since we started the Ndiyo project, with the aim of providing computing access to people for something “closer to the cost of a VGA lead than the cost of a computer”. Ndiyo has now formally closed, but it led to many other activities, including the founding of DisplayLink, and successful past projects in collaboration with the GSM Association, No-PC, and others.

Today, there’s some more good news.

Towards the end of Ndiyo’s life we started to experiment with a model we called ‘Hubster‘ – the name coming from using a USB hub as the core of a thin-client terminal, something made possible once DisplayLink’s evolution of the Ndiyo technology allowed monitors, as well as keyboards and mice, to be connected over USB. The idea is to share the power, cost, and the carbon footprint of a PC between two or more users at once, simply by plugging in enough USB peripherals to give the extra users access to it. This is important for everybody, but especially for the poorer parts of the world where the cost of owning one PC per person is prohibitive.

Well, over the last few years, Bernie Thompson at Plugable.com in Seattle has been beavering quietly away to make this more of a reality, by providing DisplayLink-based terminals at reasonable prices and by maintaining the Open Source software to drive them. There are two bits of good news coming from his recent efforts:

  • First, he’s been working with RedHat to get support for USB terminals built in to the standard distribution of Fedora 17. We’re getting very close now to the ideal situation where you can turn any Linux box into a multi-user box simply by plugging in enough components for a new user, and a new login prompt will appear when you do so. Imagine, say, a disaster-relief information centre where one person with a laptop connected to a satellite link can easily provide access for half a dozen members of the team just by plugging them in. And it’s all Open Source. Zero extra software cost per user.
  • Secondly, in an attempt to get the cost of these terminals down and support ongoing development of the Open Source components, he’s launched a Kickstarter project called ‘The $50 Computer‘.

Some may, quite sensibly, ask how this compares to the RaspberryPi, which is, after all, even cheaper, and is a standalone machine. Well, this one comes with a box!

No, seriously, they are both excellent projects – I have a RaspberryPi on order, too – but they fill different roles. RaspberryPi, in the early years at least, will be about teaching people the basics of how computing works.
Bernie and the Plugable team are creating a system where providing fully-featured applications to multiple users at very low cost is something that a non-technical user can do simply by plugging in USB devices.

In some situations, perhaps in an internet cafe, you may just want to give extra users access to a Chrome browser. But with this system you also have the option of providing them with OpenOffice, with Scrivener, with Blender, with Corel Aftershot Pro, with Sublime Text, with Skype, with… well you get the idea! And, of course, with access to however many giga- or terabytes of storage you care to put in the PC.

I think the Kickstarter plan is a great one – I wish it had been around when we started Ndiyo.

Please support it if you can… Even if you don’t need more affordable computing devices, for your kids, your school, your internet cafe, your office, there are millions who do. Billions, in fact.

Every $10 we can shave off the cost of access to IT makes it accessible to many thousands of new people globally… and now you have a chance to help.

Cashing in on Open Source

I’ve recently switched to using GnuCash to manage my accounts, and have been quite impressed with it. A friend recently asked about software for simple bookkeeping, and I wondered whether GnuCash might be appropriate, but I didn’t find an introduction that took someone unfamiliar with accounting software through the absolute basics. So here’s my attempt to do so. It’s 26 minutes long.

Direct link.

Multiple GnuCash files on Mac OS X

This is one of these ‘just in case you’re Googling for it’ posts. Most readers can probably ignore!

I’ve been experimenting again with GnuCash, the Open Source accounting package that runs on the Mac, Windows, Linux and others. There’s rather a paucity of good accounting packages for Mac users, especially now that the good old MYOB (“Mind Your Own Business”) is no more, but last time I looked at GnuCash it was something that you’d have to be a pretty die-hard Open Source enthusiast to want to use.

Well, it’s come on a long way, and I’ve been merrily importing statements downloaded from my banks and setting it up for both personal and business use. If you’re familiar with double-entry bookkeeping it all works very nicely, and it stores all the data in Gzipped XML files, so I can be pretty confident that I could get it out and into another format if I ever needed too.

Only one hiccup so far – the standard build for the Mac (which no longer requires X11) assumes that you only want one set of accounts (ie. one file) open at one time, and it helpfully closes your current project when you open a new one. But it can be very handy to have your personal account and business account, or more than one business, open at the same time, especially if you’re shifting money between them.

Here’s how to do it:

Instead of running the GnuCash app and using File > Open, start it from the command line, e.g.:

open -a Gnucash -n --args MyBusiness.gnucash
open -a Gnucash -n --args Personal.gnucash

The ‘-n’ means ‘start a new instance even if the app is already running’. Note that some of the preferences etc are still per-user rather than per-file, but assuming you’re not doing anything too complex, this works fine.

You could use Automator to create a handy one-click launcher in your dock for each account to do the same thing. Or, create an Automator application using the ‘Run Shell Script’ action, and specify the script as:

open -a Gnucash -n --args $*

Save the application, and any .ledger files dropped onto it will be opened in their own session.

Alright, Mr. DeMille, I’m ready for my close-up.

OpenCV is a wonderfully full-featured computer vision library. I’ve just written a very simple demo of the built-in face recogniser. It finds a face and scales it to a fixed size. If you watch my eyes in the viewfinder window, you’ll see they stay pretty much in the same place however I move around the room.

All sorts of things could be done to improve the frame-rate if needed, but this was just a quick test I put together over a couple of hours while learning about the library. Back in ancient history, when I did my PhD, this kind of thing would have taken weeks…

The title of this post, if you’re not familiar with it, is from the famous closing scene of Sunset Boulevard, which you can see here.

Of course, as soon as I thought of using this title, I realised that I could also grant Gloria Swanson’s greatest wish. So here’s my version…

It needs some smoothing, but still quite fun.

Speed up slow console in VirtualBox

This is one of those geeky posts that is here for the benefit of (a) my failing memory – so I can search for it later – and for (b) anyone who happens to Google for the right keywords.

At Camvine, most of our development is done on Linux, but many of us have Macs as our preferred desktop machines. So we regularly install Ubuntu Linux Server as a virtual machine using VirtualBox. We’re not interested in a graphical interface to the VM – the console is fine. Unfortunately, the default video driver that Ubuntu uses when running under VirtualBox is painfully slow, to the extent that we usually minimize the window and SSH into the machine.

Fortunately, the guys on this thread found a solution – thanks everyone!

On the VM, edit /etc/modprobe.d/blacklist-framebuffer.conf and add a line:

blacklist vga16fb

Reboot the VM and speedy scrolling should be yours again.

(Of course, you may still want to ssh in to get more convenient cut & paste, etc., but it’s nice to have a speedy console anyway, and you may not always have network access to the VM.)

Duplicate mail messages

In my various shufflings, copyings, archivings of email messages between my IMAP folders, I often end up with duplicates.

Sometimes, a copy or move goes badly wrong and I end up with hundreds of duplicates.

Many years ago I wrote a bit of Java code which would find and remove duplicates, but I’ve now converted it to a Python script and released it as Open Source, in case it’s useful to anyone else.

You can find IMAPdedup here.

Feedback and improvements welcome!

Good indentations

I saw and liked this Python / Google T-shirt at Opentech.
(If you don’t understand it, don’t worry – it’s a very geeky joke!)

VNC2DL

vnc2dl2Warning – for geeks only…

I’ve just posted an alpha version of VNC2DL on github.

This is a VNC viewer which uses the new Open Source library from DisplayLink to display a VNC session on a USB-connected display, rather than in a window.

Just in case it’s useful to anyone…

© Copyright Quentin Stafford-Fraser