Daily Archives:November 16th, 2013

Thoughts on home data storage: Drobos and Time Capsules

Drobo Right AngleI am fortunate to be the owner of three Drobos. One I bought, and two I have inherited from past companies etc, and they’ve served me very well. I have had many hard drives die over the years – and another goes every six months or so – but I’ve never lost any data if the drive was in a Drobo. And the flexibility just to plug in a new drive of any size at any time is great. They’re not perfect, but overall, they’ve been a very good place to put my music, photos and video-editing projects, without covering my desk with lots of different individual drives and their related cables and power supplies. The youngest one is more than four years old now, and they’re not particularly fast, but I’ve never had one fail.

Having said that, there is the problem of ‘How do you back up the Drobo’? If the unit should fail, I suspect getting the data off the drives would be very tricky, since they use their own custom filesystem – though, to be fair, that’s true of quite a lot of RAID systems. In some circumstances you can take the drive set out of one Drobo and plug it into another, so that’s probably the best route, unless you have the luxury, as I do, of backing your Drobo up to another Drobo! (A good use for older, slower Drobos). In any case, it’s worth remembering that however reliable the underlying hardware may be, filesystems can get corrupted, malware can attack, fire, burglary, or lightning strikes can take you by surprise, or users can accidentally delete things. Having a system which is resilient to hard drive failures isn’t the whole solution to the data storage problem. But it certainly helps!

None of my units, however, have network interfaces: they’re too old for that. And once you’ve dealt with the hard disk failure problem, most of those other threats are not going to be mitigated simply by backing up onto another Drobo sitting next to it on the shelf. In the past I’ve made Drobos available on the network by plugging them into various other machines: home-built Linux fileservers etc, but the Linux support for the Apple filesystems is not great, so it was never wholly satisfactory, especially for Time Machine backups.

timecapsuleThen, some months ago, I decided I needed a new wifi router which supported dual-band wireless, and so splashed out and got an Apple Time Capsule, which is essentially their Airport Extreme base station with an internal hard disk added, and it gave me both very good wifi and Time Machine backup space for every machine in the house. That’s really how it’s marketed, but you can also just use it as a generic file server, and in my experience, it’s a very good one.

I was nervous that I was being too much of an Apple fanboy in paying a premium price for a router plus disk, but I have never once regretted it. I don’t think I’ve ever used any networked storage which has been so simple and so reliable. I should mention that this is the previous generation of TC, not the latest, that I’m in an almost-all-Apple environment, and that I haven’t required it to do anything particularly unusual in the way of file-serving or router configuration, but for this scenario, it’s been quite superb.

And that’s not all. It has a USB port. So, as well as the internal disk, which I use for the Time Machine backups, I’ve plugged in a Drobo, and now have a few TB of nicely-redundant file storage for all my other backups humming away in a cupboard. Occasionally, I’ve opened the door and everything’s quiet, and after a moment of worry I realise that the Time Capsule is just very good about putting its disks into standby when not in use.

So this is really just a recommendation, both for the Time Capsule, and for Drobos (even elderly ones), and for the combination. If I were starting from scratch and looking for networked storage, I’d have to consider Synology, who also have a very loyal following, and whose devices can arguably do rather more than even the newer, networked Drobos.

But, for now, this arrangement is working well for me.

IMAP, Sent mail, Apple Mail and Mavericks

This is one of those posts that’s chiefly intended for those Googling for a particular problem. It might still make gripping reading, though, for those of you interested in the internals of email protocols…

Most email programs nowadays allow you to specify the folder in which you want to save your outgoing messages, and choose whether that should be stored locally or on your email server. (Assuming you’re using IMAP to fetch your mail, that is. If you’re still using POP, you should get another mail provider. And if you’re using Exchange… well, you have my sympathy…)

But different apps have traditionally had different names for this folder: some call it ‘Sent’, others ‘Sent Items’ or ‘Sent Messages’ and some will use a folder with one name and display it as something else to the user. (The same is sometimes true of ‘Drafts’, ‘Trash’, and ‘Junk Mail/Spam’). So, over the years, I’ve tended to standardise on ‘Sent’, and when I set up a new mail app or a new machine, I configure it to use that folder.

But recently, that setting didn’t always seem to be stick, and I found some of my mail would end up in different folders when sent from some devices. Still, I persevered, until I installed Mavericks on my Mac, and found that the setting wasn’t even available on Apple Mail, at least, not for my main account – it was greyed out! What could be going on?

Screen Shot 2013-11-16 at 08.37.45

So I started to investigate. I dug into the file that Mail uses to store information about its accounts (currently ~/Library/Mail/V2/MailData/Accounts.plist) and I came across a setting which gave me a clue: it was called HasServerDefinedSentMailbox, and for this account it was set to YES. Mmm…

In the past, IMAP basically just provided you with a smart filing system for your mail, and it’s proved a remarkably resilient one, when compared to other formats. As an aside, I felt very old recently when I told a colleague in the lab that I had used the same method for storing my mail for ages, and had emails from 1991/92 in there that were just as accessible now as they had been then. He laughed, and said, “That’s the year I was born!”. Sigh… Still, compare that to data stored n tapes and floppies.

Anyway, a few extra features have been added since then, and one of these came just a couple of years ago. RFC 6154 describes ‘new optional mailbox attributes that a server may include in IMAP LIST command responses, to identify special-use mailboxes to the client, easing configuration’. In other words, the server can tell your app which folders to use for these key functions. This makes a lot of sense, particularly when your email provider also has a webmail interface, for example. I use Fastmail, which has a really good one, and, of course, it needs to know what you want to use for sent mail, drafts, etc when you’re using it via the web. Fastmail reflect these folder choices in the IMAP protocol, to keep everything consistent. Which is fine by me: I now simply stick to using the ‘Sent Items’ folder that the server recommends, and all is well on all my devices.

Anyway, all of that is a long way of explaining why you may find the ‘Use this mailbox for’ menu items are greyed out, and why on iOS devices you may try changing the ‘Sent Mailbox’, only to find that your new setting doesn’t stick. If your server is specific about which folders should be used, Apple will take that setting seriously, which I think makes sense, but they aren’t yet very clear in the UI about why you can’t then change it yourself.

Hope that’s useful to somebody!


Continuing the electric bike theme…

© Copyright Quentin Stafford-Fraser