Tag Archives: backups

The Fragile Free Cloud

Here’s a timely reminder, if one were needed, that you should never assume anything you store online is going to be there for very long, unless it’s on a system (a) that you are paying for and ideally (b) that you run or manage.

Flickr has announced that it’s going to start removing photos from its free accounts: everyone can still have 1,000 images, but that’s much less storage than they offered for free in the past. If you have more than that, they’ll start deleting the older ones first. I starting uploading things to Flickr about 13 or 14 years ago, so 90% of my 10,000 Flickr images will vanish over the next few months.

Most of the Snapchat/Instagram generation are probably not interested in anything that happened more than 1000 images ago! But people who have used Flickr for archiving the first pictures of their children or grandchildren may be in for a surprise. The name ‘Flickr’ might have a certain irony to it…

Now, this is a perfectly reasonable thing for the company to do, and there are several ways you can deal with it: you can start paying for your account, you can download your images if you don’t have local copies, or you can migrate them over to Smugmug (who now own Flickr). But only the first of those options will keep your photos nicely arranged in their albums, and, more importantly, will preserve your image URLs, so I imagine there will be a very large number of pages around the world with Flickr-shaped holes in them where an image used to be. Whichever option you choose, do it before the end of the year.

Now, I’ve been a fan of Flickr for a long time, and paid for an account for about a decade — it’s a good service and reasonably priced — but I switched to Smugmug a few years back because it was a better fit for my occasional bits of professional work. I don’t mind paying for one photo storage service, but I’d rather not pay for two, especially from the same company! So my photo archive has been copied to SmugMug, and I’ll probably need to write a bit of code to go through my blog and fix Flickr URLs. The album arrangements, though, will vanish if I take this approach.

Anyway, the moral of the story is this: You need to look after your own data. Don’t assume that anyone else will do it for you, on a long-term basis, and especially if you’re not paying for the service! In particular, don’t assume that any URL is going to continue to work in the future unless it’s on a domain that you control and manage.

And lastly…

Remember that this will almost certainly also happen at some point to the pages you have on Facebook, the images you have on Instagram and the videos you have on YouTube. Don’t assume that a service will continue indefinitely because the company is large or because it has a model based on advertising revenue. I had stuff on Google Video too…

Update: Thanks to John for pointing me at Thomas Hawk’s post explaining that Flickr’s action is a good thing; yes, I agree overall!

Disk Risk

Mmm. I seem to have had a lot of hard drive failures recently – Seagate drives, mostly, though, to be fair, the majority of my drives are Seagate just because my favourite supplier happens to like them, so I would expect see more failures there. The last one, though, is just 18 months old and has started making ominous clicking noises. They don’t make ’em like they used to. Stuff I’ve read online tends to suggest that it’s hard to assign blame to particular drive manufacturers, but particular models do tend to have rather different failure rates.

I do, I realise, have rather a lot of hard disks. I have three 4-bay Drobo enclosures, for a start, so that’s 12 drives even before I start adding on the miscellaneous backup disks, TV-recording disks, etc. Not to mention the internal ones in all our various machines. There must be 20-25 hard disks around here, and even though manufacturers’ specs talk about a <1% annual failure rate, studies tend to suggest that real-world figures are rather higher. One of the biggest studies, done by Google a few years ago, showed failure rates of 1.7% in the first year, rising to over 8% in the third year.

Yes, many of my drives are about that age, so if I really have 25 of them, I guess I should expect one to die every six months or so. Bother.

This suggests to me that money spent on things like my Drobo enclosures is worthwhile, because, though they are pricey, especially once you’ve filled them up with drives, any single drive failure is unlikely to be catastrophic – as disks die, you just replace them with whatever size is currently in vogue. My main Drobo currently has two 2TB drives, one 1.5TB, and a 1TB. There are those, I know, who have had less positive experiences with some Drobo kit – I found a DroboShare networking add-on to be decidedly wobbly at a past company – but in the simple use case of a Drobo plugged into a computer, I’ve been very happy and have replaced several drives without ever losing data.

The other thing that the Google study found was a strong correlation between when disks start reporting errors (which they can do using the S.M.A.R.T technology built into modern drives) and a failure soon afterwards. It’s worth, therefore, having something that checks the S.M.A.R.T status and lets you know about issues as soon as they are reported, even if the drive is still apparently working OK. On the Mac, Disk Utility can tell you about issues, but only when you go and look, so I use SMARTreporter to give more regular checks.

OK, things are getting better. There is another issue, though.

On the Mac, at least, most external drives are connected by USB or Firewire, and in general S.M.A.R.T information is not read through those interfaces – if you look in Disk Utility, you’ll see it’s ‘Unavailable’. More sophisticated enclosures like the Drobo will check the S.M.A.R.T status themselves and warn you when things look dubious, but your average USB-connected backup drive may give you no such warnings.

So I was interested to discover this kernel driver project which enhances the standard OSX USB and FireWire drivers to make S.M.A.R.T available for a lot more interfaces. (Download v0.5 here). I’ll try it on my Media Mac Mini, which has three external drives, and see how it goes…

© Copyright Quentin Stafford-Fraser