Tag Archives: storage

The Fragile Free Cloud

Here’s a timely reminder, if one were needed, that you should never assume anything you store online is going to be there for very long, unless it’s on a system (a) that you are paying for and ideally (b) that you run or manage.

Flickr has announced that it’s going to start removing photos from its free accounts: everyone can still have 1,000 images, but that’s much less storage than they offered for free in the past. If you have more than that, they’ll start deleting the older ones first. I starting uploading things to Flickr about 13 or 14 years ago, so 90% of my 10,000 Flickr images will vanish over the next few months.

Most of the Snapchat/Instagram generation are probably not interested in anything that happened more than 1000 images ago! But people who have used Flickr for archiving the first pictures of their children or grandchildren may be in for a surprise. The name ‘Flickr’ might have a certain irony to it…

Now, this is a perfectly reasonable thing for the company to do, and there are several ways you can deal with it: you can start paying for your account, you can download your images if you don’t have local copies, or you can migrate them over to Smugmug (who now own Flickr). But only the first of those options will keep your photos nicely arranged in their albums, and, more importantly, will preserve your image URLs, so I imagine there will be a very large number of pages around the world with Flickr-shaped holes in them where an image used to be. Whichever option you choose, do it before the end of the year.

Now, I’ve been a fan of Flickr for a long time, and paid for an account for about a decade — it’s a good service and reasonably priced — but I switched to Smugmug a few years back because it was a better fit for my occasional bits of professional work. I don’t mind paying for one photo storage service, but I’d rather not pay for two, especially from the same company! So my photo archive has been copied to SmugMug, and I’ll probably need to write a bit of code to go through my blog and fix Flickr URLs. The album arrangements, though, will vanish if I take this approach.

Anyway, the moral of the story is this: You need to look after your own data. Don’t assume that anyone else will do it for you, on a long-term basis, and especially if you’re not paying for the service! In particular, don’t assume that any URL is going to continue to work in the future unless it’s on a domain that you control and manage.

And lastly…

Remember that this will almost certainly also happen at some point to the pages you have on Facebook, the images you have on Instagram and the videos you have on YouTube. Don’t assume that a service will continue indefinitely because the company is large or because it has a model based on advertising revenue. I had stuff on Google Video too…

Update: Thanks to John for pointing me at Thomas Hawk’s post explaining that Flickr’s action is a good thing; yes, I agree overall!

Drive to live?

hddOne of the very valuable things to come out of large data centres is large-scale reliability statistics. I’ve written before about my suspicions that my Seagate drives weren’t as reliable as they might be, but I had insufficient data for this to be anything other than anecdotal.

And then a couple of weeks ago, I pulled a couple of old 2.5″ drives off a shelf — Western Digital ones, I think — intending to reuse them for backups. They both span up, but neither would work beyond that.

So I was very interested by this Backblaze blog post which discusses their experience with a few thousand more drives than I have at my disposal. They use consumer-grade drives, and are very price-sensitive.

A quick summary:

blog-survival-drives-by-month

Some quotes:

Hitachi does really well. There is an initial die-off of Western Digital drives, and then they are nice and stable. The Seagate drives start strong, but die off at a consistently higher rate, with a burst of deaths near the 20-month mark.

Having said that, you’ll notice that even after 3 years, by far most of the drives are still operating.

Yes, but notice, too, that if you have four computers with Seagate drives, you should not expect the data on one of them to be there in three years’ time. And, quite possibly, not there by the Christmas after next.

The drives that just don’t work in our environment are Western Digital Green 3TB drives and Seagate LP (low power) 2TB drives. Both of these drives start accumulating errors as soon as they are put into production. We think this is related to vibration.

and

The good pricing on Seagate drives along with the consistent, but not great, performance is why we have a lot of them.

If the price were right, we would be buying nothing but Hitachi drives. They have been rock solid, and have had a remarkably low failure rate.

and

We are focusing on 4TB drives for new pods. For these, our current favorite is the Seagate Desktop HDD.15 (ST4000DM000). We’ll have to keep an eye on them, though.

Excellent stuff, and worth reading in more detail, especially if longevity is important to you. It’s tempting to fill old drives with data and put them on the shelf as archival backups, but this would suggest that you should only use new drives for that!

Oh, and if you’re wondering about which SSDs to buy, this report suggests that Intel ones are pretty good.

Update: Thanks to Dominic Plunkett for the Backblaze link, and for Rip Sohan for a link in the comments to the TweakTown article that attempts (with some, but not a great deal, of success) to debunk some of this. The previous article I mentioned above links to an older Google study which didn’t distinguish between manufacturers and models, but did say that there was a correlation between them and the failure rates. It also catalogued failure rates not too dissimilar to the Backblaze ones after 3 or so years, so the general implication for home archiving remains!

© Copyright Quentin Stafford-Fraser