Status-Q

A few years ago, the ‘University Information Services’ (UIS) team at Cambridge made the decision that the University would no longer run its own email service — something it had done since email began — and instead was going to outsource it to a commercial entity.

This in itself, I think, was probably a mistake, because email is enormously important, not just as a communication method, but as an archive. Many people make this mistake. “We use Slack for our internal chat, we use Facebook and Twitter and WhatsApp and Zoom for our external communications. We hear some old people still use email, but why is that any different?”

Email, though, for most people, is not just an instant-messaging system. It’s a store, a journal, a database, a record, of decisions made, invoices received, relationships formed, ideas generated, news exchanged, and so on. I have most of my email for the last thirty years or so, and find it incredibly valuable. Last year, I wrote a post about why I think it’s one of our most successful and important data archive formats, and its success for so long, when compared with almost anything else, has depended substantially on open standards and Open Source software.

Running an email server is easy. Doing it well, however, is not, and requires expertise and dedicated staff. In the past, I have run my own personal server, and we also did this in one of my startup companies. (In both cases these were based on the excellent, widely-used and highly-regarded Exim software which was, ironically, developed here at Cambridge in that same UIS department!) But these days there are lots of good hosting services, and it doesn’t normally make sense for most organisations to run their own.

The University of Cambridge is not, however, a small organisation, and surely it should really be willing to employ the skilled staff and provide the resources to run its own email system, in the same way that it runs its own libraries. (Maybe they’ll be next?)

However, out-sourcing is the name of the game these days, and it’s not that that is causing me, and several of my colleagues, concerns.

Exchanging open for closed

No, it’s the fact that that the University decided to opt for Microsoft Exchange Online. A couple of days ago, most staff who had not yet transitioned were moved from our own servers to Microsoft’s, and the forced migration will be complete by the end of next week, and the old servers switched off at the end of the year. For most people, in the short term, this will just be a minor inconvenience. So why does this worry me? Because I take a longer view.

Since email was invented, all of the email of many tens of thousands of University staff and students has been stored in formats defined by open standards, on machines that we control, generally by open email software running on open operating systems which were storing it on open filesystems… and so on. We had full control, and we always knew we could manipulate, improve, archive, extract and back up any part of this system which stores such a valuable archive of data for the entire community.

This week, however, saw the transition of that entire archive to a system running proprietary software, stored in proprietary formats on proprietary operating systems and filing systems, on servers that we no longer control. And the most important question, of course, when putting any important data into any new system is “How easily can I get it out again, in a usable format for the future, if I change my mind?”

Now, normally, one of the many wonderful things about an IMAP-based email archive is that this is trivial: you can decide to move it from one hosting service to another, or just shift it to your own machine, simply by dragging and dropping it using the email program of your choice. I have done this many times over the decades, as I’ve moved between jobs, email providers, academic institutions, and also as my email-reading devices have switched between different operating systems and different email apps.

Exchange, however, has always had problems supporting IMAP access to email reliably. Experts disagree on how serious those problems currently are, but the fact that the debate continues is worrying. It’s a lower priority for Microsoft, because they’d rather you used their own protocols and their own software.

The slippery slope

The UIS’s own pages telling Mac users how to connect to the new email service, for example, start with this dire warning:

Note that this is referring to Apple Mail, one of the most-used email clients in the world, and there’s a similar warning for those accessing their email and calendars from iOS devices.

What hope is there for those of us who prefer less-common email clients, such as my long-term favourite, Mailmate? Or who use less-mainstream operating systems? Or who have older hardware that won’t run the latest software?

This rings alarm bells for those of us who remember Microsoft’s (thankfully failed) attempts in the past to control another open system — the Web — by getting people to run Microsoft software both on the desktop and the server. When you control both ends of a system, you can freely modify the protocols used to communicate between them until it becomes very hard for users to adopt any other system. Microsoft used predatory and sometimes illegal business practices in the past to try and kill off Netscape and other desktop competition in pursuit of this goal, but the overall strategy failed, partly because their own server-side software was so inferior to the Open Source offerings that the techies responsible for most web servers refused to depend upon it.

I hope we are not seeing a repeat here — and to be frank, I doubt it — but I do wonder whether this shift to Exchange will prove to be a one-way-only transition. Suppose Microsoft were to say, in a few years’ time, that only a very small proportion of Exchange users now depended on IMAP, and they would be switching it off shortly. What options would the University have?

And what choice would users then have about the software they ran on their desktop to access this valuable archive?

Could we rely on getting thousands of email archives out and into a usable form elsewhere? What about any enhanced metadata? Attachments? Calendars? Contacts? There are open standards for all these things, but what incentives would Microsoft have to ease that transition, if it were even viable?

No, I fear that this contract, once made, has been made for ever.

And some of the implications are starting to be felt elsewhere. It recently came to light, for example, that the contract with Microsoft covered the staff and students of the University, but we have traditionally continued to provide email cam.ac.uk addresses to retired academics, emeritus professors, others helping with the activities of the University. “Ah”, said Microsoft — after the deal was signed, as I understand it, but perhaps before our chaps had cottoned on to all the implications — “You do realise that they’re not included in this deal?”.

This has made a lot of people very upset, because they had been given to understand that the email address that they’d always had, often the only one they’d ever had, that had been published at the top of their academic papers for decades, and had been used, for example, to set up all of their other online accounts, would remain a valid way of contacting them indefinitely. Not any more. This, of course, is standard practice in the business world: you leave a company and your email address is gone. The company closes, or is bought, and your email address is gone. But ancient universities haven’t always been run on those purely corporate lines or with those same expectations of transience.

Where does this leave us?

Essentially, therefore, there has been only one choice for those of us who wish to keep our email in standard formats that we know will be accessible in the long term. Reliable archiving is not a service we can now expect our eight-century-old University to provide, so we take all incoming email to our University addresses, and forward it to our own accounts held elsewhere.

Many of my colleagues in the Computer Science department are now doing this. We have to hope that this will continue to be allowed, both by the University and, of course, by Microsoft. We can’t ignore the Microsoft servers completely, because they will probably soon also be the only way to send email that will be recognised by the outside world as coming from a valid cam.ac.uk address. And being able to receive email under such an alias can be important for accessing academic journals, university service etc.

But some friends have decided that a better option is to start using their own personal email addresses on academic projects, papers, websites etc. to ensure that they have control in the future. (If you’re doing this, make sure you use your own DNS domain or a long-term reliable forwarding service, so you aren’t indefinitely tied to your current ISP or your ancient Hotmail account for your future professional communications!)

me.com

And perhaps some of this is inevitable. I used to rely on my employer to provide me with a telephone and a telephone number, for example, but the one sitting on my University desk for the last several years has almost never been used.

For many of us, it’s important that we control our own ‘brand’. Our own phone number, our own website, our own LinkedIn pages are more important to us than how we appear on any particular employer’s website, and as a result we need to pay for those out of our own pockets. Email should perhaps be the same, and wise academics should probably ensure that they control their own destiny and don’t use their current university email address on their publications.

But it all gradually erodes the sense of belonging to an ancient institution with which you might have a lifelong relationship. And that isn’t good for the institution either, especially when it comes to fund-raising. Ask any development office.