Tag Archives: software

Location, location, revisited

Previously on Status-Q…

Regular readers may remember that a couple of months ago, I lost my glasses and found them again. If you missed that particularly gripping episode, turn back to Location, location, location (or, ‘How technology saved me a few hundred quid yesterday’) and then you’ll understand the background here.

After posting it, my friend Phil Endecott got in touch with me. “Am I to understand”, he said, “that you would find it useful if you had a map app that could show both the locations of your photos and your current location at the same time? If so, I may have just what you need….”

And he did indeed. Phil, you see, is the author of UK Map, an iOS app that I’ve had for as long as I can remember, and one I should talk about more, because I use it all the time, especially when looking for new dog-walking routes. Yes, I may use Google Maps to find out how long it’ll take me to drive there, and Streetview to check that there’s likely to be a parking spot when I get there, but once I’ve laced up my walking boots, then I generally switch to UK Map. It combines free or paid-for Ordnance Survey maps with footpaths from Open Street Map, and, certainly round here, it’s a much better guide than almost anything else as to where you can actually go for a walk.

New features get added periodically and, like much of the cool stuff, are often buried deep in some menu below some unassuming icon in the corner, making them very easy to miss, so you really do want to go to the home page and read it carefully to see what the app can do, and then to the help button in the app to see how to do it. I don’t check these often enough, but when Phil’s message prompted me, I had another explore and found that, yes, it can show your photos on your maps. This is the site of my aforementioned adventure:

and if I had been there at the time of writing this, the little blue dot would indeed have shown me exactly what I needed to know. (The violet-coloured triangle there, by the way, is showing the rough direction in which the phone was pointing when it took the picture, and therefore shows what you may be able to see in it. Neat, eh?)

Anyway, I’ve mentioned UK Map before, but I’ve used it for so many years that I take it for granted. I do think that if you put in a little time learning what it can do, you’ll find it repays its very modest purchase price. Actually, it’ll repay that even if, like me, you really only scratch the surface.

P.S. If you happen to be anywhere other than the UK, this will be of limited use, but if you’re in North America, Phil is also the author of the highly-rated Topo Maps

When not to Excel

Back in about 1993, I was doing the bookkeeping for a big project being undertaken by my local church. Donations were flooding in, and we needed to keep track of everything, send out receipts, forms and letters of thanks, and note whether each donation was eligible for the UK tax relief known as ‘Gift Aid’.

I was keeping track of this using on a PC running the now long-gone Microsoft Works, which, for those less familiar with last-millennium computing, was a software suite incorporating basic and much cheaper versions of the things you found in Microsoft Office: a word-processor, database, and spreadsheet. If your needs were simple, it worked rather well.

Anyway, at one point I was printing out a list of recent donations on my dot-matrix printer, and I noticed what appeared to be some data corruption. In the midst of the sea of donors’ names, there were a couple of dates being printed out. Was this a software bug, or was my database file corrupt? I started investigating, while wondering just how much data I’d entered since my last backup and whether I could recreate it…

In the end, the answer was simple, and I breathed a huge sigh of relief. I tried recreating the entry for one of donors and the same thing happened again, at which point I dug a bit deeper and discovered a ‘feature’ of the app. The lady’s first name was ‘June’, and, though it displayed just fine on the data-entry screen, behind the scenes, it had been turned into 1/6/93! I skimmed quickly through the congregation to find the other problematic record, and found it was for a donation from a lady named ‘May’!

When I came back to doing some scientific computing in academia a few years ago, I was surprised and slightly worried to see several of my colleagues processing their data with Excel. It’s a wonderful program and very appealing, because of the ease of viewing, checking and plotting graphs of your results, but it comes with lots of problems of its own and shouldn’t be used as a substitute for a proper database (if you’re a church accountant), or for something like Jupyter Notebooks, if you’re a scientific researcher, unless you’re exceedingly careful. Last year, more than a quarter of a century after my issues with May and June, 27 human genes were actually renamed because of the number of errors caused in scientific papers by the use of Excel by researchers. The genes’ previous names were things like ‘SEPT1’.

All of this came to mind when reading Tim Harford’s enjoyable piece in yesterday’s FT, The Tyranny of Spreadsheets. Harford follows the origins of spreadsheets, double-entry bookkeeping and other ways of keeping track of things, up to the famous case last year where 16,000 people weren’t promptly told they had positive Covid test results, because somebody had used Excel’s old XLS file format, which can only store about 64,000 rows of information, instead of the newer XLSX. That’s not really a problem; the problem is that Excel doesn’t give you adequate warning when it’s discarding data, or changing it in an attempt to be helpful. And the results can be serious.

To quote the article:

Two economists, Thiemo Fetzer and Thomas Graeber … decided that no catastrophe should be allowed to occur without trying to learn some lessons. They combed through the evidence from Public Health England’s mishap. And by comparing the experiences of different regions, they concluded that the error had led to 125,000 additional infections.

Fetzer and Graeber have calculated a conservative estimate of the number of people who died, unknown victims of the spreadsheet error. They think the death toll is at least 1,500 people.

Think about that, as you click ‘Save As…’ and pick your data format.

(Thanks to my sister-in-law Lindsey for the link to the Harford article, which also traces some of the origins of spreadsheets from the 14th century; that’s before even I was using them!)

Getting Things Done Together

Like many people of approximately my generation, I have long been an advocate of David Allen’s famous ‘Getting Things Done‘ methodology. In case you’ve been living in a cave for a couple of decades and haven’t come across ‘GTD’, it was an appealing and genuinely useful system for handling, originally, the ever-growing flood of paperwork that people were experiencing towards the end of the last millennium. Allen was fortunate (or canny) in that most of his book, originally published in 2001, translated very nicely into the emerging predominantly-digital world. He did for filing cabinets and in-trays what Marie Kondo now does for wardrobes and sock drawers.

Over the following couple of decades, many software products sprang up to help you adapt the GTD task-management techniques to your new digital world; the most complete and sophisticated probably being OmniFocus. If you juggle lots of big and complex projects and are really into this stuff, OmniFocus is immensely capable, and I started using it as soon as it was first available as a beta release about 13 years ago. I typically have two or three part-time jobs at any one time, and many projects within each one, and something like OmniFocus can definitely help keep your world manageable especially if, like me, your brain is not one that is naturally drawn to rigorous and careful planning and organisation! In recent months, though, I’ve switched to the rather wonderful ‘Things‘, which is for me the perfect half-way point between a simple to-do list and the all-encompassing and baroque structures I had previously created within OmniFocus.

Anyway, all of this meant that I was very interested when John linked to this New Yorker piece by Cal Newport, talking about the history of GTD and some of its limitations in the current climate. It’s a good read; here are a couple of short extracts:

In this context, the shortcomings of personal-productivity systems like G.T.D. become clear. They don’t directly address the fundamental problem: the insidiously haphazard way that work unfolds at the organizational level. They only help individuals cope with its effects. A highly optimized implementation of G.T.D. might have helped Mann organize the hundreds of tasks that arrived haphazardly in his in-box daily, but it could do nothing to reduce the quantity of these requests.

It seems likely that any successful effort to reform professional life must start by making it easier to figure out who is working on what, and how it’s going. Because so much of our effort in the office now unfolds in rapid exchanges of digital messages, it’s convenient to allow our in-boxes to become an informal repository for everything we need to get done. This strategy, however, obscures many of the worst aspects of overload culture. When I don’t know how much is currently on your plate, it’s easy for me to add one more thing. When I cannot see what my team is up to, I can allow accidental inequities to arise, in which the willing end up overloaded and the unwilling remain happily unbothered.

In a distributed working-from-home world, he argues, techniques like Kanban boards — or the electronic versions encapsulated in products like Trello — can be more appropriate ways to manage tasks when your workforce is distributed. You need to make your to-do list more public, so others in your organisation, and particularly those responsible for managing you, can see what you’re working on and whether you have too much (or too little) on your plate. Software developers have been doing this for years, of course, but it’s interesting to think about how many other kinds of work might benefit from some of our techniques in the Covid age.

My favourite example of a public to-do list, though, probably just predated the publication of GTD, and was not digital, even though I was working in a cutting-edge high-technology lab at the time.

The sysadmins — about four of them, I think — all worked in a shared office, and people like me would wander in and ask them to do things which, of course, always needed to be done immediately if they possibly could. So at one point, they adopted a brilliant scheme.

On the wall outside their office, they put up a big board showing the queue of things that people had asked them to do. If you came in with a request, they’d say ‘Sure! We can do that!’, and hand you a little card. You’d write your name and your request on the card and go and pin it on the board at the bottom of the queue. When one of them finished dealing with the current issue, they’d go and take the next card off the board. If you felt that your card was more important than some of the other ones there, you could try to make the case that the queue should be re-ordered. But if the queue was really long, you sometimes discovered that actually you didn’t need this as much as you thought you did, or perhaps you could sort it out yourself.

It was, of course, just a ticketing system, but run very much on a first-come, first-served basis and, since the tickets were all public, it was one which gave everybody, not just management, a very clear idea of what was going on. I’ve always thought it was brilliant, and something which should be replicated more often in the digital world.

Why do you have a ‘Knowledge Base’ on your website?

Question 1. Which of the following are indicated by having a ‘Knowledge Base’ on your company’s ‘support’ pages?

  • (a) A lack of the discipline needed to write proper documentation.

  • (b) Insufficient staff to provide proper support.

  • (c) Inadequate engineers to provide a reliable product.

  • (d) A poor user-interface experience in your product.

  • (e) All of the above.


The best thing about creating a website in blog format, I’ve always thought, is that you don’t need to maintain it to nearly the same degree as you would most other sites. Why? Because everything has a date, and that date is paramount; it is clearly marked. This makes it obvious to the reader that a post was written in a particular context, and, while the content may still be useful some years later, you can’t blame the author for not being able to predict the future if it’s no longer accurate or relevant now! As an author, therefore, you don’t need to be constantly re-reading, deleting, updating, clarifying, as you would with most other forms of documentation.

The appeal of the so-called ‘Knowledge Base’ — a searchable database of support articles on a company website — is that it also promises an alternative solution to this challenge of maintaining structured documentation, but instead of using old Father Time to winnow out inaccurate material, it uses a search engine to help you ignore it.

If you force your customers to type things into the Knowledge Base search box before you let them get anywhere near the page with your phone numbers or email addresses, there’s even a chance they may find out the answer to their question and not bother you! Bliss!

What’s the problem?

Search engines are wonderful things, of course, and this isn’t a bad idea in theory, but if your experience is anything like mine, these knowledge bases are chiefly a source of frustration for customers.

Why is this?

  1. Users don’t know what to search for. They may not know the phrases you use internally to categorise certain issues. A human support person would know that when you said you had an issue with your mouse, you might be referring to a problem labelled ‘trackpad’ or ‘cursor’ in the docs.

  2. The articles aren’t well-indexed. Do you, for example, make sure that the error message the user actually sees appears in plain text in the articles, so the search engine has a chance of finding it? Do your articles have extensive tagging with relevant keywords which may not be in the text? If the error is slightly different because they used a different filename, will it still be found?

  3. The search engines aren’t good enough. We’ve been spoiled by Google. Does your search engine understand synonyms? Does it rank pages well? Does it show a good summary so your users don’t have to read all 26 articles returned by their query?

  4. The content is outdated. If the user actually succeeds in finding a relevant article, is it clear that it only refers to version 4.2 of the product running on Android, and not version 5.0 running on iOS, because the latter didn’t exist at the time it was written? Will they know that the solution it proposes can’t possibly work for them? Because this advice is buried in a database, it may be harder for your support staff to know this is a problem.

How to fix it!

I have four pieces of advice for anyone who has, or is considering implementing, a Knowledge Base on their website:

  1. Make sure the full text of your knowledge base is searchable by Google. They will do a better job than you do.

  2. Allocate more, not less, in the way of resources, if you want your documentation in this form. Knowledge Bases are not like blogs. You can’t dump things in there and forget them. Articles need to be reviewed, re-written, re-indexed, re-keyworded and regularly purged if the contents are to remain useful. You need to do this in addition to writing the actually text itself. Customers will not thank you for anything which suggests you believe that their time is much less important than yours. If they have to search through dozens of articles to try and find an answer, it probably indicates you aren’t doing your job. There may be good reasons for using this format for your documentation. Cheapness isn’t one of them.

  3. If you insist on putting your users through this and they still contact you, you should provide better support once they do. For heaven’s sake, don’t just connect them to somebody in a remote location who only has access to the same information! Don’t offer a chat link to somebody who is ‘Very sorry to hear about your problem’ but knows less about it than they do. Connect them to the engineers who produced the flawed product, or the writers who produced the inadequate documentation.

  4. This is the key one: The bigger your Knowledge Base, the worse job you are doing. In most situations, an entry in your Knowledge Base indicates a failure in communication elsewhere. Insufficient documentation, unhelpful errors, unreliable products. Your users won’t generally be consulting a Knowledge Base if everything is going well for them. Treat it as an issue-ticketing system, and reward those whose work means that articles can be removed from it. And make sure you have the processes in place that this actually happens!

Geek wisdom of the day

Some people, when confronted with a problem, think ‘I know, I’ll use multithreading’.
Nothhw tpe yawrve o oblems.

Eiríkr Åsheim

The Modern Lab Notebook

I’ve just uploaded my longest YouTube video yet!

Entitled The Modern Lab Notebook: Scientific computing with Jupyter and Python, it’s a two-and-a-quarter hour blockbuster! But you can think of it as three or four tutorial seminars rolled into one: no need to watch it in one sitting, and no need to watch it all! It starts with the basics, and builds up from there.

It’s intended for people who have some Python programming experience, but know little about the libraries that have become so popular recently in numerical analysis and data science. Or for people who may even have used them — pasted some code into a Jupyter notebook as part of a college exercise, say — but not really understood what was going on behind the scenes.

This is for you. I hope you find it useful!

Watch it full-screen, and turn the resolution up 🙂

Also available on Vimeo.

Breaking good

breaktimeI keep hearing about research that shows how your life will be dramatically shorter and more problematic if you spend too much of it sitting in front of a computer.

Some of this, no doubt, is encouraged by the manufacturers of the standing desks, and even treadmill desks, which are to the young entrepreneurs of today what the Aeron chair was to the dot-com startups of yesteryear.

But whether or not you believe the more worrying claims of reduced life expectancy, I think we can agree that it’s not a bad idea to get up and stretch your legs from time to time. Maybe have a bottle of chilled water, if you’re from California, or a nice cup of tea, if you’re British.

So I’ve been rather taken with a little Mac app called BreakTime, which will pop up and nag you when you’ve been working at your computer for too long at a stretch. You can choose the time periods: mine requires me to have a four-minute break after 56 minutes, for example, and you have some control over how persistent it will be: are you allowed to dismiss it before the four minutes are up? It also makes sensible decisions if you leave the machine of your own accord first, and resets the timer when you return.

I find, to my surprise, that I really like it: I’ve put it on all my machines, and what it highlights is just how difficult it is to keep track of time myself. I’m amazed how quickly an hour of sitting still can fly by when I’m deep in concentration. Even if I do little more than stand up and tidy some things off my desk, I’m sure it’s a good discipline.

There are several other similar utilities out there, but BreakTime works well for me. Recommended.

Update: Tim Green, on Facebook, pointed out Workrave, which does something similar for Windows and Linux. I’m linking to it here because, of course, you can’t search Facebook – even your own history (something I still find incomprehensible).

The Subscription Dilemma

money

Ten years ago, I wrote a piece for the IEE Review entitled “If You Love Your Data, Set it Free”, where I warned that Microsoft and other similar companies were experimenting with a subscription-based model of software.

This is a perfectly reasonable way of running the IT economy, but it has an important implication. If your data is stored in a proprietary format tied to one software package, as much of it probably is today, you may not have access to it if you don’t keep paying. Do you want to finish working on that book you started a few years ago? Sorry, that will cost you. In such a world, it’s worth asking yourself who actually owns your creative work…

Well, it’s taken a while, but Microsoft and Adobe are now actively pushing the subscription-based ‘Office 365’ and ‘Creative Cloud’ respectively. If you go to their web sites, it’s getting harder and harder to find a traditional buy-and-install product.

Software prices have been dropping dramatically recently, and it must be hard to persuade people who are used to paying under a fiver for the latest iPad app that it’s worth dropping hundreds on the latest Office or Creative Suite, however good those may be. This is particularly true if they already have an older copy. I’ve never felt a desire to upgrade my Office 2008 or Photoshop CS3, but I don’t use them very often. However, my wife, who uses Word all day, every day, also has no reason to upgrade, and in fact would probably view it as a retrograde step. So they had little choice. When you can’t innovate enough in your product, you have to innovate in your marketing.

Now, the subscriptions are not extravagant (at least compared to these companies’ traditional prices). If I used the software on a regular basis I wouldn’t mind paying. The problem is that you’re not just paying for upgrades, you’re paying for continued use. If you stop paying, you don’t, as in the past, continue happily using your current version. You get dramatically reduced functionality, in the case of Microsoft, or none at all, in the case of Adobe. So this is not a decision to pay for ongoing updates, it’s a commitment to continue paying indefinitely unless you want to go through the process of exporting all of your documents to some other format. The issue is particularly acute since these are apps into which you are likely to pour a large amount of your creative output, something you’re unlikely to want to discard. If you want to keep upgrading your software to the latest version, the pricing isn’t bad. But what you’re losing is any option about whether or not to keep upgrading.

So, on the one hand, this spurs me on to even greater enthusiasm for open file formats. And on the other, it makes me wonder about upgrading my copy of Office. Why? Well, it looks as if I won’t have the option very much longer of buying Office 2011, which, though already two years old, may be the last version for which I only have to pay once…

Inventing on Principle

This is an interesting and unusual talk, given about a year ago at a Canadian software engineering conference. I’d seen it before, but a friend reminded me of it recently (thanks, Aideen!) so I’ve just watched it again.

Bret Victor starts by talking about new ways to design software, and finishes with some suggestions on how to live your life. This is dangerous, because you may only find him credible on one of these points, and one could perhaps argue that the one-hour talk would be better delivered as two half-hour talks. And the first couple of minutes, delivered in his slow, careful style in a badly-lit brown room, don’t jump out and grab you. However, I think he pulls it off, and it certainly has the merit of being very different from your typical software-engineering talk.

Recommended.

© Copyright Quentin Stafford-Fraser