Status-Q

I’ve just been through the interesting process of applying for (and failing to get) a TSB Proof of Concept grant for an idea I’m working on.

For those who haven’t tried this, it involves filling in a substantial form where each section has strident word limits – it always seems to have plenty of space for the things that would appear to me to be relatively unimportant at the ‘Proof of Concept’ stage, and not enough space for the things about which one can say quite a bit. It took me a couple of days to fill out the forms. None the less, it’s worth persevering, because the size of the grants on offer can be significant – up to £100K for the Proof of Concept, for example, though that would have to be matched with £67K from elsewhere.

I wasn’t surprised at not getting the grant – there were complaints recently that the board had spent most of the available funds in the first half of the year, meaning that only around 10% of applicants even got past the first stage if, like me, they were unlucky enough to apply after the autumn (as opposed to over 40% beforehand). But I was surprised to get what appeared to be useful feedback from the three assessors – something which had never happened in the days of the Regional Development Agencies.

So I decided to re-apply – you’re allowed a second chance – taking the feedback into account, and in particular addressing the one section where I’d had a very low score due to a slight misunderstanding of what was required. To most of the other sections I only made minimal alterations because the scores were pretty good, but I hope I improved them a bit based on the feedback. Another day of work, and then a six-week wait, before another rejection. Oh well – c’est la vie!

But I am interested in one aspect of this. There were only two assessors on my second attempt, and, almost without exception, the scores were substantially lower. If we leave out the question on which I had concentrated, where the score did go up by a small amount, on average my ‘improved’ version scored 1.6 points less on each section. That’s 1.6 out of 10, which is quite a variation, if you remember that the changes were pretty minimal and always in line with the feedback from the first attempt. But it’s also pretty consistent – on none of the other sections did my score go up, and on most of them it went down between 1.3 and 2.3 points.

All of which makes me think that the assessors for the second attempt may have had rather different scoring habits from the first ones, and I wonder if that is taken into account in the final reckoning of this closely-fought competition?

I make no claims about whether the higher or lower score is a better assessment of my own project’s worth, and my knowledge of statistics is too rusty to work out what reasonable error bars might be here: we’re talking, after all, about very small statistical samples. But it does suggest that applicants should take any scores and feedback with a big pinch of salt. This doesn’t mean that you shouldn’t re-apply, and in particular if you can spare the time for the paperwork you shouldn’t be put off if you get a low score – the lottery may go in your favour second time around…