Tag Archives: camera

Behind the Tesla ‘Full Self Driving’ system

If I were giving advice to somebody considering buying a Tesla at the moment, it would be (a) buy it and (b) don’t believe the ‘full self-driving’ hype… yet.

You’ll be getting a car that is great fun to drive, has amazing range, a splendid safety record, a brilliant charging network, etc… and, in the standard included ‘autopilot’, has a really good cruise control and lane-keeping facility. One thing I’ve noticed when comparing it to the smart cruise control on my previous car, for example, is that it’s much better at handling the situation where somebody overtakes and then pulls into the lane just in front of you. Systems that are primarily concerned with keeping your distance from the car in front have difficult decisions to make at that point: how much and how suddenly should they back off to maintain the preferred gap. The Tesla, in contrast, is constantly tracking all the vehicles around you, and has therefore been following that car and its speed relative to yours for some time, so can react much more smoothly.

The dubiously-named ‘Full Self-Driving’ package is an expensive optional extra which you can buy at the time of purchase or add on later with a couple of clicks in the app. At the moment, it doesn’t give you very much more: the extra functionality (especially outside the US) hasn’t been worth the money. If you purchase it now, you’re primarily buying into the promise of what it will offer in the future, and the hope that this will provide you with significant benefits in the time between now and when you sell the car!

But at sometime in the not-too-distant future, the new version –currently known as the ‘FSD Beta’ — will be released more widely to the general public. ‘Full Self Driving’ will then still be a misnomer, but will be quite a bit closer to the truth. YouTube is awash with videos of the FSD Beta doing some amazing things: people with a 45-minute California commute essentially being driven door-to-door, for example, while just resting their hands lightly on the steering wheel… and also with a few examples of it doing some pretty scary things. It seems clear, though, that it’s improving very fast, and will be genuinely valuable on highways, especially American highways, before too long, but also that it’s likely to be useless on the typical British country road or high street for a very long time!

What Tesla has, to a much greater degree than other companies, is the ability to gather data from its existing vehicles out on the road in order to improve the training of its neural nets. The more cars there are running the software, the better it should become. But the back-at-base process of training the machine learning models on vast amounts of video data (to produce the parameters which are then sent out to all the cars) is computationally very expensive, and the speed of an organisation’s innovation, and how fast it can distribute the results to the world, depends significantly on how fast it can do this.

Last week, Tesla held their ‘AI Day’, where Elon Musk got up on stage and, in his usual way, mumbled a few disjointed sentences. Did nobody ever tell the man that it’s worth actually preparing before you get up on a stage, especially the world stage?

However, between these slightly embarrassing moments are some amazing talks by the Tesla team, going into enormous detail about how they architect their neural nets, the challenges of the driving task, the incredible chips they are building and rolling out to build what may be the fastest ML-training installation in the world, and the systems they’re building around all this new stuff.

For most people, this will be too much technical detail and will make little sense. For those with a smattering of knowledge about machine learning, you can sit back and enjoy the ride. There are lots of pictures and video clips amidst the details! And for those with a deeper interest in AI/ML systems, I would say this is well-worth watching.

There are two key things that struck me during the talks.

First, as my friend Pilgrim pointed out, it’s amazing how open they’re being. Perhaps, he suggested, they can safely assume that the competition is so far behind that they’re not a threat!

Secondly, it suddenly occurred to me — half way through the discussions of petaflop-speed calculations — that I was watching a video from a motor manufacturer! An automobile company! If you’re considering buying a Tesla, this is a part of what you’re buying into, and it’s astonishingly different from anything you’d ever see from any other car-maker. Full self-driving is a very difficult problem. But this kind of thing goes a long way to convincing me that if anybody is going to get there, it will be Tesla.

You may or may not ever pay for the full FSD package, but it’s safe to assume much of the output of these endeavours will be incorporated into other parts of the system. So, at the very least, you should eventually get one hell of a cruise control!

The livestream is here, and the interesting stuff actually starts about 46 minutes in.

No, sorry, you can’t Zoom in that far

Having been a big fan of Zoom and extolled its virtues in the past, I thought it only fair to share a current criticism. (I’m talking about the videoconferencing app, of course. I’m an even bigger fan of the other Zoom and have relied on their products for years… definitely recommended!)

Anyway, back to video calls. I was playing recently with virtual cameras in OBS so I could do fun things like adding lower-thirds titles to my video stream…

or blending multiple video streams into one….

and my friend Nicholas commented that it was very clever, but any text was not actually that readable. At which point we delved into the Preferences > Statistics menu on the Zoom app and discovered that the video resolution was only 640×360; definitely lower than it used to be.

Now, this is perfectly fine for having a conversation with somebody, so for the vast majority of Zoom use, it’s not an issue. And if you turn on screen-sharing, your screen image is sent at a much higher resolution, so that’s fine too.

But it is an issue for some of my colleagues who like using pointing cameras at whiteboards or documents while giving remote lectures, or even if you’re just trying to hold something up to your camera for the person at the other end to read.

If you search online, you can find various references to ‘Enabling HD’, or to different resolutions being possible for Business or Education accounts, but as far as I can gather, these are all currently disabled or have no effect. I think Zoom may be restricting things to manage the load on their servers, which makes me wonder how much actually goes through their servers? At least for a 2-person call, like the one Nicholas and I were in, it really ought to be peer-to-peer. (Like Skype used to be in the early days before Microsoft bought and ruined it.) Still, to be fair, even the otherwise-abominable Teams does do a much better job at the moment when it comes to resolution.

Well, this may resolve itself in Zoom, but bizarrely, in the meantime, if you care about resolution of your camera more than you care about framerate or latency, the solution is probably to show it on your local display in high resolution, and then share your screen.

Progressing parallelograms

Progressing parallelograms

Pretty abstract for me, eh?

There’s an app called ‘Camera for iPad’ which allows your iPhone to be used as a remote camera for an iPad, which doesn’t have a camera of its own. Quite fun. It shows a ‘viewfinder’ on the iPad, so of course I pointed the camera at that.

So this is a view, taken on an iPhone, of a view on an iPad of what an iPhone is seeing when the iPhone camera is pointed at the iPad. The kitchen ceiling light is reflected in the iPad screen.

© Copyright Quentin Stafford-Fraser