Autonomous self-driving cars keep inching ever closer to reality. More companies are getting into the self-driving game, but one big problem remains: Self-driving cars have a really difficult time identifying people on bikes. And they are even worse at predicting where those cyclists are headed.

Autonomous cars & their issues identifying cyclists

courtesy of Waymo

A couple of new reports out since the start of 2018 have summarized the huge progress that self-driving car technologies have made in the last year or two. On one side industry group IEEE has credited progress overall, but stagnation regarding cyclists. And then there is Waymo – the evolution of Google’s self-driving car project – that released a safety report showing the strides they’ve made, specifically calling out cyclist detection.

These autonomous vehicles are becoming smarter and much more accurate at identifying, tracking, and even predicting the movement of objects in their field of view. The problem remains, though, that progress doesn’t appear to be being made with regards to cyclists.

courtesy Uber

We reported on the same issue a little over a year ago when Uber admitted their own self-driving cars were having trouble identifying bike lanes. That made it difficult, both to keep the cars out of the lanes and to recognize that they should be carefully looking there for cyclists.

Cyclist detecting technologies

courtesy of Waymo

And a lot of that comes down to the most common technology that self-driving cars utilize – namely analyzing a constant inflow of two dimensional images to identify objects and their direction of travel. Many vehicles combine that optical sensing with additional tech like lidar & radar which help identify locations of objects. But those sensing systems are less effective at determining their direction of travel, limiting their effectiveness on their own.

Essentially the autonomous car sees the world as a succession of images, and software automatically models boxes around things like cars (green & purple), pedestrians (yellow), and cyclists (red) over the vehicle’s 3D basemap data.

Then it anticipates what happens next, so it can decide how to proceed driving.

Cyclists are a difficult thing to identify and track. We are less massive than other vehicles on the road. Plus, we are able to both move speedily with traffic and react quickly, changing direction faster than almost anything else in the roadway.

But maybe the biggest obstacle for the automated machine analysis made by self-driving cars is that cyclists are so individual. Whether we are roadies in skin tight lycra, mountain bikers in baggies, or a commuter in regular street clothes, we all look very different, even though we move in similar ways. While autonomous cars have effectively learned how to identify mostly standard looking cars by studying huge numbers of images, those images haven’t had a lot of cyclists in them, so there is much more learning to be had.

How can self-driving cars and cyclists co-exist?

Volvo Leads the Way to Safety, with First Cyclist Detection Braking System
courtesy Volvo

Some cycling & car industry minds have been developing technologies, using reflective paint, specially identifiable helmets, a smartphone app, and even bikes equipped with technology that communicates directly with the self-driving cars. That seems like a band-aid over the problem to us (and many others). It certainly could help those equipped with the special tech, but would ultimately leave anyone without it more vulnerable.

For now it seems like the best solution is more machine learning. The programmers behind the development of these self-driving cars need to get more images of cyclists into the robot brains of their cars, so they can make the right decisions.

The same problems exist with regular drivers who don’t always react properly to seeing cyclists on the road. (Not helped by some cyclists not obeying the rules of the road to start with.) Much like we need to use cycling advocacy to teach human drivers how to share the road, it seems we need to do the same with self-driving cars. And as more autonomous vehicles encounter more cyclists out on the road, they can better learn to identify us and operate safely around us.

23 COMMENTS

  1. They should look into re-creating a full-body skeleton in a similar way video game technology does with Kinect and gesture prediction. Because just analyzing 2D images and radar data seems like a limited approach to predict the path of a fast moving agile object. Having an 3D representation and an understanding of how the cyclist can turn would help. That’s how the human brain seems to work at least. If you see the bike lean right, but the cyclist still has his body weight centered, you know it’s possible he can still flick left at any moment. But not if his body weight also leans with the bike. Good luck predicting that with a box representation.

  2. It’s still relatively early in autonomous car development. I’m confident this problem will be resolved. I don’t see the use of a signal broadcast by a device on a bike as a bandaid. Instead, it’s making use of technology that’s available. It’s not a bandaid when military aircraft use IFF (Identification friend or foe) to identify aircraft. It’s not bandaid when aircraft use transponders. It shouldn’t be difficult for an industry to develop a low cost transponder for bicycles and integrate a receiver that transponder in cars. Developing a common spec would allow bike manufacturers to directly integrate that transponder into bikes and offer transponders for older bikes.

    • No, that’s not a solution. If your transponder stops working you won’t have air traffic control to warm you about it, you’ll just get run over.

    • I too am confident that the problem will be resolved. However, the use of a broadcast device (active) or specific reflective type device (inactive) is a bandaid solution and a poor one at that. The instances where these are used are in well controlled environments with multiple back-up solutions.

      Case in point. A kid grabs a bike that hasn’t been retrofitted, or the transponder malfunctions without their knowledge. They are now extremely likely to be hit.

      Self driving cars should navigate and avoid objects as a human does…that is by observing surroundings and adjusting correctly to suit. With the fact that they can use sight modes i can’t (thermal, radar) and should never be tired, intoxicated, distracted, impatient, angry, poorly trained, etc, we should eventually see a truly safe vehicle.

      • I’d prefer a multi-mode detection scheme, given the apparent difficulty in detecting cyclists. Active detection (with some EM radiation frequency or possibly with multiple frequencies) coupled with passive detection (the transponders) would make for a robust system.

        • I think you underestimate how robust a system can be without relying on these devices. The “apparent difficulty” is only now, at current tech. 20 years ago, nobody could fathom smartphones. Proper detection will improve.

          The main problem is that these devices cannot inherently provide failure detection/alert to the vehicle that relies on them to not hit said object. Imagine one failing on our commute or a kids failing on their ride to school.

          All control should be inherent to the vehicle where it can detect failure, and employ redundancy as required (or safely shutdown as needed). It should not rely on devices it has no control over and no way to run diagnostics on.

          • So you’re in favor of single failure. Interesting.

            A transponder could in fact allow a vehicle to take corrective measures. It makes little difference whether a sensor is detecting a beam on a round trip flight or a bean on a one way flight (except for time of flight sensors, diffractive or interferometric sensors). A vehicle sensor can absolutely act on that incoming transponder beam, especially given that over a few micro seconds in can calculate the velocity of that transponder.

            I also believe that it will part of the natural progression of autonomous vehicles that said vehicles will in some way communication with each other, even if that communication is just a transponder signal. I don’t want to see bicycles left out of that loop.

            • Single failure? What part of redundant sensing systems on the car with fault detection logic and planned compensation/action don’t you understand?

                • a system designed/tested to be effective based on equipment mounted to a bike will fail to be effective if that bike mounted equipment fails or is not installed and there is no way for the vehicle to detect said failure or lack of equipment.

  3. ill just note, living in SF and commuting on bike daily that some automated cars see me. theres a gazillion prototypes around here. some are super dangerous and some are quite good.

    remember also that most cyclists basically throw themselves into cars fully expecting the car will stop (ive seen quite a few direct hits in the last 5-6 years here on my daily commute.. which is actually quite low considering that 99% of the cyclists i see blow through lights without even a look)

  4. “The same problems exist with regular drivers who don’t always react properly to seeing cyclists on the road.”
    Automate the flaw! Efficiency gained!

    • Human drivers have actually developed a pretty brilliant workaround that could be easily translated to machine language and incorporated into the decision loop. Train the machine to hate cyclists and assure it that all it has to do is blame the cyclist and it’ll get off the hook every time.

  5. why don’t we just require people to actually have responsibility and learn to drive a car? all these safety systems do is make people dumber and then when the system fails (they always will at some %) the human is too stupid to know what is going on or how to correct it. look at back up cameras/sensors. they are great 99% of the time but people no longer actually look and simply rely on the system to save them. when it fails they run over their 1 year old. That actually happened in Minneapolis.

  6. “Some cyclists not obeying the rules of the road to begin with”. That describes about 70% of the cyclists around here. Anyone not in technical cycling clothing (or at least wearing a helmet) is more likely to be cycling on the sidewalks (note, NOT “shared use/mixed use paths”), or cycling on the road AGAINST traffic, wearing dark clothing (and no lights or reflectors) at night… It’s hard enough sometimes for another CYCLIST to avoid these road hazards. (Add to the mix, pedestrians walking in streets without sidewalks, or whose sidewalks have been rendered impassible by snow plows dumping their loads onto them to keep the ROADS clear.)

  7. It seems yet again that the ‘blame the victim’ brigade are out in force.
    If these so-called autonomous systems cannot ‘see’ cyclists then they probably cannot ‘see’ people either. People should always have priority (whether on foot on cycles) over motor vehicles in civilised societies and should not be expected to have to carry whatever totem the motorists’ lobby is currently advocating, whether that be hi-viz, reflectives, or, heaven forbid, some kind of transponder, in order to prevent themselves being struck by an out-of-control motor vehicle!
    If these devices cannot negotiate expected hazards on roads and streets then they should be banned until they can be proven to be able to do so.

    • I agree. Much of the discussion here revolves around somehow modifying bikes or cyclists. While cyclists do have responsibilities as road users, their behavior is not relevant to the discussion about the ability or inability of self-driving cars to detect bicycles. The onus is on the companies introducing the new technology to make it safe for the roads, rather than vice versa.

What do you think?