Explainer: cameras versus lidar — the sensor argument that shaped an industry

Every autonomous system begins with the same problem: turning photons into a model of the world accurate enough to act on. Two families of sensor dominate the answer, and the argument between them defined a decade of autonomy development.

How each one sees

Lidar — light detection and ranging — is a measuring instrument. It fires pulses of laser light, times their return, and computes distance directly from the speed of light. Sweep that across a scene and you get a point cloud: hundreds of thousands of range measurements per second, each one a fact. A lidar knows the cyclist is 43.2 metres away because it measured the distance. It works identically at noon and midnight, and it cannot be fooled by a photograph of a road painted on a wall.

A camera is an inference engine. It captures a 2D grid of colour values, and everything else — depth, object identity, motion, intent — must be reconstructed by software, today almost always a neural network. Depth from cameras is estimated, from stereo geometry, from motion parallax, or from a network that has learned what far-away things look like. The estimate can be superb. It is still an estimate.

The trade-offs follow directly. Lidar gives you geometric certainty but semantic poverty: a point cloud shows a person-shaped object, not whether it's a pedestrian or a mannequin, and struggles with rain, fog, and dark, absorbent surfaces. Cameras give you rich semantics — text on signs, brake lights, a police officer's hand signal — at the cost of inferred geometry, and they degrade with the light. Radar, the quiet third party, sees velocity and through weather, but at coarse resolution.

The argument

The industry split into two camps. One holds that redundancy is the safety case: measure the world with lidar, radar and cameras, cross-check the modalities, and never depend on a single sensor's failure modes. Waymo and most robotaxi programmes are built this way. The other camp — Tesla, most vocally — argues that humans drive with two cameras on a swivel, that vision plus enough neural network is sufficient, and that lidar is an expensive crutch that lets you postpone solving the real problem, which is perception software.

For years, economics did the arguing. Early automotive lidar cost tens of thousands of dollars per unit, which made camera-only the only path to an affordable consumer vehicle. That argument has since collapsed: volume manufacturing, chiefly by Chinese suppliers such as Hesai and RoboSense, has pushed capable automotive lidar into the hundreds of dollars, and it now ships as standard on mid-priced Chinese EVs. When a sensor costs less than a headlamp assembly, refusing it is a philosophical position, not a financial one.

The quiet truce

Which is where the industry has landed: fusion. Every operating driverless service today runs multiple modalities and treats disagreement between them as a signal in itself — if the camera sees empty road and the lidar sees an obstacle, the vehicle believes the lidar and brakes. Meanwhile the camera side's core claim has aged well too: modern vision networks do most of the semantic heavy lifting in every stack, lidar included.

The same settlement, incidentally, was reached years earlier on the factory floor, where "machine vision" originally lived. Industrial systems mix 2D cameras for inspection and reading with 3D techniques — structured light, time-of-flight — whenever geometry matters, and nobody there ever considered it a culture war.

The questions that matter

For anyone evaluating an autonomous system, the sensor list on the datasheet is the least useful line. Ask instead: what happens when the modalities disagree, and which one wins? What is the detection range for a low-reflectivity object at the system's maximum operating speed? And how does performance degrade — gracefully or suddenly — in rain, fog, glare and darkness? How a system sees matters less than what it does when it can't.