I hope after tonight's announcement we will be able to have an equivalent diagram for Tesla to the Dolgov one you have here. Not something I know much about but I hear that it will be based on a sort of LLM-like "Large [Tesla driver driving responses to visual inputs] Model " trained on data collected from Tesla cars being driven. Is there anything to that notion?
Yeah, likely Tesla uses similar architecture, in the spirit, at least. The devil is in the details. There are still major architectural decisions to be made that will affect reliability. The tech still needs to be matured. Then, Tesla does not use lidar, so one less data stream. I think Waymo's engineering team overall has more technical depth.
Waymo uses a compound AI system of perception to action, with modules for perception, prediction and driving policy where AI is embedded along with "glue code" that ties it all together, and tries to maximize each strength and minimize each weakness of the components.
Tesla uses a simple AI black-box of raw images in, driving policy decision out.
The black-box neural-network is far simpler and easier to implement, but not proven to be able to generalize enough on rare events to stay safe with a very high MTBF (mean time between failure). Tesla also has to prove that it can address specific failures with an ability to quickly and comprehensively fix any big failure patterns. A black-box system isn't likely to be so fixable for specific problems, since the engineers don't know why FSD does what it does. All they can do is keep training the system with data and hope the problems go away.
When Waymo has a big crash, like when they recently crashed into a pole in a back alley of Phoenix, they likely fixed the issue within a few days, convincingly enough to regulators that they see no further issue. Waymo has already proven that their Waymo Driver has solved most of the driving problem, based on over 30 million safe miles of robotaxi.
Indeed, Tesla does not have any magic way of leap-frogging Waymo. Same hard problems everywhere.
Besides, Tesla does not use lidar. Vision alone was repeatedly shown to be not robust enough.
That said, Waymo still likely has work cut for it. Freeway driving, rare events. It is not only a matter of having more cars. But unlike Musk, Dolgov does not think bluster will pave over issues.
I'll be more convinced when I see Waymo cars driving around one of the notorious east coast cities (let's say my home of the Boston area) where you've got some really sneaky 5 and 6 way intersections that even humans have a hard time figuring out. That's not to say Tesla is performing perfectly there either. I've been testing mine and while it generally gets things right, I have to be pretty trigger happy to make sure it doesn't do something really stupid since I don't want to be the subject of one of those breathless Tesla crashes while on autopilot stories. OTOH, for long distance trips mainly on Interstate highways, I rarely have to intervene. This is a game changer since I'm of an age such that driving for driving's sake just isn't a priority - getting there is.
Waymo will be able to handle Boston. It's not much different than San Fran and L.A. Using HD maps is a huge advantage for complicated intersections. Waymo Driver knows exactly how every intersection is layed out, where the lanes are, where to be ahead of time to end up in the lane of preference. Boston snow is a problem, but Waymo is working hard to solve winter driving, with lots of testing in recent winters. Waymo trains a lot in NYC. They'll be able to drive anywhere soon.
North American cities are generally easy mode; I'll be interested to see self driving capabilities in more difficult driving conditions, say London, then Shanghai, then Delhi, then Lagos.
I'm right there with you. Tesla's software will help me extend my abilities to drive beyond my 80s - if I live that long. Now, it's enabling me to take multi day trips that I used to do when in my early 70s. My kids enjoy these visits, and my wife and I see the grandkids more often. I just have to convince my wife that the technology is going to benefit her too in the same way.
This made me chuckle: "Tesla has a software problem. Waymo has a hardware problem. And I think that software problems are much easier"
AI software is the great intellectual challenge of the 21st century. Building cars at scale was perfected by Henry Ford in 1910. Cars have been a commodity for the 100+ years since.
In Karpathy's defense, as your article explains, building a lot of cars can take a few years, and perhaps that will partly offset Waymo's many-year lead on software. But what he says is still a chuckle
It's not a "chuckle" as you yourself pointed out. It will take many years for Waymo to bring down cost from $100K and have thousands of cars on the roads. On the other hand, progress on FSD appears to be must faster in the last year.
How many obstacles are there on the regulatory/compliance side? I would imagine that Google has a more robust lobbying operation than Tesla does if rollout depends on negotiating with individual municipalities. It seems like most of regulations are being established at the state level though?
I have not looked into this in detail, but my sense is that getting regulatory approval is easy in most states. That's especially true in Republican (or formerly Republican) sun belt states like Florida, Texas, and Arizona.
California regulates AVs at the state level but it issues geographically-specific permits. Like currently Waymo is authorized to operate in SF and down the SF Penninsula, but not in the East Bay.
I've been told that New York has fairly strict regulations that could delay the entry of self-driving cars by a few years. I would not be surprised if some other northeastern states have restrictions. I expect these states to be at the back end of Waymo's rollout anyway because snow is still an unsolved problem. I expect they will launch in every major city in the south and southwest, then work their way northward to areas where snow is more common. I think they can grow for 2-3 years in southern states (and maybe some other countries) before snow becomes a binding constraint.
Pretty much all AV regulations are at the state level. The feds regulate car safety after accidents, and give recommendations for AV companies to fix the problems. They don't prevent a car with pedals and steering wheel from deploying driverless in any state.
CA and AZ have comprehensive AV laws with licensing, testing programs, reporting of incidents, intervention-rate reporting during testing, limited testing areas, times of day, etc. CA is stricter in that they have more political enemies of automation than AZ; there are almost zero political enemies of AVs in Arizona.
Texas has a very simple AV law, which Florida, Georgia, and Nevada have pretty much emulated. In TX, no specific AV license is required; AVS have an implicit right to drive there, if they meet the few legal requirements of an AV system (an ADS); AVs don't even have to register in TX to start driving around, but they do register in FL and NV. The owner of the AV is the legal driver in TX. But, all states have normal driving laws, with laws against reckless driving, and felonies for severe bodily injury and property damage, and a system for removing a driver's license for enough violations. So the owner of a Tesla that is driving around driverless that crashes, would face felonies and loss of license if his car does serious harm.
In all states with a specific AV law, any ADS (automated driver) car with no driver has to have a "fallback" system, in language straight from the SAE J3016 Level-3, Level-4, and Level-5 definitions, which prevents reckless driving when the ADS fails at dynamic driving; in other words, the driverless car can't just crash, it has to have a fallback system where the car knows it's failing, and pulls over and gets out of the way to where it's safe. Waymo has a very good fallback system which includes remote operators, and a vast record of avoiding faulty accidents. If a driverless car just crashes and causes harm, it will be shown to be an illegal system in court, and face charges like any other driver for the damages, and presumably loss of driver's license. In Florida, the driver is the maker of the ADS, which is Tesla.
So FSD cars can go driverless in TX and other southern states now, but the owner of the car would face severe problems when they crash and are shown to not have a legal fallback system that should have prevented the crash. One crash could end the program if it's shown to not have a legal fallback system to prevent crashes, something any decent lawyer could easily show by comparing FSD to the sophisticated Waymo Driver, resulting in a court ruling that the ADS is illegal to go driverless. Also, in all states, any car in a crash must, if possible, get out of the way to a safe area. If a driverless car could move after a crash, and is in a dangerous spot on the road that could cause more damage, it must move to the side of the road, or it will face more charges. So FSD will have to be sophisticated enough to pull off this kind of maneuver after an accident. Cruise was dong this kind of legally-required maneuver when it pulled over, dragging a woman under the car, after the famous SF crash.
"They don't prevent a car with pedals and steering wheel from deploying driverless in any state."
I do not think this is true. The FMVSS effectively requires cars to have steering wheels and pedals. Cruise applied for an exemption for the Origin in 2022 and I don't think NHTSA ever granted it.
That's what I meant. The rules require pedals and steering wheel. If the car does have pedals and a wheel, like all current Teslas, FMVSS doesn't have much else to say about AVs.
So a current Model Y could deploy now in TX and any other state for driverless operations, with NHTSA leaving it up to the state.
The new Cybercab with no pedals and wheel will have FMVSS problems with NHTSA, unless NHTSA updates the rules or Elon pays Trump to make that go away.
BTW, you have really great AV coverage. You are one of the very few who understands what is going on.
"Waymo has long acknowledged the existence of these remote operators, but it hasn’t provided much detail about where they’re located, what they do, or how often they interact with Waymo’s vehicles."
If you watch lots of Waymo drives by Autonomy Central or JJRicks, you'll see plenty of remote operator interventions. It's obvious that they never directly drive the vehicle remotely, and they can intervene really fast, or sometimes not for several minutes of the Waymo Driver flailing around in circles. Remote Ops appears to be understaffed at times. If the remote ops team decides they want to come get the car, it could take ten to twenty minutes or more being stuck.
They always intervene by giving the Waymo Driver a path or plan to get out of the situation, like back up all the way out, or do a u-turn, or pull into that driveway and turn around, or reroute around the accident, etc.
The "remote guy steering Waymos with a joystick" trope by Tesla stans is 100% nonsense.
To deploy driverless, Tesla will need remote operations, like you said.
Remote Operations are part of the legally-required fallback system in all states, and a way of not letting the Waymo Driver freelance on corner cases in these early days. Waymo is serious about building the "world's most trusted driver", so they are trying to never cause a crash. They'd rather delay and annoy their customers pretty often while always keeping them safe, than saving money and time at the cost of a few more crashes per year.
A key point is, the Waymo Driver is always able to keep the car safe while doing DDT, with no remote operator actions, often pulling over and sitting there until remote ops get involved.
The worst situations can be when a Waymo gets stuck, with the robotaxi going around in circles or just sitting there for no apparent reason, waiting for help that is late to the situation. That can potentially infuriate customers because a bush is sticking out into the road, or it's confused for some other dumb reason that passengers find trivial. They likely lose some customers because of this.
Per usual, I found all this educational and excellently put. I do have a counterpoint of sorts though to your end gloss. It would seem to me (perhaps naively!) that Tesla's scale advantages would apply here too as they tackle the IRL complexities of any major rollout.
Take eg. roadside service. They already have 100+ service centers in the US, along with what I understand to be mobile dispatch units (sketchy on current numbers here). This seems much easier to scale up when you can cross-apply techs to both normie teslas and robotaxis, esp. when the combined number is orders of magnitude larger than Waymo's no? (This would be especially relevant if Musk goes back to his original idea of letting people rent out their own vehicles to the taxi fleet, as then eg. cleaning services also become relevant for both. While I'm a bit skeptical that this is a near-term plan, seems there would be utilization and forecasting gains if they did. Plus then you can take bookings for larger group sizes and offer an effective ownership rebate etc.)
The speed at which they create and deploy new infrastructure has also been super impressive. They've gotten new Supercharger locations up in just weeks at times and have long employed the MFABT philosophy (for better and worse!). I still think your main points here all hold though, and that this is going to be a very painful process as they battled through a million edge cases.
Tesla's roadside service will have to operate everywhere, just like Uber/Waymo. There's no shortcut to employing all those remote operations teams and service hubs with parking for thousands of cars in a metro area.
Maybe it would be somewhat of a headstart to already have service centers in some cities, but they'll end up with the same number of employees doing robotaxi services as Waymo/Uber. And Tesla won't be ready to do real robotaxi business for many years, likely beyond 2030. Waymo/Uber will have a gigantic headstart by then.
Shortcut? Perhaps not. Scaling advantage? Absolutely. When you only have eg 200 cars in a metro area it’s difficult to balance service levels and resource utilization, which in turn makes it very expensive to optimize for service levels. When you can grow your denominator by servicing a much much larger volume of cars (in this case all Teslas) that problem goes away.
And no doubt that Waymo has the edge. But as Tim outlined here they have serious headwinds of their own and it’s not obvious to me that they can capitalize that much on their head start. They aren’t getting a data advantage, no longer have a compute advantage, and will always be at a massive production disadvantage.
Waymo will have far more than 200 cars in a metro area. They have more than that now, with an increasing pace of growth.
The Waymo Driver isn't quite ready for full-scale growth yet, still not serving freeways, and they have a protocol of "responsible scaling" that moves slowly, making sure they really can stay safe after taking on a new patch of a metro. So they expand the service area a little, add a few more cars, and drive a lot to make sure they're safe, and then they can add cars, and then another patch. But they're adding patches in four markets now, five next year, and adding cars to eliminate waiting lists. They may have close to 2000 cars next year at some point, unless they have a setback. They're still pre-business, but the scaling business phase is coming soon.
The pace is slowly ramping up because they so far haven't had any big setbacks, despite now driving about one million miles per week. So Waymo will soon be able to handle over 1000 cars in Phoenix, and likely that many in the Bay Area in a few more years. They think L.A. is their biggest market.
They're not stuck at 200 cars per metro. At some point soon they'll likely be able to expand to an entire new metro area with 1000 cars in a year.
Waymo has all the data they need; the proof is in the great safety performance. In a few years the Waymo Driver will be ready to drive anywhere. Driving in Atlanta and Austin is pretty much the same as driving in L.A., S.F, and Phoenix. It's all the same once the AV driver can handle 100 million miles safely on all public roads in a metro. The key is, Waymo has really good simulation, and their driverless data is very high quality. They encounter the long tail of unusual events every day now that they do a human lifetime of driving every few days, and they can simulate variations on rare events to train the Driver for general driving skills. The Waymo Driver is built to be robust at learning and solving any new rare event they encounter, without causing side effects to weaken other skills. Their slogan of "The World's Most Trusted Driver" is not hype.
Tesla's black-box end-to-end FSD driver, on the other hand, is not at all proven to be robust enough to reach high MTBF. Tesla is still trying to solve simple things like where the lanes are, and reading signs and road markings. They have a very long way to go.
And Waymo doesn't need a compute advantage. They're one of the titans of compute. Enough is enough.
Tesla's production advantage could go away with Waymo's parnership with Hyundai, who wants robotaxi but their Motional robotaxi unit is likely being wound down. Waymo is the perfect partner for their robotaxi-ready cars. If they show they can make money soon, then do an IPO to scale Driver hardware and service, and have Hyundai invest for cars, and Uber handle the app and service hubs, they'll be in good position to scale nationwide. The Waymo Driver likely won't be very expensive. The only expensive part could be the 360-degree roof lidar, but that's only one part, and at scale will likely be a few thou$and at most.
Elon's nonsense about "expensive, stupid lidar" is stupid wrong.
Waymo only has 700ish cars on the road today. Given that ~300 are in SF, seems difficult for them to have 200+ anywhere else. While sure they have plans to add more, we’ll see how many cars they can secure from Jaguar and Zeekr before those doors shut. While getting up to 2k in 2025 sounds plausible, ramping to real numbers may not happen until Hyundai production kicks in maybe late 2026.
Your view of Tesla’s FSD sounds outdated to me. It’s not as good as Waymo yet in most respects, but is on a very healthy curve.
Agree with you and Tim that LiDAR will come down. Dunno what the floor is there, but even at eg $2k they’re not going to get to the same ballpark on unit economics vs Tesla. Making your own cars at scale is always cheaper. Same for fees to Uber.
Waymo’s slow and steady approach has its advantages no doubt. And they have Google’s deep pockets. But imo the gap won’t take that long to close. Their bull case is that Tesla’s vision-only FSD progress plateaus hard. And it could. Many certainly believe that. We shall see!
Tesla doesn't need to build much infrastructure if the RT service is being provided by owners of Tesla vehicles. If they compensation offered is sufficient it's an easy way to run a small business. Finance some vehicles and manage the logistics of charging, cleaning, storing and maintenance/repairs.
Elon has said in the past that Tesla will just have supplemental services in areas where there is not enough owner supply.
The bigger issue (off course withe main question of when will L4 or L5 autonomy be reached) would be actually starting a service and sorting through all the inevitable issues. To do that they should start a ride hail service for Teslas w a driver. The transition to driverless can be relatively seemless once the service is established (all operational issues resolved). Owners will drive their cars and once autonomy arrives in your area, you just send the vehicle instead of driving yourself. Tesla HAS to start the service w human driver's and switch to autonomous zone by zone as approval is achieved per zone. I can't see it any other way.
A driverless robotaxi operation is vastly different than an Uber system with a driver. The company will have to charge, fix, clean, retrieve stuck vehicles, deal with emergency situations, crashes, and the legal requrements of a "fallback" system to prevent drivng failures and not obstruct at crash scenes.
Tesla will not be able to just pull the driver and cut most of the costs. It will be far harder than that. What Elon is selling is magic mumbo jumbo.
FSD won't go from Level 2 to a magic Level 5 car that can handle every situation. It will be a long, hard slog to super-cheap robotaxi services.
Ultimately, Tesla robotaxi will look the same as Waymo robotaxi or Uber, Cruise, Zoox robotaxi.
All technologies were labelled 'mumbo jumbo' before they became reality.
Read my comments again. The technology will be created, managed and maintained by Tesla. But the operations will be run by vehicle or fleet owners.
You either believe the technology will work or you don't. Once it does, the driver gets replaced by the software (in geographies where licenses are granted) and the vehicle owner executes logistics.
It will be a piecemeal process. But as the busieness case is proven it will expand quicker as people don't want to be left behind.
Most people had no use for a phone that had to be charged with spotty connections and hogh fees. And most did not think they would want to buy a computer to dial up into the internet and wait 10 minutes to download 1 page of text. The Internet was also considered a fad by most 30 years ago. Who would want to stream movies if you could go to Blockbusters?
FSD is currently at about the driverless safety level that Waymo/Google was at in 2012, when Google was giving driverless demos and completing 100-mile intervention-free rides around the Bay Area.
And Tesla is trying to achieve a harder goal than Waymo; they're trying for Level-5 driving on all public roads, in all conditions, with far fewer sensors and redundancy, and lousy maps, with a black-box end-to-end AI driver that no engineer understands in detail.
The idea of FSD suddenly waking up to be super-duper human is idiotic, considering that it currently might, if being generous, be able to complete one hour of safe city driving with no human help. It took Google eight more years to open a public robotaxi service in parts of Chandler AZ, from a starting point at least as good as FSD is at now, with four major hardware upgrades along the way. And Waymo is still not quite ready to scale nationally, in 2024.
And then the plan is for Tesla to use their magic driver in a massive nationwide ride-hailing system they build from scratch, with no experience at all. What could possibly be hard about ride-hailing, finding and managing all those customers, pleasing the regulators, servicing the cars, pleasing the public? And they want to do it with random Tesla dudes at home in their pajamas, expecting their magic cars to print money for them all day long.
One claim not touched on here is Musk's assertion that "it's not possible to solve the self-driving problem without having millions of vehicles on the road", or more generally that Tesla has an unassailable advantage when it comes to training data that only grow as MTBF increases. Does that hold water?
Obviously, Waymo is driverless in busy cities for a million miles per week, with a very impressive safety record that has yet to have a faulty major bodily-injury accident in now over 30 million driverless miles. They are steadily scaling up to five major metro areas.
Waymo has obviously mostly solved city-street driving. If they have this huge data deficit compared to FSD, then why is Waymo so much better at driving than FSD, which can't even do driverless testing yet in an easy suburb? FSD can't last a single day driverless in a real city, probably not even one hour in busy traffic.
The goal of AV engineers is to safely go driverless over big mileage volume in busy markets, not to accumulate the most Level-2 driving data, most of which is useless.
Waymo is proving that they have all the data they need to solve the long tail of edge cases that come over 30 million miles. They'll likely soon prove they can stay safe over 100 million miles.
The proof of the AV safety is in the safe driverless driving at scale.
Excellent post. If Tesla FSD requires "human intervention once every 13 miles," then that does seem better than Cruise in 2023 at "roughly every four to five miles," per CNBC: https://www.cnbc.com/2023/11/06/cruise-confirms-robotaxis-rely-on-human-assistance-every-4-to-5-miles.html .... Should we regard these numbers as comparable? Would Tesla's number be inflated by uneventful highway travel? And despite Waymo's resistance to giving us a similar metric, is there any way we can come to an educated guess?
No we shouldn't regard those numbers as comparable. The Tesla model is that the human is monitoring the vehicle 100 percent of the time and only intervenes if the car is about to make a serious mistake. The Waymo/Cruise model is that the car itself identifies situations where it might need remote assistance and phones home proactively. In most of these cases the vehicle will have a default action that would have worked out just fine, but the car is checking in with HQ out of an abundance of caution. Most human interventions in the Waymo/Cruise are likely just the human saying "yes that route you chose makes sense" or answering questions like "is this object ahead a fire truck?"
For a sense of perspective, back in 2016 Waymo/Google was reporting that their vehicles had a safety-related disengagement once every 5,000 miles. I think we should take that with a grain of salt since they may have had an unreasonably narrow definition of safety-related disengagement, but still it's safe to say Waymo cars can go a lot more than 13 miles without making a serious mistake. Still, the vehicle may be checking in much more frequently because it checks in when it's confidence on the best action is below (say) 98 percent.
The key is, Tesla isn't yet driverless, so their intervention rate numbers are a very noisy statistic that is only a rough estimate of how capable FSD is. Tesla drivers intervene for any number of reasons. And Tesla doesn't release their intervention data anyway.
Waymo, and Cruise before their suspension. were driverless, which is a very different world. There are no sudden interventions when driverless. Remote operations aren't driving the car, they are a fallback safety team that solves difficulties that the driverless ADS avoided, then pulled over and called home to let the humans approve the solution.
FSD Level-2 intervention rates by car owners, often sudden moves to prevent catastrophe, compared to Level-4 driverless remote operations, are cherries to coconuts comparisons. They have almost nothing in common. There is no way to compare Waymo's "intervention" rate to FSD's.
Waymo completes 100,000 rider-only trips per week, with over one million trips total, without having had a single major faulty accident. They have lots of smaller accidents, all of which are reported to NHTSA and available to safety researchers for studies. The vast majorty of those are other cars smashing into them one way or another, and about half are scrapes under 1 mph. Their crash rate for higher-impact airbag deployment crashes is 6x better than comparable human stats on the same roads.
Waymo is far, far more transparent than Tesla, because they know they have the goods: a really safe driverless robotaxi, that they want everybody to know about. They just opened a data hub that makes it easy for researchers to do their own studies on their NHTSA crash data. Waymo's own studies are very professional, always comparing their safety record to drivers on the exact same public roads. Tesla's "studies", in contrast, are misleading apples to oranges comparisons to make them look better, and not reproducible.
I don't see any mention of Tesla's "imitiation learning" advantage. Potentially every Tesla (at least those who opt-in now) is collecting driving data in a multitude of situations -- lighting, terrain, weather, landscape, traffic, you name it -- and recording how the driver/FSD is treating every situation. This is a massive dataset growing at rate Waymo can never match.
Good AI is a dataset problem once you have fast-enough hardware. Collecting enough data for effective imitation-learning/behavior-cloning is Karpathy's "software problem".
From the hardware side, Waymo doesn't build it's own cars, and probably won't be able to install Starlink mini for rural use, so Tesla is ahead there. However, Waymo has access to Google's TPU engineers so that might give it a power/compute advantage over Tesla's Nvidia Drive PX 2.
"Waymo is currently building out a network of depots in Los Angeles, Phoenix, and San Francisco and hiring people to clean, charge, and repair its cars. Recently Waymo signed a deal to offload some of these functions to Uber in its next two markets: Austin and Atlanta. But it will still take time for Uber to build the infrastructure required to provide these services."
Tesla already has hundreds of service centers, so at least the physical depot part is solved. It'd be funny if there ends up being a job of "Fostering a Car", similar to those who chase down scooters and electric bikes in cities for charging.
is it true that Waymo depends on a lot of sensors on the streets, where as Tesla only depends on the onboard cameras? therefore scaling could be limited by co-operation from cities? I didn't understand your explanation of how Waymo navigates through construction areas? are Waymo taxis equipped with cameras as well that inform the rest of the fleet? Can you explain how Waymo navigates an area in which there is not an accurate map please? thanks
Waymo has cameras as well as lidar sensors so there's no obvious advantage for Tesla there. Waymo hasn't shared all the details of how its technology works, but my guess would be that when Waymo encounters a construction site it assembles a world model "on the fly" identifying the locations of traffic cones, construction workers, pavement, etc. Then it uses this map instead of the default one to navigate the scene. And yes I believe it will then notify Waymo headquarters so Waymo can update the map and push it out to other vehicles.
This is the ABSOLUTE BEST ARTICLE yet on Tesla vs Waymo. Thank you.
I have a 50+ years experience in IT. (1st AI course at NCSU in 1973). I was a tester for a USNavy weapon system. Highly reliable software is very difficult. I've owned a Tesla YP with FSD ($10,000) since 2021.
IMHO, FSD-Not-Supervised is at least 2-years from "mostly reliable." Certainly not good enough to put my child into for the trip to school. Just last week, I had to brake hard to avoid getting t-boned at an admittedly difficult intersection, albeit at 35mph, using the latest version of FSD-12.
I'd guess 2027...or later for any realistic level 5 certification. Elon will continue to promise "early next year" in the interim. Especially when TSLA needs a pump.
I hope after tonight's announcement we will be able to have an equivalent diagram for Tesla to the Dolgov one you have here. Not something I know much about but I hear that it will be based on a sort of LLM-like "Large [Tesla driver driving responses to visual inputs] Model " trained on data collected from Tesla cars being driven. Is there anything to that notion?
Yeah, likely Tesla uses similar architecture, in the spirit, at least. The devil is in the details. There are still major architectural decisions to be made that will affect reliability. The tech still needs to be matured. Then, Tesla does not use lidar, so one less data stream. I think Waymo's engineering team overall has more technical depth.
Waymo uses a compound AI system of perception to action, with modules for perception, prediction and driving policy where AI is embedded along with "glue code" that ties it all together, and tries to maximize each strength and minimize each weakness of the components.
Tesla uses a simple AI black-box of raw images in, driving policy decision out.
The black-box neural-network is far simpler and easier to implement, but not proven to be able to generalize enough on rare events to stay safe with a very high MTBF (mean time between failure). Tesla also has to prove that it can address specific failures with an ability to quickly and comprehensively fix any big failure patterns. A black-box system isn't likely to be so fixable for specific problems, since the engineers don't know why FSD does what it does. All they can do is keep training the system with data and hope the problems go away.
When Waymo has a big crash, like when they recently crashed into a pole in a back alley of Phoenix, they likely fixed the issue within a few days, convincingly enough to regulators that they see no further issue. Waymo has already proven that their Waymo Driver has solved most of the driving problem, based on over 30 million safe miles of robotaxi.
Indeed, Tesla does not have any magic way of leap-frogging Waymo. Same hard problems everywhere.
Besides, Tesla does not use lidar. Vision alone was repeatedly shown to be not robust enough.
That said, Waymo still likely has work cut for it. Freeway driving, rare events. It is not only a matter of having more cars. But unlike Musk, Dolgov does not think bluster will pave over issues.
I'll be more convinced when I see Waymo cars driving around one of the notorious east coast cities (let's say my home of the Boston area) where you've got some really sneaky 5 and 6 way intersections that even humans have a hard time figuring out. That's not to say Tesla is performing perfectly there either. I've been testing mine and while it generally gets things right, I have to be pretty trigger happy to make sure it doesn't do something really stupid since I don't want to be the subject of one of those breathless Tesla crashes while on autopilot stories. OTOH, for long distance trips mainly on Interstate highways, I rarely have to intervene. This is a game changer since I'm of an age such that driving for driving's sake just isn't a priority - getting there is.
Waymo will be able to handle Boston. It's not much different than San Fran and L.A. Using HD maps is a huge advantage for complicated intersections. Waymo Driver knows exactly how every intersection is layed out, where the lanes are, where to be ahead of time to end up in the lane of preference. Boston snow is a problem, but Waymo is working hard to solve winter driving, with lots of testing in recent winters. Waymo trains a lot in NYC. They'll be able to drive anywhere soon.
North American cities are generally easy mode; I'll be interested to see self driving capabilities in more difficult driving conditions, say London, then Shanghai, then Delhi, then Lagos.
I'm right there with you. Tesla's software will help me extend my abilities to drive beyond my 80s - if I live that long. Now, it's enabling me to take multi day trips that I used to do when in my early 70s. My kids enjoy these visits, and my wife and I see the grandkids more often. I just have to convince my wife that the technology is going to benefit her too in the same way.
This made me chuckle: "Tesla has a software problem. Waymo has a hardware problem. And I think that software problems are much easier"
AI software is the great intellectual challenge of the 21st century. Building cars at scale was perfected by Henry Ford in 1910. Cars have been a commodity for the 100+ years since.
In Karpathy's defense, as your article explains, building a lot of cars can take a few years, and perhaps that will partly offset Waymo's many-year lead on software. But what he says is still a chuckle
It's not a "chuckle" as you yourself pointed out. It will take many years for Waymo to bring down cost from $100K and have thousands of cars on the roads. On the other hand, progress on FSD appears to be must faster in the last year.
How many obstacles are there on the regulatory/compliance side? I would imagine that Google has a more robust lobbying operation than Tesla does if rollout depends on negotiating with individual municipalities. It seems like most of regulations are being established at the state level though?
I have not looked into this in detail, but my sense is that getting regulatory approval is easy in most states. That's especially true in Republican (or formerly Republican) sun belt states like Florida, Texas, and Arizona.
California regulates AVs at the state level but it issues geographically-specific permits. Like currently Waymo is authorized to operate in SF and down the SF Penninsula, but not in the East Bay.
I've been told that New York has fairly strict regulations that could delay the entry of self-driving cars by a few years. I would not be surprised if some other northeastern states have restrictions. I expect these states to be at the back end of Waymo's rollout anyway because snow is still an unsolved problem. I expect they will launch in every major city in the south and southwest, then work their way northward to areas where snow is more common. I think they can grow for 2-3 years in southern states (and maybe some other countries) before snow becomes a binding constraint.
Pretty much all AV regulations are at the state level. The feds regulate car safety after accidents, and give recommendations for AV companies to fix the problems. They don't prevent a car with pedals and steering wheel from deploying driverless in any state.
CA and AZ have comprehensive AV laws with licensing, testing programs, reporting of incidents, intervention-rate reporting during testing, limited testing areas, times of day, etc. CA is stricter in that they have more political enemies of automation than AZ; there are almost zero political enemies of AVs in Arizona.
Texas has a very simple AV law, which Florida, Georgia, and Nevada have pretty much emulated. In TX, no specific AV license is required; AVS have an implicit right to drive there, if they meet the few legal requirements of an AV system (an ADS); AVs don't even have to register in TX to start driving around, but they do register in FL and NV. The owner of the AV is the legal driver in TX. But, all states have normal driving laws, with laws against reckless driving, and felonies for severe bodily injury and property damage, and a system for removing a driver's license for enough violations. So the owner of a Tesla that is driving around driverless that crashes, would face felonies and loss of license if his car does serious harm.
In all states with a specific AV law, any ADS (automated driver) car with no driver has to have a "fallback" system, in language straight from the SAE J3016 Level-3, Level-4, and Level-5 definitions, which prevents reckless driving when the ADS fails at dynamic driving; in other words, the driverless car can't just crash, it has to have a fallback system where the car knows it's failing, and pulls over and gets out of the way to where it's safe. Waymo has a very good fallback system which includes remote operators, and a vast record of avoiding faulty accidents. If a driverless car just crashes and causes harm, it will be shown to be an illegal system in court, and face charges like any other driver for the damages, and presumably loss of driver's license. In Florida, the driver is the maker of the ADS, which is Tesla.
So FSD cars can go driverless in TX and other southern states now, but the owner of the car would face severe problems when they crash and are shown to not have a legal fallback system that should have prevented the crash. One crash could end the program if it's shown to not have a legal fallback system to prevent crashes, something any decent lawyer could easily show by comparing FSD to the sophisticated Waymo Driver, resulting in a court ruling that the ADS is illegal to go driverless. Also, in all states, any car in a crash must, if possible, get out of the way to a safe area. If a driverless car could move after a crash, and is in a dangerous spot on the road that could cause more damage, it must move to the side of the road, or it will face more charges. So FSD will have to be sophisticated enough to pull off this kind of maneuver after an accident. Cruise was dong this kind of legally-required maneuver when it pulled over, dragging a woman under the car, after the famous SF crash.
"They don't prevent a car with pedals and steering wheel from deploying driverless in any state."
I do not think this is true. The FMVSS effectively requires cars to have steering wheels and pedals. Cruise applied for an exemption for the Origin in 2022 and I don't think NHTSA ever granted it.
That's what I meant. The rules require pedals and steering wheel. If the car does have pedals and a wheel, like all current Teslas, FMVSS doesn't have much else to say about AVs.
So a current Model Y could deploy now in TX and any other state for driverless operations, with NHTSA leaving it up to the state.
The new Cybercab with no pedals and wheel will have FMVSS problems with NHTSA, unless NHTSA updates the rules or Elon pays Trump to make that go away.
BTW, you have really great AV coverage. You are one of the very few who understands what is going on.
Oh I see what you mean. Thanks for reading!
“So FSD will have to be sophisticated enough to pull off this kind of maneuver after an accident”
Well Musk announced “Unsupervised FSD” coming in SF and TX in 2025. We’ll see how that goes 😀
"Waymo has long acknowledged the existence of these remote operators, but it hasn’t provided much detail about where they’re located, what they do, or how often they interact with Waymo’s vehicles."
On "what they do", there's this: https://waymo.com/blog/2024/05/fleet-response/
Yes, that's better than nothing but I'd like to see a lot more information.
If you watch lots of Waymo drives by Autonomy Central or JJRicks, you'll see plenty of remote operator interventions. It's obvious that they never directly drive the vehicle remotely, and they can intervene really fast, or sometimes not for several minutes of the Waymo Driver flailing around in circles. Remote Ops appears to be understaffed at times. If the remote ops team decides they want to come get the car, it could take ten to twenty minutes or more being stuck.
They always intervene by giving the Waymo Driver a path or plan to get out of the situation, like back up all the way out, or do a u-turn, or pull into that driveway and turn around, or reroute around the accident, etc.
The "remote guy steering Waymos with a joystick" trope by Tesla stans is 100% nonsense.
To deploy driverless, Tesla will need remote operations, like you said.
Remote Operations are part of the legally-required fallback system in all states, and a way of not letting the Waymo Driver freelance on corner cases in these early days. Waymo is serious about building the "world's most trusted driver", so they are trying to never cause a crash. They'd rather delay and annoy their customers pretty often while always keeping them safe, than saving money and time at the cost of a few more crashes per year.
A key point is, the Waymo Driver is always able to keep the car safe while doing DDT, with no remote operator actions, often pulling over and sitting there until remote ops get involved.
The worst situations can be when a Waymo gets stuck, with the robotaxi going around in circles or just sitting there for no apparent reason, waiting for help that is late to the situation. That can potentially infuriate customers because a bush is sticking out into the road, or it's confused for some other dumb reason that passengers find trivial. They likely lose some customers because of this.
Per usual, I found all this educational and excellently put. I do have a counterpoint of sorts though to your end gloss. It would seem to me (perhaps naively!) that Tesla's scale advantages would apply here too as they tackle the IRL complexities of any major rollout.
Take eg. roadside service. They already have 100+ service centers in the US, along with what I understand to be mobile dispatch units (sketchy on current numbers here). This seems much easier to scale up when you can cross-apply techs to both normie teslas and robotaxis, esp. when the combined number is orders of magnitude larger than Waymo's no? (This would be especially relevant if Musk goes back to his original idea of letting people rent out their own vehicles to the taxi fleet, as then eg. cleaning services also become relevant for both. While I'm a bit skeptical that this is a near-term plan, seems there would be utilization and forecasting gains if they did. Plus then you can take bookings for larger group sizes and offer an effective ownership rebate etc.)
The speed at which they create and deploy new infrastructure has also been super impressive. They've gotten new Supercharger locations up in just weeks at times and have long employed the MFABT philosophy (for better and worse!). I still think your main points here all hold though, and that this is going to be a very painful process as they battled through a million edge cases.
Tesla's roadside service will have to operate everywhere, just like Uber/Waymo. There's no shortcut to employing all those remote operations teams and service hubs with parking for thousands of cars in a metro area.
Maybe it would be somewhat of a headstart to already have service centers in some cities, but they'll end up with the same number of employees doing robotaxi services as Waymo/Uber. And Tesla won't be ready to do real robotaxi business for many years, likely beyond 2030. Waymo/Uber will have a gigantic headstart by then.
Shortcut? Perhaps not. Scaling advantage? Absolutely. When you only have eg 200 cars in a metro area it’s difficult to balance service levels and resource utilization, which in turn makes it very expensive to optimize for service levels. When you can grow your denominator by servicing a much much larger volume of cars (in this case all Teslas) that problem goes away.
And no doubt that Waymo has the edge. But as Tim outlined here they have serious headwinds of their own and it’s not obvious to me that they can capitalize that much on their head start. They aren’t getting a data advantage, no longer have a compute advantage, and will always be at a massive production disadvantage.
Waymo will have far more than 200 cars in a metro area. They have more than that now, with an increasing pace of growth.
The Waymo Driver isn't quite ready for full-scale growth yet, still not serving freeways, and they have a protocol of "responsible scaling" that moves slowly, making sure they really can stay safe after taking on a new patch of a metro. So they expand the service area a little, add a few more cars, and drive a lot to make sure they're safe, and then they can add cars, and then another patch. But they're adding patches in four markets now, five next year, and adding cars to eliminate waiting lists. They may have close to 2000 cars next year at some point, unless they have a setback. They're still pre-business, but the scaling business phase is coming soon.
The pace is slowly ramping up because they so far haven't had any big setbacks, despite now driving about one million miles per week. So Waymo will soon be able to handle over 1000 cars in Phoenix, and likely that many in the Bay Area in a few more years. They think L.A. is their biggest market.
They're not stuck at 200 cars per metro. At some point soon they'll likely be able to expand to an entire new metro area with 1000 cars in a year.
Waymo has all the data they need; the proof is in the great safety performance. In a few years the Waymo Driver will be ready to drive anywhere. Driving in Atlanta and Austin is pretty much the same as driving in L.A., S.F, and Phoenix. It's all the same once the AV driver can handle 100 million miles safely on all public roads in a metro. The key is, Waymo has really good simulation, and their driverless data is very high quality. They encounter the long tail of unusual events every day now that they do a human lifetime of driving every few days, and they can simulate variations on rare events to train the Driver for general driving skills. The Waymo Driver is built to be robust at learning and solving any new rare event they encounter, without causing side effects to weaken other skills. Their slogan of "The World's Most Trusted Driver" is not hype.
Tesla's black-box end-to-end FSD driver, on the other hand, is not at all proven to be robust enough to reach high MTBF. Tesla is still trying to solve simple things like where the lanes are, and reading signs and road markings. They have a very long way to go.
And Waymo doesn't need a compute advantage. They're one of the titans of compute. Enough is enough.
Tesla's production advantage could go away with Waymo's parnership with Hyundai, who wants robotaxi but their Motional robotaxi unit is likely being wound down. Waymo is the perfect partner for their robotaxi-ready cars. If they show they can make money soon, then do an IPO to scale Driver hardware and service, and have Hyundai invest for cars, and Uber handle the app and service hubs, they'll be in good position to scale nationwide. The Waymo Driver likely won't be very expensive. The only expensive part could be the 360-degree roof lidar, but that's only one part, and at scale will likely be a few thou$and at most.
Elon's nonsense about "expensive, stupid lidar" is stupid wrong.
Waymo only has 700ish cars on the road today. Given that ~300 are in SF, seems difficult for them to have 200+ anywhere else. While sure they have plans to add more, we’ll see how many cars they can secure from Jaguar and Zeekr before those doors shut. While getting up to 2k in 2025 sounds plausible, ramping to real numbers may not happen until Hyundai production kicks in maybe late 2026.
Your view of Tesla’s FSD sounds outdated to me. It’s not as good as Waymo yet in most respects, but is on a very healthy curve.
Agree with you and Tim that LiDAR will come down. Dunno what the floor is there, but even at eg $2k they’re not going to get to the same ballpark on unit economics vs Tesla. Making your own cars at scale is always cheaper. Same for fees to Uber.
Waymo’s slow and steady approach has its advantages no doubt. And they have Google’s deep pockets. But imo the gap won’t take that long to close. Their bull case is that Tesla’s vision-only FSD progress plateaus hard. And it could. Many certainly believe that. We shall see!
Great post Tim!
Tesla doesn't need to build much infrastructure if the RT service is being provided by owners of Tesla vehicles. If they compensation offered is sufficient it's an easy way to run a small business. Finance some vehicles and manage the logistics of charging, cleaning, storing and maintenance/repairs.
Elon has said in the past that Tesla will just have supplemental services in areas where there is not enough owner supply.
The bigger issue (off course withe main question of when will L4 or L5 autonomy be reached) would be actually starting a service and sorting through all the inevitable issues. To do that they should start a ride hail service for Teslas w a driver. The transition to driverless can be relatively seemless once the service is established (all operational issues resolved). Owners will drive their cars and once autonomy arrives in your area, you just send the vehicle instead of driving yourself. Tesla HAS to start the service w human driver's and switch to autonomous zone by zone as approval is achieved per zone. I can't see it any other way.
A driverless robotaxi operation is vastly different than an Uber system with a driver. The company will have to charge, fix, clean, retrieve stuck vehicles, deal with emergency situations, crashes, and the legal requrements of a "fallback" system to prevent drivng failures and not obstruct at crash scenes.
Tesla will not be able to just pull the driver and cut most of the costs. It will be far harder than that. What Elon is selling is magic mumbo jumbo.
FSD won't go from Level 2 to a magic Level 5 car that can handle every situation. It will be a long, hard slog to super-cheap robotaxi services.
Ultimately, Tesla robotaxi will look the same as Waymo robotaxi or Uber, Cruise, Zoox robotaxi.
All technologies were labelled 'mumbo jumbo' before they became reality.
Read my comments again. The technology will be created, managed and maintained by Tesla. But the operations will be run by vehicle or fleet owners.
You either believe the technology will work or you don't. Once it does, the driver gets replaced by the software (in geographies where licenses are granted) and the vehicle owner executes logistics.
It will be a piecemeal process. But as the busieness case is proven it will expand quicker as people don't want to be left behind.
Most people had no use for a phone that had to be charged with spotty connections and hogh fees. And most did not think they would want to buy a computer to dial up into the internet and wait 10 minutes to download 1 page of text. The Internet was also considered a fad by most 30 years ago. Who would want to stream movies if you could go to Blockbusters?
FSD is currently at about the driverless safety level that Waymo/Google was at in 2012, when Google was giving driverless demos and completing 100-mile intervention-free rides around the Bay Area.
And Tesla is trying to achieve a harder goal than Waymo; they're trying for Level-5 driving on all public roads, in all conditions, with far fewer sensors and redundancy, and lousy maps, with a black-box end-to-end AI driver that no engineer understands in detail.
The idea of FSD suddenly waking up to be super-duper human is idiotic, considering that it currently might, if being generous, be able to complete one hour of safe city driving with no human help. It took Google eight more years to open a public robotaxi service in parts of Chandler AZ, from a starting point at least as good as FSD is at now, with four major hardware upgrades along the way. And Waymo is still not quite ready to scale nationally, in 2024.
And then the plan is for Tesla to use their magic driver in a massive nationwide ride-hailing system they build from scratch, with no experience at all. What could possibly be hard about ride-hailing, finding and managing all those customers, pleasing the regulators, servicing the cars, pleasing the public? And they want to do it with random Tesla dudes at home in their pajamas, expecting their magic cars to print money for them all day long.
The whole plan seems a bit fishy, I do say.
One claim not touched on here is Musk's assertion that "it's not possible to solve the self-driving problem without having millions of vehicles on the road", or more generally that Tesla has an unassailable advantage when it comes to training data that only grow as MTBF increases. Does that hold water?
No, it doesn't hold water.
Obviously, Waymo is driverless in busy cities for a million miles per week, with a very impressive safety record that has yet to have a faulty major bodily-injury accident in now over 30 million driverless miles. They are steadily scaling up to five major metro areas.
Waymo has obviously mostly solved city-street driving. If they have this huge data deficit compared to FSD, then why is Waymo so much better at driving than FSD, which can't even do driverless testing yet in an easy suburb? FSD can't last a single day driverless in a real city, probably not even one hour in busy traffic.
The goal of AV engineers is to safely go driverless over big mileage volume in busy markets, not to accumulate the most Level-2 driving data, most of which is useless.
Waymo is proving that they have all the data they need to solve the long tail of edge cases that come over 30 million miles. They'll likely soon prove they can stay safe over 100 million miles.
The proof of the AV safety is in the safe driverless driving at scale.
Excellent post. If Tesla FSD requires "human intervention once every 13 miles," then that does seem better than Cruise in 2023 at "roughly every four to five miles," per CNBC: https://www.cnbc.com/2023/11/06/cruise-confirms-robotaxis-rely-on-human-assistance-every-4-to-5-miles.html .... Should we regard these numbers as comparable? Would Tesla's number be inflated by uneventful highway travel? And despite Waymo's resistance to giving us a similar metric, is there any way we can come to an educated guess?
No we shouldn't regard those numbers as comparable. The Tesla model is that the human is monitoring the vehicle 100 percent of the time and only intervenes if the car is about to make a serious mistake. The Waymo/Cruise model is that the car itself identifies situations where it might need remote assistance and phones home proactively. In most of these cases the vehicle will have a default action that would have worked out just fine, but the car is checking in with HQ out of an abundance of caution. Most human interventions in the Waymo/Cruise are likely just the human saying "yes that route you chose makes sense" or answering questions like "is this object ahead a fire truck?"
For a sense of perspective, back in 2016 Waymo/Google was reporting that their vehicles had a safety-related disengagement once every 5,000 miles. I think we should take that with a grain of salt since they may have had an unreasonably narrow definition of safety-related disengagement, but still it's safe to say Waymo cars can go a lot more than 13 miles without making a serious mistake. Still, the vehicle may be checking in much more frequently because it checks in when it's confidence on the best action is below (say) 98 percent.
What Timothy replied is excellent.
The key is, Tesla isn't yet driverless, so their intervention rate numbers are a very noisy statistic that is only a rough estimate of how capable FSD is. Tesla drivers intervene for any number of reasons. And Tesla doesn't release their intervention data anyway.
Waymo, and Cruise before their suspension. were driverless, which is a very different world. There are no sudden interventions when driverless. Remote operations aren't driving the car, they are a fallback safety team that solves difficulties that the driverless ADS avoided, then pulled over and called home to let the humans approve the solution.
FSD Level-2 intervention rates by car owners, often sudden moves to prevent catastrophe, compared to Level-4 driverless remote operations, are cherries to coconuts comparisons. They have almost nothing in common. There is no way to compare Waymo's "intervention" rate to FSD's.
Waymo completes 100,000 rider-only trips per week, with over one million trips total, without having had a single major faulty accident. They have lots of smaller accidents, all of which are reported to NHTSA and available to safety researchers for studies. The vast majorty of those are other cars smashing into them one way or another, and about half are scrapes under 1 mph. Their crash rate for higher-impact airbag deployment crashes is 6x better than comparable human stats on the same roads.
Waymo is far, far more transparent than Tesla, because they know they have the goods: a really safe driverless robotaxi, that they want everybody to know about. They just opened a data hub that makes it easy for researchers to do their own studies on their NHTSA crash data. Waymo's own studies are very professional, always comparing their safety record to drivers on the exact same public roads. Tesla's "studies", in contrast, are misleading apples to oranges comparisons to make them look better, and not reproducible.
I don't see any mention of Tesla's "imitiation learning" advantage. Potentially every Tesla (at least those who opt-in now) is collecting driving data in a multitude of situations -- lighting, terrain, weather, landscape, traffic, you name it -- and recording how the driver/FSD is treating every situation. This is a massive dataset growing at rate Waymo can never match.
Good AI is a dataset problem once you have fast-enough hardware. Collecting enough data for effective imitation-learning/behavior-cloning is Karpathy's "software problem".
From the hardware side, Waymo doesn't build it's own cars, and probably won't be able to install Starlink mini for rural use, so Tesla is ahead there. However, Waymo has access to Google's TPU engineers so that might give it a power/compute advantage over Tesla's Nvidia Drive PX 2.
"Waymo is currently building out a network of depots in Los Angeles, Phoenix, and San Francisco and hiring people to clean, charge, and repair its cars. Recently Waymo signed a deal to offload some of these functions to Uber in its next two markets: Austin and Atlanta. But it will still take time for Uber to build the infrastructure required to provide these services."
Tesla already has hundreds of service centers, so at least the physical depot part is solved. It'd be funny if there ends up being a job of "Fostering a Car", similar to those who chase down scooters and electric bikes in cities for charging.
This is pretty funny. Claim #4 is not a claim, it’s a fact and it is the reason that Tesla is already far ahead of Waymo.
is it true that Waymo depends on a lot of sensors on the streets, where as Tesla only depends on the onboard cameras? therefore scaling could be limited by co-operation from cities? I didn't understand your explanation of how Waymo navigates through construction areas? are Waymo taxis equipped with cameras as well that inform the rest of the fleet? Can you explain how Waymo navigates an area in which there is not an accurate map please? thanks
Waymo has cameras as well as lidar sensors so there's no obvious advantage for Tesla there. Waymo hasn't shared all the details of how its technology works, but my guess would be that when Waymo encounters a construction site it assembles a world model "on the fly" identifying the locations of traffic cones, construction workers, pavement, etc. Then it uses this map instead of the default one to navigate the scene. And yes I believe it will then notify Waymo headquarters so Waymo can update the map and push it out to other vehicles.
This is the ABSOLUTE BEST ARTICLE yet on Tesla vs Waymo. Thank you.
I have a 50+ years experience in IT. (1st AI course at NCSU in 1973). I was a tester for a USNavy weapon system. Highly reliable software is very difficult. I've owned a Tesla YP with FSD ($10,000) since 2021.
IMHO, FSD-Not-Supervised is at least 2-years from "mostly reliable." Certainly not good enough to put my child into for the trip to school. Just last week, I had to brake hard to avoid getting t-boned at an admittedly difficult intersection, albeit at 35mph, using the latest version of FSD-12.
I'd guess 2027...or later for any realistic level 5 certification. Elon will continue to promise "early next year" in the interim. Especially when TSLA needs a pump.