Forums

Why Tesla Is Betting On Cameras For Full Self-Driving

Why Tesla Is Betting On Cameras For Full Self-Driving

I almost never look at Seeking Alpha articles as they are usually garbage created by Tesla shorters. This one caught my eye and is overall positive and brings up some excellent points, especially about LIDAR. One of the more reasoned articles I've come across.

https://seekingalpha.com/article/4106093-tesla-betting-cameras-full-self...

Pungoteague_Dave | September 15, 2017

@inconel "It is kinda fun though to read through some of my fellow forum members who appoint themselves expert in AI and Computer Vision and decide that Elon and his engineers are dead wrong."

No, we accept the words and guidance of ex-Tesla engineers, the CEO of Mobileye, NASA's analysis of AP functionality in flight, google's engineers, ford's analysis about what's possible, Audi's conclusions, and our own eyes regarding the Mr. Magoo functionality that Tesla has already been able to deliver given current sensors. The sensor and cameras on our cars simply cannot see far or well enough to allow a car to drive unattended, which is what FSD entails. This view is backed up by Tesla being consistently unable to deliver on Mr. Musk's promises - some just late, but many NEVER being delivered. The only person who believes FSD can happen with only a camera suite is Elon Musk - and I do not believe that actually believes it either, despite saying so. His engineers certainly do not.

DonS | September 15, 2017

Most experts are wrong, just not all the time. It is the very definition of competition.

LIDAR vs cameras is an economic choice more than a technical choice. LIDAR is a better data set, but multiple cameras and processing also get the needed view of vehicle surroundings. Tesla thinks multiple cameras will be cheaper than LIDAR. Both sides are developing so rapidly that it will take years to declare a winner.

inconel | September 15, 2017

Yeah I guess time will tell. I can't comment since I am not an expert and can't argue against second-hand experts or real experts who might have an agenda.

SbMD | September 15, 2017

A succinct summary of the three technologies (cameras, LIDAR, and radar) in autonomous driving:
http://www.eetimes.com/author.asp?section_id=36&doc_id=1330069

SbMD | September 15, 2017

An interesting example of a relatively inexpensive hardware, camera-only, self-driving car. This shows that it is not so impossible to consider a camera-only solution:
https://electrek.co/2017/03/19/self-driving-car-camera-only-autox-tesla/

From EM's TED talk, which is really not really "left field thinking", if you read a critical mass of papers on this approach (and I have):

"There’s no LiDAR or RADAR being used here. This is just using passive optical which use essentially what a person uses. The whole road system is meant to be navigated with passive optical or camera, so once you solve cameras, or vision, then autonomy is solved. If you don’t solve vision, it’s not solved. That’s why our focus is so heavily on having a vision neural net that’s very effective for road conditions. You can absolutely be super-human with just cameras. You could probably do ten times better than humans with just cameras."

TeslaTap.com | September 15, 2017

PD - All those people you cite may be correct, or totally wrong:

- ex-Tesla engineers - Fired or left to go to a competitor - they may have other reasons for their statements and/or they were not smart enough to do what was required. They could be right too, so it is hard to evaluate the credibility of this.

- CEO of Mobileye - clearly a competitor with an agenda to denigrate Tesla

- NASA's analysis of AP functionality in flight - This is a new one. I'm not aware of NASA saying Tesla cannot do FSD, but please cite something. They are the most believable of the group cited, but also have an almost limitless budget for FSD for things like the mars lander. Hardware cost is not a concern.

- Google's engineers, Ford's analysis about what's possible, Audi's... - These are all direct competitors with nothing to offer the consumer today. Of course they are going to say Tesla is totally wrong and they have all the answers, if you can wait 5 years.

I'm not an expert in the field, but as both a software and hardware engineer, I have a fairly good background to evaluate the technology. I have yet to see LIDAR is a requirement for FSD. Actually, my thoughts are the hardware is actually not nearly as important as the software, and the software for any FSD system is exceptionally complex.

SamO | September 15, 2017

@TeslaTap,

@PD has been wrong about Tesla's self driving program both early and often. Even Dave would have to admit that his understanding of Tesla's program, especially with regard to who between Tesla and Mobileeye's, was designing Tesla's software was 100% wrong. I think he called it "bolt on" if I recall.

But Dave is awesome about owning up to his errors in judgement.

NKYTA | September 15, 2017

Sarcasm SamO? You? ;-) ;)

I'm guessing here, but I imagine the only engineers PD has worked with, don't live in SV or LA.

Does John Jay give out Masters and PhDs in Computer Science?

Google say no, unless you are into Criminal Justice. Oh, but wait, CS students are still in the Math Department. http://jjay.smartcatalogiq.com/en/2016-2017/Undergraduate-Bulletin/Cours...

That changed at most major colleges circa 1979.

PD is -under his skis-.

Silver2K | September 15, 2017

I'm going to explain this to everyone like you're 2 years old.

When we drive we use our eyes to evaluate situations and respond accordingly. We don't need to see 5 miles down the road and we don't need sensors or sonars. Elon believes that he can do this with cameras only and he's right. The cameras see something like our eyes do, feed it to the car like we feed it to our brain and respond.

Silver2K | September 15, 2017

sorry but my response was to the OP's tittle "Why Tesla Is Betting On Cameras For Full Self-Driving"

I did not read the article until after my post due to pure laziness :)

Madatgascar | September 16, 2017

@Silver - right. Everyone is dissecting the seeing part. I'm questioning the braining part. LIDAR vs cameras - no idea. I have zero expertise here, I'm just an end user bemusedly watching the evolution of AP.

My esteemed forum friends seem to think the challenge is akin to landing a rocket on a barge. Am I the only one who thinks there are infinitely more variables involved here? Either Elon is missing something or I am overthinking the whole driving business. Over a year ago he already said AP1.0 was better than a human driver. Yes, AP PLUS an attentive human driver may be statistically better than a human driver alone- HUGE difference. I cannot count the number of times I have had to save AP's ass. If we are extrapolating AP success to FSD and telling people they can go to sleep, we are in for some rough seas. You don't need a physics degree to see it coming.

TeslaTap.com | September 16, 2017

@Madatgascar - great points - I hadn't thought about the AP1 + driver being better than driver alone as far as stats go!

I do think FSD software is far more difficult than landing a rocket on a barge :)

Pungoteague_Dave | September 16, 2017

@SamO "@PD has been wrong about Tesla's self driving program both early and often. Even Dave would have to admit that his understanding of Tesla's program, especially with regard to who between Tesla and Mobileeye's, was designing Tesla's software was 100% wrong. I think he called it "bolt on" if I recall. But Dave is awesome about owning up to his errors in judgement."

I've been wrong about a LOT of things, but generally not AP capability. I was one of the very first P85D orders, within a minute of website loading. Despite that, I stated, from a hospital bed after open heart surgery five hours prior, on this forum, exactly what I thought the sensor/camera suite could do and what it could not based on then-available sensor products on the shelf. With the partial exception of summon, which does not do what was promised (but I said the car would likely never drive at all unattended, clearly proven wrong to my delight), there is not one thing about AP1 functionality on which I have been incorrect. I also consistently stated that I HOPED that I would be proven incorrect. I have not been, with the noted exception.

And the "bolted on" element being wrong? Seriously? Why do you think there is no AP2 parity yet? Tesla is using largely the same camera/sensor suite, but cannot get the software up. If ANYTHING has been proven correct by the AP2 debacle, it is that I was 100% correct that most of the AP functionality was bolted on, purchased from Mobileye. I ALWAYS said that Tesla's software integrated it on the front end, but the core functionality (seeing and reading signs, seeing and identifying adjacent cars, understanding relative speeds, virtually all inputs on lane keeping, etc., were bolted on and Tesla integrated the analysis and outputs to steering. braking, and other firmware/hardware.

Pungoteague_Dave | September 16, 2017

@Madatgascar 'If we are extrapolating AP success to FSD and telling people they can go to sleep, we are in for some rough seas."

Bit isn't that EXACTLY what we are doing here? FSD in Elon Musk's world means that you can put your kid in the car with no driver and have it drop them off at school. It means putting the car into the Tesla mobility (Uber-like taxi) program and have it drive around with paying passengers, and no driver. That has been specifically promised for every AP2+ car with FSD. It means that Tesla can summon the car to a service center for service, with no human at all. So, yes that is what FSD entails.

Pungoteague_Dave | September 16, 2017

@Silver2K "I'm going to explain this to everyone like you're 2 years old. When we drive we use our eyes to evaluate situations and respond accordingly. We don't need to see 5 miles down the road and we don't need sensors or sonars. Elon believes that he can do this with cameras only and he's right. The cameras see something like our eyes do, feed it to the car like we feed it to our brain and respond."

Notwithstanding the insult, no camera has anywhere near the capacity of the human eye. Not in the same universe, much less ballpark. Muscles movement alone makes human eyes way more capable and aware of priorities. No camera has the capacity to move independently, except those gimbaled on things like drones, and those are in infancy, certainly not being proposed for cars, and definitely not in AP 2.0 or 2.5 cars.

I love your statement "Elon believes that he can do this with cameras only and he's right." On what do you base your analysis that he's right? Your systems expertise? His stellar track record of delivering on promises? Like AP parity by year end last year? His own engineers disagree - the people who led the damn program and then gave up in frustration - saying that Elon is trying to do dangerous things with tech that's not ready. Any response that they're disgruntled so shouldn't listened to is pure fanboism.

Pungoteague_Dave | September 16, 2017

@NKYTA "Sarcasm SamO? You? ;-) ;). I'm guessing here, but I imagine the only engineers PD has worked with, don't live in SV or LA. Does John Jay give out Masters and PhDs in Computer Science? Google say no, unless you are into Criminal Justice. Oh, but wait, CS students are still in the Math Department. ttp://jjay.smartcatalogiq.com/en/2016-2017/Undergraduate-Bulletin/Cours... That changed at most major colleges circa 1979. PD is -under his skis-."

Wow, the incorrect data flies. Before retiring, I employed with thousands of engineers over the years in various capacities, including software, creating and developing cutting-edge, and more pedestrian retail projects. We were the first, for example to do in-mall video advertising - and yeah, it was cutting edge at the time. Were the first to use high-resolution cameras for vandalism mitigation. We used animatronics in several locations. And some were in LA and SV. Large projects that you would know about in Santa Monica, Downtown LA, Ontario, Orange, San Jose, San Fran, San Mateo, are projects that I was involved in developing or financing and had or have an ownership interest, with many hours on the ground at each (and dozens of others in CA, including several factory outlet centers). Every Mills project in the country has my fingerprint on it somewhere. I have hired and supervised software development teams, mostly for business software, but also for financial analysis. Tease me all you want, but you do not know me.

And where the heck do you get John Jay? MY prior reference to JHU was because I am an adjunct professor at John Hopkins University (since 1993). We own the Applied Physics Lab, the Whiting School of Engineering, and developed and run things like the Hubble Space Telescope, the Van Allen Probes, New Horizons (investigating Pluto), Messenger, and many many others. No institution is close to JHU in terms of developing and deploying cutting edge sensing technology, and that's on top of their medical robotics and technology expertise, which is unequalled in the world. Not that I have know most of those folks, but I do know many, and have had some drive my car (which attracts techies on campus) and discussed what's possible in terms of FSD. They aren't Elon Musk, but they are true geniuses.

Among the programs at the Whiting school, where I know a few of the professionals, are the Lab for Computational Sensing and Robotics, SMARTS Lab (Sensing, Manipulation, and Real-Time Systems Laboratory), ASCO Lab (Autonomous Systems, Control and Optimization Laboratory), and the DSCL (Dynamical Systems and Control Laboratory) Lab. One of their senior professors, a relative, christened my car Mr. Magoo.

From the lab's description: "The Autonomous Systems, Control and Optimization Laboratory (ASCO), aims to develop intelligent robotic vehicles that can perceive, navigate, and accomplish challenging tasks in uncertain, dynamic, and highly constrained environments. The lab performs research in analytical and computational methods for mechanics, control, motion planning, and reasoning under uncertainty, and in the design and integration of novel mechanisms and embedded systems. Application areas include mobile robots, aerial vehicles, and nano satellites."

I do not claim to know jack beyond layman's observations about this type of science. I was a simple builder/developer/financier and investor in large-scale retail projects, and now smaller scale. I do, however, get out, and do know a lot of experts in these fields. And they say Musk is over his skis. I do not know one expert who believes the 2.0 or 2.5 cars will provide FSD under all driving conditions and locations except Musk. Even his lieutenants are not out there in public supporting his exaggerated view. They will talk about battery technology, but not self-driving. Because they do no know what they do not know about this technology, which is in its infancy. You may believe him, but it is an act of faith, not reason.

I do not believe that even Elon Musk knows whether the currently-sold technology can do what he says. Like Trump, he sets out a marker and watches what happens, then modifies his public stance, or simply ignores his prior promises. Yet, there it is in the marketing materials. The very definition of P.T. Barnum. Anyone who pays extra for FSD is being suckered in my opinion - not necessarily in a negative way. Hell, I have paid for Elon promises that I knew at the time would never be delivered. Still love my car, greatest thing I've ever owned. But I live in the world of reality, so haven't been disappointed when not receiving what was promised because I never expected it in the first place - I feel that I received value for payment. Still a Tesla fan, but not buying BS.

carlk | September 16, 2017

Silver2K
"Elon believes that he can do this with cameras only and he's right. The cameras see something like our eyes do, feed it to the car like we feed it to our brain and respond."

So true but so hard for some to understand. You connect LIDAR to a bad driver he/she will still be a bad driver. The article I put linked in a post above showed current cameras sensors are already better than human eyes in pretty much every aspect. The rest is just brain power and training, or machine learning in the case of self driving cars. The problem with LIDAR is you are at best only able to put it in a few hundred test cars to train the machine. Elon's approach allows the machine to learn hundreds or thousands times faster NOW. There is no doub in my mind Tesla will get there before anyone else does.

SbMD | September 16, 2017

@PD - you are missing a simple fact: there are experts who believe that autonomous driving can be accomplished with cameras. There are others who believe in LIDAR, Radar, Sonar in combination. Your contacts notwithstanding, but they are a small swath of the experts in the field. See some of the articles I posted above, and some of the embedded references within them if you want to learn more. There are others which I have not posted for the sake of some brevity.

In all scenarios, software is the route to success, but there is a hardware component to consider which might offload some of the processing needed. I am sourcing this opinion from research in this area and from visual and auditory processing research for both biological and mechanical systems.

Sidebar: JHU is a fine place, but your claim here is wrong: "No institution is close to JHU in terms of developing and deploying cutting edge sensing technology, and that's on top of their medical robotics and technology expertise, which is unequalled in the world."

I'll refrain from a CV regurgitation, as such actions are really unnecessary, but suffice to say that I am certain that your assessment is myopic here. Say what you want, but consider toning down your rhetoric to less bombastic terms.

carlk | September 16, 2017

PD "I do not believe that even Elon Musk knows whether the currently-sold technology can do what he says. Like Trump, he sets out a marker and watches what happens, then modifies his public stance, or simply ignores his prior promises. "

It can not be further from the truth, about Elon not Trump. Elon is a physicist he always make decisions according to the first principle. He always says he sees a clear path and then set an aggressive goal to get there. You're missing the first part and could only drill on the the second part all the time. Just look at his original secret master plan. Not only he followed through everything he planned but most people now agree that's the ONLY way to do it. Some may still don't get everything he knew from the beginning from long range EV to low cost battery manufacturing to fast charging infrastructure but that's their problem and why they are not Tesla.

Look at what I said in the post above that LIDAR may look better on paper and it's a safe bet because Google/Waymo has been using it, you know no one gets fired because he selected an IBM computer syndrome. But Elon would only look at the most fundamental aspects and build from there. I'm sure he figured cameras make more sense than LIDAR for whatever reasons. As I speculated above using cameras would make the most important part of this whole equation, machine learning, to go much faster. Not just a speculation some AI or computer experts from George Hotz to Chris Lattner, even while he's leaving Tesla, said the same thing that this is Tesla's greatest advantage over the rest too.

SamO | September 16, 2017

@NKYTA,

I'm trying a new tack. ;-)

@PD,

I want to acknowledge that you did say Autopilot 1 would be incapable of Full Self Driving. Except neither Tesla nor I ever made that assertion. :-)

carlk | September 17, 2017

In comparison here is what Ford is thinking.

https://www.yahoo.com/finance/news/ford-invests-michigan-apos-autonomous...

A test ground for autonomous cars? Can you believe how backward this is compares to what Tesla is doing? It's like sending you kids to learn swimming in the classroom while other kids are training in the swimming pool everyday. Play it safe usually is the riskiest move. Those companies have zero chance to compete with Tesla.

Pungoteague_Dave | September 17, 2017

@NKYTA - still trying to figure out your John Jay reference and sarcastic diatribe

TaoJones | September 17, 2017

*twitch*

Pungoteague_Dave | September 17, 2017

@carlk "Those companies have zero chance to compete with Tesla."

And yet Super Cruise has just leapfrogged and schooled Tesla's AP.

georgehawley.fl.us | September 17, 2017

Who beats whom doesn't matter. The point is to stimulate a more rapid transition to BEVs. Tesla seems to be succeeding in that effort. We will know better in a couple of years if they are. The market is so large that Tesla will be unable to meet demand, even with effective competition. EAP/FSD? Whatever works that saves thousands of lives are year. That's the holy grail.

georgehawley.fl.us | September 17, 2017

Lives a year, that is.

SO | September 17, 2017

@Pungoteague_Dave | September 17, 2017

"And yet Super Cruise has just leapfrogged and schooled Tesla's AP."

Schooled how?

Pages