Software update failed - twice !!

Software update failed - twice !!

The software update on my M3 has failed twice in the past 5 days and has disabled the use of autopilot (no idea why) for which I paid oh-so-many-thousand dollars.

The support rep admitted that there was a bug in the update and there's no ETA for a fix yet. Any decent software writer would always have a rollback option in case something goes wrong. Not so with Tesla engineers !! Adding to the frustration is the fact that you cannot track your case status online. You always have to call and be on hold for 15-20 minutes just to be able to speak with someone, who doesn't have a clue about what's going on and just repeats a scripted answer.

I can't see how Tesla can rise up to be a world class motor company it's striving to be, with this kind of service. It makes good cars, but that's just about it.

Horrible support and incompetent developers. Sorry.

Tijodallas | 11 septembre 2018

I had the same issue, call the local Service center and have them pushed out to your car. It took three times for me.

lao | 11 septembre 2018

Same here, called the delivery team, and told that the car needs to update the software. In fact, it's not the first time to have reason to postpone the delivery, was told that it needs time to polish, then a seatbelt issue, now is the software update... I'm really fed up with the way they treat customer.

Atoms | 11 septembre 2018

Tesla service had to try 5 times before it loaded. They were ready to give me a loaner.

Yes, Tesla software engineers on the non-drivetrain side do not understand concepts of fault recovery, fault tolerance, etc. They are more typical programmers who believe everything goes just exactly as specified in the software specs. Such programmers create systems which are sporadically unstable due to lack of fault tolerance.
I had the backup camera go black on me. You would think that the UI software would detect no signal and force a reset of the camera module and report to Tesla each occurance. Anyway I sent in a suggestion for auto detection and statistics collection. Can only hope the UI software team improves fault tolerance of its programming.

I’ve seen cases of file servers using NFS and the programmers assume that if they wrote to a file, that the file would be there and the data updated. On a distributed non synchronous system, you can never assume that a file written to is actually updated immediately after the write. Timers are the worst hack solution. Really need to use semaphores and other techniques when feeling with interconnected non synchronous system.

gmkellogg | 11 septembre 2018

@Atoms sounds like Elon needs to hire you.

garibaldi | 11 septembre 2018

In software engineering you must assume that everything can fail, and you have to be able to recover from it, or log the error for future reference. I certainly hope they're at least collecting telemetry.

I do wonder what's the most common cause of failure for Tesla, though.

hokiegir1 | 11 septembre 2018

Our update doesn't seem to want to go through, either. We're close to a service center, so hubby is running by to see if they can force it -- or eve if being on their wifi instead of our bad connection will help.

walnotr | 11 septembre 2018

This is kind of like in the early days of computers. You would cringe every time you saw your kid near the keyboard. Only they could come up with the random keystrokes needed to cause the machine to drift off to never never land. They could also find things you may never have thought of.

It is sort of the same thing with all the new owners (I am one) many have no clue of the abilities of the 3 before sitting behind the wheel for the first time. This is obvious by the number of posts here that get the RTFM reply. It is also easy to sit back and proclaim to have the answers without knowing either the coders or the code behind the software. Do you not believe the first and foremost thought in Tesla’s mind is the safety of their vehicles? I have not done the research, but I am confident in my own mind their engineers and coders have pretty good credentials and are aware of what is happening.

I still see the (BETA) on the settings. That makes us all testers. We should do our job and report what we find, good or bad.

Alex_SD | 11 septembre 2018

Same problem here... Six months old car, every update was fine until now... Tesla started seeing failed updates... Need to go to service center, they can't "force" the update OTA... First available appointment is 10 days out...

Black_and_White | 11 septembre 2018

Same problem here ... I have an appointment at the Service Center for Friday. The painful part is that both TACC and AutoPilot are disabled!!!

SCCRENDO | 11 septembre 2018

I have a Model S with dozens of software updates never failing since April 2013. I got my Model 3 in January and starting around June I had a string of software update failures. They told me at the time that there was a partitioning issue with SD card on early Vin numbers (mine is 30XX). The successfully reformatted the card over the air and it was fine for a while. I am on 26.3. Over the past 2 weeks they have pushed multiple updates include a string where I tried to update, it failed and immediately they pushed another. I went through this rinse-wash cycle at least a dozen times. They tried another over the air reformat an hour ago and the follow up update again failed. I am told that 32.5 has a fix for the partitioning issue. My car is going in tomorrow to get a new SD card. Most of the time you get a failure and they say wait for the next update which usually comes in about 48 hours. I had 2-3 incidents in June when it partially updated but failed to update my autopilot so I had a non-functional autopilot until they reformatted my card. Previous ones resolved on the next update. So this is a definite issue but meant to be confined to early vins.

M3BlueGeorgia | 11 septembre 2018

Would be nice to understand why OTAs fail.

I work in an entirely different business and I can assure you that OTAs are super painful.

At home, where my Model 3 gets its updates, I put in a WiFi repeater so there is excellent WiFi in and around where the car is normally parked. Previously it was weak, because of both distance and a brick wall.

jimglas | 11 septembre 2018

I will hold off on my update ...

EVRider808 | 11 septembre 2018

My update failed twice too. Called service center each time it failed. The second time I managed to get mobile service out for this Friday. Remember to be courteous and patient with reps. They are swamped with calls.

EVRider808 | 11 septembre 2018

No AP, TACC, or auto headlight due to failed update.

walnotr | 11 septembre 2018

34.1 finally loaded on the third try. Haven’t driven yet to see if there is noticeable change.

walnotr | 11 septembre 2018

I should add, I parked the car facing the opposite direction so the nose is closer to my router for the third try.

rxlawdude | 11 septembre 2018

Like @SCCRENDO, we have an early VIN (138x). But we have not seen any issues with updates so far.

billlake2000 | 11 septembre 2018

Your nose knows

MMKB | 11 septembre 2018

How did you get it to try again. My just says to contact Tesla support

thecakeisalie | 11 septembre 2018

I also had a failed update last night, version 2018.32.2 I think, but the version number was updated and I do have new features that were added between the original version and the failed update. Haven't tried driving yet to see if anything actually broke. Going to try tonight to see if I also need to call support about it.

SCCRENDO | 11 septembre 2018

They usually send another update in 48 hours. But if it says contact Tesla support I would contact them and they will get the software team onto it. If the version updates then it is a partial failure where not everything was updated and pushing the version again will likely fix it.

asif_alvi | 11 septembre 2018

I just wish there was a better way to rollback a failed update and/or retry automatically. One shouldn't have to call in (and be put on hold till eternity - exaggeration intended) to get the update again.

Also, online status and opt in/out capability would be truly useful.

SCCRENDO | 11 septembre 2018

If the update failed a new one will automatically come. If you lost your autopilot or it continues to fail you should call them.

EVRider808 | 11 septembre 2018

After 4 tries, software finally updated to 34.1. Even with these hassles, Tesla is still the gold standard for EVs!

hokiegir1 | 11 septembre 2018

I think we're on try #7 or so. Autopilot is gone, as is traction control and auto high beams. We have a persistent "contact service" notification. After an hour on hold (not an exaggeration), they are supposed to be updating us remotely -- where they will start it and everything. We asked about plugging in and they said to do we are currently charging. Hopefully they can either get it to work or roll it back before tomorrow.

SCCRENDO | 11 septembre 2018

@hokiegirl. They will take care of you within 48 hours. They say they need up to an hour but it is far less. In my case 10 mins. You need to just give them a time when your car is not being driven. They reformat the SD card and then send an update immediately. If it doesn’t work you may need to take it in like me. But they seem to have early appointments. I called them on Sunday and got an appointment tomorrow (Wednesday) at Costa Mesa an extremely busy SC

Black_and_White | 11 septembre 2018

My update failed last week and have not had EAP and TACC for almost a week now. I had multiple calls with Tesla Support.
Finally, got an email this afternoon from Tesla Diagnostic Support that they were able to remotely resolve the issue and a new software package was sent to my vehicle. I came home and updated and the install was successful.
At 34.1 now.

thecakeisalie | 11 septembre 2018

Update: car seemed to work despite the message that it couldn't start (I was able to put it in reverse and move a couple inches). After I locked it and called service, they had me try the car again, but it was dead other than normal MCU functions and the same errors about an update being required and to call service. Having them try an update again remotely tomorrow. Hopefully there's no actual hardware issue.

nwfan | 12 septembre 2018

In August I attempted software update to my 3.
It failed miserably. Car would not charge, operate
and release the rear brakes. Had to be loaded on a
flatbed and transported to nearest SC.

Had SD card fail, it was replaced, reformatted and 12 volt battery
replaced. Know issue that needs to be fixed.
Since then, 2 more OTA have come across and both loaded without
any problems. On 34.1

nwfan | 12 septembre 2018

Early Model 3 vin 83xx

Ron.Olsberg | 12 septembre 2018

The firmware is/should be digitally signed which means that you SHOULD not even get a notification of a new firmware version available for install if the OTA download failed because of a bad network connection or any other reason. This signature should also be checked during the actual firmware update process and a hash of the newly installed firmware should be checked BEFORE switching over to it. I think they store the new firmware on SDCARD maybe many versions? Even if the SDCARD fails for any reason, it should not cause a partial firmware load or firmware update problem; however, can't fix bugs that are not caught during regression testing. Bottom line, if there is a problem with the SDCARD you should get an error indicating such and firmware updates might be impossible until the SDCARD problem is rectified. Now if they are using the SDCARD to store the actual boot firmware and are booting directly from it that is an entirely different problem (I hope they are NOT doing this!). Nand Flash on SDCARDS is not nearly as reliable as soldered on flash or SFF (small form factor) SSD's (solid state drives). SDCARDs are designed to be cheap and therefore are not the most reliable.


hokiegir1 | 12 septembre 2018

They were able to remote in last night and do something. We think they were trying as we were driving home from my sister in law's house at about 9:30 (heard a random chime that didn't relate to any screen info), but when we got home about half an hour later, plugged in, it charged up (yay! That was our first on the new charger), and they did their thing, finishing around 1am. Hubby went out and checked on things and unplugged the car -- all seemed good then, and no issues when we drive me to work today.

pko.888 | 12 septembre 2018

So does the service center resend the update or does it have to be some other tech support? My service center told me someone else from support would call me back to help.

Kalpasemasinghe | 12 septembre 2018

My update failed yesterday for the first time since buying the model 3. I called the service center and they said to connect to wifi and wait through the week for re-attempts that will occur. It just sucks that my autopilot has been deactivated due to this failure

madkim23 | 12 septembre 2018

An OTA update failure is a rare experience. I have had an S for more then 2 years and my 3 since January. The 34.1 update is the first software failure I have expeirenced. If you continue to get failures after retries, you should be able to get mobile service to come out and address it or take to the service center. They reformatted my SD card and that corrected the problem for me this morning.

pko.888 | 12 septembre 2018

So the service center ended up having to reformat my sd card before the update would work. For those of you told to wait and see, I would suggest if it fails again call and get a more proactive technician

asif_alvi | 12 septembre 2018

Just received the following from Tesla Support :

"I wanted to touch base in regards to your firmware update failure. We have resolved the issue and have pushed another update to the vehicle. Your vehicle is now ready to schedule a firmware update. Please be sure to schedule the update at a time you do not anticipate needing to drive or charge the vehicle. If there are any issues please feel free to respond directly to this email and I will follow up with you directly for next steps."

Will try the upgrade in a short while..... fingers crossed.

asif_alvi | 12 septembre 2018

Third time was a charm !! Update completed.... phew.

Magic 8 Ball | 12 septembre 2018

All this drama with the "horrible support and incompetent developers". Phew!! I am glad this one is over.

rxlawdude | 12 septembre 2018

Well, I spoke too soon. The update failed yesterday. Tesla will not push an update. Say we must wait 24-48 hours. :-(

thecakeisalie | 12 septembre 2018

Whew, 2018.34.1 that they pushed has fixed my car. Updated pretty fast, too - probably not a huge change from 32.2 that originally bricked it.

fishheadted | 12 septembre 2018

For what is is worth I have been on 32.2 for many weeks without any issues. Trolling the forums here it seems I am one of the first to get updates as after getting an update and looking around I see few others on my version. I am in So Cal and have a car made in May 2018 with a 26XXX VIN. Today I got prompted for an update, presumably 34.1, and after reading this thread the last couple of days will be holding off.

SCCRENDO | 13 septembre 2018

Update, They kept my car for 24 hours and did my service bulletins. However they did not have an SD cards in stock. So why did they bring me in if they did not have cards. They knew they would have to replace it. I guess there is no "Best Buy" in Costa Mesa. So they are going to send a mobile service guy to do it at my home. I have generally been very happy with Tesla Service but this is a royal screw up

rxlawdude | 13 septembre 2018

@SCC, sorry to hear you're having that tsuris.

SCCRENDO | 13 septembre 2018

My wife has my car and they are tormenting me. Just sent me another update. I am going to call the central team. They seem overwhelmed and confused at Costa Mesa

spuzzz123 | 27 septembre 2018

Happened to me on Sunday. I made a service center apt for Thursday. They proactively called me Tuesday while I was at work and forced an ota update. Fixed before I drove home Tuesday!

Lbrooks51115 | 27 septembre 2018

I have had the car for 3 weeks and have never seen an update. Is that normal? From I read in the manual it seems to indicate that I would need to enable the update to happen and I should also be notified when one is available. Is there a way I can check to make sure I am on the correct version?

spuzzz123 | 27 septembre 2018

LBrooks ok so my tenure is all of 6 months so take it for what its worth. But yes I've gone sometimes 6-8 weeks without an update. Other times I get two updates in the same week (I'm thinking about the brake OTA update in the spring).

You can see what version you are on in your phone app (or in the car). Mine is on 2018.36.2 but not sure if that is the most recent update or not. Assuming you have enabled OTA updates in your settings, the version you are on is the "correct" version for you. If there is something urgent or safety-related, Tesla will make sure you get it quickly.

The last several updates I got were simply labeled 'minor bug fixes.' The last significant feature update I got was auto high beams I think. If you have that, you probably aren't missing anything functionality-wise

The release everyone is waiting for is what they are calling release 9 or maybe EAP version 9? To my knowledge no one has it yet -- but it supposedly comes with significant autopilot updates, navigation interface changes, maybe a dashcam, and believe it or not video games.

kirkmiller | 6 novembre 2018

Version 9 software upgrade not installing after a number of attempts. This has caused charging issue with Model S. I am now being told to come in for a $150 diagnosis to be performed, warranty is up. Don't understand why i have to pay for this.

bblevis | 8 novembre 2018

I have a long Service history in Fremont of the error with Fremont Service helping me diagnose and I can say the error is a “hard nut” for Tesla Engineering to crack because it is both intermittent, has latency, and appears specific hardware dependent possibly certain VINs more susceptible. I have evidence the failure is always in the “handshaking” stage before download to car has even started... 2nd seqeuncial attempt has always worked if the second attempt happens immediately after a handshaking error in my case always works. If I could find a customer facing friendly Engineer in Palo Alto I’d be happy to loan my car to Tesla while their Engineers use my car to debug. I can supply evidence to show the bug was introduced with a Software update I received in April 2018 possiblly around time that WiFi 5Ghz connectivity hardware and software was rolled out to Model S.