Voice command processing

Voice command processing

Ok, so I got bored and did what any self-respecting nerd would do. I did an experiment.

I went downstairs and woke my car up. I brought my phone so that I could look at my DNS server logs. And I started issuing voice commands. The verdict? The car attempts to process the command. If it can't, it sends the audio to the mothership.

Commands that are listed in the release notes worked instantly. "Dome lights on", " my feet are cold", " set climate auto" worked without a DNS request, while things it had a harder time with, "Navigate to Xanadu, Virginia" generated api calls to

I could be wrong, but a "quick and dirty" experiment seems to indicate that voice recognition occurs on the car.

spuzzz123 | 30. desember 2019

Cool Sarah I had assumed all requests went to mothership since I was experiencing so much flakiness with even basic commands. Siri and google home can’t even interpret/parse a voice command without being connected. I assumed the same for tesla, but I guess more local cpu horsepower in tesla.

Sarah R | 30. desember 2019

I am curious, though, why tapping the Teslacam icon generated two requests to AWS.

spuzzz123 | 30. desember 2019

Hmmm maybe events tesla wants to capture and store. When we tap that it’s usually because something significant occurred. Eg an accident, near accident, crime, etc. could confirm or refute claims that customers make, like “I was in autopilot when the car swerved off the road.” Video evidence combined with detailed log data could serve to insulate tesla legally.

Can you see how much data is sent? (Does it look like video-sized amounts of data?)

Sarah R | 31. desember 2019

I was only looking at the DNS server requests. What you're asking would require a packet sniffer.

jebinc | 31. desember 2019

-@Sarah R

Great nerdy detective work, Sarah. Kinda makes sense that Tesla would implement it this way. I suspect with each subsequent release, more and more commands will be run locally as each release will bring with it a larger command library.

hokiegir1 | 31. desember 2019

That is interesting, since in the past, if there was no LTE, voice commands didn't work -- and that was with a much smaller list of options. And asking for a location makes sense since it has to get the address in order to navigate. It can't possibly store all addresses locally.

wiscy67 | 31. desember 2019

I'm still on 40.50.1 and tried a similar experiment yesterday while in an area of my work that offers no LTE signal. I wanted to see if any voice commands would work with 0 LTE connectivity. I also tried the dome light on command and basic "Call ". The result for me was that no commands were recognized. I believe the window on screen with example commands came up when I pressed the voice (right steering wheel) buttno.

spuzzz123 | 31. desember 2019

wiscy67 - may not be a definitive test. I get that same error all the time while at home, connected to a strong wifi signal, and still have some LTE signal. So maybe your commands are failing due to not having any signal, and maybe its failing for the same mysterious reason I am failing at home. Who knows?

leo33 | 31. desember 2019

Sarah, good experiment. Which firmware were you on, and which hardware do you have? As hokiegir1 said, a network connection used to be required so the local on-car processing might be new.

rxlawdude | 31. desember 2019

@hokie, the 3 actually has an on-board database for "offline address entry." It does not know business or POI names, so you need the actual street address.
Tested this the first day of M3 ownership (12/29/17) when the LTE was glitchy. I'm not sure how well or if nav would calculate Supercharging stops in no connectivity areas.

wiscy67 | 31. desember 2019

@spuzzz123 I agree completely. It definitely requires more testing.

Sarah R | 31. desember 2019

@leo33 2019.40.50.5 on HW3