Voice command processing

Voice command processing

Ok, so I got bored and did what any self-respecting nerd would do. I did an experiment.

I went downstairs and woke my car up. I brought my phone so that I could look at my DNS server logs. And I started issuing voice commands. The verdict? The car attempts to process the command. If it can't, it sends the audio to the mothership.

Commands that are listed in the release notes worked instantly. "Dome lights on", " my feet are cold", " set climate auto" worked without a DNS request, while things it had a harder time with, "Navigate to Xanadu, Virginia" generated api calls to

I could be wrong, but a "quick and dirty" experiment seems to indicate that voice recognition occurs on the car.

spuzzz123 | 2019年12月30日

Cool Sarah I had assumed all requests went to mothership since I was experiencing so much flakiness with even basic commands. Siri and google home can’t even interpret/parse a voice command without being connected. I assumed the same for tesla, but I guess more local cpu horsepower in tesla.

Sarah R | 2019年12月30日

I am curious, though, why tapping the Teslacam icon generated two requests to AWS.

spuzzz123 | 2019年12月30日

Hmmm maybe events tesla wants to capture and store. When we tap that it’s usually because something significant occurred. Eg an accident, near accident, crime, etc. could confirm or refute claims that customers make, like “I was in autopilot when the car swerved off the road.” Video evidence combined with detailed log data could serve to insulate tesla legally.

Can you see how much data is sent? (Does it look like video-sized amounts of data?)

Sarah R | 2019年12月31日

I was only looking at the DNS server requests. What you're asking would require a packet sniffer.

jebinc | 2019年12月31日

-@Sarah R

Great nerdy detective work, Sarah. Kinda makes sense that Tesla would implement it this way. I suspect with each subsequent release, more and more commands will be run locally as each release will bring with it a larger command library.

hokiegir1 | 2019年12月31日

That is interesting, since in the past, if there was no LTE, voice commands didn't work -- and that was with a much smaller list of options. And asking for a location makes sense since it has to get the address in order to navigate. It can't possibly store all addresses locally.

wiscy67 | 2019年12月31日

I'm still on 40.50.1 and tried a similar experiment yesterday while in an area of my work that offers no LTE signal. I wanted to see if any voice commands would work with 0 LTE connectivity. I also tried the dome light on command and basic "Call ". The result for me was that no commands were recognized. I believe the window on screen with example commands came up when I pressed the voice (right steering wheel) buttno.

spuzzz123 | 2019年12月31日

wiscy67 - may not be a definitive test. I get that same error all the time while at home, connected to a strong wifi signal, and still have some LTE signal. So maybe your commands are failing due to not having any signal, and maybe its failing for the same mysterious reason I am failing at home. Who knows?

leo33 | 2019年12月31日

Sarah, good experiment. Which firmware were you on, and which hardware do you have? As hokiegir1 said, a network connection used to be required so the local on-car processing might be new.

rxlawdude | 2019年12月31日

@hokie, the 3 actually has an on-board database for "offline address entry." It does not know business or POI names, so you need the actual street address.
Tested this the first day of M3 ownership (12/29/17) when the LTE was glitchy. I'm not sure how well or if nav would calculate Supercharging stops in no connectivity areas.

wiscy67 | 2019年12月31日

@spuzzz123 I agree completely. It definitely requires more testing.

Sarah R | 2019年12月31日

@leo33 2019.40.50.5 on HW3