Support for vehicle command proxy / FleetAPI #546

RichieB2B · 2024-01-20T00:31:58Z

The legacy OwnerAPI will soon be dropped by Tesla. Newer cars will support only signed commands using the Vehicle Command protocol. Pre-2021 Model X/Y will move to the FleetAPI. This PR supports both. For the Vehicle Command protocol it uses the proxy supplied by Tesla.

This PR adds 3 new configuration settings:

teslaApiUrl with as default the FleetAPI URL for North America. If this setting is missing in the config, the old OwnerAPI URL is used. This ensures that TWCManager keep working as-is with existing configurations.
teslaApiClientID is the client_id of the app registered with Tesla. This is needed to refresh the tokens.
httpProxyCert should point to the certificate of the proxy if teslaApiUrl is set to a proxy URL.

Fixes #545

MikeBishop · 2024-01-22T19:31:17Z

Just to clarify, this still requires a public endpoint and developer account for each user, correct?

MikeBishop

Relieved to see that the diff is so small. Thanks for putting this together.

RichieB2B · 2024-01-22T19:38:26Z

Just to clarify, this still requires a public endpoint and developer account for each user, correct?

A Tesla developer account is needed to register the app with Tesla. During this process the public key for the app is loaded from a public website. So there will be a shakeout of home use of the API because this is certainly not for everyone.

RichieB2B · 2024-01-22T19:57:58Z

Yes, this surprised me as well. The Fleet API is basically the same as the Owner API except it needs a token approved by both Tesla (by registering an app) and the user (by using the Tesla app). The Vehicle Command protocol is hidden away behind a proxy which understands the Fleet API. A real elegant solution that makes the transition quite easy.

ChutneyMary · 2024-01-22T23:09:33Z

During this process the public key for the app is loaded from a public website.

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

So there will be a shakeout of home use of the API because this is certainly not for everyone.

Agreed... Although any assistance/guidance would be appreciated to do this with minimum 'exposure'.

ngardiner · 2024-01-23T09:52:56Z

Many thanks for this @RichieB2B

Just one concern around the certificate verification logic. If using the local proxy, self.proxyCert is set to the cert specified. If using fleet api and commented out, it will be set to None. I can't find any definitive explanation of what verify=None will do, it is a little bit too close to verify=False (which disables SSL validation) for comfort.

I think this could be easily fixed with using this instead:

self.proxyCert = config["config"].get("httpProxyCert", True)

I believe Tesla are actively disabling segments of the owner-api as we speak, in particular the vehicles endpoint.

RichieB2B · 2024-01-23T10:00:41Z

You're right. The default should be True not None. I'll push a fix.

https://github.com/psf/requests/blob/96b22fa18c00831656ee4b286bf1c9062459b00a/src/requests/sessions.py#L416-L424

RichieB2B · 2024-01-23T10:15:20Z

verify=None indeed disabled the certificate verification. Horrible idea to mix a boolean and string but it is well documented. Good catch, I totally assumed setting a string variable to None would fail over to the default of using the supplied certificate bundle. This is not the case:

https://github.com/psf/requests/blob/96b22fa18c00831656ee4b286bf1c9062459b00a/src/requests/adapters.py#L249-L274

RichieB2B · 2024-01-23T10:36:41Z

None does seem to enable certificate validation after all, see psf/requests#6624

RichieB2B · 2024-01-23T10:48:00Z

It seems on the Owner API vehicles endpoint has today been disabled by Tesla. TeslaMate found an easy fix already. This will help a lot of TWCManager users that are not developers. I'll make PR for it.

RichieB2B · 2024-01-23T10:56:48Z

During this process the public key for the app is loaded from a public website.

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

No, you just need to host the generated public key (a text file) on a public website. There is absolutely no need to run TWCManager on the public internet.

So there will be a shakeout of home use of the API because this is certainly not for everyone.

Agreed... Although any assistance/guidance would be appreciated to do this with minimum 'exposure'.

I have recorded all the steps during registration. I'll make a write up when I have more time.

ngardiner · 2024-01-23T11:56:12Z

One thing nagging at me is whether we can auto-detect the best URL to use. We'll soon be forced to move all API calls over to this API and having people hardcode their geo DNS entry just feels a bit messy (and who knows what will happen down the track when Tesla changes their minds)

If we were to do it, any thoughts how? I had one or two - calling a geolocation API based on source IP or maybe a DNS lookup that could be used, but an even easier option might be to make HTTPS calls to a list of the API endpoints and see which one responds fastest? Just some thoughts.

RichieB2B · 2024-01-23T12:35:34Z

To use the Fleet API (for older Model S/X without using a proxy) you still need to register an app with Tesla and fill in the teslaApiClientID in config.json. Since this is a manual step anyway, setting teslaApiUrl to the correct region is a small thing to ask.

RichieB2B · 2024-01-23T12:38:07Z

One way to automate setting the correct region URL is to decode the API token using PyJWT.

RichieB2B · 2024-01-23T18:24:57Z

Decoding the token is not very helpful:

>>> token=settings['carApiRefreshToken']
>>> decoded = jwt.decode(token, options={"verify_signature": False, "verify_aud": False})
>>> decoded['data']['aud']
['https://fleet-api.prd.na.vn.cloud.tesla.com', 'https://fleet-api.prd.eu.vn.cloud.tesla.com']
>>> token=settings['carApiBearerToken']
>>> decoded = jwt.decode(token, options={"verify_signature": False, "verify_aud": False})
>>> decoded['aud']
['https://fleet-api.prd.na.vn.cloud.tesla.com', 'https://fleet-api.prd.eu.vn.cloud.tesla.com', 'https://auth.tesla.com/oauth2/v3/userinfo']

RichieB2B · 2024-01-23T18:30:57Z

Or maybe the ou_code points to the region?

>>> token=settings['carApiBearerToken']
>>> decoded = jwt.decode(token, options={"verify_signature": False, "verify_aud": False})
>>> decoded['ou_code']
'EU'

MikeBishop · 2024-01-23T21:35:15Z

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

Agreed... Although any assistance/guidance would be appreciated to do this with minimum 'exposure'.

The device publishing the key need not be the same device running TWCManager, they just need to have the same key. You could host the key on any Internet-facing device or service, so long as the domain name you use resolves to that endpoint. (A dynamic DNS name from afraid.org or similar might be sufficient, for example.) Static hosting of one file isn't exactly a big attack surface.

RichieB2B · 2024-01-23T22:21:42Z

According to their documentation Tesla does the verification on TLD only, not FQDN. I would not recommend using a domain name you do not own. OTOH only you have the private key so there is little chance of abuse.

mshoe007 · 2024-01-25T01:18:00Z

Just to clarify, I would need to set up a public facing website hosted on my Raspberry Pi, open appropriate ports for said website, and hope my home network is not compromised by the onslaught of attacks? Are there any guides you can suggest for users to safely do the above? I'm not sure if I understand how to host a public key.

I set up a developer account (and hosted the public key on my public server) using this tutorial

https://shankar-k.medium.com/tesla-developer-api-guide-account-setup-app-creation-registration-and-third-party-da24aba1bddd

MikeBishop · 2024-01-29T21:03:57Z

I've created an application that can help with this, for anyone comfortable spinning up a Docker container. Anyone willing to take https://github.com/MikeBishop/fleetapi-tokens for a spin and give me suggestions, I'd appreciate it.

ChutneyMary · 2024-01-29T22:36:07Z

I would love to help but I still can't get an account set up on the Developer portal. Has anyone had issues? I usually get 'Can't onboard app'. Any suggestions?

MikeBishop · 2024-01-29T23:00:30Z

When you're trying to register an app, or earlier?

ChutneyMary · 2024-01-29T23:13:04Z

At the screen where the API & Scopes is checked (Profile Information, Vehicle Information, Vehicle Commands etc.). The Submit button results in an instantaneous message - 'Unable to onboard app'.

MikeBishop · 2024-01-29T23:36:14Z

I've found that it rejects certain app names - try something that doesn't include the word Tesla?

ChutneyMary · 2024-01-29T23:57:10Z

Yes, I've given that a go without success. What are you selecting from the options - 'Authorization Code and Machine-to-Machine' or 'Machine-to-Machine only'?

MikeBishop · 2024-01-30T00:52:24Z

That doesn't ring a bell. For scopes, I'm selecting everything. Maybe send a screenshot? If you'd prefer DM, I'm on Discord as @evequefou.

ChutneyMary · 2024-01-30T01:02:04Z

The options - 'Authorization Code and Machine-to-Machine' or 'Machine-to-Machine only' appear if USA is selected as the country. My country selection (Australia) doesn't offer the option.
I've discussed the problem with creating an account on other forums. It seems to be an issue for others as well.
The suspicion is that Telsa hasn't provisioned enough server capacity and/or mis-configured local servers.

RichieB2B · 2024-01-30T07:32:49Z

At the screen where the API & Scopes is checked (Profile Information, Vehicle Information, Vehicle Commands etc.). The Submit button results in an instantaneous message - 'Unable to onboard app'.

When you click submit it tries to download the public key from your website. But I think the error message was different when it is missing.

I have read somewhere the app names must be the unique.

alexeiw123 · 2024-02-04T02:12:21Z

With this merged, are we able to update to latest version and behaviour is returned or are we required do something else in order to restore API functionality?

RichieB2B · 2024-02-04T08:14:27Z

No,this enables the option to use the new Fleet API. To keep using the old Owner API we need to merge #548 as well.

ChutneyMary · 2024-02-16T11:09:59Z

My TWCManager has stopped working for API control and I hope someone could provide guidance on where to go from here...
There are two options in the Tesla Developer portal -

Option 1: 'Authorisation Code and Machine-to-Machine'.
Option 2: 'Machine-to-Machine only'.

Option 1: Requests a domain name and callback URL. I've set up a web-server with the public key com.tesla.3p.public-key.pem being available at my domain. I can successfully test retrieve the key. I get variations of 'Unable to onboard app' or 'Unable to fulfil request, try again later' when I try to set up an account. I've done this dozens of times over a few weeks.

Option 2: I appear to have had success using the option 'Machine-to-Machine'. In the Tesla Developer portal, it provides details of OAuth 2.0 Client ID and Client Secret.

What I don't yet understand is whether the OAuth 2.0 Client ID and Client Secret is sufficient to continue? Is the Option 1 method with public key retrieval absolutely required?

RichieB2B · 2024-03-11T21:31:52Z

@ChutneyMary Here is a nice tutorial that can be used: https://shankar-k.medium.com/tesla-developer-api-guide-account-setup-app-creation-registration-and-third-party-da24aba1bddd

alexeiw123 · 2024-03-31T23:58:03Z

Can someone provide advice on how I can point my TWCManager to a working fleet API that I'm hosting.

I have set up access to fleet API through home assistant and it is now working for the home assistant integration.

I have done a git pull and make install on the main branch but I'm not able to see the following lines in my config.json. I feel that I'm making some basic error...

TWCManager/etc/twcmanager/config.json

Lines 139 to 159 in f4dcb13

    
           # Since 2024 the Tesla Vehicle Command protocol is required for all 
        
           # vehicles manufacturered after 2021. The Tesla Vehicle Command proxy 
        
           # will automatically fall back to the FleetAPI for older vehicles. 
        
           # 
        
           # TWCManager does not implement the Tesla Vehicle Command protocol. 
        
           # Instead it relies on the Tesla Vehicle Command proxy which translates 
        
           # the FleetAPI REST calls if the destination vehicle supports the 
        
           # new protocol. This requires the URL of the proxy to be set as 
        
           # "teslaProxy" and the proxy certificate for validation as 
        
           # "teslaProxyCert". 
        
           # See https://github.com/teslamotors/vehicle-command 
        
           # 
        
           # The URL of the Tesla HTTP Proxy running locally 
        
           # "teslaProxy": "https://localhost:4443", 
        
           # 
        
           # Only set "teslaProxyCert" when "teslaProxy" is set. 
        
           #"teslaProxyCert": "/path/to/public_key.pem", 
        
           # The cloudUpdateInterval determines how often to poll certain 
        
           # data retrieved from the Tesla API to evaluate policy. 
        
           "cloudUpdateInterval": 1800,

RichieB2B added 6 commits January 19, 2024 21:31

Preliminary support for fleet-api and Tesla Vehicle Command proxy

de506f2

Add baseURL to CarApiVehicle class

fc00db2

Use FleetAPI instead of fleet-api

dbef9ce

Don't set httpProxyCert by default

c2e712f

Use VIN as vehicle_tag for commands

cce56df

Use proxy client_id for token refresh

4b9fa1e

RichieB2B changed the title ~~Support for vehicle command proxy / FleetAPI, fixes #545~~ Support for vehicle command proxy / FleetAPI Jan 20, 2024

Clarify teslaApiClientID is also needed for FleetAPI

5bbb92c

RichieB2B mentioned this pull request Jan 20, 2024

Move to Fleet API and Tesla Vehicle Command SDK tdorssers/TeslaPy#150

Open

MikeBishop approved these changes Jan 22, 2024

View reviewed changes

ngardiner self-requested a review January 23, 2024 09:59

Default requests verify to True (not None)

6657ac7

ngardiner merged commit eef63d1 into ngardiner:main Jan 23, 2024
1 check passed

ngardiner removed their request for review January 23, 2024 12:00

RichieB2B mentioned this pull request Jan 23, 2024

Get baseURL from ou_code in bearer token #548

Merged

RichieB2B deleted the vehicle-command branch February 4, 2024 11:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for vehicle command proxy / FleetAPI #546

Support for vehicle command proxy / FleetAPI #546

RichieB2B commented Jan 20, 2024 •

edited

Loading

MikeBishop commented Jan 22, 2024

MikeBishop left a comment

RichieB2B commented Jan 22, 2024

RichieB2B commented Jan 22, 2024

ChutneyMary commented Jan 22, 2024

ngardiner commented Jan 23, 2024

RichieB2B commented Jan 23, 2024 •

edited

Loading

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

ngardiner commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024 •

edited

Loading

RichieB2B commented Jan 23, 2024

MikeBishop commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

mshoe007 commented Jan 25, 2024

MikeBishop commented Jan 29, 2024

ChutneyMary commented Jan 29, 2024

MikeBishop commented Jan 29, 2024

ChutneyMary commented Jan 29, 2024

MikeBishop commented Jan 29, 2024

ChutneyMary commented Jan 29, 2024

MikeBishop commented Jan 30, 2024

ChutneyMary commented Jan 30, 2024

RichieB2B commented Jan 30, 2024

alexeiw123 commented Feb 4, 2024

RichieB2B commented Feb 4, 2024

ChutneyMary commented Feb 16, 2024

RichieB2B commented Mar 11, 2024

alexeiw123 commented Mar 31, 2024

Support for vehicle command proxy / FleetAPI #546

Support for vehicle command proxy / FleetAPI #546

Conversation

RichieB2B commented Jan 20, 2024 • edited Loading

MikeBishop commented Jan 22, 2024

MikeBishop left a comment

Choose a reason for hiding this comment

RichieB2B commented Jan 22, 2024

RichieB2B commented Jan 22, 2024

ChutneyMary commented Jan 22, 2024

ngardiner commented Jan 23, 2024

RichieB2B commented Jan 23, 2024 • edited Loading

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

ngardiner commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

RichieB2B commented Jan 23, 2024 • edited Loading

RichieB2B commented Jan 23, 2024

MikeBishop commented Jan 23, 2024

RichieB2B commented Jan 23, 2024

mshoe007 commented Jan 25, 2024

MikeBishop commented Jan 29, 2024

ChutneyMary commented Jan 29, 2024

MikeBishop commented Jan 29, 2024

ChutneyMary commented Jan 29, 2024

MikeBishop commented Jan 29, 2024

ChutneyMary commented Jan 29, 2024

MikeBishop commented Jan 30, 2024

ChutneyMary commented Jan 30, 2024

RichieB2B commented Jan 30, 2024

alexeiw123 commented Feb 4, 2024

RichieB2B commented Feb 4, 2024

ChutneyMary commented Feb 16, 2024

RichieB2B commented Mar 11, 2024

alexeiw123 commented Mar 31, 2024

RichieB2B commented Jan 20, 2024 •

edited

Loading

RichieB2B commented Jan 23, 2024 •

edited

Loading

RichieB2B commented Jan 23, 2024 •

edited

Loading