Best way to refresh the auth token for a fully programatic authentication

[SomeDevWeb]

Hi guys.
I thought I would start a discussion topic rather than an issue, as this is not one as such.
So, ever since I started working with this Monzo API, I had trouble understanding and applying the truly automated authentication.
Now, even in this lovely repo, the auth in the examples files is done in two steps and part of that procedure involves clicking links in the browser and waiting for a confirmation link in the email and moreover, you also need to click the "accept" button in the app in order to get the final auth tokens. The problem is actually that the returned tokens expire in about 1 day, so you need to do the procedure again, which completely defies the purpose of any API automation script... :(
I understand there is a way to truly check and refresh the auth token automatically, but I can't put my finger on it in order to implement it in a working/production script.

As a thought, @petermcd, would this file work from the project you gracefully shared previously:
https://github.com/petermcd/monzo-api/blob/v0.1.6/monzo/authentication.py

Maybe a little code example of how that can be implemented in a real script would be great.

Thanks a lot for any support or ideas!

Best regards,
Andy

[SomeDevWeb]

Hi. Sorry for my silly comments or questions, my python knowledge is still quite basic.

So, obviously, there is the auth file already used in this very repo: main/monzo/authentication.py :)

I did a little step forward, I added a detailed error print on the transactions example and I now get "Could not refresh the access token", which at least is more descriptive than 'Failed to retrieve transactions'.

...
except MonzoError as e: 
    print(e)
    print('Failed to retrieve transactions')

Now we only need to find out why that happens and how to make it work... :)

[petermcd]

Hi @SomeDevWeb If authentication was successful but you get that error it would usually be one of the following:

1. Tried to fetch old transaction when the token is older than 5 minutes.
2. Token has expired.

To combat the first issue you should only fetch transactions older than 90 days when you first create the token.

To combat the 2nd, tokens only last for so long, this is of course not convenient when you use an automated process to fetch transactions. Tokens are accompanied by a refresh token. The code will refresh the token when required and can provide the new token details when a new token is created from that refresh token. The following explains how this works:

https://monzo-api.readthedocs.io/en/latest/refreshing_tokens.html

The error you are receiving is potentially that the code automatically refreshed the token and you tried to use the original token details again, the above would get around this.

[SomeDevWeb]

Thanks a lot, @petermcd, that shed a bit more light on this issue!
Still, I tried everything I knew and still I can't retrieve the new token.
Again, sorry my python level is still not too good.

Firstly, as a thought, I see Echo is imported from monzo.endpoints.echo, but in this repo I only found it in the
https://github.com/petermcd/monzo-api/blob/main/monzo/handlers/echo.py
I presume it's the same one.

So this is some sample code I tried to use - store info in a local file and used it from there, but it still says "Could not refresh the access token". It doesn't seem to refresh anything, but probably some of the code is not doing the right thing...

from monzo.authentication import Authentication
from monzo.handlers.echo import Echo

with open('tkn') as f:
    tkn_lines = f.readlines()

access_token = tkn_lines[0]
expiry = int(tkn_lines[1])
refresh_token = tkn_lines[2]

monzo = Authentication(
    client_id=client_id,
    client_secret=client_secret,
    redirect_url=redirect_uri,
    access_token=access_token,
    access_token_expiry=expiry,
    refresh_token=refresh_token
)

# Instantiate our handler
handler = Echo()

#Register the handler
monzo.register_callback_handler(handler)


try:
    who = WhoAmI.fetch(monzo)
    print(who.user_id)
except MonzoError as e: 
    print(e)
    print('failed to fetch whoami endpoint')
    if str(e) == 'Could not refresh the access token':
        print('Retrieving new token...')
        # Instantiate our handler
        handler = Echo()

        #Register the handler
        monzo.register_callback_handler(handler)

        f = open("tkn","w")
        f.write(access_token + str(expiry) + '\n' + refresh_token)
        f.close()

try:
    who = WhoAmI.fetch(monzo)
    print(who.user_id)
    print(who.client_id)
    print(who.authenticated)
except MonzoError as e: 
    print(e)

[petermcd]

Hi @SomeDevWeb Really sorry I never got the chance to properly have a look at the issue you were facing, things have been a bit crazy at work.

Did you manage to overcome your issues or are you requiring assistance?

[SomeDevWeb]

Hi @petermcd and thanks for the reply.
I know exactly what you mean, same here... :)
BTW I really wanted to say a big THANK YOU for your effort because I know full well how it is to try to maintain both a job and contribute to the community projects, so hat off!

No, I'm afraid I abandoned it again, as I didn't find any solution and it was getting so agravating that it wasn't worth it anymore.
See, I'm a midlevel webdev with little python knowledge and I neither need any more python nor monzo research than this and I presume there are a lot of people like me, so I think if we can put together a short code example that clearly illustrates how to renew the token programatically, it would be great for all.

So, I think we should re-clarify a few aspects:

1. The procedure from the examples step1 and 2 - where the script asks you to manually login to the dev console, then approve the request in the mobile app - needs to be done only once to get the access_token and the refresh_token.
2. Then you can use the returned refresh_token refresh it programatticaly. You don't need to redo the first sequence, right?
3. I actually tried direct api calls with various options, but all returned a "401 unauthorized" error:
   {
   "code": "unauthorized.bad_refresh_token",
   "error": "invalid_request",
   "error_description": "Bad refresh token",
   "message": "Bad refresh token"
   }
   I tried putting the values in the URL, keeping them in the body as frorm fields, form-encode, json etc, but everytime I get the same error...
   So here I stopped because in the docs too says pretty much the same thing, so I don't kwow what else I can try: https://docs.monzo.com/#refreshing-access

Actually, if it helps, I also tried doin the whole procedure from scratch, which gave me new access_token and refresh_token and with those ones I could indeed retrieve the accounts for example, but this time when I tried to get the refresh token, I got some other error "URL could not be accessed"...
So I don't know...

[SomeDevWeb]

Wow, I think I got it. So a few thoughts in conclusion:

1. The only way I made it work in the end was with the values only in the "form-encode" format.
2. Those values only work in the body, not as URL/Query parameters, as it appears in the docs...
3. Does that mean that the documentation is wrong? https://docs.monzo.com/#refreshing-access
   Sorry, I'm just pointing out some facts, but personally I find it confusing and annoying that they're called "Parameter" and the example is given in the curl format, as a lot of people are not familiar with that, so maybe it should be called "Query parameter".
   Also, why doesn't it work as described, but as a Body Form-Encode style? Am I doing something wrong or should the docs be changed or will the api be changed and it will work that way in the future?
   All those are quite confusing, not to mention there is no code example about this anywhere I searched.
4. Another little trap that I fell into (not from the API) is after you get the new refreshed token, you only get it ONCE and you have to save it, cause any subsequent requests will return the "401 unauthorised" error cause obviously you are using still the old token :)
   So you gotta be careful to catch and save it and use that newly generated one for any following requests.
   BTW, I used the Thunder client for the tests, but I presume it will behave the same with Postman or any other...
   Anyway, I'll try to put together a code snippet soon, if it might help anyone.

[petermcd]

Hi @SomeDevWeb you are correct the parameters are sent as the body of the request so are encoded when you send them to Monzo.. I agree that the way Monzo document this is a tad clunky.

The refresh token is only given when you initially get a token (and a new refresh token when you use the last refresh token). This is where using the handlers come in handy. For example I have a Django site I have on my local network, I have created a handler that uses Django models to store and fetch those details. To handle the initial logging in I have created a page that asks for the relevant details and carries out the hard work of exchanging the initial secrets for the tokens and refresh tokens.

A lot of the detail here is very much dependant on how they are going to be used so not easy to come up with a solution someone can just pick up and use (although I might tidy up the Django stuff I did and create a plugin).

It is good to hear that you have made some headway.