Auth does not work using cronjobs (cronR)

[schmalte04]

I am currently using googlesheets in production and want to switch now to googlesheets4, but the auth method is giving me a hard time.

Previously i authed through a .rds file where my google token was stored. Using googlesheets4 now i auth via my mail sheets_auth(email="...@gmail.com"). It works fine in RStudio but as soon as i run it in a cronjobs via cronR it fails and give me the following error:

Fehler: Can't get Google credentials.
Are you running googledrive in a non-interactive session? Consider:

• drive_deauth() to prevent the attempt to get credentials.
• Call drive_auth() directly with all necessary specifics.

I tried to use OOB, which didnt worked. I tried to auth via service account and token in a json file. Didnt worked..

[jennybc]

In RStudio, look at the location and content of your gargle oauth cache:

gargle::gargle_oauth_sitrep()

Write a cron job that consists of just that line (no googlesheet4 stuff (yet)) and make sure to send output to a file (or use rmarkdown::render() on it), so you can inspect it after the fact.

Compare the results and I imagine you will get info on why the cron job is not finding the same token cache.

Similar instructions, I imagine, for using a service account token, which requires specifying the path.

In RStudio, check the working directory:

getwd()
file.exists("path/you/are/using/to/the/service/account/token.json")

Create a cron job with just those lines and treat same as above. I suspect you will gain some info on why the service account token can be found at the path in one case but not the other.

OOB is only about getting a token initially. It just creates a regular OAuth2 token that behaves the same as tokens obtained through the usual browser flow.

[schmalte04]

Thanks for the quick support. The paths and location of the token were similar. I did the described auth process again and again and suddenly it works. I currently don't know what the reason has been. Lets see

[higgi13425]

I appear to be having a similar problem. I can render a Rmd or qmd to pull data, analyze, and then hand off to blastula for an automated email. But when I run it as a cron job, the auth fails.

Running this on a Rserver behind a university firewall (I use this at work, or VPN in from home).
The R file and qmd file sit on the server, so they cooperate without a firewall in between them

So far, the following only fail with a cron job - but work fine when I source the R file manually (the email is sent successfully), but my goal is to automate this process.

gs4_auth(email = "pdr.higgins@gmail.com", cache = "~/.cache/gargle/", use_oob = TRUE)
gs4_auth(email = "pdr.higgins@gmail.com", cache = "~/.cache/gargle/")
gs4_auth(email = "pdr.higgins@gmail.com")

When I run
gargle::gargle_oauth_sitrep() %>% rmarkdown::render()
ℹ Reporting the default cache location.
5 tokens found in this gargle OAuth cache:
~/.cache/gargle

email client scopes hash...

---

pdr.higgins@gmail.com gargle-erato 528d412...
pdr.higgins@gmail.com tidyverse-erato ...spreadsheets 54a075e...
pdr.higgins@gmail.com tidyverse-erato ...spreadsheets.readonly 843ef1d...
pdr.higgins@gmail.com tidyverse-erato ...drive 928dfaa...
pdr.higgins@gmail.com gargle-erato ...spreadsheets.readonly ced3782...
Error in Sys.readlink(x) : invalid 'paths' argument

This is the output

I have read @jennybc 's response to @schmalte04 several times, but I am still puzzled as to how to use this output to help me to fix the problem. I have been at this a while, with a variety of attempts, so there are several oauth tokens. Is there a way to :

1. figure out the right one (probably a spreadsheets.readonly)?
2. tell gs4 the right one to use? If there is a right one?
3. remove the unnecessary tokens?

or is the Error in Sys.readlink(x) : invalid 'paths' argument a clue to what is wrong?

I am also getting no response to a related Q on posit.community, so this does not appear to be an obvious forehead-slapping fix.

All help greatly appreciated.

Peter

[jennybc]

Is the output above obtained via a cron job?

[higgi13425]

the output of a successful email with an Rmd report works when I source the R file manually on the RStudio server.

The R file looks like this

library(blastula)
setwd("~/data_space/phiggins/R/cronR_scripts")

generate the email OBJECT using blastula

my_email_object <- render_email("blastula_email.Rmd")

smtp_send(my_email_object,
from = "higgins-server@outlook.com",
to = "phiggins@med.umich.edu", "higgi13425@yahoo.com",
subject = "weekly report email",
credentials = creds_file("outlook_server_creds"))

print(paste("script ran at", Sys.time()))

However, when I run it as a cron,
and use gs4_auth(email = "pdr.higgins@gmail.com")
this does not work, because the auth fails.

When I switch this to
gs4_deauth(), the cron job works.

within the crontab, the path is listed as:

PATH=/app/apps/rhel8/R/4.3.2/bin:/app/apps/rhel8/R/4.3.2/lib64/R/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

I am not sure whether the problem is with the gs4_auth, or the crontab path, and I am not sure how to figure this out.

I tried to sort this by running gargle::gargle_oauth_sitrep() %>% rmarkdown::render() manually in the console, and got:

ℹ Reporting the default cache location.
5 tokens found in this gargle OAuth cache:
~/.cache/gargle

email client scopes hash...

pdr.higgins@gmail.com gargle-erato 528d412...
pdr.higgins@gmail.com tidyverse-erato ...spreadsheets 54a075e...
pdr.higgins@gmail.com tidyverse-erato ...spreadsheets.readonly 843ef1d...
pdr.higgins@gmail.com tidyverse-erato ...drive 928dfaa...
pdr.higgins@gmail.com gargle-erato ...spreadsheets.readonly ced3782...
Error in Sys.readlink(x) : invalid 'paths' argument

The last statement makes me think that this is a path conflict/problem, but I don't know what to do next to debug and fix.

If I can figure this out and make it work, it will be pretty awesome (for my level)

thanks,

Peter

[jennybc]

I'd like to see

I tried to sort this by running gargle::gargle_oauth_sitrep() %>% rmarkdown::render() manually in the console, and got:

the result of ☝️ run via a similar cron job. Also throw in print(getwd() and print(normalizePath("~/")) for good measure. You need to get data on the context that's in effect when you're cron job is running.

[higgi13425]

OK, results in hand...

print(getwd))
returns
[1] "/nfs/isilon/path-ibdx/phiggins/R/cronR_scripts"
which makes sense, that is where the R script and the qmd file live

in contrast,
print(normalizePath("~/"))
returns
[1] "/home/phiggins"

I am guessing that this difference is part of the problem.

Would it make sense to change the PATH statement in the cron job to

PATH=/home/phiggins
instead of
PATH=/app/apps/rhel8/R/4.3.2/bin:/app/apps/rhel8/R/4.3.2/lib64/R/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

• the latter only works with gs4_deauth(), which only works sometimes...

[jennybc]

In a cron job, what does gargle::gargle_oauth_sitrep() show?

[higgi13425]

When run within a cron job, I get back

ℹ Reporting the default cache location.
5 tokens found in this gargle OAuth cache:
'~/.cache/gargle'

email client scopes hash...

---

pdr.higgins@gmail.com gargle-erato 528d412...
pdr.higgins@gmail.com tidyverse-erato ...spreadsheets 54a075e...
pdr.higgins@gmail.com tidyverse-erato ...spreadsheets.readonly 843ef1d...
pdr.higgins@gmail.com tidyverse-erato ...drive 928dfaa...
pdr.higgins@gmail.com gargle-erato ...spreadsheets.readonly ced3782...

[jennybc]

OK thanks. My interpretation is "same as he saw interactively". Do you agree?

Can you also show me the output of gargle::gargle_oauth_client_type() in an interactive session vs. in the cron context? Still hunting for what the different is.

[higgi13425]

Yes, they appear to be the same.
now for oauth client type.

Interactive:
gargle::gargle_oauth_client_type()
[1] "web"

cron
gargle::gargle_oauth_client_type()
[1] "installed"

which are clearly different.

I am also noticing a what I think is a change in

print(getwd()) - which gives me
[1] "/nfs/isilon/path-ibdx/phiggins/R/cronR_scripts"

while
print(normalizePath("~/")) gives me
[1] "/home/phiggins"

I'm hoping that these differences are helpful clues.

thanks,
Peter

[jennybc]

OK we are getting somewhere!

I had noticed the difference in home directory, so you will want to be extra deliberate and explicit when specifying cache here: gs4_auth(cache = "~/.cache/gargle/") because ~/ isn't necessarily going to expand to the same thing. If you are expecting to obtain and cache a token in one context (interactive work as yourself) then rediscover it in another (non-interactive work via cron job), this is an easy place to goof things up.

But the more meaty problem is the mismatch of oauth client type. If you are hoping to pull a token from the cache, we need the client type to match or else there will be a cache miss. I assume you get and cache your token in a context corresponding to "interactive" above, where the client type is "web". So you need to make sure that the workload executing via cron job also uses a "web" client type. In the code run via cron, add this line before any auth happens:

options(gargle_oauth_client_type = "web")

[viswaduttp]

just opening it to report my observation as I have struggled with it and left it for almost 2 years before finding a work around.

so we generate token in an interactive way and for cron job we have to read the token from the path (~/.cache/gargle/name_of_token_file) using readRDS(token_path) and pass it to the gs4_auth(token = read_token) - then whole thing works.

The auth inexplicably fails if i give the correct cache path

Taking cache location from the"gargle_oauth_cache"option. 1 token found in this gargle OAuth cache: --gives cache path here-- -- token details here--- tryingtoken_fetch()tryingcredentials_byo_oauth()Error caught bytoken_fetch(): inherits(token, "Token2.0") is not TRUE

If i read the token, it says that the token is of Token2.0, so not sure why this happens.