Linewise evaluation with async queue

[lionel-]

This might be an alternative to tagged prompts (#1008) that is potentially more robust with a better console output.

The idea is to send inputs linewise to R, and wait for output with a very short delay. If we see a trailing prompt (continuation or otherwise) after that delay, send the next line immediately. Otherwise, this means the expression was complete and R is busy computing it. We save the rest of inputs in an accumulation buffer. When we receive new output in the filter, we check for a trailing prompt (continuation or otherwise) and send the next accumulated line if any. If the user sends new inputs but R is busy, this goes in the accumulation buffer. If the user sends an interrupt, we cancel all pending inputs.

This should result in interlaced inputs and outputs like when ess-eval-visibly it t but without the freezes. Actually we might even deprecate this option and only support linewise evaluation.

Could this work? The only issue I can think of is that if an output batch artificially contains a trailing prompt, we'll send the next line even though R is still busy. This will mess up the interlacing of inputs and outputs but shouldn't otherwise have deleterious effects. Also, I can't imagine this will be a common edge case.

[mmaechler]

This is definitely preferable to ESS sending its own newlines, and I think conceptually this is really fine.
Indeed, you are right the robustness should be our primary concern. We need something where ESS/Emacs never locks up (the user should be able to notice when R "locks up", i.e. needs too long, but useR should never be lead to see emacs lockup and confusing that with R lock up).

As we have to deal with scenarios where a very large region is sent to R for evaluation, or a medium size region but one that has some parts that take 2-3, even dozens of seconds, etc, the idea of

• try to expect a quick answer, if we get it ... we get correctly formatted output,
• if not leave it to the R "terminal/console" I/O how the result will look like... but definitely don't spend CPU waiting for R.

is a very good idea ((but possibly very close to what we had done years ago already ??))

Choosing the "very short delay" may be somewhat crucial: We can afford say 500ms on startup (of the R process) trying to measure speed [which may differ from the next R process that is started in the same emacs/ess instance]. I think it may be important to "message" to the user about this delay setting and that (s)he can easily customize it. I think an "optimal" delay is really a tradeoff between user <-> *R* interaction and "beauty" of the *R* process console itself, and e.g. I may want a different delay when "hacking" as opposed to when using ESS interactively when teaching applied statistics.

What I really like about this proposal that we would finally stop doing things in R behind the user's back; which is something I've loathed more and more....

( (and which is still a problem with tracebug: If I debug a basic R function and ESS is using that basic R function "behind" my back .. something which happens to me every few weeks, I'm severely frustrated ;-) but yes, that's for another issue))

[lionel-]

Actually since we can only react to R output in the process filter, we don't need to wait for process ever, we just continually monitor new input coming in the filter to figure out if we should send the next line.

The performance impact on large inputs is a good point that we'll need to assess. I don't imagine it causing too much problem though, since in most cases, the very large inputs are evaluated by source()ing the file.

[mmaechler]

The performance impact on large inputs is a good point that we'll need to assess. I don't imagine it causing too much problem though, since in most cases, the very large inputs are evaluated by source()ing the file.

{{well, once source(*, echo=TRUE) is finally fixed .. ;-) }}
More seriously, the problem is that you assume an ESS user who knows when to use C-c C-l and when do .. what everyone knows how to do : Mark with the mouse and send. There will be many users who'll do the latter all the time... and while it's okay if they are "slightly punished" for that,
we do have to ensure that it won't be a huge hit.

Also, people (including me) like to use

C-c <C-up> runs the command ess-eval-buffer-from-beg-to-here, which is
an interactive compiled Lisp function in ‘ess-inf.el’.

and that may be a large part of an R script... and there I really want it to be fast primarily before "beautiful".

[mmaechler]

>>>> Lionel Henry >>>> on Mon, 25 May 2020 01:05:54 -0700 writes: Actually since we can only react to R output in the process filter, we don't need to wait for process ever, we just continually monitor new input coming in the filter to figure out if we should send the next line.

Yes of course, you're right. One thing we could tune: At this point we can (cheaply) know the size of the chunk that (ess-eval-* ..) is trying to send to the R process and hence we could adapt the monitoring timeout/delay to be "less patient" when there's a large chunk than when there's only a few lines (or few dozen characters; don't know if we'd know both).

[lionel-]

I think with this setup there would be no timeout at all because process filtering is asynchronous.

[vspinu]

This is remarcably similar to ess-eval-linewise with the difference that we use an asyncronuous queue instead of blocking. We do have busy detection inside the filter, and I think it's pretty reliable, so no need to even wait a small amount. My concern is the interrupt, it's basically inserting a customer layer between the user interrupt and process interrupt which might be very reliable. People might even send their own process interrupts directly, or from outside emacs. Can try for sure, but needs more thinking.

[lionel-]

For these cases we could improve this with R 4.0's global calling handlers. If an interrupt reaches our global handler we could send a sentinel output to ESS to empty the queue.

I think if you're sending inputs with comint, you should interrupt with comint as well though.

[lionel-]

Your comment about interrupts makes me realise the fundamental difficulty with this approach: emptying the queue on errors. globalCallingHandlers() will help to reliably detect unhandled errors and interrupts but not for things like invokeRestart("abort") (I guess we would need globalRestarts() to deal with this case). With R < 4.0, we'll need to use regexps for error detection and we'll get false positives in corner cases such as try(stop("foo")). This might get much trickier than I first thought.

Are there any other cases than errors, interrupts, and abort restart where we'll want to empty the evaluation queue? The other cause of longjump I can think of is Q in the debugger but I guess the eval queue will be empty in that case.

[lionel-]

With R < 4.0, we'll need to use regexps for error detection and we'll get false positives in corner cases such as try(stop("foo"))

I think that means we can't use this approach in older Rs and in languages where we can't reliably detect that evaluation has been aborted. So we'll need to maintain fallback evaluation strategies (i.e. nowait and nil) after all.

[vspinu]

Yeh, emptying on error is tricky.

If we decide to go with managing-the-queue direction, I would rather go full blown TCP communication. Then we can pass back and forth messages as we wish. But it's a whole different jar of worms in itself.

[lionel-]

Perhaps worth noting that currently R and ESS are not that careful with multiple inputs that throw errors:

(comint-send-string (ess-get-process) "stop('foo')\nstop('bar')\n")

> Error: foo
> Error: bar

[mmaechler]

Indeed. I did not think that the case of errors should stop "us" (you, @lionel- ) from following this route.
If an error is thrown it is important that R (via ESS) user gets the error message "as usually" in the *R* buffer. If there's an extra > or + in front of the error message, who cares?
And even if some more R code lines are sent after the error (often typically then also leading to errors, because one previous expression did not produce the desired result (say, "variable assignment")), that's no catastrophe. The user will have to see the error (message including call, printed) and see what to do about it. If there's a bit of extra "noise" (extra R code sent) that should not completely confuse the user.

[lionel-]

Just to be clear, I think it's important to have the intermediate prompts, and we should definitely keep them. The only problem with these prompts is that they mess up output of tabular data. Compare the first and second output:

> df <- data.frame(x = 1, y = 2, z = 3)

> df
+ df
  x y z
1 1 2 3
>   x y z
1 1 2 3

Regarding extra +, I think it's worth removing them. First for tidyness of the output. But also I'm using an evaluation mode where the result is printed as #> comments in the R buffer below the evaluated expression. Having the + in these inlined outputs is quite distracting. (I plan to contribute this eval mode at some point, I find it quite useful to keep track of things without going full reproducibility.)

[lionel-]

WIP at lionel-/ESS@lionel-:fix-ns-eval...lionel-:fix-eval

The approach looks promising!

[lionel-]

Challenge. Shiny promises use the top-level event loop to run code. This means we might get output that is never followed by a prompt. This will trip up our busy detection and stall our evaluation queue.

p <- promises::promise_resolve(1)
promises::then(p, ~ message("foo"))

> 
foo

This is not a problem with addTaskCallback() and options(error = ) which execute before prompt.