Closed
Description
Hello, I am using joblib.Parallel to parallelize my computations. Parallel is able to print out to stderr the progress of the calculations. This works fine in IPython console where the progress is updated every few seconds, but in jupyter notebook the progress is only printed once the computation has completed.
If you want to try it:
import time
from math import sqrt
import joblib
def my_sqrt(x):
time.sleep(0.1)
return sqrt(x)
joblib.Parallel(n_jobs=2, verbose=11)(joblib.delayed(my_sqrt)(i**2) for i in range(50))
I talked with the joblib devs and they said that it must be a problem in the Notebook since stderr should be printed directly.
It works fine in the Notebook as well if you put n_jobs=1 which just runs it serially so it seems like Notebook doesn't update stderr when there is threading.
I would appreciate any insight :)