Skip to content

Parallel printing to stderr #700

Closed
ipython/ipykernel
#85
@stefdoerr

Description

@stefdoerr

Hello, I am using joblib.Parallel to parallelize my computations. Parallel is able to print out to stderr the progress of the calculations. This works fine in IPython console where the progress is updated every few seconds, but in jupyter notebook the progress is only printed once the computation has completed.

If you want to try it:

import time
from math import sqrt
import joblib

def my_sqrt(x):
    time.sleep(0.1)
    return sqrt(x)

joblib.Parallel(n_jobs=2, verbose=11)(joblib.delayed(my_sqrt)(i**2) for i in range(50))

I talked with the joblib devs and they said that it must be a problem in the Notebook since stderr should be printed directly.
It works fine in the Notebook as well if you put n_jobs=1 which just runs it serially so it seems like Notebook doesn't update stderr when there is threading.

I would appreciate any insight :)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions