Skip to content

加载模型报错ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None. #453

Open
@licamla

Description

@licamla

Describe the bug

from fastHan import FastHan

model = FastHan()
model.set_cws_style('cityu')

model('加载模型报错')

加载模型报错

  File "/lib/python3.10/site-packages/fastNLP/core/dataloaders/torch_dataloader/fdl.py", line 152, in __init__
    super().__init__(**dl_kwargs)
  File "/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 245, in __init__
    raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.

Desktop (please complete the following information):

torch 2.0.0+cpu
FastNLP 1.0.1

Additional context
torch的DataLoader参数num_workers=0时,prefetch_factor需要设置成None。
而fastNLP里的子类TorchDataLoader的默认num_workers=0,prefetch_factor=2。按道理来说TorchDataLoader应该像DataLoader默认prefetch_factor=None

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions