Skip to content

Commit cb257d2

Browse files
authored
Add warning in a multiprocessing special case (#6830)
Related to discussion #6657 This code at least adds a warning if `set_track_meta(False)` and `multiprocessing_context='spawn'` are used in the same code. However this warning only triggers if `set_track_meta(False)` has been called before the DataLoader has been initialized. I will append some example code where this is not True, still the bug is triggered. Imo this is still a MONAI bug even though in the discussion it was claimed otherwise. The multiprocessing_context='spawn' has unintended consequences and this is only true for MONAI and not for torch. (I believe the problem is that with 'spawn' Python and all the libs get reinitialized and thus _TRACK_META is reset to being True). ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Matthias Hadlich <[email protected]>
1 parent 49a1ae5 commit cb257d2

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

monai/data/dataloader.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,13 @@
1111

1212
from __future__ import annotations
1313

14+
import warnings
15+
1416
import torch
1517
from torch.utils.data import DataLoader as _TorchDataLoader
1618
from torch.utils.data import Dataset
1719

20+
from monai.data.meta_obj import get_track_meta
1821
from monai.data.utils import list_data_collate, set_rnd, worker_init_fn
1922

2023
__all__ = ["DataLoader"]
@@ -88,4 +91,16 @@ def __init__(self, dataset: Dataset, num_workers: int = 0, **kwargs) -> None:
8891
if "worker_init_fn" not in kwargs:
8992
kwargs["worker_init_fn"] = worker_init_fn
9093

94+
if (
95+
"multiprocessing_context" in kwargs
96+
and kwargs["multiprocessing_context"] == "spawn"
97+
and not get_track_meta()
98+
):
99+
warnings.warn(
100+
"Please be aware: Return type of the dataloader will not be a Tensor as expected but"
101+
" a MetaTensor instead! This is because 'spawn' creates a new process where _TRACK_META"
102+
" is initialized to True again. Context:_TRACK_META is set to False and"
103+
" multiprocessing_context to spawn"
104+
)
105+
91106
super().__init__(dataset=dataset, num_workers=num_workers, **kwargs)

0 commit comments

Comments
 (0)