pydantic causes pickling (in worker multiprocessing) to fail
Bug description
pydantic has an effect on pickling data which is used in the multiprocessing module
Steps to Reproduce:
- Install latest worker with dependencies
- send request to run model on worker
- error returned as below. Appears to fail as soon as data is handled by multiprocessing module
Expected Behaviour:
No error, model run should complete.
Actual Behaviour:
File "/home/sarsonl/repos/em1/ramsis.sfm.worker/worker/resource.py", line 427, in post
p.start()
File "/home/sarsonl/miniconda3/envs/sfm_models38/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/home/sarsonl/miniconda3/envs/sfm_models38/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/sarsonl/miniconda3/envs/sfm_models38/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/home/sarsonl/miniconda3/envs/sfm_models38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/home/sarsonl/miniconda3/envs/sfm_models38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/home/sarsonl/miniconda3/envs/sfm_models38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/home/sarsonl/miniconda3/envs/sfm_models38/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'pydantic.dataclasses._Pydantic_ReservoirGeometryEdges_94865931686688'>: it's not the same object as pydantic.dataclasses._Pydantic_ReservoirGeometryEdges_94865931686688
@schmidni I will talk to you about this and see if pydantic can still be used, or if it has to be removed at least until this issue is fixed in the module.