Personal tools
You are here: Home Forum How to use Salome? Issue with salome library combined with concurrent.futures (multiProcessing)

Issue with salome library combined with concurrent.futures (multiProcessing)

Up to How to use Salome?

Issue with salome library combined with concurrent.futures (multiProcessing)

Posted by Franco Otaola at August 30. 2021

Hello,

I have been trying to find a way to fuse faster a big number of elements. with this I finished trying the multiprocessing library (concurrent.futures) inside my python script.

The script firstly I created without any salome specific library (for debugging purpose). where the script uses multiprocessing for calculation of a sum. this works as entended.

The fonction that is being parallelized is inside a loop as the idea is to apply it in a pseudo-recursive way. 

When I enter the salome libraries (geometry in this case) to fuse objects in different processors at the same time, I have a strange (and problematic behavior), the first time the parallel function runs, it works correctly and it returns the correct list of objects. the issue comes from the second time it runs (as the loop goes back to the function with the list of objects that is being output from the first time.).

when this happens, salome gives an error "concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending."

the strange thing is that if I intentionally stop the script in only one loop it works perfectly.

I am attaching the script without any salome library. and in the first comment the script with the geometry library.

Attachments

Re: Issue with salome library combined with concurrent.futures (multiProcessing)

Posted by Franco Otaola at August 30. 2021

Here is the script with the salome example.

also some things worth noticing:
A. if we change the range(step) to range(1) in line 93 the script will run correctly.
B. if we use concurrent.futures.ThreadPoolExecutor() instead of concurrent.futures.ProcessPoolExecutor() the code (even with the salome libraries) runs for n numbers of times (not limited for 1 as with ProcessPool)

best regards,

franco

Attachments
Powered by Ploneboard
Document Actions