How multiprocessing terminates the main process and all child processes when the child process catches an exception

Hello, everyone. Recently, we have been using multiprocessing to deal with problems. The approximate code is as follows:

def work(i):
    -sharp
    print("child success")
    pass
    
def throw_error(e):
    raise e
       
def main():
    pool = multiprocessing.Pool(processes=10)
    for i in range(0,5000):
        pool.apply_async(work, (i,), error_callback=throw_error)
    pool.close()
    pool.join()
    print("success")
    pass

when this code is running, if there is an exception in the child process, it will be caught and thrown by the main process, and the whole process will not continue to run but will not end, so it needs to be forced to end on the command line ctl+c. I would like to ask if the main process catches a child process exception and the whole process ends directly, how should the code part be modified?

Dec.30,2021

self-question and answer, quote https://www.cnblogs.com/Tour/.: when creating a child process in the main process, the main process belongs to the same grouping as the child process it created, and the concept of this grouping is called a process group in linux. It is a collection of one or more processes. Processes in the same process group have the same process group ID.
add:

to the method that captures exception
def throw_error(e):
    print(e.__cause__)
    os.killpg(os.getpgid(os.getpid()), signal.SIGKILL)

this not only outputs the error, but also terminates the main process and related child processes


the original program problem lies in: the child process exception try Pool itself has been done for you. Once you call back to throw_error, it means that the exception has occurred. All you need to do at this time is to obtain the exception information alarm or notify the exit. Here, raise e will cause the main process to directly exception, the Pool scheduling is based on the queue, and the whole queue of the main process exception will be blocked by hold, and the program will be stuck there.

as for exiting when a child process is abnormal, in addition to the inelegant way of process group, you can also call kill to achieve elegant shutdown, which is not the core problem. For the closure of parent and child processes, please refer to https://blog.csdn.net/wenzhou.

.
Menu