[c404-063.stampede2.tacc.utexas.edu:22558] [[INVALID],INVALID] ORTE_ERROR_LOG: The system limit on number of children a process can have was reached in file ess_singleton_module.c at line 633 [c404-063.stampede2.tacc.utexas.edu:22558] [[INVALID],INVALID] ORTE_ERROR_LOG: The system limit on number of children a process can have was reached in file ess_singleton_module.c at line 172 -------------------------------------------------------------------------- It looks like orte_init failed for some reason; your parallel process is likely to abort. There are many reasons that a parallel process can fail during orte_init; some of which are due to configuration or environment problems. This failure appears to be an internal failure; here's some additional information (which may only be relevant to an Open MPI developer): orte_ess_init failed --> Returned value The system limit on number of children a process can have was reached (-119) instead of ORTE_SUCCESS -------------------------------------------------------------------------- -------------------------------------------------------------------------- It looks like MPI_INIT failed for some reason; your parallel process is likely to abort. There are many reasons that a parallel process can fail during MPI_INIT; some of which are due to configuration or environment problems. This failure appears to be an internal failure; here's some additional information (which may only be relevant to an Open MPI developer): ompi_mpi_init: ompi_rte_init failed --> Returned "The system limit on number of children a process can have was reached" (-119) instead of "Success" (0) -------------------------------------------------------------------------- *** An error occurred in MPI_Init *** on a NULL communicator *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, *** and potentially your MPI job) [c404-063.stampede2.tacc.utexas.edu:22558] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed! Command exited with non-zero status 1 {"realtime":24.73,"usertime":34.05,"systime":45.10,"memmax":123288,"memavg":0}