-------------------------------------------------------------------------- By default, for Open MPI 4.0 and later, infiniband ports on a device are not used by default. The intent is to use UCX for these devices. You can override this policy by setting the btl_openib_allow_ib MCA parameter to true. Local host: c404-063 Local adapter: hfi1_0 Local port: 1 -------------------------------------------------------------------------- -------------------------------------------------------------------------- WARNING: There was an error initializing an OpenFabrics device. Local host: c404-063 Local device: hfi1_0 -------------------------------------------------------------------------- [c404-063.stampede2.tacc.utexas.edu:24828] mca_base_component_repository_open: unable to open mca_osc_rdma: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_osc_rdma.so: failed to map segment from shared object (ignored) [c404-063.stampede2.tacc.utexas.edu:24828] mca_base_component_repository_open: unable to open mca_osc_sm: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_osc_sm.so: failed to map segment from shared object (ignored) c404-063.stampede2.tacc.utexas.edu.24828Out of memory for malloc at /build/libpsm2-GHG80G/libpsm2-11.2.86/psm_mpool.c:498 Command exited with non-zero status 255 {"realtime":28.39,"usertime":32.46,"systime":40.75,"memmax":123280,"memavg":0}