-------------------------------------------------------------------------- By default, for Open MPI 4.0 and later, infiniband ports on a device are not used by default. The intent is to use UCX for these devices. You can override this policy by setting the btl_openib_allow_ib MCA parameter to true. Local host: c401-004 Local adapter: hfi1_0 Local port: 1 -------------------------------------------------------------------------- -------------------------------------------------------------------------- WARNING: There was an error initializing an OpenFabrics device. Local host: c401-004 Local device: hfi1_0 -------------------------------------------------------------------------- [c401-004.stampede2.tacc.utexas.edu:168174] mca_base_component_repository_open: unable to open mca_osc_pt2pt: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_osc_pt2pt.so: failed to map segment from shared object (ignored) [c401-004.stampede2.tacc.utexas.edu:168174] mca_base_component_repository_open: unable to open mca_osc_rdma: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_osc_rdma.so: failed to map segment from shared object (ignored) [c401-004.stampede2.tacc.utexas.edu:168174] mca_base_component_repository_open: unable to open mca_osc_sm: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi3/mca_osc_sm.so: failed to map segment from shared object (ignored) c401-004.stampede2.tacc.utexas.edu.168174Out of memory for malloc at /build/libpsm2-GHG80G/libpsm2-11.2.86/psm_mpool.c:498 Command exited with non-zero status 255 {"realtime":23.42,"usertime":25.81,"systime":31.87,"memmax":113084,"memavg":0}