How to tell solid_dmft to only run wannier90 in serial

[sAimen94]

Dear all,

I have the problem that the (mpi supported) wannier90 version I have installed is very faulty. I do not know what the problem is exactly, it gives me an error message like:

simon@simon-ubuntu:~/solid_dmft.src/doc/tutorials/Ce2O3_csc_w90$ mpirun solid_dmft > dmft.out
Starting run with 6 MPI rank(s) at : 2023-07-05 12:14:54.252289

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0  0x7ff83d64251f in ???
	at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
#0  0x7fd59364251f in ???
	at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
#1  0x562eea3bf470 in __w90_comms_MOD_comms_scatterv_cmplx_4
	at ../comms.F90:1247
#1  0x559a2d9af470 in __w90_comms_MOD_comms_scatterv_cmplx_4
	at ../comms.F90:1247
#2  0x562eea2ffda7 in __w90_overlap_MOD_overlap_read
	at ../overlap.F90:207
#3  0x562eea2a580e in wannier
	at ../wannier_prog.F90:204
#4  0x562eea2a6ed0 in main
	at ../wannier_prog.F90:55
#2  0x559a2d8efda7 in __w90_overlap_MOD_overlap_read
	at ../overlap.F90:207
#3  0x559a2d89580e in wannier
	at ../wannier_prog.F90:204
#4  0x559a2d896ed0 in main
	at ../wannier_prog.F90:55
#0  0x7ff4ec24251f in ???
	at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
#1  0x55bf3fbf3470 in __w90_comms_MOD_comms_scatterv_cmplx_4
	at ../comms.F90:1247
#2  0x55bf3fb33da7 in __w90_overlap_MOD_overlap_read
	at ../overlap.F90:207
#3  0x55bf3fad980e in wannier
	at ../wannier_prog.F90:204
#4  0x55bf3fadaed0 in main
	at ../wannier_prog.F90:55
#0  0x7fd77ba4251f in ???
	at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
#1  0x55cd63f5d470 in __w90_comms_MOD_comms_scatterv_cmplx_4
	at ../comms.F90:1247
#2  0x55cd63e9dda7 in __w90_overlap_MOD_overlap_read
	at ../overlap.F90:207
#3  0x55cd63e4380e in wannier
	at ../wannier_prog.F90:204
#4  0x55cd63e44ed0 in main
	at ../wannier_prog.F90:55
#0  0x7ff4afc4251f in ???
	at ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
#1  0x55dcca4d8470 in __w90_comms_MOD_comms_scatterv_cmplx_4
	at ../comms.F90:1247
#2  0x55dcca418da7 in __w90_overlap_MOD_overlap_read
	at ../overlap.F90:207
#3  0x55dcca3be80e in wannier
	at ../wannier_prog.F90:204
#4  0x55dcca3bfed0 in main
	at ../wannier_prog.F90:55
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 5 with PID 0 on node simon-ubuntu exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/simon/miniconda3/bin/solid_dmft", line 45, in <module>
    run_solid_dmft.main(sys.argv)
  File "/home/simon/miniconda3/lib/python3.9/site-packages/solid_dmft/main.py", line 103, in main
    csc_flow_control(general_params, solver_params, dft_params, advanced_params)
  File "/home/simon/miniconda3/lib/python3.9/site-packages/solid_dmft/csc_flow.py", line 275, in csc_flow_control
    _full_qe_run(general_params['seedname'], dft_params, 'initial')
  File "/home/simon/miniconda3/lib/python3.9/site-packages/solid_dmft/csc_flow.py", line 121, in _full_qe_run
    qe_wrapper('win')
  File "/home/simon/miniconda3/lib/python3.9/site-packages/solid_dmft/csc_flow.py", line 100, in <lambda>
    qe_wrapper = lambda calc_type: qe.run(dft_params['n_cores'], calc_type, dft_params['dft_exec'],
  File "/home/simon/miniconda3/lib/python3.9/site-packages/solid_dmft/dft_managers/qe_manager.py", line 255, in run
    _start_with_piping(mpi_exe, arguments, qe_file_ext, env_vars, seedname)
  File "/home/simon/miniconda3/lib/python3.9/site-packages/solid_dmft/dft_managers/qe_manager.py", line 208, in _start_with_piping
    subprocess.check_call(mpi_arguments + [seedname], env=env_vars, shell=False)
  File "/home/simon/miniconda3/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/simon/miniconda3/bin/mpirun', '-np', '6', 'wannier90.x', 'ce2o3']' returned non-zero exit status 139.

when running mpirun solid_dmft > dmft.out in the Ce2O3 csc tutorial. Is there a way to tell solid_dmft to only execute wannier90.x with one core while still utilizing parallel computation for QE and DMFT cycle?

Thanks and have a nice day,
Simon

[the-hampel]

Hi @sAimen94,
I guess the problem has three ingredients:

1. Quantum espresso builds per default wannier90 in serial mode. So if you are using that version it will be always be in serial
2. the current stable release of wannier90 3.1.0 cannot be build with mpi and more recent versions of gfortran. This is fixed however in the development branch of wannier90: Allocate arrays on non-root nodes to avoid SIGSEG wannier-developers/wannier90#373 , but not in the release. One solution is to just use the develop branch of wannier90
3. in solid_dmft we should allow to use a non mpi version of wannier90 to be used. Let me figure out a way to do this. Either we will add another flag to do this, or automatically detect if wannier90.x is linked against mpi. However, this might take a few days / weeks.

For now I recommend you to either build the develop branch of wannier90 with mpi when using solid_dmft, or manually remove the mpirun in qe_managers.py in solid_dmft.

Best,
Alex

[sAimen94]

Hey and thank you for your quick reply.

I have installed the develop version of W90 now and it seems to work so far.

Best,
Simon