Compiling IPOPT Mex File with Mumps Linear Solver

[anilvrao]

I am trying to compile an IPOPT Mex file for the Silicon Mac using just the MUMPS linear solver. I can successfully compile the Mex file, but when I try to run it I get the error,

*** Error using Ipopt Matlab interface: ***

Invalid value for IPOPT option "linear_solver"

---

I am running the examples examplehs071 and examplehs038, both which have the linear solver set to "mumps", but I get the above error. In case it helps, I am compiling the MATLAB interface written by Enrico Bertolazzi on an Apple Silicon Mac.

I am confused.

[rdzman]

(It appears this and #724 are essentially duplicate issues.)

I'm afraid I don't have the answer for you. I too tried unsuccessfully to build a working MATLAB MEX interface for Apple Silicon (see ebertolazzi/mexIPOPT#24).

But, it looks like you got further than I did. If you care to share the details of how you are building what you have now, I'd be happy to at least try it and see if I get the same error.

[anilvrao]

I attach the shell script I am using. If you do not want to compile with HSL, you can change the flag "--with-hsl" to "wihtout-hsl" on line 165 and comment out the line "install_hsl" on line 214. It should then compile with just Mumps. If you have the HSL source code, the COIN HSL file needs to be named "coinhsl-2022.11.09.tar.gz" or you need to change HSL_NAME, HSL_TARBALL_NAME, and COMPILE_IPOPT_HSL_VERSION within the script. Everything aside from the HSL library should be self-contained from within the script.

By the way, change the name of the attachment to "compile_ipopt.sh" and set the permissions to ugo+rwx so that it can be executed from the terminal window. This discussion page does not support files with a ".sh" extension.

compile_ipopt.txt

[rdzman]

I got the MEX file to build using your script (with HSL stuff disabled), but I'm running into the following runtime error when calling ipopt() ...

OMP: Error #15: Initializing libomp.a, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

MATLAB is exiting because of fatal error

FWIW, I had previously installed IPOPT via brew install ipopt, so aside from what's been installed in /usr/local by this script, I also have IPOPT and it's dependencies in /opt/homebrew.

[svigerske]

The script patches up the mex build script to explicitly link to /opt/homebrew/opt/libomp/lib/libomp.a (function compile_mex). You could try adapting this to link against a shared omp library (or try not to mention any omp lib there).

[svigerske]

The script looks as if it should include Mumps into the Ipopt build, and then it should be possible to set linear_solver to mumps.
You may want to check whether that works with the Ipopt installation itself (run some of the Ipopt tests or examples).

If you can share also the log of your whole installation procedure, maybe we can say more.

[rdzman]

In my case it looks like it's in /usr/local/lib ...

-rw-r--r--@ 1 root  wheel  14216480 Nov 15 11:04 libipopt.a
-rwxr-xr-x@ 1 root  wheel      1286 Nov 15 11:04 libipopt.la
-rw-r--r--@ 1 root  wheel    313496 Nov 15 11:02 libmetis.a
-rw-r--r--@ 1 root  wheel   1432616 Nov 15 11:04 libsipopt.a
-rwxr-xr-x@ 1 root  wheel      1316 Nov 15 11:04 libsipopt.la

[svigerske]

Yes, you configure Ipopt with --prefix="$COMPILE_IPOPT_INSTALL_PREFIX" and have COMPILE_IPOPT_INSTALL_PREFIX="/usr/local", so the lib is installed into /usr/local/lib.

You can run Ipopt tests (make test) also without having it installed first.

[anilvrao]

Here is the output of "make test":

CXX hs071_main.o
CXX hs071_nlp.o
CXXLD hs071_cpp
ld: warning: -bind_at_load is deprecated on macOS
CC hs071_c.o
CCLD hs071_c
CXX emptynlp.o
CXXLD emptynlp
ld: warning: -bind_at_load is deprecated on macOS
CXX getcurr.o
CXXLD getcurr
ld: warning: -bind_at_load is deprecated on macOS
ln -s ../examples/hs071_f/hs071_f.f hs071_f.f
F77 hs071_f.o
F77LD hs071_f
CXX parametricTNLP.o
CXX parametric_driver.o
CXXLD parametric_cpp
ld: warning: -bind_at_load is deprecated on macOS
CXX MySensTNLP.o
CXX redhess_cpp.o
CXXLD redhess_cpp
ld: warning: -bind_at_load is deprecated on macOS
chmod u+x ./run_unitTests
./run_unitTests

Running unitTests...

Testing AMPL Solver Executable...
no AMPL solver executable found, skipping test...
Testing C++ Example...
0
---- 8< ---- Start of test program output ---- 8< ----
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "/Users/anilvrao/Desktop/ipopt-mex-temp/Ipopt/src/Common/IpLibraryLoader.cpp" at line 67:
Exception message: dlopen(libhsl.dylib, 0x0002): tried: 'libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/usr/lib/libhsl.dylib' (no such file, not in dyld cache), 'libhsl.dylib' (no such file)

EXIT: Library loading failure.

*** The problem FAILED!
---- 8< ---- End of test program output ---- 8< ----

******** Test FAILED! ********

Test program existed with nonzero status.
Testing C Example...
0
---- 8< ---- Start of test program output ---- 8< ----
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "/Users/anilvrao/Desktop/ipopt-mex-temp/Ipopt/src/Common/IpLibraryLoader.cpp" at line 67:
Exception message: dlopen(libhsl.dylib, 0x0002): tried: 'libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/usr/lib/libhsl.dylib' (no such file, not in dyld cache), 'libhsl.dylib' (no such file)

EXIT: Library loading failure.

ERROR OCCURRED DURING IPOPT OPTIMIZATION.
---- 8< ---- End of test program output ---- 8< ----

******** Test FAILED! ********

Test program existed with nonzero status.
Testing Fortran Example...
./run_unitTests: line 21: 69853 Abort trap: 6 $@ > tmpfile 2>&1
0
---- 8< ---- Start of test program output ---- 8< ----
libc++abi: terminating due to uncaught exception of type Ipopt::DYNAMIC_LIBRARY_FAILURE

Program received signal SIGABRT: Process abort signal.

Backtrace for this error:
#0 0x1039bdb07
#1 0x1039bcb53
#2 0x186f8ba23
#3 0x186f5ccbf
#4 0x186e6ca3f
#5 0x186f146d7
#6 0x186f047c7
#7 0x186baf8a3
#8 0x186f13a9b
#9 0x186f16a47
#10 0x186f169f3
---- 8< ---- End of test program output ---- 8< ----

******** Test FAILED! ********

Test program existed with nonzero status.
Skip testing Java Example (Java interface not build)
Testing sIpopt Example parametric_cpp...

---- 8< ---- Start of test program output ---- 8< ----
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "/Users/anilvrao/Desktop/ipopt-mex-temp/Ipopt/src/Common/IpLibraryLoader.cpp" at line 67:
Exception message: dlopen(libhsl.dylib, 0x0002): tried: 'libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/usr/lib/libhsl.dylib' (no such file, not in dyld cache), 'libhsl.dylib' (no such file)

EXIT: Library loading failure.
sIPOPT: Aborting sIPOPT computation, because IPOPT did not succeed

#-------------------------------------------

Sensitivity without bound checking

#-------------------------------------------

--------------= Warning =--------------

Internal abort has been called for the sensitivity calculations.

#-------------------------------------------

Sensitivity with bound checking

#-------------------------------------------

--------------= Warning =--------------

Internal abort has been called for the sensitivity calculations.
---- 8< ---- End of test program output ---- 8< ----

******** Test FAILED! ********

Output of the test program does not contain 'EXIT: Optimal Solution Found'.
Testing sIpopt Example redhess_cpp...

---- 8< ---- Start of test program output ---- 8< ----
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "/Users/anilvrao/Desktop/ipopt-mex-temp/Ipopt/src/Common/IpLibraryLoader.cpp" at line 67:
Exception message: dlopen(libhsl.dylib, 0x0002): tried: 'libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/usr/lib/libhsl.dylib' (no such file, not in dyld cache), 'libhsl.dylib' (no such file)

EXIT: Library loading failure.
sIPOPT: Aborting sIPOPT computation, because IPOPT did not succeed

--------------= Warning =--------------

Internal abort has been called for the sensitivity calculations.
---- 8< ---- End of test program output ---- 8< ----

******** Test FAILED! ********

Output of the test program does not contain 'EXIT: Optimal Solution Found'.
Testing EmptyNLP Example...
./run_unitTests: line 21: 69861 Abort trap: 6 $@ > tmpfile 2>&1
0
---- 8< ---- Start of test program output ---- 8< ----

*** Solve for 0 variables, 0 feasible constraint, feasible bounds
Exception of type: DYNAMIC_LIBRARY_FAILURE in file "/Users/anilvrao/Desktop/ipopt-mex-temp/Ipopt/src/Common/IpLibraryLoader.cpp" at line 67:
Exception message: dlopen(libhsl.dylib, 0x0002): tried: 'libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libomp/lib/libhsl.dylib' (no such file), '/usr/lib/libhsl.dylib' (no such file, not in dyld cache), 'libhsl.dylib' (no such file)

EXIT: Library loading failure.
Assertion failed: ((status == Solve_Succeeded) == (!infeasbounds && !infeascons)), function runEmpty, file emptynlp.cpp, line 338.
---- 8< ---- End of test program output ---- 8< ----

******** Test FAILED! ********

Test program existed with nonzero status.
Testing GetCurr Example...
Test passed!
make: *** [test] Error 1

[anilvrao]

By the way, I have HSL disabled in the compilation of IPOPT using the flag --without-hsl. Thus, I am not sure why I am getting the error that the library libhsl.dylib is missing.

[svigerske]

--without-hsl only disables that Ipopt's configure looks for HSL libraries, so you can disable linking HSL into Ipopt even when having HSL libs installed (which you do not have). Ipopt may still look for a HSL library to load at runtime if no other linear solver has been build in (or a HSL linear solver has been explicitly selected). So it seems that your Ipopt build is without Mumps. As said, if you share a full log of configure and build, maybe one sees what is going wrong. Or you inspect the configure output of Ipopt, and the config.log generated, by yourself and check why it didn't pick up your Mumps installation.

[rdzman]

FWIW, I found a simple solution to a working native Apple Silicon IPOPT MEX file. It's not statically linked with everything it needs, and I'm pretty sure it doesn't include HSL, so it may not have everything you need. But its finally running on MATLAB R2023b on my M2 MacBook Pro.

[anilvrao]

I have a follow up question regarding IPOPT and MUMPS.

Is it possible to change the settings within MUMPS for use with IPOPT? I have been given some information suggestinf that MUMPS may work better if some of the default settings are changed, but I do not know how to change any MUMPS settings when using the IPOPT Mex file.

Thanks!

[svigerske]

A few MUMPS options are available as Ipopt optoins. For others, you would have to modify the Ipopt/Mumps interface code.

[anilvrao]

Please see my new post #772 which I believe might provide more information on my troubles compiling the Mex file. In particular, please see that I am unable to successfully run "make test" in the Ipopt/build directory because the HSL libraries are not found and Ipopt is searching for the dynamically loaded library libhsl.dylib. Thanks!