Support for Arduino libraries?

[maartin0]

I'm trying to get arduino-esp32 to compile using the esp32-led-blink-sdk example as a template, which would be amazing if it worked.

I'm pretty sure it shouldn't be super difficult given how it's literally just a C++ library, but the only issue I've run into is that swiftc doesn't seem to like the preprocessor macros that Arduino uses to handle different architectures etc.

Here's the bit of arduino's CMakeLists.txt that defines the compiler flags:

target_compile_options(${COMPONENT_TARGET} PUBLIC
    -DARDUINO=10812
    -DARDUINO_${idf_target_for_macro}_DEV
    -DARDUINO_ARCH_ESP32
    -DARDUINO_BOARD="${idf_target_caps}_DEV"
    -DARDUINO_VARIANT="${CONFIG_ARDUINO_VARIANT}"
    -DESP32)

This is what the swiftc call then looks like:

/usr/bin/swiftc -j 8 -num-threads 8 -c 
-DESP_PLATFORM -DIDF_VER=\"v5.1.4\" -DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\" -DSOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -D_GNU_SOURCE -D_POSIX_READER_WRITER_LOCKS 
-parse-as-library -static -emit-module 
-emit-module-path esp-idf/main/__idf_main.swiftmodule
-module-name __idf_main 
-module-link-name main 
-wmo -color-diagnostics
-target riscv32-none-none-eabi
-Xfrontend -function-sections -enable-experimental-feature Embedded -wmo -parse-as-library -Osize 
-Xcc -march=rv32imc_zicsr_zifencei -Xcc -mabi=ilp32 
-pch-output-dir /tmp
-Xfrontend 
-enable-single-module-llvm-emission 
-Xcc -I... -Xcc -I....
-import-bridging-header main/BridgingHeader.h
-cxx-interoperability-mode=default
-DARDUINO=10812 -DARDUINO_ESP32C3_DEV -DARDUINO_ARCH_ESP32 -DARDUINO_BOARD=\"ESP32C3_DEV\" -DARDUINO_VARIANT=\"esp32c3\" -DESP32 
-output-file-map esp-idf/main/CMakeFiles/__idf_main.dir//output-file-map.json 
-I ... -I ....

Which raises the following warnings:

warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'IDF_VER="v5.1.4"')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'MBEDTLS_CONFIG_FILE="mbedtls/esp_config.h"')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'SOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'ARDUINO=10812')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'ARDUINO_BOARD="ESP32C3_DEV"')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'ARDUINO_VARIANT="esp32c3"')

Which is then used by a lot of Arduino libraries like OneWire for example (list):

#if ARDUINO >= 100
#include <Arduino.h>       // for delayMicroseconds, digitalPinToBitMask, etc
#else
#include "WProgram.h"      // for delayMicroseconds
#include "pins_arduino.h"  // for digitalPinToBitMask, etc
#endif

Creating an error like this:

<unknown>:0: error: failed to emit precompiled header '/tmp/BridgingHeader-swift_1PSIJ0LN6QHW8-clang_1Y9RGGHIW2TB2.pch' for bridging header 'main/BridgingHeader.h'
2 warnings and 4 errors generated.
components/OneWire/OneWire.h:15:10: error: 'WProgram.h' file not found
 13 | #include <Arduino.h>       // for delayMicroseconds, digitalPinToBitMask, etc
 14 | #else
 15 | #include "WProgram.h"      // for delayMicroseconds
    |          `- error: 'WProgram.h' file not found
 16 | #include "pins_arduino.h"  // for digitalPinToBitMask, etc
 17 | #endif

So it seems that the issue is swiftc only supports boolean preproccessor macros; I've tried putting the attributes in my BridgingHeader.h as #defines but that doesn't seem to work, and I don't know enough about swift's internals how to know what else to try?

Here's what is basically a MNWE (minimal not working example): https://github.com/maartin0/esp32-arduino-swift-test but I'm happy to test any ideas you might have so you don't have to set up a workspace (Arduino is like 1.5 GB)

[rauhul]

You likely want to pass the defines with -Xcc -DARDUINO=10812 so they go to the clang importer instead of the swift compiler

[maartin0]

That makes a lot of sense, thanks!
Although that doesn't seem to be the only issue because I'm now getting this:

[11/13] Linking CXX executable main.elfFAILED: main.elf
...
esp-idf/main/libmain.a(Main.swift.obj):(.data.DW.ref._swift_exceptionPersonality[DW.ref._swift_exceptionPersonality]+0x0): undefined reference to `_swift_exceptionPersonality
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

(full error)

I'll investigate a bit more tomorrow

[rauhul]

I haven never seen that symbol! @kubamracek have you?

[maartin0]

I've figured out that exact error, it's because before you updated the example to use CMake's native Swift support I had included -cxx-interoperability-mode=default in swiftc's arguments, and didn't remove it when I updated the rest of the file to try and use the same configuration as you. I guess I hadn't properly read the docs because it explicitly says that's for when you have a clang module map which I don't have since this is manually just -I including everything

That original MNWE now builds, but the original project that I was working on still doesn't compile because it doesn't seem like it's treating the arduino library as C++? e.g.

components/arduino/libraries/Wire/src/Wire.h:55:17: error: unknown type name 'public'
 53 | #endif /* SOC_I2C_SUPPORT_SLAVE */
 54 |
 55 | class TwoWire : public HardwareI2C {
    |                 `- error: unknown type name 'public'
 56 | protected:
 57 |   uint8_t num;

I've updated the MNWE's bridging header to #include "Wire.h" which gives it that same error now,

Reading those docs more carefully now I'm assuming the correct argument to pass to swiftc is -Xcc -std=c++20? (or alternatively I was using -std=gnu++11 with the project I had originally written in C++ and want to try and convert to swift, but I guess newer versions are generally better)

However, using either of these flags now gives the following error which I have no clue where to start with debugging - does swiftc get run twice, once for C and once for C++?:

$ idf.py build
...
/usr/bin/swiftc -j 8 -num-threads 8 -c ... -I ... ...
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'IDF_VER="v5.1.4"')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'MBEDTLS_CONFIG_FILE="mbedtls/esp_config.h"')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'SOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'ARDUINO=10812')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'ARDUINO_BOARD="ESP32C3_DEV"')
warning: conditional compilation flags do not have values in Swift; they are either present or absent (rather than 'ARDUINO_VARIANT="esp32c3"')
error: generate-pch command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: invalid argument '-std=c++20' not allowed with 'C'
<unknown>:0: error: clang importer creation failed

ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in build/log/idf_py_stderr_output_370636 build/log/idf_py_stdout_output_370636

[maartin0]

Any ideas?

[maartin0]

I think this just is a question of getting C++ to compile:

I'm not 100% clear on these docs but after re-reading I think -cxx-interoperability-mode=default can generate a clang module map if it doesn't find one. Re-including it gives the undefined reference to _swift_exceptionPersonality again which I have no clue where that's coming from, but further down in the docs it explains how you can use -emit-clang-header-path SwiftModule-Swift.h to emit the swift module if you use swift code in C++ sources. I'm not, but including it anyway gives a different error which I don't know if this is helpful, but seems to be slightly later in the process?

swiftc call:

/usr/bin/swiftc
-j 8 -num-threads 8 
-c 
-DESP_PLATFORM -DIDF_VER=\"v5.1.4\" -DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\" -DSOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -D_GNU_SOURCE -D_POSIX_READER_WRITER_LOCKS 
-parse-as-library -static -emit-module -emit-module-path esp-idf/main/__idf_main.swiftmodule -module-name __idf_main -module-link-name main 
-wmo -color-diagnostics -target riscv32-none-none-eabi 
-cxx-interoperability-mode=default -emit-clang-header-path SwiftModule-Swift.h 
-Xfrontend -function-sections -enable-experimental-feature Embedded 
-wmo -parse-as-library -Osize -Xcc -march=rv32imc_zicsr_zifencei -Xcc -mabi=ilp32 -Xcc -std=c++20 -Xcc -DARDUINO=10812 
-pch-output-dir /tmp
-Xfrontend -enable-single-module-llvm-emission 
-Xcc -I... 
-import-bridging-header main/BridgingHeader.h 
-v 
-DARDUINO=10812 -DARDUINO_ESP32C3_DEV -DARDUINO_ARCH_ESP32 -DARDUINO_BOARD=\"ESP32C3_DEV\" -DARDUINO_VARIANT=\"esp32c3\" -DESP32 
-output-file-map esp-idf/main/CMakeFiles/__idf_main.dir//output-file-map.json 
-I build/config -I main 
-I ...
main/Main.swift main/Led.swift

Which calls swift-frontend, and gives the following error:

error: compile command failed due to signal 6 (use -v to see invocation)
swift-frontend: /home/build-user/swift/include/swift/IRGen/Linking.h:852: static swift::irgen::LinkEntity swift::irgen::LinkEntity::forTypeMetadata(swift::CanType, swift::irgen::TypeMetadataAddress): Assertion `!isEmbedded(concreteType) || isMetadataAllowedInEmbedded(concreteType)' failed.
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the crash backtrace.
Stack dump:
0.      Program arguments:
/usr/bin/swift-frontend -frontend
-c main/Main.swift main/Led.swift 
-emit-module-path esp-idf/main/__idf_main.swiftmodule 
-emit-module-doc-path esp-idf/main/__idf_main.swiftdoc 
-emit-module-source-info-path esp-idf/main/__idf_main.swiftsourceinfo 
-emit-objc-header-path SwiftModule-Swift.h 
-target riscv32-none-none-eabi 
-disable-objc-interop 
-cxx-interoperability-mode=default 
-I build/config -I main -I ~/esp-idf/...
-color-diagnostics 
-module-link-name main
-static
-Osize 
-D ESP_PLATFORM -D IDF_VER=\"v5.1.4\" -D MBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\" -D SOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -D _GNU_SOURCE -D _POSIX_READER_WRITER_LOCKS -D ARDUINO=10812 -D ARDUINO_ESP32C3_DEV -D ARDUINO_ARCH_ESP32 -D ARDUINO_BOARD=\"ESP32C3_DEV\" -D ARDUINO_VARIANT=\"esp32c3\" -D ESP32
-function-sections 
-enable-single-module-llvm-emission
-enable-experimental-feature Embedded 
-empty-abi-descriptor 
-resource-dir /usr/lib/swift 
-Xcc -march=rv32imc_zicsr_zifencei -Xcc -mabi=ilp32 -Xcc -std=c++20 -Xcc -DARDUINO=10812 
-Xcc -I~/.espressif/...
-import-objc-header main/BridgingHeader.h
-pch-output-dir /tmp 
-module-name __idf_main 
-plugin-path /usr/lib/swift/host/plugins -plugin-path /usr/local/lib/swift/host/plugins
-enable-default-cmo -parse-as-library -num-threads 8 
-o esp-idf/main/CMakeFiles/__idf_main.dir/Main.swift.obj 
-o esp-idf/main/CMakeFiles/__idf_main.dir/Led.swift.obj
1.      Swift version 6.0-dev (LLVM 3602748d4ec9947, Swift 09b8b0861a69528)
2.      Compiling with effective version 5.10
3.      While generating Clang header
 #0 0x0000559b51ff30b7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/bin/swift-frontend+0x7aeb0b7)
 #1 0x0000559b51ff0e0e llvm::sys::RunSignalHandlers() (/usr/bin/swift-frontend+0x7ae8e0e)
 #2 0x0000559b51ff372a SignalHandler(int) Signals.cpp:0:0
 #3 0x00007fb0a2849320 (/lib/x86_64-linux-gnu/libc.so.6+0x45320)
 #4 0x00007fb0a28a2b1c __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007fb0a28a2b1c __pthread_kill_internal ./nptl/pthread_kill.c:78:10
 #6 0x00007fb0a28a2b1c pthread_kill ./nptl/pthread_kill.c:89:10
 #7 0x00007fb0a284926e raise ./signal/../sysdeps/posix/raise.c:27:6
 #8 0x00007fb0a282c8ff abort ./stdlib/abort.c:81:7
 #9 0x00007fb0a282c81b _nl_load_domain ./intl/loadmsgcat.c:1177:9
#10 0x00007fb0a283f507 (/lib/x86_64-linux-gnu/libc.so.6+0x3b507)
#11 0x0000559b4b4bed8c (/usr/bin/swift-frontend+0xfb6d8c)
#12 0x0000559b4b4c1b2f void llvm::function_ref<void (llvm::raw_ostream&)>::callback_fn<swift::printSwiftToClangCoreScaffold(swift::SwiftToClangInteropContext&, swift::ASTContext&, swift::PrimitiveTypeMapping&, llvm::raw_ostream&)::$_0::operator()(llvm::raw_ostream&) const::'lambda'(llvm::raw_ostream&)::operator()(llvm::raw_ostream&) const::'lambda'(llvm::raw_ostream&)>(long, llvm::raw_ostream&) PrintSwiftToClangCoreScaffold.cpp:0:0
#13 0x0000559b4b4a2175 swift::ClangSyntaxPrinter::printExternC(llvm::function_ref<void (llvm::raw_ostream&)>) const (/usr/bin/swift-frontend+0xf9a175)
#14 0x0000559b4b4bf517 void llvm::function_ref<void (llvm::raw_ostream&)>::callback_fn<swift::printSwiftToClangCoreScaffold(swift::SwiftToClangInteropContext&, swift::ASTContext&, swift::PrimitiveTypeMapping&, llvm::raw_ostream&)::$_0::operator()(llvm::raw_ostream&) const::'lambda'(llvm::raw_ostream&)>(long, llvm::raw_ostream&) PrintSwiftToClangCoreScaffold.cpp:0:0
#15 0x0000559b4b4a1f79 swift::ClangSyntaxPrinter::printNamespace(llvm::function_ref<void (llvm::raw_ostream&)>, llvm::function_ref<void (llvm::raw_ostream&)>, swift::ClangSyntaxPrinter::NamespaceTrivia, swift::ModuleDecl const*) const (/usr/bin/swift-frontend+0xf99f79)
#16 0x0000559b4b4a2081 swift::ClangSyntaxPrinter::printNamespace(llvm::StringRef, llvm::function_ref<void (llvm::raw_ostream&)>, swift::ClangSyntaxPrinter::NamespaceTrivia) const (/usr/bin/swift-frontend+0xf9a081)
#17 0x0000559b4b4bf22b void llvm::function_ref<void (llvm::raw_ostream&)>::callback_fn<swift::printSwiftToClangCoreScaffold(swift::SwiftToClangInteropContext&, swift::ASTContext&, swift::PrimitiveTypeMapping&, llvm::raw_ostream&)::$_0>(long, llvm::raw_ostream&) PrintSwiftToClangCoreScaffold.cpp:0:0
#18 0x0000559b4b4a1f79 swift::ClangSyntaxPrinter::printNamespace(llvm::function_ref<void (llvm::raw_ostream&)>, llvm::function_ref<void (llvm::raw_ostream&)>, swift::ClangSyntaxPrinter::NamespaceTrivia, swift::ModuleDecl const*) const (/usr/bin/swift-frontend+0xf99f79)
#19 0x0000559b4b4a2081 swift::ClangSyntaxPrinter::printNamespace(llvm::StringRef, llvm::function_ref<void (llvm::raw_ostream&)>, swift::ClangSyntaxPrinter::NamespaceTrivia) const (/usr/bin/swift-frontend+0xf9a081)
#20 0x0000559b4b4befec swift::printSwiftToClangCoreScaffold(swift::SwiftToClangInteropContext&, swift::ASTContext&, swift::PrimitiveTypeMapping&, llvm::raw_ostream&) (/usr/bin/swift-frontend+0xfb6fec)
#21 0x0000559b4b4aa8d6 swift::printModuleContentsAsCxx(llvm::raw_ostream&, swift::ModuleDecl&, swift::SwiftToClangInteropContext&, bool, llvm::StringSet<llvm::MallocAllocator>&) (/usr/bin/swift-frontend+0xfa28d6)
#22 0x0000559b4b498e22 void llvm::function_ref<void ()>::callback_fn<swift::printAsClangHeader(llvm::raw_ostream&, swift::ModuleDecl*, llvm::StringRef, swift::FrontendOptions const&, swift::IRGenOptions const&, clang::HeaderSearch&)::$_3>(long) PrintAsClang.cpp:0:0
#23 0x0000559b4b492cd7 emitCxxConditional(llvm::raw_ostream&, llvm::function_ref<void ()>, llvm::function_ref<void ()>) PrintAsClang.cpp:0:0
#24 0x0000559b4b492a69 swift::printAsClangHeader(llvm::raw_ostream&, swift::ModuleDecl*, llvm::StringRef, swift::FrontendOptions const&, swift::IRGenOptions const&, clang::HeaderSearch&) (/usr/bin/swift-frontend+0xf8aa69)
#25 0x0000559b4b2197bd bool llvm::function_ref<bool (llvm::raw_pwrite_stream&)>::callback_fn<printAsClangHeaderIfNeeded(llvm::vfs::OutputBackend&, llvm::StringRef, swift::ModuleDecl*, llvm::StringRef, swift::FrontendOptions const&, swift::IRGenOptions const&, clang::HeaderSearch&)::$_11>(long, llvm::raw_pwrite_stream&) FrontendTool.cpp:0:0
#26 0x0000559b4aff8dc4 swift::withOutputPath(swift::DiagnosticEngine&, llvm::vfs::OutputBackend&, llvm::StringRef, llvm::function_ref<bool (llvm::raw_pwrite_stream&)>) crtstuff.c:0:0
#27 0x0000559b4b21728f performEndOfPipelineActions(swift::CompilerInstance&) FrontendTool.cpp:0:0
#28 0x0000559b4b2159d8 generateCode(swift::CompilerInstance&, llvm::StringRef, llvm::Module*, llvm::GlobalVariable*) FrontendTool.cpp:0:0
#29 0x0000559b4b2109c0 performCompileStepsPostSILGen(swift::CompilerInstance&, std::unique_ptr<swift::SILModule, std::default_delete<swift::SILModule>>, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) FrontendTool.cpp:0:0
#30 0x0000559b4b20f808 swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) (/usr/bin/swift-frontend+0xd07808)
#31 0x0000559b4b220205 withSemanticAnalysis(swift::CompilerInstance&, swift::FrontendObserver*, llvm::function_ref<bool (swift::CompilerInstance&)>, bool) FrontendTool.cpp:0:0
#32 0x0000559b4b2129c7 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) FrontendTool.cpp:0:0
#33 0x0000559b4b21171a swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) (/usr/bin/swift-frontend+0xd0971a)
#34 0x0000559b4afce34e swift::mainEntry(int, char const**) (/usr/bin/swift-frontend+0xac634e)
#35 0x00007fb0a282e1ca __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#36 0x00007fb0a282e28b call_init ./csu/../csu/libc-start.c:128:20
#37 0x00007fb0a282e28b __libc_start_main ./csu/../csu/libc-start.c:347:5
#38 0x0000559b4afcd3f5 _start (/usr/bin/swift-frontend+0xac53f5)
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the build/log/idf_py_stderr_output_89909 and build/log/idf_py_stdout_output_89909

(full error)

[maartin0]

I just noticed that 2. Compiling with effective version 5.10 isn't the same as 6.0-dev - could this be the cause of the missing symbols? Although with -emit-clang-header-path SwiftModule-Swift.h enabled, you get Assertion '!isEmbedded(concreteType) || isMetadataAllowedInEmbedded(concreteType)' failed. not the undefined reference 'to _swift_exceptionPersonality'