"AWS CRT binary not present" in Lambda

[srischard]

I started the migration from the v1 to v2 and I'm bundling the software with webpack to run it as a Lambda.
Within the lambda I got the following error

  "errorType": "Error",
  "errorMessage": "AWS CRT binary not present in any of the following locations:\n\t/var/bin/linux-arm64/aws-crt-nodejs",
  "trace": [
    "Error: AWS CRT binary not present in any of the following locations:",
    "\t/var/bin/linux-arm64/aws-crt-nodejs",

my package.json

    "dependencies": {
        "aws-crt": "^1.15.5",
        "aws-iot-device-sdk-v2": "^1.9.0",

In v1 I created only a new device and it assumed the correct credentials from the container and it connected.
I don't know if I really need the "crt" if I'm in the lambda. Maybe it is in implementation issue. I tried several builders. All are raising the missing CRT binary error. My lastest builder is:

    const builder = iot.AwsIotMqttConnectionConfigBuilder.new_with_websockets({
        region: "eu-central-1",
        credentials_provider: auth.AwsCredentialsProvider.newDefault()
    });
            const config = builder
            .with_clean_session(true)
            .with_client_id(clientId)
            .with_endpoint(host)
            .build();
        const clientBootstrap = new io.ClientBootstrap();
        const client = new mqtt.MqttClient(clientBootstrap);

A local setup and using certificates works pretty well on the other hand.

    return iot.AwsIotMqttConnectionConfigBuilder
        .new_mtls_builder_from_path(certPath, keyPath)
        .with_certificate_authority_from_path(undefined, caPath);
}

If it is a coding issue does anyone have the correct code for Lambda.
If it is about the missing CRT binary does anyone have hints how to deploy it along with webpack? I can see #146 (open) and has been opened in 2021 and awslabs/aws-crt-nodejs#220 (closed) .

It seems a lot of people are running the v2 maybe someone has a solution or workaround already.

I wish there would be a migrations guide from v1 to v2 for Lambda usage.

---- update 230123 ---
webpack.config

const path = require('path');
const nodeExternals = require('webpack-node-externals');
const webpack = require('webpack')

function srcPath(subdir) {
    return path.join(__dirname, "src", subdir);
}

var config = {
    entry: path.join(__dirname, '/src/lambda.ts'),
    watch: false,
    target: 'node',
    plugins: [
        new webpack.IgnorePlugin({ resourceRegExp: /^hiredis$/ })
    ],
    output: {
        path: path.join(__dirname, 'dist'),
        filename: 'xxxxx.js',
        library: 'xxxx',
        libraryTarget: 'umd',
        libraryExport: 'default',
        umdNamedDefine: true
    },
    optimization: {
        moduleIds: "named", // deprecated namedModules: true,
        chunkIds: "named",// deprecated namedChunks: true,
        nodeEnv: false,
        minimize: false,
        flagIncludedChunks: false,
        concatenateModules: false,
        splitChunks: {
            hidePathInfo: false,
            minSize: 10000,
            maxAsyncRequests: Infinity,
            maxInitialRequests: Infinity,
        },
        emitOnErrors: true,
        checkWasmTypes: false,
        removeAvailableModules: false
    },
    externals: [
        {
            "aws-sdk": "aws-sdk"
        },
        // nodeExternals({
        //     whitelist: []
        // })
    ],
    module: {
        rules: [
            {
                test: /\.tsx?$/,
                loader: 'ts-loader',
                exclude: /node_modules/,
            },
            {
                // test: /.\/mqtt\/.\.js$/, // add all files that are known to have a shebang, (alternatively could use it for all files)
                loader: "shebang-loader",
            }
        ]
    },
    resolve: {
        extensions: [".tsx", ".ts", ".js"],
        alias: {
            "backend": srcPath('backend'),
  .....
        },
    }
};

module.exports = (env, argv) => {
    if (argv.mode === 'development') {
        //place specific dev configuration here
    }

    if (argv.mode === 'production') {
        //place specific prod configuration here
    }
    return config;
};

[jmklix]

While we don't currently have a migration guide for v1 to v2 (we are working on one), we do have a sample for how to use this sdk with webpack. so you might want to take a look at that below. You do the AWS CRT as it is a dependency of this sdk.

Browser Pub/Sub sample

This is a browser based version of the PubSub sample in JS.

To run the sample:

1. Configure your credentials and endpoint URL in settings.js.

   • You will need to setup a Cognito identity pool to run this sample. You can read how to setup a Cognito identity pool here: AWS documentation.

2. Build the script via npm install

3. Open index.html from your browser.

Please let me know if you have any more questions.

[srischard]

Thanks for the hints but unfortunately it did not really help me.
I have already have the "aws-crt" as a dependency (see above).
In your sample the target is "browser" but not a node application. And in the sample there is no "aws-crt" package as a dependency . I guess you skipped the crt in the sample.
I attached my "webpack.config" to the post. Maybe I missed something there. Or is there so

[jmklix]

Have you tried importing the sdk as a layer?

[srischard]

No I have not yet build any layer. I included the AppConfig extension to my Lambdas - so at least I'm aware of how to benefit from them.
But is the idea of a distinct encapsulated layer something you can provide for all lambda users? A small repo with webpack or esbuild we can clone and build our layer from?

[jmklix]

But is the idea of a distinct encapsulated layer something you can provide for all lambda users?

No, this is not something that this sdk currently provides. But you can find instructions for creating a layer here. Please let me know if you have any problems with this

[johannnishant]

Hello!
I am running into the same issue!
Trying to run the MQTT5 Client in an AWS Lambda
Tried a lot of different options but nothing works,

Get a AWS CRT binary not present in any of the following locations:\n\t/bin/linux-x64/aws-crt-nodejs

Even tried using layers,
What modules needs to be in the layers? only the aws-iot-device-sdk-js-v2 ?

[srischard]

I skipped the sdk-v2 as long as there is no proper migration guide for lambda consumers.
If I could make a wish I would like to see these binaries installed in the node 18 containers like the AWS-SDK-V3 and the iot-sdk-v2 will use this transparently for me.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.