AWS Bedrock isn't accessible from SvelteKit Project folder, getting a socket connection timeout error #6432
Labels
guidance
General information and guidance, answers to FAQs, or recommended best practices/resources.
p3
This is a minor priority issue
response-requested
Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days.
I'm trying to access the llama 3.1 405B model on AWS bedrock. Here's the code in my Sveltekit server side file:
`
import { json } from '@sveltejs/kit';
import {PRIVATE_AWS_ACCESS_KEY, PRIVATE_AWS_SECRET_KEY} from '$env/static/private'
import {
BedrockRuntimeClient,
InvokeModelCommand
} from "@aws-sdk/client-bedrock-runtime";
const client = new BedrockRuntimeClient({ credentials:{ accessKeyId:PRIVATE_AWS_ACCESS_KEY,
secretAccessKey: PRIVATE_AWS_SECRET_KEY},region: "us-west-2" });
console.log(client)
const modelId = "meta.llama3-1-405b-instruct-v1:0";
/** @type {import('./$types').RequestHandler} */
/** @type {import('./$types').PageLoad} */
export async function load() {
try{
const userMessage =
Tell me about donuts
const prompt =
<|begin_of_text|> <|start_header_id|>user<|end_header_id|> ${userMessage} <|eot_id|> <|start_header_id|>assistant<|end_header_id|>
;// Format the request payload using the model's native structure.
const request = {
prompt,
// Optional inference parameters:
max_gen_len: 1024,
temperature: 0.7,
top_p: 0.9,
}
const response = await client.send(
new InvokeModelCommand({
contentType: "application/json",
accept: "application/json",
body: JSON.stringify(request),
modelId,
}),
);
// Decode the native response body.
/** @type {{ generation: string }} */
const nativeResponse = JSON.parse(new TextDecoder().decode(response.body));
// Extract and print the generated text.
const responseText = nativeResponse.generation;
console.log(responseText);
return {resp:responseText}
}catch(err){
console.log(err)
return{message: "This isn't working"}
}
}`
All I'm able to get is this:
Error [ERR_SOCKET_CONNECTION_TIMEOUT]: Socket connection timeout at new NodeError (node:internal/errors:399:5) at internalConnectMultiple (node:net:1099:20) at Timeout.internalConnectMultipleTimeout (node:net:1638:3) at listOnTimeout (node:internal/timers:575:11) at process.processTimers (node:internal/timers:514:7) { code: 'ERR_SOCKET_CONNECTION_TIMEOUT', '$metadata': { attempts: 1, totalRetryDelay: 0 } }
It does work on a separate, simple typescript project that only calls the model and prints.
Anything, even a clue as to what could be causing this will be a huge help.
Thank you.
The text was updated successfully, but these errors were encountered: