[Question]: could i use my local embedding model ?

[ucas010]

Steps to reproduce

i have my own embedding encoder, and just request the http url to get vectors,
then how to realize it
and i also have my own LLM , and also i ask it just through the URL with ChatOpenai or OpenAI func ,
then how to replace the default Openai ?
thx

Relevant log output

no

What did you expect to happen?

replace the embedding encoder and the LLM

Version

no

Docker Versions (if used)

No response

OS

Linux

On which CPU architecture did you run Pathway?

ARM64 (AArch64, Apple silicon)

[dxtrous]

@ucas010 please take a look at this guide: https://pathway.com/developers/templates/private-rag-ollama-mistral#_3-embedding-model-selection.
It describes both selection of local embeddings, and of a connection to a custom LLM via LiteLLM wrapper.

The code described comes from this template https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines/private-rag but can be used with any template.

For relevant documentation, see:
https://pathway.com/developers/api-docs/pathway-xpacks-llm/llms
https://pathway.com/developers/api-docs/pathway-xpacks-llm/embedders

[ucas010]

thx,
if i do not use pw vector store ,
how to deal with it ?
or i just use faiss ?
@dxtrous

[dxtrous]

I think there is no problem here: pw Vector Store is a built-in vector index in-memory (it is built around Tantivy, at similar performance levels as the FAISS implementation). You do not need any extra steps to set up pw Vector Store, it will work out of the box. Try running the code.

Setting up an external vector store or integrating with FAISS is significantly more work. We currently do not provide templates for this.

[dxtrous]

Hi @ucas010 did you give the built-in vector store a try?
If we should close the issue, let me know.