Why are the embedding inference results different from the Python version?

[coolbeevip]

I’m new to this field and have been using sentence_transformers to embed inference. Recently, I tried text-embeddings-inference and noticed it was significantly faster. However, I found that the inference results varied when I used the same model. What might cause these differences, and can the results be consistent across both approaches?

Python demo

import sentence_transformers
transformer = sentence_transformers.SentenceTransformer(
    model_name_or_path="/Volumes/SD/huggingface-models/bge-m3",
    cache_folder="/Volumes/SD/huggingface-models",
    device="cpu"
)

texts = ["Hello, world!"]
texts = list(map(lambda x: x.replace("\n", " "), texts))
embeddings = transformer.encode(
    texts,
    show_progress_bar=False
)
print(embeddings.tolist())

[[-0.01615568809211254, 0.026993485167622566, -0.04258325323462486, 0.013542222790420055, ...]

text-embeddings-inference demo

./text-embeddings-router --model-id /Volumes/SD/huggingface-models/bge-m3 --port 8080

curl 127.0.0.1:8080/embed \
    -X POST \
    -d '{"inputs":"Hello, world!"}' \
    -H 'Content-Type: application/json'

[[-0.016229825,0.026583018,-0.043181285,0.013070445,-0.019435262,-0.04520771,-0.053681854,...]

These two examples clearly show that the initial digits of the outputs differ.