[blog] Choosing Embedding Models

[davidmyriel]

PREVIEW

[netlify]

✅ Deploy Preview for condescending-goldwasser-91acf0 ready!

Name	Link
🔨 Latest commit	a998fbf
🔍 Latest deploy log	https://app.netlify.com/sites/condescending-goldwasser-91acf0/deploys/679cfba93b42300008524206
😎 Deploy Preview	https://deploy-preview-1439--condescending-goldwasser-91acf0.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[AstraBert]

Hi there, the blog is fantastic, thank you soooo much!❤️

There is just a little imprecision in sentrev usage example:

This (line 76-81)

from sentrev import SentrevEvaluator

evaluator = SentrevEvaluator(client=client, data_files=["data1.pdf", "data2.docx"], models=["all-mpnet-base-v2", "mini-lm"])
evaluator.evaluate()

Should be:

from sentrev.evaluator import evaluate_dense_retrieval
from sentence_transformers import SentenceTransformer
# Define all the encoders
dense_encoder1 = SentenceTransformer('sentence-transformers/all-mpnet-base-v2', device="cuda")
dense_encoder2 = SentenceTransformer('sentence-transformers/all-MiniLM-L12-v2', device="cuda")
dense_encoder3 = SentenceTransformer('sentence-transformers/LaBSE', device="cuda")
# Evaluate retrieval!
evaluate_dense_retrieval(files = ["data1.pdf", "data2.docx"],  encoders = [dense_encoder1, dense_encoder2, dense_encoder3], encoder_to_names = { dense_encoder1: 'all-mpnet-base-v2', dense_encoder2: 'all-MiniLM-L12-v2', dense_encoder3: 'LaBSE'}, client = client, csv_path="~/evals/dense_stats.csv", chunking_size = 1500, text_percentage=0.3, distance="dot", mrr=10, carbon_tracking="USA", plot=True)

Would be also great if we could put the actual link to the GitHub repo :)

Should I implement these changes or can you do it?
Thanks a lot!🥰

[AstraBert]

This looks awesome, thank you so much! :)