Andrew Gross @gross.systems · Jan 6 Bonus, I was testing on Nous Llama 3 8B, and the MIPROv2 optimizer added a funny set of stakes (similar to my grandma will die):
0 likes 1 replies
?
Replies
Though related to that RAG example, anyone have good examples of using vLLM to serve an embedding API? Ran in to a lot of issues following dspy examples with api rate limits so switched to my local hardware (a fun sidequest thanks to iommu). However, I didn't figure out how to serve embeddings yet.