e08961ab25
Fixed openai embeddings to be safe by batching them based on token size calculation. ( #991 )
...
I modified the logic of the batch calculation for embedding according to
this cookbook
https://github.com/openai/openai-cookbook/blob/main/examples/Embedding_long_inputs.ipynb
2023-02-15 23:02:32 -08:00
91c6cea227
Harrison/batch embeds ( #972 )
...
Co-authored-by: John Dagdelen <jdagdelen@users.noreply.github.com >
Co-authored-by: Harrison Chase <harrisonchase@Harrisons-MBP.attlocal.net >
2023-02-10 06:59:50 -08:00
d564308e0f
rfc: instruct embeddings ( #811 )
...
Co-authored-by: seanaedmiston <seane999@gmail.com >
2023-02-02 08:44:02 -08:00
7b4882a2f4
Harrison/tf embeddings ( #817 )
...
Co-authored-by: Ryohei Kuroki <10434946+yakigac@users.noreply.github.com >
2023-01-31 00:00:08 -08:00
b94244eb12
nits ( #210 )
...
use json.dump
move test to integration tests (since it requires huggingface_hub)
2022-11-27 13:03:09 -08:00
b90e25f786
Add HuggingFace Hub Embeddings ( #125 )
...
Add support for calling HuggingFace embedding models
using the HuggingFaceHub Inference API. New class mirrors
the existing HuggingFaceHub LLM implementation. Currently
only supports 'sentence-transformers' models.
Closes #86
2022-11-27 00:24:59 -08:00
28282ad099
Issam9/cohere embeddings ( #105 )
...
Add support for cohere embeddings
2022-11-09 13:44:27 -08:00
95dd2f140e
Make Integration Tests "work" again ( #106 )
...
This fixes Issue #104
The tests for HF Embeddings is skipped because of the segfault issue
mentioned there. Perhaps, a new issue should be created for that?
2022-11-09 13:26:58 -08:00
990cd821cc
Issam/hf embeddings ( #68 )
...
Add support of HuggingFace embedding models
2022-11-07 05:46:44 -08:00
76aff023d7
FAISS and embedding support ( #48 )
...
also adds embeddings and an in memory docstore
2022-11-01 21:29:39 -07:00
Hasegawa Yuya