Commit Graph

886 Commits

Author SHA1 Message Date
Bagatur ec362ecbe2 Fixed regex bug in RetrievalQAWithSources in previous update (#9898)
- Description: In my previous PR, I had modified the code to catch all
kinds of [SOURCES, sources, Source, Sources]. However, this change
included checking for a colon or a white space which should actually
have been only checking for a colon.
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
2023-08-29 17:32:24 -07:00
Nikhil Suresh 56a0165a4e cleaned up unit test example 2023-08-29 23:37:54 +00:00
William FH cedfad541d don't emit none from eval config (#9963) 2023-08-29 16:14:32 -07:00
Nikhil Suresh b31475c622 minor updates to regex 2023-08-29 23:13:31 +00:00
Bagatur 8fb0a9594c Add LLMonitor Callback Handler Integration - open-source observability & analytics (#9870)
Adds support for [llmonitor](https://llmonitor.com) callbacks.

It enables:
- Requests tracking / logging / analytics
- Error debugging
- Cost analytics
- User tracking

Let me know if anythings neds to be changed for merge.

Thank you!
2023-08-29 15:49:01 -07:00
William FH d799963870 Wfh/async tool (#9878)
Co-authored-by: Daniel Brenot <dbrenot@pelmorex.com>
Co-authored-by: Daniel <daniel.alexander.brenot@gmail.com>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-29 15:37:41 -07:00
Bagatur 16eb935469 Fix for similarity_search_with_score (#9903)
- Description: the implementation for similarity_search_with_score did
not actually include a score or logic to filter. Now fixed.
- Tag maintainer: @rlancemartin
- Twitter handle: @ofermend
2023-08-29 15:04:48 -07:00
Bagatur c70bb0ec28 Activeloopai runtime arg (#9961) 2023-08-29 15:01:46 -07:00
Bagatur 0f85671630 fmt 2023-08-29 14:55:25 -07:00
Bagatur 78c014399f fmt 2023-08-29 14:53:15 -07:00
Eugene Yurtsev 5cce6529a4 Speed up openai tests (#9943)
Saves ~8-10 seconds from total unit tests times

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-29 14:30:41 -07:00
Guy Korland 7cbe872af8 Add support for Falkordb (ex-RedisGraph) (#9821)
Replace this entire comment with:
  - Description: Add support for Falkordb (ex-RedisGraph)
  - Tag maintainer: @hwchase17
  - Twitter handle: @g_korland
2023-08-29 14:22:33 -07:00
Bagatur 9f2d908316 cr 2023-08-29 14:16:48 -07:00
Bagatur 3c1547925a fix 2023-08-29 14:02:13 -07:00
William FH fbd792ac7c Fix import (#9945) 2023-08-29 12:38:42 -07:00
Zizhong Zhang 8bd7a9d18e feat: PromptGuard takes a list of str (#9948)
Recently we made the decision that PromptGuard takes a list of strings
instead of a string.
@ggroode implemented the integration change.

---------

Co-authored-by: ggroode <ggroode@berkeley.edu>
Co-authored-by: ggroode <46691276+ggroode@users.noreply.github.com>
2023-08-29 12:22:30 -07:00
Predrag Gruevski 8dbf4cbe80 Add notice about security-sensitive experimental code to experimental README. (#9936)
It renders like this:
https://github.com/langchain-ai/langchain/tree/pg/experimental-readme/libs/experimental


![image](https://github.com/langchain-ai/langchain/assets/2348618/a5f9569d-96f6-44c6-8559-921adb3e337d)
2023-08-29 14:21:30 -04:00
Predrag Gruevski b5cd1e0fed Add security notices on PAL and CPAL experimental chains. (#9938)
Clearly document that the PAL and CPAL techniques involve generating
code, and that such code must be properly sandboxed and given
appropriate narrowly-scoped credentials in order to ensure security.

While our implementations include some mitigations, Python and SQL
sandboxing is well-known to be a very hard problem and our mitigations
are no replacement for proper sandboxing and permissions management. The
implementation of such techniques must be performed outside the scope of
the Python process where this package's code runs, so its correct setup
and administration must therefore be the responsibility of the user of
this code.
2023-08-29 13:51:56 -04:00
Jan-Luca Barthel f5faac8859 addition of cosine distance function for faiss (#9939)
- Description: added the _cosine_relevance_score_fn to
_select_relevance_score_fn of faiss.py to enable the use of cosine
distance for similarity for this vector store and to comply with the
Error Message, that implies, that cosine should be a valid distance
strategy
- Issue: no relevant Issue found, but needed this function myself and
tested it in a private repo
  - Dependencies: none
2023-08-29 10:29:51 -07:00
Eugene Yurtsev 880bf06290 x 2023-08-29 11:15:41 -04:00
Eugene Yurtsev 9efc29e3d1 x 2023-08-29 11:13:42 -04:00
Bagatur d6957921f0 bump 276 (#9931) 2023-08-29 08:00:38 -07:00
Tomaz Bratanic db13fba7ea Add neo4j vector support (#9770)
Neo4j has added vector index integration just recently. To allow both
ingestion and integrating it as vector RAG applications, I wrapped it as
a vector store as the implementation is completely different from
`GraphCypherQAChain`. Here, we are not generating any Cypher statements
at query time, we are simply doing the vector similarity search using
the new vector index as if we were dealing with a vector database.

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-29 07:54:20 -07:00
Bagatur 49ebbe4bcd fix pydantic import (#9930) 2023-08-29 07:53:01 -07:00
Mike Nitsenko c80e406e95 Cube semantic loader: allow cubes processing (#9927)
We've started to receive feedback (after launch) that using only views
is confusing.
We're considering this as a good practice, as a view serves as a
"facade" for your data - however, we decided to let users decide this on
their own.

Solves the questions from:
- https://github.com/cube-js/cube/issues/7028
- https://github.com/langchain-ai/langchain/pull/9690
2023-08-29 07:21:01 -07:00
Nikhil Suresh dd10cf945c fixed minor linting issues 2023-08-29 14:15:59 +00:00
adilkhan bbae8cb88f Added runtime argument 2023-08-29 12:12:49 +06:00
Ofer Mendelevitch 4454204455 reformat black 2023-08-28 23:04:57 -07:00
Ofer Mendelevitch 318a21e267 fixed typo in spelling 2023-08-28 23:01:11 -07:00
hughcrt e71f4760db Change multiline comment width 2023-08-29 07:55:10 +02:00
Ofer Mendelevitch a5450be32e fixed lint 2023-08-28 22:31:39 -07:00
Ofer Mendelevitch 8b8d2a6535 fixed similarity_search_with_score to really use a score
updated unit test with a test for score threshold
Updated demo notebook
2023-08-28 22:26:55 -07:00
hughcrt 7979cef06a Replace | by Union 2023-08-29 06:22:50 +02:00
Nikhil Suresh 23ef836b48 matches colon and any number of white spaces after colon 2023-08-29 04:18:33 +00:00
Nikhil Suresh 64eb5a6082 removed unnecessary white space in regex that breaks qa with sources chain 2023-08-29 03:54:38 +00:00
Nikhil Suresh 8a4670e127 updated formatting changes 2023-08-29 03:54:38 +00:00
Nikhil Suresh b1f649bca5 fixed issue with white space and added unit tests 2023-08-29 03:54:38 +00:00
Nikhil Suresh 6d3485e798 fixed regex to match sources for all cases, also includes source 2023-08-29 03:54:25 +00:00
Predrag Gruevski 47499c6db4 Avoid type: ignore suppression by adding mypy type hint. (#9881)
Mypy was not able to determine a good type for `type_to_loader_dict`,
since the values in the dict are functions whose return types are
related to each other in a complex way. One can see this by adding a
line like `reveal_type(type_to_loader_dict)` and running mypy, which
will get mypy to show what type it has inferred for that value.

Adding an explicit type hint to help out mypy avoids the need for a mypy
suppression and allows the code to type-check cleanly.
2023-08-28 17:53:33 -07:00
maks-operlejn-ds f327535eda Add conftest file to langchain experimental (#9886)
In order to use `requires` marker in langchain-experimental, there's a
need for *conftest.py* file inside. Everything is identical to the main
langchain module.

Co-authored-by: maks-operlejn-ds <maks.operlejn@gmail.com>
2023-08-28 17:52:16 -07:00
William FH 907c57e324 Add collect_runs callback (#9885) 2023-08-28 15:30:41 -07:00
William FH 3103f07e03 Use existing required args obj if specified (#9883)
We always overwrote the required args but we infer them by default.
Doing it only the old way makes it so the llm guesses even if an arg is
optional (e.g., for uuids)
2023-08-28 14:40:22 -07:00
William FH b14d74dd4d iMessage loader (#9832)
Add an iMessage chat loader
2023-08-28 13:43:59 -07:00
Predrag Gruevski eb3d1fa93c Add security warning to experimental SQLDatabaseChain class. (#9867)
The most reliable way to not have a chain run an undesirable SQL command
is to not give it database permissions to run that command. That way the
database itself performs the rule enforcement, so it's much easier to
configure and use properly than anything we could add in ourselves.
2023-08-28 13:53:27 -04:00
hughcrt 97741d41c5 Add LLMonitorCallbackHandler 2023-08-28 19:24:50 +02:00
eryk-dsai 7f5713b80a feat: grammar-based sampling in llama-cpp (#9712)
## Description 

The following PR enables the [grammar-based
sampling](https://github.com/ggerganov/llama.cpp/tree/master/grammars)
in llama-cpp LLM.

In short, loading file with formal grammar definition will constrain
model outputs. For instance, one can force the model to generate valid
JSON or generate only python lists.

In the follow-up PR we will add:
* docs with some description why it is cool and how it works
* maybe some code sample for some task such as in llama repo

---------

Co-authored-by: Lance Martin <lance@langchain.dev>
Co-authored-by: Bagatur <baskaryan@gmail.com>
2023-08-28 09:52:55 -07:00
William FH cb642ef658 Return feedback (#9629)
Return the feedback values in an eval run result

Also made a helper method to display as a dataframe but it may be
overkill
2023-08-28 09:15:05 -07:00
Bagatur 5e2d0cf54e bump 275 (#9860) 2023-08-28 07:27:07 -07:00
Leonid Kuligin 00baddf34c fixed enterprise search returning an empty array 2023-08-28 15:38:56 +02:00
Eugene Yurtsev 5edf819524 Qdrant Client: Expose instance for creating client (#9706)
Expose classmethods to convenient initialize the vectostore.

The purpose of this PR is to make it easy for users to initialize an
empty vectorstore that's properly pre-configured without having to index
documents into it via `from_documents`.

This will make it easier for users to rely on the following indexing
code: https://github.com/langchain-ai/langchain/pull/9614
to help manage data in the qdrant vectorstore.
2023-08-28 09:30:59 -04:00