Commit Graph

970 Commits

Author SHA1 Message Date
iocuydi e8d9cbca3f Add prompt and completion token tracking (#2080)
Tracking the breakdown of token usage is useful when using GPT-4, where
prompt and completion tokens are priced differently.
2023-03-27 23:41:25 -07:00
Michael Gokhman b5020c7d9c docs: fix promptlayer link typo (#2005)
tiny typo, just stumbled upon it when reading the docs

Co-authored-by: Michael Gokhman <michaelg@ai21.com>
2023-03-27 23:35:54 -07:00
Deepankar Mahapatro 5bea731fb4 docs(deployment): add langchain-serve (#2006)
Adds documentation to deploy Langchain Chains & Agents using Jina.

Repo: https://github.com/jina-ai/langchain-serve
2023-03-27 23:32:04 -07:00
Harrison Chase 0e3b0c827e Harrison/ai plugin (#2084)
Co-authored-by: Xupeng (Tony) Tong <tongxupeng.cpu@gmail.com>
2023-03-27 23:31:53 -07:00
Harrison Chase 365669a7fd Harrison/fix save context (#2082)
Co-authored-by: Saurabh Misra <misra.saurabh1@gmail.com>
2023-03-27 23:10:46 -07:00
blob42 b7f392fdd6 [agent_executor] convenience func: lookup tool by name (#2001)
A quick convenience function to lookup a tool by name

Co-authored-by: blob42 <spike@w530>
2023-03-27 23:10:34 -07:00
Ace Eldeib 4be2f9d75a fix: numerous broken documentation links (#2070)
seems linkchecker isn't catching them because it runs on generated html.
at that point the links are already missing.
the generation process seems to strip invalid references when they can't
be re-written from md to html.

I used https://github.com/tcort/markdown-link-check to check the doc
source directly.

There are a few false positives on localhost for development.
2023-03-27 23:07:03 -07:00
Harrison Chase f74a1bebf5 Harrison/duckdb (#2064)
Co-authored-by: Trent Hauck <trent@trenthauck.com>
2023-03-27 19:51:34 -07:00
Harrison Chase 76ecca4d53 redis retriever (#2060) 2023-03-27 19:51:23 -07:00
Ankush Gola b7ebb8fe30 enable streaming in anthropic llm wrapper (#2065) 2023-03-27 20:25:00 -04:00
Francisco Ingham 41c8a42e22 Improve chat tool prompt (#1989)
I have found that when the user has not asked an explicit question the
agent might have trouble answering the latest comment and might instead
try to answer a question that came before in the conversation which
would not be what is desired.

I also found that the agent might get confused with the current prompt
and talk about the tools themselves instead of the results obtained from
them.

I added two changes to the tool prompt so that the agent answers only
the last comment/question and only returns information from tool
results.
2023-03-27 16:34:01 -07:00
Francisco Ingham 1cc9e90041 Solve small bug in the kg prompt (#1988)
I think that the 'Person' line should be under 'Last line of
conversation' as is the case in the other examples in the kg prompt
2023-03-27 16:33:26 -07:00
Harrison Chase 30e3b31b04 Harrison/document cleanup (#2062)
Co-authored-by: Delip Rao <delip@users.noreply.github.com>
2023-03-27 16:32:55 -07:00
Harrison Chase a0cd6672aa Harrison/site map (#2061)
Co-authored-by: Tim Asp <707699+timothyasp@users.noreply.github.com>
2023-03-27 16:28:08 -07:00
Arttii 8b5a43d720 Correctly pass filter down to the similarity_search_with_score function for chroma filtering logic (#1934)
Should slightly fix the work in #1869
2023-03-27 15:50:46 -07:00
Jonathan Pedoeem 725b668aef Updating PromptLayer request in PromptLayer Models to be async in agenerate (#2058)
Currently in agenerate, the PromptLayer request is blocking and should
be make async. In this PR we update all of that in order to work as it
should
2023-03-27 15:24:53 -07:00
Peter Shi 024efb09f8 feat: add function similarity_search_limit_score to vectorstores.redis (#1950)
# Description
***
Add function similarity_search_limit_score and
similarity_search_with_score

# How to use
***
``
rds = Redis.from_existing_index(embeddings,
redis_url="redis://localhost:6379", index_name='link')

rds.similarity_search_limit_score(query, k=3, score=0.2)

rds.similarity_search_with_score(query, k=3)
``

---------

Co-authored-by: Peter <peter.shi@alephf.com>
2023-03-27 15:05:09 -07:00
Rajat Saxena 953e58d004 similarity_search is not accepting filters (#1964)
I have changed the name of the argument from `where` to `filter` which
is expected by `similarity_search_with_score`.

Fixes #1838

---------

Co-authored-by: Rajat Saxena <hi@rajatsaxena.dev>
2023-03-27 15:04:53 -07:00
Gerard Hernandez f257b08406 Removed duplicate "revision_request" in constitutional_ai/prompts.py (#2046)
Removed a duplicate "revision_request" in the second example within
[this
file](https://github.com/hwchase17/langchain/blob/master/langchain/chains/constitutional_ai/prompts.py).
2023-03-27 15:04:23 -07:00
Krulknul 5e91928607 Added .as_retriever() to from_llm() calls (#2051) 2023-03-27 15:04:03 -07:00
Harrison Chase 880a6a3db5 Harrison/redis id key (#2057)
Co-authored-by: Fabrizio Ruocco <ruoccofabrizio@gmail.com>
2023-03-27 15:03:51 -07:00
cragwolfe 71e8eaff2b UnstructuredURLLoader: allow url failures, keep processing (#1954)
By default, UnstructuredURLLoader now continues processing remaining
`urls` if encountering an error for a particular url.

If failure of the entire loader is desired as was previously the case,
use `continue_on_failure=False`.

E.g., this fails splendidly, courtesy of the 2nd url:

```
from langchain.document_loaders import UnstructuredURLLoader
urls = [
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-8-2023",
    "https://doesnotexistithinkprobablynotverynotlikely.io",
    "https://www.understandingwar.org/backgrounder/russian-offensive-campaign-assessment-february-9-2023",
]
loader = UnstructuredURLLoader(urls=urls, continue_on_failure=False)
data = loader.load()
```

Issue: https://github.com/hwchase17/langchain/issues/1939
2023-03-27 14:34:14 -07:00
Daniel Chalef 6598beacdb PydanticOutputParser unit test (#2047)
Unit test for PydanticOutputParser

---------

Co-authored-by: Daniel Chalef <daniel.chalef@private.org>
2023-03-27 14:32:56 -07:00
William FH e4f15e4eac Add support for YAML Spec Plugins (#2054)
It's common to use `yaml` for an OpenAPI spec used in the GPT plugins. 

For example: https://www.joinmilo.com/openapi.yaml or
https://api.slack.com/specs/openapi/ai-plugin.yaml (from [Wong2's
ChatGPT Plugins List](https://github.com/wong2/chatgpt-plugins))
2023-03-27 14:27:48 -07:00
weiyang e50c1ea7fb Fix the parameter error of 'Qdrant.maximal_marginal_relevance' (#1921)
Hi, first and foremost, I would like to express my gratitude for your
outstanding work; it's truly remarkable!


https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/qdrant.py#L134
It appears that there might be a minor issue with the `limit` parameter
being passed incorrectly in the `Qdrant.maximal_marginal_relevance`
function. This seems to be a typographical error.

Signed-off-by: weiyang <weiyang.ones@gmail.com>
2023-03-27 08:29:07 -07:00
goka 62e08f80de feat #1915 support for google custom search site restricted api (#1920)
#1915 

https://developers.google.com/custom-search/v1/site_restricted_api

It is possible to search unrestricted to specific sites.
2023-03-27 08:28:55 -07:00
david qiu c50fafb35d fix Poetry 1.4.0+ installation (#1935)
Temporary fix for #1801 until upstream issues with `pydata-sphinx-theme`
wheel are resolved.
2023-03-27 08:27:54 -07:00
Jason Holtkamp 3d3e523520 Update getting_started with better example (#1910)
I noticed that the "getting started" guide section on agents included an
example test where the agent was getting the question wrong 😅

I guess Olivia Wilde's dating life is too tough to keep track of for
this simple agent example. Let's change it to something a little easier,
so users who are running their agent for the first time are less likely
to be confused by a result that doesn't match that which is on the docs.
2023-03-27 08:19:13 -07:00
Eduard van Valkenburg c1a9d83b34 Added Azure Blob Storage File and Container Loader (#1890)
Added support for document loaders for Azure Blob Storage using a
connection string. Fixes #1805

---------

Co-authored-by: Mick Vleeshouwer <mick@imick.nl>
2023-03-27 08:17:14 -07:00
Harrison Chase 42d725223e Harrison/num token calculation (#2041)
Co-authored-by: Aratako <127325395+Aratako@users.noreply.github.com>
2023-03-27 08:16:32 -07:00
Harrison Chase 0bbcc7815b Harrison/open search kwargs (#2040)
Signed-off-by: Marcel Coetzee <marcelcoetzee@tutanota.com>
Co-authored-by: Marcel <34739235+Pipboyguy@users.noreply.github.com>
2023-03-27 07:56:09 -07:00
Harrison Chase b26fa1935d fix headers (#2039) 2023-03-27 07:55:57 -07:00
Harrison Chase bc2ed93b77 fix doc tags (#2019) 2023-03-26 21:43:51 -07:00
Ankush Gola c71f2a7b26 small nit on index page (#2018) 2023-03-27 00:15:24 -04:00
Harrison Chase 51681f653f fix docs (#2017) 2023-03-26 20:50:36 -07:00
Harrison Chase 705431aecc big docs refactor (#1978)
Co-authored-by: Ankush Gola <ankush.gola@gmail.com>
2023-03-26 19:49:46 -07:00
Harrison Chase b83e826510 plugin tool (#1974) 2023-03-24 12:30:08 -07:00
Mario Kostelac e7d6de6b1c (ChatOpenAI) Add model_name to LLMResult.llm_output (#1960)
This makes sure OpenAI and ChatOpenAI have the same llm_output, and
allow tracking usage per model. Same work for OpenAI was done in
https://github.com/hwchase17/langchain/pull/1713.
2023-03-24 08:51:16 -07:00
Harrison Chase 6e0d3880df bump version to 122 (#1970) 2023-03-24 08:24:44 -07:00
Harrison Chase 6ec5780547 add docs for openai retriever ingest (#1969) 2023-03-24 08:24:33 -07:00
Harrison Chase 47d37db2d2 WIP: Harrison/base retriever (#1765) 2023-03-24 07:46:49 -07:00
Enwei Jiao 4f364db9a9 Add milvus for ecosystem (#1951) 2023-03-23 22:01:28 -07:00
Tim Asp 030ce9f506 fix import error of bs4 (#1952)
Ran into a broken build if bs4 wasn't installed in the project.

Minor tweak to follow the other doc loaders optional package-loading
conventions.

Also updated html docs to include reference to this new html loader.

side note: Should there be 2 different html-to-text document loaders?
This new one only handles local files, while the existing unstructured
html loader handles HTML from local and remote. So it seems like the
improvement was adding the title to the metadata, which is useful but
could also be added to `html.py`
2023-03-23 21:56:13 -07:00
Harrison Chase 8990122d5d retrievers interface (#1948) 2023-03-23 19:00:38 -07:00
Harrison Chase 52d6bf04d0 tracing improvements to docs (#1947) 2023-03-23 19:00:18 -07:00
Harrison Chase 910da8518f hotfix (#1928) 2023-03-23 07:11:15 -07:00
Naoki Ainoya 2f27ef92fe Fix typo in VectorStoreIndexWrapper method (#1922)
Fixed a typo in the argument of the query method within the
VectorStoreIndexWrapper class. Specifically, the argument `retriver` has
been changed to `retriever`. With this correction, the correct argument
name is used, and potential bugs are avoided.
2023-03-23 07:08:04 -07:00
Harrison Chase 75149d6d38 bump version 120 (#1918) 2023-03-22 23:21:56 -07:00
Harrison Chase fab7994b74 Harrison/retrieval code (#1916) 2023-03-22 23:15:04 -07:00
Harrison Chase eb80d6e0e4 Harrison/from methods (#1912)
Co-authored-by: shibuiwilliam <shibuiyusuke@gmail.com>
2023-03-22 21:10:09 -07:00