mirror of
https://github.com/kennethreitz/langchain.git
synced 2026-06-05 23:00:18 +00:00
e05bb938de
* feat: Add Google Cloud Translation document transformer * Merge branch 'langchain-ai:master' into google-translate * Add documentation for Google Translate Document Transformer * Fix line length error * Merge branch 'master' into google-translate * Merge branch 'google-translate' of https://github.com/holtskinner/lan… * Addressed code review comments * Merge branch 'master' into google-translate * Merge branch 'google-translate' of https://github.com/holtskinner/lan… * Removed extra variable * Merge branch 'google-translate' of https://github.com/holtskinner/lan… * Merge branch 'master' into google-translate * Merge branch 'google-translate' of https://github.com/holtskinner/lan… * Removed extra import
308 lines
9.2 KiB
Plaintext
308 lines
9.2 KiB
Plaintext
# Google
|
|
|
|
All functionality related to [Google Cloud Platform](https://cloud.google.com/)
|
|
|
|
## LLMs
|
|
|
|
### Vertex AI
|
|
|
|
Access PaLM LLMs like `text-bison` and `code-bison` via Google Cloud.
|
|
|
|
```python
|
|
from langchain.llms import VertexAI
|
|
```
|
|
|
|
### Model Garden
|
|
|
|
Access PaLM and hundreds of OSS models via Vertex AI Model Garden.
|
|
|
|
```python
|
|
from langchain.llms import VertexAIModelGarden
|
|
```
|
|
|
|
## Chat models
|
|
|
|
### Vertex AI
|
|
|
|
Access PaLM chat models like `chat-bison` and `codechat-bison` via Google Cloud.
|
|
|
|
```python
|
|
from langchain.chat_models import ChatVertexAI
|
|
```
|
|
|
|
## Document Loader
|
|
### Google BigQuery
|
|
|
|
> [Google BigQuery](https://cloud.google.com/bigquery) is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.
|
|
`BigQuery` is a part of the `Google Cloud Platform`.
|
|
|
|
First, we need to install `google-cloud-bigquery` python package.
|
|
|
|
```bash
|
|
pip install google-cloud-bigquery
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/document_loaders/google_bigquery).
|
|
|
|
```python
|
|
from langchain.document_loaders import BigQueryLoader
|
|
```
|
|
|
|
### Google Cloud Storage
|
|
|
|
> [Google Cloud Storage](https://en.wikipedia.org/wiki/Google_Cloud_Storage) is a managed service for storing unstructured data.
|
|
|
|
First, we need to install `google-cloud-storage` python package.
|
|
|
|
```bash
|
|
pip install google-cloud-storage
|
|
```
|
|
|
|
There are two loaders for the `Google Cloud Storage`: the `Directory` and the `File` loaders.
|
|
|
|
See a [usage example](/docs/integrations/document_loaders/google_cloud_storage_directory).
|
|
|
|
```python
|
|
from langchain.document_loaders import GCSDirectoryLoader
|
|
```
|
|
See a [usage example](/docs/integrations/document_loaders/google_cloud_storage_file).
|
|
|
|
```python
|
|
from langchain.document_loaders import GCSFileLoader
|
|
```
|
|
|
|
### Google Drive
|
|
|
|
> [Google Drive](https://en.wikipedia.org/wiki/Google_Drive) is a file storage and synchronization service developed by Google.
|
|
|
|
Currently, only `Google Docs` are supported.
|
|
|
|
First, we need to install several python packages.
|
|
|
|
```bash
|
|
pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
|
|
```
|
|
|
|
See a [usage example and authorization instructions](/docs/integrations/document_loaders/google_drive).
|
|
|
|
```python
|
|
from langchain.document_loaders import GoogleDriveLoader
|
|
```
|
|
|
|
### Speech-to-Text
|
|
|
|
> [Google Cloud Speech-to-Text](https://cloud.google.com/speech-to-text) is an audio transcription API powered by Google's speech recognition models.
|
|
|
|
This document loader transcribes audio files and outputs the text results as Documents.
|
|
|
|
First, we need to install the python package.
|
|
|
|
```bash
|
|
pip install google-cloud-speech
|
|
```
|
|
|
|
See a [usage example and authorization instructions](/docs/integrations/document_loaders/google_speech_to_text).
|
|
|
|
```python
|
|
from langchain.document_loaders import GoogleSpeechToTextLoader
|
|
```
|
|
|
|
## Vector Store
|
|
### Vertex AI Vector Search
|
|
|
|
> [Vertex AI Vector Search](https://cloud.google.com/vertex-ai/docs/matching-engine/overview),
|
|
> formerly known as Vertex AI Matching Engine, provides the industry's leading high-scale
|
|
> low latency vector database. These vector databases are commonly
|
|
> referred to as vector similarity-matching or an approximate nearest neighbor (ANN) service.
|
|
|
|
We need to install several python packages.
|
|
|
|
```bash
|
|
pip install tensorflow google-cloud-aiplatform tensorflow-hub tensorflow-text
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/vectorstores/matchingengine).
|
|
|
|
```python
|
|
from langchain.vectorstores import MatchingEngine
|
|
```
|
|
|
|
### Google ScaNN
|
|
|
|
>[Google ScaNN](https://github.com/google-research/google-research/tree/master/scann)
|
|
> (Scalable Nearest Neighbors) is a python package.
|
|
>
|
|
>`ScaNN` is a method for efficient vector similarity search at scale.
|
|
|
|
>`ScaNN` includes search space pruning and quantization for Maximum Inner
|
|
> Product Search and also supports other distance functions such as
|
|
> Euclidean distance. The implementation is optimized for x86 processors
|
|
> with AVX2 support. See its [Google Research github](https://github.com/google-research/google-research/tree/master/scann)
|
|
> for more details.
|
|
|
|
We need to install `scann` python package.
|
|
|
|
```bash
|
|
pip install scann
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/vectorstores/scann).
|
|
|
|
```python
|
|
from langchain.vectorstores import ScaNN
|
|
```
|
|
|
|
## Retrievers
|
|
### Vertex AI Search
|
|
|
|
> [Google Cloud Vertex AI Search](https://cloud.google.com/generative-ai-app-builder/docs/introduction)
|
|
> allows developers to quickly build generative AI powered search engines for customers and employees.
|
|
|
|
First, you need to install the `google-cloud-discoveryengine` Python package.
|
|
|
|
```bash
|
|
pip install google-cloud-discoveryengine
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/retrievers/google_vertex_ai_search).
|
|
|
|
```python
|
|
from langchain.retrievers import GoogleVertexAISearchRetriever
|
|
```
|
|
|
|
### Document AI Warehouse
|
|
> [Google Cloud Document AI Warehouse](https://cloud.google.com/document-ai-warehouse)
|
|
> allows enterprises to search, store, govern, and manage documents and their AI-extracted
|
|
> data and metadata in a single platform. Documents should be uploaded outside of Langchain,
|
|
>
|
|
|
|
```python
|
|
from langchain.retrievers import GoogleDocumentAIWarehouseRetriever
|
|
docai_wh_retriever = GoogleDocumentAIWarehouseRetriever(
|
|
project_number=...
|
|
)
|
|
query = ...
|
|
documents = docai_wh_retriever.get_relevant_documents(
|
|
query, user_ldap=...
|
|
)
|
|
```
|
|
|
|
## Tools
|
|
### Google Search
|
|
|
|
- Install requirements with `pip install google-api-python-client`
|
|
- Set up a Custom Search Engine, following [these instructions](https://stackoverflow.com/questions/37083058/programmatically-searching-google-in-python-using-custom-search)
|
|
- Get an API Key and Custom Search Engine ID from the previous step, and set them as environment variables `GOOGLE_API_KEY` and `GOOGLE_CSE_ID` respectively
|
|
|
|
There exists a `GoogleSearchAPIWrapper` utility which wraps this API. To import this utility:
|
|
|
|
```python
|
|
from langchain.utilities import GoogleSearchAPIWrapper
|
|
```
|
|
|
|
For a more detailed walkthrough of this wrapper, see [this notebook](/docs/integrations/tools/google_search).
|
|
|
|
We can easily load this wrapper as a Tool (to use with an Agent). We can do this with:
|
|
|
|
```python
|
|
from langchain.agents import load_tools
|
|
tools = load_tools(["google-search"])
|
|
```
|
|
|
|
### Google Places
|
|
|
|
See a [usage example](/docs/integrations/tools/google_places).
|
|
|
|
```
|
|
pip install googlemaps
|
|
```
|
|
|
|
```python
|
|
from langchain.tools import GooglePlacesTool
|
|
```
|
|
|
|
## Document Transformers
|
|
|
|
### Google Document AI
|
|
|
|
>[Document AI](https://cloud.google.com/document-ai/docs/overview) is a `Google Cloud Platform`
|
|
> service to transform unstructured data from documents into structured data, making it easier
|
|
> to understand, analyze, and consume.
|
|
|
|
We need to set up a [`GCS` bucket and create your own OCR processor](https://cloud.google.com/document-ai/docs/create-processor)
|
|
The `GCS_OUTPUT_PATH` should be a path to a folder on GCS (starting with `gs://`)
|
|
and a processor name should look like `projects/PROJECT_NUMBER/locations/LOCATION/processors/PROCESSOR_ID`.
|
|
We can get it either programmatically or copy from the `Prediction endpoint` section of the `Processor details`
|
|
tab in the Google Cloud Console.
|
|
|
|
```bash
|
|
pip install google-cloud-documentai
|
|
pip install google-cloud-documentai-toolbox
|
|
```
|
|
|
|
See a [usage example](/docs/integrations/document_transformers/docai).
|
|
|
|
```python
|
|
from langchain.document_loaders.blob_loaders import Blob
|
|
from langchain.document_loaders.parsers import DocAIParser
|
|
```
|
|
|
|
### Google Translate
|
|
|
|
> [Google Translate](https://translate.google.com/) is a multilingual neural machine
|
|
> translation service developed by Google to translate text, documents and websites
|
|
> from one language into another.
|
|
|
|
The `GoogleTranslateTransformer` allows you to translate text and HTML with the [Google Cloud Translation API](https://cloud.google.com/translate).
|
|
|
|
To use it, you should have the `google-cloud-translate` python package installed, and a Google Cloud project with the [Translation API enabled](https://cloud.google.com/translate/docs/setup). This transformer uses the [Advanced edition (v3)](https://cloud.google.com/translate/docs/intro-to-v3).
|
|
|
|
First, we need to install the python package.
|
|
|
|
```bash
|
|
pip install google-cloud-translate
|
|
```
|
|
|
|
See a [usage example and authorization instructions](/docs/integrations/document_transformers/google_translate).
|
|
|
|
```python
|
|
from langchain.document_transformers import GoogleTranslateTransformer
|
|
```
|
|
|
|
## Chat loaders
|
|
### Gmail
|
|
|
|
> [Gmail](https://en.wikipedia.org/wiki/Gmail) is a free email service provided by Google.
|
|
|
|
First, we need to install several python packages.
|
|
|
|
```bash
|
|
pip install --upgrade google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client
|
|
```
|
|
|
|
See a [usage example and authorization instructions](/docs/integrations/chat_loaders/gmail).
|
|
|
|
```python
|
|
from langchain.chat_loaders.gmail import GMailLoader
|
|
```
|
|
|
|
## Agents and Toolkits
|
|
### Gmail
|
|
|
|
See a [usage example and authorization instructions](/docs/integrations/toolkits/gmail).
|
|
|
|
```python
|
|
from langchain.agents.agent_toolkits import GmailToolkit
|
|
|
|
toolkit = GmailToolkit()
|
|
```
|
|
|
|
### Google Drive
|
|
|
|
See a [usage example and authorization instructions](/docs/integrations/toolkits/google_drive).
|
|
|
|
```python
|
|
from langchain_googledrive.utilities.google_drive import GoogleDriveAPIWrapper
|
|
from langchain_googledrive.tools.google_drive.tool import GoogleDriveSearchTool
|
|
```
|