This commit is contained in:
Jason Liu
2023-11-15 10:24:48 -05:00
parent 5706585595
commit 5f38300541
2 changed files with 5 additions and 3 deletions
+4
View File
@@ -16,6 +16,10 @@ authors:
Today, I will introduce you to various approaches for using asyncio in Python. We will apply this to batch process data using `instructor` and learn how to use `asyncio.gather` and `asyncio.as_completed` for concurrent data processing. Additionally, we will explore how to limit the number of concurrent requests to a server using `asyncio.Semaphore`.
!!! notes "Github Example"
If you want to run the code examples in this article, you can find them on [jxnl/instructor](https://github.com/jxnl/instructor/blob/main/examples/learn-async/run.py)
We will start by defining an `async` function that calls `openai` to extract data, and then examine four different ways to execute it. We will discuss the pros and cons of each approach and analyze the results of running them on a small batch.
## Understanding `asyncio`
+1 -3
View File
@@ -9,6 +9,4 @@ pytest
aiohttp==3.8.2
yarl==1.8.1
frozenlist==1.3.1
git+https://${GH_TOKEN}@github.com/squidfunk/mkdocs-material-insiders.git
mkdocs-minify-plugin
mike
mkdocs-minify-plugin