A lot has changed since I last published my post on handling long-running async tasks in Python using celery. Last time we used it to run a few async tasks to fetch data from some service which took around 1–5 minutes.

自从我上次发表关于使用celery处理Python中长时间运行的异步任务的文章以来，发生了很多变化。 上一次，我们使用它运行一些异步任务来从某些服务中获取数据，这大约需要1-5分钟。

Although I will still use celery for these types of long-running tasks, there is a subset of tasks that are better handled inside the execution thread itself. Today we will discuss a way of handling such operations.

尽管我仍将芹菜用于这些类型的长时间运行的任务，但仍有一部分任务可以在执行线程内部更好地处理。 今天，我们将讨论处理此类操作的方法。

To the people who know me, I switched to a new role which allowed me to write Node.js over a year ago and I have been working on it since then. During my time working and building APIs using it, I have started falling for the way in which Node handles Async behavior and long-running tasks.

对于认识我的人来说，我转用了一个新角色，该角色使我一年多以前就可以编写Node.js，并且从那时起我一直在努力。 在使用它工作和构建API的过程中，我开始迷恋Node处理异步行为和长时间运行的任务的方式。

The idea is simple, if the task is waiting for some IO, we can run some other task in the mean-time. Only the part that needs it, will wait for it.

这个想法很简单，如果任务正在等待一些IO，我们可以同时运行其他任务。 只有需要它的部分才会等待。

I wanted to use the same concept in Python.

我想在Python中使用相同的概念。

Fortunately, in the Python version 3.4, they have introduced asyncio for the same purpose. In this post, we are going to talk a little about it and try to understand its importance.

幸运的是，在Python版本3.4 ，他们出于相同的目的引入了asyncio 。 在这篇文章中，我们将讨论一下它，并试图理解它的重要性。

By the end of the post, you will be able to understand the importance of async functions and will be able to start using them into your codebase.

在本文的结尾，您将能够理解异步函数的重要性，并将能够开始在代码库中使用它们。

先决条件 (Prerequisite)

I am using Python 3.8 for this tutorial and you might want to use the same to run and try out examples. You can try them online as well.

我在本教程中使用的是Python 3.8 ，您可能希望使用相同的代码来运行并试用示例。 您也可以在线尝试它们。

Python中的基本异步功能 (Basic async function in Python)

This is how we write a basic async function in Python.

这就是我们用Python编写基本异步函数的方式。

import asyncioasync def func1(a): print(f"started func 1: {a + 1}") await asyncio.sleep(1) return a + 1asyncio.run(func1(1))

Response

响应

started func 1: 22

You have to start by importing the asyncio module. Async functions are defined just like a normal function, you just have to add the keyword, async. If you want to wait for the execution of some coroutine, you have to use the keyword, await. Finally, you can run the function using asyncio.run method.

您必须先导入asyncio模块。 异步函数的定义与普通函数一样，只需添加关键字async 。 如果要等待某些协程的执行，则必须使用关键字await 。 最后，您可以使用asyncio.run方法运行该函数。

Anytime you use the await keyword inside the async function, the thread will not move forward until we get a response from the called function.

每当您在async函数中使用await关键字时，线程都不会向前移动，直到我们从被调用函数获得响应为止。

This simple implementation will not help you understand the advantage of using async functions. In the next section, we will talk about the time taken for the execution and there you will really start to see the difference.

这个简单的实现不会帮助您了解使用异步功能的优势。 在下一节中，我们将讨论执行所需的时间，您将真正开始看到其中的区别。

完全执行所需的时间 (Time taken for the full execution)

To understand the benefits of using async functions, we will compare the time taken to run the same code in async and sync behavior.

为了了解使用async功能的好处，我们将比较在async和sync行为中运行相同代码所花费的时间。

Let’s first write the async code.

首先让我们编写异步代码。

import asyncioimport timestart = time.time()async def async_sleep(): await asyncio.sleep(1)async def func1(a): print(f"started func 1: {a + 1}") await async_sleep() return a + 1async def func2(a): print(f"started func 2: {a + 2}") await async_sleep() return a + 2async def func3(a): print(f"started func 3: {a + 3}") await async_sleep() return a + 3async def func4(a): print(f"started func 4: {a + 4}") await async_sleep() return a + 4async def main(): tasks = (func1(1), func2(1), func3(1), func4(1)) await asyncio.gather(*tasks) print(f"Completed after: {time.time() - start}")asyncio.run(main())

I have defined 4 different functions doing almost the same thing. Let’s assume there is a long-running function called async_sleep which is doing some IO operation. For example: Fetching data from the database.

我定义了4个执行几乎相同功能的不同函数。 假设有一个名为async_sleep的长时间运行的函数正在执行一些IO操作。 例如：从数据库中获取数据。

asyncio.gather() is used to run the tasks passed to it concurrently.

asyncio.gather()用于同时运行传递给它的任务。

The response after running this code is as follows.

运行此代码后的响应如下。

started func 1: 2started func 2: 3started func 3: 4started func 4: 5Completed after: 1.1852593421936035

The function completes the execution somewhere between 1–2 seconds for every execution. (Ran it 5 times to check).

对于每次执行，该函数都会在1-2秒之间完成执行。 (运行5次以进行检查)。

Now let’s try to run the sync version of the same code.

现在，让我们尝试运行相同代码的sync版本。

import timestart = time.time()def sync_sleep(): time.sleep(1)def func1(a): print(f"started func 1: {a + 1}") sync_sleep() return a + 1def func2(a): print(f"started func 2: {a + 2}") sync_sleep() return a + 2def func3(a): print(f"started func 3: {a + 3}") sync_sleep() return a + 3def func4(a): print(f"started func 4: {a + 4}") sync_sleep() return a + 4def main(): func1(1) func2(1) func3(1) func4(1) print(f"Completed after: {time.time() - start}")main()

The response that we got after executing this was:

执行此操作后得到的响应是：

started func 1: 2started func 2: 3started func 3: 4started func 4: 5Completed after: 4.168870687484741

This saved us ~3 seconds for executing 4 functions. Now if we wanted to run 10000 such functions, we would save ourselves ~2000 seconds/ i.e. 35-40 minutes.

这为我们执行4个功能节省了~3秒钟的时间。 现在，如果我们要运行10000个这样的功能，我们将节省~2000秒/ 35-40分钟。

That’s awesome, right.

太好了，对。

Even if we are just running 2–3 such functions in each iteration these small savings can be very important to provide better user experience to your customers.

即使我们在每次迭代中仅运行2-3个这样的功能，这些少量节省对于为您的客户提供更好的用户体验也非常重要。

Now that we have understood the advantage of using async functions, we must know a few more things about it.

既然我们已经了解了使用async函数的优势，那么我们必须进一步了解它。

异步功能的执行顺序 (Order of execution for async functions)

We should understand that while running the functions in async mode, we don’t really know about the order in which functions are executed. Let’s understand the concept with an example.

我们应该理解，以异步模式运行函数时，我们实际上并不知道函数的执行顺序。 让我们通过一个例子来理解这个概念。

The functions will return as soon as the execution is completed. We can mock the different execution time by sleeping for a random time between 0 and 1 using, random.randint(0, 10) * 0.1.

执行完成后，函数将立即返回。 我们可以通过使用random.randint(0, 10) * 0.1Hibernate0到1之间的一个随机时间来模拟不同的执行时间。

Here is the full code.

这是完整的代码。

import asyncioimport timeimport randomstart = time.time()def random_sleep_time(): return random.randint(0, 10) * 0.1async def async_sleep(): await asyncio.sleep(random_sleep_time())async def func1(a): await async_sleep() print(f"completed func {a}: {a + 1}") return a + 1async def main(): tasks = [func1(a) for a in range(0, 5)] await asyncio.gather(*tasks) print(f"Completed after: {time.time() - start}")asyncio.run(main())

The response to the following code is,

对以下代码的响应是：

completed func 0: 1completed func 3: 4completed func 1: 2completed func 2: 3completed func 4: 5Completed after: 0.9764895439147949

实际使用 (Practical Usage)

After the release of Python 3.8, which moved the asyncio.run() method to stable API, you can start using it without any problem.

Python 3.8发行后，将asyncio.run()方法移至稳定的API，您可以毫无问题地开始使用它。

The only issue with moving from a synchronous approach to an async one, is to change the way of thinking about the problems. You have to change the way you think about every little detail. Once you reach that point of thinking in terms of async functions, believe me, you will love it.

从synchronous方法过渡到异步方法的唯一问题是改变对问题的思考方式。 您必须更改对每个小细节的思考方式。 一旦您达到了关于异步功能的思维点，请相信我，您会喜欢它的。

它是如何工作的？ (How does it work under the hood?)

Let’s check the type of the async function.

让我们检查一下异步函数的类型。

type(main())# <class 'coroutine'>

So, the async function is of type coroutine.

因此，异步函数的类型为coroutine 。

This GeeksForGeeks article does a very good job of explaining coroutines.

这篇GeeksForGeeks文章在解释协程方面做得很好。

To sum up what they wrote in the post, subroutines are the set of instructions that have one entry point and are executed serially.

总结一下他们在帖子中写的内容，子例程是一组具有一个入口点并按顺序执行的指令。

On the other hand, Coroutines can suspend or give away the control to other Coroutines, allowing multiple tasks to run at the same time.

另一方面，协程可以暂停或将控制权交给其他协程，从而允许多个任务同时运行。

Python uses this concept to allow async behavior.

Python使用此概念允许异步行为。

在异步函数中处理超时 (Handling timeout in async functions)

It is necessary to have a timeout for waiting for the task to get completed. We are not supposed to wait forever for it to complete. asyncio also provided the ability to add a timeout to the async function so that you can skip the execution before its completion.

必须有一个超时等待任务完成。 我们不应该永远等待它完成。 asyncio还提供了向async函数添加超时的功能，以便您可以在执行完成之前跳过执行。

A practical application of this can be a case when you are calling a third party API in your application and the third party itself is down. In that case, you don’t want to wait for a long time.

当您在应用程序中调用第三方API且第三方本身宕机时，可能是这种情况的实际应用。 在这种情况下，您不想等待很长时间。

You can use the timeout method to solve your problem.

您可以使用timeout方法来解决您的问题。

See the code for example:

参见代码示例：

async def async_sleep(): await asyncio.sleep(2) print('Execution completed')async def main(): try: await asyncio.wait_for(async_sleep(), timeout=1.0) except asyncio.TimeoutError: print('Timeout error')asyncio.run(main())

The async function will raise TimeoutError if the coroutine doesn't return before the given timeout is passed. Response after executing the above code is

如果协程在给定timeout之前未返回，则异步函数将引发TimeoutError 。 执行以上代码后的响应是

Timeout error

There are other awesome methods in the asyncio module. You can check them at python's official website.

asyncio模块中还有其他很棒的方法。 您可以在python的官方网站上查看它们。

结论 (Conclusion)

If you want to provide a better and faster experience to your users, you can start using the asyncio module in your application. This will definitely help you find cases in which you can reduce the execution time of your API or whatever else that you are doing.

如果要为用户提供更好，更快的体验，则可以开始在应用程序中使用asyncio模块。 这绝对可以帮助您找到可以减少API或其他操作时间的情况。

Just forcing yourself to write in this way for a few months can yield big returns in the future.

仅仅强迫自己用这种方式写作几个月，就可以在将来获得丰厚的回报。

Originally published at https://ranvir.xyz on August 30, 2020.

最初于2020年8月30日在https://ranvir.xyz上发布。

升级编码 (Level Up Coding)

Thanks for being a part of our community! Subscribe to our YouTube channel or join the Skilled.dev coding interview course.

感谢您加入我们的社区！ 订阅我们的YouTube频道或参加Skilled.dev编码面试课程。

翻译自: https://levelup.gitconnected.com/achieving-asynchronous-behavior-using-asyncio-in-python-e09153defbd9