A Simple Websocket
Here we make a simple echo websocket using
asyncio. We define coroutines for connecting to a server and sending/receiving messages. The communcations of the websocket are run in a
main coroutine, which is run by an event loop. This example is modified from a prior post.
Note: Uses the Python 3.5+ async/await syntax
asyncio supports the use of
Executor objects found in
concurrent.futures for scheduling tasks asynchronously. Event loops have the function
run_in_executor() which takes an
Executor object, a
Callable, and the Callable's parameters.
Scheduling a task for an
Each event loop also has a "default"
Executor slot that can be assigned to an
Executor. To assign an
Executor and schedule tasks from the loop you use the
There are two main types of
ThreadPoolExecutor and the
ThreadPoolExecutor contains a pool of threads which can either be manually set to a specific number of threads through the constructor or defaults to the number of cores on the machine times 5. The
ThreadPoolExecutor uses the pool of threads to execute tasks assigned to it and is generally better at CPU-bound operations rather than I/O bound operations.
Contrast that to the
ProcessPoolExecutor which spawns a new process for each task assigned to it. The
ProcessPoolExecutor can only take tasks and parameters that are picklable. The most common non-picklable tasks are the methods of objects. If you must schedule an object's method as a task in an
Executor you must use a
Common Misconception about asyncio
probably the most common misconception about
asnycio is that it lets you run any task in parallel - sidestepping the GIL (global interpreter lock) and therefore execute blocking jobs in parallel (on separate threads). it does not!
asyncio (and libraries that are built to collaborate with
asyncio) build on coroutines: functions that (collaboratively) yield the control flow back to the calling function. note
asyncio.sleep in the examples above. this is an example of a non-blocking coroutine that waits 'in the background' and gives the control flow back to the calling function (when called with
time.sleep is an example of a blocking function. the execution flow of the program will just stop there and only return after
time.sleep has finished.
a real-live example is the
requests library which consists (for the time being) on blocking functions only. there is no concurrency if you call any of its functions within
aiohttp on the other hand was built with
asyncio in mind. its coroutines will run concurrently.
if you have IO-bound jobs running, you may run them concurrently using
Coroutine and Delegation Syntax
Before Python 3.5+ was released, the
asyncio module used generators to mimic asynchronous calls and thus had a different syntax than the current Python 3.5 release.
Python 3.5 introduced the
await keywords. Note the lack of parentheses around the
await func() call.
Before Python 3.5, the
@asyncio.coroutine decorator was used to define a coroutine. The yield from expression was used for generator delegation. Note the parentheses around the
yield from func().
Here is an example that shows how two functions can be run asynchronously:
Synchronization Primitive: Event
Event to synchronize the scheduling of multiple coroutines.
Put simply, an event is like the gun shot at a running race: it lets the runners off the starting blocks.
Consumer B waiting
Consumer A waiting
Consumer B triggered
Consumer A triggered
uvloop is an implementation for the
asyncio.AbstractEventLoop based on libuv (Used by nodejs). It is compliant with 99% of
asyncio features and is much faster than the traditional
uvloop is currently not available on Windows, install it with
pip install uvloop.
One can also change the event loop factory by setting the
EventLoopPolicy to the one in