HOW TO DEAL WITH BLOCKING CODE WITHIN ASYNCIO EVENT LOOP

HOW TO DEAL WITH
BLOCKING CODE WITHIN
ASYNCIO EVENT LOOP
Nikolay Novik
http://github.com/jettify

I AM ...
Software Engineer: at DataRobot Ukraine
Github:
My Projects:
database clients:
aiomysql, aioobc, aiogibson
web and etc:
aiohttp_debugtoolbar, aiobotocore,
aiohttp_mako, aiohttp_sse, aiogearman,
aiomysql_replicatoin
http://github.com/jettify

POLL
YOU AND ASYNCIO:
1. I am using asyncio extensively
2. I am using Twisted, Tornado, gevent etc. extensively
3. I think async programming is kinda cool

ASYNCIO
The asyncio project was ofﬁcially launched with the release
of Python 3.4 in March 2014.
Bare: almost no library
One year later, asyncio has a strong community writing
libraries on top of it.
But what to do when available libraries work in sync way,
potentially blocking event loop?

RULES OF ASYNC CLUB
RULE #1
You do not block event loop
RULE #2
You never block event loop

BLOCKING CALLS IN THIRD PARTY
LIBRARIES
Network IO
API wrappers
Database clients
Message queues
FileSystem IO
CPU

Set environment variable
DEBUGGING BLOCKING CALLS TIP
PYTHONASYNCIODEBUG=1
import asyncio
import time
loop = asyncio.get_event_loop()
loop.slow_callback_duration = 0.01
async def sleeper():
time.sleep(0.1) # we block here
loop.run_until_complete(sleeper())
Executing <Task finished coro=<sleeper() done, defined at
code/debug_example.py:9> result=None created at
/usr/local/lib/python3.5/asyncio/base_events.py:323>
took 0.102 seconds

APPROACH #1 IS THERE ANY
SUITABLE LIBRARY?
Search asyncio compatible library on:
1. google ~ 98k results
2. pypi ~200 packages
3. asyncio wiki page:
4. aio-libs:
https://github.com/python/asyncio/wiki/ThirdParty
https://github.com/aio-libs

Read the (f*g) source code of your libraries! Example of
python code from OneDrive SDK
Most of the time you want to do HTTP requests using event
loop not thread pool.
THIRD PARTY LIBRARIES PRO TIP
@asyncio.coroutine
def get_async(self):
"""Sends the GET request using an asyncio coroutine
....
"""
future = self._client._loop.run_in_executor(None,
self.get)
collection_response = yield from future
return collection_response

Most hipsters databases use REST API as primary access
method:
Easy to implement required subset of APIs.
APPROACH #2 IS REST API AVAILABLE?
DynamoDB
Neo4j
Elasticsearch
HBase
HDFS
CouchDB
Riak
VoltDB
InﬂuxDB
ArangoDB

is your friend
Connection pooling helps to save on expensive connection
creation. (PS: checkout new aiohttp 0.18.x release)
REST CLIENT TIP
aiohttp.ClientSession
import asyncio
import aiohttp
# carry the loop Luke!
async def go():
session = aiohttp.ClientSession(loop=loop)
async with session.get('http://python.org') as resp:
data = await resp.text()
print(data)
session.close()
loop.run_until_complete(go())

APPROACH #3 IS THERE SIMPLE TEXT
OR BINARY PROTOCOL?
Example of databases and message queues with binary
protocol:
redis
memcached
couchebase
grearman
beanstalkd
disque
Do not afraid to get your hands dirty.

Example: Simple binary protocol implementation
Simple but no protocol pipelining.
import asyncio, struct
from aiogibson import encode_command
async def read_from_connection(host, port, *, loop=None):
reader, writer = await asyncio.open_connection(
host, port, loop=loop)
cmd = encode_command(b'GET', 'key')
writer.write(cmd)
header = await reader.readexactly(4 + 2 + 1)
unpacked = struct.unpack(b'<HBI', header)
code, gb_encoding, resp_size = unpacked
payload = await reader.readexactly(resp_size)
print(payload)

Most binary protocols support pipelining
More info: http://tailhook.github.io/request-pipelining-
presentation/ presentation/index.html
PROTOCOL PIPELINING

Example: Simple pipelined binary protocol implementation
See for reference implementation.
def execute(self):
cmd = encode_command(b'GET', 'key')
self.writer.write(cmd)
fut = asyncio.Future(loop=self._loop)
self._queue.append(fut)
return fut
async def reader_task(self):
while True:
header = await self.reader.readexactly(4 + 2 + 1)
unpacked = struct.unpack(b'<HBI', header)
code, gb_encoding, resp_size = unpacked
# wait and read payload
payload = await reader.readexactly(resp_size)
future = self._queue.pop()
future.set_result(payload)
aioredis

In good sync database clients IO decoupled from protocol
parsers why not just rewrite IO part?
APPROACH #4 IS SYNC PYTHON CLIENT
AVAILABLE?
1. Locate socket.recv()
2. Replace with await reader.read()
3. Make function coroutine with async def
4. Call this function with await
5. Call parent functions with await

Yes. Make every blocking call in separate thread
APPROACH #5 IS THERE UNIVERSAL
SOLUTION TO ALL PROBLEMS?
import asyncio
from pyodbc import connect
executor = ThreadPoolExecutor(max_workers=4)
async def test_example():
dsn = 'Driver=SQLite;Database=sqlite.db'
conn = await loop.run_in_executor(executor, connect, dsn)
cursor = await loop.run_in_executor(executor, conn.cursor)
conn = await loop.run_in_executor(executor, cursor.execute,
'SELECT 42;')
loop.run_until_complete(test_example())

For python code
For Cython
BUT HOW I KNOW WHICH METHOD TO
CALL IN THREAD?
requests.get()
with nogil:
[code to be executed with the GIL released]
For C extension
Py_BEGIN_ALLOW_THREADS
ret = SQLDriverConnect(hdbc, 0, szConnect, SQL_NTS,
0, 0, 0, SQL_DRIVER_NOPROMPT);
Py_END_ALLOW_THREADS

Only good way to use ﬁles asynchronously by using thread
pools.
WHAT ABOUT FILESYSTEM IO?
asyncio does not support asynchronous operations on the
ﬁlesystem due to OS limitations.

On background uses ThreadPoolExecutor for
blocking calls.
aiofiles library workaround
async def go():
f = await aiofiles.open('filename', mode='r')
try:
data = await f.read()
finally:
await f.close()
print(data)
loop.run_until_complete(go())
aiofiles

WHAT ABOUT CPU INTENSIVE TASK?
executor = ProcessPoolExecutor(max_workers=3)
def is_prime(n):
if n % 2 == 0: return False
sqrt_n = int(math.floor(math.sqrt(n)))
for i in range(3, sqrt_n + 1, 2):
if n % i == 0: return False
return True
async def go():
n = 112272535095293
result = await loop.run_in_executor(executor, is_prime, n)
loop.run_until_complete(go(loop, executor))

HOW TO DEAL WITH BLOCKING CODE WITHIN ASYNCIO EVENT LOOP

Recommended

More Related Content

Similar to HOW TO DEAL WITH BLOCKING CODE WITHIN ASYNCIO EVENT LOOP (20)

Recently uploaded (20)

HOW TO DEAL WITH BLOCKING CODE WITHIN ASYNCIO EVENT LOOP