So what if you have an existing python code base but you are unable to scale it because of varoius dependencies like blocking I/O, network connections e.t.c
Look into eventlet !!! (thanks to my co-worker who suggested not looking into twisted to solve the problem since it was just plainly painful .. but look into eventlet)
easy to install , easy to integrate into existing code base ( if its not too late and if you are not using any C based libraries in your python code base)
They call it "greening"
So I ran into a similar blocking issue and the way i greened my application is simply by monkeypatching the standard library.
Now try to think how on highlevel you can sort of create threads on the blocking process and spawn threads of that blocking call.
( for ex:- lets say the blocking function has a network connection or some other blocking I/O and it is called foo(p1,p2) and it is using a standard python library )
Look into eventlet !!! (thanks to my co-worker who suggested not looking into twisted to solve the problem since it was just plainly painful .. but look into eventlet)
easy to install , easy to integrate into existing code base ( if its not too late and if you are not using any C based libraries in your python code base)
They call it "greening"
So I ran into a similar blocking issue and the way i greened my application is simply by monkeypatching the standard library.
pip install eventlet
Then add these lines as soon as your programme starts ( to avoid late binding problems)
import eventlet eventlet.monkey_patch()
Now try to think how on highlevel you can sort of create threads on the blocking process and spawn threads of that blocking call.
( for ex:- lets say the blocking function has a network connection or some other blocking I/O and it is called foo(p1,p2) and it is using a standard python library )
pool = eventlet.GreenPool()
pool.spawn_n(foo, p1, p2)
pool.waitall()
voila! you are async again!!! no matter how much time foo takes to complete .. you can move ahead :)
No comments:
Post a Comment