Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use tornado's AnyThreadEventLoopPolicy #991

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

fagu
Copy link
Contributor

@fagu fagu commented Aug 16, 2018

Since tornado 5.0 (with python 3), event loops by default have to be created explicitly in each new thread. This commit reinstates the old behavior in AWS and CWS.

http://www.tornadoweb.org/en/stable/asyncio.html#tornado.platform.asyncio.AnyThreadEventLoopPolicy

Apparently, the alternative is to create a new event loop inside each new thread, as described here: tornadoweb/tornado#2183

I wasn't sure where to set the AnyThreadEventLoopPolicy, but the top of main() seemed reasonable...

RWS seems to work without this change.


This change is Reviewable

Since tornado 5.0 (with python 3), event loops by default have to be
created explicitly in each new thread. This commit reinstates the old
behavior in AWS and CWS.

http://www.tornadoweb.org/en/stable/asyncio.html#tornado.platform.asyncio.AnyThreadEventLoopPolicy
@fagu fagu force-pushed the any-thread-event-loop-policy branch from d4bdbc8 to a9f81cb Compare August 16, 2018 16:13
@codecov
Copy link

codecov bot commented Aug 16, 2018

Codecov Report

Merging #991 into master will decrease coverage by 0.23%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #991      +/-   ##
==========================================
- Coverage   65.05%   64.82%   -0.24%     
==========================================
  Files         230      230              
  Lines       17731    17731              
==========================================
- Hits        11535    11494      -41     
- Misses       6196     6237      +41
Flag Coverage Δ
#functionaltests 48.03% <ø> (-0.16%) ⬇️
#unittests 45.88% <ø> (-0.09%) ⬇️
Impacted Files Coverage Δ
cms/server/admin/handlers/base.py 68.1% <0%> (-3.66%) ⬇️
cms/io/service.py 69.94% <0%> (-3.47%) ⬇️
cms/db/util.py 60.14% <0%> (-2.9%) ⬇️
cms/service/ScoringService.py 66.19% <0%> (-2.82%) ⬇️
cms/io/PsycoGevent.py 78.04% <0%> (-2.44%) ⬇️
cms/service/Worker.py 80.41% <0%> (-2.07%) ⬇️
cms/service/workerpool.py 64.35% <0%> (-1.49%) ⬇️
cms/io/rpc.py 94.77% <0%> (-1.05%) ⬇️
cms/service/ProxyService.py 59.79% <0%> (-1.04%) ⬇️
cms/io/triggeredservice.py 90% <0%> (-1%) ⬇️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d70589c...a9f81cb. Read the comment docs.

@lw
Copy link
Member

lw commented Aug 16, 2018

Your change is breaking the functional tests.

The policy you are using was introduced in Tornado 5.0. The tests pin it to 4.5, as in Ubuntu 18.04.

@fagu
Copy link
Contributor Author

fagu commented Aug 16, 2018

Yes, sorry! I've added a check for "tornado version >= 5.0" now.

@lw
Copy link
Member

lw commented Aug 16, 2018

Ok, the commit you just pushed should fix that. I'm not understanding however what you're trying to fix. The Tornado doc page you link mentions utils for bridging Tornado with asyncio. Your code also imports asyncio. We're not using asyncio, however, so how can it cause problems?

@fagu
Copy link
Contributor Author

fagu commented Aug 16, 2018

I was getting errors like this when opening the AWS web page:

Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/gevent/pywsgi.py", line 975, in handle_one_response
self.run_application()
File "/usr/lib/python3.7/site-packages/gevent/pywsgi.py", line 922, in run_application
self.result = self.application(self.environ, self.start_response)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/io/web_service.py", line 120, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/admin/authentication.py", line 128, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/local.py", line 228, in application
return ClosingIterator(app(environ, start_response), self.cleanup)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/admin/authentication.py", line 158, in wsgi_app
return self._app(environ, my_start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 826, in call
return app(environ, start_response)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/file_middleware.py", line 76, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 48, in
return update_wrapper(lambda *a: f(*a)(a[-2:]), f)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/file_middleware.py", line 86, in wsgi_app
original_response = Response.from_app(self.wrapped_app, environ)
File "/usr/lib/python3.7/site-packages/werkzeug/wrappers.py", line 939, in from_app
return cls(
_run_wsgi_app(app, environ, buffered))
File "/usr/lib/python3.7/site-packages/werkzeug/wrappers.py", line 59, in _run_wsgi_app
return _run_wsgi_app(*args)
File "/usr/lib/python3.7/site-packages/werkzeug/test.py", line 923, in run_wsgi_app
app_rv = app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 766, in call
return self.app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 766, in call
return self.app(environ, start_response)
File "/usr/lib/python3.7/site-packages/tornado/wsgi.py", line 83, in call
return WSGIAdapter(self)(environ, start_response)
File "/usr/lib/python3.7/site-packages/tornado/wsgi.py", line 242, in call
self.application(request)
File "/usr/lib/python3.7/site-packages/tornado/wsgi.py", line 207, in
application, request)
File "/usr/lib/python3.7/site-packages/tornado/web.py", line 2097, in call
return dispatcher.execute()
File "/usr/lib/python3.7/site-packages/tornado/web.py", line 2228, in execute
**self.path_kwargs)
File "/usr/lib/python3.7/site-packages/tornado/gen.py", line 297, in wrapper
future = _create_future()
File "/usr/lib/python3.7/site-packages/tornado/gen.py", line 187, in _create_future
future = Future()
File "/usr/lib/python3.7/asyncio/events.py", line 644, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'DummyThread-7'.
2018-08-16T15:22:30Z {'REMOTE_ADDR': '::1', 'REMOTE_PORT': '47688', 'HTTP_HOST': 'localhost:8889', (hidden keys: 24)} failed with RuntimeError

::1 - - [2018-08-16 17:22:30] "GET / HTTP/1.1" 500 161 0.012203
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/gevent/pywsgi.py", line 975, in handle_one_response
self.run_application()
File "/usr/lib/python3.7/site-packages/gevent/pywsgi.py", line 922, in run_application
self.result = self.application(self.environ, self.start_response)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/io/web_service.py", line 120, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/admin/authentication.py", line 128, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/local.py", line 228, in application
return ClosingIterator(app(environ, start_response), self.cleanup)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/admin/authentication.py", line 158, in wsgi_app
return self._app(environ, my_start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 826, in call
return app(environ, start_response)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/file_middleware.py", line 76, in call
return self.wsgi_app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 48, in
return update_wrapper(lambda *a: f(*a)(a[-2:]), f)
File "/usr/lib/python3.7/site-packages/cms-1.4.dev0-py3.7.egg/cms/server/file_middleware.py", line 86, in wsgi_app
original_response = Response.from_app(self.wrapped_app, environ)
File "/usr/lib/python3.7/site-packages/werkzeug/wrappers.py", line 939, in from_app
return cls(
_run_wsgi_app(app, environ, buffered))
File "/usr/lib/python3.7/site-packages/werkzeug/test.py", line 923, in run_wsgi_app
app_rv = app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 766, in call
return self.app(environ, start_response)
File "/usr/lib/python3.7/site-packages/werkzeug/wsgi.py", line 766, in call
return self.app(environ, start_response)
File "/usr/lib/python3.7/site-packages/tornado/wsgi.py", line 83, in call
return WSGIAdapter(self)(environ, start_response)
File "/usr/lib/python3.7/site-packages/tornado/wsgi.py", line 242, in call
self.application(request)
File "/usr/lib/python3.7/site-packages/tornado/wsgi.py", line 207, in
application, request)
File "/usr/lib/python3.7/site-packages/tornado/web.py", line 2097, in call
return dispatcher.execute()
File "/usr/lib/python3.7/site-packages/tornado/web.py", line 2228, in execute
**self.path_kwargs)
File "/usr/lib/python3.7/site-packages/tornado/gen.py", line 297, in wrapper
future = _create_future()
File "/usr/lib/python3.7/site-packages/tornado/gen.py", line 187, in _create_future
future = Future()
File "/usr/lib/python3.7/asyncio/events.py", line 644, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'DummyThread-8'.
2018-08-16T15:22:30Z {'REMOTE_ADDR': '::1', 'REMOTE_PORT': '47690', 'HTTP_HOST': 'localhost:8889', (hidden keys: 24)} failed with RuntimeError

::1 - - [2018-08-16 17:22:30] "GET /favicon.ico HTTP/1.1" 500 161 0.001120

@fagu
Copy link
Contributor Author

fagu commented Aug 16, 2018

In case that helps, here's another reference to the AnyThreadEventLoopPolicy fix:
https://github.com/tornadoweb/tornado/blob/branch5.1/tornado/wsgi.py#L186
I'm guessing only tornado is using asyncio, and for some reason needs help setting it up. :)

@lw
Copy link
Member

lw commented Aug 16, 2018

Tornado should have nothing to do with IO loops or asynchronicity. We take care of it, through gevent, and just use Tornado via WSGI for routing and handling. I didn't look deep into it (yet) but it appears that v5 became "asyncio-only", which could mean that asyncio stuff has creeped into places where it shouldn't be. I don't think we should tolerate Tornado's use of asyncio, so the fix for this issue should go another way (in your fix, Tornado would start an asyncio loop, which most likely won't play nicely with gevent, be starved, and lead to who knows what chaos).

While I was looking at the changelog I read that they deprecated WSGI support (as if one could ever have called it "support") and will remove it in v6. Maybe it's time to replace Tornado by something else. Or to replace gevent by asyncio. Let's see what @stefano-maggiolo is less disgusted by.

@stefano-maggiolo
Copy link
Member

image

@stefano-maggiolo
Copy link
Member

I suppose removing Tornado is more pressing. There are talks to have gevent and asyncio play nice and include that in gevent... gevent/gevent#982

@cantonios
Copy link

I get the same error. Ubuntu 18.10 seems to install tornado 5.0.2

RezwanArefin01 pushed a commit to RezwanArefin01/cms that referenced this pull request Dec 27, 2021
Tornado still remains pinned to the 4.x series (see cms-dev#991). Fortunately,
Ubuntu 20.04 has a python3-tornado4 package, which provides just that.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants