[{"user_id": 5639, "stars": [{"date_created": 1297367448.637373, "user_id": 1376}, {"date_created": 1297369343.0723209, "user_id": 38}, {"date_created": 1297390690.305614, "user_id": 222}, {"date_created": 1297396575.4651799, "user_id": 3751}, {"date_created": 1297413037.1223991, "user_id": 6810}], "topic_id": 3716, "date_created": 1297356737.506587, "message": "A coworker just posted this on his blog. Cool stuff. http://metachris.org/2011/01/scaling-python-servers-with-worker-processes-and-socket-duplication/", "group_id": 292, "id": 71309}, {"user_id": 5639, "stars": [], "topic_id": 3716, "date_created": 1297363896.3500149, "message": "yeah. I'm kind of wondering what else isn't documented in there. Maybe instead of a 'code sprint' we can have a 'code sift' for people interested in a common Python module or something at PyCon :-D", "group_id": 292, "id": 72589}, {"user_id": 1152, "stars": [], "topic_id": 3716, "date_created": 1297362082.915597, "message": "Niiiiiiiiiice", "group_id": 292, "id": 72344}, {"user_id": 1376, "stars": [], "topic_id": 3716, "date_created": 1297367566.406249, "message": "@jonesy Woah, that's hot shit!", "group_id": 292, "id": 73550}, {"user_id": 2768, "stars": [], "topic_id": 3716, "date_created": 1297370156.3915949, "message": "Bookmarked. Twice.", "group_id": 292, "id": 74083}, {"user_id": 1078, "stars": [], "topic_id": 3716, "date_created": 1297376105.256532, "message": "Yeah, I know. It was just \"if\" :-)", "group_id": 292, "id": 75388}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297376091.218632, "message": "the GIL is never going anywhere on CPython, sorry :(", "group_id": 292, "id": 75384}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297376117.2018099, "message": "Cpython being the key word", "group_id": 292, "id": 75392}, {"user_id": 927, "stars": [{"date_created": 1297399579.0200861, "user_id": 1376}], "topic_id": 3716, "date_created": 1297375707.5385089, "message": "@jonesy A \"stuff which should be more discoverable\" sprint could be very interesting", "group_id": 292, "id": 75273}, {"user_id": 1078, "stars": [], "topic_id": 3716, "date_created": 1297376061.0253201, "message": "Nice. In terms of performance, I suppose that threading solution (if there won't be GIL) will work faster anyway, due to absent IPC stuff (marshalling and co)", "group_id": 292, "id": 75377}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297390497.3470869, "message": "IPC adds a little overhead, and also the duplication of the socket which happens once when the client connects. \nI suspect this approach might not always be the best for an HTTP server, but I think for long lived connections with multiple requests per session this can be a very well performing solution.", "group_id": 292, "id": 77562}, {"user_id": 507, "stars": [], "topic_id": 3716, "date_created": 1297392632.5360079, "message": "I'm not entirely sure how the is implemented in multiprocessing but we use a similar trick in gunicorn to share a listening socket across all worker processes.", "group_id": 292, "id": 77874}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297394243.1209061, "message": "[1] https://github.com/facebook/tornado/blob/master/tornado/httpserver.py#L175", "group_id": 292, "id": 78083}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297393697.5223501, "message": "I have been playing a bit with listening socket preforking too, but found that often it doesn't bring much performance. For example tornado (1) has this built in as well, but it actually yields lower performance when tested with httperf. If it would work well that would be a really good approach too. @davis Do you have any benchmarks on the preforking?", "group_id": 292, "id": 78051}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297397291.2588029, "message": "until they get COW'd", "group_id": 292, "id": 78561}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297397363.2385299, "message": ":)", "group_id": 292, "id": 78567}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397263.293618, "message": "You end up paying the cost of garbage collection really heavily, as every process needs to page in just about everything in the universe to touch its reference count.", "group_id": 292, "id": 78559}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397369.012094, "message": "and the GC has to do that read on every single object in the universe in order to do full cycle collection every so often", "group_id": 292, "id": 78568}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397372.3511109, "message": "don't prefork in python", "group_id": 292, "id": 78571}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397374.3691649, "message": "Socket-sharing can be accomplished via means other than preforking though", "group_id": 292, "id": 78572}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397430.4615459, "message": "and maybe that'll be in Twisted itself someday (the idea is for it to be, but you know, too much to do)", "group_id": 292, "id": 78577}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397369.963078, "message": "so yeah", "group_id": 292, "id": 78569}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297397355.905731, "message": "yeah I was about to type that", "group_id": 292, "id": 78565}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397230.3136511, "message": "Forking in python is bad news.", "group_id": 292, "id": 78554}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397348.372426, "message": "even a trivial read on a python object has to write the reference count twice though (incref/decref)", "group_id": 292, "id": 78564}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297397394.722687, "message": "the benchmarks i did looked very promising", "group_id": 292, "id": 78574}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297397417.573056, "message": "CalendarServer does this for Twisted: http://trac.calendarserver.org/browser/CalendarServer/trunk/twext/internet/sendfdport.py", "group_id": 292, "id": 78576}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297397547.55796, "message": "twisted is somewhat of a special case, at it's epoll implementation and other internals are written in c", "group_id": 292, "id": 78588}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297397564.564491, "message": "please move to using stdlib epoll so it works on PyPy, thanks :)", "group_id": 292, "id": 78590}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297400385.5609879, "message": "@alex: you are welcome to contribute patches at any time: http://twistedmatrix.com/trac/ticket/3114", "group_id": 292, "id": 78712}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297400480.9696391, "message": "also I don't see how Twisted is a special case; nothing works differently due to our epoll being weird.", "group_id": 292, "id": 78714}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297400584.213845, "message": "@glyph well pypy won't work with your custom C-extension, but we do implement the stdlib select.epoll", "group_id": 292, "id": 78717}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297400642.258739, "message": "There's 0 way for CPyext to ever be as fast as native", "group_id": 292, "id": 78720}, {"user_id": 6671, "stars": [], "topic_id": 3716, "date_created": 1297400623.8645339, "message": "@alex: go work on cpyext then, our C extension is very plain :-P", "group_id": 292, "id": 78719}, {"user_id": 222, "stars": [], "topic_id": 3716, "date_created": 1297406400.0027189, "message": "Why not have Twisted's implementation use the stdlib epoll if it's there? Is it broken in CPython?", "group_id": 292, "id": 78899}, {"user_id": 5701, "stars": [], "topic_id": 3716, "date_created": 1297430437.1712079, "message": "@bob: \"Why not\" is only that no one has finished  http://twistedmatrix.com/trac/ticket/3114 yet.", "group_id": 292, "id": 80233}, {"user_id": 5701, "stars": [], "topic_id": 3716, "date_created": 1297430521.1620049, "message": "@alex: \"won't work\" sounds like a different thing than won't \"be as fast as native\" :)", "group_id": 292, "id": 80236}, {"user_id": 7, "stars": [], "topic_id": 3716, "date_created": 1297430953.424232, "message": "@jpcalderone well I hav eno idea if it works :)", "group_id": 292, "id": 80273}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297445356.7781219, "message": "@davisp I did some gunicorn benchmarks: http://dpaste.de/5wPN/ It also yields lower results with increasing worker processes. Is there another benchmark for a situation where it actually performs better?", "group_id": 292, "id": 81622}, {"user_id": 5701, "stars": [], "topic_id": 3716, "date_created": 1297445737.9989641, "message": "@metachris: Your benchmark doesn't tell the whole story.  There's even a hint that there's more to investigate in the max connection time reported by httperf.  Notice how it goes the opposite direction as the mean.  To really understand performance, you have to do more than look at a couple of aggregate statistics.  What does the distribution look like?", "group_id": 292, "id": 81654}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297448693.980469, "message": "ups, my bad... i wasn't aware that httperf does doing synchronous requests :P", "group_id": 292, "id": 82022}, {"user_id": 6310, "stars": [], "topic_id": 3716, "date_created": 1297449323.518038, "message": "'ab' clearly shows improving performance", "group_id": 292, "id": 82125}]