convore.json/groups/python/so-i-want-to-write-a-convore-bot-would-twisted-be-good-for-that/messages.json

[{"user_id": 872, "stars": [{"date_created": 1297280374.4981091, "user_id": 1057}], "topic_id": 2165, "date_created": 1297280347.75333, "message": "We use Diesel for http://afeedapart.com/ and we love it.  Simple, easy to use, and blazingly fast.", "group_id": 292, "id": 53434}, {"user_id": 8, "stars": [], "topic_id": 2165, "date_created": 1297280302.1444061, "message": "not webscale. dislike.", "group_id": 292, "id": 53412}, {"user_id": 1284, "stars": [], "topic_id": 2165, "date_created": 1297279936.798419, "message": "I don't think twisted works with the cloud", "group_id": 292, "id": 53214}, {"user_id": 872, "stars": [], "topic_id": 2165, "date_created": 1297280976.237057, "message": "@dreid The first two.  I know twisted is disgustingly fast, but the last time I tried to use it (a few years ago, to be fair) the documentation was basically a gigantic wall of unstyled text.", "group_id": 292, "id": 53777}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297280802.865927, "message": "@stevelosh Which of those 3 things are you implying that Twisted isn't?", "group_id": 292, "id": 53671}, {"user_id": 872, "stars": [], "topic_id": 2165, "date_created": 1297281207.563978, "message": "Example: this is the first thing you are supposed to read and understand if you want to \"use Twisted for web development\": http://twistedmatrix.com/trac/wiki/WebDevelopmentWithTwisted", "group_id": 292, "id": 53914}, {"user_id": 1284, "stars": [{"date_created": 1297286089.7792871, "user_id": 2313}, {"date_created": 1297314608.1792109, "user_id": 4156}], "topic_id": 2165, "date_created": 1297280861.6496551, "message": "8 minutes to the first convore argument: a strong statement for the quality of the service", "group_id": 292, "id": 53700}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297280890.477948, "message": "(Full disclosure: I am a long time twisted contributor and I spoke about twisted at Pycon 2010.  However it has been >2 years since I mostly traded Twisted for Erlang.  I'm very curious about what the public perception of Twisted is these days and am not overly sensitive about it.)", "group_id": 292, "id": 53724}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297281332.4963109, "message": "We do have a bad habit of not cleaning up after ourselves.  These days this is what should be first on the reading list: http://twistedmatrix.com/documents/current/web/howto/web-in-60/index.html  Open to suggestions on how to make that more visible.", "group_id": 292, "id": 53975}, {"user_id": 927, "stars": [], "topic_id": 2165, "date_created": 1297282427.220963, "message": "Which is a long way of saying that Twisted could really use a bit of cleanup and some white-hat SEO like Mozilla's promoteJS effort to get current, high-quality docs higher in search results.", "group_id": 292, "id": 54445}, {"user_id": 872, "stars": [], "topic_id": 2165, "date_created": 1297281891.6000619, "message": "@dreid Right now, a good portion of the front page (and the most important, front and center spot) is filled by text asking for money: http://i.imgur.com/gW0Ki.png You could reduce the amount of real estate spent on that and replace it with a 2-sentence, 2/3-link summary.", "group_id": 292, "id": 54216}, {"user_id": 927, "stars": [], "topic_id": 2165, "date_created": 1297282221.8787589, "message": "@dreid I usually need HTTP-based stuff. Last PyCon I played with switching from PyCurl to Twisted (https://github.com/acdha/webtoolbox/tree/twisted); my impression was simply that a doc refactor would be good - esp. with a quickstart - but mostly it just added a big framework and a 40% performance drop vs. libcurl (throughput; I made no attempt to measure memory).", "group_id": 292, "id": 54339}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297283310.5316241, "message": "@stevelosh For comparison: this is what I get to when I go to http://dieselweb.org http://cl.ly/2b1g151G2D2O061s1A1B", "group_id": 292, "id": 54903}, {"user_id": 872, "stars": [], "topic_id": 2165, "date_created": 1297283510.3698759, "message": "@dreid Ah, they must have taken down the site for some reason.  It used to have the content of https://github.com/jamwt/diesel/blob/master/docs/quickstart.rst  -- looks like Diesel has some work to do as well.", "group_id": 292, "id": 55003}, {"user_id": 2506, "stars": [], "topic_id": 2165, "date_created": 1297286390.330512, "message": "@dreid Twisted's doc need a serious sex appeal upgrade. There's a guy on the list that started something along those lines but he's gotten serious pushback from the powers that be...pretty typical for the Twisted community.", "group_id": 292, "id": 56816}, {"user_id": 2344, "stars": [], "topic_id": 2165, "date_created": 1297286608.376514, "message": "I think the Diesel docs are down while they work on getting them caught up to the code changes.", "group_id": 292, "id": 56928}, {"user_id": 2344, "stars": [], "topic_id": 2165, "date_created": 1297286625.080333, "message": "Generators/yield were replaced by greenlets a while back.", "group_id": 292, "id": 56934}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297287659.8142049, "message": "@jasonjwwilliams Really?  My impression from pycon last year was that everyone was very excited about the work he was doing.  It looks like it'll even be done soon?  http://buildbot.twistedmatrix.com/builds/sphinx-html/sphinx-html-15742/projects/web/howto/web-in-60/static-content.html", "group_id": 292, "id": 57551}, {"user_id": 507, "stars": [], "topic_id": 2165, "date_created": 1297291840.941849, "message": "@dreid I prefer the greenlet frameworks over Twisted.", "group_id": 292, "id": 59550}, {"user_id": 1736, "stars": [], "topic_id": 2165, "date_created": 1297291951.647866, "message": "Eventlet FTW. Twisted is a great example of why I prefer \"libraries\" to \"frameworks\".", "group_id": 292, "id": 59569}, {"user_id": 4580, "stars": [], "topic_id": 2165, "date_created": 1297312464.9707999, "message": "You might take a serious look at http://tornadoweb.org .. it's very powerful and very nice to work with", "group_id": 292, "id": 64747}, {"user_id": 4851, "stars": [], "topic_id": 2165, "date_created": 1297325377.962811, "message": "I'll second eventlet, I've been doing some pretty cool stuff with it using websockets and zeromq. It keeps everything nice and simple!", "group_id": 292, "id": 67027}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297336716.249176, "message": "Ok, so what is the apparent deficiency in Twisted that means I should be spending time investigating diesel/tornado/eventlet etc?", "group_id": 292, "id": 68510}, {"user_id": 5388, "stars": [], "topic_id": 2165, "date_created": 1297340355.969707, "message": "What kind of bot are we talking about?", "group_id": 292, "id": 68703}, {"user_id": 5296, "stars": [], "topic_id": 2165, "date_created": 1297354575.5720119, "message": "I prefer gevent over eventlet... here are some benchmarks http://nichol.as/asynchronous-servers-in-python", "group_id": 292, "id": 70953}, {"user_id": 2506, "stars": [], "topic_id": 2165, "date_created": 1297357621.551796, "message": "@dreid The \"deficiency\" in Twisted is the darn documentation. The documentation project you're talking about (converting to Sphinx) is still going on. But there hasn't been a usability improvement...the guy getting pushback on a new layout is this project: https://github.com/tdavis/twisted-docs", "group_id": 292, "id": 71519}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297356875.60674, "message": "@Zolomon Irrelevant.  It'll spend most of it's time writing to and reading from convore over their HTTP API.  It exist in several groups but only as one user.", "group_id": 292, "id": 71336}, {"user_id": 242, "stars": [], "topic_id": 2165, "date_created": 1297358612.2797649, "message": "The bit about making blocking calls is OK if they're short doesn't really make sense.  Perhaps you specifically mean blocking network calls?  All the code you write in Twisted is going to block the reactor while your code is running.  There are well defined points when control is returned to the reactor and also mechanisms for explicitly relinquishing control for a period of time.  (IReactorTime.callLater) and all filesystem calls block.", "group_id": 292, "id": 71668}, {"user_id": 5639, "stars": [], "topic_id": 2165, "date_created": 1297359397.2025919, "message": "I think he was talking in the context of calls that could potentially be made async if there were a driver. I don't think \"blocking calls are ok if they're short\" is so much tornado's philosophy, really. I think they just don't have all the drivers they might. They also get around it at deployment time using various techniques (usually involving nginx and/or two 'tiers' of tornado instances)", "group_id": 292, "id": 71776}, {"user_id": 2506, "stars": [{"date_created": 1297454404.475261, "user_id": 6310}], "topic_id": 2165, "date_created": 1297357488.785522, "message": "@dreid Tornado vs. Twisted comes down to two things in my opinion: are you invested in Twisted? Does everything need to be async? For me, I've put the time into Twisted so there's not a lot of need to learn Tornado. On the second point, there are more async drivers for things like Riak etc. for Twisted than Tornado...the Tornado approach seems to be that making blocking calls is OK if they're short. There's more and more async drivers for Tornado every day but for a small project I'm not sure it's worth the hassle if you don't know Twisted already.", "group_id": 292, "id": 71481}, {"user_id": 5639, "stars": [], "topic_id": 2165, "date_created": 1297358002.6328051, "message": "+1 @jasonwilliams (and I'm on the tornado side of the equation, mostly)", "group_id": 292, "id": 71565}, {"user_id": 5639, "stars": [], "topic_id": 2165, "date_created": 1297359433.684979, "message": "er, they get around the impact of the blocking calls that is.", "group_id": 292, "id": 71786}, {"user_id": 4219, "stars": [], "topic_id": 2165, "date_created": 1297371796.574501, "message": "do all of these run inside their own process i.e. ip + port ?", "group_id": 292, "id": 74606}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297375316.655571, "message": "Twisted just got too much burden fior keeping python2.3 or before as support target...", "group_id": 292, "id": 75152}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297376030.5227301, "message": "@number5 well yeah, I rather like that :)", "group_id": 292, "id": 75369}, {"user_id": 6113, "stars": [{"date_created": 1297454515.8906219, "user_id": 6310}], "topic_id": 2165, "date_created": 1297374435.568419, "message": "I see many comments about the Twisted documentation. As an user of twisted I can only agree. But Dave Peticolas did a great tutorial on Twisted : http://krondo.com/?page_id=1327\nTwisted is a great library, learning it is a pain in the ass but you will be rewarded.", "group_id": 292, "id": 75004}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297375348.323195, "message": "there are too much ugly hacks solely for that purpose", "group_id": 292, "id": 75158}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297375981.46738, "message": "@ericflo said something about them a while ago", "group_id": 292, "id": 75360}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297376113.704098, "message": "@alex and errback and callback will come back to same result variable, and multiple inlineCallbacks in chain might hide the exception and very, very hard to debug", "group_id": 292, "id": 75391}, {"user_id": 6113, "stars": [], "topic_id": 2165, "date_created": 1297375614.5785911, "message": "I use it with Python 2.5 (yes, it's old too) and I have no problem.", "group_id": 292, "id": 75236}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297375797.326375, "message": "twisted tips no. 1: Don't use Nevow...", "group_id": 292, "id": 75311}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297375857.015765, "message": "twisted tips no. 2: Don't abuse @inlinecallbacks (or try not use it at all)", "group_id": 292, "id": 75333}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297375968.3807449, "message": "What's wrong with @inlinecallbacks", "group_id": 292, "id": 75358}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297376147.3027191, "message": "@number5 I don't know enough about @inlinecallbacks is implemented but isn't htis \"fixable\" with .throw() on generators", "group_id": 292, "id": 75396}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297375985.8504961, "message": "but I never got the full story", "group_id": 292, "id": 75361}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297376038.609364, "message": "as long as you know what you're doing it's a much saner way to write code IMO", "group_id": 292, "id": 75372}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297376274.621182, "message": "@alex in simple situation, yes. but not when you have more than 3 inlineCallbacks in chain", "group_id": 292, "id": 75405}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297376410.2145121, "message": "especially when you need to dealing with a DB layer behind deferToThread and friends", "group_id": 292, "id": 75428}, {"user_id": 3751, "stars": [{"date_created": 1299203128.322526, "user_id": 14578}], "topic_id": 2165, "date_created": 1297376012.4623239, "message": "@alex it just make you think you are doing non-async programming", "group_id": 292, "id": 75368}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297376309.606427, "message": "@alex I know this is design problem not inlineCallbacks problem but ...", "group_id": 292, "id": 75408}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297380436.082633, "message": "it's fucking mindblowing how much more readable it gets, and it wasn't very much code to start with", "group_id": 292, "id": 75992}, {"user_id": 507, "stars": [], "topic_id": 2165, "date_created": 1297380478.6323111, "message": "Although in that article he claims that Python doesn't have powerful enough coroutines but now he uses eventlet like no other so maybe he's a bit crazy", "group_id": 292, "id": 75997}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297380423.6317079, "message": "@ericflo, can you paste the rewrite of our feedscraper from twisted to eventlet?", "group_id": 292, "id": 75989}, {"user_id": 507, "stars": [], "topic_id": 2165, "date_created": 1297380242.1456599, "message": "@dreid The reason that I prefer eventlet is because it hides most of the visibility that you're writing async code. The code ends up much easier to read and understand. The downfall is that it can play weirdly with some seeming innocuous library code.", "group_id": 292, "id": 75944}, {"user_id": 507, "stars": [{"date_created": 1297380724.111238, "user_id": 1736}, {"date_created": 1297381496.079756, "user_id": 3751}], "topic_id": 2165, "date_created": 1297380433.5370231, "message": "@dreid See this article where @ericflo nails http://www.eflorenzano.com/blog/post/how-do-we-kick-our-synchronous-addiction/", "group_id": 292, "id": 75990}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395287.2660379, "message": "there is a pretty epic amount of FUD in the history here", "group_id": 292, "id": 78254}, {"user_id": 7, "stars": [], "topic_id": 2165, "date_created": 1297395323.2539871, "message": "/me hopes he isn't involved in it", "group_id": 292, "id": 78264}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396185.372036, "message": "But for the sheer enormous volume of stuff that Twisted does, it's doing pretty well in the documentation department.", "group_id": 292, "id": 78446}, {"user_id": 5701, "stars": [], "topic_id": 2165, "date_created": 1297394634.907249, "message": "@number5: You are incorrect about the errback and callback \"coming back to same result variable\" with inlineCallbacks.  This is not how it works now, nor has it ever worked this way.", "group_id": 292, "id": 78115}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297396376.1997521, "message": "@glyph thanks!", "group_id": 292, "id": 78466}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396412.960443, "message": "Ahem.  Anyway, yes, you should use Twisted to write all your Convore bots.  And since it's an HTTP based API, you should use the new HTTP client, twisted.web.client.Agent.  http://twistedmatrix.com/documents/10.2.0/api/twisted.web.client.Agent.html", "group_id": 292, "id": 78469}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396863.732923, "message": "I'd rather not create yet another group on yet another site that I have to keep open in a browser all the time :).  You can find us on freenode IRC, in #twisted.", "group_id": 292, "id": 78525}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395542.6210859, "message": "@alex: not from what I can see, but scrolling this web page seems to keep about 3 of my cores totally busy at once, so I haven't gone back to re-read too carefully :)", "group_id": 292, "id": 78320}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395585.327431, "message": "although I might disagree that Twisted->eventlet is \"more readable\"", "group_id": 292, "id": 78333}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395879.1947911, "message": "But, Twisted has great documentation.", "group_id": 292, "id": 78396}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395883.837044, "message": "It has so much of it.  Thousands of pages.", "group_id": 292, "id": 78398}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395928.6684749, "message": "Plus we have several good tutorials for different applications, most notably the \"web in 60 seconds\" stuff which has been broadly praised.", "group_id": 292, "id": 78406}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395598.880451, "message": "most of the time when you do a rewrite like that, it's just more misleading and easier to ignore the edge cases", "group_id": 292, "id": 78335}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395266.4496119, "message": "hi", "group_id": 292, "id": 78248}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395903.6349649, "message": "We have a reference documentation requirement where every new API needs to come with damn near as much prose as code.", "group_id": 292, "id": 78399}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395946.2629039, "message": "The problem is we don't have very good index pages and we don't get very good search results when people spend 45 seconds googling for the one question they have.", "group_id": 292, "id": 78414}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396324.9508729, "message": "@number5: Calendar Server is _full_ of them :)", "group_id": 292, "id": 78460}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396364.1738541, "message": "(oh, what's that you say?  you can't think of another server that speaks all those protocols?  well, I wonder why it is that it's difficult to think of a straightforward way to document all of it at once.)", "group_id": 292, "id": 78463}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297395990.5628231, "message": "We don't have a clear story around what introductory documentation you should read and when you should read it and how much you need to know.", "group_id": 292, "id": 78422}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396149.6444681, "message": "There is also an issue with \"best practices\"; there's consensus among Twisted developers about which parts of the API you should be using and which parts are old and destined for deprecation or replacement, and communication isn't super clear about that stuff.", "group_id": 292, "id": 78440}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297396249.7055709, "message": "@jpcalderone it wasn't if there are only 1 or 2 level of inlineCallback chain, maybe I did something wrong in the middle", "group_id": 292, "id": 78453}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297396302.2700651, "message": "@jpcalderone @glyph is there any real life code examples of inlineCallbacks I can refer to?", "group_id": 292, "id": 78455}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396304.9135661, "message": "I'd be happy to go toe to toe with all the other XMPP/IRC/HTTP/DNS/IMAP/POP/SMTP/SSH/TLS/SIP/NNTP servers out there though, and see how our documentation compares to theirs", "group_id": 292, "id": 78456}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396838.344629, "message": "I was just floating past this documentation because someone referred me to it", "group_id": 292, "id": 78524}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396335.4572051, "message": "http://trac.calendarserver.org/", "group_id": 292, "id": 78462}, {"user_id": 3019, "stars": [], "topic_id": 2165, "date_created": 1297396986.581943, "message": "How about a convore bot with appengine?", "group_id": 292, "id": 78535}, {"user_id": 3751, "stars": [], "topic_id": 2165, "date_created": 1297396708.303736, "message": "@jpcalderone @glyph since you guys are all here, maybe you can create a Twisted group...", "group_id": 292, "id": 78502}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396877.5371561, "message": "bit of a freudian slip there", "group_id": 292, "id": 78528}, {"user_id": 6671, "stars": [], "topic_id": 2165, "date_created": 1297396872.3687639, "message": "ahem not this documentation, this conversation", "group_id": 292, "id": 78527}, {"user_id": 2226, "stars": [], "topic_id": 2165, "date_created": 1297584523.3513939, "message": "trt", "group_id": 292, "id": 88857}, {"user_id": 1, "stars": [], "topic_id": 2165, "date_created": 1297584701.917192, "message": "@test123 Looks like it worked!", "group_id": 292, "id": 88862}, {"user_id": 3019, "stars": [], "topic_id": 2165, "date_created": 1297669024.4446449, "message": "Check out @evilbot", "group_id": 292, "id": 96860}, {"user_id": 18571, "stars": [], "topic_id": 2165, "date_created": 1299203452.262367, "message": "Also, I'm in your Convore, bumping your ancient topics. :3", "group_id": 292, "id": 265612}, {"user_id": 18571, "stars": [], "topic_id": 2165, "date_created": 1299203382.988189, "message": "Man, I was going to question why you would want a bot for this silly site, but I completely concur with @glyph about the CPU usage. Dear God, was it really necessary just for the stupid scrollbar trick? >:3", "group_id": 292, "id": 265596}]