convore.json/groups/django-community/django-orm-caching/messages.json

[{"user_id": 18845, "stars": [], "topic_id": 21194, "date_created": 1303710924.1139641, "message": "I have a site that is heavy of ORM calls.   I am thinking about using http://packages.python.org/johnny-cache/ to cache ORM calls.", "group_id": 81, "id": 800222}, {"user_id": 18845, "stars": [], "topic_id": 21194, "date_created": 1303710942.3046529, "message": "I was just wondering what folks thought the best solution is.", "group_id": 81, "id": 800229}, {"user_id": 1127, "stars": [{"date_created": 1303736450.451884, "user_id": 13912}], "topic_id": 21194, "date_created": 1303712766.233289, "message": "Rethink what you're querying, tune your database, and then look at caching again", "group_id": 81, "id": 800495}, {"user_id": 1127, "stars": [], "topic_id": 21194, "date_created": 1303712776.9418969, "message": "(and ideally look at doing it yourself so you understand how to deal w/ invalidation)", "group_id": 81, "id": 800496}, {"user_id": 6894, "stars": [], "topic_id": 21194, "date_created": 1303724967.630383, "message": "+1 for rethinking. ORM caching is usually unnecessary (and very confusing)", "group_id": 81, "id": 802106}, {"user_id": 6894, "stars": [], "topic_id": 21194, "date_created": 1303724994.5676939, "message": "you are usually better off doing some query profiling and fixing the performance issues", "group_id": 81, "id": 802107}, {"user_id": 6894, "stars": [], "topic_id": 21194, "date_created": 1303725042.103158, "message": "often a few select_related('foo') calls can go a long way to improving performance", "group_id": 81, "id": 802111}, {"user_id": 8736, "stars": [], "topic_id": 21194, "date_created": 1303731908.473887, "message": "Disclosure: I work with one of the authors.", "group_id": 81, "id": 802671}, {"user_id": 8736, "stars": [], "topic_id": 21194, "date_created": 1303731897.2328391, "message": "If you decide you need caching, I can vouch for Johnny.  We use it in production on flavors.me, and shortly will be on goodsie.com, and we've found it to be easy to setup and understand.", "group_id": 81, "id": 802668}, {"user_id": 18845, "stars": [], "topic_id": 21194, "date_created": 1303799076.700644, "message": "Thanks for the advice folks.  I am going to try http://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related see if that boosts my speed first", "group_id": 81, "id": 810891}, {"user_id": 14923, "stars": [], "topic_id": 21194, "date_created": 1303845485.700666, "message": "I'd look at not using johnny-cache before trying it, and I wrote it.  It's a general purpose library that works very well in some circumstances and poorly in others.  If your reads greatly outweigh your writes, it can help a lot for almost no development cost;  and that's the sweet spot.  It has very zealous invalidation in order to achieve transparency, so while its operation might be confusing, using it doesn't add any conceptual baggage.", "group_id": 81, "id": 817894}, {"user_id": 13550, "stars": [], "topic_id": 21194, "date_created": 1303893135.8723669, "message": "Johnny-cache can give you a BIG reduction in your db load however it is not a magical solution either. My general advise would be :", "group_id": 81, "id": 823463}, {"user_id": 13550, "stars": [{"date_created": 1303919806.072619, "user_id": 2588}, {"date_created": 1304035818.8455901, "user_id": 927}], "topic_id": 21194, "date_created": 1303893366.6809731, "message": "1. profile your pages and optimise as much as you can \n2. Learn about the invalidation schema used by Johnny-cache\n3. test extensively all your use cases on a staging environment", "group_id": 81, "id": 823491}, {"user_id": 3580, "stars": [], "topic_id": 21194, "date_created": 1304005377.197746, "message": "@gourneau I'm not sure I agree that ORM caching is a Bad Thing, but I do agree that the solutions available for Django's ORM are (mostly). Especially if you're querying objects in many different ways and if you need intelligent invalidation.", "group_id": 81, "id": 842562}, {"user_id": 26100, "stars": [], "topic_id": 21194, "date_created": 1304012046.0247891, "message": "@jmoiron I think keeping to a simple contract like this is the best strategy in this situation. I haven't had a call for this level of caching, but if I ever do, I would want something this straight forward.", "group_id": 81, "id": 844139}, {"user_id": 14923, "stars": [], "topic_id": 21194, "date_created": 1304011188.0858309, "message": "@phill I'm sorry you feel that way..  Johnny-cache's goal is delivering on a simple promise:  any read you make from a set of tables is cached until the next write on any of those tables.  Automatically.  This actually requires some degree of sophistication to keep, but it's transaction aware, supports save points, multi-db, invalidates joins properly, barely takes any setup or configuration, and adds almost zero cognitive load to your application's code.  It's not always a good or useful caching strategy, but that's just how these things go.", "group_id": 81, "id": 843929}, {"user_id": 3580, "stars": [], "topic_id": 21194, "date_created": 1304014653.8723741, "message": "It's also worth mentioning that I wrote my first comment in this thread days ago.. before @jmoiron joined us and mentioned johnny.. I hit 'send', realized I wasn't logged in and didn't bother. Then I logged in today and Convore auto-posted it for me. I'm actually a bit peeved by that 'feature'", "group_id": 81, "id": 844805}, {"user_id": 3580, "stars": [], "topic_id": 21194, "date_created": 1304014313.762706, "message": "@jmoiron no offense intended, I'll take another look at Johnny. I think I'm just kind of married to the cache strategy that Hibernate uses in the Java camp.. entities are cached, not queries.. so the cache isn't bloated if you happen to query something three different ways, nor are you going to get extra misses.", "group_id": 81, "id": 844743}, {"user_id": 14923, "stars": [], "topic_id": 21194, "date_created": 1304015321.5704091, "message": "@phill heh no problem..   I joined because the original post mentioned it and I thought I'd have something to contribute.  Hibernate-style caching might be better if your entities are huge, or if you have lots of slightly different queries.  Entity caching is hard to get transparent; you have to duplicate much of what the db is doing, as it's hard to know whether or not an UPDATE will affect a particular row.  In general, you get way more mileage by caching big computationally expensive application-level things, like doing \"widget\" memoization, and you can probably design frameworks around making that type of thing easier, but that's just not what johnny is.", "group_id": 81, "id": 844934}, {"user_id": 14923, "stars": [], "topic_id": 21194, "date_created": 1304015682.0023701, "message": "it was designed in a very particular environment to solve a few issues:  one, we wanted to disable mysql queryset caching because we wanted to use that precious memory for indexes but had cache in our MC pool to waste, two, we had a really poorly written app with tons of reads that were difficult to get rid of;  think complex data grid processing stuff that treated each cell independently, three, our app was really huge code-wise, so we couldn't really fix it in the time we had, and fix-the-scaling-issue whack-a-mole just wasn't preventing outages.", "group_id": 81, "id": 845009}, {"user_id": 1127, "stars": [], "topic_id": 21194, "date_created": 1304023344.2232051, "message": "I would wager money that every single person using Johnny Cache doesn't need to (not saying it can't be useful, just saying there's usually better ways to solve things rather than a blanket cache-all solution)", "group_id": 81, "id": 847021}]