convore.json/groups/django-community/logging-favorite-approaches/messages.json

[{"user_id": 1736, "stars": [], "topic_id": 11576, "date_created": 1299623974.0880549, "message": "Once Sentry2 stabilizes a bit more we are hoping to adapt our object logs to use that and then make it public.", "group_id": 81, "id": 300211}, {"user_id": 1736, "stars": [{"date_created": 1299626625.229183, "user_id": 5778}, {"date_created": 1299717727.5088699, "user_id": 177}], "topic_id": 11576, "date_created": 1299623941.37238, "message": "Celery + SQL mostly. We decided long ago that it is better to eat the cost of SQL-backed logging than to deal with parsing text files back into SQL to do parametric queries on them.", "group_id": 81, "id": 300208}, {"user_id": 3580, "stars": [], "topic_id": 11576, "date_created": 1299624170.2179611, "message": "@coderanger - makes perfect sense re: SQL.. I was thinking about doing something like logging json and dumping in MongoDB though for flexibility (no migrations when changing the logged structure over time, etc)", "group_id": 81, "id": 300229}, {"user_id": 3580, "stars": [], "topic_id": 11576, "date_created": 1299624221.9406841, "message": "@ericflo that's where I was headed.. where do you put it though?", "group_id": 81, "id": 300236}, {"user_id": 1, "stars": [], "topic_id": 11576, "date_created": 1299624236.622283, "message": "syslog", "group_id": 81, "id": 300239}, {"user_id": 3580, "stars": [], "topic_id": 11576, "date_created": 1299623165.9461679, "message": "tab delimited? json? python logging? Scribe? Splunk? MongoDB? I'd really love to hear about logging strategies for django apps that folks have loved or run into trouble on. Trying to sort out what we're gonna do for our stuff.", "group_id": 81, "id": 300167}, {"user_id": 1736, "stars": [], "topic_id": 11576, "date_created": 1299624244.7242751, "message": "(in JSON)", "group_id": 81, "id": 300242}, {"user_id": 3580, "stars": [], "topic_id": 11576, "date_created": 1299623222.179565, "message": "(barely) more detail: We're wanting to log as much as humanly possible, and running django via modwsgi.", "group_id": 81, "id": 300168}, {"user_id": 1736, "stars": [], "topic_id": 11576, "date_created": 1299624242.53337, "message": "@ericflo Yeah, we store some generic stuff (object ref, log type, etc) as columns and then the rest as a TextField", "group_id": 81, "id": 300241}, {"user_id": 1, "stars": [], "topic_id": 11576, "date_created": 1299624207.195962, "message": "And much more extensible, if you just do csv or tsv, adding a field later means your code needs to have all kinds of conditionals", "group_id": 81, "id": 300234}, {"user_id": 1, "stars": [{"date_created": 1299626645.948113, "user_id": 5778}], "topic_id": 11576, "date_created": 1299624174.573324, "message": "I'm a fan of logging things in JSON format, so much easier to parse later", "group_id": 81, "id": 300230}, {"user_id": 19707, "stars": [], "topic_id": 11576, "date_created": 1299626457.095439, "message": "hello", "group_id": 81, "id": 300433}, {"user_id": 5778, "stars": [], "topic_id": 11576, "date_created": 1299626669.4566331, "message": "@ehabinism greetings earthling, I'm new here too", "group_id": 81, "id": 300447}, {"user_id": 1, "stars": [], "topic_id": 11576, "date_created": 1299626566.289264, "message": "@ehabinism hi", "group_id": 81, "id": 300440}, {"user_id": 927, "stars": [], "topic_id": 11576, "date_created": 1299687057.0471699, "message": "@coderanger Are you suing a particular JSONField implementation? I've been really happy with the django-extensions one for a project which had a ton of under-scoped metadata outside of the core schema", "group_id": 81, "id": 306506}, {"user_id": 3580, "stars": [], "topic_id": 11576, "date_created": 1299706599.0617909, "message": "so.. json -> syslog was what @ericflo said convore was doing... But are there tools that you use to make sense of the structured logs? Or do you just write scripts to pull out interesting data, etc.. Like.. let's say I wanted to look at the last error, in which maybe I've got a traceback, or request payload. What to do?", "group_id": 81, "id": 308621}, {"user_id": 281, "stars": [], "topic_id": 11576, "date_created": 1299705447.188894, "message": "+1 on syslog and shipping logs to a central server", "group_id": 81, "id": 308443}, {"user_id": 281, "stars": [{"date_created": 1299713283.2423179, "user_id": 5456}], "topic_id": 11576, "date_created": 1299705485.9523389, "message": "Frank Wiles has a pretty good intro post on setting up a central logging sever with syslog at http://www.revsys.com/blog/2010/aug/26/centralized-logging-fun-and-profit/", "group_id": 81, "id": 308448}, {"user_id": 7378, "stars": [], "topic_id": 11576, "date_created": 1299705861.1046579, "message": "Agreed. If you have > 1 server set up a logging server. Makes debugging and fixing shit a lot faster.", "group_id": 81, "id": 308504}, {"user_id": 7378, "stars": [], "topic_id": 11576, "date_created": 1299705324.6812251, "message": "Loggly is a big pile of fun. Use logstash if you want something free, or possibly graylog2. Splunk is usually overkill.", "group_id": 81, "id": 308413}, {"user_id": 7378, "stars": [{"date_created": 1299705459.916008, "user_id": 1}], "topic_id": 11576, "date_created": 1299705425.9769919, "message": "Syslog is your friend.", "group_id": 81, "id": 308440}, {"user_id": 3580, "stars": [], "topic_id": 11576, "date_created": 1299714655.3988421, "message": "Yeah, I like the idea of shipping off to something like Mongo early since the data will _always_ be ready to query on", "group_id": 81, "id": 309390}, {"user_id": 5456, "stars": [], "topic_id": 11576, "date_created": 1299713506.9300799, "message": "For our API we are using MongoDB. Basically we have a middleware that saves all data requests, user, response times, etc.", "group_id": 81, "id": 309313}]