convore.json/groups/python-scientific-computing/using-eigenimages-to-cluster-dna-sequences/messages.json

[{"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300609512.6411049, "message": "This may be crazy but bear with me for a minute. At Pycon I presented a poster on comparing and visualizing genome DNA regions, and a few people commented that I should try using Fourier transforms. This seemed somewhat outlandish to my ignorant biologist self (all I knew of FTs was what I glimpsed when I ran SETI@home on my Mac many, many years ago) but I dutifully went and bought a bioinformatics book that explains the basics of FTs, how to apply them to DNA sequences, and how that is done in Python (thank you SciPy).", "group_id": 6727, "id": 391372}, {"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300611643.5566199, "message": "So in brief the idea would be to produce a set of eigenimages for each plasmid (derived from chopped up sequence segments of standardized length), then run those eigenimages sets through a comparison and clustering routine, and shazam! give or take some optimization and parameter adjustment, outs pops a comprehensive plasmid clustering scheme that defines groups of related plasmids that I could then process through more conventional means for fine-tuned analysis. Can anyone tell me if that sounds even remotely feasible, or do I need to lay off the caffeine for a while and back the heck away from this crazy idea?", "group_id": 6727, "id": 391439}, {"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300610987.3224959, "message": "Although that was a challenging read, I muddled through and I do think I will be using some form of FTs in my current work, as it happens, but that's not the point here. The point is that the book then got kind of wild (for me) with a chapter that introduced chaos theory, eigenvectors and further on, eigenimages, with the tantalizing proposition (illustrated in a rather convincing example) that eigenimages generated from genome DNA can be used to identify clusters of related species. My grasp of these concepts is tenuous at best but I am greatly intrigued by this possibility, and more importantly, I have a very specific application in mind. If I understand the idea correctly (and that's a very big if) this could solve a huge problem I've been trying (and failing) to get to grips with, which is how to efficiently sort ALL POSSIBLE PLASMIDS (a kind of DNA element that exists independently of the chromosome in bacteria-- they're very important for microbial evolution) EVER into phylogenetically valid clusters without any initial assumptions or curation and without spending millions upon millions of CPU-years running and parsing all-against-all BLAST (DNA pattern matching algorithm) iterations.", "group_id": 6727, "id": 391397}, {"user_id": 10411, "stars": [], "topic_id": 14046, "date_created": 1300644063.053489, "message": "@gglobster I don't think this is a crazy idea!  One thing to keep in mind though is that once you take the transforms of two sequences, you'll need a metric to compare how similar they are.\n\nPeople in spectroscopy have done some work on this kind of thing.", "group_id": 6727, "id": 392991}, {"user_id": 10411, "stars": [], "topic_id": 14046, "date_created": 1300652002.301013, "message": "@gglobster Also, if you need any help with FTs or implementations just ask here.  I think that convore is working pretty well as a resource for this kind of thing.", "group_id": 6727, "id": 393586}, {"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300655474.337343, "message": "Excellent, it's nice to know I'm not completely off base. Thanks for taking the time to read that long rant of a question... I will definitely be back with more practical implementation questions. First I need to spend a bit more quality time with my book to try out their examples and grasp the concepts better.", "group_id": 6727, "id": 393859}, {"user_id": 10411, "stars": [], "topic_id": 14046, "date_created": 1300658358.8270221, "message": "@gglobster Always a sound plan ;)", "group_id": 6727, "id": 393973}, {"user_id": 21157, "stars": [], "topic_id": 14046, "date_created": 1300748823.474834, "message": "Are plasmids commonly represented by images? From your earlier description, it sounded like they were just sequences, in which case you probably just want eigenvectors -eigenimages are just the eigenvectors of image data. In general, people tend to stick \"eigen\" in front of whatever their datapoints represent to describe the eigenvectors of their data (or more generally to describe objects whose direction doesn't change under certain transformations), e.g. \"eigenfaces\" in face recognition or \"eigenfunctions\" in functional analysis.", "group_id": 6727, "id": 403440}, {"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300829147.408534, "message": "@dplepage No they are indeed DNA sequences, and they're not usually represented by images apart from the occasional linear genetic feature map. I have to admit I'm new to the concept of eigen-anythings, really, and your explanation there actually makes more sense to me than some things I've been reading recently :)  What I was reading was the possibility of producing an image that represents properties of the sequence; this has been used a little bit to represent genomes (genomic signatures).", "group_id": 6727, "id": 411983}, {"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300829320.6644139, "message": "@dplepage And thinking about it, I think the image generation part is mostly for the benefit of us biologists who get extremely skittish at the mention of eigenvectors; show us images though, and we can see how similar or different they look.", "group_id": 6727, "id": 412006}, {"user_id": 10411, "stars": [], "topic_id": 14046, "date_created": 1300847278.2183111, "message": "@gglobster  @dplepage It should be noted that you can have a 1D eigenimage.  (Or 2D, 3D...ND)", "group_id": 6727, "id": 413563}, {"user_id": 21157, "stars": [], "topic_id": 14046, "date_created": 1300885792.907819, "message": "@scopatz The eigenvectors of a matrix are inherently one-dimensional; you can treat them as unraveled higher-dimensional tensors, certainly, but I've only ever heard them called \"eigenimages\" when each was treated as a raster grid of pixels (the pixels can have arbitrarily high dimension, of course, but the array itself will have either two (grayscale) or three; this is a confusing overloading of the word \"dimension\").", "group_id": 6727, "id": 416954}, {"user_id": 10411, "stars": [], "topic_id": 14046, "date_created": 1300909757.4253819, "message": "@dplepage Yes, dimension is often overloaded.  As is eigen-*.    My point was that you can define math such that the eigen concept can be applied to the dimensionality idea of choice.  Put that generalization in your pipe and smoke it :).", "group_id": 6727, "id": 420139}, {"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300922381.7877319, "message": "@scopatz @dplepage I'll tell you what's being overloaded, that would be my poor biologist brain. I'll have a puff of that pipe now, please ;)", "group_id": 6727, "id": 421724}, {"user_id": 10411, "stars": [], "topic_id": 14046, "date_created": 1300928841.485549, "message": "@gglobster  @scopatz I am sorry officer... It's not mine. I was just holding this eigenimage for a friend!", "group_id": 6727, "id": 422291}, {"user_id": 21310, "stars": [], "topic_id": 14046, "date_created": 1300977659.4613969, "message": "@scopatz LOL", "group_id": 6727, "id": 427255}]