convore.json/groups/inscight/episode-7-perl-data-language/messages.json

[{"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1301256626.216212, "message": "History?  How did it start?", "group_id": 3435, "id": 452582}, {"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1301256647.6908679, "message": "Why are graphics included in PDL rather than a separate lib?", "group_id": 3435, "id": 452583}, {"user_id": 25953, "stars": [{"date_created": 1301282891.121078, "user_id": 25953}], "topic_id": 15653, "date_created": 1301282852.81407, "message": "@scopatz They are not included in PDL!  PDL includes interfaces to external libraries.  The dominant external library is PGPLOT, but PLPlot is also used by many users.  There has been discussion of a Gnuplot interface, since Gnuplot appears to be more stable (in a long-term development sense) than either PGPLOT or PLPlot.", "group_id": 3435, "id": 454528}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301283948.877789, "message": "Some other topics to discuss could be: PP (IMHO this combo is  the hidden jewel in PDL - a whole metalanguage, like XS for Perl, designed for implementing vectorized C code as Perl/PDL subroutines.  Coupled with Inline it makes writing new/custom high speed code a snap); the importance of \"CISC\" vectorization and the threading engine; drawbacks of vectorizing interpreted languages", "group_id": 3435, "id": 454650}, {"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1301284725.0114801, "message": "@zowie Excuse me, my ignorance is showing ;)    PP sounds awesome though.  Anything that kicks vectorization down another level gets a +1 in my book.", "group_id": 3435, "id": 454726}, {"user_id": 25999, "stars": [], "topic_id": 15653, "date_created": 1301300492.489958, "message": "The 'threading' capability of PDL is what really makes me use it day after day. Once you get the idea behind this, you can write concise code for algorithms.", "group_id": 3435, "id": 456192}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301315909.3257451, "message": "Implementing threaded code was once difficult, but the threading engine that exists now (which was written by Tuomas Lukka and expanded by several of us over the years) implements a metalanguage (\"PP\") that accepts a concise description of your central algorithm and writes all the necessary loops around it.", "group_id": 3435, "id": 457458}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301316390.201812, "message": "(The result for IDL is that you end up writing a lot of interpreted loops to do things that are very concise in idiomatic PDL).", "group_id": 3435, "id": 457489}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301316361.7932911, "message": "Contrast that to, say, IDL, which also does vectorization but not in a simple way: the vectorization engine there really only operates along a single dimension at a time, so that if you implement a 1-D operation such as a collapse you have nothing left - collapse operators don't generalize to operate on higher dimensional constructs in IDL.", "group_id": 3435, "id": 457481}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301316077.4488659, "message": "I do a lot of image processing and computer vision application stuff, so I end up working with 2-D operations a lot -- operations that do something to little 2-D pieces of an image.  For that kind of work, we have the range() operator, which can extract an arbitrary collection of patches out of a large data structure, and stack them up along a new dimension for subsequent vectorization (\"threading\").  That turns out to be extraordinarily useful (to me, anyway).", "group_id": 3435, "id": 457461}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301316249.476054, "message": "What got me going on PDL in the first place is that it is a rather simple extension to Perl -- although, like any scientific language, it has a lot of structure and tricks, there are really only a very few concepts to pick up.  From a user standpoint, the threading engine is very simple -- corresponding dimensions in any vector operation have to match, to have size 1 (in which case the size 1 dimension is repeated to generate a match), or to not exist (in which case they are treated as size 1 dimensions).", "group_id": 3435, "id": 457470}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301315811.2092471, "message": "@scopatz, no worries.  As @matthewkenworthy points out, the bread and butter of PDL is the threading engine, and it is the first such engine I've encountered that works \"right\" -- you really can take an arbitrary  0, 1, 2, or n-dimensional operation and vectorize it by feeding it a higher-dimensional object (which is then treated as a stack of your n-dimensional objects).  Code that (for example) finds the centroid of a 1-D spectrum will generally work fine on a 2-D array, treating it as a collection of spectra to centroid.", "group_id": 3435, "id": 457453}, {"user_id": 10421, "stars": [], "topic_id": 15653, "date_created": 1301701368.634665, "message": "@zowie I'm glad you brought up IDL. It's an obvious comparison to need to make, since the first question for many physicists, etc. will be \"What makes it better than IDL\" (which, I'm sure, many interested scientists will already know). I think we should be sure to flesh out this comparison on the podcast since the audience will involve a lot of people to whom IDL is better known than PDL.", "group_id": 3435, "id": 506805}, {"user_id": 10421, "stars": [], "topic_id": 15653, "date_created": 1301866156.733815, "message": "Somce thoughts I just expressed via email follow...", "group_id": 3435, "id": 517970}, {"user_id": 10421, "stars": [], "topic_id": 15653, "date_created": 1301866299.7491851, "message": "I hope you'll each take a crack at explaining piddles conceptually (and how they're not really perl arrays, but they're kind of like perl arrays, but they're not).", "group_id": 3435, "id": 517983}, {"user_id": 10421, "stars": [], "topic_id": 15653, "date_created": 1301866322.0607519, "message": "Finally, I personally think you PDL users have some fun lingo, so I'd love to hear more about pumpkin holders, etc.", "group_id": 3435, "id": 517985}, {"user_id": 10421, "stars": [], "topic_id": 15653, "date_created": 1301866257.8306501, "message": "I know perl, and a decade ago learned enough to believe I could do everything with it (after all, There Is More Than One Way To Do It). That said, I can't name another young person (I'm 25) who knows perl. Maybe it's just me, but it seems as if python is kicking some butt with the kiddos. To use PDL, you need perl. In your experience, are the young people learning perl? What happens to PDL when python wins/steals the hearts and minds of the youth?", "group_id": 3435, "id": 517980}, {"user_id": 10421, "stars": [], "topic_id": 15653, "date_created": 1301866239.4485569, "message": "I hope you'll talk about the ups/downs/ins/outs of managing such a huge open source project primarily via an email list. I would be interested in the basics (how many people have commit privileges today) as well as the details (where are the kinks in the armor? certainly bug tracking has improved the system a bit?).", "group_id": 3435, "id": 517978}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301896314.490525, "message": "@Katy: Cool, we can talk about some of that stuff.  Python has a reputation as being a lot easier to learn than Perl.  I think that the truth is closer to \"Python is easier to teach than Perl\".  With strict B&D style indenting and TIOWTDI philosophy, it's easier to grade student projects and to inculcate coding discipline in Python.   That is reflected in many intro CS curricula.", "group_id": 3435, "id": 520895}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1301896398.7839479, "message": "OTOH, despite about a decade of people calling that \"Perl is dead\", CPAN seems to be alive, kicking, and (dare I say?) vibrant.  Perl 6 may be Larry's HURD, but Perl 5 still seems to drive much of the world.", "group_id": 3435, "id": 520906}, {"user_id": 27591, "stars": [], "topic_id": 15653, "date_created": 1302005122.6522591, "message": "Is Python like what Top Gear calls your 'family car'. PDL would be a Lambo - dunno IDL? I'd be interested in discussing what makes people choose what they do. Number 1 reason is peer pressure. Number 2 reason is the opposite - rebelling against your peers.", "group_id": 3435, "id": 540041}, {"user_id": 27591, "stars": [], "topic_id": 15653, "date_created": 1302005473.6917491, "message": "How did it start? Would you believe trying to impress a girl? Or how about an insatiable desire to hack and getting bored when not doing it? Or that IDL was ugly and too expensive? The truth is complex.", "group_id": 3435, "id": 540063}, {"user_id": 27591, "stars": [], "topic_id": 15653, "date_created": 1302005200.623775, "message": "Where I work all my students use IDL. :-( Everybody else's use IDL, or SciPy or IRAF", "group_id": 3435, "id": 540048}, {"user_id": 25999, "stars": [], "topic_id": 15653, "date_created": 1302040739.638536, "message": "I remember Karl showing me PDL about 12 years ago, and I used it for a while and ten dropped it. I picked it up again and (a bit embarrassingly) simply didn't grok threading until about four years ago.", "group_id": 3435, "id": 547491}, {"user_id": 25999, "stars": [{"date_created": 1302040866.486264, "user_id": 10411}], "topic_id": 15653, "date_created": 1302040832.0930519, "message": "I keep meaning to try python, but PDL is simply my day to day multitool, and I'm too much of an old fart to learn whatever these young whippersnappers use. Get off my lawn!", "group_id": 3435, "id": 547518}, {"user_id": 25999, "stars": [], "topic_id": 15653, "date_created": 1302041138.806318, "message": "My big fear with the newer science languages is that the libraries do not handle the edge cases with any grace - PDL isn't necessarily better, but at least I know where Karl, Craig et al. buried the bodies. Or at least I can see the uneven lumps....", "group_id": 3435, "id": 547858}, {"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1302042926.2690179, "message": "Also, if everyone is online now", "group_id": 3435, "id": 549179}, {"user_id": 25999, "stars": [], "topic_id": 15653, "date_created": 1302042817.2837701, "message": "Zowie - I was referring to the additional modules, not the PDL::Core routines. They've been bulletproof all the time I've used them.", "group_id": 3435, "id": 549150}, {"user_id": 25999, "stars": [], "topic_id": 15653, "date_created": 1302043277.0580809, "message": "I assume anthony will send connect requests...?", "group_id": 3435, "id": 549259}, {"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1302043262.09811, "message": "@zowie I'll call into you", "group_id": 3435, "id": 549257}, {"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1302042875.781476, "message": "Speaking of additional modules, how popular is PDL::IO::HDF5?", "group_id": 3435, "id": 549167}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1302042186.5497179, "message": "@Matt, I disagree -- there are indeed uneven lumps in PDL at the edge cases, but one thing that made me fall in love with it is that Karl, Tuomas, Christian, and (to the extent I've walked in their footsteps) I have tried to think carefully about edge cases when designing new blocks of functionality.  Respectfully, I think it shows -- at least in the core infrastructure like the threading rules, the infrastructure for collapse, etc.", "group_id": 3435, "id": 549024}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1302043212.964927, "message": "Er, I'm online but forget where we're supposed to call with Skype.", "group_id": 3435, "id": 549241}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1302043280.0285079, "message": "OK.  Skype is starting up now -- give it 10 seconds.", "group_id": 3435, "id": 549260}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1302042948.407593, "message": "I'm online but still organizing my thoughts.", "group_id": 3435, "id": 549187}, {"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1302042938.19836, "message": "we could probably get started early...", "group_id": 3435, "id": 549182}, {"user_id": 10411, "stars": [], "topic_id": 15653, "date_created": 1302042984.0202711, "message": "OK, I am as well", "group_id": 3435, "id": 549191}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1302043123.647404, "message": "@Matt, Ah, well in that case we're in good agreement.  I'll expound on that and perhaps air a bit of dirty laundry:  IMAO PDL's only major failing seems to be a lack of large-scale organization in the project -- that held us back for years (perhaps through the \"window of opportunity\" before SciPy was developed) because it was such a pain in the butt to install.  Now it is easier but the same issue means that it's hard to keep track of all the peripheral modules.  MatLab, for example, has the same extensibility-through-loading-modules (except that you pay for it) , but at least the pay-per-module aspect keeps it halfway organized.  To the extent that PDL is still cohesive, we all owe a big debt to Chris Marshall for cracking the whip and generally tracking organizational/build issues.", "group_id": 3435, "id": 549222}, {"user_id": 25999, "stars": [], "topic_id": 15653, "date_created": 1302043221.7331409, "message": "Absolutely. I'm guilty of not helping with that, as I'm not familiar with the guts of PDL and the detailed build process. I know just enough to blow a toe off :)", "group_id": 3435, "id": 549242}, {"user_id": 25953, "stars": [], "topic_id": 15653, "date_created": 1302043246.4827061, "message": "Well, if Perl is the Swiss army chainsaw, then PDL is the nuclear Swiss army chainsaw.", "group_id": 3435, "id": 549250}, {"user_id": 25999, "stars": [], "topic_id": 15653, "date_created": 1302043287.478642, "message": "Right, finishing making a cup of tea.", "group_id": 3435, "id": 549262}, {"user_id": 27591, "stars": [], "topic_id": 15653, "date_created": 1302047765.0585101, "message": "# Points in Polygon problem, note one line heart!\n\n# Test points testx,texty to be in a polygon following \n# http://www.ecse.rpi.edu/Homepages/wrf/Research/Short_Notes/pnpoly.html\n# Nice fast VECTOR code!\nsub pnpoly{\n   my($tx, $ty, $vertx, $verty) = @_;\n   my $testx = $tx->dummy(0);\n   my $testy = $ty->dummy(0);\n   my $vertxj = $vertx->rotate(1);\n   my $vertyj = $verty->rotate(1);\n   $c =    sumover(    (($verty>$testy) != ($vertyj>$testy)) &\n\t         ($testx < ($vertxj-$vertx) * ($testy-$verty) / \n\t         ($vertyj-$verty) + $vertx)\n\t         ) %2;\n  return $c;\n}\n\n# Note here is an IDL version (from Ivan)\n# \n# http://www.dfanning.com/tip_examples/inside_vector.pro\n\n\n# Test points testx,texty to be in a polygon following \n# http://www.ecse.rpi.edu/Homepages/wrf/Research/Short_Notes/pnpoly.html\n# This is the unvectorized code (keep for reference/testing)\n# sub pnpoly_novec{\n#    my($testx, $testy, $vertx, $verty) = @_;\n#    my($i,$j,$k);\n#    my $c = zeroes($testx);\n#    my $nvert = nelem($vertx);\n#    my @c = list $c; my @testx = list $testx; my @testy = list $testy; my @vertx = list $vertx; my @verty = list $verty;\n#    for ($k=0; $k<nelem($testx); $k++) {\n#       for ($i = 0, $j = $nvert-1; $i < $nvert; $j = $i++) {\n#          if ( (($verty[$i]>$testy[$k]) != ($verty[$j]>$testy[$k])) &&\n# \t         ($testx[$k] < ($vertx[$j]-$vertx[$i]) * ($testy[$k]-$verty[$i]) / \n# \t         ($verty[$j]-$verty[$i]) + $vertx[$i]) ) {\n#                 $c[$k] = ! $c[$k];\n#              }\n#       }# ij\n#   }#k\n#   return pdl(@c);\n# }\n# \n\n1;\n", "group_id": 3435, "id": 550569}]