« first day (176 days earlier)      last day (2999 days later) » 

1:27 AM
@Andrie - I personally poured out a bottle of my best whiskey today in mourning
 
 
4 hours later…
5:40 AM
Good morningy room. It's 7:40 and let's see if there are any open questions.
 
6:14 AM
It's still only 7:13 here. No answerable questions...
 
 
2 hours later…
7:56 AM
I've learnt something useful today. In the right conditions, tapply is twice as fast as aggregate, and 4 times faster than ddply.
When learning R, I couldn't get my head round the confusion of apply functions and adopted plyr whenever I could.
But I have an application where execution time is more important than programming time. So I'll have to go and modify those ddply statements...
 
I'm baseist all the way. :)
I steer clear of plyr, mostly because I'm don't understand it well enough to be comfortable when using it.
But I think it's great that people show plyr solutions - I compare them to apply version and learn a bit every day.
 
I can't wait to test the new compiler package in 3.13 that's due out today.
 
Today, already?
 
From Dirk's blog, it seems we should be able to expect a 4-times increase in performance in most cases.
 
Geesh, time sure flies.
Yeah, I'll believe that when Avril Lavigne scratches my back.
 
8:02 AM
You're sceptical?
 
No, I'd just like her to scratch my back.
A guy can always dream, can't he?
 
:-)
 
@Andrie Remember that was an odd example, as Dirk himself notes. I think very few people outside of Luke have much of an inkling of how much the byte compiler will speed up production R code. The main developer on the vegan package noted some modest but useful speed-ups for some of the functions in that package. One lengthy example had ~2 times speed up.
 
@GavinSimpson Ah, that's interesting, thank you. Dashed hopes...
To be honest, for most of my applications, the speed is good enough. The only thing I really hope gets speeded up is ggplot. I frequently write reports with 200+ ggplot graphics. At ~2 seconds each, this means execution time exceeding 6 minutes.
Lattice graphics is much faster, but I am not yet fluent enough to create plots that are reasonably similar...
 
mbq
ggplot is written using plyr (-;
 
8:14 AM
And we've made a full circle. Champagne all around!
 
So if plyr gets a performance improvement, so should ggplot. I know Hadley is working on it. Reshape2 is already much quicker...
OK, now have to get back to paid work. Am building a business simulation in system dynamics using Vensim.
 
@Andrie split() followed by sapply() or lapply() is twice as quick again over tapply() for the problem you refer to.
0
A: How to improve this Algorithm?

Gavin SimpsonIn addition to the approaches provided by @Andrie, the split() then lapply() approach is faster still: > system.time(ZZ <- tapply(train_data$ACTION, train_data$USER_B, mean)) user system elapsed 1.025 0.011 1.062 > system.time(WW <- unlist(lapply(split(train_data$ACTION, ...

 
Amazing. I saw your post yesterday, and have already forgotten. Thank you for reminding me.
I'll add it to the list.
 
Neat.
 
Which of the split, apply, combine solutions one uses will be an individual choice. plyr provides a consistent, powerful interface to solves the problem, but because it works in some many cases, and just works, you pay the price in compute time.
If you are long-time R user, like me, you've learned the syntax idiosyncrasy of the base R offerings for doing these sorts of things. In the end, the difference on the problem over the loop is negligible - not sure what I'd do with the extra 4.5 seconds I gained by using split() then sapply() over the plyr version. :-)
 
8:28 AM
4.5 seconds is just enough time for a popular singer to scratch your back.
BTW, Gavin, how's the baby doing? Do you have time to shave? :)
 
@RomanLuštrik Emma is doing fine - my wife has taken her to Spain to see her grandparents for a few weeks, so I have time to shave but am too lazy. Sporting about 9 days growth at the moment, but that is usual for me!
Thanks for asking
 
Your wife's originally from Spain?
 
@RomanLuštrik nice Q on pgfSweave by the way. Sorry I can't help there; I've not used Windows for years for anything LaTeX related.
 
I'm not worried. Ton of LaTeX guru people crawling SO. I'm bound to be nudged into the right direction sooner or later.
 
@RomanLuštrik Danish actually (mum is from Guatemala and dad is Danish), but they've lived all round the world at various times (Mexico and Venezuela when she was quite young, and several stints in Kenya in later life). Her parents decided on settling in Spain after their last Kenya stay; too cold in Denmark and they know the language in Spain.
 
8:41 AM
And you pay less tax than in Denmark
 
Right, have to get a move on and get to work after the rush/crush on the trains/tube is over. Back online later.
@Andrie Indeed!
 
Well isn't that sweet. I've been looking for my scissors for a week now. It turns out they've been in my pen holder 20 cm away the whole time.
 
9:18 AM
@RomanLuštrik I just checked my Eclipse installation. I get syntax highlighting out of the box for Rnw files.
 
yeah, highlighting, but not code folding or checking for syntax errors
 
K
 
Add an extra comma into one of the chunks and see that nothing happens. In .R files, you will get a wiggly red underline if something is not according to R.
 
That's why I never started to use SWeave, but write Latex directly from my R.
 
Interesting. How does that look?
 
9:55 AM
Broadly as follows:
1. Create an outline tex file that contains the headers and preamble.
2. Inside the outline file have one line that imports a detail tex file
3. Write some R that appends to detail tex file, primarily using xtable statements, but could also use latex() or latexTranslate() in package Hmisc.
Caveat. I don't ever need to have code and output in the same document, which SWeave is very good at. My system is a report writer.
 
You're basically Sweaving by hand.
I haven't worked with functions that translate R objects to LaTeX code (other than some tables). Looks like this is an avenue I need to explore some day.
 
Yes, except it's all packaged up into high level wrappers.
Have you looked at the brew package? It does something similar, AFAIK
 
I think I took a peek a while ago.
 
 
1 hour later…
11:22 AM
@Dirk : cheers mate! Now that's one hell of an interesting blogpost. Thx for the info.
 
I think there was a discussion about this in the morning. Maybe you should scroll up.
If you can't find it, search for "Avril Lavigne". :)
 
12:17 PM
@RomanLuštrik "scratch my back" uhu...
 
12:27 PM
you can't beat the guys of our IT department
now they found out that there is a shortage of computers in our faculty. So they'll be installing 50 "fixed laptops" in one of the auditoria.... How so, fixed? Bolted to the tables?
 
Hallo. @JorisMeys. Thanks, once again, for your answer last night.
 
@GavinSimpson : aren't you just far too kind?
@Andrie Hey Andrie
you're welcome
 
I'm sorry that my question wasn't more clear.
 
sorry for misunderstanding you
 
I've had a look at your answer.
As far as I can tell, you've turned my balanced design into an unbalanced one!
But otherwise the algorithm is fantastic!
I've scanned through most of the paper you sent me. I guess you know of the AlgDesign package?
 
12:34 PM
I haven't been using it extensively
 
It has an implementation of Federov as well as optimum block designs.
One can specify the desired optimisation criterion, e.g. D, OB, Dp, DPC in optBlock.
And D, A, or I in optFederov.
 
I'll check it again. We use mostly hand-written code in the practical exercises to show what is going on, but for other assignments this might very well be a nice addition.
 
I agree that students need to code this type of solution with hand-written code. Otherwise you'll never understand how it works.
 
True, but we had codes submitted for homework assignments that had to run overnight :)
 
@JorisMeys and aren't I regretting not doing that now!
 
12:43 PM
well, you have 3 sympathizing votes for it, so it ain't too bad :)
 
Would it be rude of me to comment on that post that I charge very reasonable rates for commercial support on cluster analysis?
 
but boy, that guy is getting on my nerves.
 
I just tried to include a LMGTFY link to answer a comment from the OP on @Dirk's answer to a gpplot Q from a couple of days ago asking for the link the the R FAQ
1
A: problem saving pdf file in R with ggplot2

Dirk EddelbuettelIt is in the R FAQ, you need a print() around your call to ggplot() -- and you need to close the plotting device with dev.off() as well, ie try pdfFile <-c("/Users/adam/mock/dir/structure.pdf") pdf(pdfFile) ggplot(y=count,data=allCombined,aes(x=sequenceName,fill=factor(subClass))) + geo...

 
@Andrie : that might get you in trouble for spamming. I've seen already quite some accounts disappear for those kind of jokes
@GavinSimpson they know that joke here on SO, alas...
 
Unfortunately, SO won't let you include LMGTFY links in comments which curtailed my fun somewhat!
 
12:45 PM
Exactly why I haven't done it :-)
 
@JorisMeys Party poopers - and I even included a ;-) after the LMGTFY link!
 
@GavinSimpson yeah, boring people, that SO team. Pretending to be Steve Jobs already, bah. ;)
 
Oh please give me strength! Read the OPs comment on an answer of mine:
1
A: How to calculate ranking of one column based on groups defined by another column?

Gavin Simpsondf <- read.table(con <- textConnection("USER_A USER_B SCORE 1 6 0.2 1 7 0.1 1 10 0.15 2 6 0.2 2 9 0.12 3 8 0.15 3 9 0.3 "), header = TRUE) close(con) One way is to split the data: sdf <- with(df, split(...

I need to retire to a dark room to regain some composure. They're all coming out the woodwork today. And there's me thinking we'd get some respite for a while after Catherine was banned ;-)
 
Even Catherine didn't ask such stupid questions.
 
LOL
@GavinSimpson You could've warned me, I spilled my tea over my keyboard
 
12:53 PM
g'morning gents
 
morning JD
 
I used to think that the SO voting system is well thought out and rewards good behaviour.
 
In for a good laugh? the [r] tag is surely entertaining today
 
@JorisMeys Oops sorry :-)
 
Now I realise that it assumes people are reasonably rational.
Which they clearly aren't.
 
12:54 PM
Morning @JDLong
 
Hello, @JDLong
Catherine has cloned herself into @sridher and @user677743
 
@Andrie very true. Folks are insane. That's why I married a divorce attorney. It's the only way I could figure out how to benefit financially from people being bat shit crazy.
 
@Andrie Actually, I still admire the SO model as being very well thought out. Compare this to yahoo answers, and you see it works. Eventually, the community weeds out. But you'll always have new accounts trying the old trolly "gimmedacodezplz"
 
And I'm married to a psychotherapist
 
oh boy... is that an explanation why you lot are hanging out here so often?
 
12:57 PM
@Andrie very good choice there
 
(btw, I'm not married at all, which is indeed an explanation why...)
 
@Andrie I still say we need a "catherine" tag
 
when the crazy questions come along in [r] I just quickly down vote and move on. I used to try to jump in and straighten them out. I did have to post a pissy comment to "catherine" though
 
@JoshuaUlrich How about a "stupid question" tag? Will the SO police allow it?
 
@Andrie I think the SO community will argue that that's what the 'close question' button is for
 
1:10 PM
@GavinSimpson : you beat me again. But I'd use Axis instead of axis with POSIXlt data
 
Yeah. My reputation is now greater than the year.
 
@JorisMeys There was a typo in the solution I provided as I copy pasted old code in. Now fixed. But I prefer your Axis() as you don't need to care what class your data are.
 
hehe, I was about to delete my answer in fact :)
 
@JorisMeys No don't - it (yours) is a better solution than mine, but the two complement each other showing a couple of different ways.
 
No worries, I didn't.
I'm amazed at the behaviour though
I checked my histograms (I always use filled ones), and my axes are never the same color as my fill.
until now...
 
1:23 PM
@JorisMeys Don't know if it has changed recently (i.e. plot.histogram has become more complex lately)
I was being Ripley-esque with my suggestion that the behaviour was intended :-) It is unexpected yes, but intended? Probably. Just an unintended consequence I think.
 
this is crazy. I try it with my old code, and no prob. I guess it has something to do with the data type.
 
@JorisMeys there is a hist.POSIXt()method which does things differently
 
<rant> when Google releases code into the open source community why the heck does someone always say "where's the benchmark? why didn't you do XYZ for me?" GOOG is a public company who's kinda doing us a favor by kicking out some code into the community. google-opensource.blogspot.com/2011/04/… </rant>
 
@GavinSimpson Just figured that out, I'm writing to r-devel as we speak
 
@JorisMeys Yep, got it too. hist.default() calls the plot method and sets col, passing on other args in ... so as far as the underlying plot call and subsequent axis() calls are concerned, ... doesn't contain col. But with the hist.POSIXt method, ... does contain col. Sneaky!
 
1:32 PM
@JDLong That goes for anyone who releases their code to the community.
 
@JoshuaUlrich very true.
 
@GavinSimpson Sneaky indeed. and the hist.default doesn't even call axis() in any way.
 
@JorisMeys it does, r is an object of class "histogram" and the plot() method calls axis(...) - by subsequent I meant the axis() calls in the subsequent plot() method.
 
I mean, it doesn't call it within the function hist().
 
1:49 PM
@JorisMeys semantics ;-)
 
Ha, I came up with a similar solution as Barry Rowlingson on the r-sig-geo list. Too bad I didn't send my answer in.
Makes a person feel pretty OK. :)
No wonder I'm cold, it's 15°C in my room.
Temps plummeted yesterday, there was even some snow on low laying hills.
 
2:17 PM
here too, almost 10 degrees less yesterday as in the weekend
but no snow alas, we have only rain (and plenty of it)
 
I was just talking to a colleague in Dublin. When I asked him about the weather he replied simply, "Meh. It's Ireland."
 
-4
Q: Clustering in R

sridherI'm new to R and want to do some clustering. What packages should I look at?

with a bit extra help, @Gavin could get a Reversal badge for this
 
Have done my bit already. He sure deserves it.
 
one extra downvote for the question, 11 extra upvotes for the answer, and there we go
 
Have added a question downote
 
2:29 PM
just up-voted
 
I want to make a sandwich. I understand I need a refrigerator. So I bought a refrigerator at Sears. But there's no sandwich in it. What do I do next?
4
 
fortune("brain")
 
@JDLong Plug it in.
 
@JorisMeys Going to need quite a few more up-votes for that to happen. Haven't managed a +20 answer yet for any of the answers I've provided here.
But thanks for the support - I'm touched! :-)
 
Worth a tweet?
 
2:37 PM
@JDLong Get in and if you wait a (possibly) long but finite amount of time, a complete sandwich will pop into existence from seemingly nothing. You'll have to be quick mind if you want to eat said sandwich; It will disappear again almost immediately.
@Andrie I don't twitter I'm afraid
Hi @Jugglingnutcase what brings you to our little corner of the interweb?
 
@GavinSimpson just checking out stackoverflow chats in general. i've never used them. Plus i've seen R around the web lately, figured i'd see what the community is like :)
 
@GavinSimpson is this Schrödinger's sandwich, per chance?
 
@Jugglingnutcase Well don't hold the transcripts against us! :-)
 
@Jugglingnutcase the R community is mostly harmless :)
 
@GavinSimpson oh man am i trying ;)
@JDLong then i wont panic.
 
2:41 PM
We're a closely knit community. I hope you know how to knit.
 
@Jugglingnutcase : Be very welcome, and let us convert you to the right belief. (I've noticed some grave sins of the VBA and Excel nature in your posting history, but all will be forgiven by the great Powers that aRe if you redempt)
 
@JDLong Dirac's --- IIRC he is to blame for antimatter and the quantum foam
 
@Jugglingnutcase perfect. BTW, we don't declare variables around here. That's a common faux pas.
 
Variable just is, no need to declare it. :)
 
Quite a gathering we have in here today - nice to see!
 
2:42 PM
It's like announcing every person that walks into a room and commenting on his or her apparel.
 
I'd almost get out the fridge with cold beer... if I knew where to buy such a fridge
 
@JorisMeys haha. i'm hoping people wont hold my job against me! We're firmly in the Microsoft camp here. Which is (kinda) fine. i dont mind .net in general, but when i mess around with VBA i want to die.
 
Large gatherings seem to be prompted by idiots posing stupid questions. Did I miss something?
 
This is my.vec. He's sporting a character suit.
 
@JorisMeys I was just down in our firm kitchen and there was beer and 4 bottles of wine. I may have to hide in there later.
@RomanLuštrik ha! seems pretentious.
 
2:44 PM
Exactly.
 
@JDLong If it were Schrodinger's, would you be the cat?
 
@Jugglingnutcase just kiddin' Some people in the R community have a certain feel towards microsoft. I often have to do penance for allowing Windows 7 on my computer
 
@GavinSimpson I wasn't sure if I was the cat or the sammich was the cat. my IQ is too low to be good at analogies.
 
@JDLong - thanks for the tip re: This Programmers Life interview with Jon Skeet
 
@Stedy oh cool. glad you enjoyed it
 
2:47 PM
It was a while ago but I finally got around to listening to it last night, very interesting view on competition
think how much better he would be with a more reliable wireless network on the train! scary
 
no kidding!
I find "This Developer's Life" really interesting. I think because I'm not a professional developer, but clearly wired like one.
 
have you listened to any of the other episodes?
 
@JDLong The problem is, you're also inside the box (fridge) and from what I recall of Schrodinger's thought experiment, that's somewhere I wouldn't want to be waiting for a sandwich ;-)
 
@GavinSimpson as long as there is beer and wine. Or is there...?
 
@Stedy yes. Some are really interesting, some I've just skipped after 15 minutes.
 
2:49 PM
Depends how good the sandwich will be I guess
 
@GavinSimpson ah, good point.
 
@JorisMeys understandable. After school and lots of Unix training it was a weird transition. i miss having a good command prompt that i'm familiar with.
 
or the quantity of alcohol
 
hmm I will have to check the rest out
 
@Stedy It seems there is a trend in the right direction. The podcast is quite new. So they will refine it over time. It has (mostly) good editing, which is huge
 
2:50 PM
@Jugglingnutcase Oh, the command prompt of R is just delicious. You'll love it right away. Even the good ol' ls() is at your service :-)
 
Can you guys tone it down in here? Some of us are trying to work and I feel like I'll miss some of Gavin's wisdom or JD's antics if I don't follow the conversation.
 
@RomanLuštrik sorry. I'll keep my antics to a minimum.
 
@RomanLuštrik I'm about wisdomed out
 
yeah I really liked the editing
 
2:52 PM
I was showing some of my models to one of our Python devs this morning. I had to show him the obligatory plot(density(rnorm(1000))) as an illustration of why I got hooked on R
 
You could also show him curve(x/1+x, from = 0, to = 10). I adore this function. :)
Whoops, curve(x/(1+x), from = 0, to = 10). Missed some braces there.
 
@RomanLuštrik For a minute I was struggling to interpret your complex equation.
 
Did you see the forum thread I posted a while ago? It was from a body building forum...
 
@RomanLuštrik I missed it, care to share again?
 
Now, I haven't checked all 51 pages, but it seems interesting how an equation like that can generate so much chatter.
 
2:58 PM
@RomanLuštrik I loved reading bits of this when you posted it last week. I didn't get the body-building link mind...
 
So, what's the answer?
42
 
@Andrie 42
@Andrie bugger, you beat me to the obvious! :-)
 
Yes, don't panic
 
Oh, I didn't notice they have a poll at the top.
Close call.
 
@GavinSimpson somehow I'm missing the obvious...
 
3:04 PM
42 (forty-two) is the natural number following 41 and preceding 43. The number has received considerable attention in popular culture because of its appearance in The Hitchhiker's Guide to the Galaxy, as the "Answer to the Ultimate Question of Life, the Universe, and Everything". In mathematics * Given 27 same-size cubes whose nominal values progress from 1 to 27, a 3×3×3 magic cube can be constructed such that every row, column, and corridor, and every diagonal passing through the center, comprises 3 cubes whose sum of values is 42. * Forty-two is a pronic number and an abundant nu...
 
mornin gents.
 
See the bit about Hitchhikers...
 
aaah, I really have to read that book again :)
 
Morning @Jeff
 
Hi, Jeff.
 
3:05 PM
oopsie, I'm on the verge of getting Ripleyed again...
forgot a reproducible example, bollocks
 
You've got to love the opening line in that Wikipedia entry I just posted:
"42 (forty-two) is the natural number following 41 and preceding 43"
I wonder which poor sod wrote those and where they intend to stop on the number line?
@JorisMeys @BenBolker provided one though
 
It was probably generated by a robot
The existence of robots is one of the reasons I stopped contributing to WikiPedia
 
@GavinSimpson He did, I just saw. Which is what triggered my "oh no, I'm going to get Ripleyed" reaction...
 
@JorisMeys I think @BenBolker's quick intervention might save you this time. There is, at the very least, an infelicity in the way the POSIXt method handles the plotting. I have half a mind to write a new version
 
I'd try it myself if I wasn't so busy <s>strolling SO</s> working my butt off.
 
3:13 PM
thanks for the intro guys, back to work with me :)
 
@Jugglingnutcase enjoy the work
 
@RomanLuštrik hey that's nifty. I'm partial to PDFs though :)
 
3:30 PM
I love @Chase's first comment on this one: stackoverflow.com/questions/5651108/…
 
@JDLong Nice blog entry about synching.
 
@Andrie thanks! I need to get the follow up post written. I've been working some kinks out of my implementation.
 
In case you haven't seen this article: Why Most Published Research Findings Are False
Really interesting. And topical.
 
3:53 PM
Nice provocative title. I'm sharing it with everyone I know. Facebook, exit, stage left.
 
back to writing docs for Rack...
 
All this time, it hasn't struck me to call summary on a list.
 
 
2 hours later…
5:57 PM
hey, can I get transfer credits (reputation) on SO for R-help posts? :-)
 
6:16 PM
@BenBolker Yes, write your own Q copying the OP on R-Help, then post your own answer. That way you get rep for the Q and the answer ;-)
 
6:53 PM
@BenBolker you just ping me when you answer a really good one and I'll ask the same question in SO. You cut and paste your answer. We both win! Yeah Arbitrage!
 
 
1 hour later…
8:04 PM
Can questions be merged?
 
mbq
@JoshuaUlrich Sure; but use a flag to notify a mod.
 
@mbq Thanks; done.
 
8:21 PM
@Andrie It's a classic, and alas not too far from the truth either... Experimental design is a heavily underrated field in statistics I'm afraid...
 
Ciao for now folks.
 
@GavinSimpson Have a good night, oh patient one.
 
@GavinSimpson good night
 
@joshua - are you related to this fine fella? dartmouth.edu/~rogerulrich
 
@Chase Not that I know of.
 
8:34 PM
@gotcha - recognized the last names and wondered
 
@JoshuaUlrich : err, and this guy? metallicaband.net/images/LARS-ULRICH1.jpg
 
@JorisMeys Nope.
 
shame, really :-)
one of my youth heroes in fact, that member of the Ulrich tree
 
Maybe related to them in some distant manner, but it's not like either of them are my uncle or something.
 
You didn't recognize the guy?
 
8:42 PM
which guy?
 
@joris - i could see that making for interesting family reunion weekends
 
@JoshuaUlrich Lars Ulrich, drummer of Metallica
 
@JorisMeys I know who he is. I'm a Metallica fan.
In fact, I was listening to the black album on the way to work today.
 
@JoshuaUlrich Reputation : +10 :-)
Epic band, although I am a fan of all material until black album. And Justice for All was an epic masterpiece if you ask me, but they kind of lost me on Load and Reload I'm afraid...
 
@JorisMeys I don't own those two.
 
8:48 PM
@JoshuaUlrich You didn't miss much
 
That's why I don't own them. :)
My collection starts with Ride the Lightning and ends with Death Magnetic
 
No Kill'em all?
 
@JorisMeys No, but I've thought about picking it up.
 
Not as good as The Great Four (Master of Puppets, Ride the Lightning, And Justice for All and Black Album) , but definitely fun to listen to
It's raw, the good ol' garage days :-)
 
Do you have S&M?
That's another good one to own.
 
8:57 PM
Alas not. I've heard the album, but never got to buy it
I have to admit that I've moved more towards the ska, funk, jazz and balkan music later on.
 
I'm not too big on music. Metallica is the only thing I listen to from my younger days. I often listen to my friend's techno mixes to drown out noise at work.
 
ow... I yelled at some students outside my window to turn down that bloody noise. Hope it wasn't your friend's techno mixes :-)
 
I'm sure it was his. You're a jerk. ;-)
 
bloody right I am. Somebody has to scare those little knowledge-suckers into studying evil grin
 
9:21 PM
OK, I'm off to sleep. Cheers
 
@JorisMeys 'night
 

« first day (176 days earlier)      last day (2999 days later) »