« first day (2402 days earlier)      last day (773 days later) » 

10:17 AM
"cricket" %>% paste0("s")
 
10:36 AM
ooh, tidy crickets
 
10:52 AM
@MonicaMonica feel free to visit chat.stackoverflow.com/rooms/25312/r-public
 
11:12 AM
Someone is killing it on r-devel by suggesting to add ggplot2, dplyr, tidyr to base R.
Popcorn, please.
And as an aside suggest to port the GPL parts of RStudio into the base R GUI.
That's a good one. Too bad nobody else ever thought of that. Maybe add Emacs while we're at it?
 
@DirkEddelbuettel srsly?
oh yeah.
CAPS MAN
They want the CRAN and R project pages updated too.
Tempted to point them to the internet archive for that
 
11:28 AM
I was thinking of you on that one :)
 
CRAN still uses iframes
 
Yes, the web working group got to update www.r-project.org, but CRAN is, well, a world onto its own.
 
11:57 AM
Someone needs to tell this person how many dependencies Hadley's packages have and how careful he is about not masking base functions.
 
 
2 hours later…
2:23 PM
Lately when I see folks debating Hadley's packages I find myself thinking: "You know, Jesus seemed like a really groovy dude. I think just maybe some of his followers took things a bit too far."
 
data.table is.... weird... setDT(df1)[df1[, .I[.N < 10], country]$V1, country := "other"][]
 
@joran Or, "Jesus may be great for you, but please don't force him on me."
 
@JoshuaUlrich the rest of us are still waiting for the true messiah
messyr?
 
2:42 PM
@JoshuaUlrich Yeah, that would fall under "taking things a bit too far", imo.
@Spacedman What, you don't find that readable?
 
@Spacedman if we get a having= parameter, it'll be less weird for that case github.com/Rdatatable/data.table/issues/788 like setDT(df1)[, country := "other", by=country, having=.N < 10] or something
 
@joran .N is magic, .I is magic, the last [] seem to be there as a magic way of making it print...
I assume V1 is there because some intermediate thing has only one column....
plus if you do df0=df1 before running that it overwrites df0 too. Looks like a reference bug.
 
@Spacedman you'll need to make a copy(df1) else the by reference behaviour (overwriting df0) is acutally on purpose
 
yeah, those [] and reference quirks aren't going away, i guess, nor V1... not really any viable, intuitive alternative in any of those cases, is my understanding
 
@Cath data table considered harmful then.
 
2:52 PM
@Spacedman At least it's fast magic. ¯_(ツ)_/¯
 
@Spacedman documentedly so, fwiw, kind of the point of the package, really
 
especially if someone (akrun) tosses out data table solutions as one-liners to newbies who will not have understood all that. Heck, I don't, and I'm an oldie.
 
@Spacedman then akrun is harmful not data.table ;-)
 
I mean, my df0 is a data frame, wtf is it data table's business to turn it into a data table and muck with it?
 
well, gotta read the docs when using a package. anything could happen... part of why i use so few of them
 
2:54 PM
Keep your government hands off my data frame!
 
docs say (paraphrasing): "set* breaks all R conventions wrt references. have at it."
 
@Spacedman if you did df0=df prior to setDT(df) then it's strange df0 was modified...
 
@Cath yup. df1=data.frame(id=1:5,country=rep(c("USA","Oman"),c(3,2)));df0=df1;df0[1,2];setD‌​T(df1)[df1[, .I[.N < 10], country]$V1, country := "other"];df0[1,2]
 
@Cath strange, but internally consistent, eh
 
you may want t print() those elements
 
2:57 PM
@Frank I tried df2<-df; setDT(df)[, newcol="a"], it modified df not df2, copying a data.frame prior to conversion to data.table makes a copy and that cannot be change by a call to setDT
 
and cut paste from here seems to fail a bit...
> df1=data.frame(id=1:5,country=rep(c("USA","Oman"),c(3,2)));
> df0=df1
> df0[1,2]
[1] USA
Levels: Oman USA
> setDT(df1)[df1[, .I[.N < 10], country]$V1, country := "other"]
> df0[1,2]
country
1: other
 
@Cath well it modified df2 to become a data.table via setDT
 
@Spacedman ok that is strange
 
I have to go empty the compost bin now, which stinks slightly less than this does...
 
heh
 
3:00 PM
@Frank indeed I get a data.table for df2 but not the newcol, though I get it for df... I don't understand what's happening
 
@Cath probably a bug
@Cath yeah, df = data.table(a=1); df2 <- df; df[, b := 2][]; df2 does add the new col, but with data.frame + setDT it does not. seems report-worthy to me
 
though it does modify the df0 with Spacedman example...
@Frank probably indeed, you go ?
 
@Cath it seems to be a question of whether the set of column pointers is kept consistent across the two. somehow sapply(df, address); sapply(df2, address) show that the first col is the same in your example, but somehow the second col just doesn't get added. sure, i can report it, will stop cluttering this chat
 
@Frank thanks :-) (you'll explain much better than I would ;-) )
 
3:40 PM
Anyone got experience with rlang? It's the next iteration in Hadley rewriting base in his own image
0
Q: Clean way to do nested lazy evaluation in rlang

Hong OoiLet's say I have a function f that takes a bunch of arguments, along with an optional extra argument. f <- function(..., extra) { arglst <- lapply(quos(...), get_expr) if(!missing(extra)) { extra <- get_expr(enquo(extra)) arglst <- c(arglst, extra=extra) } arg...

ISTR that the problem in that question also exists in base; I can solve it by abusing eval and parent.frame but that way is just horrible
 
 
3 hours later…
6:23 PM
@BenBolker this is an interesting question. Can you chip in about what we are assuming in random effects? stackoverflow.com/questions/44025540/…
 
 
1 hour later…
7:50 PM
@Spacedman I think it is because df0 and df1 have the same address in memory:
> df1 <- data.frame(id=1:5,country=rep(c("USA","Oman"),c(3,2)))
> df0 <- df1
> address(df1)
[1] "0x109f44980"
> address(df0)
[1] "0x109f44980"
 
@Jaap yeah,
 
but when you use df0 <- df1[, 1:2], df0 gets another address
 
but its still a bad thing that data.table modifies both
theres probably a reference count somewhere that data table should check ==1 before it starts mashing up things
 
agreed on that; I wouldn't expect that either
(and warns me to use the copy-function when making a copy)
@Spacedman actually, it surprised me that making a copy of a dataframe gets the same address; I always thought that a copy of a dataframe got assigned to a new address
 
I've seen some bugs related to that before so didnt suprise me
 
 
1 hour later…
9:23 PM
3rd most voted data.table question : stackoverflow.com/a/14293056/403310
I think that covers it. I'm not aware of any bugs in this area.
 
9:56 PM
@MattDowle I just wasn't expecting data table to fiddle with my data frame... sigh
 
10:11 PM
That's what setDT is for though. And you used setDT. setDT changes the data.frame to be a data.table by reference. People wanted us to provide setDT so they could do just that deliberately. Say if the data.frame is large and they don't want to copy it.
Use copy() or as.data.table() instead if you don't want by reference.
You had to explicitly call setDT yourself, and you did.
All the set* functions, and :=, operate by reference. You have to explicitly use them though. <- and = work just as base and copy-on-write.
And they want to do that because it doesn't work at all otherwise ... out-of-memory.
It's not just about speed but making it fit at all and getting a result at all.
 
@MattDowle the user needs to be mindful of just tossing a setDT into the middle of their code, in case they had <-s for data.frames in there earlier. answerers on SO always write setDT, but users are liable to sometimes have cases like Spacedman identified, in which case, somewhat weird things can happen, like github.com/Rdatatable/data.table/issues/2170 so maybe we ought to more regularly tell people "read ?setDT or use data.table()" rather than just answering with it
besides the answers on SO, the common line "just use data.table when you need speed" also leads people to copy-paste data.table code into the middle of their existing code base, i guess
 
@MattDowle I used setDT on df1, I wasnt expecting df0 to change.
 
I haven't seen that issue yet. Looking ...
 
@Spacedman You are as a Python user should be familiar with that behavior I would think... For instance
a = b = [1,2,3]
a.pop(1)
b
## [1, 3]
Same goes for C++
 
@DavidArenburg yeah, but when R - a so-called "functional language" does it - I freak out.
 
10:24 PM
@Spacedman But this is what modifying by reference is all about. So in this case data.table isn't behaving like you would expect from R, rather how you would expect from any other language that modifies in place
 
But R's keeping of df0 as a reference to df1 (data frames) is a purely low-level convenience.
If something in base R does something to df1, df0 doesn't change value.
If DT does something to df1, wtf, df0 changes.
now if instead of df1 and df0 I have dt1 and dt0 (data.tables) then no argument, do whatever the package wants. But leave my dfs alone!
 
I expected Python to have vectors. It doesn't and I freaked out. You have to install a package to get an array in Python. That broke my expectations massively.
 
@Spacedman That's because once that happens, they aren't linked to the same pointer anymore, while this isn't true for data.tables
df <- df2 <- data.frame(a=1)
address(df)
address(df2)
df[, "b"] <-2
address(df)
address(df2)

df <- df2 <- data.table(a=1)
address(df)
address(df2)
df[, b := 2]
address(df)
address(df2)
 
Yes, I get that.
What I would have preferred would be if DT tries to work on something that isn't a DT, then if that object's value has a refcount !=1, meaning there's copies, it makes a copy first.
 
The trouble is R's refcount gets easily bumped (e.g. by base functions along the way and when passed as an argument). It has improved over the years but it's still too eager last time I checked.
 
10:36 PM
but bedtime.
 
ditto ^
 
11:02 PM
That issue (2170) helps a lot. After a setDT(), in this case of df1<-df0<-data.frame(), adding a new column using := (works according to expectations) is different to updating existing columns (breaks expectations). Ok got it. Maybe there is a solution ...
 

« first day (2402 days earlier)      last day (773 days later) »