« first day (2674 days earlier)      last day (501 days later) » 

12:35 AM
@BrodieG well my biggest concern is that it doesn’t capture environments
 
 
1 hour later…
1:59 AM
@hadley Right, that's a conscious decision to minimize complexity in the simple/common use case. One question I've been wondering about: what's the reason to require unquoting by default for evaluation? Would it not have been more natural to auto-evaluate and mark instead the rare cases that should not be evaluated?
At a minimum, it would have allowed hiding the !! business from most users who just want the basic features, which would make it a lot more accessible.
@hadley Not sure if you got to the vignette, but as per my point above even the simple quote/unquote adds complexity that isn't needed in the simple case.
 
 
1 hour later…
3:08 AM
The failure of constructed languages seems to be an instance of "Worse is Better". Barbara Tversky makes the point somewhere that human languages benefit enormously from hundreds of years of user testing and modification.
 
3:18 AM
@DirkEddelbuettel What a great reflection on the value of user-focused, changing languages (linguistic or programming)! I think probably any programming language is closer to Esperanto than English in this analogy, but I love this affirmation of the value of embracing change and the value of users.
 
3:54 AM
I always knew confirmation bias was alive and well.
2
 
 
6 hours later…
10:12 AM
@hadley "it doesn't capture environments." There I didn't understand what you mean. You are talking about subset, no?
 
 
4 hours later…
1:48 PM
@JorisMeys I'm talking about subset specifically and oshka in general. See the second problem in adv-r.hadley.nz/evaluation.html#base-subset
@BrodieG "more straightforward" = "more magical". I don't like that because (as with substitute()) the amount of context you need to read to understand a statement is much larger than with tidy eval
(I'm also reasonable certain that eval/bquote will not work in general because each element of ... can potentially have a different argument)
 
2:31 PM
@JoshuaUlrich A truly hilarious rejoinder ("I always knew..."). But you'll forgive me if I maintain that my original joke is funnier.
2
 
2:46 PM
@hadley oshka is a term not familiar to me. What do you mean by that? And regarding subset: I don't even understand why people bother using it. Plus, you show subset() use inside functions, which is specifically warned against in the help page. It's one of the few R functions that come with a warning, so why keep people ignoring that?
subset() is a NSE wrapper around indices. Something that should've never been created in the first place imho.
2
and I don't see how subset should "capture" environments. subset() does what it does: look in the calling environment, and if it doesn't find anything there, it moves up into the enclosing environment of the calling environment. Which is the reason why it shouldn't be used in functions in the first place.
So when you say subset2() catches the environment, that begs the question: which environment? the binding one? the enclosing one?
 
3:17 PM
 
3:29 PM
ah OK, that explains why we had to wait on BrodieG. So that's basically a one-function package to change the lookup sequence from the enclosing environments of the calling environments to the callstack then?
 
seems to be. i haven't tried it myself
 
I think I still misinterprete what it does when looking at the code.
 
3:46 PM
@JorisMeys It is a mechanism for expanding quoted language. A simple example:
a <- quote(x + y)
b <- quote(a + z)
c <- quote(b * w)

oshka::expand(c)
## (x + y + z) * w
It can be used with NSE to allow user to specify symbols indirectly.
@JorisMeys Do you also think data.table should not use NSE? dplyr? with? They are all quite convenient IMO.
 
@BrodieG allright, got it.
@BrodieG These are convenient functions/packages. I've never seen anything convenient about subset() though. It just confuses students and is one of the biggest frustrations until they get the slide "Do Not Use subset() when Programming." with the explanation as to why it is a bad idea.
And then I don't mean the intro stats students that use R, I'm talking about the ones in statistical computing that previously learnt to use subset instead of indices.
So I might be biased against subset(), but I've never found a good use for it myself.
 
4:48 PM
@hadley did you mean different parent frame (or some such)?
 
 
2 hours later…
7:01 PM
@hadley, I'm aware that you disagree strongly here. I respect that you have a very differing opinion from myself. Furthermore, thank you for making available your tidyeval video and for emphasizing the existing resources to learn more about tidyeval. I took the opportunity to peruse the video earlier this morning.
I would like to take a few moments to respond to some of your criticisms. I've organized my response into three parts: organic evolution of tidyeval design, designing for dplyr, and adoption rates. Before I begin, I do want to stress that again I highly respect the body of work you have produced; I just have severe reservations about the approachability of the latest mental model. My opinions are solely my opinions.
To begin, let's visit the stated goal of the tidyverse:
> An opinionated collection of _R_ packages designed for *data science*.
>
> All packages share an underlying *design philosophy*, *grammar* and *data structures*.
Regarding the design philosophy and grammar these came into being through an organic nature. That is, part of the current framing was based on contextual inquiry and mental user profiles based on your graduate appointment as an analyst in ISU's consulting office.
However, the underlying theory was made on the fly as indicated by Lionel (c.f. chat.stackoverflow.com/transcript/message/40690552#40690552 ). This is one reason we see drastic swings from year-to-year regarding tidyverse design and are forced to update existing code to still live in the 'verse.
The partial cause for this is there still hasn't been a tried and true document produced that explicitly describes the tidyverse design outside of tidy manifesto that ships with the tidyverse (c.f. chat.stackoverflow.com/transcript/message/40679093#40679093).
Again, I understand it takes time to write down such a philosophy, just look at how long it took for probability to receive an axiomatic system since its introduction in the middle ages. Moreover, the philosophy request comes while the framework is being built as its being used at scale (c.f. "building planes in the sky" youtube.com/watch?v=Y7XW-mewUm8 ), but I can say that stability of definitions is crucial to adopting tidyverse.
Now, this isn't completely a fair characterization as in the earlier days you did not manage a team at RStudio to build out the data science platform. However, going forward, such swings rest solely on you as you head the development team.
With this being said, I think over time your own mental models with R have evolved considerably. My worry here is the context switch being lost between an expert and a novice.
My comment relating the "simplest possible design" with the current iteration the development team is taking arises in part due to my second point of dplyr evolution; but, I'll end this remark by saying without novice user design stories the likelihood of groupthink done at an abstracted level appears to be more prevelant in this latest version.
Let's switch gears to talking about a specific interface example within the DSLs designed. I'm going to first introduce code between dplyr <= 0.5.0 and dplyr >= 0.6.0 that shows a dynamic filter subset. Something that a lot of data scientists will likely need to incorporate into their code on a daily basis.
set.seed(1412)
df = data.frame(
x1 = sample(5, 10, replace = TRUE),
x2 = sample(5, 10, replace = TRUE)
)

library("dplyr")

# Example old NSE
my_subset_old = function(df, col_var = "x1", obs_val = 1) {

# Two variables
# Use NSE interface
# Create an expression

df %>%
filter_(.dots = paste(col_var, "==", obs_val))

}
# Example New tidyeval
my_subset_new = function(df, col_var = "x1", obs_val = 1) {

# Casts... Which one to use?
# Note sym is not made available by dplyr
active_col = rlang::sym(col_var)
active_val = obs_val

# Check value
df %>%
filter((!!active_col) == active_val)

}

my_subset_new(df)
In the previous iteration, it was clear that the NSE interface for filter() was being chosen by function call and in pararmeter specification. It was further apparent that variables were variables. There was no "cast" to type X or cast to type Y and then inside the function uncast.
The cost under the new version places a significantly higher burden on users to understand what is happening compared to the simpler mental model that existed before of "variables can vary". Thus, to what ends does this new approach benefit them? The benefit of this new approach is largely in retaining the ASTs that many users never needed to know about previously.
These users are now faced with the difficulty of visualizing and thinking in these highly abstracted states that piaget would argue is not ideal. It is for this reason that I stated the new iteration of the tidyverse design has shifted causal users to sophisticated package developers.
2
The average analyst, student, and collaborator is not a computer scientist. Their end goal here is to take data, analyze, and communicate results to ensure the appropriate data driven decisions are made. This was the selling premise of the tidyverse. The new approach forces them into a role that should be hidden from them.
3
To emphasize this, when was the first time you heard about the concept of an AST? Was it in a CS course or through independent study? I ask as a portion of my work largely revolves around differencing between similarly constructed ASTs. When I talk about my work, I focus the details more on the underlying model than how I've collected and processed the data as I can visually see the pain of processing a tree based model.
Lastly, regarding adoption rates in terms of downloads, I'm not sure that is an appropriate metric in this case. For two reasons:
1. What is the dividing line between "mass scale" adoption and "forced" adoption? For example, knitr and rmarkdown have been established as de facto communication tools by the populous will vs. Sweave. The overall dependency structure of the tidyverse makes it hard to quantify the percent of pure downloads.
2. Given your current stature in the #rstats community, one tweet will reach ~ > 58k. Thus, instantly publicizing a new package and/or framework will likely yield short-term adoption due to popularity alone. How long does this adoption last though? Are scripts being written in a generic sense or formulate to solve only one problem?
I would be more interested at this point in the number of either package converts or new packages that are maintained over the year being analyzed. I think that would give a clearer picture. The only issue presently is CRAN is slowly starting to have too many "development" packages being hosted on it instead of "GitHub". But alas, I digress...
 
7:53 PM
@coatless That's the impression I get too. With the ideal of producing a full-powered and uncompromising NSE framework you get something that ends up bewildering to the uninitiated, at least from my own experience, and from my anecdotal sampling of reaction on twitter and SO. I think the number of iterations on documentation and presentations (and now video) are partially a reflexion of this (at least I'm guessing).
For the record, the two presentations I found most useful are the useR2017 and the fexpr one which are actually kind of hard to find.
Don't get me wrong, I think it is better to have rlang than not. There is no question it adds a lot of useful functionality that doesn't otherwise exist. It just feels that there could be a more accessible interface.
 

« first day (2674 days earlier)      last day (501 days later) »