« first day (1182 days earlier)      last day (2121 days later) » 

1:33 AM
can anybody hammer this ? It wasn't tagged r initially.
 
 
3 hours later…
5:20 AM
 
5:32 AM
@DavidArenburg I don’t; I got a promo email from a publisher and was just wandering what other R books they had & thus just browsed through their collection
and: goodmorning @ all!
 
Hello everyone :-)
 
@Jaap oh now I see, they always looking for free reviewers. I've got many mails from them over the years
sup kate
 
6:27 AM
@DavidArenburg hey Dave :-)
 
sup Kate
 
@DavidArenburg you just asked ;-p I'm without kids this week (also previous and next ones), it's oh so quiet :-)
 
Sorry @MoeinHosseini you need over 1K rep in order to get write access here, please see the rules on top right
 
@DavidArenburg such free reviews are a bad deal imo: you get nothing for something the publisher gets serious money for
2
 
@Cath what sorcery is that?
@Jaap ofcourse
 
6:32 AM
@DavidArenburg grand parents ! :-D (it's summer break in France)
 
this is their business model
you get a free book or something
 
I know, but I dont want to play by their rules
 
@Cath not for grand parents though
 
@DavidArenburg definitely not ! :'D
 
my and my wife's parents both have their own small kids and also work
people dont get old around here
 
6:41 AM
probably just because you have kids when you're young. I was almost 30 when I met my husband so that gave both my parents and his the time to grow old enough to be retired :-)
 
Good morning all
 
sup Sot
@Cath I've never been in a vacation without kids
or not in a vacation
anyway, back to work
 
7:00 AM
@DavidArenburg yep, that's where I am: not vacation but no kids. I wouldn't want to be on vacations without the kids though
@DavidArenburg have fun... ;-)
 
@Cath no one said ever
I would pay a million $ for a vacation without kids
 
@DavidArenburg that's because you have plenty of them, I only have 2 ;-)
 
I was once with a single kid and all I did is babysitting her. vacation is the absolute opposite of kids
 
This week-end though, it will be almost that: we only see the kids on sunday because we're at a music festival on saturday :-)
 
the official definition of vacation should be "Any activity that has no kids involved in it"
 
7:10 AM
@DavidArenburg so work basically
 
yes, each day I finish work, I'm telling everyone that I'm leaving for work
3
 
@DavidArenburg I sometimes say I'm about to start my second (or even third) day ;-)
 
My wife haven't worked for years, she recently started working again, she never wants to come home anymore
I'll probably need to bring my kids to work for the next few weeks too
 
@DavidArenburg I can understand. It's like finally being ablo to breath without being constantly interrupted (I felt that each time I went back to work after maternity leave)
@DavidArenburg you're allowed to to that?
 
I think it's more about not being in a constant tension about where are the kids right now and who are they trying to kill this time
@Cath yes, in Israel everyone do that
August is like kindergarten at work
 
7:16 AM
@DavidArenburg isn't that always "each other" for the killing (it works with mine)
@DavidArenburg so you're not peaceful even at work... that's sad... ;-)
 
@DavidArenburg That's sounds pretty terrible to be honest
 
@Axeman welp, it reduces a bit the tension at work as no one really very productive
 
haha, yeah that's one way to look at t
it
 
k, now really back to work
 
7:39 AM
the targets might have been not the right ones, but the information in the Q is too sparse to judge
so imo the Q should have stayed closed
 
@Queen k
 
@Jaap IMO when you get an error , "cannot allocate vector of size ... " those are the basic steps to perform and check.
 
7:56 AM
@RonakShah closed
 
@DavidArenburg the way you describe your kids really makes me laugh. 'who are they trying to kill this time'
 
@RonakShah haha... I keep thinking someone saying.. 'Hmmm... R and Python are not capable for such big data manipulation. I better turn to SAS' :D
3
 
@RonakShah Also closed with a better target
 
@Axeman thanks..that is more specific target. :)
 
@RonakShah true; they seem to be the right dupetargets (but we don't 100% for sure)
 
8:23 AM
Hello
 
what is that ???:
I bet Sotos' algorithm wouldn't be very nice to me ! o_O
 
@Florian He is describing every kid-parent relationship.
 
@Cath stop abusing the system :P
 
@RonakShah lol I wonder what I did (I mean, it's pointless to abuse the system if you don't even know what they consider an abuse...)
 
I just got a captcha while posting an answer on SO. It asked "Are you sure, you are not a robot?" :D
 
8:30 AM
@RonakShah wut!?
 
@Cath those things count volume and come up with rates...you either did something repeatedly or you are now a member in a botnet herd :D
 
yep. Even I was surprised by this, got it for the first time.
 
@Sotos I think I type too fast for them ;-p
(I just made a search for "bsseq")
 
haha
@RonakShah oh and speaking for captcha
user image
5
 
haha..that's a good way to get your code reviewed ;-)
 
8:41 AM
@Sotos excellent !
 
@Florian both
 
11:22 AM
@Queen k
 
12:13 PM
Has anyone tried to password protect a shiny app without the pro server? Joris has mentionde shinyproxy which seems like it's something that can handle authentication. Any other options out there?
 
@RomanLuštrik depends on how Shiny Server is hosted I think; if it is hosted on a Apache Server, you could use .htaccess
 
1:09 PM
Hello guys !
Hello @Jaap
Hello @Florian
 
Hello @eliasah (even though I did not get e personal greeting) :D
 
Oh sorry @Sotos
I didn't see you were present :/
Hello @Sotos ! :)
 
well now the magic is gone
 
I was looking for Florian actually hehe
 
Just kidding :)
 
1:14 PM
how have you been ?
 
not bad. Almost at the point where vacation is a must
how about you?
 
I'm not getting any vacation this year
trying to manage finances :)
but I definitely need one
 
am I making you guys jealous when I tell you that I leave for a three week holiday on friday?
 
@Jaap 3 weeks!??!? Oh, I will find out where you 'll be and I will seek you out!
:P
 
not at all @Jaap :)
I have 30 days to take :P
 
1:30 PM
can anyone here read Ruby?
 
nope sorry
 
@Axeman yes
 
@Tensibai I'm trying to figure out what this does:
def split_into_potential_neuropeptides(sequence)
        potential_nps = []
        results = sequence.seq.scan(/(?<=^|#{NP_CLV})(\w+?)(?=#{NP_CLV}|$)/i)
        headers = %w(di_clv_st mono_2_clv_st mono_4_clv_st mono_6_clv_st np
                     di_clv_end mono_2_clv_end mono_4_clv_end mono_6_clv_end)
        results.each { |e| potential_nps << Hash[headers.map(&:to_sym).zip(e)] }
        sequence.potential_cleaved_nps = potential_nps
end
The NP_CLV comes from somewhere else and is a regex to match I think. Basically this part:
DI_CLV        = 'KR|RR|KK'.freeze
      MONO_NP_CLV_2 = '[KR]..R'.freeze
      MONO_NP_CLV_4 = '[KR]....R'.freeze
      MONO_NP_CLV_6 = '[KR]......R'.freeze
      NP_CLV = "(#{DI_CLV})|(#{MONO_NP_CLV_2})|(#{MONO_NP_CLV_4})|" \
"(#{MONO_NP_CLV_6})".freeze
So it does some kind of reverse and forward regex lookup I think? But then I don't understand what the headers are about.
And I don't understand the |e| notation, which keeps coming back
 
ok, so |e| is the variable to use in the each block
results = sequence.seq.scan(/(?<=^|#{NP_CLV})(\w+?)(?=#{NP_CLV}|$)/i) <- This line makes a regex amtch case insensitive to split the sequence
(?<=^|#{NP_CLV}) <- this is a lookbehind, ensure you can match that before matching (\w+) (which is the first capture
 
1:47 PM
@Sotos mission accomplished ;-)
 
(?=#{NP_CLV}|$) <- this is a lookahead, which means ensure you can match this after (\w+), it match either NP_CLV or end of line
Detailed explanation of the regex here: regex101.com/r/0b83Xh/1
 
ok, yes I follow so far
 
So mostly, the line scan the array and retrieve only the text between omitting leading and trailing NP_CLV if present
headers = %w(di_clv_st mono_2_clv_st mono_4_clv_st mono_6_clv_st np
di_clv_end mono_2_clv_end mono_4_clv_end mono_6_clv_end) <- this line just create and array, it's the same as ["di_clv_st", "mono_2_clv_st", "mono_4_clv_st", "mono_6_clv_st", ...]
 
Hi @eliasah, sorry I was in a meeting
 
1:52 PM
trying to do some actual work between SO'ing ;)
 
results.each { |e| potential_nps << Hash[headers.map(&:to_sym).zip(e)] } <- here it's a loop over the results obtained in first line, for each entry, put it in e variable and then create a hash using as key the value of the variable headers using current index of results
 
Hey @Florian ! No worries
 
@eliasah in total I have 54 this year (bought 10 extra days), but most of them are invested in my PhD thesis
 
@Florian I just wanted to tell you that I've closed a question you've answered on
no hard feeling
@Jaap now i'm jealous ^^
 
add the resulting hash (something like { "di_clv_st" => < value of results[0] > }) into potential_nps
 
1:53 PM
@eliasah no problem, which one?
 
I had to re-open it because the OP was nagging
0
Q: Filling not null values as 1 in pyspark dataframe

Mani RzI have a pyspark dataframe named dataframe_freq as given below, +-----------+----+----+----+----+---+----+ |customer_id| p1| p2| p3| p4| p5| p6| +-----------+----+----+----+----+---+----+ | null|null| 0.6| 0| 0.4| 0| 0.6| | 2| 0|null| 0.4|null|0.4| 0| | null| 0.5...

 
aww, I did miss a point in the regex, there's a variable here #{NP_CLV} which will be replaced by its value at time of run
 
@eliasah you don't have to be jealous about working in your free time ;-)
 
I am trying to answer some pyspark questions to improve and learn myself, also not yet very familiar with what are common questions so I might sometimes miss the dupe ;)
 
@Jaap but I don't have a PhD :D
no worries @Florian
 
1:55 PM
@Tensibai Yes that's the other part I posted, but I think I understand that
 
you can always ping me here or in the apache-spark chat room :)
 
@Tensibai Does this mean that results and headers have the same length?
 
@Axeman I assume so
 
atm me neither ;-)
(but I'm pretty close)
 
that's why I'm jealous Hehehe
 
1:57 PM
@Axeman The map and zip is quite hard to get
 
@Tensibai Yes I had no intuition as what was going on there.
 
main idea is to pair headers and values, some basic exemple with zip here: blog.fourk.io/zip-arrays-ranges-hashes-in-ruby-f63c255dd07c
(that's not far of the vector way of R)
 
@Tensibai that's really useful
 
.map is more or less the same as .each
&:to_sym is use the value and treat it as a symbol (unmutable string in ruby)
So it's mostly pairing the headers and the results arrays into a hash :)
 
Thanks a bunch. Life is hard sometimes when you're an R hack like me, with no formal training or knowledge of other languages
@Tensibai yes looks like a named vector, or named list in R basically
 
2:02 PM
Yep
 
@Axeman you aren't he only one ;-)
 
@Jaap It's pretty common for R users, probably
 
The not natural part is that they loop over results first
I suspect each entry will get the last results value
for <an array>.map(&:to_sym): stackoverflow.com/a/32970835/3627607
The part I'm not sure about is the suquence.seq, I suspect it's from a specific gem (package)
 
@Tensibai yes I think so
 
My guess is you should end up with a (to keep R terminology) list of named vectors
(in ruby a hash of hash)
 
2:07 PM
it's supposed to look for specific amino acid sequences
 
correctign myself, a list of named list
 
@Axeman yes, I think so too; I'm planning to start learning a new language though
but I'm not sure which would be the best choice
 
@Jaap I've been planning to learn Python for a while now, but don't have time
 
time is indeed a scarce resource, I will have some time from november onwards
@Axeman I'm doubting between C, C++, Python and Rust
 
2:26 PM
@Jaap unless you want to work on optimizing low level processes
I advise you to give python a try
 
2:46 PM
@Jaap I suggest python as well. I m happy with it and can easily migrate the code to ...say.. spark
 
that's true
but there is also great python libraries
whether for statistics
machine learning, system engineering, data scrapping, etc.
 
downside is that It does not have the statistical capabilities of R
anyway...off to pick up me lady
you guys have a good one.
See you all tomorrow
 
take care !
 
3:02 PM
thx @eliasah @Sotos I'm indeed leaning towards picking up python as next language; next to R, it is another often used language in our institute
anyway, I still have about half a year to decide ;-)
 
or you can do scala @Jaap :P
 
@eliasah now you're giving me choice stress :P
 
hehe that was my point ^^
I can even give you more choices
 
no worries, I can do that by myself as well ;-)
 
Java/C/C++/Scala/Python/Php
Those are the languages I use :P
I didn't say R ofc Hehe
 
3:08 PM
@eliasah isn't Scala a kind of evolution of Java?
 
well no
scala is a functional language
but it runs on a java virtual machine
 
5:01 PM
Any Rcpp user here?
Oh... looks like I come at a wrong time; no one else in the room expect the Queen bee
@Queen hello bee
 
@李哲源 yup named "GMTs" for a reason, also nice histograms illustrating it here chat.stackoverflow.com/rooms/info/75819/gmts
(also, doubtful i can help re rcpp, only used it a couple times years ago)
 
No, I am about to ask something really basic
no actual coding
 
ok
 
I am a C programmer not a C++ one. I have been using the conventional R's C interface but feels I might soon become an outdated alian
alien.
So, I want to have a look at Rcpp to whether I could benefit from it.
But, the very first question is: who is Rcpp for? Is it purely for C++ users?
 
afaik, r is written in c (and fortran) and package developers sometimes use it as well, so if you're more comfortable using it with r, i expect that'll remain a viable option for a while (though maybe there's something changing with c that i don't know about?)
@李哲源 i think it's for stats/data analysts who need to improve the performance of their code and aren't super sophisticated. rcpp + armadillo is very easy to learn and use without knowing much c++, i guess
 
5:14 PM
There are so many things associated with Rcpp. What are they? I am extremely confused.
 
Dirk has a bunch of examples somewhere, can't find it browsing here at the moment, though ... rcpp.org
 
Rcppeigen, Rcpp armadillo, etc?
Alright, let me narrow my question a little bit.
 
(ah here are those examples, though the formatting has changed since i last saw it: gallery.rcpp.org)
 
C does not seem to have its own numerical libraries. For example, if I decide a write a section of code to add two vectors, I need to write a loop. Of course I could write a function than reuse it later. But I feel that if I am going to write a big C project for a complicated computational module, I have to start from scratch.
 
@李哲源 ok, that makes sense. (i've never used c.) insofar as eigen and armadillo are handy for math stuff, yeah, looks like they are c++ libraries
 
5:20 PM
So eigen and armadillo are C++ languages, not with Rcpp?
 
the rcpp.org site explains that they're c++ libraries that the rcpp team have added r packages to interface with
 
Do all the computer have them? I mean, I surely need to include some header files to use functions from some libraries. Are those libraries and header files shipped with computers already?
 
@李哲源 i guess the vignettes answer questions like that (over my head / beyond my recall, as i haven't used rcpp in a while)
but yeah, seems like you need gcc or whatever c++ stuff installed. i bet RcppEigen comes bundled with Eigen but don't recall
(looks like i installed "rtools" to get gcc since i'm on windows)
 
I have no problem with gcc, as I am on Ubuntu Linux. I also have g++ so I should have all C++ libraries already.
I feel so difficult just to find a right starting point. Do I need to learn C++ programming language first?
 
5:36 PM
@李哲源 my experience with it: i had found a part of my code that was doing something very simple very slowly; looked at arma.sourceforge.net/docs.html and some examples to work on replacing just that small part of my r code; dropped it in and saw a performance improvement. repeated that experience a couple more times and didn't regret the couple hours it took me to learn the minimum i needed to take advantage of rcpp
granted, i'm not proficient in rcpp as a result, but my goal as a user is to take advantage of it, not to code my entire project in it
also, i've used it just to mess around with / learn about some data structures that don't exist in r, like unordered sets stackoverflow.com/questions/35544706/… performance improvement in my benchmark there was tiny ...
 
Emm. Thanks. Let me just start look at armadillo first. Hopefully I could read it.
 
np, curious if you end up finding it useful
 
Basically I just want to know whether Rcpp can help me build up a complicated computational module quicker
If I can use something existing (and well optimized), that would saves me a lot of efforts.
 
if you have more specific questions after getting a rough familiarity with it, could always ask Dirk et al in the other chat room or post a question
 
@李哲源 btw, there's also r's c interface adv-r.had.co.nz/C-interface.html
(again, i have no experience with it at all)
 
That page is for the conventional interface I am rather familiar
 
ok
 
Hadley in its book actually introduces Rcpp before this conventional interface. Looks like he wants to say: if you want to write new stuff, use Rcpp; but if you want to see how old stuff work, read this part
 
@Jaap Also recommend Python (or else Scala). No open-source language has the statistical capabilities of R, but the most useful things get backported to Python etc: ggplot, patsy for the formula interface to building models, and now the Python port of datatable
Anyway you can pick up the basics of Python in a couple of days. I found "Python Essential Reference" by David Beazley a better quickstart than textbooks. Plus online tutorials or quickstarts.
 
 
1 hour later…
7:18 PM
@smci I know about the Python port of data.table and it is certainly an important reason for considering Python as a 2nd language
 
7:34 PM
@smci hmm, I didn't know about this... If and when this is stable and competitive with Rs data.table I think I would be able to leave R for good in order to finally to get some peace of mind from the tidyverse madness
 
my impression after looking at numpy was that it would never compete for my daily usage since they forgot to implement missing values. whoops. (guess i might have brought this up before.) it writes really cleanly and has data structures that r doesn't, so it's nice as a second language to learn some CS and maybe someday use in some small way
 
@DavidArenburg I remember having seen a Youtube video a while ago of Matt talking about this
 
@DavidArenburg was even posted in this room at the end of last year, demo starts at around 33:00
demo video itself can be found here
 
8:53 PM
Coming across a dev who can type regex without looking it up https://t.co/rndqwZcwDb
 
9:09 PM
Cool package, but never had the need to install/use it.
 
9:48 PM
yeah, cool stuff, i could imagine using it
 
10:03 PM
@Frank what do you mean? There is np.nan, np.isnan, etc.
@Jaap interesting, I haven't reached that point in the presentation
though the sntax looks quite annoying
 
10:32 PM
@DavidArenburg NaN and NA are distinct in R for a reason. i don't want to overlap "mathematically not a number" and "no measurement / unknown number". i think this is related numpy.org/NA-overview.html (i'm talking from a point of relative ignorance here, not having used python for anything yet. this was just my takeaway/disappointment when i initially learned some python)
re that link, those bullets are pretty important questions, and i'm glad that i can mostly trust R Core to have found the right answers long ago (so i don't have to interrupt my real work to address them anew...)
 
@Frank What are you talking about, numpy has np.nan (float or integer). And see also pandas. The downside is that if a column has even one single NA, that bloats that column's storage requirement from 4 -> 48 bytes. You will see some contortions on e.g. Kaggle about marking NAs with a separate (boolean) column or doing imputation early, when memory is critical.
 
@smci yeah, that sounds like extra mental overhead that i do not want, for one thing
given that they did not implement NA and NaN separately from the start (nor did it even occur to them, apparently), can i trust mean and similar built-ins to behave as expected wrt NA? will factors with NAs make sense? etc
the gains (cool data structures, cleaner syntax, and more i'm sure) are not immediate for me, given the work i do, and i don't think they'd ever wipe away the weaknesses built into a language not built for statistics. so, i'm just saying that unlike David, i don't see myself ever switching from r to python
 
@Frank That was numpy/pandas. I expect the Python port of datatable will implement it better. Look, NaN being a float is due to numpy being created in 1995. numpy was for numerical algorithms, not data science. pandas came a decade later (2008), pandas.Categorical came almost a decade after that... and pandas.Categorical is still a second-class citizen.
Whoever takes a look at the Python port of datatable first, let us know your findings.
 
well, yeah, categorical data is important to me. ditto bools that have NAs and correctly return true for TRUE | NA, and so on. i look forward to trying py datatable, though, as i said in the link a ways up
 
@DavidArenburg What is your objection to tidyverse, in brief?
 
10:44 PM
btw:
@pydatatable
7 tweets, 2 followers, following 3 users
 
Btw, what's the one-line verdict on Julia?
 
i hope julia sticks around long enough for me to use it. as far as i can tell, it is built for both stats (like r) as well as math, eg has a class for rational numbers, which sounds useful for projects where floating point rounding isn't tolerable. i don't know if numpy has such a class
 

« first day (1182 days earlier)      last day (2121 days later) »