« first day (1403 days earlier)      last day (3770 days later) » 

00:01
oh hey. I have enough credits to graduate
Uh.. hurray?
I've had a dissapointing summer.
why?
I cannot afford to attend my university. I've had to drop it.
Anyways, doing anything fun in Python?
just some flask shenanigans
you?
I've been working on a GUI api. Based on SDL2. Using their python bindings.
Technically it's a waste of time since Python bindings = an api. But I've been trying to make it simpler. Making Button classes, checkboxes, labels, all that fun stuff.
Flask sounds fun though. I was looking at a book on Amazon for that. Thinking of doing something better than a Google blog.
00:15
The Miguel Grinberg one?
Yes.
I own it, it's good
Got a . . scanner handy? :D
Just kidding. I don't know if It's worth buying a book yet.
I wouldn't move my blog unless I had a real reason to. Right now it's pretty adequate. If I finish my API and decide to write some documentation I might see.
Google Blogs are nice but get limited when it comes to organizing stuff.
01:03
Hello, anyone know if docs.scipy.org is mirrored anywhere, it's not been working (for me) for a while now.
Huh
It's not spectacular, but if that's any help.
Yeah, I tried the cached version, weirdly, it doesn't work either, I think perhaps because it uses redirection.
I dunno, maybe you can use the git repo
Trying to build the docs now
the git repo has some dependencies that I need to track down
01:32
That worked. Had to copy in the git project for scipy-sphinx-theme. Ta
 
4 hours later…
05:41
cbg all
06:03
cabbage @PaoloCasciello & all other
06:24
cabbage
07:23
Cbg
Cbg all
cbg @vaultah
07:46
cbg
'Y2Jn\n'.decode("base64")
08:02
bytes([99, 98, 103])
(in py3k)
cbg()
08:35
Anybody want to answer this? Classic "I load data outside a WSGI function and expect it to change between requests" question.
09:26
Can somebody with more than 250 rep close this question stackoverflow.com/questions/24549359/…
@vaultah Sounds more like "I have this not working code on Github and I expect you to download it, test it, debug it and fix it".
SO block me for asking question so i fixing my old qus
09:48
@rajasimon how is your question being closed going to fix your old question?
You shouldn't be closing your questions, you should be editing them into acceptable questions.
In the hopes of getting them re-opened.
Also don't delete your questions unless they really are unsalvagable. Deleting down-voted questions is a sure-fire way of making you banned from asking questions for longer.
@Ffisegydd Oh.. Thanks for the info. ok i ll try to edit. but i saw somewhere if close questions are eligible to delete completely from SO .. Am i right ?
I'm sorry I don't understand what you mean.
10:18
@Jerry @Ffisegydd cabbage :)
user559633
@rajasimon not all on-hold/closed questions are deleted from stackoverflow. stop trying to get around an automated feature to keep the site from getting flooded with lazy questions.
cbg
user559633
cbg jerry
after all these months of hassels , finally managed to get an accuracy of 93% on the data.
user559633
10:19
sounds like your model doesn't fit your data.
where data = bad , junk , unstructured , dumb spelled
93% isn't bad.
Although it very much depends on what you mean by "accuracy"
@tristan : if u look at the data , you will know how bad it is
user559633
okay, show me the data then :)
each address had upto 21 extractions.. 19 extractions were 100% correct , the last 2 would spoil the address
user559633
10:21
elaborate on that.
input address:
indus bogh appts 202 2nd flr 80feet rd koramangala 6th blk
output:
apartments 202,2nd floor,indus bogh,6th block, 80feet road,koramangala 6th block
@Swordy have you used machine learning to sort it or have you hard-coded the solution?
no damn format , it ate the hell out of my life with loads of regex conditions.. @Jerry knows
Oh I see
Not machine learning
10:24
huh?
user559633
"What's the deal with regex" -- Jerry
Who!?
Used multiple conditions with the base assumption that highest occuring words are correct in spellings
where?
user559633
when? NOW til? QUESTION MARKS
10:24
LOUD NOISES.
uh what?
Used Soundex codes , Bigrams , And some clean manipulations to get away with something decent that can fill my tummy with food :)
You eat regex?
no , regex gives me clean output , clean output gives client statisfaction and client gives me currency , currency gets me food :D
not to forget i got a new cell too :)
Cabbage
10:59
How can I get "Item" ranking position based on rating value? So "Item" that has a most "rating" value is on the first place in ranking etc. dpaste.de/rdCG
user559633
@mamasi sample data?
@mamasi order_by()?
@IntrepidBrit: and how can I get position for specific object?
@mamasi Well, depends on what you're trying to do. Is this going to be a standalone position? Or part of a listing page with a rating? I'd pursue them different ways
@IntrepidBrit This should be standalone position. Please look at dpaste.de/rdCG
11:10
I've looked at it. I'm not entirely sure I would do it that way. I'd be tempted to create an ItemRanking model and regenerate all the rankings at discrete time periods (say everyday)
@IntretpidBrit: Thanks
@mamasi (Depending on which architecture you're using, I would recommend implementing the ranking in a management command - then execute that command from cron)
By using an ItemRanking model, you'd also have more flexibility to track changes in ranks as well
11:43
@mamasi You're welcome
raw strings are really convenient when I want to paste in a path from Explorer, without the slashes getting converted to escape characters.
I would like "C:\temp" to not be interpreted as "C: tab emp", thanks
That's the most fun
you can do C:/temp as well
Okay, I've got a question based on a first post review
But I am lazy, and "put an r in front" is faster than "look at the path and click on each \ and then delete it and then type /"
11:55
The person's answer isn't wrong, it's just not "pythonic" Stack Overflowic
Or to put it another way, it could be clearer. Should I just charge in there and modify it to what I think it should be?
I don't think it'll be the top answer either way
You should always be doubly careful editing answers.
Part of the reason why I'm asking
But for what reasons?
Are you going to change actual content or just tidy it up (separate it out into paragraphs, add code formatting, etc)
Because tidying it up is fine, but changing actual content I can't really say without seeing the answer.
user559633
It would be mostly formatting, but that is the longest sentence I've seen for awhile
12:00
Eh yeah edit it to make it more grammarful.
cbg Haha, today is a good day, I want to share that with you guys :-)
user559633
@Raoul neat, thanks!
number of the beast :-D
user559633
12:08
i need to get to 2048
@Tristan why 2048?
user559633
Because 2**11
user559633
Then I'll have to get to 4096 (2**12).
user559633
eventually getting to 2**16, at which point i will screenshot and close my account
@tristan And start again, obviously?
:P
user559633
12:14
edit: sorry, OYC.
In ipython sometimes I get "In [23]" and sometime "In [*]". I can't execute code when it is in the latter form. Can't find out how it gets into that state. Does anyone know ?
user559633
Because the interpreter is doing a thing when in [*] state. @user994572
user559633
now give me candy
so I just have to wait till the interpreter finishes ? I am just doing a few imports.
user559633
something is blocking or doing something
user559633
12:18
show me code or begone with you, back to the land of the no context question askers
3
I expect the code you want to be shown is just import some_module
user559633
More if it's a module that's common or some custom thing
based on tristan's comments, I think what might be happening is: I got some some error
and got the "*" thing.
user559633
e.g. if the entire ipython notebook is import os, that's concerning. if it's import cool_jamz_2k14_weedlord_420, that's different.
then I went to import block.
there also I got the "*".
12:21
I suspect a non-standard-library module that doesn't have the usual if __name__ == "__main__": guard, so a whole slew of code gets executed on import
quotes = izip(dates, df.ix[:,'op'], df.ix[:,'cl'], df.ix[:,'hi'],
df.ix[:,'lo'])
fig, ax = plt.subplots()
candlestick(ax, quotes, width=0.01, colorup='g')
ax.xaxis_date()
ax.autoscale_view()
plt.show()
this is where I started getting the "*" thing.
Is this [*] thing anything like the ... that appears in the normal interactive interpreter? Have you tried pressing enter?
@user994572 You'll need to close your matplotlib plot before you can continue.
user559633
man, however typed those variable names (you) is not very nice. it's pythonic to use the honorific and the surname of the variable.
I blame ten thousand years of mathematics tradition of naming variables with a single letter
E = MC^2, wtf is this. Come on Einstein, would it kill you to document a little better?
4
user559633
12:23
e.g. mr_ax, mrs_fig
@F
sorry @Ffisegydd. Thanks
user559633
yeah you're welcome too, no problem at all to give you an immediate correct answer
@user994572 you could put interactive mode on if you so wish, meaning you'd be able to keep it open and continue typing commands.
@tristan. Thank you :)
@Kevin *E = Δmc²
Sorry, personal bugbear of mine ;)
user559633
12:26
:) i'm just messing with you @user994572. good luck! (also, you can upload ipython notebooks, so if you get super stuck...)
user559633
i'm feeling Super Snark64 today.
Oops, I forgot physics is case sensitive. That's right. c is the speed of light, C is the number of spiders you've unknowingly eaten in the last year.
@Kevin Correct.
There's branches of physics where they've had to start using the Hebrew alphabet as they ran out of Latin and Greek letters.
(actually, it was the delta bit ;))
user559633
12:28
learning russian has made me faster at math, no joke.
The universe divides by C a lot, btw, so I recommend that you all make your mouths as accommodating to spiders as possible, so as to preserve the spacetime fabric.
user559633
also, ugh hebrew letters. terrible for quick scripting and has a very small base of speakers for the related language.
We have a hebrew stack exchange, but they spend all day arguing about documentation ;-)
Did you know you're allowed to eat pork if doing so is necessary to save someone's life? I appreciate the flexibility there.
user559633
I've decided that I'm okay with the fact that I don't respect religious beliefs. I respect the right for people to have them, but the ideas themselves? No.
I respect your respect for peoples right to have religious beliefs and furthermore I respect your lack of respect of religious beliefs themselves.
user559633
12:34
:)
The invisible pink unicorn will smite thee! Just, not in any falsifiable way.
I disagree though.
The next time you feel crummy or just kind of weird, that's the IPU delivering cosmic justice
user559633
Great rules and anecdotes for keeping peasants docile, behaved, and somewhat healthy, but not for me.
12:35
@tristan: still interested in hearing from a Mozilla contact?
He just emailed me (with apologies for the delay)
I have respect for religion (while being totally atheist). I respect that they're pretty much trying to do good in the world and trying to help people. What people do in the name of religion is a completely different thing however...
user559633
@Ffisegydd and I respect that you disagree and if religion does it for you, I'm happy that you've found enjoyment in something enjoyed by many.
user559633
@MartijnPieters very much so! That would be great. And thank you again for helping, mighty kind of you.
Cool, I've located your email address, will send you an intro.
Today is mandatory XML parsing day. I have to extract data from this file, but its format is not well-documented.
user559633
12:39
Thanks again @MartijnPieters.
EnableKeepTogether="False". Yes, I see. I definitely don't want this item to keep together enablingly.
Google suggests it means "Gets or sets the keep area together option." I have a bad feeling that all attributes will have documentation like "Gets or sets the [attribute name goes here] option"
user559633
Next year is my year of adventure, so if I can also feel great about my daily labors, I'd be so happy.
@Ffisegydd I personally don't respect religion for being a religion. I believe that you can do whatever mad frothingness you desire, as long is it adheres to the law of the land. But that's just me ;)
I just think everyone should be excellent to each other.
@IntrepidBrit Bow to the Great Spaghetti Monster, you miserable unbeliever!
user559633
12:54
Church of the ABC (always be cool)
@Raoul Froth away!
user559633
first time running into flask globals: flask.pocoo.org/docs/api/#application-globals
morning
Yo
13:13
cbg
@tristan They are 'global' to the request.
So instead of stuffing extra things on the request, you stuff them on the g object.
a quick question , how do I add a new column of 1's to my current dataframe df without using something like df['new_col']= pd.Series(n obs *[1])
I use it to hold on to the backend API object for the duration of the request, as various parts (including response processors) need access to it.
@Swordy have you tried just df[new_cp'] = 1 ?
@JonClements cbg :-)
13:15
i dint :D
@JonClements thanx :)
hope u've heard of Jonception
@Swordy nope - but I'm sure Jon probably has...
pokerface
We're moving onto Lady Gaga now? Didn't see that curve in the conversation coming...
I'm feeling less productive than usual today. Could it be because of the energy drink I had yesterday? Can you get a hangover from caffeine?
@Kevin no, but if you're not use to lots of caffeine and don't "top up" as it were, then yes, it can make you feel a little sluggish
13:25
I think I enjoyed the caffeine fueled day more. My default neurobiology is unsatisfactory.
I need something that will make me productive and upbeat, without any side effects or dependency or loss of effectiveness in the long term. Get on that, scientists
Like all drugs coffee should be enjoyed in moderation...And by "moderation" I mean you should drink as much as humanly possible.
Scientists have already resolved the issue. If you feel this is unsatisfactory, then please file a ticket when it finishes
Augh why can't I find any quick and easy solutions that will change my life for the better in an instant
(I refer you to the previous link)
My work internet is so fast that I have to constantly be paying attention to download these TV episodes as they download in like 10-15 seconds. God! FirstWorldProblems...
13:33
Torrentz??
No.
I have a subscription to PA so can download the PATV episodes rather than streaming them.
13:53
I've got a feeling that datascience.stackexchange.com might not make it out of beta :(
@Ffisegydd why? it is kind of cool place and its popularity is emerging
@Maxim I agree that it's a cool place but it's on day 97 of the beta and it's only getting 2.6 questions per day.
ok, anyway, people are helping each other there, I've got a good advise there recently
Yes I know that but to become a full site there has to be a critical mass of users and it doesn't look like they're getting that.
I'm not saying it should be shut down, I'd love it if it wasn't, but I've got a feeling it might not make it.
I think it's too close to Cross Validated
@Ffisegydd ha, such place exists? ppl ask about vowpal wabbit in statistics and mathematics and stackoverflow already :D
14:00
I'm hoping to start using it in the near future. Potential client is asking about investigative machine learning techniques
@IntrepidBrit once you've picked up some ML techniques you can come help us with Nidaba, right? :P
@Ffisegydd Absolutely. My undegrad dissertation was on mood sentiment analysis, classification and prediction ;)
Wow really? That's awesome.
Can you model the sentiment of the Scottish people to independence? Or does the drunkenness mess with your model?
@Ffisegydd Yes, I've often wanted to do it, but it's lot of work without a lot of (financial) gain.
And I'm sure you'd have to compensate by blackballing any sources that refer to "bucky" or "white lightning"
I'm thinking of asking a question on DataScience.SE about Nidaba actually.
Need to formulate the idea in my mind first.
14:04
@Ffisegydd It's just that I'm in a prolonged "crunch" time right now. My ToDo list is actually longer than me
I feel like drunkenness would facilitate analysis, since you can more easily identify "I FRIGGIN LOVE INDEPENDENCE!" over "yes, I quite think independence might be beneficial to us, in my opinion"
Hah yeah I understand that all too well.
1
Q: Text categorization: combining different kind of features

elmilleThe problem I am tackling is categorizing short texts into multiple classes. My current approach is to use tf-idf weighted term frequencies and learn a simple linear classifier (logistic regression). This works reasonably well (around 90% macro F-1 on test set, nearly 100% on training set). A big...

this is a damned good question
@Ffisegydd bc that is what I am doing currently :(
At the moment I'm spending most of my spare time reading books/articles/blogs on ML. There's just so much to learn.
@Ffisegydd I mean that Q there:
14:07
@AnttiHaapala If I looked at that 10 years ago I'd have run out the room, blubbering about not wanting to go to University. Instead, I'm going to favourite it and keep an eye on all the answers
we are using sparse features (bag of words) and also dense features...
and our classifier is doing good job
Using VW?
but maybe it is just classifying by the dense features only
yes, logistic online
I wanna learn some scikit-learn as well. Even if we end up using VW for Nidaba in the end.
@Ffisegydd I did nltk,
ended up just waiting for hour for the classifier to run, and the end result was garbage anyhow
we have very very very good dense features, but
since this is about semantics of the post (spam filtering by content too) then surely sparse must have say too
14:13
What is the difference between sparse and dense features? Are dense features features that are in every single instance of your data? For example tweets will always have a length feature and a date feature. And sparse features are features that are only in a few instances? Such as your bag of words when obviously not all words are always going to be used in the same tweet.
Ok cool.
@Ffisegydd we are trying to filter twitter msgs as user post (with sentiment) about topic in english
so the easiest way to drop japanese, russian etc is to scan the distribution of unicode classes in the msgs
the problem is ofc that we just want 1 supervised input which is
(When my project accidentally became a pseudo-spam filter for a bit, I often found that the sparse data was more useful when processing LiveJournal posts)
Are you using the distribution of latin letters to find English too? Or is there some fancy NLTK-or-something-else to guess what language a tweet is?
14:18
"is this message a user post (not a spam/marketing/newslink/whatever) in English"
so we have jsut this 1 input, and want to make a classifier
Actually you probably don't have enough letters to get an accurate frequency distribution...
@Ffisegydd true, msot tweets are impossible to classify with all the trigram etc approaches
bc too little content
and also msot of the existing tables also assume nonequal distribution of probability...
I was thinking more of looking at the frequency distribution of a, b, c, etc and comparing it to the frequency distributions of languages to try to estimate the language. For example Welsh has a lot more ys and ws than English I'd imagine.
@Ffisegydd we do not need to separate english about welsh really
I don't like cordova :|
14:20
And I can imagine that Twitter would be a pain in the arse because people will shorten words/terms to nonstandard english just to fit into a tweet
I don't like crows.
welsh is not as much a problem as japanese, russian, chinese
@IntrepidBrit true.
what we could try ofc is to
calculate the sparse features into a result and feed the prediction as a value for the dense...
nobody likes crows, they're not very nice
14:23
@Ffisegydd our code does so that it looks each letter in a tweet from unicodedata.category
if it is L, N, M, then we ask the name for the letter, take the first word
which is like LATIN, CYRILLIC, CJK, DIGIT, COMBINING etc :P
and calculate their frequencies
Would that be entirely accurate? I would be tempted to look at entire words on an unigram by unigram basis
You might have someone using a loan word, but the majority of the tweet is in English, but a long word or one that has a fair few unicode characters in it could throw off the detection
maybe...
@IntrepidBrit actually, the tweets have already been filtered by content
so they must have latin too
Well, take this tweet for example:
"Come to Vatnajökulsþjóðgarður!"
highly theoretical in this case
user559633
That's a lovely bar in Iceland
14:29
Damnit I wanna go to Iceland.
probably would be classified as nonwanted as no other evidence would exist...
@IntrepidBrit actually...
user559633
Iceland is pretty great. If I wanted to write stackless py with a bunch of MIT grads, that would be my first pick of locations
Could you look at the percentage of words that contain "strange" letters rather than the percentage of letters themselves? That way if you only have 1 or 2 that do it could let them pass through.
@tristan definitely.
A pity I didn't stick with my Icelandic employers back in the day ;)
user559633
a great place to go if you're single and not icelandic.
14:32
@IntrepidBrit actually, it would show 2 combining and 27 latin...
which might not be much yet
cbg again
@JonClements Re-cabbage
DSM
DSM
Cabbage to Jon!
Cabbage to DSM!
@AnttiHaapala Sorry mate, you've lost me there
14:34
@IntrepidBrit the simple algorithm we're using
all in all we just try to reduce spam
bc we need to crowdsource the sentiment analysis anyhow
@AnttiHaapala Pfft. Just do what we did. Enforce daily staff quotas across the company to do it xD
@Antti yeah seen that before, it's where I found out about VW from
Crowdsourcing it is!
actually
we want the "manager" to do something useful, could as well be clicking the mouse
actually our spam filter works already about on par with the crowdsourcing platforms.
so I said co should change biz into "clowd"sourcing.
:P
14:38
Have you ever heard of Rainbow for classification?
I wonder if the dude is still maintaining it
@AnttiHaapala What stemmer are you using (if any)?
snowball
as i said I have to run some tests if teh bags of words and ngrams actually have any effect at all :P
@AnttiHaapala ^_^
bu tI am not now at work so not now :D
Lucky for some ;)

« first day (1403 days earlier)      last day (3770 days later) »