« first day (2978 days earlier)      last day (1963 days later) » 
00:00 - 21:0021:00 - 00:00

12:02 AM
evening rbrb
 
12:59 AM
1
Q: Speeding up a dash visualization applet

pyeR_bizI need to visualize sensor data with below features of the data: Per Second data granularity (around 12-25 days of data), timestamp index Around 20 channels with different scales Around 50 datasets with above features, each will have its own plot The plot should be zoomable to an area of the...

 
 
2 hours later…
2:39 AM
@AaronHall: It's not just that question, I'm an advanced Python user but like many others I didn't know the full benefits of slots; your answer here reads at a quick glance "faster and less memory", but the "faster" part is only true for attribute access, not object creation. It only takes adding eight words to your answer to make that basic but important fact explicit. — smci 15 mins ago
 
"I hear you but I don't think saying something doesn't do a thing adds much to what I'm already saying..."
@WayneWerner I want to see Andy Kaufman do that talk.
 
@AaronHall lol. I'd pay money for that
 
reveals he was Andy Kaufman all along!!!
 
wim
3:03 AM
slots is a kinda dumb feature in practice
 
@wayne
 
@nader
 
@wim :P
 
wim
you can't even really use it in one of the most obvious use case (dataclasses) because when you want to provide a field default there is a conflict in the namespace
 
i dnt think 20 h will enough for leaning thing i think it;s only the basics
 
3:05 AM
did you watch the video?
 
yes i watch it 2 years ago and yesterday too again
and also i read the book
 
Have you tried practicing for 20h?
 
yes i try with django , i made it in 15 h
but i dn't reach all topics
 
So would you say that you learned Django?
Not that you mastered Django
but learned
 
you can say that yes
 
3:08 AM
watches The Matrix for 20 hours - knows Kung Fu
 
LOOL XD
 
now i am stuck with JavaScript i think it needs 20 year to learn it XD
 
@NaderElsayed see if you can get away with just learning the good parts, or maybe Typescript...
 
3:20 AM
the problem i face is fully understand the idea of functional programming ,
i don't know why i jumbled between it and OOP
 
people who fall for the "learn XYZ in <insert unreasonably short span of time here>" end up not understanding 80% of what they are forced to breeze through, and land up here on Stack overflow
 
Most people don't take a measured approach to learning, though (part of the point of that video) - it's more just wild flailing
which is definitely why most people end out here instead
 
 
2 hours later…
5:42 AM
recbg
 
cabbage
 
wim
am I remembering wrong or did you used to be able to check the time of your submission on the hover event or something
 
hi any one ther
who could help me in webscraping
 
6:14 AM
Which website?
 
wim
advent of numpy day 11 spoiler
 
6:33 AM
hello everyone
 
cbg
 
6:45 AM
@wim you don't mean this?
 
wim
7:02 AM
I thought there was a place you could see last star time for other users too
 
7:22 AM
you have to have star-based ordering
 
There has to be a faster way.
 
yup
 
I finished but I went all O(n^3) with numba. I'm debating if I should try to figure it out or sleep
my hamster wheel is turning so I guess I'll be up a while
 
wim
8:02 AM
@AndrasDeak ah, that was it. the hover only works when it's the otherwise useless star ordering
 
yup
 
 
1 hour later…
9:33 AM
variable variables dupe in disguise stackoverflow.com/questions/53720481/…
 
10:07 AM
\o
 
@SurajRao hey. Don't often see you here :)
 
@AndrasDeak you should probably spend a bit more time answering some questions and get a hammer for yourself :)
 
Meh. I'm too lazy to dupe hunt
I just got 20k
I'd already be a long-time gold badger if half my rep wasn't from MATLAB :)
 
10:38 AM
just a couple more documentation-length canonicals and you should be good to go :D
 
> every problem has a solution that completes in at most 15 seconds on ten-year-old hardware.
having a very hard time getting there for AoC 11.2
 
10 yo hardware? Welp, my mac is certainly getting there.
 
 
3 hours later…
1:44 PM
cbg
 
cbg
 
2:08 PM
recbg
I came up with an I-think-improved solution but it's still too slow :(
 
I had something that was somewhat worse than O(N^3). For the first minute or so it produced a lot of diagnostic info, then went quiet. I thought "while I'm waiting, I may as well plug in the last candidate region it printed". It turned out to be the right answer :>
 
(-: way to nail it down. I'm still thinking of how to memoize or something to reduce big-O
 
Afterwards I came up with a recursive implementation for total_region_power(x,y, size) that memoizes well, so now I'm at least at O(N^3) instead of beyond it
I notice that for both the example test cases, the region size of the solution is considerably smaller than the grid size. 16 is approximately the square root of 300. I wonder if this means anything.
 
yeah, I was wondering about the same thing
 
@AndrasDeak as_strided was good thinking.
 
2:17 PM
Thanks, though I'm not sure it gave too much of a benefit
at least it's worth street cred
 
I want to say "the law of large numbers predicts that the total power of a large region would be very close to the average power of a cell, times the number of cells in the grid" but I don't know if that's how the law of large numbers works
The average power of a cell is 0, so maybe large regions tend to sum up to zero. Outliers would only be found in small regions
Even if this were true I'm not sure how I'd take advantage of it in my program. if size > 20: break? Seems a little brittle.
 
For very large regions it would work that way. Mostly because the total power of a region is the sum of the cells in the grid which is equal to the mean of the cells times the number of cells ;)
the question is whether the average power is indeed 0
there's ~discrete math~ going on in generating the grid so anything can happen
 
The last two steps of power level determination is "take this one digit, and subtract five from it". If that digit is well-distributed from 0 to 9, then the result is well-distributed from -5 to 4. (Big "if", I know)
Hmm, but then the average value of a cell would be -0.5, not 0... So that's even worse for big regions.
 
yup
 
I think the admin wants the values to be evenly distributed, since he's basically implemented a hash algorithm and nice hashes are evenly distributed
My new improved O(N^3) approach has been going for ten minutes. I think I did something wrong.
300^3 is only like twenty million, even my potato powered laptop should be able to churn through that in a reasonable amount of time
Maybe it's because I'm caching twenty million region power levels? How slow is a page swap these days...
Ah, it would probably help if I saved my changes -_-
Complete in 51 seconds. Good enough for government work.
 
2:36 PM
Cbg
(Memoisation is a good idea)
I also did another thing that seemed cool, but is probably still dog-slow compared to scipy
 
@FélixGagnon-Grenier Well having British humor sensibilities, I wasn't sure...
Apparently Intel is hawking a bespoke Python dist that's faster than CPython?
cbg, btw
 
Why is that page using "*" everywhere it should be using "â„¢"? Are they afraid of encoding issues?
 
@RobertGrant looping through with scipy is also slow.
 
> Versions: Python* 2.7 and 3.6
the world really does turn at a slower pace than expected
 
2:53 PM
Cabbage
 
cbg
 
@Arne Some call it legacy support... others call it a bad decision
 
Writing new software today with python2 support sounds like a nightmare to me
 
*nightware
 
3:10 PM
Introducing Python Nights, the 18+ spinoff of Python
If you thought we couldn't incorporate a 1980s Miami murder mystery into a programming language, you thought wrong
 
ha, my improved version runs 5x longer than the original
guess I don't have to commit that one
 
3:32 PM
Hi folks, was wondering if anyone could help with converting an input straight into a list. Trying this splits the input into a list of characters: a = list(input('enter string: '))
 
That's one reasonable way to convert a string to a list. If you need something else you'll have to be specific.
 
Sorry, so if I were to enter keyboard, I'd want a =['keyboard']
not a=['k','e','y','b','o','a','r','d']
 
you always want a one-element list with the full string inside?
 
a = [inp]
 
3:36 PM
ah
 
though this is a surprising thing to want ;)
 
yeah, I have a scenario where a list of multiple indices is created, so I just want to keep it the same type
thanks @AndrasDeak
 
no problem
 
Hey, I am very much overthinking this but do you do os.path.split(path)[-1] or [1] ? I want to keep consistent and I'm not sure what to use
My logic for using -1 is that split could change and split into more than 2 components that's why I use -1
I was just wondering if somebody else overthought this so much^^?
 
I'm inclined to use 1
 
3:42 PM
why?
 
I find it unlikely that os.path.split will change in the foreseeable future
After the backwards compatibility break between 2 and 3, the Python dev team is really really unwilling to break backwards compatibility unless there is an extremely good reason
 
I see, I guess than 1 does make more sense
 
So taking "but what if it changes?" out of the equation, it just comes down to a question of "when selecting elements from a tuple of length two, what is the idiomatic way to get the second one?" which is a matter of opinion but I happen to like [1]
 
I agree with 1 since it is one less char, smaller is always better. But thinking about a path with possibly n splits and I want the last one -1 would make more sense.
I found it odd at first anyway that split splits twoways
Btw does somebody know a tool that measures code quality automatically? Some things like num_lines per function and file, num warnings per function/file etc
 
for the record smaller is not always better
 
3:54 PM
smaller as long as it stays the same clarity is always better
 
shorter isn't always clearer
 
yes
 
take [val2,val1][flag=='potato'] vs val1 if flag=='potato' else val2 for instance :P
@Hakaishin hopefully someone will write a WTFs per minute neural network one day
 
:D
I mean it is not for anybody, just for myself to see over what code I could have another look
I just saw old code, where I created two folders, then os.listdired that folder to then afterwards remove the two folders that I just created :D :facepalm:
 
4:14 PM
Nvm what I did before was right, I just forgot why it made sense to do it this way. Atleast it is documented better now :D
 
4:29 PM
It'll be interesting to see if my part 2 for today actually finishes. I know there are some better algorithms for it, I'm just not aware of what they are and I don't have the time right now to derive and implement one lol
 
I improved my original after which it ran for 5x longer
 
4:53 PM
lol
 
Is that an "improved" definition of "improved"? Or is your algorithm making sport of you?
 
5:04 PM
probably just jealousy for my trying to improve upon it
 
5:18 PM
How many different (but not redundant) Python linters are there? flake8 for one, right?
mypy, pydocstyle, cpychecker (extension checker), mccabe (complexity)...
huge list here (but lots of redundant stuff, it seems) github.com/vintasoftware/python-linters-and-code-analysis
 
Hot damn boys! I finally got day 7 p2. Feast your eyes on this magnificent implementation Not really magnificent but I’m pretty jacked about it.
 
@W.Dodge gratz. I felt pretty good when I finished slogging through it as well.
 
words = re.split(" ", instruction) seems like not the most concise way to split a string on whitespace
 
5:33 PM
rather than words = instruction.split()?
 
I used re in my own implementation, but I used re.match
 
@piRSquared tyty
 
i.e. a,b = re.match("Step (.) must be finished before step (.) can begin.", line).groups()
Something like that. The source is on my other computer.
 
Oh, so (.) will just pull standalone uppercase letters?
 
single character, any character
will match ""Step : must be finished before step & can begin."
 
5:43 PM
But re.split(“ “, instruction) is concise in terms of splitting on whitespace, or is there something shorter for that?
 
instruction.split(" ")
 
Ahh.. ok, and just noticed that piR just said that too, oops
 
And instruction.split() if you don't mind splitting on newline, tab, etc in addition to literal space
If you're using re and your pattern doesn't have any actual special characters in it, that's a good time to reevaluate whether you actually need re
 
Well that makes a lot of sense, yup
I had a tough time on the polymer problem with regex as well. Could not figure how to match a letter followed by itself, where the case between the two letters is always inverse. I ended up using aA = re.compile(r'aA|bB|cC|dD|eE|fF|gG|hH|Ii|jJ|kK|lL|mM……. and its opposite regex expression. It’s terrible, I know.
 
That's not something regex is good at. You'd probably just match a repeated character (case insensitive) and then use code to check if the cases are different
 
5:54 PM
morning cabbage
 
that would have been smarter, for sure
 
6:37 PM
Day11 view spoiler Much Improved Big-O
 
@W.Dodge generally, regex has no memory of what was previously matched. The polymer problem is similar to checking a string for matching parens which isn't solveable with regex alone.
 
wim
@piRSquared github just says "Sorry, something went wrong." loading those.
which makes me curious, where does the code actually execute when loading a .ipynb file on github
 
wim
blocked
 
hmm, ok, one sec
 
6:53 PM
@Code-Apprentice Yeah, my fastest solution for the second polymer problem was almost all re stuff with a while loop that broke whenever the length of the string did not change with subsequent iterations.
 
@W.Dodge how long does it take to run your solutions?
 
Too long for some The times for everything are in the comments on my github AoC repository. The solutions for day seven were fast
 
Hi guys I was told I need premium proxies for web scraping a certain site
I have the code that requires a proxy as an argument where can I get this proxy?
 
@W.Dodge wow, you commented your solutions pretty thoroughly. That's pretty impressive.
 
wim
7:02 PM
numba, interesting
should compare it with pypy
 
my original solution to the polymer problem took much, much longer than 8 seconds. I had to completely rethink my approach.
@Pherdindy who told you that?
 
the person who I paid to program my web scraping thing
The thing is the website is a bit sensitive to webscraping
And beautiful soup doesn't work
    proxies = []
    with open("proxies.csv","r") as r:
        reader = csv.reader(r)
        for line in reader:
            proxies.append(line[0])


    for i in range(1,page_number+1):
        link = 'https://www.lazada.com.ph/catalog/?_keyori=ss&ajax=true&from=input&page='+str(i)+'&q='+str(search_keyword)+'&spm=a2o4l.home.search.go.239e6ef06RRqVD'
        proxy = random.choice(proxies)

        req = requests.get(link,headers=headers,proxies={"https":proxy})
 
@Code-Apprentice Yeah, my slow solution took eight seconds and the fast one I just posted took 0.09292 seconds on actual
 
Imagine paying someone to write '&q='+str(search_keyword)... ouch
 
It was only $20
Worth every penny
It gets the job done
I just need to know about proxies
She said I needed to get a premium proxy to get this script to work
 
7:09 PM
@wim not as if numba reduces complexity but...
 
The one in the csv file contains this: 104.238.146.146:8118
But it does not work anymore
 
Googling "web proxy" seems like a reasonable way to find proxies.
 
Is it safe though? I am quite noob in this
Any known proxies?
:/ i guess i'll just ask her again to give me a proxy lol
 
quora.com/… looks relevant
 
@W.Dodge that's a great improvement. Is that after changing implementations?
 
7:22 PM
yes, my slow and fast solutions where much different
 
Ugh, 40 is spelled forty. I'm not sure I've ever noticed that.
 
Does anyone know a good dupe target for "why do my bytearrays show a bunch of ASCII characters where I was expecting hex escapes"? I don't see anything on the sopython list.
 
7:38 PM
What happens if I scrape
without a proxy and get blacklisted
 
Hold on, isn't the reason you need a proxy in the first place that the website's owner doesn't want you to scrape to begin with?
 
cbg
 
cbg
 
I didn't know the girl just told me
To get a proxy
But I removed the proxy
And scraped using my https
i mean my
IP whatever
I did it once pls don't save me from getting jailed
save*
 
Is anyone familiar with Flask-user or user permission groups in general?
 
7:40 PM
I web scraped world bank
Website
 
I'm merely talking about the possible ethical questions raised by scraping a site that doesn't want you to scrape it
 
Just kidding but yeah you're right
 
don't they provide an API by any chance?
 
Time I get proxies
Nah they don't
This is an online selling platform
They don't want to give a lot of information to people
 
What? There are a ton of world bank datasets; you just have to look for them
unless that's not the point of your scraping at all, in which case... carry on
 
7:44 PM
I was just kidding about the bank
 
FWIW their ToS doesn't seem to mention scraping lazada.com.ph/terms-conditions/…
 
Yeah that's good
But it's still something they would discourage
So i'll just use a proxy
I'm just concerned about some proxy sites
 
If you're asking "will I go to jail if I scrape without permission and the owner finds out?", that seems very unlikely to me.
 
They don't look legit at all lol
If I put my card in I don't know what will happen
 
well either you'll be scammed or you won't
 
7:48 PM
Can suggest any good proxies?
 
I'd already have done if I could
 
Ok lol might ask the girl who coded the thing
 
8:05 PM
@AndrasDeak not sure if you saw my link I successfully reduced the time complexity of Day11. Maybe you saw it already (-: but I was pretty excited about it and thought you would be one to appreciate it.
 
Thanks, I saw it, but didn't get the gist of it after a quick glance so I closed it. It reminded me of something I've heard mentioned from a reddit thread or something, which I couldn't make much sense of in the time frame available to me
unfortunately today is really busy, it's past 9 PM and I'm still grading mid-term resits...
 
This explains it a tad view spoiler
 
Yeah, I gathered as much. I just can't make sense of that in my head, partly due to lack of experience with <spoilery mathematical operations>
I can't easily see why the spoiler does what we need it to do, at which point I've not much to look at in that kind of approach
 
elf magic
 
working on diagram
 
8:11 PM
I appreciate it, but you don't have to :) as I said my resources are all allocated for the time being
 
I'll leave you to it.
 
Day 700: Still searching for the confidence to attempt AoC puzzles
 
jump right in, there's nothing you can lose :)
you can even play without joining any leaderboards, so nobody will see how you're doing
the only harm that can come is to one's own ego :P
 
@coldspeed WT-Yam! I've spent so much time convincing complete noobs to try it and someone like you lacks the confidence? Have the courage to look foolish and jump in. Do it now!
 
joins with a whole cabbage um, can you? I thought the second star is hidden until you complete the first one
 
8:20 PM
Yes, it is hidden. You need to get the first, for which you need to join the game. But you don't need to join a leaderboard.
 
@AndrasDeak yes, that's likely the biggest factor keeping me from attempting :P
 
oh well ;)
 
anyway, it's almost Christmas and I'm in India with nothing to do... might take a stab
 
It's fun and you learn
 
@AndrasDeak ah, I meant including the global leaderboard
 
8:21 PM
@coldspeed watch your memory need, and if it runs for more than half a minute odds are you have a bug/you need to redo it entirely
@EKons well that's only public for the first 2x100 contenders or so, right? Beyond that you can't see people
 
okay, will do. Can't wait to get involved in AoC discussions one of these days!
 
Stuff like this answer is what makes SE really awesome. A NASA scientist that worked on Voyager stopping by to answer questions.
 
Actually, India time isn't that bad for competing for global scores.
 
@AndrasDeak 1x100, actually :P
 
Ah, right. 2x100 places get scores each day, but the global list is only 100 long
 
8:25 PM
there are per-day leaderboards, yeah
2x100 is for 2 stars and 1 star in this order
 
considering that the puzzles go live at 6 AM for me I find the global leaderboard largely irrelevant
 
yeah, I think it's pretty flawed too
 
well, there's no uniquely fair solution
 
maybe change it a bit to count the time separately?
I mean, first criterion is number of stars, tie-breaker is earliest time
 
Would that change anything?
 
8:29 PM
well, if you got 172 points for 3 stars and somebody else got 35 points for 35 stars, wouldn't it be unfair?
not that my change makes it totally fair
 
surely the leaderboard will be determined among people all having all the stars
Or maybe I'm missing your point. As I've just explained I'm a bit pre-occupied in terms of cognitive capacity
 
actually, I don't think that having 50 stars is at all an advantage in front of a very quick person...
 
Points for the leader board are determined by the who answered first relative to others on the leader board. Someone answering all questions on the last day will not have earned many points because most others will have answered before then. If you want to compete with other top competitors then you must be fast. I'm not in this to be fast. I think the puzzles are fun.
 
yeah, and GMT-5 midnight is definitely either sleepy time or school time, not PC time for me
 
Which is true for many people. Which is why they do it when they can and accumulate stars
 
8:36 PM
tbf, I've actually stopped participating in AoC 2018, there is a reason why
many of the inputs I've got with the puzzles make them unsolvable
 
sunspot activity
 
that got an audible laughter from me and made it awkward while those around me stared
 
@EKons are you sure?
 
yes, because I've read over the code maybe 10-15 times and found a few flaws and fixed them and then yet again and still no fruit
 
there's only one puzzle that had buggy submission checks, one in 4 years of AoC
@EKons you should try someone else's code with your input (and be surprised ;)
 
8:39 PM
spoil myself? never! :D
 
it justs seems like you're blaming the game for your own mistakes :P
 
well... maybe
tbf, I'm totally unsure at this point
 
note that I find "I stopped because I got stuck" is a perfectly acceptable reason to me
 
a few others have also experienced such issues though
 
This year's day 6 had a bug where 33% of users were stuck with correct solutions. It was fixed later and day 6 scores are ignored. This is the first such incident in the history of AoC that I know of
so I very strongly suspect that you have bugs or misunderstandings (test inputs usually avoid edge cases on purpose)
 
8:42 PM
yeah, maybe, I've been also having some lack of confidence that the description is clear enough (I've heard that day 9 didn't even say anything at first...)
 
by the time I got there it said everything ;)
 
yeah me too
so, for example, I can't seem to get the second star of day 1
and yes, I've accounted for looping back to the beginning of the array etc.
 
Share your approach! Sometimes a second set of eyes is the best thing
 
if I'm lucky and repl.it saved it... ;-)
(also, repl.it's STDIN is weird)
ah, doesn't look like it :-(
 
not having a .py file on your hard drive is the first mistake :P
 
8:50 PM
repl.it doesn't save your previous revisions anymore :P
oh wait...
woah, it has saved like ~50 revisions
however, they're all too late
you know what? maybe a fresh start helps :P
 
it often does ;)
 
so, I've put the input back into the I file
"inp = open('I').read()"... ah, the feel of a fresh start :P
 
don't forget to .strip() :P
 
it doesn't have anything to strip there though, only .split() (er, .split('\n'), I don't wanna deal with that)
and then a nice map(int,...)
it's day 1 part 2
 
00:00 - 21:0021:00 - 00:00

« first day (2978 days earlier)      last day (1963 days later) »