« first day (3342 days earlier)      last day (1607 days later) » 
00:00 - 20:0020:00 - 00:00

8:03 PM
I'm coming in part way through the discussion, but rather than instantiating inside __init__ can't you use setUp? or, possibly def __init__():
self.setUp = super(self, OutliersTest).setup
 
@toonarmycaptain cant use init there
 
define "there". Are you saying you can't call __init__ from inside setUp? Why would you want to?
 
mistake
 
What's a mistake?
 
previous message
 
8:06 PM
Anyway. The answer to "how can I have a variable number of [whatever]s?" is almost always to create a list of whatevers.
 
windows is hard coded
the problem is in the function where they are used
 
self.windows = [[3,10,20,40,80], [4, 8, 23, 42], [1,2,3,4]]
 
no i meant liked [2,5,7] or [2,5,6,8] or [2,3,4,6,8,10,12]
 
self.windows = [[2,5,7], [2,5,6,8], [2,3,4,6,8,10,12]]
 
@Permian I meant define setUp in the Base class, then inherit via regular, or by explicitly assigning it as an attribute/method.
 
8:08 PM
@toonarmycaptain i dont what the syntax is
 
Or perhaps you're saying "how can I modify the definition of outliers_IQRm so that it can work on a windows object of any size?"
 
yes
list of variable size
 
def __init__(self, *args, **kwargs):
    super(OutliersTest, self).__init__(*args, **kwargs)
    self.setUp = super(OutliersTest, self).setUp
 
@toonarmycaptain how does this help?
 
Use a for loop.
 
8:10 PM
@Kevin dpaste.com/3NR2JEQ this bit is hard coded though
 
#Whenever you have code like

doThing(0)
doOtherThing(0)
doThing(1)
doOtherThing(1)
doThing(2)
doOtherThing(2)

#... It can more concisely be expressed as

for i in range(2):
    doThing(i)
    doOtherThing(i)

#You can even do the things a variable number of times by changing the `2` to a variable.
 
@Kevin yes but dothing and dootherthing are hardcoded
 
@Permian I'm not sure...I misread and missed that you're referring to an init from within it's own definition? I'm not sure how or why?
 
If you're thinking "but I have five variables Qli through Qlv, how can I dynamically create each one?", use a list.
 
@Kevin yes hmmm
 
8:12 PM
#instead of doing
thing0 = doThing(0)
thing1 = doThing(1)
thing2 = doThing(2)

#do
things = []
for i in range(3):
    things.append(doThing(i))
I see this kind of "dynamic variable" antipattern a lot. Naming them with roman numerals is a new one, though.
 
i know
 
He doesn't even need a list for Qli through Qlv, they're only used locally, they can just be generated in each loop.
 
Oh, in that case don't bother with a list.
 
# or
[doThing(i) for i in range(3)]
 
sometimes i feel so dim
olrs_per_win[4] these are generated dynamically
 
8:15 PM
@Kevin It worked!
 
for j in range(5):
    for i in range(windows[4],self.file.shape[0]):
        Q1i = self.file['Price'][i-windows[j]:i].quantile(0.25)
        Q3i = self.file['Price'][i-windows[j]:i].quantile(0.75)
        IQRi = Q3i - Q1i
        if ( self.file['Price'][i] > Q3i+1.5*IQRi or self.file['Price'][i] < Q1i-1.5*IQRi):
            olrs_per_win[j].append(i)
Reduce 5 repeated steps to 1 for loop over range(5)
 
i tthink you can go further
 
Probably, but that's your job.
 
i know
im thinking
 
@KieranMoynihan and then get rid of the i suffix
 
8:18 PM
@Code-Apprentice Of course, I was just too lazy to do it for a snippet I'm not using ;)
 
;-)
"exercise for the reader"
 
@KieranMoynihan should the for loops be the other way round
 
Yes.
Drank too much bread at lunch, my bad.
Actually, I don't think it matters.
Either way, you iterate range(windows[4], self.file.shape[0]) for each of the 5 olrs_per_win.
 
better
its this that is holding it back
olrs_per_win = [[],[],[],[],[]]
 
@KieranMoynihan Mmm liquid bread.
 
8:25 PM
olrs_per_win = [[] for _ in range(windows)]...... beautiful
doesnt work haha
 
range(len(windows)), perhaps
 
olrs_per_win = [[] for _ in windows]
in the console
but now i have for i in range(windows[4],self.file.shape[0]):

IndexError: tuple index out of range
works
 
no this is non obvious
as the argument is variable lenght which causes the problem
 
If you're saying "roganjosh, don't accuse my problem of being a dupe, it's actually pretty difficult", he wasn't talking about your problem.
 
8:37 PM
oops
 
I earned a hat!
 
FWIW, I also didn't know the behaviour of pandas in this particular instance until that question came up, which is why I'd kinda like to not have it diluted by that question getting answered instead of closed :)
 
for i in range(max(windows),self.file.shape[0]):

TypeError: 'list' object cannot be interpreted as an integer
 
<sigh> too late.
 
@roganjosh hammered
 
8:42 PM
@AndrasDeak thanks :)
 
@roganjosh Just blame the 70s
 
@Permian have you been asking questions for the past 5 hours?
 
@AndrasDeak not quite
 
OK, I just saw you start there at 5 PM my time and it's almost 10 now
 
im sorry
 
8:46 PM
@KieranMoynihan <grabs the pitchfork>
 
@roganjosh yeah, it's very weird
wonder why datetime does that...probably just a crap heuristic
 
MaxU gives some pointers, but the logic from that snippet alone doesn't quite explain the logic, I guess you'd have to dig from there
 
yeah, passes the blame onto POSIX and ISO C
this week is not a digging week for me, alas
 
args are awful
 
Nor for me, I'm being battered by issues at work. A lot of what I've built with spreadsheets at the foundation of my system are falling apart quite drastically.
 
8:52 PM
So some stuff is going to break in 2069... What else is new ;/
 
Of course, I'd never choose to build something this way. I thought I was onto a winner when I could get the data straight from the central source of the company that had been parsing them for years. Only this week, I discovered their parsers were even more broken than the ones I'd built but they limped on and hid tonnes of issues. I have never hated Excel more
 
9:09 PM
@roganjosh "panda"
 
Oh wow, how did I miss that? We should tell them they need at least 3 rooms of infinite pandas to get this done :P
"It was the best of datetimes, it was the blurst of datetimes"
 
Infinite pandas with infinite typewriters...
 
Well, we'd probably give them something more sophisticated than typewriters. As YouTube adverts keep extolling Grammarly, I suppose we could give them that. It's that yamming good, according to the un-skippable bits of the advert, probably a single panda could do it these days
 
9:27 PM
I must be seriously misunderstanding something about scope...
 
EE([3, 10, 20, 40, 80],) what does this mean?
whats the EE for?
 
@KieranMoynihan I was assuming that you had a question coming. Or was it a comment on what I was saying earlier?
 
@roganjosh Was contemplating how much of an idiot I was going to look like ;)
Given the following function:
def execute():
    def func():
        print old_list
    ...
    old_list = []
    ...
    func()
Where '...' represents some other unrelated code, func() should be able to see old_list, unless everything I know is wrong. Right?
 
NameError: free variable 'some_list' referenced before assignment in enclosing scope
That's from my own example. It seems you need to define old_list before func
 
>>> def func():
...  def func2():
...   print x
...  x = []
...  func2()
...
>>> func()
[]
Not by my tests? :(
 
9:37 PM
Even:
def execute():

    def func():
        print(some_list)
    func()
    some_list = 2

a = execute()
Fails for me with the same error. So it's not about mutability
 
Well, of course if you assign some_list = 2 after calling func() it won't know what some_list is.
 
Ah, hold on, you're right, I misread
I can't repeat the issue in any case (for that matter, I don't really know what it is)
print old_list are you using Python 2?
 
he always is
 
yes, but I can't imagine how that would be causing the issue.
 
python 2 has no nonlocal statement so I bet there might be scoping differences other than list comps
 
9:42 PM
The more interesting thing is that the inner function can see other variables that were assigned within the outer function, but not this one specifically.
 
you can try looking at dis.dis of your function
 
wim
[mcve]
 
@Permian variables and function names must be defined before they are used. By itself, EE doesn't stand for anything magical here. It is a function or a class that was defined earlier in your code somewhere (or in a module that you import).
59
Q: MCVE shortcut link in chat

Code-ApprenticeMany of the shortcut links to help topics work in chat, including [ask], [answer], [main], and [meta]. This is a very useful feature. However, [mcve] does not work. Can we get the [mcve] shortcut in chat? While you are at it, enable the help/* links as well.

 
list(set(olrs_per_win[0]) & set(olrs_per_win[1]) & set(olrs_per_win[2]) & set(olrs_per_win[3]) & set(olrs_per_win[4]))
i think i can avoid the other problems by fixing this
somehow this can be written as a list comprehension type thing
 
wim
@Code-Apprentice [mcve] this thing doesn't work on editing old message... ;)
 
9:45 PM
nm, so it breaks on edits
 
@AndrasDeak According to dis.dis the variable is local (LOAD_FAST), meaning that var is somehow not available to the scope of the function.
 
wim
Is that correct? -->
Unfortunately though, this requires installation for it to work. Which means any users without the extension won't see it (and these users are probably the majority) — Zoe May 22 '18 at 13:57
if so, that's the dumbest script ever.
 
That might be why nonlocal was started, so that functions can see scopes between local and global. But I'm just guessing, you'd have to give an MCVE that wim asked for.
@wim I think someone here made an [mcve] poster userscript
 
Have a question regarding an API for a probably common design pattern
have some object which, in a separate thread, just adds data to some queue
And then in the main thread, I want to just dump all the data currently in the queue
 
@AndrasDeak Yeah I'll slowly deconstruct down towards an MCVE, and I might solve it anyhow on the way.
 
9:49 PM
@wim Glancing at the code, I'm 99% sure it's true
 
in other words, provide a polling mechanism for what is otherwise a stream of data
 
Mar 31 '18 at 22:33, by Aran-Fey
Behold my userscript that enables [mcve] in chat
@KieranMoynihan that's how it usually works
 
Does anyone know how to compile a C file that includes the Python.h header file on a Mac (Python 3)?
 
I have this implemented fine but...is there a name for this? Is there a typical interface? Idk what to call the class or the polling method.
 
wim
Don't come in here with your non-reproducible problems! That's what main site is for.
 
9:51 PM
lol
 
@KieranMoynihan Can you clarify your confusion? func does see old_list just fine.
 
cbg
Hey, would you look at that ... I got a nice little hat on my profile picture.
Well, isn't that just great ...
 
@alkasm "Hunter-Gatherer"? :P Tbh, I don't think I've seen a single class that operates this way (IIUC) with one thread dumping into a queue and the same class in another thread processing that queue
 
@alkasm in how far is your setup different from a plain Queue with Queue.empty() to poll and Queue.get() to fetch what's available?
 
Does empty() empty the queue? I thought it was an is_empty() method
Oh I see what you meant, read that wrong
It is quite close miyagi, with the exception that I want to just dump all; in other words, provide a method like dump() that does exactly that: while not empty, append q.get() to some list that is returned.
 
10:03 PM
sounds like a buffer, then.
 
@MisterMiyagi I have a similar case where the equivalent of func cannot see the equivalent of old_list. I'm trying to simplify to an mcve.
 
Buffer is probably better than what I was using! I'll poke around those methods
 
@KieranMoynihan does your func write to old_list by any chance?
 
The only thing I could think to name it was listener, but listeners are...well, not what im doing exactly
 
@MisterMiyagi Yes, though after it attempts to read from it.
 
10:07 PM
@alkasm To a queue you can have "publisher" and "subscriber" but that probably has connotations of bigger systems where there are different topics, so maybe "consumer"
 
Yeah, its sort of like a pub/sub, sort of like observer, but a little diff from either. It is a consumer for sure, in the same way that a buffer is a consumer. I think buffer fits the bill the best.
It's a stream buffer. It exists to hold the data until it gets dumped.
boom
ty chat
 
@KieranMoynihan this says you can see but not rebind nonlocals. Note that you didn't even give us an error message (have you?)
 
@AndrasDeak Well that would explain it. No, I didn't provide you with an error message explicitly, though the relevant error message was given by Roganjosh.
Actually that's my mistake, it wasn't quite the same error.
It was UnboundLocalError: local variable 'old_radarsat_2_files' referenced before assignment
 
@roganjosh FYI, the basic reason for my use-case is that I have live streaming data. But if people want to plot that live data, most python plotting libs update on some interval. So I need some way to provide "all the data since the last interval". Hence, enqueueing the data in another thread from the live stream, and dumping it all from one method call.
 
Clever me didn't pay enough attention to the UnboundLocalError part of the error message ;)
Which would have solved my problems to begin with.
 
10:16 PM
@alkasm That makes a lot of sense. However, the two aspects (at least in my head; pushing data and consuming it) are sufficiently different to be different classes. On the flip side, I've not done this particular thing (taking live data in batches) and maybe I'm thinking too much on the side of decoupling with the queue being its own thing e.g. redis
 
@roganjosh I almost just used Redis for this, lol
the redis stream datatype is super perfect for this.
 
wim
Are there any widely agreed upon conventions for "intersection of no sets" and "union of no sets" ?
similar to how sum([]) is 0, prod([]) is 1, any([]) is False, all([]) is True ...
 
:) Even if it doesn't a merit redis, it's a bit confusing to me that the class operates in two differes "modes" (?) between two threads, and it could be separated out. But I have nothing to back that up but my own logic
 
@wim Hm. Well, "no sets" is an empty grouping, i.e. empty set.
 
@wim can that be anything else than "error" or "empty set"?
 
wim
10:20 PM
Hmm. I could see the intersection of no sets being "the universe" :)
 
only if no sets contain the universe
 
what about {the universe} tho
checkmate
 
wim
because empty sum implicitly sums up from zero, right? so intersections could implicitly intersect down from "everything"
union of no sets being empty does seem reasonable, though.
 
@roganjosh Yea thats why it works best with buffer, since you do both read and write to buffers.
@wim where is there a sum?
 
@alkasm Aha, I misinterpreted what you took from the discussion sorry. I thought you were gonna put a buffer into the single class.
 
wim
10:23 PM
@alkasm when you sum up a bunch of values, you're accumulating them onto an implicit initial value, which is zero.
 
@wim yes but where in this discussion are we talking about summing?
 
5 mins ago, by wim
similar to how sum([]) is 0, prod([]) is 1, any([]) is False, all([]) is True ...
 
wim
yeah summing is just the analogy
 
oic
missed that
 
wim
but it's pretty common in mathematics to formulate numbers on top of set theory
for some perverse reason
 
10:25 PM
sum([]) is 0 because 0 is additive identity, imo
 
I've heard people build up analysis from set theory :|
 
(I have a degree in math, fwiw)
 
wim
mathematical navel gazing
 
intersection of sets = {x | x in s forall s in sets}. if sets is empty, then the intersection is simply {} from the definition
similarly for union
 
wim
hmmm.
 
10:28 PM
OTOH probably any set theory definition is going to say "nonempty set of sets"
like intersection of a nonempty set of sets is....
 
wim
empty doesn't make sense to me intuitively - the fewer subsets to intersect, the larger the intersection is. why should it suddenly jump to empty again, when you're intersecting the fewest subsets?
 
wikipedia: "The most general notion is the intersection of an arbitrary nonempty collection of sets. If M is a nonempty set whose elements are themselves sets, then x is an element of the intersection of M if and only if for every element A of M, x is an element of A."
blah
 
@wim Because intersection will give you a subset from the "inputs", so my intuition is fine with "no data" equalling "no output"
 
@wim I mean, it doesn't get smaller, but doesn't necessarily grow.
yea, intersection must be a subset of all sets
 
@alkasm but the kicker is that there's no sets from which to take a subset, so you can only rely on intuition
 
10:31 PM
the problem of course is {} does not have any sets in it
yea
exactly
lol
 
to be fair I have terrible intuition when it comes to discrete math
 
same.
wait theres actually a section on this on the wiki page
LOL
"nullary intersection"
 
there you go
 
In mathematics, the intersection of two sets A and B, denoted by A ∩ B, is the set containing all elements of A that also belong to B (or equivalently, all elements of B that also belong to A), and nothing else. == Notation and terminology == Intersection is written using the sign "∩" between the terms; that is, in infix notation. For example, { 1 , 2 , 3 } ∩ { 2 , 3 , 4 } = { 2 , 3 } {\displaystyle...
 
seems to support wim's notion
Good news: When M is empty the condition given above is an example of a vacuous truth. So the intersection of the empty family should be the universal set (the identity element for the operation of intersection). Bad news: Unfortunately, according to standard (ZFC) set theory, the universal set does not exist.
and if you fix it it becomes empty, apparently
 
10:34 PM
lol exactly
"Now if M is empty there is no problem. The intersection is the empty set, because the union over the empty set is the empty set."
 
so now let's discuss the merits of ZFC...
 
alright ima head out
lol
 
to be honest most of what I know about ZFC is that it stands for Zermelo--Fraenkel--axiomofChoice, and that last one breaks up friendships
 
@roganjosh err well...I...was. I mean I can separate it out but now its on the user to set up what will be used every time, so the class overall is just a wrapper to do both. There is a queue data structure to write/read from (just a queue.Queue()), and a thread worker that fills it.
 
Hello.. one question, I’m new to uwsgi and when I run the code pastebin.com/raw/ZagmAMkc using the command uwsgi --http :9090 --wsgi-file wsgi.py --enable-threads --processes 2, I get two processes, but the thread that I’m creating is running only in the main process.. I was expecting the thread also to be part of the fork, but that is not the case. How is it working internally?
 
10:46 PM
@alkasm It's still something of a misread of mine spawned from "have some object which, in a separate thread, just adds data to some queue". I had weird images of something else spawning threads with your object in different modes, rather than the object spawning a worker thread. I guess I'm too sleepy at this point :P
 
@JAamish "but the thread that I’m creating is running only in the main process" Do you mean that the second process doesn't have its own thread? Do expect the same thread to run inside two different processes?
 
:480770609 ohh yea i see, nah the thread/consuming would be launched with a context manager or something
like
with StreamBuffer(iterable) as streambuf:
    time.sleep(1)                   # fill buffer for a second
    print(list(streambuf.flush()))  # dump all the data from the buffer
 
@Code-Apprentice, yes.. I was assuming that uwsgi is spwaning two workers.. and they would be identical.. and the thread that gets created in the first worker, it will also be in the 2nd worker.. but obviously, it isn't.. so was wondering why it is that way
th = threading.Thread(target=daemon, args=()) - is only in the first worker, but not in the second
 
@JAamish If anything, each process will spawn its own thread, not share a thread.
 
yes.. that’s Okay too.. but the 2nd process is not spwaning that thread
 
10:50 PM
hmm...interesting. I don't know anything about uwsgi, either, so not sure how that works.
 
hmm yes @Code-Apprentice.. just confused how uwsgi works on this area..
 
I'm fearful of screwing up my anaconda setup to upgrade to walrus-times, but in relation to this answer, does it hold true for a = b = (b := 1)?
 
>>> b = 42
>>> a = b = (b := 1)
>>> a,b
(1, 1)
actually, I see no way it could not hold true
 
That's perhaps an overly-simplified example, but are the two bs completely separate?
 
What do you mean?
 
11:05 PM
As I understand it, you take the right-most expression, evaluate it, and then start going from the left-most assignment. But the walrus operator made an assignment to a name that happens to also occur in the middle
 
So it's overwritten later
>>> a = b = (b := 1) + 1
>>> a,b
(2, 2)
 
Ok, so the pattern holds. Thanks :)
 
Hmm, close enough.
> Such a cool question! Makes a lot of fun! :) Can be used at interviews :)
 
--> proceeds to not make a great example and put a bizarre comment under Shadow's answer (now deleted)
Although, they have since edited their example but I think they downvoted Shadow as it coincided with something like "Ooooo! Now it falls apart <comment-formatting-error><comment-formatting-error>"
 
wim
11:29 PM
@alkasm Thanks
@AndrasDeak because the constructivists insist on choosing their friends one-by-one ...
I had a professor who would always write in the margin of his proofs (ZF) when it didn't need to use choice and (ZF+C) when it used choice somehow (this was usually like an "indirect dependency", i.e. by having used a prior result which needed to use choice)
choice is weird because a surprising amount of obvious and intuitive things are not provable without it, but it also brings along a bunch of very bizarre friends to the party (e.g. banach tarski "paradox" - which is banach tarski "theorem" if you like choice).
 
that's my go-to "needs axiom of choice" example (and the only one)
 
wim
that's the famous one but the prof could rattle off a whole bunch more
 
11:53 PM
It has to be. I never learned any discrete math at university.
 
00:00 - 20:0020:00 - 00:00

« first day (3342 days earlier)      last day (1607 days later) »