« first day (3268 days earlier)      last day (1666 days later) » 
00:00 - 21:0021:00 - 23:00

12:20 AM
I don't have performance issues at the moment thank goodness, and I do turn off my laptop regularly so no reason for me to look into that it appears. Interesting nonetheless
 
wim
12:49 AM
🦡🦡🦡🦡🍄🍄🦡🦡🦡🦡🦡🦡🐍
 
Ahh.. it's a badger, was wondering what the little creatures were
 
 
5 hours later…
5:44 AM
cbg guys o/
 
 
1 hour later…
6:56 AM
cbg
 
 
1 hour later…
7:57 AM
cbg Arne
stackoverflow.com/questions/58129806/… typo but it's hanging around. I sense the vultures swirling
argg... 1 CV short before it got answered :/
Closed and the answer deleted. Thanks
 
8:23 AM
I'm curious whether anyone has an anecdote for why Stack Exchange and Stack Overflow are moving to CC BY-SA 4.0 is such a contested issue? I'm trying to plough through all the answers and comments but in my mind I'm just thinking "I just gave my answers for free"
I mean, I guess SO could flip their terms radically and start charging royalties for anyone using code they found on SO but outside of that dream world where such a thing would be possible to implement, I don't see any way it would impact contributions. Maybe that's why I'm not a Lawyer, though.
 
8:38 AM
@roganjosh most people would volunteer but the change is illegal
You can't relicense retroactively without explicit consent
Or so they say in that thread
 
Sure, I'll take that as true. I'm still struggling to understand what the worst-case ramifications of making such an illegal move could be
 
And yes, not being a lawyer is relevant
Technical differences are at creativecommons.org/Version4
no matter how harmless or well-meaning a change is if it's illegal
And having licenses is all about law
 
Perhaps your approach when answering is different, but mine starts at the lowest possible denomination. "I'll never get anything for this, I'm just bored and someone needs help" or along those lines. I suspect there is a case where this all terribly backfires and I'll say "Aha, I was blind to this issue"
 
Your argument is that licensing doesn't concern you. So it's understandable that a subtle change in license is irrelevant in your eyes
fortunately that's not how law works :P
 
In that frame of reference, my question is then "should this be bothering me?" :P
 
8:46 AM
No, it should not
But it's still a legal issue
9 mins ago, by Andras Deak
@roganjosh most people would volunteer but the change is illegal
Most people only objected to the illegality last time I checked
 
Yeah, that's pretty much the theme
 
"Yeah, whatever. But you can't do this"
 
Kinda disappointed I couldn't come up with a better riddle based on python's import mechanics, but here goes:
# You have a package with this __init__.py file.
# Unfortunately there is no "foo" submodule in this package,
# so the following import fails. How can you prevent
# an exception from being thrown without modifying the
# import or `sys.modules`?

from . import foo
 
9:02 AM
I want to add date to a H:M format string like
datetime.datetime.strptime("7:6", '%H:%m')
Is there a way to add date to this time
currently it shows datetime.datetime(1900, 6, 1, 7, 0)
 
What date would you add?
Orphaned timestamps will always default to that date and it is perfectly rational for any kind of work you'd need to do if the date isn't already in the string
 
ya like i have a date string say "20190927", which should be the date to the above time i need
 
I don't see how that relates to your initial comment. What is the full string you're parsing?
 
So combine the two strings and parse the result?
 
ya sorry, i simply did datetime.datetime.strptime("20190927_7:6", '%Y%M%d_%H:%m')
and it works
 
9:14 AM
huh, until now I've lived on my invented thought that %m needed to be zero-padded. Not sure where I got that idea
 
9:25 AM
trying to make a regex to get date out of a string: example sometext_20190927.somethin.anything
HOw can i extract on YYYYMMDD which is 20190927 in above case
something like a squeezed .[\d\d\d\d\d\d\d\d].
 
txt.split('_')[1].split('.')[0] without regex (untested, but regex scares me so I'll give an alternative)
 
ya but thats just an example, tried dparser.parse("stat_20190916.some.moresome",fuzzy=True)
looks smart
 
Why do you have such a varied format?
 
@pythonRcpp [\d\d\d\d\d\d\d\d] is just [\d], you need \d{8}
 
python strings could learn a thing or two from kotlin strings
>>> "example sometext_20190927.somethin.anything".substringAfter('_').substringBefore('.')
res1: kotlin.String = 20190927
 
9:32 AM
@Aran-Fey wow thats intutive
 
Also need to learn a dozen of handy methods. I'd rather roll my own. But I understand that tastes vary.
 
cbg
 
10:05 AM
It's been a long time since I've had a clean run of downvotes on a question and answers. Now one of the comments is that the OP had indentation issues on a question with a single line of code <head explodes>
Although I don't remember if <head explodes> is a self-closing tag. Since my embarrassment over </shrug> I'd better go with <head explodes/>
 
time for tea?
 
I think so
 
10:34 AM
Hmm, am I correct in thinking that python has special-cased behavior for attribute access on classes? Like, if you do x.y on a non-class x, python checks the namespaces of x and every class in x's MRO. But if x is a class, it also checks all parent classes
Or is that just magic in type.__getattribute__ or something
Well, I guess it doesn't matter, since type is strongly coupled to the interpreter anyways
Attempts to trick CPython have failed miserably
class A:
    x = 3

class M(type):
    __bases__ = (A,)

class B(metaclass=M):
    pass

print(type.__getattribute__(B, 'x'))
# AttributeError :(
 
 
1 hour later…
11:47 AM
cabbage
 
12:20 PM
cbg
 
cbg
 
cbg
 
капуста
 
Since the last import-based puzzle was so popular (*cough*), here's another one
# Create a module named "solution" that can be imported
# with `import solution` but not with `from solution import *`.

import solution
try:
    from solution import *
except:
    print('You win!')
 
12:35 PM
@Aran-Fey view spoiler
 
That was quick.
There's also a slightly different way to do it
 
it happened to touch on the very few things I do know :D
this is probably just another way of putting that it was easy
 
I came up with a similar answer, but I had to hunt through several pages of documentation first
 
I'm curious about the alternative, assuming it's not something weird like view spoiler
 
nope, nothing complicated like that. It just uses a different data type, that's all. You can do it by adding 2 characters
Hmm, I just found a 3rd solution
 
12:46 PM
ah, gotcha
 
Guysss, which widget should i use in tkinter to display a continuous stream of strings
imagine linux booting up, i want to display something like that
 
so you want a scrollbar? ;)
 
Text probably
 
Text is giving me that error : <function win_read at 0x0318F930>
 
That ain't an error
 
12:56 PM
That's not an error
 
oh
 
user error perhaps
 
You did something wrong, but we don't know what since we can't see your code
 
uh ok
root = tk.Tk()
S = tk.Scrollbar(root)
T = tk.Text(root, height=4, width=50)
S.pack(side=tk.RIGHT, fill=tk.Y)
T.pack(side=tk.LEFT, fill=tk.Y)
S.config(command=T.yview)
T.config(yscrollcommand=S.set)
quote = win_read
T.insert(tk.END, win_read)
tk.mainloop()
 
12:56 PM
ah yes
 
T.insert(tk.END, win_read())
 
@Aran-Fey that would be None
 
ok now i get an error : _tkinter.TclError: wrong # args: should be ".!text insert index chars ?tagList chars tagList ...?"
 
ah, I didn't realize the win_read definition was included in the code
 
in the original version, and it printed the stream in a loop
we asked Rozakos twice to fix the code format or post to a pastebin, that's how it ended up in the other room
 
12:58 PM
i can't post in pastebin
 
Every single code paste service is inaccessible to you?
that's a rough environment
 
i mean i can get here
but some of the sites are disabled
in this network
 
there's still the other option (fixing the formatting)
 
I thought it was all of the sites, considering the vast number of code paste services out there, including github gists
 
oh wait github right
 
1:00 PM
then again a programmer only needs google and SO
fortunately I'm off to commute, rhubarb
 
def win_read():
    while True:
        time.sleep(1)
        root.after(0, T.insert, tk.END, 'foo\n')

Thread(target=win_read).start()
 
Lol, nice optimism there. rbrb Andras
 
^^ proof of concept
 
Surprisingly enough, my question about the thread-safety of root.after is still unanswered, and there are indications of the answer being "it's unsafe"
 
1:06 PM
@Rozakos here's how I would do this. pastebin.com/1A10w140
I'm assuming ser.read is a non-blocking operation
 
@Aran-Fey It's interesting that Bryan has commented a few times but didn't offer up an answer. That's pretty strong confirmation that it's not a simple question.
 
@ke
@Kevin yes ser.read is a non blocking operation, i will try your way now, thank you very much
 
Hmm, is there any chance of non-blocking reads returning half of an encoded character?
 
Sounds plausible to me.
Especially if the read buffer has more than 10,000 bytes in it
 
good point
that sounds difficult to implement correctly, then
 
1:16 PM
i think it might work, i will need to convert my "string stream" to tuple though, since it gave me this error : TypeError: can only concatenate tuple (not "str") to tuple
 
Ideally you'd write a protocol on top of the serial connection that gives you some indication of how long the message is
Then you can accumulate the bytes over the course of however many reads are necessary, and only call decode once you've got all the data
This protocol could be as simple as "all string data is preceded by a two byte header indicating how many bytes the string takes up"
Or perhaps more than two bytes if you intend to send strings that are more than 65,536 bytes large
 
Every time I think I've finally discovered a use case for memoryview, reality slaps me in the face
AttributeError: 'memoryview' object has no attribute 'decode'
 
1:32 PM
@Aran-Fey Yesterday I said that after() was probably safe if all it did was append a function object to some internal to_run_later queue. But now that I look at the implementation, I'm pretty sure it's doing more than that. The self.tk.call is especially concerning, since AFAIK that's how the underlying tcl layer gets invoked
May as well be a signpost reading "Thread-Unsafe Land this way"
 
@kevin ok after some research, every string character takes about 22bytes, so you think if i do let's say len(output) and do that *22 and wait to accumulate all the data and then decode, it will work?
i am confused
 
writing correct GUI programs just got a lot more inconvenient
 
@Rozakos 22 bytes per character sounds a little too high to me. How did you get that number? sys.getsizeof? That includes header information that you won't be transmitting, so it's not a very useful measurement here.
 
I think utf8 characters are between 1-4 bytes long?
 
You can tell how many bytes will be required to transmit a string as bytes by doing len(the_string.encode())
 
1:37 PM
hm ok 1 sec
 
https://chat.stackoverflow.com/transcript/message/47409914#47409914
https://chat.stackoverflow.com/transcript/message/47409926#47409926
@Rozakos
In your github.com/Rozakos/SerialText/blob/master/Actual%20File, win_read is still returning None!
 
Hmm, I wonder if it would help to encode the string in UTF-32, since that's fixed width and every character always takes up exactly four bytes.
Then at least you know that you haven't sliced a character in half as long as read returns a bytes object whose length is divisible by four
The question then, is what to do if read doesn't return that many bytes.
 
If it returns, say, seven bytes, then you can't decode that final 3/4th of a character until a later read call returns the final 1/4th
So you're still stuck tracking some amount of state in between calls to read_and_decode_from_serial
 
1:53 PM
Yeah. If you want clean code you should probably turn it into a class
 
You can guarantee that data arrives in divisible-by-four chunks if you call read(4) with no timeout, but then the UI may hang indefinitely
 
Isn't read(n) allowed to return anywhere between 1 and n bytes?
 
If a timeout is set, yes. With no timeout, it blocks until exactly n bytes are received. pythonhosted.org/pyserial/pyserial_api.html#serial.Serial.read
 
oh, that's handy
 
hey guys, in sql is there any convenient way to select things in a group by that you aren't grouping over
 
1:56 PM
Are we just reinventing TCP here? I feel like we're reinventing TCP.
 
1	Central	6853
1	East	1467
1	West	4521
2	Central	14153
2	East	23205
2	South	10604
 
I think TCP is even lower level than this
 
I have this type of output, and want to do a group by to select the max and return the region with the max salesfor each productid
but if i aggregate to find max i get errors in oracle sql if i alos try to select the region
 
@Skyler AFAIK you can't select non-grouped columns in a group-by query unless you wrap them in SUM or COUNT or AVERAGE or some other aggregate function
 
@Kevin so i have to nest this and do a join or something like that?
god sql seems really inefficient sometimes
 
1:59 PM
Hmm, maybe I'm misunderstanding the query structure. Are you grouping by region or not?
If you're grouping by region then select region ... should work OK
 
SQL seems inefficient sometimes?
well, I guess most of the time it's more annoying than inefficient
 
I am now trying to find for product 1 2 3 etc the region where column 3 is maximized
 
Oh, the columns are product_id, region, sales? I thought it was sales, region, product_id
 
yea
 
(That's the kind of information that is best provided up-front in the problem description, for future reference)
 
2:04 PM
I guess I kind of did it a bit out of order underneath
it just seems weird I cant somehow couple the region somehow
I had to do a lot of subqueries to get that sales figure so its not like I can just join on the sales easily
 
I don't suppose you have a sql statement that constructs the table and inserts that data into it? Otherwise I'm going to spend the next twenty minutes writing one myself before I can actually start on the problem
 
create table my_table(product_id int, region text, sales int);

insert into my_table values(1, "Central", 6853);
insert into my_table values(1, "East", 1467);
insert into my_table values(1, "West", 4521);
insert into my_table values(2, "Central", 14153);
insert into my_table values(2, "East", 23205);
insert into my_table values(2, "South", 10604);
 
thanks
would that be sufficient amount of output @Kevin?
nothing too special going on so a case that works there will cover all cases
 
Yeah, that's enough sample data. I'm playing around with it now
 
select product_id, region from my_table natural join (select product_id, max(sales) as sales from my_table group by product_id); seems to work
ugh, writing SQL still feels like I need to make my brain run backwards
 
2:13 PM
stackoverflow.com/questions/7745609/… looks relevant but I think all the approaches in the top answer require a primary key, which this table doesn't have
 
@Aran-Fey yea, the my_table there is a long query so I was hoping for the sake of readability not having do it twice
with a temporary table like this can I just assign a primary in query
 
In any case it does look like you need a join
 
i had a feeling that was the case
 
I would deliver a lecture here about asking SQL questions in a non-SQL room, if I hadn't spent all of yesterday asking questions about HTML/CSS in here. I'll just politely assume that you're performing all of these queries from Python using sqlite3
 
many thanks, i needed to add a tiny modification on the suggested code to also display the sales but that did work
still annoying that it ended up needing that join and there isn't an elegant way to do something this trivial
then again I guess the elegance is saved for python
 
2:26 PM
Considering how database operations tend to be the most performance-critical part of an application, I'm surprised that vendors are content to force their users to have a double select. Maybe there's some magic going on under the hood that makes it as efficient as the single-select approach that we wish existed.
Perhaps joining a table to itself on its primary key is really fast if you've got any kind of indexing
 
It's my running assumption that they do get optimised away
I was horrified recently just how slow INSERTS are into Postgres, though. I was trying to migrate from a poxy SQLite db and just had to cut the process after ~10 mins, then just upload a CSV
 
2:49 PM
Why did someone decide to create a python library called config?
 
Could be worse. Just do import config as configlib or something
 
It definitely threw me in a question just now, I automatically assumed it was a local file they were trying to import
 
ah, that's your problem with it. True, it's an unintuitive name for a library
 
like bash's import
 
> The import program is a member of the ImageMagick(1) suite of tools.
Now I know who to curse next time I accidentally execute a python script with bash/fish/sh
 
3:10 PM
@Aran-Fey It does seem to be the case that class objects do attribute access differently compared to instance objects. If you look at the standard type hierarchy, the entries for "custom classes" and "class instances" both describe their attribute lookup strategy, and they aren't identical. In particular, only type objects search their MRO.
I believe github.com/python/cpython/blob/master/Objects/… is responsible for this, and corresponds to type.__getattribute__. The _PyType_Lookup is where the MRO search happens.
 
Interesting, thanks
 
Ultimately a class instance attribute access will search its MRO too, but not directly -- it will ask its type to do the search
Current confidence that my assessment is correct: 75%
I'm inclined to say that attribute lookup is not special-cased for classes, in the sense that the bytecode interpreter does not go out of its way to see if the target of LOAD_ATTR is a type or not. It just calls PyObject_GetAttr without looking
 
@Aran-Fey huh, didn't know that
 
3:28 PM
hi
I have a numpy question
You have an mxn array A(i,j), an nxp array B(j,k) and a pxm array B(k,i), and you want to form an m x n x k array D with elements
D(i,j,k) = A(i,j)*B(j,k)*C(k,i). What's the fastest and least intermediate space consuming way of doing that? In other words not just creating m x n x k arrays from each of A, B, C with ones in the appropriate axis and then doing element-wise multiplication.
 
np.einsum is the most concise
 
ah.. would you mind giving me a clue how to do that with einsum?
 
and "creating m x n x k arrays from each of A, B, C with ones in the appropriate axis and then doing element-wise multiplication" is not space-consuming if you use broadcasting
D = np.einsum('ij,jk,ki->ijk', A, B, C)
 
is that really the notation?!
 
try it and see, or look at the docs
 
3:31 PM
I am reading the docs as we speak
thanks!
 
and the broadcasting one is probably something like D = A[...,None] * B * C.T[:,None,:]
>>> n,m,p = 2,3,5
... A = np.random.rand(m,n)
... B = np.random.rand(n,p)
... C = np.random.rand(p,m)
...
... D1 = np.einsum('ij, jk, ki -> ijk', A, B, C)
... D2 = A[...,None] * B * C.T[:,None,:]
...
... print(np.array_equal(D1, D2))
True
 
Hmm, my assessment of attribute access must be incomplete, because I expected this code to raise an AttributeError on print(B().x).
 
and broadcasting is such that in D2 there's probably only one 3d temporary array being created on the right-hand side at the same time
 
@AndrasDeak that's cool. Let me do some timing
 
einsum is often slower, but I love its expressive power
 
3:37 PM
@AndrasDeak in this case einsum is apparently slightly faster
 
Or maybe my assessment is correct but my assessment of my assessment is incorrect
 
@Anush try doing it with CT = C.T.copy() before the timing bits, it might make the second one faster
which would mean that if you can create C with the transposed definition to begin with, that one could be faster (but I'd still stick with einsum even if this is the case)
(and of course using CT[:,None,:] as the last factor)
 
@Kevin Special methods like __getattribute__ are only executed when you're operating directly on an instance of the class where the method is defined; B.x goes through your __getattribute__ but B().x does not - it just goes through the __dict__s of all classes in B's MRO
 
@AndrasDeak thanks so much.. I have to admit this corner of numpy syntax is a slight mystery to me
 
I was under the impression that "goes through the dicts of all classes in the MRO" is done by __getattribute__ itself. And in fact this may be the case, but because of docs.python.org/3/reference/datamodel.html#special-lookup it's using type.__getattribute__ instead of Meta.__getattribute__.
"implicit special method lookup generally also bypasses the __getattribute__() method even of the object’s metaclass"
Confidence: 50%
 
3:47 PM
@Anush It needs some getting used to. The key thing is that None in an indexing expression injects a singleton dimension, so A[..., None] is A[:,:,None] which has size (m,n,1), B has size (n,p) which is broadcast compatible with (1,n,p) and the last term is reshaped to shape (m,1,p), so the three can then broadcast together to shape (m,n,p)
 
@AndrasDeak I will take note of that!
 
@Kevin I have no idea what exactly object.__getattribute__ and type.__getattribute__ do either
 
@AndrasDeak thanks
 
no problem
 
3:50 PM
by the way, can someone unstar that SQL thingy please
 
4:00 PM
Kevinned
 
I think object.__getattribute__ and type.__getattribute__ are comprehensible to us mortals, but I think knowing when exactly they get called is pretty murky
 
...among the 479 questions found by [python] check all elements list same...
...which leads us to How do I check if all elements in a list are the same?, whose accepted answer is too clever and causes lots of hashing: len(set(mylist)) == 1 instead of simply all(x == mylist[0] for x in mylist) . I upvoted the latter answer. But still don't want to use it as a dupe target.
 
ditching values into a set to check the number of unique elements is a standard pattern
problems with non-hashable elements would be more of a problem
 
it also works with arbitrary iterables and is almost certainly faster than the all-approach
 
4:16 PM
there are two very different answers on a short, clear question, I think that's perfectly fine as a dupe target
I wouldn't be surprised if the former was even faster
 
@AndrasDeak No it's not, do you think hashing long (possibly unique) strings isn't wasting CPU time? esp. when it's avoidable. This ain't just hashing integers and one-letter character strings.
 
...how do you know it's not just hashing integers?
 
My pub-ometer just received its strongest signal in a while. rbrb
 
Huh, somehow hashing 999 character long strings is faster than hashing ints on my machine
In [4]: itr = ['x'*999] * 99999
In [5]: %timeit all(x == itr[0] for x in itr)
9.18 ms ± 15.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [6]: %timeit len(set(itr)) == 1
814 µs ± 3.86 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [7]: itr = [0] * 99999
In [8]: %timeit all(x == itr[0] for x in itr)
9.11 ms ± 53.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [9]: %timeit len(set(itr)) == 1
1.01 ms ± 4.36 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
 
4:33 PM
Shouldnt the lzma module be installed when i install python3.6 on ubuntu (16.04)? Its missing for some reason, and no other installed python versions (default v2 and v3.5) have it either..
 
will@x45j7:~$ python3
Python 3.6.8 (default, Aug 20 2019, 17:12:48)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lzma
>>>
works on my machine, are you using an L not an I?
 
Is there a good reason why super() doesn't work in staticmethods? Why can't it just be equivalent to super(__class__, __class__)?
 
Yeah, i just saw that, but i do have v3.5 and v3.6 installed as well so it should be there
 
Poll: If you see `super(__class__, __class__)` in someone's code, do you think they
1) have no idea what they're doing
2) know exactly what they're doing
?
Well, thinking about it, super() has no way of knowing whether it's in a staticmethod or not. The only time it can be sure is if the function doesn't take any arguments at all.
Which means I clearly don't fall into category #2 of this poll
 
If I'm reading the source right, hash is O(size of object) for both strings and integers, so I'm surprised to hear that big strings hash faster than tiny ints for you.
Maybe the set constructor is doing some fancy optimization that leads to fewer hash calls for the string case.
 
4:49 PM
Do strings cache their hash?
 
I assume _PyUnicode_HASH(self) = x; gets macro'd into something like self->cached_hash = x;
At a glance I don't see any caching logic for longs.
 
so basically it's quite pointless trying to compare the speed of the two approaches
 
@Aran-Fey Because it isn't: source-code of Objects/unicodeobject.c. Hashing long strings is O(N). Hence, hashing long unique strings is wasteful, especially so when it's avoidable. Your benchmark didn't prove anything, other than that you cached one hash result on one string and used it 99999 times.
 
Right, but you don't know what kind of objects the code will be used on.
 
github.com/python/cpython/blob/… or github.com/python/cpython/blob/… is where the actual string-hashing sausage gets made, for the curious
Depending on your platform and #defines and such
@Aran-Fey Certainly I think you won't be able to come to a conclusion like "approach X is better than approach Y in all situations". But perhaps you can determine which one should be used on a case-by-case basis
 
5:02 PM
@Aran-Fey I stated above I was talking about hashing long (possibly unique) strings, so if code might need to handle those, then len(set(mylist)) == 1 trick is inferior to simply all(x == mylist[0] for x in mylist) . That's what I already said.
 
e.g. "I've got a list that usually contains 10,000 referentially identical strings, but 0.1% of the time one of the values is a different string": use the set approach. "I have a list that usually contains unique integers, but 0.1% of the time they're all identical": use the all() approach.
 
@smci Ok, I agree with that conclusion, but I have no idea why we're assuming a list of long strings
 
Short strings have a slightly more efficient hashing algorithm than long strings, although I expect it's not that much more efficient
I think the cutoff is 7 bytes
 
I'm making a point about scaleability: testing a toy algorithm on bullshit data then posting it as the universal algorithm to do X in all cases is bad - yet there it is as the top answer in How do I check if all elements in a list are the same?. I'm the person who's pointing out this breaks on long strings. I didn't "assume" a list of long strings.
 
Yo yo yo I am wanting to create a terminal program that refreshes the same line at some given refresh rate would curses be a good library for this?
 
5:09 PM
If your OS supports curses, sure.
 
Yeap running ubuntu so it works. I'm off to read the curses documentation
thanks my man
 
@smci I very much doubt the set solution will ever be horribly slow compared to the all solution on any realistic data
 
@smci you're unequivocally going against the room rules by swearing, though.
 
@roganjosh Oh ok. 'BS' data then. Or '*******' data. I'm clearly not swearing at anyone.
 
5:25 PM
Is it safe to assume that dict.values() is the only way to obtain an instance of collections.abc.ValuesView?
 
@AndrasDeak It's not a "short, clear question". It's a short, misleading question It has a misleadingly over-general title. But the example only shows single-character data, which is a terrible testcase, ignore scaleability, and what with caching and interning, will give falsely optimistic benchmarks. Most Python users on SO seem to be unaware hashing is O(N), so unless the strings are short and all the data fits in memory, bad for scaleability.
... Should I just leave a comment saying so? or edit the title? or add an answer showing it breaks? My motivation was to improve it to use it as a dupe target.
 
@smci This is pretty sad, especially from someone with 176K rep.
 
@PaulMcG Thanks for backing me up Paul. So, what should we do? There's a wider picture that users (including me) aren't sure that hashing is O(N), hence hashing a list of M long strings is in general non-scaleable O(MN) (unless you assume tons of duplication, and caching).
 
wim
5:37 PM
@roganjosh it sets a precedent
maybe next year they sell your content :P
 
I have a hard enough time doing that myself :P Good luck to them!
If I posted on the understanding that my contribution will be limited to a particular library, I'd be furious. But like I said later on, I post with the lowest expectation. I don't think SO could make commercial value out of disparate bits of content
 
wim
@Aran-Fey no
 
Well, not counting user-defined subclasses
actually, not even the stdlib. Just in the builtins, basically
 
wim
@roganjosh There's some excellent code on here, diamonds among the crap
 
@wim I'm probably using some of it in one way or another. How could they know?
 
wim
5:49 PM
CPython strings cache their hash, couldn't be bothered reading the rest of that discussion
 
Anyway, I was content to settle on the idea that I shouldn't be bothered in this case :)
 
wim
there is
>>> collections.ChainMap().values()
ValuesView(ChainMap({}))
>>> type({}.values()) is not type(collections.ChainMap().values())
True
 
6:08 PM
Hmm. But it's always created by calling .values() on the dict, at least?
ah, but it's not from the builtins module. I'll need to think about how I want to handle these in my serializer...
 
Hashing ONE 1200-character string takes 1ms.

import random
import string

def f(size=999, chars=string.printable):
s = ''.join(random.choice(chars) for _ in range(size))
return hash(s)


In [7]: %timeit f(1200)
1.01 ms ± 19.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
 
creating and hashing it takes 1ms
 
KPython cleverly avoids this by hashing only the first eight characters of any string
 
You could hash only the first 4 characters and double your performance!
 
wim
@Aran-Fey not really, it uses the abc implementation
and there's also odict_values, a different type to dict_values
 
wim
7:29 PM
userscript request: filter out all questions from rep 1 users from the main feed
 
:upvote:
 
Sounds like a good idea, I hope someone writes it for us
 
wim
main site slow for everyone today or just me?
 
Yes
 
writing a userscript to filter content on a page, how hard would that be on a scale from 1-10?
 
7:44 PM
depends on the content, but worth an attempt
the easiest content to filter would be a definite 1
 
Identifying the page element corresponding to a question - 2/10
Determining how much rep the author has - 3/10
Removing the element - 2/10
Monitoring the page every 1000 ms to see if the "x new questions" button has appeared or changed numbers - 4/10
Monitoring the page using the DOM mutation event framework instead, which is more responsive - 6/10
Determining the rep of the author of a question that would appear if you clicked the "x new questions" button - ???/10
Altering the HTML of the "x new questions" button to decrement its number or remove it entirely - 6/10
 
a recipe, nice
 
???/10 can be anywhere between 3 (question's HTML is present in DOM and merely hidden) to 10 (question's data exists nowhere on the page, and is retrievable only via an obfuscated API request with dynamic credentials specific to each user session)
 
My thoughts about the process were generally the same but I missed two components which you've outlined
What about simply making it appear that all one rep users have 100 rep ;)
 
Easy, just got to apply the regex s/\b1\b/100/ to the pagewide html
... Never mind that it will change the "1 new question" button to "100 new question"
 
7:56 PM
filtering out questions from all 1 rep users will, rarely, cause one to miss brilliant questions from users who are asking their first question
But really, the odds that a skilled user who will quickly attain high rep and add to the site starts off by asking rather than answering are pretty small, probably has not happened
 
@wim Lazy implementation: gist.github.com/Aran-Fey/421b9c3de872000101e79b5ec771898d (this just waits 1 second after clicking before it hides the questions)
 
Does anyone know how to vectorize an assignment of the form
for i in range(n):
a[:,i,i:i+m] = k
in the context of numpy or pytorch?
The trouble here is that I want "sliding" indices for the third dimension.
 
wim
@Aran-Fey hmm, it didn't hide this one stackoverflow.com/questions/58140787/…
maybe we can just hide if OP have the 👋 New contributor hand wave emoji?
 
ah, I forgot getElementsByClassName returns a HTMLCollection. This is the "don't remove elements from a list while iterating" of JS
 
wim
it did hide some things though... was this the old "don't mutate list while iterating over it" bug?
kevin'd
 
8:12 PM
Hmm, querySelectorAll returns a NodeList... let's see here...
> In some cases, the NodeList is live, which means that changes in the DOM automatically update the collection.
the web API yammin' sucks
there you go. The VB.net tag gets reduced to 5 questions right now, that's pretty sad.
 
@user76284 Use one of the np.ndarray methods, rather than doing a (Python) assign on a range of cells. For example, look into methods like np.put. It helps if you tell us more about the array you're trying to create: what structure does it have? triangular? sparse? semi-sparse? How large is it?
 
wim
@Aran-Fey works better but still not 100%
 
hmm, works for me though
 
wim
hmm, maybe that one appeared later with ajax or something
anyway if it removes 95% of crap that's already a win! 👍
doesn't matter if a few slip through
thanks
 
thanks to the lazy implementation, you can just left click anywhere and see if it disappears that time
It's a feature!
 
wim
8:26 PM
this is awesome
huge amount of low quality garbage is gone
 
@user76284: don't say "array slice" if you mean "tensor slice", least of all in connection with "vectorize" or else you'll just get generic advice about (2D) numpy arrays. stackoverflow.com/questions/58140866/… . Anyway, please see the PyTorch doc on tensor slice assignment.
@wim Sounds nice. Could you please post a screenshot or snippet so we can see what it looks like?
 
It looks exactly like the question feed always looks, except with fewer questions in it
 
8:42 PM
@Aran-Fey Honestly, it would be really motivational to get a glimpse...
 
@user76284 i,j = np.mgrid[:n, :m]; a[:, i, i+j] = k, assuming m and k are scalars
 
wim
@smci just install it yourself and have a look. if you don't like it uninstall it.
 
you can even use np.ogrid instead of np.mgrid I think
 
wim
@Aran-Fey just a little feature request - if posts have a positive score let them through ?
 
can do
 
8:49 PM
someone left the greedy flag on the badger
@Kevin wow, is that Duff's device in the wild?
 
@Aran-Fey 'horribly slow' is a subjective term. On arbitrary data we'd have to compute the hash on at least the first two items, hence O(N) - even though we don't need to; hasing is an overkill over a simple string compare, for long strings. Remember, unlike string comparison, hashing won't do early termination at the first character mismatch. The point stands very much. len(set(mylist)) == 1 is too clever for its own good, it's for toy data. Use string compare. That's all.
 
I think a more realistic problem is using the all solution on a bunch of nested objects whose __eq__ implementation is horribly slow and __hash__ implementation doesn't exist/is significantly faster
 
@Kevin Uhuh. On a very-non-priority basis, I will do the perfplot curves (on 2.7, 3.6, 3.7, 3.4) for hashing runtime for random string and/or unicode data, with length N as the x-axis. Should be fun for a rainy day.
@Aran-Fey That's a good point too, although in data science comparing long strings is going to be far far more common than arbitrary nested Python objects; but we agree there is no universal best solution for performance, so that target needs to be improved. What sort of cases can you imagine where __eq__ would be horribly slow? How deep would the nesting need to be?
 
00:00 - 21:0021:00 - 23:00

« first day (3268 days earlier)      last day (1666 days later) »