« first day (3314 days earlier)      last day (1648 days later) » 

6:00 PM
cbg
 
wim
nice trick for your shell rc file:
export PYTHONBREAKPOINT="IPython.embed"
 
Foolish of them to assume I follow an odd/even pattern. My behavioral cycle is actually 17 days long, to confuse predators
 
wim
it means using breakpoint() will give you an IPython repl instead of pdb repl
 
alright. ParseElement.setParseAction works flawlessly. Now the only problem is that my grammar definition exploded from a tight PEG to dozen distinct islands of {grammar, __init__, (source->instance), (instance->source), __repr__} each...
 
Now you just have to automate the creation of those islands.
 
wim
6:03 PM
fun idea for a beginner project
 
Then submit it as a PR to PyParsing, under the method name create_ast_very_easily, so I don't have to do it myself later
 
wim
implement class CamelKiller
when mixed in with another class, transforms thisCase to this_case
when used as a metaclass, rewrites thisCase to this_case
 
"beginner project"
 
@Kevin for a moment, I was tempted to define a EBNF rule set to automatically do that. Then I realised I wouldn't be happy until that tool could automatically create itself.
the idea died very, very quickly afterwards.
 
wim
@Aran-Fey I thought this might be ~100 lines , no?
 
6:06 PM
for someone familiar with metaclasses :P
 
@Kevin until the apex predator evolves Septendec-Prime
 
well, learn to replace a word. Then learn an ast compiler. Then become an expert on classes. And while you're at it, solve P/NP
 
how do I store a network path in json serialized (dictionary value) so that when I read it again using os.path.join it comes out as a useable path? my path looks like

\\networkdrive\folder1\folder2\
 
wim
you don't need P/NP
 
@piRSquared With any luck they'll go after cicadas first, which will give me time to develop other countermeasures such as toxic secretions and detachable limbs
 
wim
6:07 PM
this would be a cute/fun weekend project
 
Detachable limbs are quite easy actually. It's growing them back that's the hard part
 
I tried putting 'r' in front of the string in my dictionary value like this

lpath': r'\\neworkdrive\afolder\anotherfolder',


but it didn't work, it read it like \\\\neworkdrive\\afolder\\anotherfolder\\
 
yes. The true test of the student would be not how well they implement the CamelKiller, but that they are able to identify the irrelevance of P/NP
 
@erotavlas that looks like it worked
>>> path = r'\\foo\bar'
>>> print(path)
\\foo\bar
>>> print({'p': path})
{'p': '\\\\foo\\bar'}
just a consequence of how dicts print their contents
 
@erotavlas json.dump and load should be stable, in the sense that saving an object to file and restoring it should give you the original object*. You don't need to do anything special to your strings in order to preserve their original form.
(*well, it won't be referentially identical, but it should still be equal at least)
 
6:10 PM
@Kevin I had to use r infront of it when backslash preceded the letter a because it encoded it with some something like \u0001
 
The r has nothing to do with json though
 
Ideally, you shouldn't care how the object is encoded.
In a perfect world, all you need to be interested in is 1) what your data looks like before you dump it, and 2) what your data looks like after you load it. As long as these are the same, the data could be stored in French for all it matters to your program
 
Thanks! @Kevin @MisterMiyagi
 
Maybe I'm misunderstanding the problem. Are you using the json module? Or when you say "json", do you mean "an object composed of dicts, strings, lists, and/or numbers"? It's fine if that's what you mean, I'm just trying to figure out where to focus my troubleshooting efforts
 
a small ruprecht would help
 
6:17 PM
@Kevin just had a network path in a dictionary value that I was serializing to text file using json dump, but I think its fine using the 'r' preceding the string path
 
It's probably worth mentioning that a string will display itself differently depending on whether you're printing it directly, or printing a list/dict that contains it
>>> s = r"\\netowrkdrive\afolder\anotherfolder"
>>> print(s)
\\netowrkdrive\afolder\anotherfolder
>>> print({1:s})
{1: '\\\\netowrkdrive\\afolder\\anotherfolder'}
Aran-Fey already covered this, but it bears repeating
 
wim
@Kevin should be? yes. are? no.
 
>>> json.loads(json.dumps({1:s})) == {1:s}
False
 
wim
{0: 'potato', '0': 'spud'}
 
interesting problem: how would I use csv.reader to read a file, in which the delimiter is escaped with parens/braces?
 
6:19 PM
Oops, I forgot that json doesn't like integer keys.
 
@inspectorG4dget s/interesting/XY/
 
Ok, revised statement: serializing a string and then deserializing it should give you the original string.
 
wim
the json spec says "A name is a string." stdlib json is literally broken.
 
@AndrasDeak I have what would otherwise be a csv file that's been generated by some other program. I need to parse it. It uses { } to escape the commas. It's either "hack csv" or "use regex"
 
wim
it should be strict by default, with opt-in for implicit conversions.
 
6:21 PM
Agreed, although we'll have to wait for the next backwards-compatibility-breaking version release before anything can be done.
 
@inspectorG4dget the only thing csv can escape is quotes. You're gonna have a bad time trying to parse that with the csv module
commas are "escaped" by enclosing them in quotes, and quotes are escaped by doubling
 
wim
@erotavlas use forward slashes and avoid the whole issue
 
hrm... that's unfortunate indeed. Any way I can avoid regex here?
 
Just to make sure: Both a{,}b and "a{,}b" would be parsed as a single column with value "a,b"? Or do quotes affect your escape sequence?
 
Do a find-replace on the file, replacing {,} with some character that doesn't appear in the file. Then run the csv parser as normal, and turn the placeholder character back into comma
Cross your fingers and hope none of your users ever uploads a pseudo-csv that contains all possible characters
 
wim
6:27 PM
@Kevin can't think of a counter-example, but would not be 100% surprised if there was one in the crazy world of unicode
 
@Aran-Fey quotes shouldn't affect the escape sequence
@Kevin can't do that, because I might have {a,b} as one column
 
wim
fish slap for other program
 
I give 10:1 odds that there's a crazy unicode sequence that doesn't json.dump/load properly. Unlikely, but not astronomically so
 
wim
don't write a parser, make other program use a known format.
 
 
wim
6:30 PM
10:1 odds means I get 11 quatloos for 10 quatloos investment? or 100 quatloos?
 
Can someone please explain to me the syntax?
 
100 quatloos.
 
Wait, so the braces can appear anywhere? You could even have a column like a{b,c}d?
 
wim
are you sure this is real work and you are not doing an AoC puzzle?
 
6:32 PM
@tatan What part in particular? In any case, may I recommend finding a better tutorial? parentheses-free print statements, percent style string formatting, and not using a monospace font are red flags
 
not listing outfile.close as a red flag is a red flag
 
Oops :>
 
though, since it's inside the loop, it's actually better than outfile.close(). So there's that.
 
@Kevin Yeah sure.. a tutorial might help... I can't understand the print statement with the % signs and the outfile.write line
 
Let's see, what's a good explanation of percent-style formatting... I've seen pyformat.info get recommended once in a while.
 
6:36 PM
Neither can I, what is this print without brackets and an argument?
 
TLDR: "foo %d bar" % 23 takes the value 23 and sticks it where the %d is, producing "foo 23 bar"
 
basically "%x.yf" % some_float means "round some_float to y decimals and pad it with spaces until the whole thing is x characters long"
 
@Kevin thanks
@Aran-Fey What is the meaning of f at the end of "%x.yf"
?
 
"float"
 
Percent-style formatting was largely replaced with .format, e.g. "foo {} bar".format(23), which has now been largely replaced with f-strings, e.g. f"foo {23} bar"
 
6:38 PM
oh i see
 
All three of them support various styling options, most of which are described by the format mini language
Although we discovered this morning that there are a few discrepancies, like where padding goes by default, etc etc
 
wim
largely replaced except they're all still there
 
Yeah :-/
 
Okay.. things are finally making sense... thanks everyone :-)
 
wim
this fits in with Python's design philosophy of "There should be 3 deprecated ways to do it"
 
6:40 PM
I had feared Disney+ might be an impediment to productivity, but I've managed to watch Mandalorian, Aladdin, and part of A New Hope with no issues ;)
 
wim
is leia a disney princess yet?
 
@Aran-Fey Is this really strictly true?
I mean see this
 
@Kevin Always reassuring when this room observes that they discovered things 'this morning'.
 
 
Assuming Dr Frankenfurter and Deadpool are, absolutely!
In my book, anyway.
 
6:42 PM
here x=1 but total length of second column is not 1
 
wim
he should have said "at least x characters long"
it's a minimum width, but will overflow if number is big
 
oh thanks
 
"pad to x characters" only adds padding, it doesn't chop anything off
 
okay
 
Not very useful to specify a pad of 1, then, since all floats are at least one digit long
 
wim
6:43 PM
I guess it is implicit in the word "pad", but the word "until" is a bit troublesome
 
Ok, NaN is strictly speaking not "one digit long" because it doesn't contain any digits. You know what I mean.
 
wim
my nan has 10 digits
20 if you count the toes too
 
6:58 PM
Aw, just realised I will miss the room meeting, as I have a work meeting conflicting.
@Code-Apprentice cbg
@Kevin Is NaN a float, strictly speaking?
 
wim
yes
 
worse, it's a double
 
wim
along with infinity, and negative zero.
 
Depends on what your definition of "is" is
 
wim
search for "nan boxing" for some hilarious stuff that .js do with nans, or an elderly spinoff of UFC, I forget which
 
7:02 PM
isinstance(float("nan"), float) evaluates to True, sure, but does it really capture the essence of floatness?
 
7:12 PM
I understood float to essentially be a mathematical decimal, represented in memory as a floating point number. So, TIL something.
 
@wim Thanks @wim - should I just explicitly check the Python version in the "doing distutils just in case" part, and bail if Py2?
 
@AndrasDeak yes, that's how you get two infinity... and beyond
 
7:29 PM
Hi everyone. I am wondering if anyone knows whether it is recommended to set FLASK_APP = app.py and do flask run over python app.py? And if so what is the reason behind it?
 
wim
@PaulMcG no
Honestly I think that PR should not have been merged. Since setuptools vendor pyparsing, they should be using the vendored version of the code to do it.
The other guy had the right idea by removing it
The rationale for putting the distutils back into master branch, so that they can bootstrap setuptools but not use the vendored code which setuptools decided to use for their bootstrapping, it just doesn't make sense.
 
@variable the quickstart mentions both
 
wim
It should not be your problem that whatever distribution this guy is dealing with can not figure out how to bootstrap setuptools properly, or they want to bootstrap setuptools using a custom version of pyparsing for whatever crazy reason, that's their problem.
 
actually, no
it says either use the flask script or python -m flask
@variable where do you see example of the latter?
elsewhere it again only mentions flask run
 
wim
he asked a very good question and the other guy did not answer it at all:
> Does this mean the distro will patch setuptools to unbundle its dependencies, but not patch pyparsing? Why the distinction?
what was it, openSUSE?
 
7:42 PM
@variable I think you'll find that flask run does a whole lot more than simply run your app.py through the python interpreter like any other script.
 
wim
To clarify, the distribution doesn't care if you have incorrect metadata (since they use package manager to install/uninstall, not pip). But your end users do, since they use pip. You need features of setuptools, so you need to fail hard if setuptools is there, not fall back to a broken install (incorrect metadata and impossible to uninstall).
 
is *not there
 
wim
So putting this fallback in is adding a (harmless) practicality from the point of view of the distribution's package manager, but can be problematic for any pip user.
It's a bit hard to explain, does that make sense?
 
I think that sounds quite reasonable
 
wim
I should have wrote "need to fail hard if setuptools ISN'T there" , too late to edit now.
The alternative is not using the features of setuptools, which is currently impossible if you: 1) want to enforce python 3 only, and 2) don't want to change the build backend away from setup.py and onto PEP517 instead
 
7:50 PM
does anyone know how to extract a gzip to a folder
so far I have this but I don't know how to do the final step (extract to folder)

#read saved file
with gzip.open(os.path.join(savedir, filename), 'rb') as f:
    s = f.read()
    f.close()
    uncompressed_path = os.path.join(savedir, "myfile_" + version)
    # store uncompressed file data from 's' variable
    open(uncompressed_path, 'w')
 
wim
@erotavlas I thought you said you were a senior level dev? Read the docs?
 
Assuming s is just a regular string, write it to the file in the ordinary way
 
i did surprisingly there is no mention of how to do that in the gzip doc
 
i.e. with open(uncompressed_path, "w") as file: file.write(s)
 
with gzip.open('foo.gz', 'rb') as archive, open('foo.whatever', 'wb') as file:
    shutil.copyfileobj(archive, file)
 
7:54 PM
Hmm, perhaps I'm misunderstanding the question. Is uncompressed_path supposed to be a file, or a directory?
 
wim
are you sure you don't have a a .tar.gz ?
then you should be using tarfile.open in mode "r:gz"
 
If the zipped file contains, say, ten files, it would make sense that a single ordinary open call in w mode would not create ten destination files
 
@Kevin your line is for xompression

with gzip.open(os.path.join(savedir, filename), "wb") as f:
    f.write(bindata)
 
wim
gzip itself is just a compression algorithm and doesn't really know about folders and stuff
 
ok then I'll use another library, maybe tar to decompress it
 
7:55 PM
Oh, maybe I'm thinking of zipfile. zipfile knows about folders and stuff.
 
@erotavlas side note: it's gzip doing the compressing
 
wim
ffs do you have file.gz or file.tar.gz
how you handle them is totally different
 
sorry no tar
 
wim
then the snippet from Aran-Fey looks good
 
7:58 PM
@erotavlas If you're saying "file.write(s) will compress the data, not decompress it", are you sure? Did you try it?
 
yeah of course that's how I created it
 
But consider that file and f are different objects.
 
wim
why are you calling f.close() from within the context manager?
this looks like PEBKAC to me
 
ok good point
whats pebkac? lol
 
@erotavlas Problem Exists Between Keyboard And Chair
 
8:01 PM
no problem exists with online documentation, its such a simple task should be documented
 
examples show 4 examples of compression
 
wim
it is a simple task and it is clearly documented
 
still searching lol
 
they may have presumed that extrapolating to decompression is obvious
 
wim
gzip doesn't know about folders. only bytes.
are you with me so far?
 
8:05 PM
what is 7zip doing when you right click on it and extract to folder?
 
@wim ok what should we do about all the dupes/caveats on inspecting a variable's name chat.stackoverflow.com/transcript/message/47817872#47817872 ?
 
Not all simple tasks should be documented. If every module that returned a string explained how to write a string to a file, the documentation would be ten times as long.
 
wim
I don't know, what does it do? I don't use 7zip or clicky-clicky guis like that.
@smci it depends on the context
usually it's an XY problem
sometimes it's actually a good idea (class attributes / descriptor __set_name__ for example)
 
omg ok never mind you guys thanks for helping but I think I can leave the file gzipped I don't have to extract it myself (there is a separate tool in the software I am using to do it)
 
problem solved
more than once, actually
 
8:09 PM
yeah thank god, i hate wasting time on stuff like that
 
okay?
 
wim
I tried extracting a file.gz using a gui and it just created a file with the contents in same working directory
presumably it just stripped the extension off the file.gz
yeah I renamed to example_123.gz and it generated example_123
I guess 7zip does the same
 
import zipfile
target = "path/to/myzip.zip"
destination = "path/to/folder_that_will_be_created"
with zipfile.ZipFile(target) as z:
    z.extractall(destination)
Not sure if this is what's being asked for, but here's how you unzip a .zip file into a directory
 
wim
I don't think they have a .zip, they have .gz
different beast
 
yeah with zipfile it works but can't use that format unfortunately
 
8:12 PM
Oh, I was hoping they were basically the same
 
wim
hahaa
 
Like those file formats that have some proprietary algorithm that will be copyrighted for the next ten thousand years, so everyone just uses the open source alternative that's free and almost as good
cough cough gif cough cough LZW
 
Speaking of compression, my PC just finished compressing ~130GB of data with lzma level 9. It took the whole dang day
 
wim
you mean open source alternative that's free and 10x better
@Aran-Fey as much as I hate facebook, zstd is excellent
if you have multiple processors, pigz is good too.
if you want super fast and best compression ratio is not that important, LZ4 is my goto
 
I wanted maximum compression because I can hardly fit all of this data on my 2nd hard drive, but I didn't think it'd be this slow
 
wim
8:17 PM
check out zstd then
 
...and I didn't realize borg actually supported zstd. I wish I'd found that out this morning
 
wim
sometimes single-process zstd is even better than multiprocess gz, it's amazing
 
zstd levels go up to 22, wow. That'll take some experimentation
 
the borg from star trek?
 
@Aran-Fey why not make 10 more compressed and make that the new maximum?
 
8:19 PM
@erotavlas no, this borg
 
wim
 
@AndrasDeak not sure what you mean
 
it was a weak ref to the "goes up to 11" that wim linked
 
I got the reference but I could not add a witty rejoinder in time
 
I'm not familiar with this reference
 
wim
8:20 PM
that's why it's a weakref
 
Note to self: make 11 sticker for my -_ key.
 
wim
badum tiss
 
a youtube link that's not a rickroll, that's rare
 
I have class
 
8:25 PM
if the universe is infinite i think borg must exist somewhere out there
 
it's not infinite
it's around 15 billion light years across, give or take
 
The observable universe is finite, but what about the unobserved universe
All sorts of wacky stuff outside of our light cone I bet
That being said, even in the parts of the universe that we can't ever possibly reach or even look at, the laws of physics are probably the same. So the exact events of Star Trek are probably not happening anywhere, because warp drives don't work.
 
@wim Sure, agreed, but where do we start with cleaning up the tons of dupes? I don't think the question you cited was very definitive, it's not good enough to be canonical
 
That doesn't exclude the possibility of a spacefaring race that incorporates robotic prosthetics into its citizenry and violently conquers other worlds, but then we're getting into the gray area of what exactly counts as a borg
 
@Kevin I thought Alcubierre drives could work
And we don't really know about the laws of physics. Physical constants can easily be different in other universes, if any...
 
wim
8:31 PM
@smci I'm not terribly excited about cleaning up tonnes of old dupes like you seem to be
dupes are good to whack new questions with
but I don't mind to let old questions with low views just linger
 
do questions marked as dupes get removed from SO?
 
wim
usually no
 
I'm counting "the bits of the universe outside of the observable universe" as still being part of our universe, for the record
 
but non-registered users get redirected to dupe targets
 
If we suppose that other universes with different physical laws exist, then sure, the borg can be in one of those.
 
wim
8:35 PM
now we just need to find a way to redirect them to dupe targets before they ask their question
 
googling is an unsolvable problem
 
wim
what do you mean by that?
 
such a feature does exist in the form of "similar questions" when a user specifies their question. The problem is that not everyone does their due diligence before hitting <SUBMIT>
 
wim
there is also selection bias ..
 
I'm partial to the theory that every mathematically describable universe really exists, so not only are the events of Star Trek occurring right now, so are the events of my Star Trek fanfiction where I am the fastest-graduating cadet in history and everyone admires me
 
8:37 PM
@wim there's an endless supply of users who will ask first, google second, because apparently typing your question into the ask form is easier than using google's search bar for the same purpose
 
wim
the users that are able to solve their own problem and have good google/docs reading skills, you don't hear from them
 
often, I've written question for one or both of two non-obvious reasons:

1. rubber-ducky my question to SO
2. write out the question so that I can get to a possible dupe target that I was unable to find by googling
 
wim
so the ones you do hear from are the clueless lot
 
Hello!
Hello guys, I am having some difficulties understanding this script(community.plot.ly/t/…). I need to know if the function (some_function) iterate through all of the sheets when I import an excel file? Any help will be appreciated.
 
@wim I never said I was excited about old dupes either. But when you use the search functionality (both SO or Google or !ddso), it tends to just turn up tons of crud, and it's impossible to find good answers. Do we not think that solving that at least for a few key topics is worth it? (If not, how do you expect new users learn to navigate all that?)
 
8:59 PM
@smci I saw your answer to my match.call() question. Thanks for the links.
 
wim
hmm, I do not think it's impossible to find good answers.
In fact I know it's not impossible, because I find good answers all the time.
 
9:23 PM
@wim We're talking in the context of a new user using the search functionality. I just get lots of bad old dupes, then I have to wade through them to see what's obsolete, inaccurate, irrelevant to 3.6+ etc. Anyway I'm just checking if any small effort to improve any of this is worth doing. Maybe not.
@cs95: Re your self-answered Select rows in a pandas DataFrame which has a MultiIndex, which is great. Just I think the 5-paragraph preamble is too verbose. Is it ok for me to try to edit what you wrote? You can revert or modify it as you wish.
 
can any of you guys help me with this question(stackoverflow.com/questions/58826911/…)
 
9:47 PM
@inspectorG4dget I haven't done it in awhile, but I definitely remember finding solutions via having my question or another person's question duped, where I didn't find the target by googling.
 
right before you post there's a list of "suggested" posts to peruse, and that is often way more useful than, say, SO's internal search
 
10:47 PM
@smci Sure, the reason it was there initially was to try and convince people to not downvote it. I don't think it matters anymore, so you can go ahead and just remove it altogether if you want.
 
11:32 PM
 
@smci looks good, thanks :)
 

« first day (3314 days earlier)      last day (1648 days later) »