« first day (2847 days earlier)      last day (2322 days later) » 
00:00 - 20:0020:00 - 23:00

00:24
Hello! I have a python multi-threading script running on an old 2015 Mac running el capitain with 4 cores that takes 6 mins. Bought a 2018 18 core iMac pro running high Sierra and now that that exact same script takes 40 hours to complete. This same thing is happening in php threading as well--scripts on old machine take minutes and on new machine takes 40+ hours...? Any hints what this could be due to?
Anybody running any php or python threading scripts on high Sierra or the new iMac pro?
00:46
rubber duck question: does the new one have the same amount of memory?
Insane runtimes are often due to intense swapping once memory is gone (thrashing)
Perhaps the 18 processes eat up all the RAM?
 
7 hours later…
08:02
cbg-ning
08:49
Cabbage all. I just opened SO to find this comment:
@Simon Please accept the answer. You, of all people, should know how it works around here. Then (and not before) I will vote your question up.
Good day :)
cbg
Huh; what a great start to the morning. Their answer is decent, although not exactly what I was after (and yes I forgot about the question), now I feel like not accepting just for spite ;)
Either way the OP (with 3970 rep) needs a comment
Cabbage
I don't know the domain so I don't know the quality of the answer but the last few comments seem a bit outrageous
@Simon That's rather rude. It's ok to make a polite request, eg "Please consider accepting this answer". But it's not ok to make demands like that. :(
08:54
"You owe it to yourself to learn C++, else your skillset will be completely obsolete in a few years"
I'm in two minds about it. It's passive aggressive but just on the borderline of what I might flag
That poster is deluded if he believes that C is going to disappear anytime soon. C++ is a behemoth. Yes, it has its uses. It's not fun using a non-OOP language to build a sophisticated program. I remember doing GUI stuff in C back in the old Amiga days, but even there we used OOP-like techniques to do it.
However, there is tons of code where C++ is totally unacceptable. You wouldn't use it to write core OS routines. If you told Linus to rewrite the Linux kernel in C++ he'd laugh in your face. And think of all the code that runs on embedded processors and other small environments. They simply can't afford the waste of RAM that using C++ would entail.
And then there's the issue of optimization. Sure, modern compilers are pretty good at it, but I bet it's easier to do it on well-written C than on typical messy C++. ;) I'm getting out of my depth here, I assume Antti would have more accurate information.
Disclaimer: I learned C in the early 1980s, but I haven't used it much in the last decade, so I'm a bit rusty, and some of my info is out of date. I never bothered to learn C++. Originally I figured I'd wait until the language stabilized... I guess it has kinda done that, but it turned into an unwieldy ugly monster in the process.
Oh wow, that user is well out of whack with SO
@Simon you owe him
09:18
Welp, that's another person not getting invited to my birthday party. Thankfully they don't seem to hang around the Python tag so hopefully I won't encounter them again.
09:32
@Simon I'd custom mod flag the comment
other kinds of voting fraud are highly likely
 
1 hour later…
11:02
@AndrasDeak Yes I figured that too. Wait all the comments are gone? That solves it for me
@roganjosh You owe me. Give me up-votes :p
@PM2Ring Thank you for pointing it out, I hadn't noticed ;)
@roganjosh He has a point for business apps, but in the really low level stuff, no it's here to stay...
@PM2Ring My C isn't great, but you are totally correct. Programming drivers/things with little memory space and the like have and will be for some time coded in C and ASM, Those frameworks made for C++ specfically (Wx/Vulkan/Qt ect) all contain at least some C with C++ wrappers
That is why people still use ASM because they can use nothing else to get the job done efficiently.
@Simon presumably you flagged it, all the comments are gone. Did you get any feedback?
Nm, reading comprehension fail. I guess a moderator from here saw it. I did my best to fight your corner :)
11:28
No I didn't flag it. I had to leave. Thank you for the support ;)
 
2 hours later…
13:13
Python xml ElementTree from a string source? yet another example of the "accepted answer solved the OP's practical problem, but not the problem that was actually asked, and the next answer has three times as many upvotes as the accepted answer, thanks to all the people coming from google that actually want the answer to the asked question"
"How do I get an ElementTree from a string source?" not strictly answered by "use fromstring" because fromstring returns an Element, not an ElementTree
Hmm, is it unreasonable to expect import xml.etree.ElementTree as ET; ET.parse("input.xml").write("output.xml") to create a document whose contents are identical to the input document? Because that's what I expect, and that's not what I'm getting.
Is there a canonical dupe for just basic set intersection? I can find good ones that start with lists, but none that start with sets and just want intersection
\o cbg
Ah, I think stackoverflow.com/questions/8983041/… solves my problem.
@user3483203 Not that I know of. It's in that gray area of "too basic for SO"
Hmm that xml solution kind of only works if you know ahead of time exactly what namespaces you need to register before you open the document. I only have one piece of test data here, so I can't draw any conclusions about what namespaces might appear in all possible input
Rapidly approaching the practicality limit for this script, which should save me five minutes a month
13:31
do any of you guys use visual studio code?
Only when I'm doing C# work
I was trying to install a package in my python project using pip but in the terminal it says it doesn't know pip
hello
wow, you gone messed up when Kevin has to get you to leave room 6
I keep missing drama
Apparently that guy has been going around for the last couple days causing trouble, so I'm just following everyone else's lead
13:47
grrr figured it out...I had to add the location of pip and python to my path environment variable
That does seem like it would be important.
This form requires that I submit "a datetime" but gives zero indication of what the formatting ought to be. My day is ruined.
I guess I'll put in 2018-08-02T09:16:12Z and pray
You could try inspecting the html element and try guessing from there
I lied, it's not an HTML form, it's a SOAP request template.
That's more scarier, good luck!
Possibly SOAP has a well-established and publicly documented format for datetime fields that I could look up, but I like complaining more than I like solving my own problems
First google hit suggests "YYYY-MM-DDThh:mm:ss". Problem: first google hit is w3schools.
14:00
I think they improved a bit recently, but yeah, still a problem.
Most times I still have to hit the 2nd link to firefox dev platform, just like you have to skip py2.7 first-links.
Google hit #3, w3.org/TR/xmlschema11-2/#dateTime-lexical-mapping, says the same thing. I think.
dateTimeLexicalRep ::= yearFrag '-' monthFrag '-' dayFrag 'T' ((hourFrag ':' minuteFrag ':' secondFrag) | endOfDayFrag) timezoneFrag?
Pretty sure that's BNF-speak for YYYY-MM-DDThh:mm:ss
typo etc stackoverflow.com/questions/51653124/… I'm glad that the OP's problem is solved, but I'm not happy about all those wrong answers, some from people who should know better.
@Simon you still could, mods see deleted comments
Obligatory "you can edit your messages for up to N minutes" ;-)
Double-post on edit by buggy mobile site...
14:08
Makes sense.
@AndrasDeak but if it has already been dealt with is there any point? Ask them to keep and eye on the particular OP?
I'd ask to investigate if this is a recurring pattern for the user or if they're into different kinds of abuse. But no pressure, do as you see fit
Rhubarb for now
rbrb ok sounds like a good idea
 
1 hour later…
15:15
rb folks
What's the procedure with these types of question? User posted megabyte of data... stackoverflow.com/questions/51656851/…
Unpopular opinion: I consider the M the least important part of the MCVE.
DSM
DSM
Long weekend in sight cabbage for all!
I don't mind a hundred lines of input data if the rest of the question is good.
Not that that applies in this case, since the rest of the question is not good
DSM
DSM
@Kevin: you're right about the ranking, but it's so easy to get within, say, 10% of minimal, that it's pretty frustrating when people don't make the effort.
15:20
True. Huge data implies that the OP didn't try very hard to make it not huge.
morning cabbage
I can imagine scenarios where some nasty heisenbug only occurs for a comically big input, and any attempt to cut that input in half makes the bug vanish. But those are exceptional cases.
DSM
DSM
Sometimes it can be tough -- someone once found a Sage bug involving a very long function (mathematical function, I mean).. and AARGH Kevin just Kevin'd me, that was exactly the situation which actually happened. I had to write a pretty complicated code to reduce it down to something debuggable.
@Kevin personally I am equally frustrated by incomplete code as I am by non-minimal code. I'm quite active in and it's frustrating how many code examples are missing a simple class Foo around the methods.
In plain Java, it often isn't critical, but with Swing or Android, the containing class often extends another class which provides much-needed context.
I'm active in and 80% of questions leave out their import statements and I wish to slap each OP with a large herring
15:26
I can see that. For me, I'm generally very forgiving of missing imports...and sometimes even prefer that they are left out. A moderately complex Java program can have 25 lines of imports and I consider that just noise.
Although...there is the rare case that the error asked about is caused by an improper import...but those cases are rare enough that it is usually easy to tell if that is indeed the problem.
DSM
DSM
TBH I don't mind if someone drops import numpy as np, pandas as pd although I'm always happy to see it.
Especially problematic since they might be using any of import tkinter or import Tkinter or from tkinter import * or import tkinter as tk and any solution you implement for one won't work with any of the other ones, unless the OP is smart enough to fix the namespace, and a lot of the time they aren't
jpp
jpp
stackoverflow.com/questions/51656988/… Possible duplicate, full disclosure: my close vote was overturned by another gold badge.
yah, I can see why that would be frustrating. When the problem might be from the import but it is difficult to tell without them.
@jpp [tag:cv-pls]
DSM
DSM
I guess it does annoy me a little when somebody does import pandas..
15:28
At least there aren't any SNAFUs at the magnitude of "did you do import datetime, or from datetime import datetime?". Count your blessings.
DSM
DSM
DSMScript will learn from Python's naming mistakes.
"All identifiers must be globally unique"
@Kevin Well, with most of those it's easy to see from their code, apart from import tkinter vs import Tkinter, and the OP should know how to deal with that. Unless of course that's the total cause of their problem and they didn't post a Traceback, or any error message at all. :)
And I mean global. Sorry, you can't do count = 1, since John Smith of Springfield Nebraska used that variable in his script the other week
DSMScript best practice is to append a GUID to your variable name, e.g. count_123e4567_e89b_12d3_a456_426655440000 = 1
Most of the time these days when I answer a Tkinter question I post an answer using import tkinter as tk. And if the OP uses a star import I always lecture them as to why they shouldn't do that.
DSM
DSM
15:33
@jpp: hmm. While jez does have a bad habit of re-answering obvious duplicates -- and I've told him this in comments before, so I'm not breaking news here: if you can do it in one line, it's probably a dupe, and there's probably a preexisting answer by jez himself -- I'm not in love with that target.
@PM2Ring True. If the OP provides all code other than the import, you can deduce which one he used. Emphasis on "if".
I hate Tkinter questions that don't post (almost) runnable code. You end up spending half the time adding enough extra code just to make the thing runnable.
Just become a human tkinter runtime. That's what I did for flask questions.
jpp
jpp
@DSM, Fair enough, that's why I disclosed the disagreement. For me, it's a dup + typo (forgetting to close quote marks).
DSM
DSM
@jpp: my only hesitation is with the dataframe/series confusion also going on. I guess I'd have voted to close and added a one-line comment.
jpp
jpp
15:38
Yeh, I would have closed + commented on the typo. Because the question does state he's working with a series. No big deal, at least ppl can look at the link in the comment.
The tkinter .pack geometry manager cannot be fully comprehended by minds embedded in euclidean space. Humans should not attempt to become a runtime unless they want to become topologically equivalent to a klein bottle.
DSM
DSM
I'm not even sure I'd have used .str[:4] anyhow. .str.startswith means we could avoid having to count characters.
@davidism I can do that. But I prefer to test code on actual hardware. It's just too easy to make a simple mistake that you don't notice.
Mar 9 '17 at 8:43, by PM 2Ring
@AshishNitinPatil Old programmer saying: "Nothing is so smiple that you can't screw it up." :)
I was half joking. I usually just dump their code into whatever Flask app I'm working on at the moment.
DSM
DSM
[scribbles in notebook: new davidism attack vector discovered]
15:45
Speaking of simple mistakes, I wrote a number theory answer on SE.mathematics a few hours ago. Hopefully, I didn't make any simple mistakes or logic errors...
Speaking of SE.mathematics, there's an election on there at the moment, still in the nominations phase. One of the mods there retired recently, due to burnout. I visited the election page yesterday, and things are looking pretty dire. So far, there are only 4 candidates, and two of them strike me as definitely unsuitable.
I'm using my half-finished SE API library. Every six months or so when I need it it gains a couple new features.
Somehow openid auth is still working, I thought they disabled that.
The fact that burnout is even possible is shocking to me since I assumed every user on that site was a perfect being of logic and rationality, and thus no moderation would even be necessary.
Why argue about things when you can just deduce from first principles who is right
@kevin I take it you don't watch South Park?
Not since I was a lad
Gah, off the top of my head, I can't remember the phrases they warred over. To Google!
15:57
Fig 1. A number theorist prepares to solve m^2+1=5^n
@Kevin the episode synopsis is here but they end up warring over the name of the organisation.
Hi hwo can I filter only by date in old version of django like 1.8, where does not exists the queryset __date?
Kevin, which number theorist hurt you?
@Kevin There's pretty strong polarization in the community regarding homework questions. Some want to adopt the SE.physics approach of banning all homework or homework-like questions, some find them acceptable if the OP has made some kind of effort, and then there are the rep-farmers who'll answer anything.
Homework-dump questions there are a little different to those on SO, where you can argue that a good answer may benefit future readers. On SE.maths there's a much higher risk that the OP will self-delete as soon as they get an answer so that they don't get caught cheating.
I just listened to a new track by Buddy Guy, with guests Keith Richards and Jeff Beck. Not bad for 3 blokes with a combined age of over 230. :)
This homework distinction is silly. Don't we just want people to learn and collectively advance? I'm only interested in whether someone put some effort in. Across the board there seems to be some distinction about homework; if I was a maths professor and asked a question that someone attempted to answer but asked online, I really wouldn't care if it got answered if they showed effort.
If the student can't get an answer without asking the question online, ok. That probably makes them more productive than the 90% who basically do nothing. They just won't excel in their field.
16:11
No, we don't just want people to learn.
If someone has a legit homework question you'll never know it's homework
the problem is when homework is a defining quality of the question
I won't dispute that
And it would be nice to discard a lot of "write java in python" questions on account of "only your professor wants to do that"
But I wonder whether we (myself included) might have it wrong. We're in an age where Google searching is the norm, even for professionals. I'm playing devil's advocate; maybe the approach is to ask the hive-mind now. There will always be people who actually research the proper way and they may be accessible.
No.
Most answers are already out there
What people do is expect instant gratification without first making an effort. Like learning a language before using it. "How do I add this word to a string? I'm new to python"
That's the problem, not our attitude
Again, devil's advocate; people are answering those questions faster than I can find the dupe
16:23
It can be hard to find a good dupe for a lot of questions like that because they're so basic they don't deserve to be on SO. So the only dupes you find are to other low-quality questions, mostly with low-quality answers.
You're both starting from different places. roganjosh is trying to figure out "what's the best way to get people's questions answered" and Andras is trying to figure out "what's the best way to have a quality Q&A repository"
@davidism You say that as though being compared to a giant floating octahedron isn't the highest form of compliment
7
@roganjosh what does that imply?
@KevinMGranger and I'm absolutely not advocating this behaviour but I wonder whether there needs to be some introspection in light of the information age where you just get answers
Of course, I'm a little biased, being a rhombic dodecahedron. :)
@roganjosh Getting information is easy, getting knowledge still requires work, and actual learning.
16:26
@roganjosh as in, "maybe SO should evolve in what it's trying to be", or "maybe SO should evolve in how it's getting there" ?
@PM2Ring I never noticed that!
Again, the only clarification I can make if that I think it's completely unreasonable, but I wonder whether this is an inevitable path
I have no idea what your point is :P
If I can't answer immediately from my head, I'm going to google around the issue
I expect google to answer
Same here
16:28
Google is man-made
@roganjosh I remember that one. Pretty funny.
So why bother googling around the issue if stack overflow will answer it fully?
That's an annoying (to me) progression of thought, but it's only 1 logical step ahead of what we're doing to answer the questions.
That's why we don't answer crap questions :P
We understand why they ask, the question is what we do about it
16:31
but we are. Not us in this room (I'm speaking generally)
There's no generally
"In a recent xkcd's alt text, Randall Munroe suggested stacksort, a sort that searches StackOverflow for sorting functions and runs them until it returns the correct answer. So, I made it. If you like running arbitrary code in your browser, try it out." gkoberger.github.io/stacksort
Many people are stupid or lazy and answerers aren't an exception
I can speak to pandas. I will suggest there are two main issues (actually there are many issues but I think these are the biggest).
One: New askers of Pandas questions are more likely to be of the Data Scientist sort. That implies a higher likelihood of not having programmer like tendencies. The most important tendency lacking is the one that allows you to generalize.
Two: Pandas questions are difficult to find dups for.
Three: (I Know I said two) There are so many ways to answer pandas questions
typo from a newbie who can't read the docs stackoverflow.com/questions/51658386/…
16:38
That all translates into me trying to decide to just answer the question or take time to search for dup and explain in comments how it generalizes
@piRSquared tangential note: anyone calling themselves a scientist should have first and foremost strong abstraction skills
Heh
@AndrasDeak Otherwise, they're just a data janitor.
@piRSquared answers to pandas questions often amaze me. For a lot of people, they just solve a roadblock.
16:41
@AndrasDeak I share your purist views. So much so, I spent 30 minutes lecturing my 5 year old on the scientific method when he said he wanted to be a scientist and build things. I told him, "That' an engineer, a scientist must follow the method"
insert self depreciating joke here about how computer scientists have to have exceptional lateral thinking ability in order to convince themselves that they're real scientists
@Code-Apprentice make that run, then it'll be funny
inb4 'bold of you to consider that joke self-depreciating, since you're more in the "code monkey" category'
doesn't the TarFile.extractall method extract the contents of a tar.gz file if the files contain absolute path-names ?
inb4 Kevin is just a brain in a jar
16:44
@AndrasDeak we have no evidence to the contrary
docs.python.org/2/library/tarfile.html -- the documentation is not clear
We're all brains in jars, the jars are just these meat mechs we pilot around
@SusheelJavadi can you try and see?
yes, i did
What happened?
16:44
The canon of the room mythos is that I am a sentient collection of white triangles on an infinite green plane, as indicated by my avatar
it doesn't extract if the files inside have absolute-pathnames
@KevinMGranger Let's define jar as translucent and brittle
But perhaps I am a brain in a jar merely dreaming that I am triangles
but this is really surprising
@Kevin "self depreciating" does that mean the joke gets worse over time?
16:45
because the documentation doesn't mention anything like that
@Code-Apprentice Oops.
@SusheelJavadi the warning sounds like it should. Do you have correct permissions?
@KevinMGranger recursively deprecating?
There are definitely geometric shapes with more than 3 sides. You should get your counting abilities checked, mate :P
@SusheelJavadi It looks pretty clear to me. In fact, it warns you to be careful because it can handle absolute pathnames:
Warning

Never extract archives from untrusted sources without prior inspection. It is possible that files are created outside of path, e.g. members that have absolute filenames starting with "/" or filenames with two dots "..".
16:48
@PM
@PM2Ring - yes, I wasn't sure if that is a warning to the developer or a "warning" that the files with absolute-pathnames are not extracted
by the tarfile module
It's a security warning
Oh my science, I love this room. Heading out, take care :) rbrb
And you can edit/delete messages for 2 minutes after posting
@AndrasDeak - ok
It's warning you that those files can be extracted to anywhere in your filesystem, if the permissions on the destination directory permit it. Of course, on a sane system it shouldn't clobber anything vital that belongs to root, but it could easily overwrite files belonging to the user that runs the script.
16:52
but in my tests, it appears that the files (with absolute path-names) are not being extracted
DSM
DSM
Unexpectedly interesting reason for using exec: hmmm.
@roganjosh They're just triangles with an edge touching ;-)
DSM
DSM
@Kevin: I find it hard not to imagine you with a Sargon of Akkad-style beard because your avatar makes me think of cuneiform.
@DSM "ah, yes, of course"
brief cbg
16:57
brfcbg
I don't object to such a notion, since I strive to give the impression that I'm a philosopher that spends 99% of his time drawing circles in the dirt, and 1% repelling invading armies with an improvised death ray made from parabolic mirrors
@AndrasDeak hv ll th vwls n yr kybrd brkn? :p
The first step is having a beard that would not be out of place in a museum of roman statuary
btw guys... when do we have a next room meeting thingy?
"Soonish" was our last conclusion re: room meeting dates
16:59
trying to test creating a tar.gz file with pathnames...to post here ```sh-3.2# tar -cvzfP absolute_pathnames.tar.gz /blah
a absolute_pathnames.tar.gz
tar: Removing leading '/' from member names
a blah```
i even gave the -P option to not remove the absolute path-names, without luck
@SusheelJavadi Ok. Does the script give any error messages? I'd assume that you'd get an IOError or OSError if an attempt is made to write to a directory that the script doesn't have permission for. I admit that the docs don't mention that, but it would be surprising for such a command to fail silently.
I assume that you've made sure that all the destination permissions are set correctly. Are you running the script as a normal user, from a normal terminal? Or are you, eg running it as different user from a cron job, etc?
@PM2Ring -- it's not a permission issue i think
@Kevin okay... might be an idea to try and schedule one for August at some point?
@PM2Ring - running as a normal user
wow... Apple has finally become the first company in the world to be valued at 1 trillion dollars...
17:05
@JonClements Sounds reasonable.
@JonClements -- 998.4 billion :D or did it actually cross 1T
@Susheel they've just announced it on the radio as breaking news in the UK - so I'm guessing they might be right, or it's so close, it doesn't really matter.
One of the few contexts where 1.6 billion dollars doesn't really matter
@roganjosh I watched that "Richard Turner" Penn and Teller thingy - quite sweet. I've only watched the UK editions with Jonathan Ross hosting - didn't realise Alyson Hannigan was hosting that one.
@Kevin probably fell down the back of the couch or something... I'll happily sort that out if someone told me where the couch was :p
@SusheelJavadi Hmmm. Does the archive extract properly when you try to untar it from the command line? Sorry, I can't think of anything else, but I'm not a tar expert, and I don't use it very often.
DSM
DSM
More competition. :-/
17:42
@AndrejKesely As with any ranking of prgoramming languages, I am very critical of their metrics. But there are a couple of interesting bits of information in that article, like "[...], Python is now listed as an embedded language."
@Arne Yeah, maybe they're talking about MicroPython
> MicroPython is a full Python compiler [...]
🤔
sounds fascinating though, I had no idea
Never touched it so far, but it's in my list when I buy som Raspberry to toy with.
I've dabbled a little with MicroPython. I just ran interpreted scripts, though.
Now that I think about it, I am not 100% sure about the real, actual difference between a compiler and an interpreter.
The difference between JIT and a compiled binary seems abvious enough. Is an interpreter somewhere inbetween?
17:55
@Arne In the end, they perform the same function. The main difference is what file you install on other machines to run, whether its a compiled executable or direct source code.
a compiler translates code into an executable file. You copy that file to other machines when you install it. For an interpreted language, copy your source files to other machines.
my understanding is that most modern interpreted languages use JIT technology
It kind of sounds as if an interpreter just wraps the compilation and execution into a single step. Is that really all there is to it?
18:10
recbg
cbg
The check engine light came on in my car today. I checked, and yep, the engine is still there
thank god for those engineer skills, right?
Now I have to go to the mechanic and pretend like I'm a normal person, instead of going home and being the goblin I usually am when I'm alone
How hard is it to replace a spark plug by yourself? Should be just like screwing in a lightbulb, right?
like a lightbulb that needs a torque wrench to be screwed in.
a word that i learned two minutes ago
18:27
@Arne Not really. There's a big conceptual difference, and modern interpreters generally work a little differently to traditional interpreters. As said above, a compiler translates the source code into an equivalent program in the CPU's machine code language. An interpreter doesn't do that.
It reads each instruction from the source and then performs the operation that the instruction specifies. Of course, to actually perform that operation some machine code is going to be executed but the interpreter isn't performing a translation process.
I have a hammer and a screwdriver and a ladder, let's MacGyver this mofo
I also have a soldering kit which I can use to combine my other tools into a MegaTool
@PM2Ring what is a pyc file, then? and what stops cpython from being a jit interpreter?
@Arne I'm getting to that. :)
curiously awaiting =)
@Kevin gl hf!
What I just described applies to a classic interpreter, like Basic. Modern interpreters like CPython blur the distinction a little. First, the Python source is compiled into a simpler language, Python bytecode, and then that bytecode is interpreted by running it on a virtual machine: the bytecode instructions are effectively the machine code of that virtual machine.
That machine only understands fairly simple stuff (compared to the Python source language), but its machine code is more sophisticated than the real machine code of your CPU, eg it knows about more sophisticated data structures, not just simple integers, floats and strings of bytes.
wim
wim
18:34
weird question I found when browsing Python eval error
How does TypeError: 'NoneType' object is not subscriptable raise? What eval trying to do to trigger that?
Presumably it look for None['x']
@wim locals and globals don't have entries for x, so it checks __builtins__
It's a guess
wim
wim
don't you think that's weird that it checks __builtins__ with getitem though?
But note that the virtual machine is still not translating Python bytecode into CPU machine code. It performs each action that's specified by the bytecode. You can write an interpreter in a high level language, without any knowledge of the CPU machine code.
wim
wim
should be trying __builtins__.x, no?
I don't understand how the code in accepted answer is working, but it does.
@vaultah so how would you have unhandled TypeError then?
lol, yeah
18:47
@PM2Ring I think I get it now
Thank you for taking the time to explain that to me =)
Ahh, another guess: descriptor protocol
@Arne Oh, good. :) I'm surprised that Kevin hasn't joined this conversation, since he's actually written a full interpreter for his Kevinscript language. I've only written toy ones, and that was a while ago.
FYI I also found this post that touches some similar points regarding java.
eval must be doing some type-checking trickery, because it seems to know whether __builtins__ is a module object or not before deciding whether to use getitem or getattr.
>>> import math
>>> eval("sin", {'__builtins__':math})
<built-in function sin>
So it knows to do math.sin here, rather than math["sin"]
wim
wim
Python 2 give a much more sensible error
>>> eval('x+1', {'__builtins__': None})
NameError: name 'x' is not defined
so Python 3 doing something weird and crazy here
>>> import math
>>> __builtins__ = math
>>> sin
<built-in function sin>
>>> __builtins__ = {"foo": 23}
>>> foo
23
Initially I thought this getitem/getattr trickery was an exclusive property of the execution mode used by eval, but here it is in the regular repl also
maybe open a bug on python: "I thought we check on behaviour, and not on type."
wim
wim
@Kevin huh. that bananas.
I wonder how/why that went in.
Hmm, but the very same code when executed in a script gives NameError: name 'sin' is not defined. Revising theory to "the trickery is exclusive to the execution mode used by both eval and the repl"
Imagine a CSI-esque scene where I zoom in on "REPL" and enhance the pixels in sector 2 to reveal the existence of "eval" within it
wim
wim
19:08
assign __builtins__ to SimpleNamespace instance not working
still tries getitem and crashes
github.com/python/cpython/blob/… is where the byte code interpreter handles name resolution when the name isn't in locals or globals. The PyDict_CheckExact line supports my theory that it's checking for the dict type. But I'm surprised that it doesn't check for dict subtypes also. But whatever.
But whether builtins is a dict or not, it calls Py(Dict|Object)_GetItem, so I still don't see how it could resolve sin in my earlier code
wim
wim
19:23
eval doc says "The globals must be a dictionary and locals can be any mapping".
Right
wim
wim
So where do we branch at L2178? into the if or into the else?
else
wim
wim
so you're saying PyObject_GetItem is tried, but Kevin says PyDict_GetItem is tried
you can't both be right ...
When we're doing print(eval('x+1',{'__builtins__': None}))? I'd expect f->f_builtins to be the None object, so it would go down the else branch.
wim
wim
19:30
ah ok
catches KeyError and not TypeError
which what vaultah said originally, oops
That's consistent with OP's "not subscriptable" error. But then, why don't I get "not subscriptable" in my math example.
wim
wim
you know what
so we probably don't get to L2178 at all
@Kevin because Python somehow turns __builtins__ module into f_builtins mapping attached to a frame
*Copies the attributes of whatever __builtins__ is at that point to f_builtins
@wim I linked the sources to P2.7 and P3
There is seen thatn Python2.7 formats the error code in all cases the same, in Python3 only when KeyError is thrown
wim
wim
@AndrejKesely Yeah you're right on the 2.7 part
19:37
But None['potato'] isn't throwing KeyError
wim
wim
I think you got the wrong branch on the Python 3 part though
It's throwing TypeError
wim
wim
oh wait, I see now
we go to 2163, exit 2165 and then jump down to 2174
Sounds right so far
wim
wim
I keep forgetting that C doesn't have exceptions
only gotos
CPython dev seem to liberally use goto as if it were raise
19:42
I am getting "global name i is not defined" error. paste.ubuntu.com/p/C6w5p3nxs3
I guess i is not defined in outer scope
can I make it work somehow?
Now I'm suspicious of github.com/python/cpython/blob/…, which indicates that in its first conditional GetItem doesn't operate directly on the object, it operates on o->ob_type->tp_as_mapping. Is a module considered a "mapping"?
wim
wim
@yasar swap the loops around the other way
It seems to work
I just cannot tell why it should matter
DSM
DSM
If you wrote it out as nested for loops, do you understand why it wouldn't work the way you wrote it?
19:49
I think it works the opposite of how I think it works
I get the order wrong in nested list comps about 50% of the time.
It's always been weird to me that [[item for item in row] for row in matrix], works, but [item for item in row for row in matrix] doesn't
DSM
DSM
nested for loop order = listcomp order. That's it.
What the Data Science Man said.
00:00 - 20:0020:00 - 23:00

« first day (2847 days earlier)      last day (2322 days later) »