« first day (3413 days earlier)      last day (1528 days later) » 

1:19 AM
@AndrasDeak I have a new Samsung tablet and went to install a PDF reader from the PlayStore, and on first use, it asked me to log in with my Google or Facebook login - I JUST WANT TO READ A @#!&#&#!@#$ PDF!!!
 
 
3 hours later…
4:31 AM
hello people
 
 
1 hour later…
5:47 AM
@wim From what I can tell, you never have to touch xml. It's only used under the hood
My problem with zeromq is that the documentation is nonexistent. I literally don't know the first thing about it
 
6:25 AM
cbg guys o/
 
7:25 AM
closed
 
7:45 AM
Hi everybody. I have a question which might be too basic for SO, but I can't find the answer using a search-engine: -- How can I find the time it takes for my Python app to load the different packages I use? --

I there anyway to trace what happens when during the execution of my app? Even only the correct search phrase to use on a search engine would be great help.
 
Take a look at the timeit and profile modules
 
@Aran-Fey Thank you! I will. :)
 
I posted this to SO stackoverflow.com/questions/60280319/…. Why will it work for one model and not work for another.
 
8:03 AM
Eh, I meant to edit, not delete :/ I seem to remember caching causing issues with timeit for imports
 
oh, good point. You'd probably have to do something like timeit('import os; del sys.modules["os"]', setup='import sys', number=1000)
 
@Ideogram see this but I do think the timeit approach is broken. I've seen it mentioned a few times that the caching of imports cannot reasonably be negated to get an answer. To get around it, you end up doing more work than just the import itself so it's not easy to isolate the import time alone
 
@Aran-Fey 🙋 we replaced some PHP services with python but needed to keep the API. apparently, xmlrcp is popular in PHP. Can't tell you much about the security aspects though, it's a purely internal service, and we generally assume not to be malicious towards ourselves.
 
hi guys, when running a python command like this: python.exe d:\fol1\subfol1\file.py
 
Somehow knowing that it's popular in PHP makes me more reluctant to use it :/
 
8:11 AM
Python does not know the working directory, all it knows is the directory of the file that is subfol1 in this case
assuming working directory is d:\
 
Why wouldn't python know the current working directory?
 
That is how it python is
 
I don't think it's possible for a process to not know the working directory
 
its working directory agnostic
Hence the usefullness of the python -m keyword
 
So it knows the working directory but doesn't use it? That sounds more plausible
Anyway, what's your question?
 
8:15 AM
So coming to my question - i want to understand how pytest would work. For example if the pytest is run from d:\ and the pytest recursively searches for test_* files (or similar regex), and when it encounters a file/test, if there is a code like open('../somefol/somefile') in the test case, then how does python / pytest handle such path - with respect to working directory or directory that holds the test file?
 
Relative paths are always relative to the current working directory
 
python would try to serach for the ../somefol with respect to the entry point file
What is current working directory - in my example it is d:
but the test file directory is d:\fol1\subfol1\..
 
It's possible that pytest changes the working directory, but I don't see a reason why it would do that
Yeah, pretty sure pytest leaves your CWD alone
 
@variable after days spent trying to debug similar problems, my input is "not running pytest from the root of your project is asking for trouble"
 
@roganjosh Thanks, if that works, it looks really straight-forward. I hope it can help me decide if it's worth re-implementing some of these existing modules.
 
8:29 AM
^^
 
@Ideogram are you sure this is something that needs to be optimised?
 
@Arne thanks but if user is running tests via pycharm - by rightclicking on test folder or test file, then..
 
@variable I run my tests that way too, it works quite well as long as you don't have multiple projects within a single repository
 
True
 
which brings me to my second wisdom of the day, "having multiple projects in a single repository is asking for trouble"
 
8:33 AM
and you can configure what your working directory should be via the pycharm config. But some users dont know about this concept of working directory and current directory in python
 
Why is the CWD even an issue? If your tests rely on the CWD having a specific value then you should fix them
 
@roganjosh Well,... I am porting an application I previously wrote in PHP to Python. I must say, I have completely fallen in love with the language, but I notice it is considerably slower. As in: 2 seconds vs. instantly response. I suspect the Sympy Geometry package is part of the problem.
 
@Arne my problem is that the unit tests folder is outside the package folder (that holds some files that need to be 'open('ed)
the reason tests folder is outside as we didnt want to distribute unit tests along with package
 
@Ideogram Only if you were launching the python process from scratch each time. Which, given my supremely limited understanding of PHP, could be a pattern that you have become accustomed to. Imports are cached, and a web server will run continuously, so the import will only be incurred once
 
@variable that's a good practice, tests shouldn't be distributed with the package
 
8:36 AM
If you know where the files are then why is your code struggling to find them?
 
I disagree - often times I do prefer having a quick way to test that the package that I've installed via a wheel actually worked
I don't want to rely on figuring out how to get the tests separately from the process of getting the package.
and having the tests being part of the same package namespace avoid that problem
 
@Aran-Fey because when test is run from pychrm by right clcking on test folder/file, then the python is unaware of folders/files outside the test folder
 
How can python be "unaware" of folders that exist on the file system and can be accessed with no problem?
 
that depends on which file is run. as i said python only knows about the folder that holds the entry point file
please correct me if i am wrong
 
It sounds like your code contains paths relative to the CWD despite the fact that you don't want them to be relative to the CWD
If you know the file you're looking for is in the parent directory of your python script, then make your path relative to the location of the script
 
8:39 AM
# my projects usually look like this
myproject  # folder that pycharm recognizes as "project root"
├── src # source code, gets packaged and distributed
├── tests  # tests + fixtures, still in project root
└── pyproject.toml  # or setup.py, that is configured to only look at src for the build
 
@Aran-Fey relatives paths dont work at or outside of top level directory. Am I wrong
 
@roganjosh That would indeed be the PHP way of working. I now run the app using 'Werkzeug'. Each time time I refresh the page, I have to wait these couple of seconds. Should I look into how I use 'werkzeug' and leave the imports alone for the moment?
 
i.e. instead of writing open('../file.txt') you write open(Path(__file__).parent.parent / 'file.txt')
 
Oh I will give that a try thank you so much
 
@Ideogram is there a reason that you aren't using Flask to serve your app content?
 
8:43 AM
@variable and two more resources that explain in detail which setups produce the least amount of headache: hynek.me/articles/testing-packaging (quite old by now, but well written and still moslty relevant) docs.pytest.org/en/latest/… (pytest's docs on the matter)
 
@Ideogram Flask is the application layer on top of Werkzeug. I'm not actually sure how you'd use werkzeug directly to serve your content but I'm imagining that you're reinventing a pretty big wheel here
Potentially you are spawning processes on a per-request basis?
Well, not potentially, almost certainly :)
 
@metatoaster I was considering writing an "unless.. " part, but decided against it. I know that the scientific community of python still likes packaging tests with code
 
well, the documentation you linked from pytest did cover all the basis quite nicely, so that's fine
 
@roganjosh I am reading along! :)

I am implementing an API without a GUI. So it's like www.my-api-project.com/?A=1&B-2&C=3 and the server responds with a SVG file that does something with these variables
 
because there are very good reasons to package tests as part of the package, but there are situations and setup that doesn't necessary warrant that, though in my opinion there may be other problems with that decisions that lead to this point, I guess.
 
8:48 AM
@Ideogram I highly recommend this tutorial. You'll want to read a couple of the first chapters to make sense of the application structure, and then go to that chapter
 
and yeah, I do work in a setting that does science stuff, so hence I have my biases.
 
@roganjosh All right, that is very useful indeed. It looks like I was taking the wrong route to this. Now I need to do some reading. Thanks!
 
@Ideogram No worries, hopefully it helps. Good luck :)
 
Is tehre an alternative of os.path.exists when we use pathlib?
 
Cheers!
 
8:57 AM
perhaps you are looking for pathlib.Path.exists?
 
the only file-related thing I'm not using pathlib for is os.walk
 
9:15 AM
I was wrong to say that file from antoher outside directory cannot be access if python file from an inner directory is run. What I said is true only from imports
It works with both os. and pathlib. appraoches
 
9:36 AM
Hello People Very Good Afternoon . I am working on churn model , where model interpret ability is very important. i came across a library call shap and in that there is a force plot which gives a intuitive understanding with other variables for every customer/instances . i want to know how can i have those numbers in a data frame which seen in force plot in shap library
 
10:01 AM
@NabiShaikh You mean this? The data it already in numpy arrays that are being passed to force_plot, so you just need to grab those arguments and put them in a df
Unless you mean the results of whatever transformation force_plot is doing, in which case you'll need to dig into the library itself and I'm not familiar with it at all sorry
 
@roganjosh Any link for reference ?
 
I already linked to the library, I'm not sure what else I could link
 
10:23 AM
Hi guys I have a question
I'm currently wanting to implement a feature where facial recognition is used. So when the student's face is detected, my python server will be able to detect which class the student belongs and hence send the name of the student to the lecturer thru email...
What is this feature called?
 
...face recognition?
 
yeah face recognition, but after face recognition is done , the name is out, i want to write an algo that finds which class does this student belong to..
 
I believe that's called a database query
 
hmm, yeah there's a database , im using firebase. So the python server communicates with the firebase
is that possible?
 
I have no idea, but I assume yes
 
10:38 AM
Yes, there is a library for it. You just need to search "python firebase" in google
 
Is there a tutorial for my question? Sorry I'm kinda new to programming.... esp python
 
As a whole-project, probably not. As for the individual components that you want to put together, almost certainly yes. The first thing you need to do is break the problem down into component parts, think about how they fit together and then research your way through them in stages
If you start with a basis of hoping to find a guide that will result in the whole project being done, you will inevitably fail. It doesn't work like that
 
But my school works like that
 
@VentusZXC then frankly you should learn programming and python first. Then do facial recognition.
 
Never teach programming, and suddenly came out with some hard programming programming
projects*
 
10:44 AM
If this is a school project, facial recognition is way too advanced. It's an expert domain. Unless you've been specifically instructed to do this, I would suggest a simpler project
 
10:55 AM
Our flask is using swagger - this is openapi v2. I was thinking if there are any benefits of upgrading to openapi v3
 
11:34 AM
Fresh cabbages everyone!
 
cbg
 
what would you use for a linked list in python?
 
@Permian Why do you need a LL?
 
user10984358
@Permian are you asking how you’d implement a linked list in python? I’m assuming it’s because “My uni has asked me to implement linked lists in python for my algo / ds course”
 
11:55 AM
Probably a dumb code challenge site problem
 
- "... It will probably Fail. It doesn't work like that"
"But my school works like that"
However I do agree breaking it into components is the best way to go if you absolutely need to face this level of complexity in a challenge. But understanding face recognition systems and being able to extract the features and link it to a database and associate a name to it sounds really challenging. Plus making the code send emails to the professor. It sounds like a full stack data science job :B
 
looking up in a DB and sending an email seems trivial among that
 
user10984358
12:24 PM
Is object detection the same as face detection? Every different face is a different object? Or maybe the algorithm to detect face uses eye position nose alignment etc to enhance?
 
user10984358
If the answer to the first question is yes then you can look at TensorFlow’s object detection api. Which is relatively easy to train stuff you want to detect. If it’s not then someone can tell me how it differs ELI5
 
12:46 PM
Does named tuple support having strict types for its fields?
like only string
 
1:05 PM
you are talking about values, right?
 
Hi everyone
 
Cbg
 
@variable in case you are talking about values, the answer is no. types in python are annotations, so they have no influence on the behavior or actual types at all.
there are third party libraries that do read type annotations in order to introduce a validation step after/at the end of __init__ though, like pydantic. They support a @pydantic.dataclasses.dataclass class decorator which exposes an interface very similar to namedtuple in case you want to try it out.
 
thanks
 
1:40 PM
Hi how do I create a set of data (like a word or excel document) and then python server will do name matching
 
@VentusZXC You'll need to provide a couple of more details. What specifically do you mean by "a set of data"? What specifically do you mean by "name matching"?
And only use Word or Excel to create your data file if you absolutely have to. Start with something simple like a plain text file, or a csv file, both of which are very easy to read, with no extra Microsoft baggage.
Get "name matching" (whatever that is) working, then add in the Excel or Word stuff.
 
@PaulMcG Let say I have 2 names (Jane and Mike) in Class A and the names and class are stored in an excel file, so i want my python server to match the names in the excel file and execute some function, if the names r unmatched, nothing will happen
 
From what you describe, you want to check whether a name occurs in a document. Is that correct?
 
1:55 PM
yeah
 
Can you clarify what you mean by "Class A"? the term class is a tad overloaded.
 
oh , the Class A as in classroom
Classroom A
It would be like this:
ClassroomA:
Jane
Mike
 
do these files already exist? otherwise, there are simpler formats than word or excel.
 
So when python script is running:
Python:*scans name*
*James.... Lily...Mike*
Python:*Oh, there's Mike, he is in Class A*
@MisterMiyagi Nope they arent
 
@VentusZXC - did you miss the part where I said "start with something simple like a plain text file"? Excel and Word files have lots of extra bits in them for formatting, fonts, margins, etc. Use notepad (NOT wordpad) and just create a plain old text file to start with.
Are James, Lily, and Mike elementary children, where they are in a single classroom all day long? Or are they college students who may be in different classrooms on different days?
So before think about Word or Excel files, think about what would be a good data structure for this. Let's assume these are elementary children, so we just have to associate each child with a single classroom. What would be a good way to describe that Jane and Mike are in Classroom A, while Lily and Severus are in Classroom B?
I just like typing and typing and typing, I have nothing better to do...
 
2:19 PM
Before even thinking about data structures, it may help to solidify the project requirements. Imagine that you have just hired a diligent yet rather dim intern. He has never worked in your industry and knows nothing about it. You want him to do this task. You tell him "I need you to scan names and check them against this file". Do you think that's all the instruction he will require?
 
Listen to Kevin ^^
 
Or do you think he'll ask "what does 'scan' mean? What file? How do I open the file? What parts of the file should I be looking at? How often do I need to do this? What do I do if I find a match?"
Ideally, all of these questions should have an answer before you write a single line of code
 
No response from @VentusZXC in almost 30 minutes, I'm dropping out of this convo
 
(Proponents of the agile development process will tell you it's fine to jump out of the plane and build your parachute on the way down, but let's focus on the fundamentals before doing cool stunts like that)
 
Sorry I was on the phone
@PaulMcG at this stage i would like to assume them to be elementary children . Yeah I like ur idea to start with text file
Yeah in my college it happens like that, students with no knowledge will need to do this as part of their project to pass the semester
 
2:28 PM
If you're saying "yes, I do expect rather dim interns to use my program", then perhaps I have not communicated my metaphor properly. The dim intern is not the person using your program. The dim intern is your program.
 
@Kevin He requires more than that. So its a no from me
 
The rather dim intern asked all the right questions, at least
 
T_T i sometimes hate the education system in my country
 
In the worst case scenario, the intern listens to you, says, "sorry, I don't get it", and goes back to his game of Tetris
 
we have less practical knowledge
T_T i sometimes hate the education system in my country
 
2:51 PM
@VentusZXC You have said twice that "[your] python server" does the matching. Are you talking about an actual, separate server with a long-running Python process to which the input files are uploaded to trigger scanning?
 
I think we are talking about something very simple, like a homework assignment that asks you to model students in classrooms. If this is for a college course, one would hope the attendees would be familiar with the classroom-to-student relationship.
Oh, the heck with it. This one-sided socratic method is tedious. Just make this text file:
Jane,A
Mike,A
Lily,B
Severus,B
 
Yesterday I saw a database question where there were two tables, class and student. One of the tables had a gender field. You'll never guess which one.
 
I'm trying to accept xml requests with flask, but it breaks at ampersands, even if I use &, my google-foo is failing me badly. ruprecht
did I use a wrong setting somewhere? all my sources tell me that & is valid text in xml no matter what.
 
3:08 PM
I assume it was an intentional design choice to give each class a gender, since the sample data had one class with only typically male names, and one class with only typically female names. I wasn't aware that kind of school actually existed. If you're going to accept students of any gender, you may as well let them mingle, right?
 
That might lead to dancing
 
@Arne Could it somehow be curl's fault? Maybe it's doing some kind of "helpful" pre-or-postprocessing on your data.
 
hmm, I'll try to make sure
 
I think this might be a "having your cake and eating it too" situation. If you want your data to be sent with the application/x-www-form-urlencoded media type, then all the ampersands and equals signs in your xml will be interpreted as delimiters for the parameters' key-value pairs. If you use a different media type, then I expect Flask won't be able to automatically parse the paramters as a dict
Possible solutions: 1) abandon the parameter-based design, use a different media type, and send only the xml data by itself; 2) keep using parameters, and percent-encode your xml before packing it into the foo=... parameter
Confidence that I'm not missing a simple and easy third option: 30%. I don't often play with this layer, so my opinion has as much weight as anybody else with access to Wikipedia and five minutes of prep time
 
3:29 PM
@Kevin P.E. classes tend to be gender-separate. Also the middle school health classes where they teach about you-know-what are also usually gender-separate.
 
developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST tells me that multipart/form-data and text/plain are valid alternatives to application/x-www-form-urlencoded. Not sure which one is idiomatic here. text/plain seems promising, since it seems like it wouldn't meddle with the content.
@PaulMcG Eurgh, then those classes would have either "male" or "female" in their gender field, and ordinary integrated classes would have null. What a pain; booleans are twice as hard to work with when they're actually boolean-plus-one
 
So I'm using Django but really it's an SQL question
I have a table model
And a row model
and a separate "item" model
I would like the items to appear as "children" of multiple rows
Is there a relationship that describes this?
 
@Kevin I created an MCVE for a question I have now I hope it is sufficient. I have the following nested dictionary and I want to loop through the value2 dictionary and get the maximum value. Here is the dictionary. Here is what I have made but it does not work.
{'id1': {'time': '2020-02-10', 'value1': {1: '66', 2: '0', 3: '0', 4: '66', 9: '114', 10: '0', 11: '0', 12: '114'}, 'value2':
{5: 39, 6: -1, 7: -1, 8: 39, 13: 50, 14: -1, 15: -1, 16: 50}}, 'id2': {'time': '2020-02-10T15:52:00Z', 'value1': {1: '30', 2: '60', 3: '0', 4: '36'},
'value2': {5: 55, 6: 52, 7: -1, 8: 54}}}
 
@MalikBrahimi If each child has many parents, and each row has one child, that's many-to-one right there
 
Ya so basically theres a foreign key from row to table
But it's not exactly a foreign key from item to row
because an item can appear under multiple rows
 
3:35 PM
guys how do I place the text on the bottom of the image
from PIL import Image, ImageDraw, ImageFont
import textwrap

astr = 'Newton Distance 9'

im = Image.open("source.png")
MAX_W, MAX_H = im.size
draw = ImageDraw.Draw(im)
font = ImageFont.truetype('/Library/Fonts/Arial Bold.ttf', 100)

w, h = draw.textsize(astr, font=font)

draw.text(((MAX_W - w) / 2, (MAX_H - h) / 2),astr,(255,255,255),font=font)

im.save('test.png')
 
@Vasilis You've only shown the dictionary, not what you've tried. Are you aware of dict.values()and the max builtin?
 
@MalikBrahimi Seems like you should have a foreign key from row to item, then
 
right now I have it like this : ibb.co/v4Dbg3p
 
@Kevin no I shouldn't
There are multiple items per row
 
If each item can have multiple rows, and each row can have multiple items, then it's not many-to-one, it's many-to-many
 
3:38 PM
I cannot access the last layer
 
Think of an expandable table in HTML ... you have your base row entry, but once you click into it there are several sub-rows
 
i will be back in 5
 
An expandable table is still one-to-many even if it's nested. Everything has no more than one parent.
 
Something like this just to confirm
 
solved, sorry guys !
 
3:53 PM
@MisterMiyagi I am struggling to access the values within 'values2', I want to iterate over them but I cannot
 
Well, what have you tried so far? What error do you get?
 
TypeError: 'int' object is not iterable
because the keys in the last layer are integers when I try to iterate over them I cannot
 
Please show the code that generates this error. You shouldn't iterate over each value, you should be iterating over all values.
For that matter, you shouldn't be iterating over the values at all because max already does that for you.
 
I have renamed the values so I hope it works.
for i in myd_dict['id1']['value1']:
        for j in i:
            list1.append(j)
TypeError: 'int' object is not iterable
i for some reason I dont fully understand has the following type: <class 'int'>
is it more clear now @MisterMiyagi, sorry I am still learning so I am not 100% clear
 
what value does i take at any given moment?
 
4:04 PM
for i in myd_dict['id1']['value1']: iterates over the keys of myd_dict['id1']['value1'], which are 5, 6, 7, ....
What do you expect i to be?
 
I think therefore i am
 
if you're not sure, add a print immediately inside the first outer loop and print out the value of i
@AndrasDeak giggle snort.
 
Note that the code you have shown would flatten the keys of value1, but your initial description says you want the maximum of the values of value2.
 
4:28 PM
@AndrasDeak Cogito ergo partial(reduce, operator.add)
 
wim
The irony of It's time to reward the duplicate finders [duplicate] itself closed as duplicate the same day it appears .. ☜(ಠ_ಠ☜)
 
Is there an "It's time to reward the ironic posters" post?
 
wim
Not sure about the usefulness of dupe closing on meta .. sometimes it seems used to shut down a discussion, and it doesn't really have the desired effect of moving a discussion over to the dupe.
 
53
Q: The company’s commitment to rebuilding the relationship with you, our community

Teresa DietrichMy name is Teresa Dietrich. About a month ago, I joined Stack Overflow as Head of Product and Community. During my years as an engineer and technology leader I saw the impact this community and site has on people across the world and I am very excited to be here. While I have personally gotten a ...

 
4:47 PM
"once a week" uh huh...
step in the right direction but i wonder what they are going to do with the many mods and people they stepped on...
 
yeah, especially in a post-Shog society
 
This literally screams it's a PR move because why now and why not when the train was on fire? Why wait until a bowl full of mods are out the door? I don't get the reason behind doing it now.
 
@MooingRawr Going back in time will be difficult - better now than a month from now.
 
"We will be creating a Moderator offboarding process" screams, thanks for being the sacrificial lamb; to me...
 
@MooingRawr it's a new employee so that part checks out. They hired someone whom they'll actually listen to.
 
4:52 PM
@PaulMcG I mean sure? better late than never? but the damage is already done, to me it feels like the fire is burnt out, just ember chars remind, and now they are throwing water to try and save what's left... when they could have threw water while it was on fire..
 
It would all be very nice and encouraging if it weren't for the fact that we already had long-term employees who argued to listen to the community, only to be sacked on the spot
 
Once again yea, they brought in someone neutral I guess, but im pessimistic on this. feels like if something goes south shes to blame, if something goes right then great. I guess any other job is basically like this :\
 
that's her risk
 
yeah seems like a slap to the face. "hey we want you to listen to us please or we are leaving" SO/SE: "okie good bye, oh we hire this other person who want want you to tell us what we need to improve, please fill out this form while you are leaving, so we can have your good ideas one last time"
I personally wouldn't enjoy doing her job... but then again I'm not qualify to be
Anyways I'm curious on what would happen next from this company, if the mods will be coming back / employee being rehired (doubt it) and if codidact actually takes off.
 
5:21 PM
@MooingRawr It would be very difficult for codidact to get loads of people easily; Right now, their idea of Free and open-source software is only what I see has advantage over SE.
 
wim
5:54 PM
copying design mistakes from other languages.. the slippery slope ...
@MooingRawr always good to have a patsy!
it's a little weird for someone coming afresh to the debacle to "personally apologize", are we take that as apology by proxy or what...?
 
yes, it's her job to fix this (or appear to fix this, depending on level of optimism)
Part of that is apologising and assuming responsibility. And let's face it, if someone from the original leadership apologised they'd be laughed at.
it's a gesture
 
wim
6:13 PM
ok, I think I misinterpreted "personal apology"
it just means "apology coming from a real person"
it doesn't mean "apology from a person who screwed up"
 
That's how I read it. But you're the native.
She's only been here for a month so she could have only been responsible for sacking Shog and Robert, which I doubt.
 
wim
I wonder if they (SE) ever considered having a public issue tracker, like jetbrains have
meta is not an issue tracker
 
that's meta...
it partly is, or should be
Jeff might have made that point in the "meta is murder" post, but it's been a while since I read it
 
wim
oh come on, they presumably use jira or similar internally, there is no way they use meta as their only issue tracker
my point is that to really engage with the community desires making issue tracker public, at least for projects such as the website, would be much more useful for that "community engagement" purpose than stuff like the loop and the blog (which I think many users don't care about much)
 
Oh, definitely not. But the company isn't public at all.
 
6:20 PM
@AndrasDeak +1
 
wim
jetbrains is not public either, doesn't stop them from having a public issue tracker.
To me it's obvious you want the main users of your product to have access to issue tracker. Since they're the ones that have/find issues.
 
My point was that they could use meta as a public issue tracker (or at least a public mirror of sorts) if they wanted to. I agree that it could help with social relations (although it would involve hella amount of work)
 
wim
meta is / would be a terrible issue tracker
the scope is different, it has the UI of Q&A and not the features that a tracker needs
 
Yeah, I guess. A forum layout would work much batter for that.
 
wim
so, I'm certainly not in agreement that they could use meta as a public issue tracker.
 
6:24 PM
having an atual issue tracker would make it much harder to ignore requests
 
6:40 PM
yes i agree it's good to have a patsy , and i do agree that SO would be hard to rival.
 
An issue tracker would have to be objective. Meta still raises the issues and has very emotive feedback, but it's still mostly ignored or paid lip service. I doubt it would be any more effective
It's the age-old frog-in-a-pan situation. The company mentality is set internally. A fresh face appears, finds the issues, posts, then just gets beaten down by the internal dialogue of the company. They'll fall in line and give up caring in the end just to survive in a job
 
 
1 hour later…
8:01 PM
I'm trying to write a 3d graphics engine in python. Can someone help me diagnose a problem I'm having?
 
not without a clear problem description
 
I can't get the faces to render properly
 
you can optimise if True: ... if False: ... to if True: ... else: ... :P
 
Faces are rendering to the screen but not at the proper coordinates
 
The python interpreter in my head crashed at face_color += (128,128,128) with the error "can't concatenate list and tuple"
And I know you don't want to hear this, but your code has a serious case of spagetthi. I'll be quite surprised if someone can find a bug in that, considering how hard it is to follow everything that's happening. You need to put some (or a lot) of that stuff into functions
 
8:15 PM
wtf lol
 
"It's a troll please don't be mad" <- how these people's brains work will forever remain a mystery
 
Not very welcoming
 
pretty sure it might be a bot account?
 
@erotavlas do flag
@MooingRawr nah, these are all humans. Idiots, but humans.
 
@Aran-Fey Like this?
 
8:26 PM
oof.
 
rejected as spam/vandalism
 
If anyone in data analytics is bored. Today has been coming up with the most elaborate data analysis and graphs (they don't need to be useful) I can think of for some data (there is a point). All our machine operators fill in a form on their issued ipods when their machine has to stop; recording the issue category (a fixed dropdown), the start time and end time of the event that took the machine offline, their name, the department and the machine name.
 
Sorry what Rogan ?
 
Basically, the senior mgmt forced all of the operators to fill these forms in over a year ago. They're coming in around 1 every ~5 mins. It transpired in a meeting that they had forgotten that they made everyone do this and all the data was being binned. They're now planning on buying a ~£30k system to do exactly the same thing because it shows some pretty graphs. It's all meaningless because they can't interpret them. Instead, I'm just replicating the whole system with existing data
 
Oh I think I understand what you are getting at....
 
8:39 PM
So, a 3D graph showing meaningless attempts at correlation are welcome, because they've wasted all the operators' time filling out a form for zero benefit, and the supervisors on the shop floor, who actually wanted to see this data, have just been blanked by IT
 
@erotavlas it might be worth raising a custom mod flag on the user, just in case they play around on other posts
 
(I have actually started making useful graphs for the supervisors based on all the effort their teams have been putting in). Now I just want overblown analysis
 
it should involve PCA
 
I tried to replicate this to show the frequency of downtime reasons against time but it started getting a bit complex
@AndrasDeak nice!
 
and maybe some poorly defined clusters
 
8:46 PM
I definitely wanted clusters but I wasn't sure exactly how I might define them. Maybe cluster the downtime reasons around machine operator
Not a traditional clustering, but I bet there's a visualisation along those lines
 
just throw kmeans at it with a few clusters and see if you can retcon an interpretation :D
 
hahaha
 
it's only obviously dumb if you don't colour the clusters in different bright colours and markers...
 
Well, I have a lovely bubble chart already of mean time down against the number of events, where the bubble size and colour is proportional to how many machines in the dept. have the reported issue
I just need to get the spectrum of colours to scale proportionately to be alarming for all departments (some have been more diligent in obeying the complete waste of time they were compelled to take part in)
 
Violin-chart with hour-of-day on the y axis? Those look rare.
 
8:56 PM
I could probably do that. There's ~25 reasons for downtime, so I could probably have two charts side-by-side with a dropdown on each so you can compare pairs of issues
 
maybe a Sankey diagram? plot.ly/python/sankey-diagram
 
I struggled with that one. 3 of the reasons involve "engineering" like "Waiting for engineer", "electrical engineering problem", "mechanical engineering problem", so I could go down that path but it wouldn't look impressive enough
But then there is also a "custom" downtime reason, so I started doing some word-frequency analysis for the free text, so I might be able to do something similar from that if I hard-code the hierarchy of the issues
 
word cloud! :D
 
Also in the pipeline :P I'm struggling a bit with D3 plots in JS. At least I'm also learning from this as I go
 
Probably a little late, but equipment downtime accounting has been a huge effort in the semiconductor industry since the early 90's (the equipment is so expensive, that the depreciation exepense for the fab equipment makes up about 1/2 of the wafer cost). Plus there is a delicate balance between maintaining uptime vs. scheduling important maintenance (else your "uptime" is just making low-yield wafers).
There was a push to come up with 2 or 3 dozen different "reasons" for equipment to be down, and the realization (only after suffering in the field with a complicated state model) was that more than about 8 were just too many.
 
9:08 PM
@PaulMcG Oh, please don't get me wrong. I understand fully what the importance of this is. What I am not happy about is that the SMT has forced operators to do this for the benefit of the SMT, who actually forgot they made them do it, and denied access to everyone on the shop floor from seeing the output. The people on the shop floor actually want to see this
 
I didn't take your intention that way at all. I was just offering a cross-industry benchmark that might give you some ideas on grokking and presenting such data.
 
So I have some simple Pareto analysis that tells the supervisors which machines are suffering from particular issues. It's absolutely to help them. It's also so that, when the SMT introduces the new system (which also relies on machine operators going through hoops), the supervisors can make them look silly by saying that they already had the system
What became most-painfully obvious when I was talking to the SMT about the fact they already had the system was that they had zero (I mean zero) appreciation that people on the factory floor might actually want to make use of this analysis. Their view is that the SMT cares, and the shop floor is a bunch of units that magically make the targets they set for them
 
E10, E58, and E116 are the related SEMI standards in this area. The standards themselves are behind a ($$$$) paywall, but if you google for "SEMI E10" etc., you will probably find some online presentations from SEMI conferences.
semi.org/en/Standards/CTR_031244 This gives a nice brief allocation of Total Time into different uptime/downtime buckets.
 
@PaulMcG That's really cool, thanks. Both for my own interest for "proper" endeavours, and also for adding credibility to this mission
 
9:32 PM
@AndrasDeak I've decided this is the priority for the morning. It's supported by plotly.js so it should be about 30 min work. I'll report back with pics, thanks :) PCA will take a bit more time, but that's definitely on the list.
 
no worries
 
9:56 PM
For a numpy question what's going on with memory here? I'm inclined to believe the OP in the comments, despite the fact that I can easily throw MemoryError
 
Maybe lack of swap, or difference in the code on the two machines, or user-level memory limit on the large machine
 
But I can't run it on a 16GB machine with no memory restriction. I don't know what "lack of swap" refers to, though? numpy won't dump to disk when RAM is exceeded, will it? I thought that was why dask exists
Will it run for you?
 
my machine thrashes if I overallocate
Hence significance of swap.
It's not numpy that dumps to disk, it's the OS.
 
Numpy is just throwing the issue straight away for me; it doesn't even try
 
@roganjosh do you have swap?
in any case windows might be different
 
Neo
10:05 PM
Can someone please look into this:
l3 = {'results': [{'IntDes': 'Vlan1', 'MACAddr': 'F02DE99A29FE', 'Status': '2', 'Change': '2'}]}
sys_name = {'results': [{'Name': 'Access_Sw1'}]}
node = {'results': [{'Interface': 'Fa0/1', 'IPAddr': '10.10.10.1', 'hname': 'Host1', 'Node': '10'}]}

l3['results'].extend(sys_name['results'])
l3['results'].extend(node['results'])
print(l3)

"""
Result:
{'results': [{'Change': '2', 'MACAddr': 'F02DE99A29FE', 'IntDes': 'Vlan1', 'Status': '2'},
{'Name': 'Access_Sw1'},
{'hname': 'Host1', 'Interface': 'Fa0/1', 'Node': '10', 'IPAddr': '10.10.10.1'}]}
I would like to merge 3 dict into one without any nest
 
*sigh*
@Neo if I recall correctly your current state is the result of an hour's struggle, and the code you're trying to use was handed to you, maybe twice. I see another hour's struggle trying to explain this new problem...have you read some tutorials?
 
mmm, then we're coming at this from different angles. Numpy 1.18.1 is flat-out rejecting even trying to build the array for me before it even tries. In the past, I've definitely thrashed memory. At this point I don't know whether it's an OS thing or a numpy version thing. I'm assuming swap is a Unix thing that implicitly spills over into disk storage, but that wouldn't explain that question (unless the 16GB machine is Unix and the bigger machine is Windows)
 
@Neo .extend is a list method, which should be an indication to you that you are merging the wrong objects.
 
@roganjosh if the array is larger than RAM + swap it will scream. If it's in between it will probably thrash.
 
Neo
yes you are right, I made the same mistake .. I need to take notes
 
10:09 PM
@roganjosh not numpy version
 
Neo
type() shows dict but I need to try repr
 
type(l3['results']) won't show dict. same for sys_name['results'] and node['results']
 
@AndrasDeak I've asked for OS info too, since that's the last bastion for figuring this out I guess
 
Neo
@MisterMiyagi, you are right. I am dumb
I was doing type(l3)
print (l3['results'] + sys_name['results'] + node['results'])
 
Well, the link in the comments is to this, a familiar name. Once something involves 2357112, I know I'm out of my depth :P
 
Neo
10:14 PM
Even this create nested list
 
@AndrasDeak Looks like you guessed it. I didn't even know that a swap partition was a thing, so TIL. Thanks
On Windows, I'm at the mercy of the OS on swapping, but maybe that's something I can configure. It didn't cross my mind that this could be a thing until now
 
Neo
found this cool fuction
def flatten_json(nested_json):
    """
        Flatten json object with nested keys into a single level.
        Args:
            nested_json: A nested json object.
        Returns:
            The flattened json object if successful, None otherwise.
    """
    out = {}
    def flatten(x, name=''):
        if type(x) is dict:
            for a in x:
                flatten(x[a], name + a + '_')
        elif type(x) is list:
            i = 0
            for a in x:
                flatten(a, name)
 
10:34 PM
@roganjosh red herring
It's still surprising if the macbook version "runs perfectly fine". I bet it has an SSD so thrashing is probably not much of an issue there, but they should not have 64GB+ of swap lying around...
(surprising in the "I'm somewhat skeptical" sense)
either way I'm voting no repro
@Neo "finding this cool function" is how you will never learn to write code on your own. You're asking a very similar and fairly basic question yet again, which is a bad sign that you're down the path of cargo culting. I suggest that you set aside what you're trying to solve for now, and spend a week actively working through python tutorials. It will be an investment in your future as a coder.
 

« first day (3413 days earlier)      last day (1528 days later) »