« first day (4121 days earlier)      last day (810 days later) » 

12:04 AM
@PM2Ring Thank you, codidact.org looks nice. Not sure if it will solve all problems, but open source nature should allow easier impact by community. Not that I like that it is in Ruby though ;)
 
 
8 hours later…
8:04 AM
I am looking for source code similarity tool in Python. Preferably command line. Do you have any suggestions ?
 
8:33 AM
a quick google search leads me to a few options, depending on what you want. could give any one of those a try.
also, you could just use an off-the shelf tool. if your plan is to prefer command line, why the requirement for python
 
8:50 AM
Wait a second, are you asking for a "Python code similarity tool" or a "Python code similarity tool"?
If you're just aiming at code similarity in general, I guess diff is a good starting point.
Of course it depends on what you consider "similar" code.
git diff know the algorithms "patience|minimal|histogram|myers", so looking into them might be worthwhile as a next step from diff's text line mode.
 
9:15 AM
Hi
 
9:50 AM
Hello :)
 
Hello ! can somebody please help me with
C:\Users\admin\AppData\Local\Programs\Python\Python39\python.exe C:/Users/admin/PycharmProjects/project1_AMD/main.py
Traceback (most recent call last):
File "C:\Users\admin\PycharmProjects\project1_AMD\main.py", line 6, in <module>
import data_extraction
File "C:\Users\admin\PycharmProjects\project1_AMD\data_extraction.py", line 3, in <module>
from main import baseurl
File "C:\Users\admin\PycharmProjects\project1_AMD\main.py", line 53, in <module>
each_link_data = data_extraction()
 
10:08 AM
There isn't really much we can other than what the error already tells you: You are trying to call a module. That don't work.
Did you perhaps intend to call a function in the module?
It looks like there's a cyclic import between the two modules as well, so you might even deal with partially executed modules only.
 
observe the lines import data_extraction and each_link_data = data_extraction(). python is saying, "why you do this?".
 
10:44 AM
Hey, I was trying to write a python program that does splitting of the regex "/\.?\s+/". Can anyone help me out here how do I put this in python regex? The meaning of each of the regex I learnt from this regexr.com site.
\. escaped character that matches "." character
? quantifier - match between 0 & 1
\s a whitespace
+ quantifier again
All these meaning I got from regexr.com site
Can anyone help me in putting in python regex?
 
r"\.?\s+"
 
I see. regexr uses '/' character. Hence I was using this . May be thats why the program was crashing
Thank you Frey
 
11:10 AM
@golo I suggest using regex101.com, as you can select the Python flavor
 
Wow. This is amazing
Thank you
 
Hello, I have a complex matter. I need a complex SUM IF and SUBTRACT IF formula for my df. IF column (A) = X, then add the value of this row of column (B) to a third column, (C). IF column (A) = Y, then SUBTRACT the value of this row of column (B) to a third column, (C)
Thank you in advance
 
@golo You're welcome. No need to star the messages though
 
ok
Since no like options, so I starred :P
 
11:33 AM
@Baobab Something like this, maybe?
x_mask = df['A'] == 'X'
y_mask = df['A'] == 'Y'
df.loc[x_mask, 'C'] += df.loc[x_mask, 'B']
df.loc[y_mask, 'C'] -= df.loc[y_mask, 'B']
 
 
1 hour later…
12:41 PM
@Aran-Fey thank you already. Do you know what the Error "TypeError: can only concatenate str (not "float") to str
" could be in this case?
 
Sounds like you have a string value somewhere in your C column (while B contains numbers)
 
Thank you, I'll try again!
 
 
1 hour later…
2:11 PM
Is there a good dupe for setting a function-local instead of a global?
 
2:29 PM
I have a requirements.txt file that contains two packages, pA and pB, both of them requires different versions of requests, I keep getting the error below when I try to install both of them from requirements file:

The conflict is caused by:
pA 1.1.1 depends on requests==2.25.1
pB 2.3.2 depends on requests~=2.27

How is this typically resolved in a python (django) project?
I was able to temporarily resolve this by first removing one of the packages (pA) from the requirements file and installing it via cli. pip still warned about the conflict, but the rest of the dependencies in pA were inst
 
Contact the devs of pA and tell them to stop depending on a specific version
 
what version should one put in their requirements file, say if i was in pA's position for example, and i had only tested it on 2.25.1 in my env, what should i put instead
 
It depends. Is that good enough of an answer? :D
 
Either of ~=2.25.1 or >=2.25.1 would be fine.
 
2:44 PM
In an ideal world everyone would use semver and you'd write ~=2.25.1
 
Personally, I would go with >=2.25.1 and only add an upper bound when a future version actually breaks things.
 
Hmm, I never considered that. Is that decision based on a bad experience you've had with ~=?
 
Yeah. Had a few unmaintained dependencies that ~= required things even though they would still work fine technically.
Often enough the major changes just didn't affect anything that was used in practice.
 
3:03 PM
cool, thanks
 
hey guys, i am using djangoactivity-steam and I wondered if anyone know a way to make the notification disappear after it has been displayed once
 
3:20 PM
FYI: apparently pandas 1.2.0 (Dec 2020) fixed a minor bug in floating-point precision in the 17th sig fig, in read_csv/read_table. This seems to be manifesting here and here.
 
3:53 PM
if I have a list my_list=[1,2,3,4,5] and slicing creates a shallow copy, but what is the space complexity for a code that does my_list=[1,2,3,4,5] new_list=my_list[1:-1] is it n or 2n? I know 2n is n, but what is it exactly?
new_list just holds references, do I need to add references when I calculate space complexity
 
Short answer: you don't need to.
 
@Jake What do you mean? my_list just holds references, too.
 
For the same reason that O(2n) is O(n), space complexity doesn't care whether the list holds references to objects, or objects directly
 
ok, so my_list is the original list, and the objects that are referenced by it are "created" there, so that list takes up space initially (the first n in 2n), the second list references what is already created, so why add again is my question
@Kevin this clears it up for me, from a theoretical point this is the sort of explanation I wanted to hear / understand
so if I do something like zip(my_list, my_list[1:]) I am not consuming twice as much memory?
 
For efficiency reasons, the Python interpreter creates an internal array of integer objects for the integers in range(-5, 257). So when your program uses those integers, they're never created, since they exist before the interpreter even starts reading your code.
 
4:01 PM
I read here that itertools tee is a better fit for this, but why if all I do is just refer a part of the list
 
zip in particular is quite memory efficient, only generating each value the instant before you iterate it
 
@PM2Ring ok, but in a case of user defined instance, where each instance holds "1" memory, as I mentioned, I am looking for a theoretical explanation
 
So for example, in the program for x in zip(a,b): print(x)`, the total amount of memory allocated at once is: the memory for a, the memory for b, and the memory for a single two-element tuple
 
I seem to understand the above, but my confusion comes when I use slicing
 
Compare to for x in list(zip(a,b)): print(x), which allocates a complete list of every tuple that zip can yield.
 
4:07 PM
@Jake If you make 2 lists that contain the same instances, each list needs to internally store the pointers to those objects, which is 8 bytes per object, assuming you're running 64 bit Python.
 
@Kevin ok this is making things a bit clearer
"pointers", I am guessing I will have to look up that, thanks Kevin and PM 2Ring
 
@Jake Sorry. A pointer is what the C code uses for references. It's the address in RAM of the object.
 
"itertools tee is a better fit for this" -- I partially agree. tee will use less memory than slicing in almost all cases, but in many cases it won't have an effect on big O complexity. In other words, the memory you save will be a small fraction of your total memory usage.
You won't get many accolades for saving 100kb of memory if the process already uses 500mb
 
Python objects are generally much larger. Eg, a smallish integer takes 28 bytes.
 
just to confirm, if one were to ask me the space complexity of zip(my_list, my_list[1:]) is it "O(2n) which comes to O(n)" or "O(n)" ?
 
4:16 PM
Just to be sure: Are you asking for a practical understanding or for some exercises/homework? Because the answers will be a bit different in either case.
 
I'd go with just "O(n)". IIRC, strictly speaking you're not even supposed to write O(2n) in your intermediary calculations, even if it gets simplified out before the end.
 
@PM2Ring Idk C or its terminologies, so I was bound to look at it one day or other
 
Yeah, the my_list[1:] creates a new list, so you've (almost) doubled the amount of RAM consumed by references.
 
@MisterMiyagi interview perpective
 
O(n) it is, then.
 
4:19 PM
alr, thanks again you three
 
@Jake I normally avoid using C terminology when discussing Python internals. It's generally not necessary, and can cause confusion. ;)
@Jake But as soon as you've finished iterating over that the zip, my_list[1:] will be destroyed & its RAM recycled.
 
better way to avoid confusion is to not use C in the first place :) , jk, I know its super fast and everything (CPU) has a C compiler at the worse case
I guess, I should probably look into something like timeit but for memory, so it could tell me how much a cell consumed, could probably give me some insights if at all it exists
 
If my_list is huge, it's a good idea to use islice rather than a slice. But there are time & space overheads in using iterators. If the list is under 200 or so items (IIRC) there's probably no benefit in using islice.
 
What's our cannonical dupe for [[0] * n] *n?
 
if that single line is a question, I have a lot to learn in python lol
 
4:28 PM
You can get an idea of RAM usage with sys.getsizeof. But it's not recursive, so using it on a nested list or dict doesn't tell you the total RAM usage.
 
@Code-Apprentice This one here: sopython.com/canon/18/…
 
1d list should do for me, didnt know the standard packages had something like this, nice
 
@Code-Apprentice I'm pretty sure we do have one. Let me check...
Ninja'd
 
I can say with certainty that PM 2Ring reads and replies to the older messages before moving on to newer ones ;P
 
Also, I'm on my phone, so I can only see a few messages at a time.
 
4:35 PM
have a nice one guys
 
See ya
 
PM occcasionally replies to things I've written days ago. I think everyone should do that more often.
 
@MisterMiyagi thanks
@PM2Ring thanks
I guess I should check that first
 
My meatspace friends think it's weird when I tell them "by the way, the movie you were trying to remember last week was West Side Story". As far as I'm concerned, a conversation doesn't end until all questions have been resolved. Otherwise, it is merely sleeping
 
@Code-Apprentice I've occasionally missed it even when knowing it's there. It's hard to remember the proper search terms.
 
5:11 PM
JPL updated their trajectory data for the James Webb Space Telescope, so I've been making 3D plots of its orbit. space.stackexchange.com/a/57832/38535 Technically, the code is Sage, but it's mostly plain Python, although it does use Sage vector & matrix objects, and of course Sage 3D plotting functions.
 
 
1 hour later…
6:36 PM
I'm looking at a job ad for a software vendor specializing in industrial manufacturing tech. Maybe @roganjosh can tell me some trade lingo that will impress them
"Sorry I'm late, my clock was a minute slow. Just like the TPS-2342 ConveyorBeltPlannerPro, am I right?" [pause for words and noises of agreement]
"You really have to treat yourself to ConveyorBeltPlannerProPlus. Some people say that the customizable splines feature is marketing fluff, but I've found a few cases where they doubled efficiency"
 
7:07 PM
howdy folks! it's been a while
@Kevin Musk talks about where automation should not be placed on a production line. Connecting hoses, for example. Not quite the targeted name-dropping you were hoping for, but that's about the best I can come up with right now
 
is there a way to profile which line of code is consuming cpu, when the program is already running?
I can't stop the program, because it has active users
 
7:31 PM
stackoverflow.com/a/147114/953482 discusses ways to get data out of a running python process. Of particular use are the answers for talking to an "unprepared" process, which isn't listening for debug signals from other processes
 
11
Q: Profiling a long-running Python Server

WillI have a long-running twisted server. In a large system test, at one particular point several minutes into the test, when some clients enter a particular state and a particular outside event happens, then this server takes several minutes of 100% CPU and does its work very slowly. I'd like to k...

 
The answers mostly just extract a stack trace or drop into PDB, which is sort of a far cry from proper profiling... But it's a good foothold in an imposing cliff
Worst case scenario: extract a thousand stack traces, with each one 0.01 seconds apart, and see which line numbers appear most often
 
sounds like a plan to me
 
 
1 hour later…
8:43 PM
@Kevin You're looking to program the actual mechanisms of the machine itself?
 
I think they're more on the monitoring/metrics/analysis/reporting side
 
The only real exposure I've had on that is RedLion working with Programmable Logic Controllers (PLCs) and they probably ran on C++ or Java, but they handily created their own "C-like" language for the pleb users to use, which was... just awful
If it's just interfacing with the PLCs, you'd smash it. Do you wanna email me the job description to take a look at?
 
@MisterMiyagi Is it better to retitle that "Duplicating a list through multiplication (*) only makes a shallow copy of contents" rather than "...shares references for the contents"?
...there's also another question Is creating a new list object via multiplication by one with an existing list equivalent to making a deep copy? although it sohuld really ask "a deep or shallow copy?"
 
That's different from the usual meaning of "shallow copy" though. Usually copying refers to the container, not the contents
 
...ok how would you reword that question? (to not be misleading?)
 
8:55 PM
I don't see a problem with it tbh
 
Hello, can I ask for help with a coding problem here?
 
If you're being strangled by a snake or the problem is in some other way related to python then yes
 
@FrancescaC Hello. Yeah, you can ask here but subject to the room rules. Specifically, you have two questions that are under 48 hours old and we ask that they don't get raised here
 
9:16 PM
Is there a way to take np.mean but only get 3 decimals? It's for a report. Currently it spits out 10 decimals.
I've been googling and all I can find is np.around but I want the mean rounded
 
ImportError: /opt/rh/python27/root/usr/lib64/python2.7/lib-dynload/_io.so: undefined symbol: _PyErr_ReplaceException

Anyone know how to resolve this?
 
@CelesteWilson You don't want to round. You want to convert the number to text with 3 decimals. You can do that with f'{some_float:.3f}'
 
@CelesteWilson pandas Series have .mean(), .round() etc. Recommend you use pandas. It's way nicer useability than raw numpy.
 
Hello all, I have a short question. How can I in my DF convert the value in the column year , displayed as YYYY-MM-DD" into "YYYY" only? Thank you
 
@Baobab Depends is your "YYYY-MM-DD" a datetime, or a string? See all the existing questions: "[pandas] extract year..."
 
9:22 PM
whats the datatype of the column?
 
@smci
It is a DateTime value
 
Baobab: so search on [pandas] extract year from datetime and look at the top questions.
@Baobab ...if it's a series, use the .dt accessor. See that doc.
 
@smci
thank you!
 
@Aran-Fey It will be going on a histogram plot
 
I remain steadfast in my opinion that you shouldn't change the number just because you want to change the way it's displayed. Unless you literally do not have a choice in the matter
 
9:47 PM
@Aran-Fey Sometimes, but sometimes you want to make subsequent calculations based on the rounded value. Either way, doing reports in raw numpy without pandas higher-order stuff (pivot-tables, formatting, styling, coloring, HTML export etc.) sounds painful.
 
@roganjosh Thank you, yes , I am struggling with those still... hmm but its not exactly the same.
 
If you're convinced it's separate to the two other questions, then fine. They are disparate. Please make sure your question is clear, though, because I'm not sure I follow your questions on main
 
well basically I am trying now to create a code that will iterate over my files so that it will turn them all into csv using this code: df = pd.read_csv("/Users/name/Desktop/PDB/147_norm_seqprof/4xgv_A.profile", sep="\t", header=None).astype(float)
but instead of just one profile, all of them in the directory
 
@Aran-Fey I have multiple statistics, some have 5 decimals, some have 10, some have 0. I want something common and in the industry I work in, we always round to 3 decimals.
 
@FrancescaC Have you googled how to list the files in a directory? Unless I'm missing something, this problem takes 10 seconds to solve
 
10:00 PM
I am guessing its something like this:
path = ("/Users/main/Desktop/PDB/147_norm_seqprof/")
# list all files in the directory, assuming this directory
# contains only files csv files that you need to save
for file in os.listdir(path):
df = pd.read_csv(path+file
yes but for some reason I keep hitting silly errors that I feel shouldn't be, like "isnotadirectory" and then I change it and get "IsADirectory"
 
This is relevant again here:
Jan 19 at 23:17, by Aran-Fey
Throw os.path in the garbage bin and use pathlib, problem solved
Ok, almost relevant. Mentally replace os.path with os.listdir.
Although I can't see anything wrong with the code you posted
 
"like "isnotadirectory" and then I change it and get "IsADirectory"" is not what I had in mind when I asked for a clear problem statement
 
OK let me see
 
You're just paraphrasing.
 
That was just the error messages I was getting
 
10:13 PM
Without the code that produced them, that's fairly pointless information
 
But it isn't, because python isn't that inconsistent with errors that it'll give you lowercase or camelcase errors
So, paraphrasing
 
path = ("/Users/fcatalogne/Desktop/PDB/147_norm_seqprof/")
# list all files in the directory, assuming this directory contains only files csv files that you need to save
for file in os.listdir(path):
df = pd.read_csv(path+file)
print(df)


ERROR:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/fcatalogne/Desktop/PDB/147_norm_seqprof/'
 
@Baobab You're welcome. Always first try the doc and searching SO :)
 
Hmm, come to think of it, shouldn't the opposite of "information" be "formation"? The english language disappoints me
 
@FrancescaC Before we go on, can you please read the formatting guide for chat?
 
10:16 PM
" IsADirectoryError: [Errno 21] Is a directory " was what I was getting before
 
@FrancescaC But pd.read_csv("/Users/name/Desktop/PDB/147_norm_seqprof/4xgv_A.profile") works? That is un-possible
Wait, that path is different. Never mind.
Oh, it isn't. name is just a placeholder. Never never mind.
 
@Aran-Fey "inflammable" is a fun one
 
@FrancescaC and Aran-Fey, my bad, I should have picked a better duplicate target for Francesca's question. Aran-Fey can you suggest one? There is this but it's from back in 2010. What's the best Q&A for glob, pathlib, (replaces os.walk), fnmatch etc.?
 
At this point this is no longer a programming question, and more of a tech support issue. At this rate we won't be getting anywhere unless someone remotes into your PC and you show them the problem(s) first-hand.
 
@FrancescaC Did you try putting it all on your local C drive and not your desktop, changing the path and trying it?
 
10:22 PM
I don't think that's very practical. Why would that fix path issues?
 
it works, okay phew
 
@smci I don't even want to read that question to be honest.
 
thank you very much
 
@roganjosh sometimes on our work computers at work, IT has things blocked so you can only use certain drives to access things (I'm not IT so no idea)
 
On Windows?
 
10:26 PM
@roganjosh I don't seem to have the 'fixed-font' or 'upload' selections...hmph
 
I don't follow
 
Ah fixed-font pops up when typing the code
 
There's no need to take any prompts from popups, it's in your court to do the formatting
There are no popups
 
Right now I only have the option to hit 'send', unless I type ' >>>(something) '
Should I make use of "fixed font" when sending code blocks?
 
You could read the guide I linked?
 
10:30 PM
@roganjosh Yeah we have windows. At first when we switched companies we couldn't do anything, it was all read only. They had an outside company shut off all our usb ports, internet and intranet access. We said okay guess we aren't doing our jobs then.
 
Yes I am reading now, ah it is only with more than 1 line of of, got it!
code*
 
10:43 PM
Another, that'll really confuse English people, is to describe problems as "insoluble" @Aran-Fey. It's technically correct, but nobody would understand what you're talking about and you'd end with a freeze-frame-high-five. But I have a hard time understanding "inflammable" when it seems to mean the same as "flammable"
In the link I sent, there is a sandbox to test your code formatting. Please make use of that before posting here @FrancescaC
 
"insoluble" is the kind of word that I would hardly even notice at first because I'm fairly sure I know what it means, but when I'm done reading the sentence I will backtrack to it and spend 20 seconds wondering if I should google it just in case
 
Did you guys already talk about Irregardless HAHHAHAH jK
 
To 99% (I'm guessing) "insoluble" means that it cannot dissolve in water. But it can also refer to a problem that can't be solved, and most people don't know that
@CelesteWilson that isn't a word
Ugh, you're making me think of all the fake words that annoy me :P
 
Alright, I'm gonna go to bed before y'all start discussing the definition of "word"
 
@CelesteWilson there's another that I think I've blotted out from my memory. "Irregardless" was a great shout
 
10:58 PM
@roganjosh Merriam-Webster says “the most frequently repeated remark about irregardless is that ‘there is no such word.’ There is such a word, however.” It has been used (mistakenly) in place of regardless since the early 1900s and has now been admitted into dictionaries. So even though it is a word, irregardless is still far from being widely accepted. And judging by the scorn it receives online, it won’t be widely accepted anytime soon. Merriam-Webster’s advice: “Use regardless instead.”
 
Ok I think I got it, so I cannot mix plain text and code, therefore, I should write my description/problem in a separate message, and then proceed to the code, indented - always? Would this solve the formatting or am I missing something?
 
@CelesteWilson Have you ever looked into the origins of "needful"?
@FrancescaC Correct. You need to separate the text from the code block
 
I think it should be explicitly written on the guide that it is perfectly fine to separate the message and in fact necessary, as I was trying to avoid spam by merging >,<
 
Did you read the guide before your first post?
 
@FrancescaC you mean something like "you can’t mix plaintext and code in a multi-line message"?
 
11:09 PM
It doesn't really work to give feedback on a guide if you didn't even read it before you posted the first time and ignored the two links directly related to the chat room
 
I mean valid feedback is always welcome
 
We have no better way to advertise the advice
 
No, I read that part but the use of "can't" doesn't really imply that it must not be used...because I could... but I mustn't...maybe I just don't like the word can't. ok anyways, I was wondering if anyone here is using a Mac and could answer a question
 
I have no idea what you just said, but ok, I guess. I'm on a mac
 
11:18 PM
Very well, I am encountering this issue specifically, but I think it could be caused by my Macs hidden influences, do you have experience? stackoverflow.com/questions/59914977/…
 
No. It's utf-8 by the looks of it, so why would it be mac's influence?
 
@FrancescaC did the answer tell you anything?
 
No, I cant use what he suggested, doing 'rb' instead of 'r'.
@roganjosh because some folks were discussing it here...stackoverflow.com/questions/38518023/…
 
I see
 
What are you trying to do?
 
11:26 PM
path = ("/Users/name/Desktop/PDB/training_set_normseq_prof/")
# list all files in the directory, assuming this directory contains only files csv files that you need to save
for file in os.listdir(path):
    df = pd.read_csv(path+file, sep="\t", header=None).astype(float)
    df = df.to_numpy()
    arr_2d = df
    result = np.all((arr_2d == 0))
    if result:
       print('2D Array contains only 0')
    else:
       print('2D Array has non-zero items ')
 
Ok, and the error is?
 
I am obtaining the list of matrices that are ALL zeroes, there are 1252 files in total, the error is:
 File "pandas/_libs/parsers.pyx", line 1917, in pandas._libs.parsers.raise_parser_error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 3131: invalid start byte
I think I will try the if-statement to .profile files only and update
 
you really need to check the extension of those files before you do anything to them
 
yes, all the files are .profile, for certain
 
Ah, I see what you meant. Your CSV files are named *.profile. In that case yes, do that
just beware that you probably have a .profile in your home direcory
 
11:38 PM
Woo, worked, okay thank you for the morale
 

« first day (4121 days earlier)      last day (810 days later) »