« first day (3721 days earlier)      last day (1230 days later) » 

5:55 AM
hey all, merry christmas everyone. i have simple question
i just learn about coroutine. and i had this st*pid idea to put async inside my thread to break out data cleaning process into a smaller batch. is this good idea?
 
@Anusha I refer to the matplotlib docs and the exapmples referenced, they are quite comprehensive and easy to approach. Seaborn and pandas both are built on top of it and will often comply with any arguments set through **kwargs.
 
 
1 hour later…
7:14 AM
Hello
from dateutil import tz
from datetime import datetime

x = str(datetime.now(tz=tz.UTC))
print(x)
Output: 2020-12-23 07:13:46.004969+00:00
But I just need this as the output: 12/23/2020 7:15:33 AM
How to achieve the same? I want it to return an aware datetime instance.
 
7:30 AM
Use datetime.strftime to format the time in whatever style you like
 
8:14 AM
This line did the magic for me :
x = datetime.now(tz=tz.UTC).strftime('%m/%d/%Y %I:%M:%S %p')
But x is of type string
I want it in date time format.
 
Do not confuse instances with their string representations. A datetime is not a string.
datetime.now() gives you the datetime instance. With that you can subtract from another datetime to get the timedelta of the elapsed time. str(some_datetime) will use datetime's default __str__ method to give you a string for output or display. some_datetime.strftime() gives you the ability to output in different formats.
 
Thanks @PaulMcG.
This line here is producing expected output for me:
x = datetime.now(tz=tz.UTC).strftime('%m/%d/%Y %I:%M:%S %p')
But my only concern is- it is in str type.
I need that in datetime format itself.
How to achieve the same?
 
x_as_datetime = datetime.now(tz=tz.UTC)
x_as_str = x_as_datetime.strftime("%m/%d/%Y etc.")
print(type(x_as_datetime))
 
If you apply strftime() on resulted x_as_datetime , x_as_str turns out to be string type.
 
That is correct. strftime returns a str.
What exactly do you mean when you say "I need that in datetime format itself."?
 
8:29 AM
I need x_as_str in datetime type instead of string!
 
just don't use strftime then
 
use x_as_datetime if you want a datetime.
 
That will be in diffrent format right. I need it in specific format i.e., 12/23/2020 08:29:55 AM and have the type as datetime.
 
This feels similar to the confusion we see sometimes when people ask how to remove the commas and brackets from a list, because when they do print(list(range(3))), they get "[0, 1, 2]". Of course, the list itself has no commas or brackets, is just the values 0, 1, and 2, and "[0, 1, 2]" is the default output string.
 
A time doesn't have a format. The representation of a time has a format.
 
8:32 AM
datetimes don't have a format.
 
x = datetime.now(tz=tz.UTC).strftime('%m/%d/%Y %I:%M:%S %p')
print(x)
xx = str(x)
xxx = datetime.strptime(xx, "%m/%d/%Y %I:%M:%S %p")
print(xxx)
 
xx = str(x) is pointless because x is already a string
 
Here x is : 12/23/2020 08:29:55 AM
and xxx is : 2020-12-23 08:29:55
But type of xxx is datetime
 
Yes, and x is a str and xxx is a datetime. Instead of just print(x), try using print(repr(x)) to better see the distinction between str and datetime.
 
xxx has got some format using strptime() and it retained it's type as datetime.
 
8:38 AM
xxx does not have a format. What you are seeing is the default format used by the datetime __str__ method.
 
Okay. Got it @PaulMcG
Thanks
 
Here are some samples using repr - strings are explicitly shown in quotes to show they are strings.
>>> from datetime import datetime
>>> print(repr(datetime.now()))
datetime.datetime(2020, 12, 23, 2, 36, 35, 218389)
>>> print(repr(datetime.now().strftime("%m/%d/%Y %H:%M:%S %p")))
'12/23/2020 02:37:14 AM'
 
Got it. :)
But let me explain my actual problem. I have the below C# code :
DateTime dt1 = DateTime.UtcNow;
 
do you also have the feeling ubuntu is getting worse and worse in the last few years? Like since around 16.04 it keeps getting worse
 
This produces output as :12/23/2020 8:39:36 AM
 
8:41 AM
Like HOW is this a thing :O askubuntu.com/questions/777711/…
 
I want equivalent of this in python.
 
And that is probably using the default ToString() method, which emits a particular format. You can customize that by passing a format string argument to ToString().
"12/23/2020 8:39:36 AM" is not the DateTime, it is the string representation of the DateTime, created by Console.Write using the DateTime.ToString() method to convert the DateTime to a string.
I'm a little surprised that such a locale-dependent format is the default output in the C# case.
The equivalent of DateTime d1 = DateTime.UtcNow; in Python is d1 = datetime.utcnow(tz=tz.UTC). The difference in output is just a formatting issue, use strftime to make it look any way you want.
One technique I've found when doing timestamp formatting in a program is to put the desired output format up in a module-level variable, and then just using the variable name in all my strftime calls. The code looks cleaner without all those ugly formatting strings, and easier to change the format when you decide it should be something different (like when you decide you need to add milliseconds or something).
 
Thanks @PaulMcG Nice explanation!
 
8:57 AM
Glad it is clicking for you now
 
9:40 AM
@Mahi008 hello. Please don't ask for help here with fresh questions on the main site as per our rules.
 
The commonly used for i in range(list): is an antipattern, but what if I need access to i-1 and i+1 in the same iteration? Is there a python way do to that without an actuall index?
 
there's also more_itertools.windowed or something
 
hmm I wonder if that is really more readable, I had to look up tee. Besides that it looks quite neat
 
10:10 AM
@AndrasDeak I'm sorry, I read the rules after posting and it was already too late to delete :)
 
10:37 AM
@Mahi008 no worries
@Hakaishin tee's mostly there for memory efficiency
 
11:09 AM
[tag:cv-pls] [tag:dupe] [What is the most efficient way to loop through dataframes with pandas?](https://stackoverflow.com/q/7837722/4909087) -> [How to iterate over rows in a DataFrame in Pandas](https://stackoverflow.com/q/16476924/4909087)

Some dude decided to reopen the question to answer it. If anyone posts an answer to this question it is either going to be "use this <insert previously discussed loopy solution here>" or "don't iterate!!!!" (something I would say). And all of these answers are already in the target, so I would be really happy if you could help me put this to bed.
oh wow that formatting messed up.
(I've already closed it once before it was reopened, so I can't mark it myself, sadly)
 
formatting doesn't work
 
evidently :-(
 
... in multiline. never has.
 
ah, yeah. slipped my mind at the time. It's ... been a while
 
 
2 hours later…
12:52 PM
Hello everyone,

I created a python project that contains virtual environment, created by pycharm.
How can I run that script from command line and tell the python script to use that virtual environment I created?
 
do you just want to run the script? Then it would be easier to just right click the file and click run
 
That script can run fine when executing it from pycharm because everything it needs is located in the venv folder, but it does not start when executed from command line, because the necessary packages are missing
i don't want to install these packages on the default system's python path
Just only in the venv folder
 
Sounds like you have to read up on virtualenvs. Every tutorial will teach you how to activate a venv
 
Will look it up. Thank you!
 
1:37 PM
Am I the only person who does not know who Monica is?
 
1:56 PM
yes
 
yes
 
I knew it.
 
73
Q: Who or what is Monica and why so much notice from SE users?

DRPI see more and more users putting profile names related to "Monica". Is there an official post of what is all this movement around "Monica"?

 
Search MSE
and discuss it elsewhere
 
2:11 PM
I don't want to discuss it here. Mainly because I am the only person who does not know who Monica is.
 
cbg all, is any(word.startswith(my_word) for word in words) the most efficient way to check if there is any word in words list that starts with my_word? or should I build a trie?
 
How long does that expression run for your use case?
And how many times are you performing that check?
 
less than a second, and I am implementing "search all possible words in a grid"
 
Contrast those times with the possible speed increase you'd get from a trie, and the development time it would take to get it right, and the maintenance burder caused by having that additional complexity in your code. Do you need it?
 
so if I end up using a trie it will be faster?
nvm you mentioned possible, didnt read it well :D
 
2:26 PM
My main point is that your line of code is very clear and readable, and you should have very good reason to want to make it a lot more complex
 
got it, but is readable code better than a possible faster algorithm if all I care about is speed?
 
If all you care about is speed then you're using the wrong language
How to improve that clear line also depends on what will change in subsequent runs: words and/or my_word
 
my_word, I will try using this and see if I actually end up getting slow results
with interviews and my way of learning so far, it was always speed before anything
 
A poor man's alternative to a proper trie could be to store a set for each word in words, and the set should contain every possible prefix of word (with length n in range(1, len(word))). Then you need any(my_word in prefix_set(word) for word in words) which might or might not be faster, at the cost of more memory
oh, even better: just build a single big set from each of these sets
all_prefixes = set.intersection(prefix_set.values()) might work for a dict of sets
or just build it in one sitting, all_prefixes = {word[:n] for word in words for n in range(1, len(word))}
(untested hypothesis)
 
thanks for the suggestion, I will try implementing this and my current approach
 
2:35 PM
and then you'd just do if my_word in all_prefixes, no loop for all the testing words
 
I have a word list of 370k words not sure how much the total memory will be after the prefix set
 
at most the size of a set with (370k * length of max word) elements, but probably significantly less
 
some words might overlap so yeah
 
and most words are not maximally long
 
looks kinda fun to do so far, let me see how far I can go before this turns the opposite
 
2:40 PM
you should also think about whether it actually solves the problem
a fast wrong solution is useless :P
by the way if you have the testing words you can also build a set from those and take an intersection with all_prefixes to get the words that match
 
thats a fast way to test, I dont have one, but shouldnt be a problem to make one
that works :D
 
2:56 PM
morning cabbages, folks
 
you are like 12 hours behind me, or you wake up late :p
 
3:25 PM
@python_learner like AD is saying, it's a compromise between code complexity and runtime. How many prefixes of length-1? -2? -3? -4? -5? (You could even come up with a hybrid solution involving list and Trie, but that's presumably not necessary).
(i.e. You could come up with a fast check that tests for say 98% of the prefixes of length -1 or -2 or -3. I've done this with regex before. Depends if your list of prefixes is statically known or dynamic)
 
I guess this is something I can look up more into then, I have never written trie codes so idk how easy / difficult they are to write
 
3:57 PM
Using a set of all prefixes may give false hits. If "waterfall" is a valid word but for some reason "water" is not, then you would get a hit for "water" because the prefix would be present. Writing a trie would be a good exercise, and would not have this problem.
 
4:07 PM
I didnt think of that, nice intuitive guess
 
I know I asked before, but how do I find the physical location in my own network of a device? Something is blocking an ip which I need, it's not in the reservations list of the dhcp server, the dhcp server doesn't know anything about it. The advanced ip scanner doesn't show me anything besides the mac address and the port scanner doesn't show any port open besides ssh. The default ssh username doesn't work. Any idea what I can do to find it?
Ah rubber ducky. I just wanted to say that wireshark doesn't show anything without testing it. But I decided I can't just assume that and see there. Wireshark shows dst: hewlettP_8c:a3:69 So it's gotta be a printer, that's a start :)
 
 
2 hours later…
5:43 PM
What is replacement for cmp in list.sort? I can't use key, I think... I have a list of tuples [(V1,E1),(V2,E2),...,(VN,EN)] and I need to sort based on formula abs(V2-V1)+abs(E2-E1). How can I do that?
 
There's always cmp_to_key
 
That seems very interesting. Hmm..
 
Hi! Does anyone know how to make a frame for a matrix? Like this: gyazo.com/3e47a39fe136101ca2eb18e324ec464d
I used np.pad but the problem is that I dont know how to put the frame as the picture like: A B C 1 2 3...
 
@BožoStojković are you sure you want regular sorting? any-to-any comparisons usually hint at other schemes, such as topological sorting.
 
@MisterMiyagi I'm not sure. It's a coding problem. I might be getting at it from a wrong angle. I have a list of tuples (vectors), and I need to return the minimal distance between any 2 vectors, distance being given by the formula above.
I solved a similar problem by sorting the list, but this one is a tad bit harder, it seems.
 
5:58 PM
you could create a list of pairs, and apply sorted/min to that.
 
yeah, I'm pretty sure sorting won't guarantee that the pair with the smallest distance will be at the start of the list
 
basically min(dist(v1, v2) for v1 in vectors for v2 in vectors if v1 != v2), where dist is your distance metric function.
if this is a coding challenge, there's probably a more clever approach that is not O(n^2)
 
@MisterMiyagi Yeah, it's a coding challenge. But I don't think the challenge is in optimization.
@MisterMiyagi This passed the first test case. Thank you.
Now to see why it doesn't pass the second one...
@Aran-Fey Well, that isn't the point of sorting. I would iterate over the sorted list, and find the smallest distance.
Oh wait..
I'm confused lol
Ok, it now passes 11 test cases, 2 to go xD
It seems v1 != v2 is the culprit on this test case.
 
use v1 is not v2, then.
 
Ah, of course. Thanks!
 
6:12 PM
Can't recommend using is or is not with tuples. Time for a proper Vector class
 
Well, it did work out all test cases. Thank you so much @MisterMiyagi @Aran-Fey
 
a = (1, 2)
b = (1, 2)

print(a is b)  # True
^ I wouldn't risk it
 
@Aran-Fey Interesting. The test input does have 2 identical tuples. I wonder why it worked.
 
We can only speculate. How/when tuples are cached/interned is all implementation details
 
@Aran-Fey Particularly since
>>> a = (1, 2)
>>> x = "hello"
>>> b = (1, 2)
>>> a is b
False
 
6:16 PM
Aran is right. tuple will only guarantee that the same object is itself. It will not guarantee that an equal object is not the initial object.
It's implementation dependent, really.
 
Well, I will just keep that in mind, close my eyes and blindly run to the next challenge. Thanks.
 
Practically, you are good to go with that approach.
I doubt the challenge actually intended to cover such grey areas.
 
I think the challenge was made when cmp was a thing. Or something.
It is at least 4 years old, from forum posts. But it was revised at some point or another.
 
FWIW, the technically correct approach is to "tag" the tuples with their identity. So instead of (V1, E1), you have (I1, V1, E1) where I1 is unique. Though at that point you might as well go for classes (as Aran mentioned), which implicitly do that for you already.
 
I seriously hope I never see anyone put random unique integers/anythings into their tuples :/
 
6:25 PM
We should play "coding horror Bingo" some day... :P
 
noooooooo
 
@MisterMiyagi I see. Was thinking about doing the same thing, but then is not just worked and I didn't want to bother with that or creating a class for that matter. :P
The point of coding challenges is to win internet points, not learn. :D (jk)
 
7:34 PM
Happy this time of the year, folks!
8
 
Merry Xmas to you too, inspectorG4dget :)
 
Hey guys, what exactly happens when you do assignments on a pandas df columns attribute. I've been handed code that has multiple areas with code thats something to the effect of of df.columns=['a','b','c'] . Is that the exact same as just dropping all other columns or is there some behind the scene differences?
 
8:06 PM
it just overwrites the column names:
In [104]: df = pd.DataFrame([[1,2,3], [4,5,6], [7,8,9]])

In [105]: df
Out[105]:
   0  1  2
0  1  2  3
1  4  5  6
2  7  8  9

In [106]: df.columns = ['a', 'b', 'c']

In [107]: df
Out[107]:
   a  b  c
0  1  2  3
1  4  5  6
2  7  8  9

In [108]: df.columns = ['d', 'e', 'f']

In [109]: df
Out[109]:
   d  e  f
0  1  2  3
1  4  5  6
2  7  8  9
 
8:26 PM
oh ok, it only accepts same number of item list as columns in df and does position-based renaming
thanks
 
9:13 PM
happy festivities, gadget =)
 
9:33 PM
@Skyler FWIW I don't like that syntax at all. I'd much prefer df = df[['a', 'b', 'c']].
I don't actually know why df.columns = ['d', 'e', 'f'] isn't an error. I don't think the columns have gone anywhere, surely?
They're re-named but they can't be implicitly dropped?
Fantastic. I've borked my Windows laptop by testing installations :/ RuntimeError: The current Numpy installation ('C:\\Users\\jpilk\\Anaconda3\\lib\\site-packages\\numpy\\__init__.py') fails to pass a sanity check due to a bug in the windows runtime. See this issue for more information: https://tinyurl.com/y3dm3h86
 
@roganjosh I was thinking something kind of like your syntax but instead as a dictionary so its not order dependent
df = df[[1:'a', 2:'b', 3:'c']]
for example
 
I think you're mixing things. This is two steps; 1 to reduce the columns and a second to use rename
 
which would also make it so that you dont need to change all columns. It would slice as written, and if you just wanted to rename a few columns but pass the whole df you could do: df = df[[1:'a',2:'b',]]
 
Yeah, but that syntax doesn't really make sense, does it? You're mixing dictionaries and list slices
 
9:49 PM
@inspectorG4dget Happy everything!
 
@roganjosh just to make sure, when you say this you are referring to your syntax or df.columns stuff, cuz if you pass a list to df.columns whose length isnt equal that returns an error
 
@holdenweb Happy everything!
 
so that syntax cant do any column reduction
 
@Skyler I think the code you've been handed and the demo by inspector is a hack
 
@roganjosh That's a naming error, isn't it?: df.columns should have been called df.column_names.
 
9:52 PM
@roganjosh there are some janky things, lets just say it was demo/prototype code that all of a sudden became production
 
I wanna be testing this stuff but I done broke my laptop, it seems, so I need to fix that :(
 
so im first doing organizational refactoring and modularizing things, while they've asked me to not yet change some of these nasty internal and janky things they've done, hopefully some of this ugly stuff can be put next on the chopping block
 
@roganjosh is it your laptop itself though, or just the anaconda env
 
@Skyler I think I borked it myself while testing a package locally that had a requirements.txt that conflicts with conda
 
9:56 PM
lol, looking at the error I think the only other time I had to really give up and do some major fixing was when I was trying to do BLAS-related stuff
 
A package that I wrote, btw. I was trying to test it out. Silly me for using the global env
 
oof
was the package doing anything nifty?
 
Nope, just had a requirements.txt file. I don't even know that that's the problem, but I'm pretty sure it is before I start trying to fix it
 
tbh tho i am curious what the package you wrote does, i dont think ive ever gotten a big gauge on what your interests are
 
www.jpilkington.com
conda update --all.... let's hope it has elves to fix my environment. It is the festive season, after all
 
10:03 PM
lol yea
is your machine production example basically a dynamic programming problem/module
 
I don't think it can be solved outside of heuristics, but maybe
Woohoo, the elves did their work :)
 
Virtual/conda environments should be considered transient, to be recreated at will. I prefer conda because audit trail: stackoverflow.com/questions/56069924/…
Reading that answer taught me I can roll back changes to the base environment!
 
ive really warmed up to conda over this year
this audit trail stuff is really cool
 
towardsdatascience.com/… gives a lot of what you need.
 
the only issue I sometimes have with conda is that sometimes the environments I end up with are kind of funky, and sometimes its very finnicky to get vscode to detect new envs
 
10:12 PM
@Skyler Sure is!
 
thanks, ive really wanted to see if i can get down a solid grasp of conda + cuda setting up so this should be helpful
 
I used conda a long time before I found out about the history fetaures. No idea when they came in.
 
I've very lightly used conda probably for like 6 years, (though fankly I was a pretty crappy programmer for most of that) but only a fair bit more for like the last year
 
merry xmas folks.
@Skyler what IDE do you work with when working with conda?
 
10:41 PM
@AkshaySehgal so far I've really taken a liking to how lightweight vscode is, add a couple of extension to bind to the right environment. Basically I only use vscode or JupyterLab (as oppose to jupyter notebook) when I do any work in python
 
Ah ok, it took me a while to figure out how venv work with juypter. specifically with the modifications with kernel.json. Maybe this would be useful for you if you are having trouble with it. github.com/Akshaysehgal2005/Projects/blob/master/…
I am not quite sure about VSCODE though, but i would assume it would be somewhat similar.
 
my trick for doing jupyter related stuff is just to always make sure I've started up my conda virtual environment and to start jupyter from there
(not really a trick, but I guess just making sure nothing unexpected is happening)
 
oh, yea thats one way to do it, but if you dont wanna start the venv and then start jupyter but wanna switch between kernels and venv on the fly check the link.
took me a while to collate that .. once you understand it, you dont need any helper commands for it because in certain cases i find those fail to work properlt
properly*
tldr of that is, when you setup a venv, you setup a new interpreter. activating venv is basically turning on and switching to that python interpreter. .. so, the kernel.json for jupyter holds the info and path to that interpreter
once you can set that up, you can basically switch interpreters without explicitly activating venv beforehand.
and do that switch on the fly while working in jupyter.
 
yea you can get a lot done in jupyter
at some point I basically turned my jupyter env into a pyspark managing instance with direct sql query support, basically like a databricks notebook but local
 
my most painful venv to setup was the new tensorflow for mac API that is still in beta
basically tries to bring GPU tensorflow for mac.
took me days to figure out how to get that running.
@Skyler h20? or just standard pyspark?
 
11:04 PM
this was just spark
 
ah ok, so this would be with a single cluster that is your local right?
 

« first day (3721 days earlier)      last day (1230 days later) »