« first day (4489 days earlier)      last day (681 days later) » 

01:54
why is it so common for people to describe the problem with phrases like "it keeps saying"? well, yes, of course it does the same thing over again if you ask it to run the same code with the same input; that's the point of using a computer. Good job creating a reproducible example; now see about 'minimal', 'complete' and 'verifiable'
 
2 hours later…
04:17
hello, I am not sure if I should use SQL or NoSQL here, I have an ETL job, I collect around 5 data points (5 columns if its SQL), as the project grows, I would be collecting newer columns (more than 5), should I stick with SQL or is this something NoSQL can help me with?
 
4 hours later…
08:30
@Nike I'm not Aran, but there is a tutorial collection: sopython.com/wiki/What_tutorial_should_I_read%3F
@Jake ETL isn't my forté, but I can try some guiding. how are you consuming the data? writing single events is cheap in no matter what, but if you're running analyses on the target DB that'd benefit greatly from SQL then that will matter. If you're updating the shape of the data you send a lot, nosql will be easier. either way, versionize your data - every time you send a new column / update an existing one / delete one, increment the version counter you send along with it
@KarlKnechtel And I could/would have removed it from the question if that single answer wouldn't be using it. :/
@KarlKnechtel Hm, I don't think it generalises like that. Accidentally having bytes-str conflict with a file is a very specific issue that usually shouldn't be approached with the decode/encode hammer; usually, being aware of wb/wt/rb/rt is the way to go.
08:46
@Jake an example of what I mean
# my current favorite format
sink.send(
    "version": "v4",  # bump whenever the content of "payload" changes
    "timestamp": "2023-01-30T08:42:56.927479+00:00",   # don't forget timezone
    "payload":{"col_a": 25, "col_b": "rent"}  # actual data
)
@MisterMiyagi IMO, the general issue should be addressed with forethought anyway. I contributed an answer on that basis.
but I don't particularly understand what you dislike about .readlines... ?
@KarlKnechtel It pulls the entire file into memory. That is a huge waste for every usecase that just wants to iterate lines.
ah, is that all :)
because it's a demo setup, I hadn't really given any thought to whether creating a list is necessary. Of course it isn't for the code that's shown, but a real-world continuation of the code could well benefit
anyway, I think generally we only clean up style issues if they're noticeably attracting attention from comments/answers, or noticeably un-minimizing the MRE
Sorry, this is both a personal and professional pet peeve of time. Using streams inappropriately is a huge waste of resources and mostly awareness that it's even a thing is nonexistant because "it works in my tests".
I hear ya
my pet peeves are largely around Unicode and the import system
09:51
What's wrong with unicode? O.o
 
2 hours later…
12:10
is there a term to refer to functions that do things and don't have a meaningful return output vs. functions that focus on having a meaningful return output?
e.g. list.append(object) vs. something like a function sum(first_int, second_int)
@shintuku like pure function versus function with side-effects?
that's not strictly about just returning, but I guess it would somewhat describe one of the differences
that's a start, thanks
But you could make a pure append function

It would just look like

new_list = append(old_list,new_element)
right, that makes sense
13:04
@shintuku another word for functions with side effects is "mutating"
@KarlKnechtel Most file formats identify themselves with a header at the start of the file, but apparently .zip identifies itself with a footer at the end. The four-byte signature 0x06054b50 signals the "End of central directory record". It usually appears about twenty bytes before the end of the file, if the footer's optional comments field isn't used.
expressed as a bytes object, the signature is b'\x06\x05KP'. But I believe that's in little-endian. If you open a zip file in Notepad, you'll see PK, not KP.
The end of central directory record contains the address of the central directory, which will tell you where the zip data actually starts.
13:25
@Arne I wouldn't call them the same, print is a good example of function with side effects that does not mutate anything
it mutates the out-buffer
feels like a stretch
I was tempted to say it mutates the monitor. Glad I wasn't the only one thinking it ;-)
@matszwecja yeah, it is. usually with mutating you'd only mean "things that are in memory in my program"
but since you can load all system information into your program, things like write to stdout or update a file can become part of your program, so the lines are blurry
I suspect that even something like (lambda: None)() causes some small change in the process memory. Despite having strong "you know what I mean" energy, the term "mutates" may be difficult to define objectively.
14:07
How can I copy files that start with a dot, like .gitignore? Using shutil.move => works, but shutil.copy results in a permisson issue. Any ideas that do not make the code to complicated and work for python 3.9?
What exactly is the permission issue? .move should preserve permissions.
PermissionError: [Errno 13] Permission denied:
Permission denied for what?
'template\\aws-cdk-python\\.gitignore'
my file..
I mean, what are you trying to do? Read it? Write it? List the directory?
Have you checked which permission is exactly is lost? Is the owner still correct?
14:14
shutil.copy(file_path, target_file_path)
i want to copy it and I´m on windows. Any other file that does not start with a "." works perfectly
Wait, you mean you can only move the file but you cannot copy it?
that right sir
I'm on Windows and I can copy .gitignore files. Here's my code.
import shutil

#create file
with open(".gitignore", "w") as file:
    file.write("whatever")

#copy file
shutil.copy(".gitignore", ".foobar")
hm
ok, maybe the issue is somewhere else
I suspect permissions weirdness. I bet a .gitignore file created by an actual git-related process would have stricter permissions than the file I created just now using open().
Make sure you have permissions for the destination as well.
14:19
ah got my issue. Actually I´m trying to copy a mixture of files and folders and I even have some folders that start with a dot. Only the folders make the programm run into this issue
Is this windows or unix? The . shouldn't have any effect aside from visibility on unix.
its windows
I don't think you can use shutil.copy on a folder.
it works in this order
os.makedirs(os.path.dirname(target_file_path), exist_ok=True)
shutil.copy(file_path, target_file_path)
then the folder will be created upfront and afterwards the file will be copied. But this does not seem to work for folders like .vscode
but I cannot use copytree or something similar, because I have to evaluate each file on its own
shutil.copy won't work for folders, whether they start with a dot or not. Example:
import os
import shutil

#create directory
os.mkdir("vscode")

#move directory
shutil.copy("vscode", "foobar") #crashes with PermissionError: [Errno 13] Permission denied: 'vscode'
14:26
import shutil

template_files: list[str] = list(
    glob.iglob(self.template_root_path + "**/*.*", recursive=True)
)
template_dot_files: list[str] = list(
    glob.iglob(self.template_root_path + "**/.**", recursive=True)
)

files: list[str] = template_files + template_dot_files

for file_path in files:
    target_file_path: str = file_path.replace(
        self.template_root_path[:-1], self.output_path
    )
    if file_path.endswith(".j2"):
        target_file_path: str = target_file_path.split(".j2")[0]
the code basically copies the files inside the folders
If you want to selectively copy only certain files, try using the ignore argument of copytree
ah good hint, wasn't aware of this
seems to be a valid option for my usecase, thanks!
👍
I wonder if you could use path.relative_to in place of __remove_root_folder_from_path... Well, you might end up not needing either one.
14:54
nah I have to remove the root_folder, because jinja2 is using a FileSystemLoader that already takes the template_root_path
def __remove_root_folder_from_path(self, path: str) -> str:
        return os.path.relpath(path, self.template_root_path[:-1])
Whatever works, then
 
1 hour later…
16:28
cbg
any Django experts in the chat?
16:42
dontasktoask
17:08
@Arne have you done any of those?
17:53
@shintuku Functions with no return value are sometimes called "procedures"
 
2 hours later…
19:23
I can add a new item to a python dictionary , but how do i add a dictionary to another dictionary to form a nested dictionary? sorry for the basic question
@ozil dictionary keys have to be hashable... but you can just have another dictionary as a value of another key
@Nike no, I had a good mentor in Uni. And I prefer to just dig in anyway, my reading comprehension for prose has never been great
19:39
brief cbg
@JonClements - thanks. a sample , if you don't mind would be great
@Aran-Fey yeah... but the languages that make a big distinction between procedure and function are generally different in design :p
@ozil I'm not involved in this conversation, so not sure why you're asking me?
@Arne I also prefer to just dig in. It seems that Aran-Fey's suggestion wasn't meant to waste my time.
Why would it be meant to waste your time? That's the whole point of room moderation - we try to keep things around that would be useful and people will step in if it's not useful
19:55
Preferring a tutorial over experimentation was honestly the less important half of my advice. If you prefer a different method of learning, that's fine too. Just make sure you learn (the basics of) python, and not just the parts of python that you think you'll need
Once you have a solid foundation, then you can pick and choose what to learn
@roganjosh I wrote "wasn't" rather than "was". I just want to check if you might have accidentally seen it the other way?
Aran-Fey, is there a suggestion you have for learning those basics? I appreciated the list suggested by Arne, but also see that Arne hasn't tried any of those tutorials.
@Nike no, I read it correctly. What is suggests is that you expected Aran's comment to waste your time
I don't have a recommendation either. It's been a long time since I needed a python tutorial...
@roganjosh how does it suggest that?
Is English your first language? (I don't mean this in a confrontational sense)
" It seems that Aran-Fey's suggestion wasn't meant to waste my time." Says "I think he was trying to waste my time on first consideration but apparently not after further investigation"
20:09
@roganjosh I agree.
And look, English is not my first language. For me it was very clear.
@roganjosh no it doesn't!
9 messages moved to Trash can (ignore the room invites, they're incidental to moving messages. Just trying to clean up the transcript)
@ozil See next
>>> d1 = {'a': 100}
>>> d2 = {'a1': 101}
>>> d1['sub'] = d2
>>> d1
{'a': 100, 'sub': {'a1': 101}}
20:30
@HenryEcker oh hey - fairly sure I recognise you from another room/two :p
thanks @PaulMcG
I really think I want to give up the room mod side. I'm ending in arguments with lots of people and I'm always the bad guy. I'm doing it from my phone too a lot of the time
@JonClements Howdy :) I'm fairly certain I used to see three-legged dog just like you around. It's been a minute
@roganjosh I'm back now if that helps... but your choice is your own... talk about it in the backroom before you make any decisions please mate?
@HenryEcker how strange that is :p
@roganjosh hmm, from my own POV, you're doing great. I did notice one argument once, but I think you handled that fine...maybe I'm biased since I'm just lurking most of the time
20:36
@roganjosh Here in this situation that occurred just now, no one should have thought they acted wrongly, except the victim.
Wait, the victim?
I don't know, the word "victim" is a bit strong here (at least based on context)
@HenryEcker if you're going to move messages - please 1) leave it to the RO's first unless you receive flags, and 2) if you feel it necessary - move such messages to the python orobours room instead of the trash can?
@JonClements Flags were received but yes I can move to ouroboros in future
@JonClements I think in this case there was a flag.
20:37
Does the choice of trash can really matter? It's not like we recycle
@Aran-Fey Okay, no problem changing the word from "victim" to "the other party"
@Aran-Fey do you remember the problem that ffmpeg process doesn't terminate correctly?
I can see a basis behind Jon's desire to have Python chat messages moved to a specific trash can, rather than a generic one.
@ChrisP Yes
I try to make a MRE.
In that example p1.terminate() doesn't work.
But p1.communicate(b'q') terminate the process and then delete the file.
20:39
@Aran-Fey that actually makes me wonder: how would you recycle text? Deleting it? Maybe you could do something weird and use an RNN with some random distribution and use the text to create newer "good" sounding text
But in the big program this trick doesn't work. :\
@NordineLotfi Okay, I picked the wrong word so, sorry.
what's really funny is... i just instinctively wanted to go to the chat admin page to see flags lmao
@NordineLotfi Use it to train ChatGPT!
@Marco there is no need to be sorry. I wasn't judging or anything, just was voicing what I noticed...
@JonClements I see, so you can smell flags too? :O
@Aran-Fey please, don't give them any more ideas!
20:40
@Jon Also I don't think we can expect all 27 moderators + the dozen-or-so SE employees that can move chat messages, to all remember which trash can to put messages from specific rooms.
@ChrisP Hmm. The best explanation I can think of is that communicate() closes stdin/stdout/stderr, which ffmpeg takes as a sign that there's nothing more to do and it (gracefully) shuts down. But really, it shouldn't matter how you make the process shut down...
Maybe if i use pypi.org/project/python-ffmpeg it be more easier.
@roganjosh moderation from a phone is extremely difficult, and I try to avoid using chat on the phone altogether.
@Aran-Fey I really used the wrong word, I meant to use the antonym of it. I'm referring to the person I consider to be in the wrong. But I didn't understand the reference to ChatGPT. :P
@ChrisP the closest SO post I found was this: stackoverflow.com/a/63660293/12349101
20:45
@Nike agreed... but sometimes unless it's an emergency you should just let the RO team deal with...
I think I wanted to be ironic, It should be a victim in quotes.
@JonClements in this case, what I think happened was that someone from the RO team flagged for a diamond moderator.
i don't think we can assume who cast the flag in this case
Ok, no problem, thanks.
all 10k users were notified of it
20:46
@KevinB fancy seeing you here
Either that, or a non-owner flagged for the diamond moderator, but is it possible for moderators to see who it was that raised the chat flag?
moderators can see who cast the flag - 10k just users see it pop up
@Marco eh, it wasn't a good joke anyway
:P
Thanks for pointing that out. I'm a 10k user on the SE network but don't know what moderators can see.
Also, since a member of the RO team was involved in the issue that was raised in the flag, it's no surprise that the diamond moderator that was summoned by the flag, would take actions on their own (i.e. without letting the RO team handle it).
@JonClements perhaps some of these messages could be moved to the Meta Python room, if you agree.
20:52
Whatsoever you think is reasonable @Nike
You've done a good job offsetting responsibility
@roganjosh pardon me?
You're giving suggestions - what's your solution?
@roganjosh which suggestion are we talking about? The suggestion to move some of this conversation to Meta Python, or the suggestion to use a keyboard in chat?
If we move it to meta python, is it me or you picking it up?
@roganjosh I don't know what you mean by "picking it up", however the suggestion was made because,
there's at least two conversations going on here. One of them started with your message about the difficulties of moderation and Jon's comments about the movement of messages. The other one has to do with Chris P's question about ffmpeg.
The conversation about ffmpeg seems to be about Python, whereas the conversation about the difficulties of moderation and the comments about movement of messages and flags, seems to be about Meta Python. Is this correct?
21:02
Hi guys anyone know what happens behind the scene when you call print and input asynchronously?
the thing is, this only works if you enter your input before the next print
why ?
Neither print nor input is async, so how do you call them asynchronously?
using asyncio
That's... not helpful
import asyncio

flag = 1
async def user_input():
    global  flag
    while True:
        loop = asyncio.get_event_loop()
        content = await loop.run_in_executor(None, input, "> ")
        print(content)
        if content =='quit':
            flag=0

async def print_something():
    global flag
    while flag:
        await asyncio.sleep(0.1)
        print('something')


async def main():
    tasks = [user_input(), print_something()]
    await asyncio.gather(*tasks)


loop = asyncio.get_event_loop()
Works as expected, I enter quit and it exits. What should be happening?
21:10
really
u did it so fast i think.
It also outputs DeprecationWarning: There is no current event loop
wouldn't work i think
remove the await asyncio.sleep(0.1). or put a smaller sleep time
it only prints and never let me type in console
was wondering that I/O here might work singular only. printing or input. can't do it together in separate function asynchronously. means that input would end whenever you print a new data
do u agree ?
The only thing input and output have in common is that most terminals display them in the same place. I can't think of any reason why this code shouldn't work, except that your terminal is doing something stupid
so u think it's because of limitation of terminals ?
sounds reasonable.
Specifically, your terminal. Not mine
21:19
where are u testing
The default ol' Windows cmd thingy
@Nike I have to say... I've seen your message in the other room, and you seem like a reasonable person... but only after you explain yourself
@Aran-Fey ok thank you
Hello, I'm trying to classification with keras, but the predictive value for my class named 1 is always 0.
I used with 500 images for each class but the results did not change.
here is my question, classification report and what i tried stackoverflow.com/questions/75270085/…
This is sort of a semi-Python question. I have a situation where I'm parsing a scripting language and then running it as part of a Python program. Ideally, I'd like to be able to include some sane debugging where I'll be able to pause script execution, view variable values, and step through code. I'm trying to decide if I should do all of the interpreting on my own, or to compile this to something like regular Python, or byte code, and then run it from there.
The rest of it is built in Python, including the script lexing and parsing, through Sly.
@TryThinkTest how are u labeling your data?
can u describe your folder of images ?
21:34
0 and 1
i have two folders
so which line you are labeling of binary images as 0 and 1
have u tried this with flow_from_directory ?
like this
@AminS no
train_data_gen = image_gen_train.flow_from_directory(batch_size=BATCH_SIZE,
directory=train_dir,
shuffle=True,
target_size=TARGET_SIZE,
class_mode='binary')
use in this way it will work. i think there is a problem for your 1's data.
ok i am trying, thank you so much:)
and i think using ImageDataGenerator is much easier than cv2 images stuff
@TryThinkTest Your welcome :)
21:39
@AminS hım
my folder names are 0 and 1
it can be apple or banana
 
1 hour later…
23:22
stackoverflow.com/questions/75281869 Is this reproducible? It seems to me like the explicit sys.path manipulation should enable the import described unless there's a typo somewhere
Can sys.path resolve ~? I doubt it.

« first day (4489 days earlier)      last day (681 days later) »