« first day (4226 days earlier)      last day (947 days later) » 

03:02
Do we not have a canonical for "Why is constructing sql queries with string formatting bad and how do I fix it?"
 
1 hour later…
04:14
@SuperStormer There's this: stackoverflow.com/q/10950362/4014959 but it doesn't give much detail. However, this is pretty good: bobby-tables.com
 
2 hours later…
06:20
@SuperStormer I tend to use stackoverflow.com/q/902408/5320906 as the question is usually "why doesn't my query work?" as opposed to "Why is constructing with string formatting bad"
06:36
@LoopingDev it's a shame that you keep putting effort into this when people keep advising you that this isn't the way to approach security (none of this will obfuscate anybody with the skills needed to actually get hold of your db in the first place). Have you hashed the passwords? Do you use SSL/TLS on the db connection? Have you locked down user permissions? Have you used an encryption layer (I think MySQL uses MD5 which isn't great)?
Postgres might allow for a SHA-type encryption layer, I'd have to check
07:06
I'm still not even sure if they want to hash or encrypt the data ¯\_(ツ)_/¯
Yeah, SHA is not a type of encryption
^ That
And it was said earlier that wanting to encrypt UserId seems a bit weird. UserId is normally public. Passwords should (normally) be hashed, with a slow hashing function, not encrypted.
07:23
More on a data science level, if that is the scree plot that you got, then I would probably not use pca at all. First of, there is no clear way to determine a reasonable number of components but more importantly, you need almost all of your components to explain a good amount of the variance.
As you're working with many features, but little data things like SVM's might work better for an initial model.
07:36
Ah Microsoft got up and has chosen violence: support.mozilla.org/en-US/questions/1315433
Like I could have waited 10 years for them to give me a spell checker, also like I want ONLY spell checking in their program. So annoying. Now I have to write a tampermonkey script to reliably find the input field and set spellchecker="true". Why is the world getting worse as we speak... Angry emoji
also aran-frey thanks a bunch for showing me tampermonkey, it's doing me a great service
Outlook fixed, gmail is still resisting: (╯°□°)╯︵ ┻━┻
08:15
@vaultah Indeed. For whatever reason MySQL refers to them as encryption e.g. "Some encryption functions return strings of ASCII characters: MD5(), SHA(), SHA1(), SHA2(), STATEMENT_DIGEST(), STATEMENT_DIGEST_TEXT()."
Encryption implies decryption is possible, so it's strange to refer to a hash function as encryption.
Postgres has TDE which could be used. In any case, I think the current approach they are taking is over-complicating their setup without providing any noticeable benefit since it isn't really encryption. I guess it's a case of not doing anything or doing it properly, but not something in between
08:40
@IvoMerchiers Thanks, I'll have a look.
08:57
Hi, I am trying to encode categorical data into binary (0/1). I want to create new data frame after encoding. I tried the following code, but it throws error when assigning new values to the newly created data frame.
import category_encoders as ce

encoder= ce.OrdinalEncoder(cols=['Q8_1'],return_df=True,
                           mapping=[{'col':'Q8_1','mapping':{'1':0,'2':0,'3':0,'4':1, '5':1}}])

print(likert_df)

df = pd.DataFrame(columns=['Q8_1'])
df['Q8_1'] = encoder.fit_transform(likert_df)

print(df)
The error thrown is "ValueError: Cannot set a frame with no defined index and a value that cannot be converted to a Series"
09:28
Are you doing one-hot encoding? Pandas can do that.
Something about dummies
 
1 hour later…
10:50
I've got a file containing this text. I got it by copy-pasting it from Chrome's network inspector after I visited example.com. I'd like to send the same request using requests. Do I have to write my own little parser to turn my file into a headers dict, or is there a better way?
More abstractly, I'm curious if this is a well-defined file format. I get (almost?) identical data from Firefox's network tab, so I suspect it's either standardized or they're just plagiarizing from one another
this is what my python TCP server says:
GET /?account=okeanosexplorer HTTP/1.1
Host: 127.0.0.1:8000
Accept: application/atom+xml,application/xml;q=0.9,text/xml;q=0.8,*/*;q=0.7
DNT: 1
Accept-Encoding: gzip, deflate
User-Agent: Liferea/1.13.4 (Linux; lzone.de/liferea) AppleWebKit (KHTML, like Gecko)
Connection: Keep-Alive
when hit by my linux feed reader
@Kevin Isn't that just HTTP? You could probably send that over a socket and it'd work
this works, for what it's worth
>>> print(s)
GET / HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Connection: keep-alive
Host: example.com
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36
>>> dict(email.message_from_string('\n'.join(s.splitlines()[1:])).items())
11:05
Ok, I think that will suit my needs. Thanks :-)
golfed: dict(row.split(':', 1) for row in s.splitlines()[1:])
(well, not too golfed)
@Aran-Fey Looking at en.wikipedia.org/wiki/…, I think you're right. I didn't seriously consider the possibility that this is what the data looks like as it travels over the wire, because it's human-readable.
What signals the end of the header?
\r\n\r\n I think
Network protocols don't have nice spaces and newlines, they have length-encoded strings and fields that are only named in a document behind the IEEE's paywall, and everything is gzipped to make it extra inscrutable
@Aran-Fey So wasteful, they could simply use chr(0)
11:46
@Kevin That is quite suprising
Yeah, for a protocol that gets used a trillion times per hour, I'd expect it to be a tad denser
Change Host: example.com to Host:example.com, boom, just saved the industry a hundred million dollars
Hello, can i ask you guys about text cleaning after web-scraping ? :)
Doesn't that get http message get compressed before sending it trough the actual network? From my very limited understanding since http is defined at the application layer, there is still enough room for the other layers to make that a bit more efficient
Yeah, probably.
12:01
don't want to ruin the millon-dollar saving method though
@Darius If you're referring to this question, I'm looking at it now
Yes, about this :)
the one asked 13 minutes ago
The very same!
Installing Selenium... Please insert floppy disk 2 of 37
I'm running the script... For the record, you made a mistake in your question. You said your output was Įrengimas:': 'Dalinė apdailaNAUDINGA:Interjero dizaineriai', 'Ypatybės:': 'Nauja kanalizacijaNauja elektros instaliacija',, but when I run your code I don't get anything like that. I get this: pastebin.com/raw/yTGkDfgE
12:17
Well, yes, you get more. I probably need to fix question, but what i want in output is to remove that ''NAUDINGA:Interjero dizaineriai'' and to add spaced before capital letters
Generally, you should post the entire output of your program. If there's a lot of output, you should change your program so there's less output, and post the entire new output.
I might be misreading the code, but should the for loop only update d_list on the next iteration if the el.name is not equal to dt? My intuition says that you don't want an elif there, but something like an indented if or an AND statement
This is one part of constructing a good MCVE. Here's a specific example. Your code currently opens 25 different web pages and scrapes each one of them. Change your code so that it only opens one web page. Now your output is 96% smaller.
The ideal way to do it would be to remove selenium and include the relevant HTML in the code.
And I mean relevant HTML. Not the whole webpage.
Good idea. And you might even be able to remove all HTML-related code entirely, if you're only interested in turning the output dict into a different dict.
Your entire question could be boiled down to: I have a dictionary, {'Įrengimas:': 'Dalinė apdailaNAUDINGA:Interjero dizaineriai', 'Apsauga:': 'Šarvuotos durysKodinė laiptinės spyna'}. I want to remove "NAUDINGA:Interjero dizaineriai" from all values, and insert a space before any capital letter that's not on a word border. For example, my expected output is {'Įrengimas:': 'Dalinė apdaila', 'Apsauga:': 'Šarvuotos durys Kodinė laiptinės spyna'}. How should I do this?
12:27
I don't think that would be a good idea. Just scrape the data properly in the first place
True. If scraping the data properly is possible, then that's definitely what you should do.
I started python like few months ago and scraping like week ago... i'am just in learning phase now guys :) Will be better at questioning :))
Sometimes it's not possible, if your data source is poorly formed to begin with. If there is a table cell on the page that literally contains the exact text "Dalinė apdailaNAUDINGA:Interjero dizaineriai", then neither Selenium nor BeautifulSoup will have a way to separate the data sensibly.
Disclaimer: I have not bothered to actually look at the page to see if there is such a table cell
There's probably something like <parent><title>foo:</title><content>bar</content></parent> and the code is doing parent.get_text() instead of title.get_text() + ' ' + content.get_text()
It's good advise that they are giving, especially if you're learning. The most valuable skill you can develop as a programmer is the ability to reduce a problem to it's core. And thinking hard on things like MCVE's is a great way to train that (and coincidentally more often than not already helps you find the problem).
12:43
Ok, now that I've lectured for thirty minutes, here's some code.
import re

def clean(s):
    s = s.replace("NAUDINGA:Interjero dizaineriai", "")
    s = re.sub(r"\B([A-Z])", r" \1", s)
    return s

d = {'Įrengimas:': 'Dalinė apdailaNAUDINGA:Interjero dizaineriai', 'Apsauga:': 'Šarvuotos durysKodinė laiptinės spyna'}
d = {k: clean(v) for k,v in d.items()}
print(d)
#result: {'Įrengimas:': 'Dalinė apdaila', 'Apsauga:': 'ŠarvŠuotos durys Kodinė laiptinės spyna'}
... But I notice an issue:
print(clean("Hello, my name is AaaBbbŠccDdd"))
#output: Hello, my name is Aaa BbbŠcc Ddd
#                         problem ^
Time to pip install regex
The re docs mention how A-Z might match different things depending on your flags and locale, but I don't totally understand it
@Kevin Thank you !
@Aran-Fey if using BeautifulSoup... that's probably better written as soup.select_one('parent').get_text(' ')
Interesting, I didn't know about that
12:51
yeah... only found that out recently myself you can provide a join argument
Semi-related idle thought. Many of us have a moral rule, saying "don't answer questions that seem like they're being used for evil purposes". For example, a question starting with "I want to perform a SQL injection attack on Twitter." would usually qualify. But the truth is, Twitter wants you to SQL-inject them. They'll pay you $12,460 to do it.
Well, now many of us have an additional rule: Don't answer questions that seem like you would be scamming yourself out of 12 thousand bucks
One ought to take this into consideration when determining the moral value of a question. Don't make a snap decision as soon as you see "I want to perform a SQL injection attack on Twitter". Skim a little farther to see if they mention that they're doing it for cash, rather than for the simple delight of destroying things
yeah, destroying things for money is way more ethical
@Aran-Fey Hehe. Definitely don't hand a fully functioning exploit to the asker :-) but if their question is like "I want to hack twitter. How do I specify headers when I run requests.get("twitter.com")?", you're probably not hurting your own expected value by sharing your knowledge
13:06
True I guess
Really this is mostly for my benefit, so I can stop saying "example.com" and just name the site I'm really poking at
I might keep doing it though, to uphold the mystique
13:44
> AttributeError: 'ValidatingEntry' object has no attribute 'validate_fn'. Did you mean: 'validate_fun'?
Hmm, nope. Nothing fun here.
Well, that's why it raised an exception. Working as intended
14:08
@roganjosh actually, I've been listening, I was just going to add the userId as it's because it wont hurt much even incase of data breach but yes Iam using SSL along with pycryptodome to encrypt the semi-sentive data that I'm temporarily storing.
i'm storing it for like 30 minutes
14:22
cbg folks!
@LoopingDev umm... if you're only needing 30 mins... I'd be tempted to just do whatever and then put it in redis with a TTL
@inspectorG4dget cbg!
heya puppy! potato?
always SSDD but melon... potato?
Melon... Mushroom - Flask has me Peas
actually, if you'd have 10min, I'd love a quick chat about why my flask server is failing me so epicly
Do you see anything obviously wrong with this?
@JonClements redis? I'm confortable with postgres, is there is anything wrong with it ?
14:28
from flask import Flask, request
import json
import pickle

app = Flask(__name__)

@app.route('/', methods=['POST'])
def foo():
    print('foo')
    if request.method == 'POST':
        print("I got a POST")
    else:
        print("wtf")
        print(request.method)
    print('I got something')
    print(request.data)
    with open('form.pkl', 'wb') as outfile:
        pickle.dump(request.get_json(), outfile)
    return json.dumps(request.form)

if __name__ == '__main__':
    app.run(debug=True, port=3000)
there's a chrome extension that claims to be POSTing stuff to localhost:3000. Flask logs show 200s, but foo never seems to be called (I don't see the prints in stdout)
@LoopingDev nope... my favourite DB... just easier to set things in redis for ttl and temporary storage
@inspectorG4dget I wonder if it's possible for two Flask servers to run at the same time. If there's a zombie process floating around that's successfully handling requests, that might explain why the logs are present but the console is empty. Because the zombie doesn't have a console.
Maybe you can add the process id to the logs... flask.palletsprojects.com/en/2.1.x/logging/… may be useful
@Kevin hey thanks. That's really useful. I'm going to try this
In my .NET work I occasionally encounter dueling servers, because IIS and Visual Studio don't play nicely together, out of the box. If my breakpoints aren't being triggered, IIS has commandeered the port; if localhost 404s, it's because VS won't let anything get processed until I click its "debug" button.
@inspectorG4dget what are you trying to achieve with that flask code anyway?
(also looks like outdated flask code as fairly sure it's not massively recommended to have that __main__ anymore... and to run it using flask run)
14:43
@JonClements any time my chrome extension POSTS to localhost:3000, I want to read what's being posted and save it to a pickle. I am aware that I'm going to currently get only the last posted value as the file will keep getting overwritten, but I can deal with that later
"Why not just turn off IIS and let VS handle everything?" you ask. Or vice versa. The answer is, there are complicated extenuating circumstances, and enduring the problem indefinitely is the least painful strategy.
 
1 hour later…
16:07
What's the point of a database cursor? The only explanation I can find is that it "enables iteration over a query result" or something similar, and that just sounds like an iterable to me
Plus, that description makes no sense considering how cursors are created and used. It's not like you make a query and get a cursor as the result, no, you do cursor = conn.cursor() and then you use the cursor to interact with the db instead of using the connection directly for some reason
Is there anything like handling transactions or serving as context managers that cursors do?
Depending on what you do, having several cursors on one connection can be useful.
@AndrasDeak--СлаваУкраїні Not as far as I can tell
Says the guy who had to painfully discover that multiplexing over SSH is apparently arcane voodoo 1337 magic.
I'm working with pyodbc, in case it matters. But I think this design is ubiquitous in every database engine under the sun
@MisterMiyagi How so? What does the cursor do for me? Why can't I use the connection directly?
16:13
Well, you can have more than one cursor per connection.
Kind of like you can have more than one iterator per list.
That could still work with cursors being replaced with iter(conn), right? Is that not what Aran is talking about?
Ah, I just realized that cursors have state. It remembers the commands you executed and lets you commit or rollback.
I swear I read that the connection takes care of that stuff...
Will someone want to stab me if I don't bother with cursors and just use connection.execute() for everything?
FWIW, I don't get why connections are explicitly managed things. Establishing and keeping/sharing a connection in the background isn't too difficult.
16:19
@Aran-Fey that won't use transactions, right? If so, can that bite you in the butt?
sorry for all the vague questions, but DBs are also one of the things I don't really know, so I only have questions matching the vagueness of my notions
My understanding is that transactions are the way to handle a group of operations in a way that can't leave your database in a corrupt state when it's only done half way through. Either all of it works or none of it.
Good point. But I don't think I'll need something like that. It's a pretty small and simple db
Is there some practical cost to using a cursor? Additional level of indentation due to a context manager or something?
I'm not entirely sure. If you can re-use the same cursor for multiple queries/statements, then no. But if it implodes after every commit or something, then it'll be annoying
I just don't like working with them because, in my mind, they're a very weird object that does things it shouldn't
For example, if you do cursor.execute('select * from foo') you get the cursor itself as the result. Like... what the heck. On what planet does this design make sense
16:29
So that mutates its state so that you can then do things like cursor.fetchall() (give or take off-by-name errors)?
so I guess it's very stateful
haha, well, you got me there :D
Do you know how this state relates to actual db queries? I imagine that it can change the query and then submit the changed query when you fetch, stuff like that. So how would it work if it returned a new object from execute()?
I assume whatever it returned would have to be strongly tied to the cursor anyway, because it would all have to happen within the transaction. Or something... I probably shouldn't try to hypothesise based on my level of understanding.
16:47
I wonder if cursors help manage asynchronous access somehow? If you have a regular iterable on a list, and the list mutates while you're not looking, then you may end up with surprising results:
>>> x = [1,2,3,4]
>>> g = iter(x)
>>> next(g)
1
>>> x.insert(0, 5)
>>> next(g)
1
I can imagine some kind of smart iterator that would yield only results from the original state of the data. A straightforward approach would be to deepcopy the entire table, but I bet there are cleverer ways
If you're thinking "but surely transactions alone would be enough to prevent simultaneous access problems?", I'm also thinking that.
17:02
@AndrasDeak--СлаваУкраїні I have no idea how that works under the hood. But I don't think it's necessary for something like a Cursor to exist. (In other words, I don't think all of that state has to be stored in one object.) I would much prefer if there were multiple kinds of objects, like a QueryResults (i.e. an iterable) and a Transaction
@Kevin I'm thinking "it's not the cursor's job to take care of that". As far as I can tell, cursors have half a dozen different jobs that they all juggle at the same time for some reason. It's like the database hired 1 cursor to do the jobs of 6 people, instead of hiring 6 people.
Going by the PEP's introduction, it seems that the API was designed to encourage widespread adoption. Maybe keeping the interface small made it seem less imposing to database devs that were considering making a Python endpoint for their project.
Of course, cursors are widespread in other languages too. So this would only explain why Python went with the flow rather than coming up with something more modern and OOP-y
Oh wow, that's from 2001
Oh, I didn't know they had computers back then.
17:17
Computers have existed since 1993, when someone decided they'd invent something that can run DOOM
3
Unix was invented as a publicity stunt for Jurassic Park, to make the "it's a Unix system! I know this!" scene seem slightly less far-fetched
 
1 hour later…
user18499643
18:51
Hi anyone here having familiarity with pyodbc connection to SQL storedprocedure
@JoeTha You can ask directly, and see if someone can answer you.
@JoeTha According to the room rules (sopython.com/chatroom), "Don't ask for answers to your recent Stack Overflow questions. Those who can answer are already watching the queue on the main site."
user18499643
@Marco ok did not know the rule. thanks for letting me know
ok, no problem.
thank you both
@Aran-Fey so I peeked at sqlalchemy and it seems that connection.execute() returns a CursorResult which sorta-wraps a DB cursor, but only seems to have the kind of methods you'd want to use once the aforementioned state mutation is done. I imagine this is a lot closer to what you'd expect.
you presumably know this already, but it was interesting to me
Is it possible to capture all import statements, single-line and mult-iline? This deals with single lines, but how you extend this to multi-line? I want to be able to write the name of the module above the line as a comment.
19:03
@DrownedSuccess you mean when the code does from foo import bar you want to add a comment # foo on the previous line?
Yes, more like # import: foo, but same thing.
I want to be able to deal with code like:
    from foo import ( bar,
    baz,
    dee
    )
@AndrasDeak--СлаваУкраїні Interesting. Maybe I'll use that then
This seems more resilient, but also does single lines only. It also doesn't allow you to go line by line.
@Aran-Fey I looked at docs.sqlalchemy.org/en/14/core/connections.html#basic-usage and docs.sqlalchemy.org/en/14/core/… linked from therein (lower on the page, different anchor)
@DrownedSuccess may I ask why you want to do that? I can't think of a use case, so perhaps it's an XY problem, that's all.
I want to be able to mark it for further processing by another tool.
19:11
I see. Wouldn't it be better to use a dedicated data structure (exported to a file) instead of annotating the python source directly?
something as simple as a line number -> list of modules mapping
I meant that I mark it, so I can directly replace the comments with something else. However, the line numbers is good enough as well.
I don't see the problem with the ast solution
yeah, I was about to suggest that python's own parsing should be perfect for this
They don't give you line numbers, right?
You can access the line number with node.lineno
19:14
Oh, that's perfect! Should it handle multi-line imports automatically?
Yeah, if it's a multi-line statement lineno gives you the starting line
Oh, thanks! Let me try it.
19:27
It works! I did need to deal with relative imports by modifying it a little, but otherwise it works great!
19:59
Ok, it seems that ast deals with relative import .foo as foo
20:28
In the Python DB API I suspect that the cursor was mostly to conform to actual SQL where the cursor can literally iterate over tables. Given that you can iterate the cursor object, it remains consistent with that rather than providing immediate benefits
You can execute queries directly on the connection (termed "non-standard" in sqlite3 docs IIRC) but I'm not sure all RDBMS bindings actually allow this. On my phone so don't fancy trawling all the different docs just now
Then again, cursors wouldn't be used for set queries in SQL, so that logic has a major backfire on the other end of the spectrum
20:51
Ok, I have a list of lists [[string0,lineno0],...]. I want to insert stringi at linenoi. However, the minute I insert one, it messes up the placement of every other line after it. The elements are listed in increasing lineno.
iterate over one file and build another one, inserting the new lines
I mean look up the index on the input
for lineno, row in enumerate(rows):
    if lineno == next_special_lineno:
        print('one more thing...')
But when I inserted string0, I will have to pop() the list to get the next one, which is inefficient.
I'll require an MCVE
Are you just missing it = iter(import_data); next_special_lineno, modules = next(it)?
off-by-design error considering what you asked, just look at the general shape of the code
Oh, just make a variable that holds the next one, that works
Am I missing something or is this literally as easy as for string, lineno in reversed(list_of_lists): lines.insert(lineno, string)?
20:59
even with a list you could keep track of the "current" index (although using an iterator would be more elegant)
@Aran-Fey yeah, that could work too. I'm still mentally stuck in file content land.
@Aran-Fey I thought about it, and erroneously discarded it, thinking that it would just shift everything up, but obviously that isn't true.
(unless I misunderstood your last message)
this is the same trick as looping over backwards to pop off elements from a list
Yeah, I understand, as this would preserve the ordering for all the lines above string_i.
Wouldn't it be lineno-1 though?
21:04
if lineno are one-based then yes
Can we stop with the questions that you can easily figure out just by trying it please?
Observation: when I've been on the landing page of SO on mobile the last few times, there are no questions tagged with python or javascript. The only python ones are those that carry the python-3.x tag alone. I guess the home page was being flooded so they got blocked (or I've accidentally ignored them but I can't find those settings)
OK, not ignored
I have python in my "watched" tags and I always get a whole bunch of them on the homepage (yes, I confess, I visited the homepage)
Confirmed. Watching the tag swamps the landing page. I'm guessing that would be the reality if they didn't auto-block the tag unless it's actively being watched
Sorry for the question, but what happens when a tag is watched?
I started to watch now the python tag.
It shows up on the homepage a lot, weren't you listening? :P
But other than that, I honestly have no idea
Does it only show content on the home page about what you watch?
It used to get highlighted on the main page.
it still does
21:23
@Aran-Fey Yes, I was reading, but I didn't quite understand what happens
@AndrasDeak--СлаваУкраїні perfect, I understand now
@AndrasDeak--СлаваУкраїні I got it now, ty
It's nice
it's as useful as the home page itself
it seems so
@Aran-Fey ty, too, Aran
A curiosity: what is, generally, the approximate percentage of questions about Python related questions on SO? Any bets?
I'm thinking it's either below 5% or near 20%
21:37
Wow, I thought it was at least 40%
Given the popularity of Python today and for a long time
statistics about that come up on meta from time to time
@AndrasDeak--СлаваУкраїні nice to know
My guess is 8%
I think I've seen a stat like this on OS once, but I don't remember the numbers
"22,564,013 questions" total, "1,948,492 results" tagged
there's also python-2.7 and python-3.x etc, too lazy to look those up now
21:42
Just use stackoverflow trends
@Aran-Fey I'm impressed with the low guesses percentages, I wonder what topics are related to the other percentages
C, C++, java, javascript, PHP, C#/.NET
@roganjosh Nice
Maybe that's what I had seen
@AndrasDeak--СлаваУкраїні Hmm
So, it's circa 16% of all questions now are python
21:44
@AndrasDeak--СлаваУкраїні @AndrasDeak--СлаваУкраїні Am not doing one hot encoding or dummies. I am assigning to one when the column has value of 4/5 and assigning zero when the column has value of 1/2/3
@roganjosh Yeah!!
Nice
@YatShan can't you just check something like df.colname.isin({4, 5}) or something similar?
Ok, I can try that method
isin and sets might not play nicely, but I'm not a pandas user
@Marco well, is it? What does it say when the language is decades old and most people are using a pretty fixed set of libraries with the odd new show-stopper coming out?
21:48
@roganjosh sad, huh?
@AndrasDeak--СлаваУкраїні Ok
https://insights.stackoverflow.com/trends?tags=python%2Cc%2Csql%2Cjavascript%2Cjava%2Cc%23%2Cphp%2Cc%2B%2B

Adding all these languages, it seems it gives no more than 50% from total
@Marco I think for JavaScript it might be a bit more justifiable because entire frameworks come into existence every couple of hours (ok, slight exaggeration) that are pretty all-encompassing. The same can't really be said for Python. So the vast majority of the questions are likely to be dupes
@AndrasDeak--СлаваУкраїні yeah, ty
21:54
hey guys i have question
@AndrasDeak--СлаваУкраїні algorithm gives about 1% ahahaha
array([[[2],
[0]],

[[1],
[1]]], dtype=int64)
i have this mask
i want to select
array([[[0.4359949 , 0.02592623, 0.54966248],
[0.43532239, 0.4203678 , 0.33033482]],

[[0.20464863, 0.61927097, 0.29965467],
[0.26682728, 0.62113383, 0.52914209]]])
@AndrasDeak--СлаваУкраїні no problem :P
it's fun
@MohamedFathallah what is mask.shape, input.shape, expected_output.shape?
@Marco android/kotlin?
21:56
@roganjosh yes, it's true
There are so many you're missing. Dart and flutter, for example. Kotlin, rust, R ... No point going on, I'm sure you get the point
mask shape is (1,2,2,1)
and image is 2x2
each pixel has 3 classes
image is (1,2,2,3)
@roganjosh yeah, ty
i am trying to post answer to clarify segmentation task for someone
@roganjosh Don't forget kevinscript
21:59
@AndrasDeak--СлаваУкраїні android is very good, gives about 4%
@MohamedFathallah you said "image is 2x2" and "image is (1, 2, 2, 3)", which contradict each other and neither answers my question about input vs output
@Aran-Fey much like the blocking of python on the main page, I guess KS would skew the results too much so it's not displayed
input is (1,2,2,3) is singel image 2x2 each pixel have 3 classes (#image,H,W,classes)
mask is (1,2,2,1) (#image,H,W,selectedClass)
And what is the shape of the expected output?
(1, 2, 2, 1)?
expected output is (1,2,2,1)
selected values from image
do you want to see my code
@MohamedFathallah no, thanks
22:02
I think this is what you need @MohamedFathallah:
import numpy as np

rng = np.random.default_rng()
shape = (2, 3, 4, 5)
img = rng.random(shape)
mask = rng.integers(low=0, high=shape[-1], size=shape[:-1] + (1,))

indices = np.indices(img.shape, sparse=True)
res = img[(*indices[:-1], mask)]
it's better to have different dimension sizes across every axis to avoid accidentally working solutions
(I edited a bit just now to match your trailing singleton dimension)
>>> img[1, 2, 3, mask[1, 2, 3, 0]], res[1, 2, 3, 0]
(0.8614303057904695, 0.8614303057904695)
you should probably get rid of that trailing singleton though, it's semantically cleaner if you throw it away
that would look like this:
mask = rng.integers(low=0, high=shape[-1], size=shape[:-1])

indices = np.indices(img.shape[:-1], sparse=True)
res = img[(*indices, mask)]
>>> img[1, 2, 3, mask[1, 2, 3]], res[1, 2, 3]
(0.46942734938427966, 0.46942734938427966)
@roganjosh gives about 18% now, after adding Python-3.x tag (insights.stackoverflow.com/…) :P
22:19
Fair enough. I'm not actually all that concerned about it tbh. However, that alone might help you understand why you can't get to 100% (and actually, you'd vastly exceed 100% if you kept going)
100% across all the languages you're trying to incorporate, that is. Not just Python tags
And is more like 17% than that 16% :P
@roganjosh yeah
@roganjosh yes
Anyway, I'm glad you have a new toy. I'm not expecting any further updates on discoveries :P
I was looking at the TIOBE index page about the main programming languages, it seems that the main programming languages gives about 50% from total
@roganjosh Ok, sorry
Last comment: I just don't understand why Assembly (8th place) is more popular than SQL (9th place)
No worries :) It's just that everyone can mess about with it in their own time and there's not going to be anything really enlightening in there. For newer languages addressing similar problems, it might give a gauge on how fast the community is building between the two but, other than that, it's a curio IMO because there isn't any context other than what you imagine up yourself
@Marco It's not "popular". It's "number of questions asked". They are not necessarily correlated
@roganjosh yeah, it's true
@roganjosh I am not talking about SO
22:26
You really don't have to keep pinging me on every response
Ok, sorry
The TIOBE index will contain its own biases, just like any other survey/sampling method
@roganjosh Yeah, but... the people use so much Assembly to do what??
...
2 mins ago, by roganjosh
You really don't have to keep pinging me on every response
@Marco you
it was inadvertently, sorry again
22:28
@Marco realise
@Marco this pings?
:P
Can I interest you in a userscript that lets you lower the ping volume?
haha
Normally I would say yes, but actually they have multiple uses for me, and the ones where it doesn't, I'll just make a point about it instead :P
22:30
Say one really useful use please
Nowadays
@AndrasDeak--СлаваУкраїні Thanks i would like to know more about this like img[(*indices, mask)] , *indices is that a pointer ,where i can read more about cuz I am new to python
@MohamedFathallah it is certainly not a pointer. It's syntax for iterable unpacking. (*indices, mask) is the same as (indices[0], indices[1], indices[2], mask). You can learn these things from a good Python tutorial, which you should absolutely read before learning advanced numpy (e.g. advanced indexing involving 4d arrays).
@Marco literally everything that's making this communication possible? You seem to be of the understanding that somehow once you get higher languages, you never need to handle the lower-level stuff. What happens when there's a new CPU architecture or instruction set etc.? How do you think things keep working?
@roganjosh Yeah, I know about it
So I think that the main users are the processors industries, etc
22:34
@Aran-Fey nice! Thanks
uuuh
time to call it a day, rhubarb
@Aran-Fey hahaha, it's real
Now that society has managed to agree on a text encoding, next we have to standardize audio volume
@AndrasDeak--СлаваУкраїні ah I remember it is related to *args,**kwargs, Thanks for your help :)
@Aran-Fey I will try do that script in Assembly
22:37
I think this is going to start grating on a number of people @Marco. Please just drop it
(Note: I already had a course in college where I learned very little about assembly, I thought it was really cool)
It's very weird to go from a college-level newbie to a greybeard :P
Tactical black and white in the profile photo? Not seeing any greys there!
It's just that there were other messages in between, I found it useful to mention. I won't ping any messages now.
When everything is a shade of grey, is nothing?
22:43
I think my level for philosophising tonight has dropped to saying the inane as Yoda :P
Nothing wrong with that
Changing the subject, why does downloading files via curl take so long with each new file request? The server that is guilty of taking too long to respond? It is taking about 2 minutes after each request to start each download.
23:05
Maybe you're just downloading too much and getting rate limited
23:26
Hmm. Rate of requisition acceptance?
And yes, I am downloading too much.
23:45
Then you have your answer. We are not going to help break such limitations
I didn't imagine any help like that (now that I know that this is the problem) haha, I was just trying to understand the reason for the problem occurence
Because excess traffic costs the host money and might also interrupt the service to other people. I feel like we've discussed this in the past. Web servers aren't free and bandwidth isn't infinite
Yes, of course, I understand those reasons. And no, I've never dealt with that subject here.
Ok. Well, "The server that is guilty of taking too long to respond?" vs. "I already know I'm guilty of exceeding the data limits of the service" are very different ways of looking at the same situation, but I'm glad you got to the bottom of it
again, I didn't know the cause of the problem.
no matter how obvious is the reason for you

« first day (4226 days earlier)      last day (947 days later) »