« first day (3794 days earlier)      last day (1137 days later) » 

12:58 AM
hey , i forgot how to parse an ajax data using flask
@app.route("/ajax", methods=['post'])
def ajax():
    print(f'\n\n RE : {request} \n\n ')
    test = request.form['hi']
    return jsonify(test=test)
request is printed as : <Request 'http://127.0.0.1:5000/ajax' [POST]>
not sure what is the right way to parse it

what I've sent was "hi=user&test=test2"
I tried to print(test) , I got an error

werkzeug.exceptions.BadRequestKeyError: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.
KeyError: 'hi'
 
 
1 hour later…
2:03 AM
is it possible to ajax a string ?
 
 
5 hours later…
7:09 AM
@Aran-Fey You can try types.MappingProxy. It seems to have a smaller API, though.
 
 
1 hour later…
8:36 AM
@LoopingDev There isn't such a thing as "AJAX data". AJAX is an approach to send data and it depends what you sent
request is printed as : <Request '127.0.0.1:5000/ajax' [POST]> well of course it will be, because you're invoking __repr__ (or __str__ I haven't checked) on the request object
what I've sent was "hi=user&test=test2" is a GET request, but your method is set up to only allow POST
The more I look at what you've posted, the more confused I'm becoming. I don't understand how you're printing an exception in a route that would have rejected the request off-hand :/
 
 
1 hour later…
9:48 AM
I've just stumbled over PEP 561 by accident. Is there some recommended way to test whether a package properly integrates with the typing ecosystem?
 
I'm getting 'latin-1' codec can't encode characters in position 59-60: ordinal not in range(256) when running a raw query in Django with cursor.execute(sql) and the query is something like sql = "insert into foo(a, b) values ({}, {});".format(adapt(row.a), row.b) and row.a and row.b is data I read from file. I set the connection charset in settings.py to utf-8, also when I try sys.stdout.encoding I get utf8, also when I do SHOW SERVER_ENCODING; I get UTF8
so I am really not sure what I'm doing wrong
 
@Silidrone you wrote an injection vulnerability :'(
 
Oh my
 
I really don't care about that right now
 
Well you should, because it's often the case that these errors come about by thinking that string formatting is a shortcut
It never is
In any case, what is adapt?
 
9:53 AM
The data isn't uploaded by a user, it is company's files that are classified and no one has access to them other than the company. Trust me, if anyone had, it would be a bigger problem than an injection vulnerability
@roganjosh psycopg2's function
 
Knowing what complains about an encoding-you-do-not-use or an MCVE might be helpful.
 
@MisterMiyagi what complains? how do you mean what complains? You mean the character that causes the problem?
I found it in the file
I do not understand why it uses latin-1 encoding when I set everything to utf8
 
What do you get with cursor.execute("""INSERT INTO foo(a, b) VALUES (%s, %s)""", (row.a, row.b))?
 
@roganjosh wow, no error occurs
 
10:01 AM
And that, class, is the fallacy of "it's not open to injection so I'll take this shortcut" <drops mic>
 
@AndrasDeak First Comic 11 OCT 2016 . There goes my weekend...
 
Yeah, I still don't get why the error occurs the other way
 
"I am a very curious individual, so I ask a lot of questions and I like to understand how things work under the hood" is a bit at odds with "I write shoddy code on purpose"
 
Nor do I. I'm not familiar with adapt tbh
 
@MisterMiyagi you're welcome
 
10:03 AM
@AndrasDeak I'll try to be an adult and peacefully ignore your childish behavior, even though you are probably thrice my age.
@roganjosh oh so it is adapt that causes it?
 
Probably
 
My unfamiliarity with it probably disqualifies me from answering that question :P
 
@roganjosh hah ok, well I'll try the same and add adapt again to see if it occurs again, thanks for the help :)
 
But please just use parameterization. My code is less keystrokes and it works and it's hardened against injection
 
@roganjosh What do you mean parameterization? Isn't the only difference that I used .format with {} instead of %s, which I frankly don't know the difference between, I am quite new to python.
 
10:08 AM
hmm
 
Aha! I was just about to type that one up :P
Notice that I passed two arguments to execute
%s here is not the same as the %s of the old-style string formatting. It's an unfortunate placeholder, other dialects use ?
 
Oh I see what you mean, instead of replacing the string, you are passing the parameters to the execute function, which is also better against injection and makes Andras wet
 
Bingo. The second tuple is unpacked into the %s placeholders
 
@Silidrone "I really don't care about that right now" is an unusual way to put "I have no idea what you're talking about". Next time say that you don't know something rather than dismissing concerns from experts and we might start with the teaching instead of jumping through hoops.
 
Given that IIRC 68% of all web hacks originate from injection, I think he's perfectly right to call out bad practice
 
10:11 AM
@AndrasDeak I do know what injection is and why it is a problem, the problem is, I don't care about it because it doesn't actually matter in my case. As I said, if anyone accesses the files, it is as good as he hacked the whole DB
 
Can you please look up SQL Injection and Bobby Tables @Silidrone?
 
Ok I will, thanks for the help again.
this a good source?
Cuz I already knew what I read there
Ok no beefing here, seriously want information
How can someone SQL inject from a file that I am reading, if he/she has no access to that file?
 
We really need to have a www.isw3schoolsagoodresource.com site with just NO on the page
 
lol
ok give me a good resource
 
@Silidrone they can't but we can't know that detail ahead of time. And safe coding should be the default.
 
10:14 AM
@AndrasDeak Yes I agree.
But when I explicitly say it, no need to be salty after that too
 
Maybe this? I've only skimmed it
 
@roganjosh Ok thanks that'll be my handbook on injections.
 
Basically, just remember that any arbitrary string can be executed as a subquery and then do whatever it wants. You need to escape strings so they can't be considered as part of the actual query string
 
@roganjosh Yeah I mean I did know that but in my particular case I didn't care as I said, otherwise, when making regular websites, where a user can actually do the injection, I watch out for that. Anyway, thanks for pointing it out.
 
Sure, but also note:
13 mins ago, by roganjosh
But please just use parameterization. My code is less keystrokes and it works and it's hardened against injection
Nobody has ever given me a compelling argument for using string formatting. I can't see one objective benefit of not doing it the correct way
 
10:19 AM
@roganjosh makes life more exciting
 
I... stand corrected
 
11:00 AM
@roganjosh Okay so I took out row.a and just tried to print when surrounded with adapt, just to see what happens. So I tried: print(adapt(row.a)) and it gives me the latin-1 encoding error, which means adapt is the problem. One would think... I try: cursor.execute("""INSERT INTO foo(a, b) VALUES (%s, %s)""", (adapt(row.a), row.b)) and it works. How come it works with parameters but not with string formatting.
I agree it's better to use parameters to exectue function rather than formatting the string and passing that and I won't use the string formatting anyway, my issue is solved, I just don't understand why would it work this way and not the other. I was almost sure it is adapt, but now I'm confused.
Ah and also, print(row.a) works...
 
Well, it could only be that either a) the encoding is coerced by your postgres setup at the point of insertion, or b) it's just a string value and it's stored in whatever format you want. I'm not the "encoding guy"; I just can't get my head around it properly
 
Hi, can someone help me with scraping something from a webpage using Python?
In this website: https://www.walgreens.com/findcare/vaccination/covid-19/location-screening
If you enter a zipcode you get a message if appointment is available or not.. I want to get that output via Python
Doing it for a socially good project :)
 
What's the project?
 
@roganjosh Assuming its a): when I pass row.a as a parameter to the execute function, it changes the encoding of the string to the db encoding (currently utf8) but when I just put it in the query as a string, it just executes the given query so an error occurs?
Still it's weird, because print(adapt(row.a)) doesnt work
 
Well if it's (a) you gave postgres absolutely no attempt at changing the encoding before insertion because it's just a flat query string. At least with the correct approach it can inspect the value in isolation
 
11:07 AM
@roganjosh ah I see
 
Building a portal where people can see if there is an appointment available to take Covid vaccine, rather than checking every website individually @roganjosh
 
Yeah I just tried adapt(str) without the print and the error doesn't occur, so the error doesn't actually occur inside the adapt function, but when trying to use the string it returns
 
@Silidrone print and str.format calls str() on the argument directly. Parametrisation can do whatever with the string first.
 
@AndrasDeak Oh, that's it then, thanks.
 
Try str(adapt(...))
 
11:08 AM
@AjithKumar this is pretty heavy for those sites involved. Are you sure that by doing this, you don't make it harder for people who go to one site directly?
 
@AndrasDeak yep the latin-1 encoding error occurrs
 
I suspect adapt does not return a string. Does it?
 
@roganjosh yes, I just want to start with 2 websites to build a prototype. Walgreens and may be one more.. Can you please help?
 
Print its type
 
"Return the SQL representation of obj as an ISQLQuote" from the documentation
 
11:10 AM
yeah, not a string
 
@AjithKumar "yes" doesn't fill me with confidence when you're just asking for blanket help. How have you determined that your prototype doesn't impact those sites?
 
print(type(adapt(...))) will prove
 
@AndrasDeak psycopg2.extensions.QuotedString yeah some sort of string representation of their own
 
Given that the news is full of sites and apps that have toppled over during the pandemic, I don't think they'd appreciate more load
 
so str() gets called on it to convert it to actual string? when printing?
cuz it's psycopg2.extensions.QuotedString
 
11:12 AM
I haven't, to be honest. I am an undergrad student, just trying out the possibility.
 
Then please consider the implications of what you're doing. You may feel this is a good cause, but it will put a load on those sites
 
Will do. Can you refer me to a good resource? The parameter will be in a form that seems to be processed by react, which I am not familiar with..
 
Although I think I understand your intentions, I don't want to facilitate this project sorry
 
No worries, thanks.
 
@Silidrone yes
Define a class with a custom __str__ and print it
 
11:45 AM
well I wrote this based on what I've learned from the examples over the internet
for example this is cs50 the famous Course from Harvard university
https://youtu.be/xMs4ER1rcLg?t=5006 , you can see in the example he used POST not GET
I've been using python on a daily basis but I've stopped a year ago because I was working on a c# project. so things are getting mixed up and I forgot some of what I learned and practiced.
there arent that many ajax(vanilla js) -(python)flask examples out there, but I have checked plenty and yet I couldnt find what I need, also thinking about using socketIO instead of Ajax
 
I'll dig one of mine out
Here you go @LoopingDev this
rbrb for a little while
 
@roganjosh thank you so much looks quite informative!
but I wonder if it's ok to use socketIO as a substitute for ajax as my application already uses sockeitIO for chat and other stuff.
not sure if that is a bad idea or not.
 
12:19 PM
I don't have any opinions on what's best for your case sorry
 
12:30 PM
ok just for anyone who is interested I found this :
https://github.com/miguelgrinberg/Flask-SocketIO/issues/369

i think socketIO could be used instead of ajax if I'm not wrong
 
Miguel is a good resource for Flask stuff. It's not a case of whether it will work, it's a case of what your application is and whether one or the other is more appropriate
 
if im correct , in my case socket IO would be the best choice since I have already been using thing throwout this entire project. but I'm a little concered about it's secureity , is socket IO vulnerable in anyway more than Ajax ? I assume I could use tockens on the forms I wanted to submit just as if i am using ajax.
@roganjosh my applications organize and organization with notifications , tasks and so on , so since I'm already using notifications using socketIO then might as well use it for posting data to the server.
it make sense since i already have websocket running on the server anyway, not sure if ajax would add any benefit to the app
 
Like I said, I don't have any opinions on this. I'm not confident to have a pro/con discussion because I don't think I know enough
 
fair enough , well thanks a lot for helping me understanding ajax better :)
 
No worries :)
 
12:47 PM
am calling the following JSON-EndPoint under a normal for loop where i keep getting a new token for reference_number on each request. am trying now to send an async requests and receive new value for each of the requests but i failed as i keep receive the same value.
i created new client for each request. but unfortunately it's still fail.
here's my code
 
 
1 hour later…
1:59 PM
hey , I'm trying to learn request.get_data()

I think I'm still missing a step and there is hardly any tutorial on it.

here is my test code
@app.route("/ajax", methods=['post'])
def ajax():

    if request.method == 'POST':
        data = request.get_data()
        print(f'\n\n >> data: {data}\n\n')
        test = data[0]
        print(f'\n\n test: {test} \n\n')


    return jsonify(test=test)
Pretty straight forward and here is what was printed

>> data: b'{"hi":"hi Again","bye":"see you around "}'
test: 123
I've also tried
test = data['hi'] and i kept getting an error (`TypeError: byte indices must be integers or slices, not str`)
what i expected for test is
test = 'hi again'
not sure what 123 means
 
What version of python is this?
 
latest
3.7
 
I'm just looking now and I haven't seen bytestrings for ages. But ok, at least it's not pythoff (Python 2)
 
oh i see, i guess they have updated something
 
I don't know why you're using get_data() though. Is it just a JSON payload?
What is data = request.json['hi']?
 
2:05 PM
oh just to learn .. I came across it and i wanted to learn all methods just to expand my knowelage but the cs50 lecture explained how to use form data which would cover my purposes as well
lets try and see
` test = data.json['hi']
AttributeError: 'bytes' object has no attribute 'json'`
 
You need to show me your JS code to hope to understand this. Please post in dpaste and link here
 
sure hold on a sec
wait a minute i think i know why now lol

i forgot that im sending a json object, it must been one of the things i tried and i forgot to take it out.lol
function ajaxTest(){

         var xhttp = new XMLHttpRequest();
        xhttp.onreadystatechange = function() {
      if (this.readyState == 4 && this.status == 200) {
        var response = JSON.parse(req.responseText)
        // what to do if success ?
            console.log(' success ')
       }
     };

   xhttp.open('POST', '/ajax')
   xhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded")
   var d = JSON.stringify({"hi":"hi Again","bye":"see you around "});
   xhttp.send(d);
i'm actually still sending a json object , i should use data to parse it
but let me try the example i saw earlier that i thought i was using
where d = "hi=hi Again&bye=see you around"
 
I am reading some piece of django code and there is a ModelSerializer, and first in the code, is_valid is called, and if it returns true, then it calls .save on the serializer instance. However, there is no create or update method defined inside the serializer, how is that possible? Does Django have a default create method it calls when .save() is called? if so, how does it work?
 
I don't understand "application/x-www-form-urlencoded". Why is it not 'application/json'?
I haven't got time to watch the video you linked but I'm really confused by that general setup. I don't know what you're trying to do there
 
@roganjosh oh yea that also was part of an example i found on github, because i wasnt able to get it done on my own i looked around for examples and kept trying different things.
but the video i have seen explains how to ajax formdata just fine, I'm just trying to learn the other methods as well.
 
2:16 PM
I don't think that's going to be a constructive discussion for this room, though. We can try answer practical questions but not "I'm trying every method I can and things aren't working". That's a burden you're putting on yourself
12 mins ago, by LoopingDev
oh just to learn .. I came across it and i wanted to learn all methods just to expand my knowelage but the cs50 lecture explained how to use form data which would cover my purposes as well
That's gonna be on you, sorry
 
exactly fair enough, but It would be helpful if anyone could point me to any example for "get_data()" I just wanna learn but I will use Json/formData for now as there are more resources on them than data.
 
I don't know any sorry. You think I have the luxury of time to explore things that aren't immediately applicable to my work? <insert hearty laugh here>
 
@roganjosh sorry for any confusion, i've been trying for hours on this since yesterday so the test code started to get messy a bit.
@roganjosh well that is the thing I dont know if it;s applicable or not because i have no resource on them lol
if there is any i would spent few minutes to learn it.
 
The key word from my side was "immediately" :P
 
lol yes u are right , it's not the brightest decision at least from time perspective
 
 
1 hour later…
3:28 PM
chat is keeping roganjosh busy on a Saturday I presume
 
This is relaxation for me :P
 
Anyone here ?
 
3:44 PM
if you are new here you can read the room rules, but go ahead and ask a questions as long as it is within the rules sopython.com/chatroom
 
It is not fully about python but still related,
Can I deploy a dynamic web app on github pages?
a website where fx we use python on server side
 
What's a "dynamic" web app?
There is someone in this room that has deployed an app that uses github but I honestly don't know how that works at all
 
I think they want a free alternative to host their app, afaik github pages just lets you dump html?
 
I guess, but I have no idea how you serve them
I'm gonna go with "yes, it's possible" but I'm gonna be a chocolate fireguard in terms of helping that one
 
haha ok gotcha
its my supervisor who wanted github pages
I am more into heroku or something normal
 
3:56 PM
The person I'm thinking of is a regular of the room, so you might get in luck and they'll respond, but it's also possible I'm misremembering their deployment
 
Aran Fey's tutorial site?
 
Lol
<throws anonymity in the bin>
 
welp, there goes that
 
I found some blogs that combined heroku, flask and github pages to make a dynamic web, but I don't think I am ready for that much hassle
 
in my defense you were creating quite the build up
 
3:59 PM
I made a deliberate effort not to mention names because I don't want particular people hassled by people I don't know. If we've interacted a lot (positively) then I don't mind being pinged out-of-the-blue, but I don't want to put anyone in the firing line, especially if I don't know what it might lead to
 
fair point, my bad
this is why you are RO :D
 
:) Well it's done now. But I would ask that you keep it in mind for future. Maybe I did make too much of a cliff-hanger on that one
 
wont happen again
 
No worries. It's not like it's a rule, just what I consider etiquette :)
 
ok I asked my friends, have a nice day
 
4:08 PM
And you
 
or as Sarah Lynn says, ... nevermind
 
4:29 PM
cbg
 
Guys. would you choose "split("\n") or "splitlines()" for line spliting.
 
if you want to split on lines... use splitlines().
 
I was gonna say the same thing, but then I looked up the documentation for splitlines, and now I can only recommend split('\n')
Ok, maybe not. On 2nd thought, splitlines has a few advantages
 
The extra rules kinda scared me when I saw them the first time. But really, in hindsight it explains why my scripts using splitlines() did what I wanted, not what I thought I was correct.
 
But if you're planning to put the lines back together later, you have to remember to use splitlines(True)
 
4:43 PM
Why?
Wouldn't '\n'.join() be more predictable?
 
'\n'.join(text.splitlines()) will turn all kinds of characters into newlines - vertical tabs, form feeds, etc. I guess that's sometimes okay, but usually you'd want to preserve the original formatting, no?
 
Well, I'm thinking more about what you might do during processing. '\n'.join(text.splitlines()) is whole-hog and you wouldn't be doing that
I don't have a clear example in my head, but "you have to remember to use splitlines(True)" strikes me as odd is all
No, you're right
Sorry, I appear to have brain-farted. It makes sense now
 
5:41 PM
hey does anyone knows how to set minmum and maximum value in flask-sqlAlchemy ?
 
Does not compute...
 
ok , this is odd, maybe I can set it up on the database interface manually for now..
 
I still don't know what you're asking. You can set it manually or clarify your question
 
oh, I have this class with a column "type"
`type = db.Column(db.Integer, nullable=False)`
there are only 4 types that I need to record so Instead of making it string, i believe numbers are less intense so i'm looking for a way to use enum or numbers to define the 4 types
so i want to set minmum to (0) and maximum to (3)
 
roll back a bit. 'Instead of making it string"? wut?
 
5:53 PM
well numbers are just smaller bytes than strings, easier to upload and download
 
None of this is making sense to me
You don't want a min/max on the db, you want a table that enumerates certain things, and you link the foreign key id to another object, no?
 
true , that is the plan
 
Well, why do you need to have a min/max test?
 
lets say those are the tables .. numbers normally takes less space than strings , on the long run if i could use numbers instead of strings , that would be more optimize to the system
i'll just correspond each number to a type
 
Why do you have two sets of foreign keys?
 
6:03 PM
not to sets .. that is the same table , one in numbers and other is strings just to explain my point
 
Well then it's nonsense. You have to go with the left table, and not have a "foreign" column
 
@roganjosh wait , how could u tell whom this type belongs to without the foregin column ?
 
@LoopingDev by the id column
 
the id only defines the status
but doesnt tell who has it
 
This isn't how "normal form" works. You don't specify a foreign key column
The id is the foreign key for every other table, and "type" is the description of what that means
 
6:07 PM
Apologies for the formatting :/
 
@anky I'm gonna move it. You can't mix regular text and code blocks. You'll have to post them as separate messages
 
okk np
 
but how could u store that user (1 ) is in state ( 2 ) for example ? where do u store that ?
 
@LoopingDev in the user table
 
6:10 PM
oh I see , well i divided it in 8 tables so far to keep things easier
 
Nothing about your setup sounds easy at all. Something looks borked with those tables
@anky to be clear; it was an invitation to re-post, not that I'm gonna move your posts :P
 
you are right , the tables i'm talking about are suppose to help the accountants staffs
 
@roganjosh That was clear :P
 
invoices , transactions , histories , products , statues ...etc
we cant just add all those up in 1 table
 
I was worried about intermediate posts :) hence posted in 1
 
6:13 PM
@LoopingDev sure. I'm not intimidated by having to join lots of tables. I think your particular example suggests that something is wrong in the setup
@anky lol, chat
 
@roganjosh like what u mean ? what is wrong ?
 
I mean, I can try shut up for a few mins, but just acknowledge my sacrifice @anky :P
 
haha lol okay
 
where is the room for improvement ?
 
Lets post it better. Lets say I have a dataframe of multiple indices and columns:
df = pd.DataFrame(np.random.randint(0,100,(5,5)),
             columns=pd.MultiIndex.from_arrays((list('abcde'),list('defgh'))),
             index=pd.MultiIndex.from_arrays((list('abdfe'),list('somet'))))
print(df)
expected output:
cond = (df.index.get_level_values(0).to_numpy()[:,None]
        == df.columns.get_level_values(0).to_numpy())
print(cond)
I am trying to return a Boolean array where level=0 of index and columns are same
is there a better way?
Not that I am stuck (low priority)
but wanted to know if a better way exists
 
6:21 PM
I think I'd be hard-pushed to beat that!
If the df already exists in that format, then I (personally) don't know a better approach. I assume the multi-indexed df is a given? Not that I'm sure how I'd approach it if it wasn't a df, but it gives some flexibility
 
I was hoping pandas could address that , but apparently it can't, atleast without repeating the array and then comparing. I guess it should like numpy broadcasts
@roganjosh the df.shape is a given , true
I have been using an outdated pandas and I am quite inexperienced with numpy (incase you are wondering why I posted)
 
In that case, IMO, it's ugly but beautiful at the same time. I'll have a think but my immediate thought is that it's as good as it can be
Well, where does numpy come into this? You literally do one index operation on a numpy array
 
@roganjosh you read my mind, upgly is what I didnt want to call it and offend numpy experts :P
both are numpy arrays ?
 
Sure, but only because you make them
It's only an actual array after df.index.get_level_values(0).to_numpy()
 
else it doesnt work for me :) ahh list does
I get your point
 
6:30 PM
What I'm getting at is that I think most of the syntax you have is unavoidable because you're converting out of pandas to numpy
Eh, I typed too slowly :P
 
I should invest more time
 
@LoopingDev that you're creating a foreign key column
 
@roganjosh how do u connect tables to gether ?
how do u store that user1 is currently (online)
or user one have created an invoice ?
or a post
 
I misspoke, sorry. A base table should not have a foreign key, it just has an id
So you set the left-most column as a primary key in your left table, then your user table has a status_id column that stores that int
in that case, status_id is a foreign key. But none of your tables should need a "foreign key" column
 
oh, i see
but wont it be better if i made tables with 5 columns , 3 columns , 3 columns , 5 columns , instead of 1 ( 10 columns ) , 2 ,2,2,2 ?
 
6:38 PM
I'm lost
 
i mean by that store the foregine key of the invoice owner on the invoice table
do the same for the post , message , taxes .. etc
this way if i want to pull taxes for user1 , i could only lookin to taxes table for foregin id (1 )
 
Yes, that makes sense. Each person has a single entry that will get an id and that is the foreign key in every other table that references their transactions
 
thanks a lot :)
 
@anky not sure if this counts as "better"...
cond = [x == y for x, y in itertools.product(df.index.get_level_values(0), df.columns.get_level_values(0))]
np.array(cond).reshape(df.shape)
 
oof
%timeit anky(df)
66 µs ± 308 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

%timeit aneroid(df)
69.6 µs ± 144 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
Waaaay too small a sample to compare I guess
I'm not quite sure how to expand that test out without actual effort, given that there's a limited alphabet :P
 
6:54 PM
@roganjosh the indexes could just be numbers... the operation remains the same
only reason I came up with mine is coz I don't really get what df.index.get_level_values(0).to_numpy()[:,None] does
I mean, I "see" the result but since I don't use numpy directly much, I figure it's a reshape + add axis of some kind
 
There we go. Even with a 500x500 df:
%timeit anky(df)
276 µs ± 3.45 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit aneroid(df)
18.1 ms ± 132 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
 
wow, mine blew up badly. I blame itertools. but don't tell them I said that
 
itertools will never hold a candle to numpy (waits for a corner-case example)
 
@aneroid ummm.. intresting
@roganjosh okay thanks for that. Yeah I think numpy does better
@roganjosh agreed, example: I have seen np.tile outperform is_slice (IIRC)
 
If definitely does better, but I don't think you're missing anything in your numpy approach. If anything, maybe Andras will come blow us out of the water :P
 
7:04 PM
haha waiting for AD :) but thank you
 
I've used np.tile but it was slower than I expected. Hmmm. Maybe that's worth a revisit
 
than itertools? may be we had different DS
 
I don't think it's gonna win here because the calculation is relatively trivial
Oh, numpy should always beat itertools, no?
Ah, I should clarify. I meant I used np.tile in the past ...
 
by outperform I meant np. beats itertools, sorry
I meant the same I think, tile beats is_slice from itertools
 
Almost certainly
 
7:10 PM
yes :) my def of outperform was performs better :D
@aneroid bdw pandas have a product too : pd.MultiIndex.from_product((df.index.get_level_values(0),df.columns.get_level_values(0))) :)
 
yeah, I used that first but wasn't sure how to turn that into True/False based on the values... into a 5x5 array
 
sure :) Just wanted to make sure.
rbrb .. [f"good {i}" for i in ["morning","afternoon","evening","night"]] folks
 
7:29 PM
@anky oh, did you mean in place of the cond = [list comphension] line?
[x == y for x, y in itertools.product(df.index.get_level_values(0), df.columns.get_level_values(0))] is just slightly faster than pd.MultiIndex.from_product([df.index.get_level_values(0), df.columns.get_level_values(0)]) - 98.6 µs ± 7.11 µs vs 1.07 ms ± 31.3 µs (so probably both much slower than your original numpy version)
 
Gosh, we have very different opinions on "slightly faster" :P
 
@aneroid oh wait, scratch that - i'm missing a 0. so itertools.prod is 10x faster than pd.MultiIndex
 
:P
 
@aneroid I didnt mean that'd be faster, just wanted to let you know. itertools.product s much faster
especially on small dfs ;)
i havent tested honestly
 
@anky ah, understood.
 
7:43 PM
but I assume IDK why, pd.MultiIndex.from_product might even beat it when working with strings. havent tested though
i will confirm soon
rbrb finally. Late night here
 
8:48 PM
is there a way to make requests to a REST api on the serverside using another protocoll ?
then http
 
What?
You could try sockets and things like zmq but it's not clear what you're asking
 
9:05 PM
@randomscientist not if the REST API on the receiving side doesn't accept non-http/s requests
or are you trying to make requests from another app to the REST API service running on the same server/network ? (same problem though)
 
 
3 hours later…
11:42 PM
I have a string that represents a date in YYYYMMDD format: '20040322'
Is there a cheeky way to format that date as 2004-03-22?
 
Do you really want to transform the string, or will you end up parsing it as a datetime?
I.e. is this an XY problem?
 
I really want the string!
Javascript will ultimately parse it to a JS Date() object
I'm using slice operations right now but I'm wondering if there's something fun like a hidden use of format or something
 
I can think of many less readable options
 
haha exactly
if you think of something kind of cute I'm all ears!
 
not really
 

« first day (3794 days earlier)      last day (1137 days later) »