hey , i forgot how to parse an ajax data using flask
@app.route("/ajax", methods=['post'])
def ajax():
print(f'\n\n RE : {request} \n\n ')
test = request.form['hi']
return jsonify(test=test)
request is printed as : <Request 'http://127.0.0.1:5000/ajax' [POST]>
not sure what is the right way to parse it
what I've sent was "hi=user&test=test2"
I tried to print(test) , I got an error
werkzeug.exceptions.BadRequestKeyError: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.
KeyError: 'hi'
@LoopingDev There isn't such a thing as "AJAX data". AJAX is an approach to send data and it depends what you sent
request is printed as : <Request '127.0.0.1:5000/ajax' [POST]> well of course it will be, because you're invoking __repr__ (or __str__ I haven't checked) on the request object
what I've sent was "hi=user&test=test2" is a GET request, but your method is set up to only allow POST
The more I look at what you've posted, the more confused I'm becoming. I don't understand how you're printing an exception in a route that would have rejected the request off-hand :/
I'm getting 'latin-1' codec can't encode characters in position 59-60: ordinal not in range(256) when running a raw query in Django with cursor.execute(sql) and the query is something like sql = "insert into foo(a, b) values ({}, {});".format(adapt(row.a), row.b) and row.a and row.b is data I read from file. I set the connection charset in settings.py to utf-8, also when I try sys.stdout.encoding I get utf8, also when I do SHOW SERVER_ENCODING; I get UTF8
The data isn't uploaded by a user, it is company's files that are classified and no one has access to them other than the company. Trust me, if anyone had, it would be a bigger problem than an injection vulnerability
"I am a very curious individual, so I ask a lot of questions and I like to understand how things work under the hood" is a bit at odds with "I write shoddy code on purpose"
@roganjosh What do you mean parameterization? Isn't the only difference that I used .format with {} instead of %s, which I frankly don't know the difference between, I am quite new to python.
Oh I see what you mean, instead of replacing the string, you are passing the parameters to the execute function, which is also better against injection and makes Andras wet
@Silidrone "I really don't care about that right now" is an unusual way to put "I have no idea what you're talking about". Next time say that you don't know something rather than dismissing concerns from experts and we might start with the teaching instead of jumping through hoops.
@AndrasDeak I do know what injection is and why it is a problem, the problem is, I don't care about it because it doesn't actually matter in my case. As I said, if anyone accesses the files, it is as good as he hacked the whole DB
Basically, just remember that any arbitrary string can be executed as a subquery and then do whatever it wants. You need to escape strings so they can't be considered as part of the actual query string
@roganjosh Yeah I mean I did know that but in my particular case I didn't care as I said, otherwise, when making regular websites, where a user can actually do the injection, I watch out for that. Anyway, thanks for pointing it out.
@roganjosh Okay so I took out row.a and just tried to print when surrounded with adapt, just to see what happens. So I tried: print(adapt(row.a)) and it gives me the latin-1 encoding error, which means adapt is the problem. One would think... I try: cursor.execute("""INSERT INTO foo(a, b) VALUES (%s, %s)""", (adapt(row.a), row.b)) and it works. How come it works with parameters but not with string formatting.
I agree it's better to use parameters to exectue function rather than formatting the string and passing that and I won't use the string formatting anyway, my issue is solved, I just don't understand why would it work this way and not the other. I was almost sure it is adapt, but now I'm confused.
Well, it could only be that either a) the encoding is coerced by your postgres setup at the point of insertion, or b) it's just a string value and it's stored in whatever format you want. I'm not the "encoding guy"; I just can't get my head around it properly
Hi, can someone help me with scraping something from a webpage using Python?
In this website: https://www.walgreens.com/findcare/vaccination/covid-19/location-screening If you enter a zipcode you get a message if appointment is available or not.. I want to get that output via Python
@roganjosh Assuming its a): when I pass row.a as a parameter to the execute function, it changes the encoding of the string to the db encoding (currently utf8) but when I just put it in the query as a string, it just executes the given query so an error occurs?
Still it's weird, because print(adapt(row.a)) doesnt work
Well if it's (a) you gave postgres absolutely no attempt at changing the encoding before insertion because it's just a flat query string. At least with the correct approach it can inspect the value in isolation
Building a portal where people can see if there is an appointment available to take Covid vaccine, rather than checking every website individually @roganjosh
Yeah I just tried adapt(str) without the print and the error doesn't occur, so the error doesn't actually occur inside the adapt function, but when trying to use the string it returns
@AjithKumar this is pretty heavy for those sites involved. Are you sure that by doing this, you don't make it harder for people who go to one site directly?
@AjithKumar "yes" doesn't fill me with confidence when you're just asking for blanket help. How have you determined that your prototype doesn't impact those sites?
well I wrote this based on what I've learned from the examples over the internet for example this is cs50 the famous Course from Harvard university https://youtu.be/xMs4ER1rcLg?t=5006 , you can see in the example he used POST not GET
I've been using python on a daily basis but I've stopped a year ago because I was working on a c# project. so things are getting mixed up and I forgot some of what I learned and practiced.
there arent that many ajax(vanilla js) -(python)flask examples out there, but I have checked plenty and yet I couldnt find what I need, also thinking about using socketIO instead of Ajax
@roganjosh thank you so much looks quite informative!
but I wonder if it's ok to use socketIO as a substitute for ajax as my application already uses sockeitIO for chat and other stuff. not sure if that is a bad idea or not.
Miguel is a good resource for Flask stuff. It's not a case of whether it will work, it's a case of what your application is and whether one or the other is more appropriate
if im correct , in my case socket IO would be the best choice since I have already been using thing throwout this entire project. but I'm a little concered about it's secureity , is socket IO vulnerable in anyway more than Ajax ? I assume I could use tockens on the forms I wanted to submit just as if i am using ajax.
@roganjosh my applications organize and organization with notifications , tasks and so on , so since I'm already using notifications using socketIO then might as well use it for posting data to the server.
it make sense since i already have websocket running on the server anyway, not sure if ajax would add any benefit to the app
am calling the following JSON-EndPoint under a normal for loop where i keep getting a new token for reference_number on each request. am trying now to send an async requests and receive new value for each of the requests but i failed as i keep receive the same value.
i created new client for each request. but unfortunately it's still fail.
oh just to learn .. I came across it and i wanted to learn all methods just to expand my knowelage but the cs50 lecture explained how to use form data which would cover my purposes as well
lets try and see
` test = data.json['hi'] AttributeError: 'bytes' object has no attribute 'json'`
i forgot that im sending a json object, it must been one of the things i tried and i forgot to take it out.lol
function ajaxTest(){
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
var response = JSON.parse(req.responseText)
// what to do if success ?
console.log(' success ')
}
};
xhttp.open('POST', '/ajax')
xhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded")
var d = JSON.stringify({"hi":"hi Again","bye":"see you around "});
xhttp.send(d);
i'm actually still sending a json object , i should use data to parse it
but let me try the example i saw earlier that i thought i was using
I am reading some piece of django code and there is a ModelSerializer, and first in the code, is_valid is called, and if it returns true, then it calls .save on the serializer instance. However, there is no create or update method defined inside the serializer, how is that possible? Does Django have a default create method it calls when .save() is called? if so, how does it work?
@roganjosh oh yea that also was part of an example i found on github, because i wasnt able to get it done on my own i looked around for examples and kept trying different things.
but the video i have seen explains how to ajax formdata just fine, I'm just trying to learn the other methods as well.
I don't think that's going to be a constructive discussion for this room, though. We can try answer practical questions but not "I'm trying every method I can and things aren't working". That's a burden you're putting on yourself
oh just to learn .. I came across it and i wanted to learn all methods just to expand my knowelage but the cs50 lecture explained how to use form data which would cover my purposes as well
exactly fair enough, but It would be helpful if anyone could point me to any example for "get_data()" I just wanna learn but I will use Json/formData for now as there are more resources on them than data.
I don't know any sorry. You think I have the luxury of time to explore things that aren't immediately applicable to my work? <insert hearty laugh here>
The person I'm thinking of is a regular of the room, so you might get in luck and they'll respond, but it's also possible I'm misremembering their deployment
I made a deliberate effort not to mention names because I don't want particular people hassled by people I don't know. If we've interacted a lot (positively) then I don't mind being pinged out-of-the-blue, but I don't want to put anyone in the firing line, especially if I don't know what it might lead to
The extra rules kinda scared me when I saw them the first time. But really, in hindsight it explains why my scripts using splitlines() did what I wanted, not what I thought I was correct.
'\n'.join(text.splitlines()) will turn all kinds of characters into newlines - vertical tabs, form feeds, etc. I guess that's sometimes okay, but usually you'd want to preserve the original formatting, no?
oh, I have this class with a column "type" `type = db.Column(db.Integer, nullable=False)`
there are only 4 types that I need to record so Instead of making it string, i believe numbers are less intense so i'm looking for a way to use enum or numbers to define the 4 types
lets say those are the tables .. numbers normally takes less space than strings , on the long run if i could use numbers instead of strings , that would be more optimize to the system
If the df already exists in that format, then I (personally) don't know a better approach. I assume the multi-indexed df is a given? Not that I'm sure how I'd approach it if it wasn't a df, but it gives some flexibility
I was hoping pandas could address that , but apparently it can't, atleast without repeating the array and then comparing. I guess it should like numpy broadcasts
@roganjosh the df.shape is a given , true
I have been using an outdated pandas and I am quite inexperienced with numpy (incase you are wondering why I posted)
Yes, that makes sense. Each person has a single entry that will get an id and that is the foreign key in every other table that references their transactions
If definitely does better, but I don't think you're missing anything in your numpy approach. If anything, maybe Andras will come blow us out of the water :P
@anky oh, did you mean in place of the cond = [list comphension] line?
[x == y for x, y in itertools.product(df.index.get_level_values(0), df.columns.get_level_values(0))] is just slightly faster than pd.MultiIndex.from_product([df.index.get_level_values(0), df.columns.get_level_values(0)]) - 98.6 µs ± 7.11 µs vs 1.07 ms ± 31.3 µs (so probably both much slower than your original numpy version)