« first day (560 days earlier)      last day (822 days later) » 

8:54 AM
@OlegValter The only thing that can't be changed in urlfetch is the user agent.. Do you think this is reproducible in chatexchange, if user agent is fixed to Mozilla/5.0 (compatible; Google-Apps-Script)(stackoverflow.com/questions/56099139)
 
 
2 hours later…
10:33 AM
@TheMaster that is... a possibility. Although SE isn't known to be particularly sniffy about the User-Agent header, they do recommend overriding it to represent the app making the request. Given that Google overrides the UA string to be the same for everyone, it is possible that at some point the UA got blacklisted (or the IP ranges from where the requests originate). However, nothing in the response indicates the request has been denied, so... ¯\_(ใƒ„)_/¯
I will try the Über Meta this week to get this sorted out
 
 
9 hours later…
7:31 PM
0
A: Failed to log in to a website to scrape my profile name using apps script

double-beep Disable redirects by setting followRedirects to false: var options = { "method" : "post", 'payload': payload, 'muteHttpExceptions': true, "headers": { "Content-Type": "application/x-www-form-urlencoded", "User-Agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like ...

(fwiw)
 
8:13 PM
@double-beep Nice answer๐Ÿ‘ How did you figure out to stop following redirect?
 
I noticed the status code was 302
and wondered what happened if redirects were disabled
2
 
@double-beep I see. I think the issue is similar to this. Follow redirects in urlfetchapp probably made another post request instead of a get request in response to 302
 
9:04 PM
That... really should not be happening. But since it works, nice job, @double-beep ๐Ÿ‘
 
how does chatexchange handle redirects?
 
9:18 PM
@double-beep actually... I think it does not follow redirects either, so that's why...
 
chatexchange in Java uses jsoup, followRedirects is by default true
and I don't think the code specifies otherwise
the C# library also disables redirects
 
10:20 PM
@double-beep does not seem like the Jaca client disables redirects. Frankly, I am not sure why not following redirects helps unless I am missing something
 

« first day (560 days earlier)      last day (822 days later) »