« first day (728 days earlier)      last day (711 days later) » 

6:44 AM
in SO Close Vote Reviewers, 2 hours ago, by richardec
Nobody use my ShowRemainingVotes userscript if you are...it gets rate-limited too fast...
 
 
1 hour later…
7:58 AM
Hmm
Looks like they forgot to throttle pagination
 
 
7 hours later…
user17242583
2:30 PM
@OlegValteriswithUkraine what's your suggestion regarding my script's getting rate-limited?
 
user17242583
Note that it's not actually making any requests to the API. It's scraping stackoverflow.com/users/current?tab=votes once (or sometimes twice) for upvotes, and once (or sometimes twice) for downvotes.
 
user17242583
@OlegValteriswithUkraine So there's really a way to avoid rate-limiting that?
 
@richardec yeah, I know, I examined the source :)
@richardec first, a two-second throttle on paginated requests - the rate-limiting guide requires that
I am not sure why exactly your script is getting rate-limited that fast, so a 500ms delay can be tried first
I can make a PR if you want
 
user17242583
It only happened after I opened like 7 pages all at the same time.
 
user17242583
@OlegValteriswithUkraine I'd love that, if you don't mind!! :)
 
2:40 PM
@richardec ah, that makes more sense - in general, when scraping pages, SE's pretty tolerant to sequential scrapes, but you can get in trouble pretty quickly if requests overlap
 
user17242583
If it's not possible in that case, maybe it can be changed to a button that shows the information on click? or maybe only put it in votes panel of the activity page...
 
@richardec I'll try to do that today/tomorrow then, agreed :) might require storing a throttle value in storage for really safe scraping, but should be enough to at least throttle
 
user17242583
Okay, thank you :) Please don't feel obliged to - you're kind enough just to link to the rate-limiting page. I didn't realize there were RL rules specifically regarding scraping the web pages.
 
well, as mentioned above, I think multi-tabs problem can be easily handled with storage (maybe just localStorage to not overcomplicate things)
 
Yeah, remaining daily votes definitely seems like something that should be handled by LocalStorage, not scraping.
Or, actually, system-provided info, but now I'm just being silly.
 
2:46 PM
@richardec ah, no issues, I love making / contributing to userscripts - besides, I'd like to use the script myself too
@CodyGray well, there's no way not to scrape it to get the initial data, but then it can be stored and refreshed only periodically, thus greatly reducing the chances of being throttled
as for the system-provided info - yeah, sweet dreams...
 
Uh, why can't you just track what buttons the user clicks?
 
user17242583
@CodyGray That sounds like a very safe way...except for what if the API is used to vote?
 
How often do people do that?
I don't think I've ever used the API to vote.
 
it is an option, however, it's a bit too involved and has a higher chance of breaking due to SE "reorganising" stuff
at least with votes there is only one point of failure
 
user17242583
@CodyGray I've never done it. :D
 
2:49 PM
You've got a single point of failure for the scrape, too.
Imagine they redesign the profile pages. *cough*
 
unimaginable! /s
@CodyGray well, with tracking buttons clicked you get multiple points of failure, though (it can be argued, though, that SE is still the single point of failure :))
 
Haha
I mean, I guess
You've just got to look at clicks on the vote container. That's still just a single point of failure, it seems.
 
I mean, yeah, but don't forget that votes summary also contains close votes, and I see it as a possible extension of @richardec's script
 
Fair. But! The system does already show you that info, so a better way to get it would be scraping the close-vote dialog.
 
plus tracking button clicks requires making sure the vote actually went through
since you can't just rely on the fact the button was clicked
 
2:57 PM
Don't forget that the post wasn't deleted! :-)
 
the one with some "fun" experiments? :)
 
Ah, no. I mean that the vote you cast is returned to you if the post you voted on is deleted.
 
Glad to be of service :-)
 
I am not even sure how to handle that
 
3:56 PM
btw, quick note, @richardec - you might want to add the missing custom domains of the network like stackapps.com, superuser.com, askubuntu.com, serverfault.com, and mathoverflow.net to @include headers (as well as switch to @match headers - Tampermonkey deprecated @include headers not that long ago)
In the meantime, I made a work-in-progress PR with throttling
 
user17242583
@OlegValteriswithUkraine You're right. I was uncertain about what to add, but I was being kinda lazy.
 
btw, we have a package that autogenerates the heck out of those headers, but it needs NPM/Node.js
 
user17242583
About include vs. match - I copied that from some other userscript...I'm not too knowlegeable about this. I'll fix those later today :)
 
user17242583
@OlegValteriswithUkraine Okay, I've got those. What is it? Sounds really nice.
 
@richardec sorry, forgot a link :) It can actually fetch a live list of SE sites and generate headers for all of them
 
 
2 hours later…
6:10 PM
heya, @Rob! Either I am going crazy, or something went wrong with the 1.3.8 build of AdvancedFlagging, it still contains deprecated grid--cell Stacks classes that are not present in the latest version of the source or the dist, can you take a look? I can too since I am now a part of SOBotics, but wanted to clarify if there is a specific reasoning for that
 
 
4 hours later…
user17242583
10:24 PM
From chat.stackoverflow.com I'm trying to make requests (with fetch) to stackoverflow.com, but I'm getting CORS errors. Is there a way to solve that?
 
Cody had this problem
you might be able to find how it was solved with a chat search
alternately, it's somewhere in here: github.com/SOBotics/UserStalkerHelper/blob/master/…
 
are you making fkey requests?
scratch that, I remember
 
user17242583
My code is like this:
```
const downvote = (postId) =>
fetch(`https://stackoverflow.com/posts/${postId}/vote/3`, {
method: "POST",
headers: {
"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
},
body: `fkey=${fkey().fkey}`,
}); // 2 = upvote, 3 = downvote
```
 
the problem is that there is no Access-Control-Allow-Origin header on SE's pages
so as soon as any mismatch happens - boom
there is a long-standing declined MSE request because reasons
I tried to revive it exactly after a long discussion with Cody, but got a cold welcome even from Makyen
 
user17242583
10:36 PM
I guess I can have a local server running that acts as a proxy makes the requests to SO itself
 
user17242583
@OlegValteriswithUkraine Hmm, well he must have had a good reason then.
 
@richardec nah, Makyen just works around it via GM_xmlhttpRequest
 
user17242583
Maybe something like GM_XML_HTTP_REQUEST evades the CORS? github.com/SOBotics/UserStalkerHelper/blob/…
 
user17242583
LOL
 
since that bypasses client-side fetches
@richardec yes
that's the only workaround we have as of now
 
user17242583
10:39 PM
didn't see your message :D
 
basically, balpha dismissed it in 2013 with a "nah, do better" non-response
speaking of the post:
11
Q: Please, set some unrestrictive Access-Control-Allow-Origin headers

UndoI was working on a userscript today, which had a goal of displaying chat pings immediately at the main site. This would allow me to respond more quicker-like to chat pings. However, I was met by this fearsome warrior: Can we please set some decent header things? I want to be able to do this ...

 
user17242583
Well that's not a terrible one. It's at least a lot better than my proposed workaround! :P
 
oh, that made me notice a silly typo of mine :)
oh, well, and since balpha is no longer a part of the SE team, chances of things changing there are basically 0
 
user17242583
Oh, so balpha is part of staff again!?
 
nope
wait
wut?!
since when?
 
user17242583
10:45 PM
 
oh! Since this month!
huh
since May 8-9th it seems, no wonder I didn't know
cool
sounds like it's time to nag them about CORS again then :)
 
user17242583
haha
 
user17242583
There should be an announcement of that.
 
there actually should, come to think of it
 
user17242583
11:00 PM
Who'll write it?
 
normally, it's SE that writes it - I am surprised
 
user17242583
 
yeah, that's the usual new CM'd announcement
thought balpha deserves to be announced separately, but oh well
 

« first day (728 days earlier)      last day (711 days later) »