« first day (1906 days earlier)      last day (1486 days later) » 

3:18 PM
@Makyen Does USRR use the same API key as my other apps? I noticed something is giving me timeouts and eating up my quota
:O my quota just jumped up to 9995 What is going on?
 
@Dharman The URRS uses it's own API key. However, the SE API quota of 10k/day/IP is shared among all uses on that IP address with any API key, unless it also has acquired an API token, which the URRS doesn't attempt to obtain. The stock settings in the URRS actually use only a couple/few hundred API requests per day, even with multiple tabs open to chat, even in multiple browsers, many chat transcript pages opened, searches, etc.
Using API quota was something I was concerned about when developing the new features of the URRS. It's something which I did try to keep at a minimum. It is possible to change the settings such that it uses more, but even at max, it's not really all that much.
 
I am not concerned about running out of it, because I make on average 2 requests a minute. What I am concerned about is that I can't keep track of backoff for some reason, because I never see it until I get error 502
 
@Dharman Your SE API quota rolled over. The quota is for a 24 hour period. That period starts upon the first request you make from the IP address, if there has been more than 24 hours since the most recent prior request from that IP address. Thus, your quota can rollover at anytime during the day.
 
But now it's back to normal. 9695...9693
How can I avoid the backoff ban?
 
@Dharman Unfortunately, the SE API is poorly designed. My experience is that it expects all applications on an IP address to be tracking when a backoff is received for any request from that IP, even if it was a different API key. Obviously, that's not actually possible, given that the requests could be coming from different processes, or even different machines.
 
3:31 PM
That is terrible. So If I receive error 502 what am I supposed to do? Just catch that error and let the application sleep for 10 minutes?
 
@Dharman In some cases, you will get an error, and an effective backoff without actually receiving a backoff, even if you only have one application. You should assume that the error is a backoff.
@Dharman In the vast majority of cases, a backoff is 10 seconds.
 
@Makyen If I ignore backoff it doesn't mean that I am banned from making requests from that IP then?
 
@Dharman If you have more than one request in flight, then you may also only get a backoff on one of them and the rest may be errors.
@Dharman If you ignore the backoff, and keep making requests, then, yes, your IP will be banned. The ban may be anywhere from 10 or 15 minutes to 24 hours, or even indefinitely. Although, I've only seen indefinite bans as a result of misconfiguration/bugs on the part of SE.
 
How does URRS know if there was a backoff received or not?
 
@Dharman It looks for the backoff property in the response.
 
3:40 PM
@Makyen Yes, but if the backoff was received on another machine, then the other script will receive error and a ban for ignoring the backoff.
 
@Dharman Yes, it will get an error, or may also get a backoff. Yes, if it keeps making requests then you may get into a ban.
As for the URRS, it's a rare situation where the URRS needs to consider it, but it does. Given that backoff is on a per-endpoint basis, there are very few times when the URRS, if running in one tab, will attempt to make requests which are to the same endpoint within a period of time shorter than 1 minute. The SE API also says that you should not make identical requests more than once per minute, so the URRS also limits itself to that time-frame.
20 messages moved from SO Close Vote Reviewers
 
@Makyen If I keep refreshing a tab in the browser and my application makes calls to API, will my application be rate limited, because of my actions as a normal user?
 
@Dharman Potentially. It's really going to depend on what your application does, how many requests it makes, how it handles backoff, etc. If you're making lots of requests, then I'd recommend that you store something in localStorage, or userscript storage, so your application can see across page loads that it needs to be complying with a backoff.
The URRS does store backoffs in localStorage, so that URRS instances in other tabs, or a reloaded tab, can comply with a backoff.
 
3:57 PM
I am trying to store the backoff information too, but the problem right now seems to be me using all these different userscripts and my bot is getting the timeouts
I can't watch for backoff everywhere.
 
4:07 PM
@Dharman No, you can't. It's just not possible. You have to design your bot with the assumption that there may be other things happening and to not use all of the available request rate. The SE API has a hard request rate limit of 30 requests/s, but my experience is that it's better to stay well below that number. You should also assume that some of that rate will be used by other applications.
If you are issuing requests such that only one is in flight at a time (e.g. make request, wait for response, make next request), then it's difficult to hit that request rate. It is possible to hit that if you have more than one request in flight at a time.
 
I make two request/minute in sequence
I doubt I am abusing any fair use policy.
 
@Dharman BTW: I've run an active instance, and an intermittent test instance, of SmokeDetector on the same IP I use for normal browsing. The only times I recall having an ongoing issue with SE API request backoff is if I've also been using a userscript which is designed to put a quite heavy load on SE API requests (multiple requests in flight, with several requests per second).
For most uses, you shouldn't be hitting the SE API rate limiting, or even backoff, unless you are making quickly lots of requests.
For the userscript which makes lots of requests, it will get a backoff after anywhere from 10 ro 20 requests to a couple hundred, depending on server load, which appears to be cyclic during the day, with high load near the end of the UTC day.
 
 
5 hours later…

« first day (1906 days earlier)      last day (1486 days later) »