« first day (2231 days earlier)      last day (903 days later) » 

4:34 AM
NOOOOOOOOO! Not META! — user4581301 30 secs ago
:)
 
 
1 hour later…
5:36 AM
@cigien I'm confused
 
That happens when a mod sees it before you, and deletes all the context.
The comment @cigien quoted here was a response to his comment (the one that does still exist), pointing out that the question is being discussed on Meta.
 
@CodyGray Ah. So that was in response to cigien's comment then? Okay that makes more sense. As does it's removal.
 
In other news... anyone have some familiarity with using the SE API?
05:17:05,951  INFO StackExchangeApiClient:135 - Backing off API for 10 sec.
05:17:15,951  INFO StackExchangeApiClient:152 - API quota left: 6922.
05:17:15,951  INFO ChatBot:293 - Doing a thing at 2021-11-28T05:17:15.951703Z...
05:17:16,008  WARN StackExchangeApiClient:107 - Failed to retrieve user information from SE API.
              java.io.IOException: HTTP 400 requesting URL "https://api.stackexchange.com/2.3/users".
              Body is: {"error_id":502,"error_message":"Violation of backoff parameter"
These are serially-recorded events. Note that I receive the back-off time in the reply, actually do the back-off (for 10 seconds, as requested, which you can see in the timestamps), yet when I make a new request >10 seconds later, the API says I violated the backoff parameter...
 
5:52 AM
@CodyGray Okay all of this makes WAAAAAAAAAY more sense. Thanks!
 
And people say context is not important! :-)
 
Who says that?
They should stop whoever they are
@CodyGray Blind guess is the 10s backoff inclusive. Would sending at 05:17:15,952 have worked?
 
Uh, hmm.
Don't know. Can I borrow your time machine?
I actually guess that is the case. Looks like the other times, when it succeeded without returning an error, it happened to be 1 millisecond extra.
 
Yeah! Blind guessing!
 
That's totally ridiculous.
 
6:05 AM
You know I was just thinking the other day that that's what we're missing around here. More blind guesses
 
6:18 AM
Not more ridiculous behavior?
 
At least on my end I have more than enough... Although if it were at least fun ridiculous that would be better.
 
Defining 10 seconds as 10001 milliseconds is not sufficiently "fun"?
 
In fairness, the backoff period is 10 seconds. Your next request is accepted at 10001 ms
 
In real fairness, my initial request was made more than 1 millisecond before the reply was sent, which contained the backoff value.
 
I'll buy that from an honest user perspective. Although from a security standpoint it's much better to let the server set the backoff time and not base it in anyway off of request timestamps (which would be the only way to use your initial request time).
 
6:25 AM
Well, OK. Let me be more pedantic.
The timestamp of the response was more than 1 millisecond before the time when I emitted the log message that recorded the backoff time that it contained.
Because I had to receive the response, unpack/uncompress it, parse it, do some branching, and output stuff to the log.
 
Okay. Yes I agree to all of this.
Slight pivot, I was thinking about making an app to help manage all of my duplicates. But I haven't had time to figure out how to authenticate the API and get setup. Maybe a project over around the New Year.
Although I see a lot more "AAARGH why?!" about the SE API than most.
 
It's not actually that hard to get up and running.
The biggest issue is that you quickly run through your quota if you're doing anything remotely useful.
 
Well then I should be good. I'm not really looking to do anything too complicated API-wise. Just a little tool to calculate how many questions I've successfully closed as duplicates and what links I used to make the closure.
I hope it'll fix the "I know I've literally closed a question like this as a duplicate before and now I can't find the thread again." issue.
 
You don't need the API for that. You can just use SEDE.
That's updated every Sunday, so it should be timely enough.
 
I'd prefer to be able to close the question through the tool. So I need the API for write access if I want to do it that way.
 
6:40 AM
Ah, yes, I suppose you would, then.
Unless the tool is a userscript, then you can just hit the endpoint.
 
But again. Very much not hitting that 10,000 daily quota
Yeah. I haven't decided which way I want to go with it.
 
What is the proposed implementation language?
There are already SE API libraries available in several languages.
Although it's pretty simple if you're using something that has HTTP requests and JSON parsers out-of-the-box.
 
NodeJS or python. More likely NodeJS.
 
Isn't NodeJS a framework?
 
Yeah. It's JavaScript. But it has some features that are unique.
 
6:53 AM
The ones that make it nodular?
 
Yeah. NodeJS is technically a JavaScript runtime environment.
And there are accommodations necessary to allow it to run outside of a browser environment.
File read/write access, for example. (which is largely prohibited in a browser context for security reasons)
 
Oh, wow. I see.
 
 
4 hours later…
10:51 AM
@CodyGray related, but inconsequential for your situation: Include backoff value (or remaining time) in API backoff violation response
 
Yeah, that would not have helped.
 
yeah, unlikely, just as an FYI for everyone (and as another bookmark of mine :)). Does the above happen when you make an isolated request? Is the call recursive to fetch multiple pages of users (does the throttle happen immediately, or after N requests)?
 
Uh.. what do you mean "the above"? A backoff violation?
 
yes, sorry :)
 
I still don't understand
When you make a request, it's basically random whether you'll get a backoff field included. If so, you need to obey it. If you don't obey it a single time, it'll start throwing errors and ratchet up the future time you need to backoff.
 
10:57 AM
context, I mean. Does the error happen on the first request to the API for this endpoint, after a set of prior requests to other parts of the API, or during a recursive (or however you prefer doing that) fetch of paginated response?
 
Oh, you mean in the case where I was seeing the failure? No, it did not happen on the first request to the API. It happened after a number of prior requests.
I am not using paginated responses.
Currently, I just fetch 1 page with the max number of items (100) for each site.
The hope is that, since I'm polling every 3 minutes, there won't be more than 100 items. But that assumption isn't (currently) even verified by checking the "has_more" field.
 
ah, ok. And you do that for how many sites of the network?
 
Currently, SO plus "superuser,askubuntu,unix,drupal,arduino,codereview,ru.stackoverflow,puzzling,travel,literature,english,ell,christianity,judaism,islam,hinduism"
That's basically an arbitrary list, as far as I'm concerned. It is just what Bhargav had before, plus I added Unix and Code Review, since Jeff and Mast are at least interested in the project.
I don't know if Bhargav was specifically interested in those sites himself, or if he previously had interest from mods on those sites (years ago, when he first spun up this project).
I will probably remove some of those as defaults. However, the bot does now support dynamically adding and deleting sites from the list.
 
so... it's, like, 16 requests, right? Are they parallelized?
 
No, all sequential.
16 requests, one page for each, respecting the backoff after each, then goes back to sleep for another 3 minutes
 
11:01 AM
all righty, and the error happens around which part of the 16 requests?
 
(Well, technically, it's a periodic 3 minute timer, so it doesn't go to sleep for 3 minutes after it's done. It will wake up in 3 minutes from the last trigger.)
Uh, no consistency.
 
randomly, I see
 
Because, apparently, it's down to whenever there happens to be a 1-ms deviation
 
and from what I've gathered from the discussion above, you definitely call later than the backoff parameter happens to be
 
Ah, nuts.
I thought I had fixed this problem. I haven't.
@OlegValter Yeah. I check after each request what the backoff value that was returned to me was, and I wait that amount of time before making the next request.
I just completely rewrote the entire code, if you're interested and/or if it would help to see the Java.
 
11:03 AM
plus a couple of milliseconds needed to process this, I suppose
 
This is the root directory. This file defines a class that contains all the logic for the SE API interactions.
 
@CodyGray will not hurt - I barely know Java to write it, but can read it just fine
 
SendRequest is the workhorse function that actually calls the API. It handles either GET or POST, although I don't do any POST at the moment. It calls two helper functions, UpdateQuota (which updates a member variable) and HandleBackoff (which does as the name implies).
GetAllUsersAsJson is the public function that is called when it's time to check a site. Remember that all sites are checked sequentially, one after the other, when it comes time to check stuff (every 3 minutes). But because GetAllUsersAsJson calls SendRequest, and SendRequest handles the backoff after each call, this should not be an issue.
Even if GetUser is called at some indeterminate time (as it would be in response to a one-off command from a user to check a specific user account), that also respects the backoff, because it calls through SendRequest.
 
I have a ridiculous suggestion - can you wire it to log parameters on error as well? I am particularly interested in the String.valueOf(siteInfo.FromDate) and String.valueOf(siteInfo.ToDate )). It does not seem likely, but there was a report on StackApps that resulted in 502 due to an issue unrelated to throttling
 
I added that stuff just now, after my discussion with Henry
But yeah, I guess it could still be an issue?
That is a new update that is meant to help avoid "blind spots" during downtime, such as when the bot automatically stops checking to fetch updates to the regex patterns and then restarts itself.
It tracks the end time for each site, so when it makes the next request, it can start at the end time.
Other significant changes: I jettisoned that "checkstyle" nonsense.
 
11:12 AM
there is also "Got 502 response after very few calls" that suggests correlation, but the number of requests made is much higher than that
 
Just kidding; that's not "significant". It has no effect whatsoever on anything other than my annoyance levels.
 
re: ^^ note the OP also had correct backoff handling
 
Yeah, I saw that rene answer, but I dismissed it because I'm not making nearly that many calls that quickly.
But the IP rate-limits are brutal.
The API is nearly unusable if you don't have a dedicated machine with its own IP address to run each thing on.
 
phew, found it, sorry, lost the report because of which fromdate and todate caught my attention: API throws improper error when "date" parameters are the same. It is not very likely, but still - since you do not hammer the API, it might be relevant. Although if you make the same 16 requests, that should not be an issue. I need to read the source a bit more, I suppose
 
I personally think this rewrite is much clearer than the old version, should be more efficient, better supports the operation in two different chat rooms (one on the SO server and one on the SE server), and also adds the 'from"/"to" date support that should hopefully avoid blindspots.
One interesting thing is that the "reboot" command is designed to just terminate the entire chat bot, exiting the Java application. The startup script on the server is designed to run the Java application continuously, in a loop, so that it will restart if there is any failure. This not only means that a "reboot" will correct any memory leaks because it actually terminates the whole app, but the same logic also keeps the overall system more robust against other unexpected failures like a crash.
@OlegValter I am not making the same 16 requests... The site is different, so they're not identical requests. Also, the "to" date is going to be different... sometimes (assuming there is enough work to be done between each site's request, and there often isn't, for the small sites).
 
11:20 AM
@CodyGray yeah, that makes it an unlikely candidate for your case, but it can still be related. You never know if /users has a low threshold configured... That said, 16 sequential requests do not sound like a problem
 
But I haven't hit a "500: internal server error". That would be a different case.
I do need to add support for handling multiple pages of users, at least for the SO case.
 
@CodyGray definitely, that is what makes it unlikely
 
Although Brock's answer does suggest that he randomly got a 502 error in one case with equal from/to dates.
So it may just return 500-range errors randomly.
 
as well as the 502 error - it was also a different endpoint, so the 500 might not happen at all here
I also have an even more ridiculous suggestion: is there a possibility that whatever hosts the bot (by this I mean anything on the same IP address) might make an unrelated request in parallel while it waits for the throttle to pass, @CodyGray?
so, to summarize the discussion, can you check:
- what are the `fromdate` and `todate` parameters when the throttle violation happens (just in case)
- if there is a possibility of an intervening unrelated request (its presence would explain *a lot*) - I did not see that the code has app-wide backoff parameter set. Btw, in ElectionBot, we opted for a global backoff - if at least one backoff happens, any other call to the API will be throttled regardless of parallelization
 
11:39 AM
@OlegValter It's on an Azure VM, so... maybe? I mean, I did consider that perhaps someone else on a nearby VM that sometimes shares an IP address might be running something that interacts with the SE API. But that does seem like a remote possibility.
@OlegValter Right... I don't have this because I don't have any parallelization.
There technically is a possibility of an intervening request from a user command, but I did not run that command. And, also, as I noted before, the API hit for that user command would execute the backoff, as would any automated API hits for the scheduled task, since all of the API hits go through a single function.
What I actually think my problem is is that I'm trying to test the bot on the same server I ultimately deploy it to.
I thought that using two different API keys would solve this problem, but it doesn't, as Makyen explained to me earlier elsewhere. Very frustrating. No matter how many times or how carefully I've read the SE API documentation on throttling, I still had managed to misinterpret or misunderstand it.
Oh, BTW, I am already printing out the from/to dates in the latest version of the code.
They're printed out from another function, but they're the same values that are passed on to the API, so it is equivalent for debugging purposes.
 
@CodyGray Yeah, I realize that - just as an idea. Because other than botched date constraint parameters (for some reason - although it does not look like they can be here, who knows) and a completely random (from the API user standpoint, at least, I am pretty sure it is obvious if we had access to their configs) 502 (for which you might just give up and, if you get it, just wait the backoff again)
@CodyGray oh, nice
anything suspicious?
 
Haven't seen another 502 error yet
 
@CodyGray yeah, quite frustrating. Well, it does not really matter how well one reads the docs if the API does not behave according to the spec IRL. Come on, I had to rereport the CORS error on ~176 (?) properly throttled (??) requests with backoff never encountered (???)
 
The quotas are just so small. How am I supposed to develop/test and deploy this with these miniscule quotas?
Although... I didn't realize the API supported upvoting, downvoting, flagging, closing, and all that. So there is actually more you can do with it than I thought.
I just don't want to, because the HTTP endpoints are so much more reliable and actually allow you to do more than 2 things before throttling.
 
@CodyGray oh, yeah, it's new, sorry :) I suppose there is no rush, but do ping me if you notice something - that is important for my bots too. Frankly, I do not expect them to matter, but you know, it's like "let's make this test to ensure you are not sick with X" kind of thing (as I am sure you understand)
 
11:50 AM
Another question, though
I notice that MetaSmoke reports when its API quota has "rolled over". When does this happen? Is it every 24 hours? And where is that documented? I can't find anything about that.
 
@CodyGray yeah, quite small. I guess they are sufficient for slow small-scale development (I haven't been able to burn through quotas even once), but anything more or less serious makes you think about not burning through it much more. And the throttling is brutal too
@CodyGray ah, that :) Well, one piece of documentation is this one:
3
A: Why is my API quota not resetting after UTC reset?

Brock AdamsUpdate: Evidence now suggests that the API quota rolls over at a unique time for each app. 24 hours after that app-key, plus IP, combination made its first API call. See this chat message. The quota does reset every 24 hours, but the time seems to wander is custom for every app-instance. ...

 
So.. how does SD know when its quota has reset? It constantly compares the new value to the previously-stored value?
I mean, that just sucks.
 
@CodyGray I actually remember that it is somewhat of an educated guess - it caused an issue once when the guess was wrong. I can dig the discussion up
@CodyGray yes, yes it does
 
Once again... web programming sucks.
It doesn't seem like it has to. Each time I do it, I feel like it isn't awful, but then I have to interface with someone else's thing, and I realize that everyone else's thing sucks, which is why the whole thing sucks.
 
@CodyGray kind of :) To be fair to my field: I've worked with more APIs that do not cause these mind warps than with those that do. Actually, SE API might be one of the worst offenders on that part. It is designed pretty well, but everything related to throttling is mind-boggling
 
11:57 AM
"Mind-boggling" is definitely how I feel, repeatedly.
 
it does not help that SE and "documentation" are, apparently, too incompatible entities. As you've noticed (and participated), most of the knowledge about how the API behaves is community-provided, which..., and I will die on that hill anytime, is not what should be the case.
 
Yeah
It makes worse the concerns about people breaking things or abusing their services because they have no choice but to just try it
 
yup. Or disregard the confusing parts entirely and only fix what actually hinders the program, which results in a lot of unnecessary noise
 
You know what is even worse?
Azure.
Daniel Widdis generously gave me access to some of his credits, but the configuration interface for the Azure services is the opposite of intuitive.
And getting it to do anything is an exercise in immense frustration.
 
my knowledge is a bit out of date, but the last time I worked with it a couple of years ago, it was a slow convoluted mess of a UX. They say it got better, though
 
12:04 PM
I can't imagine it could have ever been worse.
It is very slow, and very convoluted.
I thought I had bricked my VM earlier, so, well, I don't have anything whatsoever of importance on it, so I'll just blow it away, resetting it back to the defaults, and be back in business.
NOPE! That ain't a thing.
There is, as far as I can tell, literally no way to reset the VM (i.e., its disk image) to a "fresh" install state.
 
well, if you feel nowadays it is slow and convoluted, you should have tried it in 2018 :) I've recently (~6 months ago) seen it, and it is much better than it used to be, actually
 
You can restore from a backup, but only if you've configured backups (and I haven't, because, as I mentioned, there's nothing there that I care about), and even then only with great difficulty (I was unable to figure out how. Apparently, there's some extra stuff you need to enable, which wasn't working for me at all.)
So it seems like the only option to "reset"/"restore" a VM is to delete it and create a new one. That is pretty ridiculous, I think.
 
@CodyGray I am trying to recall if AWS has a way to do that easier, but not sure. It was always my impression that Amazon is way better than the competitors when it comes to VM provisioning (although I have only a little info on Google)
 
Yeah, I really do not think it should be that hard.
And you think, well, it's supposed to be easy enough to provision a VM, so why not just delete the old one and create a new one in order to reset it? Well, because it comes with all these accoutrements! I can't even remember what all: a network virtual device, a network physical device, an IP address group, a public IP address group... just... a whole bunch of ridiculous stuff. Plus the hard disk images, and etc., which all have to be deleted separately.
And everything has all of these interdependencies, so there's no straightforward way to delete everything.
 
oh, I took a look, and it does look like it is "a backup or a delete/create action pair" on Azure
 
12:12 PM
"action pair"
That makes it sound a lot easier than it is.
It's more like: manually attempt to delete every little thing so you can see the error message indicating what the dependencies are so you can go try to delete that other thing, only to find out what its dependencies are, rinse, repeat.
And there are just so many ridiculous things that can go wrong.
 
Yes, I know :) ~6 yeah, sounds like when you modularise to the point of having a tightly coupled mess of seemingly standalone parts that end up depending on each other
 
It took me hours to do this.
Yeah
I very much dislike that kind of superfluous modularization.
If your modules have tight dependencies, then they're not separate after all, so don't try to pretend that they are separate.
 
@CodyGray yeah, setting up cloud stuff (whatever it actually involves) usually ends up in hours of chore
 
That's ridiculous!
I could set up a physical server in substantially less time.
Even if you include unboxing and installing the OS!
 
I suppose for someone like me it is still easier to provision in the cloud, even with the UX getting in the way than setting a real server, so I guess it checks out. Actually, I just logged in to the Azure portal - jeez, that is much better than it used to be
 
12:21 PM
It kinda looks decent
Although the design is still really janky. It becomes very obvious if you resize a window or something, which completely garbles the page layout.
Why this happens, I do not know. It's not like HTML is a new technology. Yet, far too many people have not yet figured out how to use it properly.
 
@CodyGray speaking of, I just opened the VM provisioning interface - and it is really much better than I remember it all to be. And kind of looks the same as AWS (I suppose that is no surprise).
 
Why, though, does it not look like VM VirtualBox?
That is what it should look and work like.
It's not how it appears, though, that's the problem It's actually trying to use it. It's another one of those pieces of software that was apparently never used by its own designers/developers.
 
@CodyGray eh, yeah, but the worst offender here is CSS if we are to be honest. With the changes it underwent over the years, it might as well be a new language. And the number of moving parts is still ridiculously high, even when you use the neatest, newest approaches...
 
Yes, through the magic of CSS, people have figured out plenty of ways to ruin otherwise valid HTML.
 
it does not help much that those who lead by example (as tons of devs copy their practices) [frameworks and major trend-setters] treat HTML as something that is not authored but rather generated in a "whatever" manner, and you can compose "utility" CSS classes to achieve the needed layout visually either way.
 
12:33 PM
Ugh! This design is so bad. How am I going to detect that my quota has rolled over? I need to do that in the class that manages the API calls. But then, if I do it there, how do I pass that info on to a higher level that can report it to chat?
There's not meant to be an arrow going up the hierarchy in that way.
 
sorry, do you refer to the SE API?
 
Yes.
Also... VIOLATION OF BACKOFF PARAMETER!
Just suddenly
12:33:54,389  INFO ChatBot:320 - Stalking english at 1638102834 (last was at 1638102654)...
12:33:55,418  INFO StackExchangeApiClient:142 - Backing off API for 10 sec.
12:34:05,420  INFO StackExchangeApiClient:174 - API quota left: 4784.
12:34:05,420  INFO ChatBot:320 - Stalking ell at 1638102845 (last was at 1638102654)...
12:34:05,452  WARN StackExchangeApiClient:114 - Failed to retrieve user information from SE API.
java.io.IOException: HTTP 400 requesting URL "https://api.stackexchange.com/2.3/users". Body is: {"error_id":502,"error_message":"Violation of backoff parameter","error_name"
 
I am trying to find the detailed explanation of what SD does (I think it was [what a surprise] Makyen that explained that, but I cannot recall if it was in chat, on MSO, or on MSE) to "detect" the rollover
 
@CodyGray Note: line 3 is advisory only, and comes from the same reply to the request implied in line 1, which line 2 is also based on.
 
@CodyGray yeah, I understand, there does not seem to be any issue with the logic handling the backoff. The only educated guesses (based on other reports over the years) are intervening requests and problems with parameters (both unlikely)
 
12:37 PM
This continued multiple times for all of the rest of the sites in the list. (Since I didn't get a backoff value in the error result, I didn't do any additional backing off. I don't have actual handling for 502 errors.
There definitely aren't any intervening requests.
I have a private IP, not shared with anyone else, and I haven't made any other requests for quite a while now, so that's definitely not possible.
 
@CodyGray well, the request to return this is field in the 502 response is close to 6 to 8 years old, so do not give up /s
 
Yeah.
No idea what the problem could be with those parameters, either.
 
I wonder if a simple repeat of the previous backoff upon receiving this error will help. I suppose you still want to know why the heck it happens (as do I), maybe we need some parent advice from Makyen here
@CodyGray anything logged? You mentioned the other function logging them
 
@OlegValter That is the log: "Stalking ell at <toDate> (last was <fromDate>)."
 
@CodyGray oh!
 
12:40 PM
Sorry, yeah, it's not labeled in the obvious way, since it was meant for debugging the actual logic for managing the time window, not for debugging the API. :-)
Unfortunately, my logic for handling the windowing was flawless, so that defensive debugging code is getting reused for something else :-|
 
ok, so the diff looks fine: 180 seconds above zero between fromdate and todate. One report eliminated for sure as it seems
 
@OlegValter It is another one of those cases where my Imposter Syndrome combined with my inexperience in this sort of thing makes me think I'm just doing something dumb/wrong, and that makes me shy about asking, e.g., on Stack Apps.
Incidentally, I said that it failed for all of the sites immediately after that one, too, but then after it waited another 3-ish minutes, everything worked fine.
Haven't seen another backoff violation since that grouping.
 
@CodyGray I know the feeling :) Only with absolute certainty that I did everything I can to debug can I ask. The last time it turned out to be a core language bug :) Although such diligence regarding SE API might not be necessary - most answers about throttling errors amount to "¯\_(ツ)_/¯". And it does look like you did everything correctly
 
That's part of why I find this so frustrating.
I spend a bunch of time trying to debug it, thinking it must be my fault, but it almost never is.
Then I feel like I should not waste so much time, but then I end up making a stupid error the very next time.
 
speaking of quota rollover - this is the guesswork Guttenberg bot does to determine rollover:
so yeah, periodic checks:
 
12:50 PM
Hard for me to even understand that logic.
@OlegValter Ugh! That'll eat up my quota!!
Oh, no, they aren't actually making any requests there.
That's just a really convoluted way of writing it...
...wow....
Setting up a periodic check (every 5 minutes) of some atomic integers.
I would never think to write code this way.
 
@CodyGray looks like just periodic intervals comparing if actual allowance is higher than what was recorded the last time
 
Why would you do that periodically, instead of doing it on each request?
 
cannot attest for the convolutedness of it all, though
@CodyGray that is a very good question :)
it actually makes sense to try to refresh with every request made
I might just be misreading, and the SelfCheckService acts as a singleton, but it looks like "self-check" is started for each of the rooms too
if this line is to be believed
 
Yeah, that looks like an utter mess.
It bothers me so much how everyone writes convoluted code that also has bad performance!
Like, at least get one or the other!
 
but since this one implies it is instantiated for each of the rooms, it does seem like start is called for each of the rooms. Thus leading to N checks where N is number of rooms. I might miss out on something obvious
hmmm. I really expected SelfCheckService to be a static class.
I suppose the decision is due to the ability of the check to send messages to all rooms at once, but arguably it is not what check method should be concerned with
 
1:05 PM
I previously had a BroadcastMessageAndWait function, which took advantage of the fact that Tunaki's ChatExchange library, which I'm using here, returns an awaitable async object from the send function. So I would just call both send functions and join them.
But I deleted it after realizing that nowhere do I actually need synchronous chat message generation. :-)
 
well, yours is intentionally limited to 2 rooms while theirs is abstracted to broadcast to N rooms (but ironically, as you noticed, is artificially limited to broadcast to 2 rooms which essentially makes it the same as BroadcastMessage
 
Well, I could easily have put a loop in that function, though.
@CodyGray Yes, I wrote and delete code before ever committing it to the repo. I'm just that awesome. (Why are so many programmers afraid to delete code that is unused?)
 
IMO, I would make everything in the SelfCheckService static and made check return a boolean status. Then, something like what you have, probably named as sendMessages, though (as a preference), that accepts a message parameter and a list of rooms as its second (or rest where available) one. Then loop over the rooms passed (and not held in the internal state) and thus broadcast
@CodyGray they do not? I thought it is rare, especially if you use a version control system - just stash / move to another branch / whatever is the preferred way and continue as normal, revisit later when you need it
 
I don't know, there's a strong reluctance among many that I've worked with to delete stuff, even with a version control system
The worst is when they comment it out :-(
 
As has been mentioned, we're also experiencing this with SmokeDetector. What we're seeing could be consistent with one, or more, servers not resetting the quota_remaining when it was reset by the others. We're seeing what looks like two independent quota_remaining numbers being reported (i.e. separately decremented), with the one which has not been reset being used much more rarely. That's consistent with, probably, a single SE API server getting out of synchronization wrt. the remaining quota. — Makyen Jul 28 at 20:39
^ this is one of the artifacts of the API rollover issue I thought of
 
1:17 PM
That's ridiculous
 
@CodyGray but, but, I spent time on it, my presciousssss! :) Honestly, I feel the reluctance too. Sometimes it even makes its way in a repo in when in rush (take a look at the unpublished editor's den script, although the presence of commented out part is largely dependent on it being unpublished)
@CodyGray you don't say :)
 
Ugh, so, I guess I'm going to try implementing a thing where, in response to a 502 error, I sleep for 15 seconds (5 seconds longer than the 10 seconds which appears to be the "normal" backoff), and then try again.
 
yup, that's what might be the solution for now - mentioned it before. Maybe previous_backoff * some_modifier will do? Since no one's loosing anything from an experiment here
 
Weird thing is, it hasn't happened again, and it's been running this whole time straight since we talked about it.
So this is really just a weird, random hiccup.
 
1:37 PM
might very well be. Fast-forward to 6 to 8 years later, the API source is made public, and everyone sees if( get_random_somehow() ) return send(502, { ... }) (in pseudocode) was the reason :)
 
Don't be silly; it won't be nearly that easy to read
 
oh, sorry, I forgot to add "after deciphering" :)
 
> While our source code is text based, we chose to diverge from standard text in one particular area. During normal operation, we guarantee that all source code is compressed, either with GZIP or DEFLATE. Both of these algorithms are rather old and commonplace, most platforms should have built-in tools for handling both and practically all will be able to handle at least GZIP.
> The motivation for this is simple, compiling or executing uncompressed source code is a loss for all parties. Bandwidth is, in comparison to CPU time, exceptionally expensive and severely limited on many devices. It's really a no-brainer to require compression accordingly.
 
well, technically, the source we get from cdn-dev is, indeed, deflated :)
 
The thing is, most of those compression algorithms are so bad that they hardly buy anything.
I've done a lot of testing with compression in embedded systems, actually.
It's not even a CPU time issue. I would be willing to sacrifice fairly extreme amounts of CPU time for a significant reduction in size. But I can't get it.
Part of it is that the sensor data we collect is not in any way compressible, no matter what compression algorithm you use.
 
1:49 PM
I thought gzip is pretty darn good at its job, is it not?
 
Maybe it works better for text than binary code
 
well, I did not make any benchmarks myself, but there are a lot of available - from the standpoint of speed + filesize, gzip is actually pretty great (I think it is also one, if not the, fastest one out there)
 
2:20 PM
Oh, it just hit me - I think Makyen and Scratte had a conversation about quota rollover recently (in SOCVR if memory serves me right). I am not sure it was about the rollover specifically, but let me check
Here it is, caught it:
in SO Close Vote Reviewers, Aug 21 at 23:32, by Makyen
If you want an example of what it's possible to maintain without actually trying to do so, you could look at when SmokeDetector's quota rolled over each day. A search for "rolled over" in Charcoal HQ will provide you with that information. Just be sure to ignore all the ones from 2021-07-28, which was the day that SE had a problem which resulted in SmokeDetector thinking its quota rolled over many times.
 
"If you want an example of what it's possible to maintain without actually trying to do so..." What? He makes it sound like SD just does this effortlessly.
 
3:03 PM
@CodyGray does he not make it seem like everything is done effortlessly? :)
 
@OlegValter No, he clearly puts in a lot of effort.
 
@CodyGray I know, he just likes to make it seem like it is effortless :)
 
The explanation itself usually involves substantial effort. I sometimes get tired just reading and/or thinking about it.
 
4:15 PM
I see a lot of clever words here
Hello)
 
 
2 hours later…
6:23 PM
Hmm, I wonder how/why the "this is being discussed on meta" comment here got deleted? I don't see any reason why it would, the meta post is still open.
 
6:57 PM
@cigien Maybe just several normal flags removed it?
 
Yeah, that's more likely than a mod deleting it, but still strange. I've seen those comments linger on posts indefinitely, so I assumed many users don't flag those. Also, why would someone want that gone anyway?
 
They don't want it discussed on meta? Just guessing. And yes, removing the comment doesn't really remove it from meta but it's the only motivation I can guess at here
 

« first day (2231 days earlier)      last day (903 days later) »