last day (14 days later) » 

10:13 AM
For context:
in SO Close Vote Reviewers, 53 secs ago, by ArtOfCode
@Cerbrus I contest. It is a room issue, because if one user in a room is a robot, then the entire room has to lose out because we can't risk it. Therefore, it's at least in part up to the room to ensure that SD is used correctly.
I have a problem with that.
 
Go on
I mean, I get that you can't control the actions of a user.
 
How can a room possibly know someone flagged a post while it shouldn't be. Not to mention deal with the incorrect flag
 
Oh, I don't mean flags. Not at all, I don't expect you to deal with that.
But Smokey feedback is something you can see.
 
@ArtOfCode Isn't that the whole problem that caused this mess in the first place?
 
If someone sd ks a post that's clearly an f, it's worth just saying to them "look, please take a little more time to review"
 
10:15 AM
@ArtOfCode did someone?
 
I can't actually say I know the full details of what caused it... I'd have to look it up, or ask Undo.
 
Okay.
 
But can you appreciate that we can't take risks with SD? If we have to shut Smokey down, then the entire network loses a layer of spam protection, which would make us a better spam target, and end up with more spam.
 
I can understand that
But I think cutting off a room isn't a solution
in SO Close Vote Reviewers, 7 mins ago, by Cerbrus
That audit was a horrible answer that could just as well have been spam, depending on the website's contents. It was flag bait. Flag bait reported by someone reputable as "Audit". Sure, everyone has to be skeptical about every single rerport, but I'll bet you a similar audit will have similar results in any room.
Especially that last line.
There's bad apples in every room
 
That's partially where it comes from. It does feel a bit harsh to me that we just remove things as a solution, but there's a balance.
 
10:18 AM
I can understand how defensive, for lack of a better word, they get about Charcoal
 
@Cerbrus so find us one that's less like spam, we can run another audit
 
I'll get back to you with that
 
Sure
It's a difficult one.
 
Review audits on SO use obvious good quality posts to root out robo-reviewers
Maybe a good answer with a link to a library should be used, instead of some kind of trash
People who still flag that just aren't paying attention. Those are the users you want (to get rid of).
 
Obviously we can't tell what goes through people's heads when they flag or feed back. But it is important that we don't end up with people trusting Smokey reports; they can't be trusted.
SD is accurate, but it's not accurate enough yet.
 
10:21 AM
I know, you know, most of us know... The problem is the few that don't...
Maybe the report could be reworded, even
 
We discussed a system for auto-casting spam flags the other day based on Smokey reports; the number of extra criteria we have to put in to make sure it flags accurately is surprising.
@Cerbrus What kind of thing are you thinking?
@Cerbrus That's a good idea. See if you can find some, I'll do the same.
 
Maybe make it a kind of review request instead of simply a report
Heck, dump REVIEW in the "report"
Any way, lunch is ready over here. I'll get back to you later :-)
 
What someone should do is to check if SOCVR really ever have nuked a post that is not spam, 1,2 wrong flags does not do any real harm (they will get denied)... but for now its better to just let all calm down, rene asked specifically for this to protect the room
 
That's probably a good idea, to be honest. An explicit "please review this post" can't hurt, at least.
@PetterFriberg 1 or 2 wrong flags is the point at which a moderator starts worrying that there's something wrong.
Spam flags aren't something that should be taken chances with.
 
while I agree, I think if you want the bot to be effective it has to report to many people and on bad answers you will always risk 1 or 2 wrong flags, for users with not enough knowledge, the only way around would be to lock room where report are in...
 
10:27 AM
I tend to agree.
There's a balance, though. We just can't risk Smokey by letting too many mistake flags through; adding load to moderators with mistake flags is something that CM's will not like.
So we're erring on the side of caution by removing answers from SOCVR for now. We will review that, it's not final.
 
I actually think the solution is a different one
 
I've also got to run now; I'll be back later. Welcome any feedback - on anything - you want to drop in here.
 
purposefully spam flagging a post without reading it is a serious offense
 
cya @ArtOfCode
 
you-can-get-suspended-for-it serious
 
10:31 AM
@Magisch We don't know if this was the case, maybe user judged it as spam...
 
cc @art I think the way to deal with smokey abuse is make everyone that gains SD privs accept explicitly and advise anyone else that they have to use their own judgement. Failing that will be harsh consequences for the user, beyond a declined spam flag
 
I aslo saw the post, linking to own project, I did not flag anything since the question was asking for that, but you need some experience for this.
 
I don't think we can pin it on a room. Its always an individuals failings.
So, not considering corner cases like that audit
If someone flags something obviously not spam as spam, give them a month or so timeout
 
perhaps treat that the same way as audits? First time - meh, no biggie. Repeated offense: wow dude, chill out
 
@Magisch however the problem of SD privs reporting incorrectly is another problem. THe problem here is that someone in room, flags incorrectly (SD priv o no SD priv)
 
10:46 AM
Thats not something we can possibly adress or control in the room
its free for everyone to join
 
yeah well, you have to go through the motions. Nobody will deny you can't prevent humans from making mistakes; that does not mean that everyone once in while you have to show and tell that you think you can anyway.
err, that's Dunglish.
It DOES mean you have to do that ;)
 
11:05 AM
I'm not having much luck finding good audit examples (yet)
I'm starring some (imo) good ideas
 
@Cerbrus An implementation idea for a "needs review" button:
in Charcoal HQ on The Stack Exchange Network Chat, 2 mins ago, by Magisch
@Undo New Feature request for SD: Add a "Needs admin evaluation" feedback type for SD posts. Currently, when errors are made or posts otherwise require special attention, we have to ping you. I propose replacing this with something like sd needsadmin <link to the chat entry of the smokey report> <reason> in order to be able to effectively give feedback on and notify admins to reports with either problematic feedback or other need of admin attention (cc @ArtOfCode)
 
I'll write up a small summary instead...
Ideas:
- "Needs admin evaluation" SD command ([link](http://chat.stackexchange.com/transcript/message/28857404#28857404))
- Make purposely flagging posts without reading bannable ([link](http://chat.stackoverflow.com/transcript/message/29834183#29834183))
- Format SD reports as a "Review request" ([link](http://chat.stackoverflow.com/transcript/message/29834052#29834052))
Hm, broken links in chat, working in the pinned message...
It's kinda difficult to find proper audit answers that aren't deleted / massively downvoted...
 
How about this?
 
That's a good option
 
11:17 AM
It is self promotion by one of the crate devs, but with disclosed affiliation and explicitly stating what the asker wanted
 
Obviously not spam, but I can understand why someone'd flag it as such, though
That may be the best option I've seen so far.
(Starring the reply, not the message itself. don't wanna draw too much attention to it.)
 
Maybe we should get someone other then Undo to report it too
Someone more inconspicuous.
 
Aye
Yet someone we know doesn't auto-flag
 
Maybe ferry
or tuna
The added benefit is that the above was reported to SD as link at end of answer, so it was never posted in SOCVR today
 
A inconspicuous RO? :P
FP in SD?
 
11:23 AM
@Cerbrus yep
 
Perfect
 
Well tuna often reports stuff in SOCVR
 
That's true
 
11:58 AM
In addition to everything else said let me add that the time and discussion between concluding that something was dubious in raising flags and the action taken without much consult with the RO's is something that strikes me as odd. To get back in control I agreed with the action currently taken because that was what Charcoal obviously wanted, immediate action. After the dust has settled my goal would be to bring back Smokey in full force.
 
I partially really dont get all this
wrongful spam flags are a very serious offense, and if anyone with SD privs or any room regular was guilty of them, both mods and ROs would take very quick action im sure
and people who just idle in the room from time to time? We have -10 control over them in any capacity
 
Indeed, and we did in the past or raised concerns when we notice wrong flags
so I don't think there was much wrong with us handling
 
We had 3 obviously abusive posts on SO this morning
staying up for longer then 40 minutes
 
It doesn't help that Undo now can see the effect of an SD report
 
Well
its still his bot
so it is in fact his decision
 
12:03 PM
There was in the past some doubt from regulars in the charcoal room
 
I dont want to open that can of worms
I can understand why they took this action partially
 
so maybe when we failed this first and only test that was enough to get everybody excited
 
The bot is under heavy scrutiny from CMs all the time, so a room misusing it even slightly could mean that the CM team decides to shut it down.
I don't know. I see both sides on this but I think it was a partial overreaction
 
My point is: people mis flag lots of stuf daily, we have an eductational purpose as well.
we're human, we fail
instead of removing the stuff that we fail on, it i s better to make sure we get more practice
 
Well
Maybe you and the other ROs can discuss this with art and undo
 
12:07 PM
I certainly will
 
We missed at least 11 smokey reports in the room today @rene
 
afk
 
in SO Close Vote Reviewers, 2 hours ago, by Cerbrus
That audit was a horrible answer that could just as well have been spam, depending on the website's contents. It was flag bait. Flag bait reported by someone reputable as "Audit". Sure, everyone has to be skeptical about every single rerport, but I'll bet you a similar audit will have similar results in any room.
That makes me think they were gunning for some incorrect flags. But that might just be my skepticism.
 
Well I was there when the audit was held
It was a "sufficiently spammy looking false positive"
so an actual audit, not a SO audit where its blatantly obvious
 
@Cerbrus looks a little bit like islands are being formed
 
12:18 PM
@Cerbrus Do you have a link or screenshot of the answer they used?
 
If the website was spammy, a spam flag would've been valid.
 
 
2 hours later…
2:25 PM
@Cerbrus Let me be clear: I don't plan on having any "audits" for Smokey. It's not really the best experience... and it starts turning the bot into a platform with meta-discussion and meta-meta-discussion and so on. :P
 
 
2 hours later…
4:33 PM
Waves
 

last day (14 days later) »