last day (2907 days later) » 
14:00 - 21:0021:00 - 00:00

2:19 PM
This room is to understand if and how we can develop a system for duplication search and enjoyable reviewing, for more info about the project see SOCVFinder on github
 
2:32 PM
As it relates to a broader effort, it has been discuss in slack (anyone is free to join just ask). And I am awaiting Sam to get feedback from the mods and Shog
Otherwise it is just another island of data and duplication of effort
 
Drew I think we need to write down specifications of what we like to do, getting some general go ahead do what you want will never happen, in github you will find exact specifications and a test program for scanning api, when those who like to develop have agreed on functionality we can ping Shog9 and see what he says.
 
I suggest leveraging Sam and the work of others as Shog has said a few times
The platform for this is already in place in fragments or even robustly by otherss
 
2:48 PM
The problem is fragments and objective specifications. I'm moving on and naturally I'm happy with all contribution.
 
Sam
Hiya
 
One of my contributions is to suggest reading what I wrote on Slack about the use of a database, and waiting to get feedback from what Sam is talking to the mods about. Then leveraging the work of others
Some wheels are round enough. New ones are often klunky and retangular
 
Drew I'm not really worried about the database and to tell the truth I hope that it the future it can be avoided (hence SO gives us possibility to filter on cv and flags).
For now I wan't to share objectives and software specifications, then we just need to some dev (we need to share what to dev, how it should work).
 
3:04 PM
FYI, my 2 cents, I'll take a look at the doc when I have clear thoughts on what I'd like to do. In the mean time, I'm working on a simple chat API for now that @Petter will be able to use to spam send messages with.
 
Great, lets all have fun and if we manage to agree on objectives and functionality then we stitch al together.
 
sounds like a plan
btw run thru 200k line text file as a database see how "fun" that is. Including inline updates :p
 
@Drew Feedback on what?
 
Sam is in talks with mods
 
... which... mods?
oh, that. Yeah, remember now
 
3:12 PM
This level of functionality if useful would require 3rd party site moderation of SO. It needs scalability, a data guy, and blessing from above imo
 
what it needs is a good meta documenting what it does and if the community agrees
3
 
^ writer needed
 
Sam
Back
Oh hey, Undo joined.
@Undo The tool we plan on building.
@Drew We never said anything about using a flat-file.
 
3:30 PM
there was commentary from others concerning "why is a db needed"
2 people
 
Lets not worry about the DB now (can't see why this is the problem) and actually it could be avoid if they add cv filter to API. Important is that we agree on objectives and specifications, then we can ping Shog to have a look and eventually maybe even post on meta
 
I think Sam has a high level something sheet with his Arctic link. Not sure what progress he has made on it.
It has been a few days, and no status on that
 
Sam
I haven't touched it since we last spoke.
I've been pretty busy.
Petter's readme is a good start though.
 
Never put any stock in functionality improvements to the API.
Been there twice
 
I'm not but I'm also not worried about the db seems like the least of problems, just saving close vote question in last 30 days in 40 main tags seems like a fairly simple issue....., however before discussing this lets share objectives and major functionality specifications
 
Sam
3:43 PM
We need a problem statement.
After we've got that sorted, we can work on the secondary goals.
And then design the infrastructure needed to solve the root problem.
Otherwise we're just wasting time here.
 
^
 
Sam feel free to update the readme
 
Sam
I'd like everyone to agree on the root issue we're trying to solve, before continuing.
As currently, it seems like several of us have varying ideas of what needs to be done.
 
There are those that think that the Problem Statement is "the current queue apparatus is insufficient". To me the PS is that people don't care. Let someone else close the stuff or meh
 
For me its closing faster duplicates and make reviewing more fun, so more people are willing to do it
 
Sam
3:48 PM
@Drew We can't fix people's lack of caring.
 
For me there is nothing fun about closing and there never will be :p
 
Sam
@PetterFriberg Right.
 
@Drew well less boring and that you feel more productive...
 
Sam
@Drew Then why bother wasting your time to create a tool?
So far, Petter's the only one who's provided a valid problem statement.
 
I created a tool for other purposes
 
Sam
3:52 PM
Right. So this is unrelated to your goals then?
 
As I have said here and there, if approached by the powers to be I will offer it up
the powers that be. That think they be.
 
Sam
CMs don't wander the chat rooms begging for help.
You've gotta take the first step.
Anyway, we're deviating.
Does anyone disagree with Petter's PS for this project?
 
the cherry picking concept might get blow-back. The pie charts on users will
of course we are cherry picking. But just saying
the users won't respond back to the bot. It is a bit dreamy
I like the output to some url
 
Sam
@Drew Wrt to user-specific data, I'm not sure.
 
On one hand it is a high level doc. On another it tries to drill
 
Sam
3:59 PM
Take Closey as an example.
 
piecharts on user (really low prio), user won't respond to bot (I think they will and if not you could autoscan batch after 30 min to close it), output to url is already in spec.
 
I would only output to a URL. I would ignore requests that DoS it. It needs auth
 
@Drew it only tries to drill to show feasibility, for me the objective's are important
 
Sam
@Drew People had to manually tell Closey that they had started/finished reviewing.
I don't think it'll be too much of an issue.
Anyway, we could probably automate that command.
 
I don't think they should have to. Sometimes they just want a link to get at later. I am saying dont expect it (an I am done cmd).
chat is nice. Captive audience. For the best experience, a data driven website is needed. Oops that suggest a db. There I go again :p
so that is my feedback at a cursory glance
 
Sam
4:04 PM
Ok, thanks.
 
I, for one, love the idea of sending chat message for possible dupes live.
 
Nothing says that you can not add a website to get result, I like a chat bot so we can notify user on incoming duplicate in real time. I think the web ui in the beginning should be to display statistics, then you can integrate same function in as chat bot (if someone like to dev the web ui)
 
and as Petter says, there are multiple end-points possible
 
Sam
So our goal is:
> To build a tool which aids the user in searching/closing duplicates faster, and to provide a more enjoyable reviewing experience (with the intention of attracting more people to join us and help).
I've built that on Petter's previous PS.
 
"more enjoyable" : )
 
Sam
4:11 PM
Have I missed anything else?
 
But I think the goals are distinct, hence we could do only one of them. If SO does not approve cherry picking we could still move on dupes.
 
what is cherry picking?
 
Sam
@Tunaki Only reviewing a select collection of questions with certain characteristics.
Whilst ignoring everything else.
 
Like picking a girl to dance with at high school
 
Sam
^ xD
 
4:12 PM
so is filtering by tag in the CV queue cherry picking?
 
it is a phrase Shog used Apr 12
 
Sam
This is where things start to become grey....
 
why is it grey? The system gives you the ability to review where you are knowledgable
 
cherry picking is the Campaigns room. In fact, it should be renamed to it
 
@Tunaki filtering on cv count...
 
Sam
4:14 PM
Yes, the system allows you to review based on tag.
But we're going one step further, by allowing users to filter by CV count (and possibly other metrics later on).
 
I don't see the issue.
 
Sam
Nor do I personally.
 
I see a huge issue with it
 
what is that?
 
it is Cherry Picking
 
4:16 PM
and how is cherry picking an issue?
 
Shog will let us know. That is all that matters
It is not a democracy
 
Well, as I see it, the point behind the review system is to get eyeballs in front of questions.
 
I love Cherry Picking. Whether an ordained system built around it is allowed is another thing
 
so I guess you have your question to ask Meta then
 
1 hour ago, by Drew
^ writer needed
 
4:19 PM
I really need to run, I would appreciate if you contribute on the git, to get the Objectives file (readme.md) in shape. When we have our goals, objective and general implementation in shape we can ping Shog, to have a eye ball on it.... most of all, we do it all since we like to have some fun..
 
Sam
@PetterFriberg Sure thing, cya soon.
 
Cya o/
 
Cya, thanks all!
 
thx Petter for all your efforts. I would stress both website work and chat bot in the doc
I would remove any language that suggests reporting on users
That is, whatever the bottom section meant
I would not remove any reference to Cherry Picking. Rather, I would highlight it. Best to get your answer as soon as possible before effort is made.
 
@Petter that's a great doc - I love the live dupe notification - and the cherry picking is good: the point is to get more eyeballs, certain eyeballs might prefer questions at CV1 while other eyeballs might prefer CV4
 
Sam
4:29 PM
I'll rewrite the doc for extra clarity.
Fix any typos, etc.
wb Undo
 
it is just filters, you filter them as you like them
the tool doesn't force you to review questions with 4 close votes
that would not be cool indeed
 
who knows what is cool. Presumably cv3 and 4 cherry picking is cool because Sam showed up in Campaigns suggesting a tool be made for it. If I understand the transcript correctly
 
Sam
Yeah
 
waffle
so I don't what they want
 
what do you want?
 
4:38 PM
I want to have fun :p
 
what do you want that "they" don't want then? :p
side-question, who is "they"?
 
some of "they" be rep-w (as for them they can have it). Answers to dupes don't bother me.
for the rest of us, I think we all generally want the same thing: a site full of good content
what I really want more than systems is the cv-mod concept
But that is way off topic
 
Sam
I've updated the first section of the readme, thoughts?
Afk for a while.
 
4:54 PM
 
5:07 PM
Related to the discussion between Gothdo and Shog: I would suggest a sentence or two somewhere in the doc concerning the close vote review queue. In particular, this system is somewhat in disregard to it, as that info is not available to us in a kosher manner
I am not saying it is unrelated, but it is the best we can do due to the fact
 
5:18 PM
It is available to us in a kosher manner @Drew
The API has all the info: close vote count, tag, reasons for close vote, comments...
 
@Tunaki where in the api is the existence in the cvrq status available?
Answer: it is not
 
Indeed but that isn't the question here, is it not?
 
It is related to the discussion. A mention of it in the doc is suggested
 
Correct me if I'm wrong, but the goal here isn't to clear the CV queue.
 
If no one wants something in the doc such as that, whatever blows your hair back
You will ultimately go to meta with it, and it will come up
We are creating a secondary review queue. Ie: the reference to the Gothdo / Shog discussion
 
5:24 PM
How is Petter's system relying on info obtained in a non-kosher manner?
 
read north again. Slowly
 
So it is relying on the fact that a question is in the queue or not?
Meaning, only questions that are in the queue are collected by the tool?
 
5:55 PM
in Room for Gothdo and Shog9, yesterday, by Shog9
@Gothdo look, if you want to create a more effective review system, don't build it on the existing review system!
 
that doesn't answer the two questions above
 
Then there was my meta feature-request to add back into the timeline that was rejected
So this is designing a secondary review system
33 mins ago, by Drew
We are creating a secondary review queue. Ie: the reference to the Gothdo / Shog discussion
 
Yes but I want to understand this system is somewhat in disregard to it, as that info is not available to us in a kosher manner
 
We are designing a review system that is making no attempt nor caring if something is in the cvrq
So it may or may not be pointing out the obvious to people. But know it before going into it
 
Okay. And what info is not available to us in a kosher manner?
 
5:59 PM
a boolean. Is it in the cvrq
 
Ah. I don't see in Petter's doc where that boolean is needed.
 
in Room for Gothdo and Shog9, yesterday, by Drew
Oh, the other thing is resignation that who cares what is in the review queue, but focus on cv count. In reality, whether or not it is in the queue doesn't make it suddenly a good question to keep and not close. And it allows for less fixation on the cv review queue count
So we are designing a new review system if this proceeds
Get comfortable with that concept if ya wanna sell it to your peers on meta
 
I am confortable with it. But I still don't see why the information of "the question is in the queue" is needed.
 
Depending on how far you want to go with this (meta), the disclosure seems important to me. It is a secondary queue
It is not our fault they decide to keep the queue status hidden
Even if they didn't, it is a secondary queue. Ie my blocked quote above
 
Yes Drew it is a secondary queue. So why would we want to tie it to the first one by fetching that boolean?
 
6:13 PM
We can't fetch it. We have to synthesize it.
 
But we don't need it, do we?
 
@Tunaki I think that Drew means that we should inform other people that our review system doesn't depend on the review queue.
 
resuming my music listening :p
 
oh
that I agree
 
@Sam Thanks
 
6:21 PM
Wb @Petter o/
 
Related to info not available, I have mentioned that here github.com/jdd-software/SOCVFinder/blob/master/…
\o
Major issues, no filter on cv count force the use of database (to keep inside api quote)
No info if question is in review, we can't redirect to review
and we cannot do inverse from closing...
 
That's not a problem for me
 
But that is not the doc we are reviewing is it
 
this force the need for a service that fills a database with question of interest --> that's not bad, we can't filter by close vote count so we workaround
 
We are talking about the current edit from Arctic
the thing that presumably Shog will be pointed to
 
6:26 PM
No information available if question is in review or is closed flag --> for simple flags, the current queue with tag filter is actually perfect for hammerers so that's not an issue. Now this new queue wouldn't show flagged questions but is it a concern? Don't think so.
 
Well in the end there is a link also to this doc : )
 
My point is tldr put a 1 liner in there that it is a secondary queue
You only get 1 shot at a first impression. With Shog or meta
 
@Tunaki No but we need to check comments, hence the "Possibile duplicate "
 
Yep, but the comments are available with the API.
This is all kosher :)
 
I actually mostly indicated the major issues to show where Shog9 could help us
Lets wait with meta for now, this is stuff that is hard to understand for people, for me I'm ready to have some fun testing it out and I'm perfectly ok informing Shog9 that we are testing this. I can hardly believe he will stop our tests, maybe he will block some, he will force us to do something else or maybe he will just help us out.
If we manage to do a chatbot that can run in different rooms, maybe we can find rooms as php and r to test it.
 
6:33 PM
There is no getting around some level of screen scraping. Just to save the suspense
 
When we completed testing (also some major rooms and people have tested) we can meta it
I can't see why screen scraping is need
 
do more dev with it. It will become apparent
 
The jar in the github.com/jdd-software/SOCVFinder/tree/master/SOCVDBService is already running try it out.
You will have only 1 day of a tag (but this is because I wan't to limit api calls, to not use an api key for now). But all my testing shows also that after 20 days there are no cv votes.... Everything is gooing on in 10-15 days
This was some test result I run in the beginning
PHP
Today DUP=29 CV1=60 CV2=37 CV3=21 CV4=9
-1 DUP=21 CV1=67 CV2=40 CV3=19 CV4=9
-2 DUP=22 CV1=42 CV2=35 CV3=21 CV4=9
-3 DUP=22 CV1=52 CV2=39 CV3=12 CV4=6
-4 DUP=17 CV1=58 CV2=29 CV3=13 CV4=5
-5 DUP=12 CV1=32 CV2=18 CV3=11 CV4=9
-6 DUP=21 CV1=52 CV2=17 CV3=6 CV4=6
-10 DUP=16 CV1=51 CV2=20 CV3=7 CV4=0
-20 DUP=19 CV1=0 CV2=1 CV3=0 CV4=0
-30 DUP=18 CV1=0 CV2=0 CV3=0 CV4=0
Today was Friday, -1 is the day before, -2 2 days ecc.
 
 
2 hours later…
8:19 PM
There is the part of the Gothdo / Shog conversation (the top chunk) that just says "Go Design Your System And Have a Nice Day"
As for that, frankly, why even bother presenting it
just do it and take the above convo as a precedent
despite the Apr 12 chat with Rob and Rene, there wasn't much complaint to Gothdo about cherry picking. It was, if the current review system is not to your satisfaction, go build a new one with your own aging
 
Yeah I'm not really worried about Shog9, I think as long as we use public api and don't touch roomba he will not have problem, the community is more difficult, they don't like that people close stuff (and there is always someone ready to pass a nasty comment) that's why my idea is to let the others room test it, get support slowly.. What is most important is that we have fun building it and clearly if we manage to do something useful for the community is even more fun.
 
I have always been a firm believer that the vocal meta folk are among the happy users of systems we have designed already or will
 
They need to see it, test it, understand it, then we can meta it if we like to make it public to all.
 
8:34 PM
Perhaps he didn't think Gothdo would design such a system. Perhaps when it becomes popular the tune will change as it threatens cvrq pushing up.
But this won't offer trinket badges.
Perhaps I should make my PHP / DB publicly exposed and be done with it
 
In the end I also think that they want people to get involved, have fun on SO, I can not see how this tool can be negative for SO, people get involved, duplicates get closed by the correct eyes ecc.
You need to do what you want for this project I think only a small db is need since it will store very few question, since it keep only cv questions and dupes, it will not exceed 20.000 questions.
 
Do you know how many questions are open currently that have a dupe reference.
Add to that number the ones from the last few weeks that have cv on them
That is a much larger number than you think
 
Try to run the jar that I have download, see the php tag example above, DUP=29 CV1=60 CV2=37 CV3=21 CV4=9
 
Petter I know the data. I have been doing this for a while.
 
Thats 1 day you can multiple for 20 days (cv votes have already aged away) add the other tags and you see that the amount is not really out of scope
 
8:48 PM
 
I know the max cv count every question has reached. There arent too many people that know that
 
I believe you but the number of questions to store does not really terrify me... I have much much larger mysql then that
 
you were the one saying we didn't need a db
so which is it lol
 
You should read more carefully, I'm saying we would not need it if we could filter on cv count and flag count in api request
 
I read pretty well before that wishing info was in an api that never will be is fruitless
Oh wait, I wrote that.
oh yeah, that is why I said sooner or later you will come to love some level of screen scraping
Because you will run into situations where the api lets your down
 
8:53 PM
I do not agree but that's maybe just me
 
You don't agree with which part
 
Tell me why you need to screen scrap?
 
Tell me which missing information you can't get to in the api
 
quick intervention: why do we need to scrape and why is it a problem if we need to?
scraping is not an issue when you do it properly
 
screen scraping is great at time. There are many bots (Sams others) that don't even use the api
why don't they use the api? cuz the data is not available
 
8:56 PM
properly as in throttled, you can't just send millions of requests
but in this specific instance, what do we need to scrape?
 
Currently I have all the information need from api to do the project described in doc.
 
When you don't have it ya screen scrape. And one does it in sane approaches
 
But we can't scrape too much:
in Room for Gothdo and Shog9, 2 days ago, by Shog9
If you find yourself making thousands of requests per day outside the API, consider that something has gone wrong.
 
what don't we have?
what specific info is missing from the API that we need to scrape something?
 
right Gothdo. I gave Rob a github chunk. But he designed his bot wrong anyway.
 
8:58 PM
I have tried to put down a detailed documentation (there is no screen scrapping), the throttle is 2s its producing a file with all info. Just download and test it the code is open
 
Petter we are talking about a design doc. I already have all the bot code I want.
 
it's the design doc that'll make it to meta though
 
If you have the project, you can just click the jar it will open a swing application, put a tag name and date
 
14:00 - 21:0021:00 - 00:00

  last day (2907 days later) »