« first day (1408 days earlier)      last day (341 days later) » 
01:00 - 20:0020:00 - 00:00

8:07 PM
Almost finished now...
 
Is that it? What exactly are you downloading? Can’t possibly be the entire dataset for all of SO’s history.
 
@VLAZ-onstrike- You're using bittorrent, right?
@Andreasdetestscensorship It's all of SO's post history
Up until ~march
Don't download directly from archive.org though. Use the torrent links
I generally average <1MBps directly from archive.org
 
@ZoestandswithUkraine DownThemAll - is the dump even available as a torrent? I'd gladly seed it, if so.
 
@ZoestandswithUkraine Seriously? That’s so small. Huh. Well. @VLAZ-onstrike- The Imgur images will only be present for as long as SE wants them to, so consider downloading all of them too. Not sure how much that’ll be. Should iterate through the posts in the data set, and select only those images present in open Q/As.
 
8:23 PM
@VLAZ-onstrike- Yep, there's links on archive.org
 
@Andreasdetestscensorship It's the data dump. It's split into one 7z file per site. Except for SO where it's split into several (I assume due to size):
 
Yeah, it's size
I think I still have an old, unpacked comment dump
I do
The 7z file is 5 gigs, but the contained file is 25 gigs
@Andreasdetestscensorship ^ - the magic of compression
 
OK, I'm blind.
 
@VLAZ-onstrike- yeah :p
The torrent maxed out my 100Mbps connection too, which is great if you don't want to wait an eternity
 
Post history considerably larger than posts. I never considered looking into the data set, myself, but maybe I should have a go now.
 
8:25 PM
I've been downloading this files since Sunday. It failed twice, and I had to restart it. Takes like 18-20 hours to download...
 
Yeah. Use the torrent :p
Well, too late now
but next time
 
I am doing this now
 
If there is a next time
 
I'll just drop the files and re-validate. It would save a massive amount of time.
 
I found out about the torrent back when I was downloading it to seed my comment moderation tool. Had a couple failed downloads and the downloads took an eternity
 
8:29 PM
@ZoestandswithUkraine I just imagined that it would be a few TBs. I didn’t care to reason about that assumption, though.
 
I had made some rough check years and years ago about different compression algorithms. Erm, before I actually knew about them. Just zipped a bunch of files with different options and compared the output. 7z was really good. It's probably better with text-only files, dunno. I've not tried the comparison with the knowledge that compression algorithms have different characteristics with different types of data.
 
There's a chunk of data that isn't exported too though
 
But I remember the 7z was probably among the top. .bh (black hole) was the top one at the time. Also, I've not seen this used since.
 
Considering the data dump is stored as XML, it wouldn't surprise me if post history and posts are well over 200GB unpacked
 
About 175 GB is what the file says
 
8:45 PM
Shesh
What about posts?
 
100 GB original 19GB packed
Both compress to about 1/5 the original size.
 
 
2 hours later…
whoa
that's a pleasant surprise
 
But why are they unable to communicate that in anything but the most imperious and "we know what's best" way?
> Our intention was never to stop posting the data dump, only to begin to collect more information on how it was being used and by whom...
So... why did you pause it back in March?
 
yeah I mean that's...transparent BS.
 
Clearly your intention was to stop posting it. Denying that makes no sense.
Yeah.
As happy as I am about this, the way it's done leaves a very bad taste in my mouth, per usual. I don't understand why they have to do this.
 
well, actually...it's sort of a scream test.
It has, in fact, given them information on how it was being used.
From all the contributors who wrote complaints.
 
10:48 PM
Mhmm.
That could have been done simply by announcing the plan to discontinue it.
 
Are there, y'know, less user-hostile ways to gather that information? Yes, obviously, but... well, that doesn't seem to be their specialty.
 
Rather than keeping it completely secret and confidential until their hand was forced.
 
404: Reputation Not Found
 
"Our intention was never to stop posting the data dump. I apologize if our previous policy of not posting the data dump implied otherwise."
(not a real quote, to be clear)
 
@CodyGray-onstrike ah, that's the menu item for a userscript returning old-style 404 page images + allowing one to upload their own
 
Oh wow, I see. I thought it was actually the rep level.
Oh, this is not your UserScripters Bot account... I thought that's what it was.
 
did you forget that rep count is on the left from the inbox? :)
I'd never do such a drastic thing as insert userscript content in front of native controls (if an RTL user reads this - do let me know!)
 
11:15 PM
I did forget that, yeah.
I honestly couldn't tell you where rep is in the current top bar without checking, as I never use it.
 
11:38 PM
0
Q: How many answers should I post to reclaim the right to ask (if need be)?

SergeyI now receive a warning that I may be denied the right to ask question on SO because some of them were badly received. I don't think it's fair since only five of them, 11%, have a negative score (in most cases just -1) and only four are deleted (they are good too, in my opinion: how is this bad o...

 
@CodyGray-onstrike I guessed as much :)
 
I think it has moved around over the years.
And I don't think that inserting userscript content in front of native controls is any more egregious of a sin than inserting it behind them.
I actually use the SE button, because useful links are hidden behind it. It's nice that I can find it in a predictable location.
(OK, I only use it when my hand is already on the mouse, instead of on the keyboard, because if my hands were on the keyboard, I'd just manipulate the URL manually. But the point still stands, because my hand is all too often wrapped around the rodent.)
 
11:56 PM
@NewPosts this is the first time I've seen someone ask how to get out of a hypothetical question ban.
 
This is also the first time you've been on strike and thus not gone to check for socks...
 
01:00 - 20:0020:00 - 00:00

« first day (1408 days earlier)      last day (341 days later) »