February 4, 2011 Followup to my “Why StackOverflow sucks” post — with some solutions
Some quick stats first.
I posted the link on Reddit only. Didn’t share it anywhere else. I got 20,000 hits the first day, 9,000 the next and not nearly as much the next. Most hits came from Hacker News and Reddit. A handful of others were from Twitter.
Surprisingly the discussion was mostly pretty good (I really recommend the discussion on Hacker News about it). Obviously there were the obligatory “author is just an idiot” statements, but overwhelmingly I appreciated the dialog about my post. Quite a few people had the same impression that I did. A few others felt that my experience was a good one — ie. it definitely keeps the riffraff out of StackOverflow.
A couple things stick in my mind from reading the responses:
- StackOverflow (and it’s siblings) is supposed to be a repository of answers with good solutions — not a game for karma/reputation [although people are naturally competitive]
- To that end, the recommended approach is to ask good questions and let the karma/rep roll in from that
- The StackExchange people understand that the “drive-by” contributor is locked out and they’re working on the ability for anonymous edits (but not answers or comments?) to be added and moderated/voted upon
- People actually get jobs based upon their SO reputation!
- Most people feel that, even with it’s flaws, StackOverflow is better than the alternative (random forums and mailing lists and ExpertSexChange)
- A good number of people believe that there is a signal to noise issue on StackExchange (but not on the other sites since they are smaller)
- Lots of people have had the same frustration that I expressed
- That I should have proposed a solution (and I actually feel bad about this one since it’s a my most common criticism of internet rants)
So I’ll take that last item to heart.
The problems I see with StackOverflow are two-fold:
- There is a distinct barrier to entry for the casual user of SO that wants to contribute, requiring them to “play the game” in order to participate — this applies to all StackExchange sites
- The signal to noise is getting very, very high on StackOverflow (this only applies to SO right now, but is a problem in any large community)
On a SE site, any random person can create an account and ask a question or provide a top level answer to a question. Questions and answers are voted up (or down) by people with enough reputation points. In order to comment on an answer or question, you have to have enough reputation (50). The ability for anyone to post or answer a question is great on the surface. No barriers to entry for novices and no barrier for people wanting to answer is really the basis of free exchange of information. The problem comes in when you limit the ability to upvote (or downvote) an answer or question and the ability to comment on/refine an answer/question. As a casual, google driven (where most of SO’s traffic comes from), user will simply not have enough rep, or patience, to be able to lend their voice to the discussion. It’s far easier to just say “whatever” and click back in the browser than to work at getting the 50 rep to add your voice. If a casual user does attempt to make an effort to actually participate in the community so that they can be a “real” user, they quickly find that it becomes a “watch the new list, answer quickly the easy ones to get a quick bit of rep” game.
The double edged sword of allowing anyone off the street to post a question and answer is that, for every interesting, non trivial/solve by first page on google, question, there are 100 (or 1000) simple, trivial, poorly thought out, homework problems posted. These simple posts get the asker reputation. They are answered by random, off the street, posters and voted up (because honestly when someone asks “how to I make a div background repeat horizontally, but not vertically” the first “repeat-x” answer is going to get accepted/voted up). This undercurrent of simple/poor questions and answers feeds more and more peoples’ reputations. It’s a self sustaining current too. Much like algae bloom in your pool caused by grass clippings, if you don’t nip it in the bud quickly, you’re going to have to take drastic action later because the pool’s going to be green quickly. An ounce of prevention, etc.
OK, so how do you fix this mess?
I think that allowing anonymous posting of answers, questions, comments, edits, etc. is not a good idea (even with moderation). Force people to sign up/in (new account/Open ID/whatever). Keep allowing anyone to answer questions. Also allow anyone to answer. Allow anyone registered to upvote an answer or question — but don’t immediately give rep to what’s being upvoted. Put the upvoter’s reputation into the equation. A more reputable person’s vote has more weight than that of a “random guy on the internet” in terms of visibility and reward to the poster. Keep downvoting rules the same (have to have 150, and you lose rep by doing it). Allow “random guy on the internet with an account” to post a comment, but hide them by default (behind a +/- in the UI, so people can see them if they want to). Allow a reputable person to moderate the comment and make it visible by default — or if enough reputable people say so, flag/delete it.
Summary: keep most of the rep based stuff in place, but allow low rep users to comment, but keep them hidden until “approved” by enough reputable users (or deleted by the same mechanism). Allow anyone to upvote. Grant karma/rep for upvotes based upon the person doing the upvoting.
This has one issue (and it’s currently an issue). If someone has avoided SO, but is a really, really good in their field, they’re still going to have to fight through the “old guard” that’s been on the site a while in order to make their voice heard and have weight. Any reputation/karma based system is going to have this issue. Unless there are periodic resets or you remove the weight of karma/rep in someone’s ability to do anything, then you’re going to have this issue.
It also doesn’t fix the signal to noise ratio. SO tries really, really hard to keep you from posting a duplicate question. Of course, if someone’s asking a simple question, they likely are frustrated or really novice and they can’t really determine how a slightly modified question has the same answer they need. The knee jerk reaction is to enforce some heavy moderation on the “new” question queue. This will add to the frustration of users seeking answers. I honestly don’t think there’s a good way to keep people from asking simple, homework, poorly worded or otherwise “junk” questions (the old adage if one person asks a question, other people in the room had the same question, but were afraid to ask applies).
So a potential solution to one problem. A punt on the other. Unless StackOverflow turns into StackOverlord, there won’t really be a solution to the latter without destroying the entire premise of the site.