About Jeff Atwood
We had a blast at Stack Overflow Co-founder Jeff Atwood’s soapbox. Jeff, of Coding Horror, was delightful and entertaining, cracking a few jokes and giving us great insight into Stack Overflow and how the entire Stack Exchange network was conceived.
Jeff delved deep into how he and Joel Spolsky got the idea for Stack Overflow from taking the best parts of Wikis, Digg, forums and blogs. Not only that, but he told us why it was crucial that Stack Overflow deploy a new version on a daily basis and the mistake he made early on in the site’s life. He told us about taking the “evil” out of Experts Exchange when it came to creating Stack Overflow. He also spoke on why the status quo is frustrating and why having an enemy is awesome.
Feel free to listen to the podcast below as you read through some of the highlights of the event below.
Taking the Evil Out of Experts Exchange
Back in 2008, Jeff and Joel created Stack Overflow as an alternative to Experts Exchange. More than just an alternative, they sought to identify existing problems in other similar systems and fix them. Jeff did a ton of research on Q&A sites, looking at what was good and what was bad about them.
We kinda wanted to build a Frankenstein monster just out of the good parts of Frankenstein. Not the bad brain jar, but the good brain jar. So that was part of the process, what is the good brain jar? What works on the internet?
As for Experts Exchange, Jeff said:
It’s awesome to have an enemy. Not in the sense that you’d go out and fight this person, because that’s ridiculous, but it’s nice to have this clear good and evil continuum.
Jeff said that Experts Exchange had a good Q&A, but it had a lot of evil associated with it, such as hiding answers behind a paywall. He said it was a good system if you take out the evil, which is what they did with Stack Overflow.
Voting from Digg and Reddit
Jeff liked the ability to vote things up and down from DIGG and Reddit. An experience on Reddit showed him the power voting can hold over a user. Jeff had posted what he called a dumb comment on Reddit, which got voted down.
Wow, that felt really bad. I felt chastised. I felt like someone slapped my wrist. “Why did you post that dumb thing here?” “You’re right I posted something dumb … why? God, I’m so dumb!” You know, I was like … it worked.
He said the down vote really bothered him, but got him into thinking about how voting could be powerful. That’s because you obviously want the best information at the top and the least relevant at the bottom. He called it “tough love” from the community of users.
When it comes to Stack Overflow, there are controls in place to ensure some balance and that voting isn’t abused. Down voting costs Stack Overflow users reputation. In order to down vote, your really have to believe you’re right, said Jeff.
Lessons from Running Blogs
Jeff has had great success with his blog and Joel also had success with a blog of his own. They wanted to take the lessons from that and apply that to Stack Oveflow, realizing that writing blogs is hard and not really actionable for a lot of people. Although, Jeff has time to time encouraged people to write blogs.
I feel a little bit like Jared from Subway. “I lost 300 pounds from eating subs. You should go eat some subs.” This isn't really actionable for most people. Whereas I can say, “Hey, no big deal, you've got five minutes to kill go onto Stack Overflow, find something you want to learn about and try to answer it.” And you’ve done this five minute unit of work that you get all the benefit from and over time will lead to the blog that I had. It lets you leave what I like to call a bread crumb of your awesomeness.
Status Quo is Frustrating
By taking the best of Digg, Wikis, blogs and forums, Stack Overflow was looking to identify existing problems and fixing them. In other words, challenging the status quo. He said the status quo is frustrating.
It's like having a cold for your entire life then one day you wake up and “Hey, I don't have cold and I feel really good. I didn't know I could feel this good.” Don't accept the status quo even if you have to rebuild … reinvent a few wheels, that's OK as long as you can identify the clear problems that you are addressing, even if they're small. Because it's the small things (that) really add up.
Rule #1: You Don’t Talk About Stack Overflow
That’s not to say challenging the status quo didn’t come without any challenges. One early mistake that Jeff said he made was to treat Stack Overflow like Fight Club. In other words, you didn’t talk about Stack Overflow — how it worked, how you wanted it to work — on Stack Overflow. The community then created their own site where they could discuss Stack Overflow. Jeff saw that there was a need to have a place where users could talk about the site and give feedback. “Telling them to go away is not an option,” he said.
That also went into the philosophy of deploying a new version of Stack Overflow on a daily basis, even if its a small change.
When you have a site like that, I think, you have to build it with the community. In other words, you have to be very transparent about things, why we're doing things, because they're the ones doing the work in the system. They're the ones asking and answering questions. And questions that don't get answered … that's ultimately not a useful site.
One way to demonstrate that is to deploy changes in front of them. Or as Jeff puts it, “walking the walk, not just talking the talk.” It was crucial to have a site that was constantly living, that wasn't outdated.
To me, unless you're changing all the time, you're kinda dead. If you have a product that hasn't changed in a year, two years, it's not really alive in a meaningful sense. People in the community can sense that.
That philosophy can also be seen in the ability to edit answers on Stack Overflow.
One additional pair of eyes makes code way better than the single pair. That second pair is huge even if it's just two people. It's a massive increase in quality. In every post, you can see side by side. This is pair programming in the small — have the person who made the post and the person who says, “I'm going edit this post and makes this more awesome.”
Our chat with Jeff continued as he talked about dealing with spammers, what he speculated was the future of Stack Overflow and the gamification of the site. We’d like to thank Jeff for dropping by ZURB and giving us his insights on the creation of the Stack Exchange network.
Transcript
Ryan: Most of us have probably used Stack Overflow to ask a bunch of questions. It's one of the best places to go to ask questions about implementation answer one and even edit one. What most of probably don't know is that Jeff Atwood and Joel Spolsky kind of looked at what was the best of Digg, Wikipedia, forums and blogs and kind of took the best of that and kind of synthesized it into the Q&A product that we know and love today. We're going to talk about that but first of all I want to introduce Mr. Coding Horror himself, Jeff Atwood. Everyone give a hand.
Jeff: Thanks for having me. Hopefully this will be a good session and we'll enjoy it.
Ryan: Very good. I kind of want to start off, Jeff, you've kind of had a philosophy for a long time and that's kind of fail early, fail often but improve and it's kind of something we take to heart here as well and I kind of wanted to know what goes into that philosophy but you guys also deploy a new stack overflow every day and so I kind of what to know why you guys do that and kind of some of the challenged you faced in doing something like that.
Jeff: Well, sure. I think, first of all, you have to discriminated like what kind of business we have and it's the business of community, right, where everything on Stack Overflow is contributed by a fellow contributor. It wasn't us, right, if it was just us write all the Stack Overflow content it'd be kind of a small site. So really, when you have a site like that you have to build with the community. In other words, you have to be very transparent of how we're doing things, why we're doing things because they're the ones doing the work that matter in the system, you know? They're the ones asking, answering questions and the questions that don't get answered, that's ultimately not a useful site and I think part of the contract we have with the community is to say look, here's how we're going to build it. Here's how we think we should build it. How do you think we should build it? In other words, to get their feedback so you're sort of running alongside the community and not herding them from behind in this really artificial way, right.
One of our early, well, I guess you'll have a question so I won't get into that.
Ryan: We can move into it now.
Jeff: But it gets to be genuine about, like, why we're here. Why we're all in the room. What we're here to do. Ultimately, Stack Overflow is about learning so you have to have an environment that cultivates learning and part of being in the classroom is that you're participating. You're participating with your fellow students in helping everybody get educated on how to do things.
Now, getting back to failure, so I think one way of demonstrating that you're actually doing, you're walking the walk and not just talking the talk, is actually deploy these changes and try them. So, that's why we have a commitment on Stack Overflow to deploy every day and some of these are very, very small. Some of them you won't actually notice but the site is constantly alive, it's living.
To me, unless you're changing all the time you're kind of dead. If you have a product that hasn't changed in a year, two years, it's not really alive in the meaningful sense. I think people in the community get sense that there's velocity. It's like we're doing things. Things are happening right? Like I give you feedback about how Stack Overflow works and eventually things change.
Maybe one day, if you're lucky, if it's a really small change that we like, the community likes, it goes in today. In the very broad sense of adjustment of the community. One example that, it took us awhile to figure this stuff out, is that questions aren't quite as valuable as answers in a Q and A system. I know this seem obvious in retrospect but it took us a while to figure that out.
So, when you ask a question for the longest time you would get, say somebody outvotes your questions; you would get ten reputation points, like thank you for asking a good, clear, useful question which is hard. Right? I think that's one of the lessons Stack Overflow is asking question is hard.
But we realized later that although questions are valuable, answers are more valuable because if you have a question that has no answers, it doesn't really matter how good the question is. What matters is was someone able and willing to answer this question and provide this artifact for everybody to find on the Internet that we can all benefit from.
So that's why we did this big re-balancing of reputation where we basically cut in half the reputation people got from questions from ten to five and you would think this would be a big controversial change and in some ways it was but we had the support of the community.
I was really surprised when we did this that people were like oh, gosh finally. This is the way it always should have been and we're happy to make those changes because I feel like it ultimately results in a better community of people that are putting more effort into providing great answers then they are just sort of constantly coming up with questions.
Nothing against questions but they're sort of like, I view them as sand. Like, you don't always have pearls but you always have sand, okay? That's sort of the way the world works. There's just a lot of questions right? But not a lot of answers.
So you have to sort of re-balance the design of the system to favor the pearls because that's really what people what to see. They don't want to see a billion questions. They want to see 500 really great questions that have really great answers.
Ryan: And kind of to piggy back off that, when you did make that, kind of, change how did you guys test that to see that people would respond to it and how did you kind of collect feedback on it?
Jeff: Well, the number one mistake we made on Stack Overflow, and it really was my mistake entirely, was early on I had this attitude that on Stack Overflow was a little like Fight Club. You didn't discuss Stack Overflow on Stack Overflow because you weren't really there to discuss Stack Overflow. You were there to get better as a programmer. You were there to teach and learn from your peers about how to become better at this thing we do professionally or at least that we love, right, even if you don't do it professionally.
And so anytime somebody would open a topic like oh, I think Stack Overflow should work this way or I should work this way. It was like no, don't, go away, basically and after a while the community said, you know what, this isn't working. We're going to have a whole other site that we're going to run where we're going to discuss nothing but how Stack Overflow works all day long, you know.
And I was like, we have failed, right? I was like, this is bad. I don't know why I didn't see this until they physically started a site about Stack Overflow. I was like, this is crazy. I'm telling you people that love Stack Overflow the most, they're like, oh, I love Stack Overflow, I'm telling them, basically, to screw off in not so many words, right and that not cool. These are the most avid users of our system. So one of the first things we learned is you have to have what we called meta, meaning a governmental site, sort of behind the scenes.
If you've ever used Wikipedia and I'm sure everybody has, there's a talk page where if you're curious about how Wikipedia, like, how did this page get to be the way it is? Then you can go to the talk page, that's very difficult to figure out which a whole other issue, it's there, right? You can figure out the history of this page, why it is the way it is, the controversies, the different ways the page is being disrupted, all that stuff.
Now not everybody knows or needs to care about this stuff but the people that do are the people that basically help you build Wikipedia, right? These are the most important people in the system in a lot of way. Not that random browsers aren't important, they are. But these are the people synthesizing the content. Making them happy is extremely important, you know? And it's also good feedback, right.
I mean, these are the people using the system and telling you this is the way it needs to work, this is the way I think it should work. They're designing your stuff for you, right? And telling them to go away is not a good option.
So the first things I would advice people is if you have a community like where the community is building the content then you have to have a meta. You have to have a place where people can go and discuss why this site works the way it does. So, with this particular change I went on to meta and I said I think the way we're doing this is broken.
I think that questions aren't as valuable as answers and we've seen problems with this where people would come on with just hundreds and thousands of questions but contribute no answers at all and I was like, this isn't wrong but I don't think they should have the same reputation levels as someone who provided a thousand really great answers that got all one up vote each or what have you.
So that was the process. The process is to go to our meta mixer, make sure you have a meta where the people who love your site can congregate and talk about it and then like vet some of these changes but they're not all tiny changes. That would be terrible tedious but big ones where this is going to affect the people using the site. We go into meta and try to talk about it and advancement community loves you for this.
And what they don't love is when you spring changes on them that are sweeping. Like, wait, wait, wait we didn't have any input into this change and we did that a fair bit to, to be honest. But generally, if we thought is going to be controversial, if we thought it was a change that would affect a bunch of users on Stack Overflow, we tried to talk about it on meta first and say is this a good idea, how should we do it, what do you think? And they would have input into how we did things.
Ryan: And kind of how do you filter through all of that feedback when you're asking about these changes and how you kind of interact to see which ones that make the most sense before you actually deploy a change.
Jeff: A lot of it can be based on data. Another thing we do on Stack Overflow is all the contributions are creative common licenses so we cannot ever put your content on a pay wall. It's literally impossible and we intentionally made that impossible by choose a creative commons license from day zero of Stack Overflow.
So, every contribution you make, you're licensing to you. You still own it but we have the right to use it. The community has the right to use it, is what I mean when I say we, not us, Stack, or Stack Exchange Inc. So when you start off on the site you have this confidence that like we're never going to turn evil, essentially, and take your content away from you.
Ryan: And kind of tends to go on, I kind of like one of the things about Stack Overflow, it's used remotely by everyone. And you kind of took that when you guys we're building it as well, this idea that it has to be built the way it's going to be used, which is remotely. You yourself, had worked from home, there we're dozens of programmers around the world. Why did you guys decide to take it like that and kind of has this remote kind of building of the system and kind of what does that kind of actually do to shape the final product?
Jeff: Essentially it was initial practicality. It wasn't an intentional design choice at that point because, [Dylan] and I were starting, we didn't have venture capital, we didn't have, it wasn't really funded in any meaningful sense. Essentially I funded it. We put in a little bit, the founders put in a little tiny bit of money.
So, I had developers that I knew that we're great that happened to be available, personal friends of mine that I worked with at a previous job that were fantastic and they just happened to be available and, of course, one of them lived in North Carolina and the other was in Oregon and they we're available. So, it was sort of like an accident at the beginning. Well, these are the people that I had to do this with.
But then as we started building it, I started to realize that the way Stack Overflow works is, our programmer in Brazil helping a programmer in New Jersey and they don't know each other, they don't need to know each other. All they have to do is have a shared love of programming and this idea that hey, if I play by the rules of Stack Overflow I get reputation but I also learn stuff and I also help other people learn things and I leave these really cool artifacts that benefit me because they're attributed to me. All the goodness in an answer, it goes back to me.
It's not like Wikipedia where who wrote the article on asphalt, well I don't know. It's not like there's a list of authors you can look at. There's really just a list of synonyms really if you dig into it. So, it's this nice virtuous cycle that you get but it doesn't require you to be in the same room and I think that for Stack Overflow it's leveraging the power of the world, I mean, in a very literal sense.
Once we got further in I found out that once I did stats that only 35% of our traffic was from the U.S. and that's going down every year. So, the world is a big place and the cool thing about programmers is there are a lot of really talented programmers just everywhere that may not be famous, they may not work for Google or Facebook or Apple or whatever but they're really good and they're kind of a little bit underemployed and they're a little bored. And that's okay.
Stack Overflow is powered by bored programmers and it really is true. It is. Yeah, it's true. But the cool thing about Stack Overflow is you can go on and it's sort of a gainful way of spending your time. It's like you're teaching other programmers. You're learning yourself. You're getting reputation and you get, this stuff is tracked and that's what let to Stack Overflow careers.
This idea that hey, if you are bored at work and you don't want to bored at work. You'd like to work somewhere everybody loves programming as much as you do, then you can go to Stack Overflow Careers and get that benefit of all the content that you put on the site. Say hey look, I really do know what I'm talking about and I actually can communicate with other human beings in a way that makes sense.
And nothing against Source Control hosting but I feel like that's the big piece you don't get from Source Control. In looking at someone's source, it tells you if they're a great programmer but it doesn't tell you if you would hate this person if you worked with them, right? That's the big missing piece. I mean I love Source, don't get me wrong, but I love human communication. I love programmers that can actually talk to other programmers in a way that is reasonable and makes sense and explains things and that's what we have to optimize for.
One of the greatest compliments we got in the history of Stack Overflow was somebody wrote me and he said, "I feel like I've been on Stack Overflow a long time an I've learned to be a better communicator. I've learned how to write clearly. I've learned how to construct answers that get uploads because they're very clear, they answer the question and they explain things in a way that makes sense to other human beings."
I'm like that's awesome. IT's like that's a skill that's going to help you in every aspect of your life. Not just how good a programmer you are but how can you communicate with your children, you're friends, you're co-workers. This is just a valuable skill to have.
So I love optimizing for communication although I love source code to and I love programming but you've got to put those things in the right order and I feel like Stack Overflow does that. It's about human communication first and source and source control second.
That's why you can't really post giant chucks of source code on Stack Overflow, and we'll tell you you're not supposed to. You're supposed to post the minimum amount of code necessary to demonstrate the problem and then explain it to us. Like use your words man. Explain the code. That's how you're supposed to do it and we can help you.
Ryan: And, kind of leads into my next question. Kind of that optimization of communication and the need to explain things. What actually kind of was the reasoning behind starting Stack Overflow? Was it because of this need to better communicate, or was it to be Experts Exchange? A lot of people actually kind of hated it.
Jeff: It was really a little bit of both because I think the cool thing, one thing I love, and if you guys are every building a company or whatever, it's awesome to have an enemy, right? Not in the sense that you actually go out and fight this person because that's ridiculous but it's nice to have this clear, good and evil continuum, right?
Because what Experts Exchange, they had a good business of Q and A but it was implemented in such a way that when you, I don't know if you guys remember experts exchange, they've kind of gone away a little bit thanks to Google and us, which is good, but you would go there and you would do a search on Google or whatever web search engine you like and you would end up on a page and it would say here's the question and the answers would be garbled. They would say oh, you want to see the answers. Well pay $19.95 a month and we'll show you the answers.
And the tricky things were because of the rules in Google they had to index the answers. You scrolled all the way down the very bottom and the answer would be there.
So, Joe and I looked and said that's actually a good system. It's kind of working if you took out all the evil. Because you don't own your contribution there and it's just done in a really sleazy kind of way. So that was certainly one of the impetuses for building Stack Overflow. Take Experts Exchange, remove all the evil and you have a pretty good system that works.
Also, just from a selfish standpoint of when I search for stuff on the Internet, the type of pages I end up on, my fantasy world is like I control those pages because I can then control their quality, you know what I mean? So, I love when I do a Google search and I end up on a Stack Overflow page. I can go in and click edit and everyone in this room can do that now. If you don't even have an account on Stack Overflow, you can click improve this answer or improve this question right now you can go in and improve the content on that page.
To me that's what it's all about. It's like ending up on pages that you like that load fast, that have reasonable information, may or may not be correct, I don't know. But it loads fast, it's easy to process, it's not filled with ridiculous amounts of ads. Just really basic stuff like that. So that was the other emphasis. We just wanted the Internet to be better for programmers selfishly, right? Like, I want good results that aren't expert's exchange that load fast and are clear. So that was sort of the other emphasis.
And then, perhaps the third things was the lessons Joel and I had seen from running our blogs and running communities was that I wanted to unlock this sense of what I call fun size participation on the Internet because I have a blog at coding whore that I've written for a long time and maybe some of you have been there but it's really hard to write a blog for a lot of people. So when I tell people you know what you should do? You should start a blog because I started a blog and I was really successful and it changed my life.
I feel a little bit like Jared at Subway, you know? It's like I lost 300 pounds eating subs, you should go eat some subs. This isn't really actionable for most people. Right, right? Whereas when I can say to someone hey, no big deal. You've got five minutes to kill. Go to Stack Overflow, find something you want to learn about, read it, try to answer it and you've done this five minute unit of work that is attributed to you. You get all the benefit from and over time will be equivalent to this blog that I had.
It lets you leave a breadcrumb trail of your awesomeness on the Internet. And it's really easy to get started. There's no friction. You don't even have to sign up. I mean literally, you can go to Stack Overflow and start answering like with no account. You just start typing and click submit like on a blog comment. So that was the other reason. Are we doing questions now or later?
Jeff: We'll do them in about ten more minutes. But I kind of wanted to, you already kind of touched upon it, Experts Exchange and communities and blogs. You guys kind of all kind of took a look at all these other forums like Digg and Wikipedia and all these other things and kind of took the best parts of that, kind of put it together into Stack Over-, I know that's an oversimplification...
Ryan: Oh, kind of actually like it is.
Jeff: Good, good, I'm on track. But you kind of took those parts. Like, how long though, what was the process there when you guys did do that and then kind of what were some of those opportunities that those sites missed that you guys though you could improve upon and kind of innovate on.
Ryan: Well, sure. So, in the process of building Stack Overflow we decided okay this community thing makes sense. We can do Experts Exchange without all the evil and then we did a lot of research. I spent a month just looking at every Q and A site on the Internet and there were so many and they're really big. I didn't know Q and A was such a large genre of the Internet but it really was even before Stack Overflow.
And part of that research we realized we kind of want to build a Frankenstein monster out of just the good parts of the Frankenstein like not the bad brain jar but the good brain jar. So that was the process of figuring out what is the good brain jar and what works on the Internet and we certainly, we loved Wikipedia. This idea that everybody can edit this, it stays up to date. You're not looking at five year old forum post that nobody has touched in five years and that may not still be valid. I mean, who knows.
So the process of editing from Wikipedia, we knew we wanted. Ultimately, we even wanted to let anonymous users edit which we just got to like a year and a half ago. It took us a while to get there. And then certainly voting from Digg and Reddit. I still remember my first Reddit experience, I'm pretty sure it was Reddit, it wasn't Digg because I didn't really care for Digg.
I posted some dumb comment and then got down voted and I was like wow that felt really bad and then I felt chastised. I felt like someone slapped my wrist like why did you post that dumb thing here and I was like, you're right. I posted something dumb. I'm so dumb. And I was like it worked. I mean why did it work? It didn't make any sense because I don't know these people. I don't care if they vote me down but it bothered me.
Because human beings are like I got down voted and I was like this is a powerful, powerful think. Obviously you want the best content at the top and the least relevant at the bottom and the really bad stuff way at the bottom, right? So this process of voting made sense even though it is kind of cool, this idea that you can get down voted. We have people telling us all the time, we have people telling us, oh you should let people down vote. If you down vote you must leave a comment explaining why you're down voting.
And I was like it's a little bit tough, right, but it's tough love. It's tough love. The community needs to love you a little bit and that's tough love. If we're not going to be honest with you and tell you, you know what that wasn't really good content, then it really wasn't appropriate content, then you're not showing the right kind of love to your users. You're not being honest with them in your love. So that's why the down voting, in particular, was important.
But one thing we did with down voting that's a little more sophisticated, down voting actually costs you reputation on answers, this is a tweaky thing. But down voting costs you a tiny, I mean a sliver, I'm talking literally one reputation and most users have hundreds if not thousands, tens of thousands of reputations.
So you would think oh, one reputation, this doesn't matter right? People will down vote all the time and all the time users are like I don't want to down vote because I'll lose rep. I'm like wow, that's a really powerful motivator. So in other words, in order to down vote something you have to really not believe that it's inappropriate or not useful or it doesn't belong on the site.
So we try to balance it right. Like we try to look down voting's powerful but we also put in these controls so that it wouldn't be done indiscriminately. It's like I'm going to down vote everything this guy did because I don't like this guy because you knew that was going to happen.
So continuing around the other thing from blogs we got was this idea of ownership. When you start a question on Stack Overflow, nobody should care more about the answer to that questions then you. You should be dedicated to your question and getting an answer to your questions. In other words, we have to have this advice and it's in the fact about look I didn't get an answer to my questions, what's wrong, what did I do wrong?
Well, did you go out and do more research on your question, did you try to make your question as good as it possibly could be, totally clear, totally simple, awesome [repro case]. I looked on the Internet and I found this, this and this but none of that worked for me.
Did you do any research? Like what did you do? Did you just come on and start typing or did you think, how can I ask that question? When other people have asked questions like this, like show us that you've put some effort in right? And that's a concept of ownership.
The other thing you get as being an owner is you can select an accepted answer and this is really a social convention. The idea of accepted answer on Stack Overflow does not mean oh, this is the correct answer. We should all look at this answer and revel in its glory as being the correct answer to this problem.
It just means that the person that started the question said hey, of all the answers I got I like this one the best for whatever reasons. It doesn't matter. It could be that was the particular solution. There's usually more than one way to do it. This is programming and there's probably a thousand ways to do it of which only 100 can be verified correct even a little. So there's always more than one way to do it.
It's just a signal to the community that this helped the questions but we would also see cases where the accepted answer will doc to the top just as a [commands] but you'll also see right under that the community might have uploaded another answer more than that. And you see that and it's like okay I got two answers. One owner liked and one the community liked and now I have two choices, right?
And you just read in descending vote order which is the default. You like okay I have these five good solutions to choose from. But that's about being in ownership. That's giving power to the person who owns the questions because this is your child. This is your responsibility to raise your child and make sure it grows up correctly and not stunted. And nobody's going to do it but you. The communities not going to go that for you.
And also, when you leave comments, like your highlighted as the owner, you're going to blog where the blogger will comment, it's in a different color because that's the owner of the blog post. It's exactly that feeling. Like you get special status in this post because you are the question owner, use your power responsibly.
And there's also, of course, attribution. Like your name is on everything, whatever pseudo name you choose, we don't require you use real names or anything like that but we want to really own the stuff and believe in it and say this is attached to my name therefore I need it to be a good representative of who I am and what I do.
And then also from the gaming world, if you look up the word gamification, it's kind of a weird word, it didn't really exist when we started Stack Overflow but when I was sitting up thinking, here's the behaviors I want to have on the site because this is what I think works for learning and for teaching in a community of professionals, technical professionals, it was about game play elements. It was like how do I make this game play fun and satisfying to do the right things. By that I mean things that benefit you selfishly and things that benefit the community and when I sat down and did that I realized these are basically all video, things I learned from video games.
And if you look at the badge system on Stack Overflow, so rather than having this long documents here's how you participate on Stack Overflow, go read this FAQ, have this video tutorial of how to play this game.
If you start a video game, who's the less person that actually read a video game manual. Who boots up Halo and goes no, no, no we can't start Halo until after we read the manual first, hold on. No, page five, I'm almost done. We can boot up the game in a minute. Nobody does this right? You start up Halo or whatever and then first level is a tutorial.
They don't say this is the Halo tutorial, it's just you start playing the game and it's like oh, you have this one enemy can you get him, right, with your giant blaster and he's like I'm unarmed and how now I learned how to fire my weapon. Look, an obstacle, how can I possibly get over this tiny obstacle. Maybe if I jumped, right?
So it's teaching you stuff. Here's how to jump. Here's how to use the weapons. Here's how to play the game, all the stuff but you're having fun. You're playing; you learn the game by playing the game.
You don't learn by, if you ever read a giant FAQ to figure out the site you're kind of failing. I mean we have a FAQ just because programmers are very anal and they must know everything little things about how things work but, in general, you learn by playing the game and the game teaches you how to play as you go in a very sort of just in time manor and the badge system on Stack Overflow supports this. All the easy badges, called the bronze badges, are there to teach you oh, here's how you do things.
Here's why you do things and as you do it you get the sensation of nobody told me to do this. I figured it out for myself because I'm awesome right? That's the way it's supposed to be and there's a really great YouTube video. It's about Mega Man, the design of the game Mega Man and it's a fantastic video and it's actually watchable for the entire ten minutes explaining why this worked in Mega Man when modern games have kind of given up.
They put a little dialog box. Here's what you must do next whereas Mega Man was like, I wasn't to show you some stuff and you'll be able to figure it out. It's like I trust you. I think you're smart enough to figure this out and they give you a controlled environment to explore your little skill and you move on to the next things.
And that was something we very much internalized in Stack Overflow, the gamification type stuff early on before it was even called gamification. And after the fact, I went to Wikipedia and was like oh, we're listed in Wikipedia under gamification. I don't even know who put that there. It wasn't me.
Ryan: You have to do some digging now to find out.
Jeff: Well, I'm afraid to touch Wikipedia, so. It gets updated somehow.
Ryan: Gotcha, gotcha. I want to ask one last question and then we'll move on to audience questions and you've kind of touched upon a lot of this already in little bits and pieces but I was kind of wondering how did you guys kind of narrow in and focus on those four features some of which you've already talked about which is vote, ability to edit, tagging and then also, user reputation and now it's based on answers as opposed to based on questions. So kind of why that narrow focus on those four, is the four core features and how have guys worked continuously to improve those features?
Jeff: Well, I think they were all examples of some sort of failures in existing systems. So, the voting is important, from a reputation standpoint but also because say you found a result in a forum thread, let's pretend it's before Stack Overflow, you found a program result on a forum thread.
Well, there's hundreds of pages, well at least maybe five pages in this forum thread. Are you on the right page? Like, where's the good content in this thread? And voting fixes that because it's okay the good content is voted by the community assuming the people are voting will at the top.
So you're never more than one or two positions from the most relevant information in the thread. So that's where voting comes from. It's like the observation that it sucks to have to figure out what's the good content in a given area for my problem. And again, within the scope of Q and A, there's a question that can be definitely answered not what's my favorite flavor of ice cream or who's your favorite X-man? But stuff that can actually have answers. I mean we have to be clear about what we're talking about here. So that was one.
And then ownership, one thing that bothered me about ownership was that programmers really like showing off what they know and I think that's cool, that's how you learn. But part of it really is showing off. Like, oh, I've seen this 30 times. Let me show you how cool and smart I am by explaining to you how to unlock the answer to your problem and you can't do that without identity.
Like on Wikipedia you can show off how cool and smart you are but you'll never really get the credit for it. The humanity gets the credit which is still a good things if you can go to that high level but I don't think you need to. I think you can get enough stuff out of identity for that to work.
So I've seen that not be, it's to diffuse on Wikipedia because you don't get credit, it's like well, it's a little bit to abstract for most people to get in and have that five minute fun size unit of participation with their name next to they thing that they contributed.
What was the other?
Ryan: Oh, ability to edit.
Jeff: Edit, of course, of course. So editing is what we call tomb stoning and I mentioned this earlier where it's the same problem on a forum. You go in and it's a five year old forum. So is it still valid? I mean it's not like anyone can go in and edit or, well I guess they could, but it's so rare as communally edited stuff stays up to date.
When I go to Wikipedia for a given topic page, I know that it's going to have the obsessively updating peel are going to have even the last ten minutes of information is going to be in there.
I don't need the last ten minutes on Stack Overflow but it's questions about C# 2005. C# has changed quite a bit in that time frame. So somebody could go in and edit it and we also have if you look at every Stack Overflow post, you know the concept of pair programming. It's like one additional pair of eyes makes code way better than just the single pair. Just adding that second pair is huge even it's only just two people it just a massive increase in quality.
So in every post you can see side by side this is pair programming in the small. You have the person who made the post and you have the person who said hay, I'm going to help you by editing this post to make it even more awesome. Together we're going to make this really cool thing collaboratively and you get that visual effect.
At the time I didn't' realize how significant that was but it really worked because it really is the principal. It's like help your peers become better by helping them and together you make this really awesome answer that everybody's proud of. It's this really, really good feeling and you get this confidence that hey, this is live content, it's alive. It's not dead. It's not just tombstone of data that's just sitting there, who knows if it's up to date or not.
So, if you didn't have the sensation of when you go to any page on Stack Overflow and I encourage you to do this. If you end up on a Stack Overflow page and you're like, you know what that's not as good as it could have been. It's like click improve. Like help us make it better. Either give another answer, that's fine to, then you get the full credit right.
Even if it's a three year old question, yes, please answer it. If you have a better answer please give us a better answer, or if it's some little thing in somebody's answer, click improve and it takes no more than two minutes, you don't even have to log in, make that content better. And then everybody has the confidence. It's like hey, this is a living thing. It's alive.
So I think a lot of that was just reactions to the problems with what was already out there. More so than us coming up with this magic list of things to do. It's like let's identify the problems with existing systems and fix them.
Ryan: Sounds like you just kind of narrowed and here's what they didn't do and here's what we can do better that they're not.
Jeff: Yeah. What do I wish the world looked like and I had a certain idea of what that is.
Ryan: Right, right, right, and it's kind of like how innovation really begins. It's like looking at things you see and you're like well, here's how I would improve it and then you go off and improve it.
Jeff: Well, I think the status quo is frustrating because the status quo, a lot of people, I'm currently looking at another problem where people are like oh, this status quo is pretty good. No the status quo is not actually very good. I was like, it's like having a cold for your entire life and then one day you wake up and hey I don't have a cold. I feel really good. I didn't know I could feel this good. Right?
It's like don't accept the status quo even if you have to rebuild a few, reinvent a few wheels. I think that's okay as long as you can identify the clear problems that you're addressing even if they're small because small things really add up I think. I mean if you look at the measurement of happiness, it's not big bad things happening to your or big good things happening to you. It's like having a day where tons of small things happen to you that feels good. That's what happiness is and those small things, in the aggregate really matter.
So I think that's when people look at the status quo, there's nothing really big wrong here. It's like yes, but there are hundreds of small things wrong and that's worse. So I think that's the way I like to look at it.
Ryan: Very good and on that, to kind of end it from my part and then open it up to audience, a couple of hands that got raised so let's get to it.
Jeff: Am I picking or are you picking.
Ryan: You go ahead and pick. I'll let you do it.
Jeff: You're in the front so.
Audience Question: How do you do quality control, how do you deal with spam and things like that of that nature? Do you [inaudible 33:18] or do you have some systems in place?
Jeff: So spam [inaudible 33:24] a problem for us particularly because we do allow anonymous contributions. So, the good news is the bad guys are not very smart. So, they really are not. So, if you can just be a tiny bit smarter then the bad guys that's usually enough. Like for example, requiring the JavaScripts to be enabled to post gets rid of 95% of bots that would do spam type of stuff.
And then like really simple here is [Six] can also catch a lot of it. Like how many links are you posting? You're a new user posting a post with ten links in it. Hmm, right, that's usually bad. If you look at the pattern of the stuff that's coming in, heuristics can get you a really long way towards just eliminating just the most common kinds and the, of course, we do rely on the community to flag spam. Usually it's more self- promotional.
A lot of the ideas that come in, one of the things I tell people is you don't really have to worry about outside attacks. It's kind of a myth that all these people are waiting to attack you. What actually happens if you do community is the attacks come from within. The attacks are people within your community doing the wrong things either because they think it's the right thing or because they want it to be the right thing.
Like, for example, self-promotion is much pernicious on Stack Overflow than any really outside attacks. It's like oh, I'm going to work in a link to my little library here and every one of my answers even if it doesn't really relate. You have to have conversations with these users going, that's not really why you should be here. You shouldn't be here to promote your library.
If it's the answer to the question yes please, by all means, post it but if the only reason every one of your answers includes a link to your little magical thing it's like I don't think you're really hear for the right reasons. So, that's the harder problem but they're both interesting problem it's just that one is way more difficult and takes more subtlety in how you handle it. The attacks from within is what I call those versus attacks from [without].
Ryan: The question was about the reputation model that Stack Overflow uses and how you actually build your reputation.
Jeff: Well, the implied problem you're getting to is how you get people to vote. Now on Stack Overflow there was a lot of voting from early on. The way to think about this is you don't ever want to give people credit for just doing something repetitively. For example, this would be the post count on a traditional forum. It's like how to I increment my post count? Well, I type a post and I click submit. Type a post and click submit. It doesn't matter how good the post is. It doesn't matter if it's on topic. It doesn't matter what it is. It's like you implemented a counter, congratulations, right? This is the wrong metric.
So you have the right idea which is reputation has to come from other users. They're saying oh, cool that really worked or that was a good post or that made sense or hey welcome to our community you can make complete sentences, that's okay because that's a par for some people. So, you have the right idea the reputation has to come from other people but getting people to vote can be really, really tricky.
A lot of the Stack, some of the other Stack Exchange communities, like the smaller ones, it's a real challenge because they have a lot of drive by users depending on the topic. Like Android has horrible voting rates but just because it's just a bunch of people that vaguely have a [inaudible 36:22] Android. That's much trickier. All I can really think of at that point is just reduce the friction to voting, make sure it's really easy to get that first vote in to sort of get somebody over the hump. It's like does this person, it has to come from another user, there's no getting around this two user interaction system that you have.
But just think about ways like how do I make it really easy to vote. How do I maybe nag people to vote a little bit. Like oh, you can vote here or give them some reward system for voting. For example, on Stack Overflow, there's a reward for basically casting your first vote, you know what I mean? That's a bronze badge. It's like oh, you got an up vote, you know what I mean?
So like make it a badge so people can see oh, can you please vote me? Like on eBay people are constantly nagging me to update my feedback so I can have my reputation go up, right, they should be doing this. You should be [building] incentives to make people nag other people to vote a little bit, not too much because it really has to come from the community. You can just say vote, you must vote. It's like, get people to nag about it a little bit, that's my main suggestion.
Ryan: So the questions was, what separates Stack Overflow from other Q and A sites like Quora?
Jeff: Quora, so Quora is interesting because it's really more tied to the social graph where we don't look at the social graph really at all. The graph we look at is interest in the topic. Quora is one big site that covers everything, right, everything, which can work. It works for some topics but whereas when you got to Stack Overflow you're there because you love fill in the blank programming, right?
And if you go to bicycles stack exchange you're there because you love bicycles right? That's what matters. It's not who you know. It's what you love. So that's the primary difference.
The other difference is that Quora really doesn't really, they support basically discussion questions. You can ask questions that can't really be answered but can only be discusses. Like, what's the coolest thing I can do in San Francisco? I don't know what is the coolest things you can do in San Francisco right?
I mean it's just not the kind of question we would really allow because it really can't be answered like show me science, show me some study that supports this. Tell me about your experience with documents why you feel this way and we are really kind of dicks about that to be honest. We require that. We are look you can't do this here. We'll tell you to go away a little bit in a nice way. Whereas Quora doesn't. They just embrace all forms of questions that I can sees. So to me that's noise, right?
Because I'm a technical OCD kind of guy and I don't really care about what's your favorite hex. But it is open up to other things. It's a lot more like a forum really in that sense. So we did incorporate a bunch of elements from Quora, probably more than I can, eventually we got to it, then we have a for real time updates which Quora always did and they were sort of like nice to have. Like oh, cool, we can do that.
Quora, I think, had anonymous user editing before we did or at least I think you still had to log in thought. I don't think they ever actually anonymous user editing but they had more premises of editing then we did. They were just nice to have. We saw something on Quora, that would be nice to have. But nothing in the core sort of way we approach the world is really the same because we're all about love the topic first, love the information first and be really strict about what information you'll accept first.
That's the premise of our system. Only support discussion in so far as it gets people good questions and good answers. Whereas Quora does not enforce that at all. So, that's the main difference.
Ryan: So the question was basically what would Jeff like to see in the future of Stack Overflow?
Jeff: Well, now nothing because I'm not actually at Stack Exchange anymore. But the philosophy lives on. One of the reasons I was comfortable leaving Stack Overflow and leave Stack Exchange was because it's a little bit like founding a religion. I don't really believe in building a product. I believe in founding a religion. It's like this is what we believe and the community, my final advice to the community is like, you've done a great job, now keep going, right? Like keep going, keep believing in the stuff because it matters, right?
And I have total confidence that they're going to figure this stuff out. The community and the rest of the team at Stack Exchange is going to figure this stuff out, you know? Like it's about setting the president and then having the belief systems that you need to have to have progress more so then what we do now. What we do now is having the vision is really what matter more than the actual super specific things.
And I will say this to, another reason I was comfortable leaving, was we kind of did what we set out to do to be honest with you. We've got an honest editing and there's nothing really missing from the original vision. It took four years but I think we achieved everything in the original vision.
Now, speculatively what can we do to get to the next level? That's tough, right, because outside of programming I don't know. I mean we have an enormous site for programmers and respectively sized sites for other topic but not nothing if you look at, go to Stackexchange.com, which has a site directory and you can go to the sites list and you can sort by size and you can see that we have a Jupiter and a bunch of Pluto's.
So, it's tricky. I don't know if we'll reach the tipping point on another topic but to be honest, I'm happy for it just to be programmers learning about other stuff and people that are technical learning about other stuff. That makes me happy. I don't need it to take over the world but that's the vision.
Ryan: Jeff, we're at 1:00 if you want to take a couple more questions?
Jeff: I think we can get to one, two, three, I'm going to three. I'm sorry you guys. All right, you sir.
Ryan: So the question was basically when building a product how do you kind of avoid the rats nest of distractions and get to that MVP?
Jeff: Sure, we'll I have different feelings about this then Joel does so my feeling on this is it is about leadership from within, about setting direction. And you have to decide the question you have to ask yourself is, is this on mission, what is our mission first of all. That's what I'm talking about. As long as you have a good mission statement you should be able to look at it and have a brief discussion with engineers, say is this really on mission? Does this advance our goal, the reason we set off to do this stuff?
And at the point, if you find out that the mission statement is unclear, that's what you need to fix. It's not so much that people want to do crazy things, it's your mission statement isn't clear about what you're trying to accomplish and I think until you fix that you're going to have that constant problem. What if we do this, what if you do this, well you can't tell if it's on mission or not because you're mission statement isn't clear enough about what it is you set off to do and how you set off to do it.
For example, anything on Stack Overflow that's not Q and A or directly supports the Q and A mission, you have to kind of disallow it and say that's a good idea but I don't know if that serves the mission. So that's my main advice is just try to get that really right and every time that comes up use the opportunity to fix the mission statement so that everybody's on the same page or at least plus or minus 10%. Yes, you sir.
Ryan: So basically the question was, how did Jeff and Joel build the community at Stack Overflow?
Jeff: Joel, obviously is a huge important ally to have because he's done a ton of community work over the years like me, sort of a blogger but he doesn't like to call it a blog and he ran the Joel and software discussion board for a long time and he based a lot of his findings on that. So I tended to wait for feedback from team members that had either sort of built communities, like Joel and I had, and had certain experiences from doing it or those that were really avid Stack Overflow users, we loved to hire from the community.
We hired a number of people that worked from Stack Overflow, right, so they had feelings about how things should work based on their-, they did it, they lived, it, they were in our system living and breathing it for a long time and I used to say one of our pet things we would do is if we hire you, you get to implement your pet feature. So one way to get your pet feature was to get hired by Stack Exchange because usually if we hired them they sort of knew what was going on, okay you were right about that and we would put it in.
And also data, I kind of eluded to this earlier but since everything is creative comments you can look at the data and analyze it to support some change that you want to make. Like I want to change a certain behavior in Stack Overflow so let's see if the data supports that, what's the current behavior? Like one frivolous change we made that I was surprised had a big effect, you can do a bounty on a question meaning you're question isn't getting answered so you're going to attach some of your own reputation to the question as a bonus.
If somebody gives me an answer I really like, you're going to get 250 of my reputation and what would happen is there's a seven day period and at the end of that seven day they had to award it or it just sort of got awarded on their behalf and people constantly complain. It's like oh, I missed the award interval. Well, like you had seven days right? And they're just like give us that one more day and I was like is that really going to matter? And I was like Okay, fine you have one more day. It's like, who cares just do it and that did matter.
We found that like, if added that one extra day, there was a significant increase in the people awarding the bounties. So this was a good way to have that discussion. It was like okay, let's try it and it's not hurting anything, just one more day on the bounty, I was a little bit against it because just get your shit together man, award your bounty, you get emails, you get all this stuff but it worked.
So that's the kind of decisions that I would hear. What kind of research can be done? Is it supported by the community asking for it? That's big as well. And then what kind of data can you assert to say this is going to work? So what kind of energy can you muster behind your suggestion? So, yeah, that's it.
Ryan: The question was, how do you actually get people to start using your service?
Jeff: Be famous. So, that's the cheating answer right? I mean we cheated right because Joel and I had these huge audiences of programmers that were listening and that's one of the reasons I wanted to do this. I had this ball of energy. I was like wow, all these people are listening to me. I was like cool, awesome, now what? And so we cheated, right? We cheated to some extend as far as bootstrapping, we definitely cheated.
If a random programmer in Topeka, Kansas started a programming site they have much more difficulty getting people to initially come there, seed it with questions and answers. So I know that's not a very satisfying answer but it's what we did.
So what I would seek out is if your starting something is try to find an ally that is a little bit famous, that's willing to like what you do and say hey, this is cool, go look at this. Because I certainly did. For example, when we had a WordPress site, I know Matt Mullenweg a little bit and I was like hey Matt, we have this word press site, could you just get, as a favor to me, tweet there's a new WordPress site on Stack Exchange, it's pretty cool and he did and it was awesome. It was like hey, that helped, right?
So try to exploit those kind of relationships. Exploit is not the right word. Be friends with people that can sort of make that stuff happen for you, that would be my main suggestion. And if what you're building is cool I think they will. I do that all the time. It's like hey, this is cool. I tweet it out to my followers or whatever. If it's cool then I'm happy to help. So make it cool and find allies. Yes, one more question.
Ryan: The question was, if Jeff was doing Stack Overflow over again will he use the same technologies as he did before or would he use some other technology?
Jeff: Sure. So for what Stack Overflow did I think it was the correct set of technologies and I think really the key decision point there is whether you want to be open source or not. If you're going to be open source then there's just too much friction in the Microsoft stack. You can do it.
People do it but I think there's a little bit too much friction to participate in those barriers to getting visual studio, getting SQL server or whatever the dependencies are although they have really great free versions of that stuff now, for the record, and they have a great incubator program called [Bismarks] so you can do it and we love the Microsoft stack.
I love C#. It's an amazing language. It's evolving very rapidly in really good ways and SQL server forms like a monster; it's great but very expensive. So I think key decision point is you going to be fully open source. If you're going to be fully open source you have to pick something else. So I think twisted technology matters a little in the sense of open source or not initially and then beyond that what type of programmers do you want to attract, how obscure do you want to be?
If you're going to pick something very experimental and new, like no JS, you might attract a certain sort of cutting edge programmer but the risk is everything's going to change in two years probably, I think, for node. Like there's no frameworks, the frameworks are going to appear, you're going to have to rewrite everything. So there's also where you are in the technology adoption curve. So whether you want to be open source or not, where do you want to be in the technology adoption curve, what type of programmers do you want to attract?
If you want just tons of programmers from, I want to choose from 100 people on Stack Overflow careers then you should probably pick PHP to be honest with you although by no means a fan, there are tons of people that do it, there's tons of tooling and resources around it. So, I think you have to sort of map on the continuum and also what your good at. If you just love PHP then go with what you love. So, okay.
Ryan: Very good, well thank you very much Jeff for coming by, it was a super good talk and enjoyed speaking with you. Thank you everyone for coming out.
