Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Coding Horror - Jeff Atwood
Syndicate content
programming and human factors
Updated: 4 hours 43 min ago

The Tablet Turning Point

Tue, 11/11/2014 - 02:36

Remember how people in the year 2000 used to say how crazy and ridiculous it was, the idea that Anyone Would Ever Run Photoshop in a Web Browser? I mean come on.

Oops.

One of my big bets with Discourse is that eventually, all computers will be tablets of varying size, with performance basically indistinguishable from a two year old desktop or laptop.

Apps are great and all, but there has to be some place for this year's bumper crop of obscene amount of computing superpower to go. I like to use history as my guide, and I believe it's going exactly the same place it did on desktops and laptops — that no-installing-anything friend of every lazy user on the planet, the inevitable path of least resistance, the mobile web browser.

For the last few years, I've been buying every significant tablet device in the run up to the big holiday sales season, and testing them all, to see how many years are left until mobile devices catch up to desktops on general web and JavaScript performance.

How are we doing? Let's benchmark some Discourse client-side Ember JavaScript code:

iPhone 4 June 2011 2031ms iPhone 5 Sept 2012 600ms iPhone 5s Sept 2013 300ms iPhone 6 Sept 2014 250ms iPad Air 2 Oct 2014 225ms

My Core i4770k desktop machine scores 180ms in the same benchmark on the latest version of Chrome x64. I'd say we're solidly within striking distance this year.

I don't like to spend a lot of time talking about news and gadgets here, since the commentary will be irrelevant within a few years. But this year marks a key turning point for mobile and tablet performance, and I've lived with every iteration of these devices for the last couple of years, so I'll make an exception.

Look at this performance rampage the iPad Air 2 goes on:

Just look at it! All the graphs are like this!

It's hard to believe we now live in a world where the Apple "Premium" is no longer about aesthetics, but raw, unbridled, class-leading performance. And you know what? That's something I can totally get behind.

Anyone who tells you the iPad Air 2 is some kind of incremental update must not actually use theirs. As someone who does regularly use his iPad, I can say without hesitation that this is a massively upgraded device. I grew to hate my old iPad Air because of the memory restrictions; I could barely have three tabs open in Mobile Safari without one of them paging out of memory. Thanks x64 and iOS7!

The bonded screen, touchid, the now-adequate-for-x64 2GB of RAM, the amazingly fast triple core CPU, the GPU, and yeah, it's a little thinner. For performance, nothing else even comes close.

It's so fast I sometimes forget I'm not using my Surface Pro 3 with its 4GB RAM and Core i5 CPU. I get hassled when I bring my Surface to meetings, but I patiently explain that it's a very nice third gen hardware design with a fully integrated keyboard cover, IE11 is a great touch browser, and that I'm mostly using the device as a tablet, as a sneak preview of what iPad 8 performance will look like. Based on today's benchmarks with the iPad Air 2 – chronologically, the iPad "6" – I believe that's about right.

I also purchased a Nexus 9. It's the first device to ship with Android 5 and the vaunted Nvidia Tegra K1.

I'm very impressed with Android 5.0; aesthetically I think it's superior to iOS 8 in a lot of ways, and it is a clear step forward over Android 4. Anyone on older Android devices should definitely upgrade to Android 5 at their first opportunity.

Performance-wise, it is what I've come to expect from Android: erratic. In our Discourse benchmarks, and the latest version of Chrome Android beta, it scores about 750ms, putting it somewhere between the 2011 iPhone 4s and the 2012 iPhone 5. That said, this is the fastest Android device I have ever laid hands on. I just wish it was consistently faster. A lot faster.

To that end, I'd like to ask for your help. We've identified some deep bugs in the Android Chrome V8 engine that cause fairly severe performance issues with JavaScript frameworks like Angular and Ember. (Desktop Chrome performance remains class leading; this is highly specific to the Android version of Chrome.) If you know anyone at Google, please ping them about this and see if it can be escalated. I'd love it if more Android users – including me – could have a better browser experience when using large JavaScript apps.

I hope over the next year the remaining Android 5 performance bumps can be ironed out. I still like the Nexus 9; if you're a big fan of Google services like GMail, Docs, and Maps like I am, I definitely recommend it. The one I have will be a gift to my mom.

[advertisement] How are you showing off your awesome? Create a Stack Overflow Careers profile and show off all of your hard work from Stack Overflow, Github, and virtually every other coding site. Who knows, you might even get recruited for a great new position!
Categories: Programming

What If We Could Weaponize Empathy?

Thu, 10/23/2014 - 19:43

One of my favorite insights on the subject of online community is from Tom Chick:

Here is something I've never articulated because I thought, perhaps naively, it was understood:

The priority for participating on this forum is not the quality of the content. I ultimately don't care how smart or funny or observant you are. Those are plusses, but they're never prerequisites. The priority is on how you treat each other. I expect spats, arguments, occasional insults, and even inevitable grudges. We've all done that. But in the end, I expect you to act like a group of friends who care about each other, no matter how dumb some of us might be, no matter what political opinions some of us hold, no matter what games some of us like or dislike. This community is small enough, intimate enough, that I feel it's a reasonable expectation.

Indeed, disagreement and arguments are inevitable and even healthy parts of any community. The difference between a sane community and a terrifying warzone is the degree to which disagreement is pursued in the community, gated by the level of respect community members have for each other.

In other words, if a fight is important to you, fight nasty. If that means lying, lie. If that means insults, insult. If that means silencing people, silence.

I may be a fan of the smackdown learning model and kayfabe, but I am definitely not a fan of fighting nasty.

I expect you to act like a group of friends who care about each other, no matter how dumb some of us might be, no matter what political opinions some of us hold, no matter what games some of us like or dislike.

There's a word for this: empathy.

One of the first things I learned when I began researching discussion platforms two years ago is the importance of empathy as the fundamental basis of all stable long term communities. The goal of discussion software shouldn't be to teach you how to click the reply button, and how to make bold text, but how to engage in civilized online discussion with other human beings without that discussion inevitably breaking down into the collective howling of wolves.

That's what the discussion software should be teaching you: Empathy.

You. Me. Us. We can all occasionally use a gentle reminder that there is a real human being on the other side of our screen, a person remarkably like us.

I've been immersed in the world of social discussion for two years now, and I keep going back to the well of empathy, time and time again. The first thing we did was start with a solid set of community guidelines on civilized discussion, and I'm proud to say that we ship and prominently feature those guidelines with every copy of Discourse. They are bedrock. But these guidelines only work to the extent that they are understood, and the community helps enforce them.

In Your Community Door, I described the danger of allowing cruel and hateful behavior in your community – behavior so obviously corrosive that it should never be tolerated in any quantity. If your community isn't capable of regularly exorcising the most toxic content, and the people responsible for that kind of content, it's in trouble. Those rare bad apples are group poison.

Hate is easy to recognize. Cruelty is easy to recognize. You do not tolerate these in your community, full stop.

But what about behavior that isn't so obviously corrosive? What about behavior patterns that seem sort of vaguely negative, but … nobody can show you exactly how this behavior is directly hurting anyone? What am I talking about? Take a look at the Flamewarriors Online Discussion Archetypes, a bunch of discussion behaviors that never quite run afoul of the rules, per se, but result in discussions that degenerate, go in circles, or make people not want to be around them.

What we're getting into is shades of grey, the really difficult part of community moderation. I've been working on Discourse long enough to identify some subtle dark patterns of community discussion that – while nowhere near as dangerous as hate and cruelty – are still harmful enough to the overall empathy level of a community that they should be actively recognized when they emerge, and interventions staged.

1. Endless Contrarianism

Disagreement is fine, even expected, provided people can disagree in an agreeable way. But when someone joins your community for the sole purpose of disagreeing, that's Endless Contrarianism.

Example: As an athiest, Edward shows up on a religion discussion area to educate everyone there about the futility of religion. Is that really the purpose of the community? Does anyone in the community expect to defend the very concept of religion while participating there?

If all a community member can seem to contribute is endlessly pointing out how wrong everyone else is, and how everything about this community is headed in the wrong direction – that's not building constructive discussion – or the community. Edward is just arguing for the sake of argument. Take it to debate school.

2. Axe-Grinding

Part of what makes discussion fun is that it's flexible; a variety of topics will be discussed, and those discussions may naturally meander a bit within the context defined by the site and whatever categories of discussion are allowed there. Axe-Grinding is when a user keeps constantly gravitating back to the same pet issue or theme for weeks or months on end.

Example: Sara finds any opportunity to trigger up a GMO debate, no matter what the actual topic is. Viewing Sara's post history, GMO and Monsanto are constant, repeated themes in any context. Sara's negative review of a movie will mention eating GMO popcorn, because it's not really about the movie – it's always about her pet issue.

This kind of inflexible, overbearing single-issue focus tends to drag discussion into strange, unwanted directions, and rapidly becomes tiresome to other participants who have probably heard everything this person has to say on that topic multiple times already. Either Sara needs to let that topic go, or she needs to find a dedicated place (e.g. GMO discussion areas) where others want to discuss it as much as she does, and take it there.

3. Griefing

In discussion, griefing is when someone goes out of their way to bait a particular person for weeks or months on end. By that I mean they pointedly follow them around, choosing to engage on whatever topic that person appears in, and needle the other person in any way they can, but always strictly by the book and not in violation of any rules… technically.

Example: Whenever Joe sees George in a discussion topic, Joe now pops in to represent the opposing position, or point out flaws in George's reasoning. Joe also takes any opportunity to remind people of previous mistakes George made, or times when George was rude.

When the discussion becomes more about the person than the topic, you're in deep trouble. It's not supposed to be about the participants, but the topic at hand. When griefing occurs, the discussion becomes a stage for personal conflict rather than a way to honestly explore topics and have an entertaining discussion. Ideally the root of the conflict between Joe and George can be addressed and resolved, or Joe can be encouraged to move on and leave the conflict behind. Otherwise, one of these users needs to find another place to go.

4. Persistent Negativity

Nobody expects discussions to be all sweetness and light, but neverending vitriol and negativity are giant wet blankets. It's hard to enjoy anything when someone's constantly reminding you how terrible the world is. Persistent negativity is when someone's negative contributions to the discussion far outweigh their positive contributions.

Example: Even long after the game shipped, Fred mentions that the game took far too long to ship, and that it shipped with bugs. He paid a lot of money for this game, and feels he didn't get the enjoyment from the game that was promised for the price. He warns people away from buying expansions because this game has a bad track record and will probably fail. Nobody will be playing it online soon because of all the problems, so why bother even trying? Wherever topics happen to go, Fred is there to tell everyone this game is worse than they knew.

If Fred doesn't have anything positive to contribute, what exactly is the purpose of his participation in that community? What does he hope to achieve? Criticism is welcome, but that shouldn't be the sum total of everything Fred contributes, and he should be reasonably constructive in his criticism. People join communities to build things and celebrate the enjoyment of those things, not have other people dump all over it and constantly describe how much they suck and disappoint them. If there isn't any silver lining in Fred's cloud, and he can't be encouraged to find one, he should be asked to find other places to haunt.

5. Ranting

Discussions are social, and thus emotional. You should feel something. But prolonged, extreme appeal to emotion is fatiguing and incites arguments. Nobody wants to join a dry, technical session at the Harvard Debate Club, because that'd be boring, but there is a big difference between a persuasive post and a straight-up rant.

Example: Holly posts at the extremes – either something is the worst thing that ever happened, or the best thing that ever happened. She will post 6 to 10 times in a topic and state her position as forcefully as possible, for as long and as loud as it takes, to as many individual people in the discussion as it takes, to get her point across. The stronger the language in the post, the better she likes it.

If Holly can't make her point in a reasonable way in one post and a followup, perhaps she should rethink her approach. Yelling at people, turning the volume to 11, and describing the situation in the most emotional, extreme terms possible to elicit a response – unless this really is the worst or best thing to happen in years – is a bit like yelling fire in a crowded theater. It's irresponsible. Either tone it down, or take it somewhere that everyone talks that way.

6. Grudges

In any discussion, there is a general expectation that everyone there is participating in good faith – that they have an open mind, no particular agenda, and no bias against the participants or the topic. While short term disagreement is fine, it's important that the people in your community have the ability to reset and approach each new topic with a clean(ish) slate. When you don't do that, when people carry ill will from previous discussions toward the participants or topic into new discussions, that's a grudge.

Example: Tad strongly disagrees with a decision the community made about not creating a new category to house some discussion he finds problematic. So he now views the other leaders in the community, and the moderators, with great distrust. Tad feels like the community has turned on him, and so he has soured on the community. But he has too much invested here to leave, so Tad now likes to point out all the consequences of this "bad" decision often, and cite it as an example of how the community is going wrong. He also follows another moderator, Steve, around because he views him as the ringleader of the original decision, and continually writes long, critical replies to his posts.

Grudges can easily lead to every other dark community pattern on this list. I cannot emphasize enough how important it is to recognize grudges when they emerge so the community can intervene and point out what's happening, and all the negative consequences of a grudge. It's important in the broadest general life sense not to hold grudges; as the famous quote goes (as near as I can tell, attributed to Alcoholics Anonymous)

Holding a grudge is like drinking poison and expecting the other person to die.

So your community should be educating itself about the danger of grudges, the root of so many other community problems. But it is critically important that moderators never, and I mean never ever, hold grudges. That'd be disastrous.

What can you do?

I made a joke in the title of this post about weaponizing empathy. I'm not sure that's even possible. But you can start by having clear community guidelines, teaching your community to close the door on overt hate, and watching out for any overall empathy erosion caused by the six dark community behavior patterns I outlined above.

At the risk of sounding aspirational, here's one thing I know to be true, and I advise every community to take to heart: I expect you to act like a group of friends who care about each other, no matter how dumb some of us might be, no matter what political opinions some of us hold, no matter what things some of us like or dislike.

[advertisement] Stack Overflow Careers matches the best developers (you!) with the best employers. You can search our job listings or create a profile and even let employers find you.
Categories: Programming

Your Community Door

Mon, 10/20/2014 - 20:32

What are the real world consequences to signing up for a Twitter or Facebook account through Tor and spewing hate toward other human beings?

Facebook reviewed the comment I reported and found it doesn't violate their Community Standards. pic.twitter.com/p9syG7oPM1

— Rob Beschizza (@Beschizza) October 15, 2014

As far as I can tell, nothing. There are barely any online consequences, even if the content is reported.

But there should be.

The problem is that Twitter and Facebook aim to be discussion platforms for "everyone", where every person, no matter how hateful and crazy they may be, gets a turn on the microphone. They get to be heard.

The hover text for this one is so good it deserves escalation:

I can't remember where I heard this, but someone once said that defending a position by citing free speech is sort of the ultimate concession; you're saying that the most compelling thing you can say for your position is that it's not literally illegal to express.

If the discussion platform you're using aims to be a public platform for the whole world, there are some pretty terrible things people can do and say to other people there with no real consequences, under the noble banner of free speech.

It can be challenging.

How do we show people like this the door? You can block, you can hide, you can mute. But what you can't do is show them the door, because it's not your house. It's Facebook's house. It's their door, and the rules say the whole world has to be accommodated within the Facebook community. So mute and block and so forth are the only options available. But they are anemic, barely workable options.

As we build Discourse, I've discovered that I am deeply opposed to mute and block functions. I think that's because the whole concept of Discourse is that it is your house. And mute and ignore, while arguably unavoidable for large worldwide communities, are actively dangerous for smaller communities. Here's why.

  • It allows you to ignore bad behavior. If someone is hateful or harassing, why complain? Just mute. No more problem. Except everyone else still gets to see a person being hateful or harassing to another human being in public. Which means you are now sending a message to all other readers that this is behavior that is OK and accepted in your house.

  • It puts the burden on the user. A kind of victim blaming — if someone is rude to you, then "why didn't you just mute / block them?" The solution is right there in front of you, why didn't you learn to use the software right? Why don't you take some responsibility and take action to stop the person abusing you? Every single time it happens, over and over again?

  • It does not address the problematic behavior. A mute is invisible to everyone. So the person who is getting muted by 10 other users is getting zero feedback that their behavior is causing problems. It's also giving zero feedback to moderators that this person should probably get an intervention at the very least, if not outright suspended. It's so bad that people are building their own crowdsourced block lists for Twitter.

  • It causes discussions to break down. Fine, you mute someone, so you "never" see that person's posts. But then another user you like quotes the muted user in their post, or references their @name, or replies to their post. Do you then suppress just the quoted section? Suppress the @name? Suppress all replies to their posts, too? This leaves big holes in the conversation and presents many hairy technical challenges. Given enough personal mutes and blocks and ignores, all conversation becomes a weird patchwork of partially visible statements.

  • This is your house and your rules. This isn't Twitter or Facebook or some other giant public website with an expectation that "everyone" will be welcome. This is your house, with your rules, and your community. If someone can't behave themselves to the point that they are consistently rude and obnoxious and unkind to others, you don't ask the other people in the house to please ignore it – you ask them to leave your house. Engendering some weird expectation of "everyone is allowed here" sends the wrong message. Otherwise your house no longer belongs to you, and that's a very bad place to be.

I worry that people are learning the wrong lessons from the way Twitter and Facebook poorly handle these situations. Their hands are tied because they aspire to be these global communities where free speech trumps basic human decency and empathy.

The greatest power of online discussion communities, in my experience, is that they don't aspire to be global. You set up a clubhouse with reasonable rules your community agrees upon, and anyone who can't abide by those rules needs to be gently shown the door.

Don't pull this wishy washy non-committal stuff that Twitter and Facebook do. Community rules are only meaningful if they are actively enforced. You need to be willing to say this to people, at times:

No, your behavior is not acceptable in our community; "free speech" doesn't mean we are obliged to host your content, or listen to you being a jerk to people. This is our house, and our rules.

If they don't like it, fortunately there's a whole Internet of other communities out there. They can go try a different house. Or build their own.

The goal isn't to slam the door in people's faces – visitors should always be greeted in good faith, with a hearty smile – but simply to acknowledge that in those rare but inevitable cases where good faith breaks down, a well-oiled front door will save your community.

[advertisement] How are you showing off your awesome? Create a Stack Overflow Careers profile and show off all of your hard work from Stack Overflow, Github, and virtually every other coding site. Who knows, you might even get recruited for a great new position!
Categories: Programming

Level One: The Intro Stage

Thu, 10/09/2014 - 23:21

Way back in 2007, before Stack Overflow was a glint in anyone's eye, I called software development a collaborative game. And perhaps Stack Overflow was the natural outcome of that initial thought – recasting online software development discussion into a collaborative game where the only way to "win" is to learn from each other.

That was before the word gamification existed. But gamification is no longer the cool, hip concept it was back in 2011. Still, whether you call yourself a "gamer" or not, whether you believe in "gamification" or not, five years later you're still playing the world's largest multiplayer game.

In fact, you're playing it right now.

One of the most timeless aspects of games is how egalitarian they are, how easy it is for anyone to get started. Men, women, children — people love games because everyone can play along. You don't have to take classes or go to college or be certified: you just play. And this is, not so incidentally, how many of the programmers I know came to be programmers.

Do you know anyone that bought the video game Halo, or Myst, then proceeded to open the box and read the manual before playing the game? Whoa there guys, we can't play the game yet, we gotta read these instructions first! No, they stopped making manuals for games a long time ago, unless you count the thin sheet of paper that describes how to download / install the game on your device. Because they found out nobody reads the manual.

The project I’m working on is critical, but it has only about 3 to 4 users, most of whom are already familiar the application. One of the users even drives the design. The manual I’m writing, which is nearly 200 pages, is mostly a safety measure for business continuity planning. I don’t expect anyone will ever read it.

It’s a project I managed to procrastinate for months, working on other projects, even outside the scope of my regular assignments. The main deterrent, I believe, was my perception that no one needed the manual. The users seemed to be getting along fine without it.

And so as the year ticked to a close, instead of learning more about Mediawiki and screencasting and After Effects, I spent my time updating a 200-page manual that I don’t think anyone will ever read. It will be printed out, three-hole punched, and placed in a binder to collect dust on a shelf.

I guess that's not surprising for games. Games are supposed to be fun, and reading manuals isn't fun; it's pretty much the opposite of fun. But it is also true for software in general. Reading manuals isn't work, at least, it isn't whatever specific thing you set out to do when you fired up that bit of software on your phone, tablet, or laptop.

Games have another clever trick up their sleeve, though. Have you ever noticed that in most of today's games, the first level is kind of easy. Like… suspiciously easy?

That's because level one, the intro stage, isn't really part of the game. It's the manual.

As MegaMan X illustrates, manuals are pointless when we can learn about the game in the best and most natural way imaginable: by playing the actual game. You learn by doing, provided you have a well designed sandbox that lets you safely experiment as you're starting out in the game.

(The above video does contain some slightly NSFW language, but it is utterly brilliant, applies to every app, software and website anyone has ever built, and I strongly recommend watching it all.)

This same philosophy applies to today's software and websites. Don't bother with all the manuals, video introductions, tutorials, and pop-up help dialogs. Nobody's going to read that stuff, at least, not the people who need it.

Instead, follow the lesson of MegaMan: if you want to teach people about your software, consider how you can build a great intro stage and let them start playing with it immediately.

[advertisement] What's your next career move? Stack Overflow Careers has the best job listings from great companies, whether you're looking for opportunities at a startup or Fortune 500. You can search our job listings or create a profile and let employers find you.
Categories: Programming

Standard Markdown is now Common Markdown

Fri, 09/05/2014 - 01:23

Let me open with an apology to John Gruber for my previous blog post.

We've been working on the Standard Markdown project for about two years now. We invited John Gruber, the original creator of Markdown, to join the project via email in November 2012, but never heard back. As we got closer to being ready for public feedback, we emailed John on August 19th with a link to the Standard Markdown spec, asking him for his feedback. Since John MacFarlane was the primary author of most of the work, we suggested that he be the one to reach out.

We then waited two weeks for a response.

There was no response, so we assumed that John Gruber was either OK with the project (and its name), or didn't care. So we proceeded.

There was lots of internal discussion about what to name our project. Strict Markdown? XMarkdown? Markdown Pro? Markdown Super Hyper Turbo Pro Alpha Diamond Edition?

As we were finalizing the name, we noticed on this podcast, at 1:15 …

… that John seemed OK with the name "GitHub Flavored Markdown". So I originally wrote the blog post and the homepage using that terminology – "Standard Flavored Markdown" – and even kept that as the title of the blog post to signify our intent. We were building Yet Another Flavor of Markdown, one designed to remove ambiguity by specifying a standard, while preserving as much as possible the spirit of Markdown and compatibility with existing documents.

Before we went live, I asked for feedback internally, and one of the bits of feedback I got was that it was inconsistent to say Standard Flavored Markdown on the homepage and blog when the spec says Standard Markdown throughout. So I changed them to match Standard Markdown, and that's what we launched with.

It was a bit of a surprise to get an email last night, addressed to both me and John MacFarlane, from John Gruber indicating that the name Standard Markdown was "infuriating".

I'm sorry the name is so infuriating. I assure you that we did not choose the name to make you, or anyone else, angry. We were simply trying to pick a name that correctly and accurately reflected our goal – to build an unambiguous flavor of Markdown. If the name we chose made inappropriate overtures about Standard Markdown being anything more than a highly specified flavor of Markdown, I apologize. Standard does have certain particular computer science meanings, as in IETF Standard, ECMA Standard. That was not our intent, it was more of an aspirational element of "what if, together, we could eventually..". What can I say? We're programmers. We name things literally. And naming is hard.

John Gruber was also very upset, and I think rightfully so, that the word Markdown was not capitalized throughout the spec. This was an oversight on our part – and also my fault because I did notice Markdown wasn't capitalized as I copied snippets of the spec to the homepage and blog post, and I definitely thought it was odd, too. You'll note that I took care to manually capitalize Markdown in the parts of the spec I copied to the blog post and home page – but I neglected to mention this to John MacFarlane as I should have. We corrected this immediately when it was brought to our attention.

John then made three requests:

  1. Rename the project.

  2. Shut down the standardmarkdown.com domain, and don't redirect it.

  3. Apologize.

All fair. Happy to do all of those things.

Starting with the name. In his email John graciously indicated that he would "probably" approve a name like "Strict Markdown" or "Pedantic Markdown". Given the very public earlier miscommunication about naming, that consideration is appreciated.

We replied with the following suggestions:

  • Compatible Markdown
  • Regular Markdown
  • Community Markdown
  • Common Markdown
  • Uniform Markdown
  • Vanilla Markdown

We haven't heard back after replying last night, and I'm not sure we ever will, so in the interest of moving ahead and avoiding conflict, we're immediately renaming the project to Common Markdown.

We hope that is an acceptable name; it was independently suggested to us several times in several different feedback areas. The intention is to avoid any unwanted overtones of ownership; we have only ever wanted to be Yet Another Flavor of Markdown.

  1. The project name change is already in progress.

  2. This is our public apology.

  3. I'll shut down the standardmarkdown.com domain as soon as I can, probably by tomorrow.

John, we deeply apologize for the miscommunication. It's our fault, and we want to fix it. But even though we made mistakes, I hope it is clear that everything we've done, we did solely out of a shared love of Markdown (and its simple, unencumbered old-school ASCII origins), and the desire to ensure the success of Markdown as a stable format for future generations.

Edit: after a long and thoughtful email from John Gruber – which is greatly appreciated – he indicated that no form of the word "Markdown" is acceptable to him in this case. We are now using the name CommonMark.

[advertisement] Stack Overflow Careers matches the best developers (you!) with the best employers. You can search our job listings or create a profile and even let employers find you.
Categories: Programming

Standard Flavored Markdown

Wed, 09/03/2014 - 21:06

In 2009 I lamented the state of Markdown:

Right now we have the worst of both worlds. Lack of leadership from the top, and a bunch of fragmented, poorly coordinated community efforts to advance Markdown, none of which are officially canon. This isn't merely incovenient for anyone trying to find accurate information about Markdown; it's actually harming the project's future.

In late 2012, David Greenspan from Meteor approached me and proposed we move forward, and a project crystallized:

I propose that Stack Exchange, GitHub, Meteor, Reddit, and any other company with lots of traffic and a strategic investment in Markdown, all work together to come up with an official Markdown specification, and standard test suites to validate Markdown implementations. We've all been working at cross purposes for too long, accidentally fragmenting Markdown while popularizing it.

We formed a small private working group with key representatives from GitHub, from Reddit, from Stack Exchange, from the open source community. We spent months hashing out the details and agreeing on the necessary changes to turn Markdown into a language you can parse without feeling like you just walked through a sewer – while preserving the simple, clear, ASCII email inspired spirit of Markdown.

We really struggled with this at Discourse, which is also based on Markdown, but an even more complex dialect than the one we built at Stack Overflow. In Discourse, you can mix three forms of markup interchangeably:

  • Markdown
  • HTML (safe subset)
  • BBCode (subset)

Discourse is primarily a JavaScript app, so naturally we needed a nice, compliant implementation of Markdown in JavaScript. Surely such a thing exists, yes? Nope. Even in 2012, we found zero JavaScript implementations of Markdown that could pass the only Markdown test suite I know of, MDTest. It isn't authoritative, it's a community created initiative that embodies its own decisions about rendering ambiguities in Markdown, but it's all we've got. We contributed many upstream fixes to markdown.js to make it pass MDTest – but it still only passes in our locally extended version.

As an open source project ourselves, we're perfectly happy contributing upstream code to improve it for everyone. But it's an indictment of the state of the Markdown ecosystem that any remotely popular implementation wasn't already testing itself against a formal spec and test suite. But who can blame them, because it didn't exist!

Well, now it does.

It took a while, but I'm pleased to announce that Standard Markdown is now finally ready for public review.

standardmarkdown.com

It's a spec, including embedded examples, and implementations in portable C and JavaScript. We strived mightily to stay true to the spirit of Markdown in writing it. The primary author, John MacFarlane, explains in the introduction to the spec:

Because Gruber’s syntax description leaves many aspects of the syntax undetermined, writing a precise spec requires making a large number of decisions, many of them somewhat arbitrary. In making them, I have appealed to existing conventions and considerations of simplicity, readability, expressive power, and consistency. I have tried to ensure that “normal” documents in the many incompatible existing implementations of markdown will render, as far as possible, as their authors intended. And I have tried to make the rules for different elements work together harmoniously. In places where different decisions could have been made (for example, the rules governing list indentation), I have explained the rationale for my choices. In a few cases, I have departed slightly from the canonical syntax description, in ways that I think further the goals of markdown as stated in that description.

Part of my contribution to the project is to host the discussion / mailing list for Standard Markdown in a Discourse instance.

talk.standardmarkdown.com

Fortunately, Discourse itself just reached version 1.0. If the only thing Standard Markdown does is help save a few users from the continuing horror that is mailing list web UI, we all win.

What I'm most excited about is that we got a massive contribution from the one person who, in my mind, was the most perfect person in the world to work on this project: John MacFarlane. He took our feedback and wrote the entire Standard Markdown spec and both implementations.

A lot of people know of John through his Pandoc project, which is amazing in its own right, but I found out about him because he built Babelmark. I learned to refer to Babelmark extensively while working on Stack Overflow and MarkdownSharp, a C# implementation of Markdown.

Here's how crazy Markdown is: to decide what the "correct" behavior is, you provide sample Markdown input to 20+ different Markdown parsers … and then pray that some consensus emerges in all their output. That's what Babelmark does.

Consider this simple Markdown example:

# Hello there

This is a paragraph.

- one
- two
- three
- four

1. pirate
2. ninja
3. zombie

Just for that, I count fifteen different rendered outputs from 22 different Markdown parsers.

In Markdown, we literally built a Tower of Babel.

Have I mentioned that it's a good idea for a language to have a formal specification and test suites? Maybe now you can see why that is.

Oh, and in his spare time, John is also the chair of the department of philosophy at the University of California, Berkeley. No big deal. While I don't mean to minimize the contributions of anyone to the Standard Markdown project, we all owe a special thanks to John.

Markdown is indeed everywhere. And that's a good thing. But it needs to be sane, parseable, and standard. That's the goal of Standard Markdown — but we need your help to get there. If you use Markdown on a website, ask what it would take for that site to become compatible with Standard Markdown; when you see the word "Markdown" you have the right to expect consistent rendering across all the websites you visit. If you implement Markdown, take a look at the spec, try to make your parser compatible with Standard Markdown, and discuss improvements or refinements to the spec.

Update: The project was renamed CommonMark. See my subsequent blog post.

[advertisement] How are you showing off your awesome? Create a Stack Overflow Careers profile and show off all of your hard work from Stack Overflow, Github, and virtually every other coding site. Who knows, you might even get recruited for a great new position!
Categories: Programming