Saturday, October 9, 2010

Today’s vowel outage

If you logged into Gmail over the last hour (or visited the Gmail homepage), you probably noticed that something looked a bit off: all the vowels are missing. We realize this makes things difficult for all of you who rely on Gmail — whether at home or at work — and we’re incredibly sorry. We take morphological issues like this extremely seriously, so we want to let you all know what happened and what we're doing about it.

At 6:01 am Pacific Time, during routine maintenance at one of our datacenters, the frontend web servers in that particular datacenter started failing to render the letter 'a' for a subset of users. As error rates escalated, the strain spread to other datacenters. We worked quickly to avoid a cascading failure of the entire alphabet by implementing a stopgap solution that limited the damage to the letters 'a,' 'e,' 'i,' 'o,' and 'u.' As a result, we're experiencing Gmail’s first temporary vowel outage. (We’re still investigating whether the letter 'y' is impacted and will post an update here shortly.)

Over the last hour we've received numerous reports of this issue via our help forums, from colleagues at Google, and via email you’ve sent us. Some of you have already found creative workarounds for communicating without vowels, like Aaron, who sent us this:


Having 80.8% of the alphabet available is significantly below the 99.9% full letter uptime reliability we strive for. Since identifying the root case of this issue, we’ve started bringing vowels back to Gmail, so you should see them back in your account within the next few hours if you don’t already. In the meantime, while you may still see this issue in Gmail's web interface, both IMAP and POP access are functioning normally. We'll post an update as soon as things are fully resolved and, again, we're v3ry s0rry.

Update (7:30 am): We’ve determined that the letter 'y' is not impacted.

Update (3:02 pm): This issue has been resolved.

Update (12:01 am): Also, this issue never happened. Happy April 1st.

No comments: