A series of unfortunate events occurred, but EditGrid is there up and running 70 minutes later. It is normal our customers will complain to us for such a long outage, especially those who use our service for the serious work. I would see this as a lucky event which allows us to experience what is disaster recovery. Lucky? Yes, not many Web 2.0 start-ups got an opportunity to demonstrate their robust recovery plan (or the lack thereof).

Since we believe that EditGrid is not only known for its amazing amount of features, but also the transparency that we deliver to our users, I will try my best to explain all the work we have done during the 70 minutes of outage. In addition to your applause for our having done a good job, this article serves another purpose, that we owe publicly our users some improvements to the system that are of critical help to prevent another such incident from happening, or reduce the affected time even when it occurs unfortunately. If you are impatient, you can skip to the last paragraph.

We begin with some background. It was just yesterday, that we got an alert on our production system, a preventive alert that warned us of our growing user base and usage exceeding the capacity of our system. As a result, we deployed an extra machine into the production server farm, and arranged another stand-by machine.

Today at around 15:10 (all time are UTC hereinafter), we received an alert on our production system, the same alert occurred yesterday. Obviously, we begin to believe that the alert is again due to the same preventive warning, which is fired when our system responds too slowly to requests. For the first few minutes, we examined the servers one by one, trying to locate the cause. We examined every component but cannot find a clue on why they suddenly become slower than normal.

Starting from around 15:20, we noticed that the number of requests going into the servers started to reduce sharply, and eventually stopped a couple of minutes later. We believed that this is due to failure of the automatic recovery mechanism. We then went through all the servers, attempting to restart each component manually. Interestingly, one specific component, after being stopped, took forever to start.

At around 15:40, we eventually identified it to be a failure of the database server. The database process is running, but when we issue it a request, we simply got no response, not even an error. Having confirmed this behavior using a few different machines, we believe that it has to be the problem of the server itself. We decided to restart it. The result is what everyone can expect. Upon start-up, the server immediately crashed.

We tried a vast number of varying configuration parameters, but the database server still refuses to start. We attempted to resort to converting the backup database server into the primary one. It turns out that due to a totally unrelated event, *all* the backup database servers stopped replicating the primary server a number of hours ago. If we wanted to promote it as a primary server, we have to wait until it as replicated all the changes till the time we stopped the primary server. We have stop-watched, and this will take over an hour to finish.

On the other hand, we began searching the web for solutions to the reasons the server crashes. We identified a bug report for the version we are running that sounds related. Coincidentally we have recently started testing a newer version of the database server in our QA environment. With a little bit of hesitation, we copied the new version into our production system (which takes some time due to the overseas file transfer) and attempted to start the new version. It worked!

It only seemed to work. It works when we created a spreadsheet, but not when we attempt to login. Then we noticed a lot of errors from the server, complaining about invalid indexes on some tables. We immediately executed some SQL queries to repair the corresponding table indexes and everything really worked. Time flew, and it is already 16:30.

Can we do better? We are not in a position to make sure our database have no bug. But we should be alerted when the backup database server failed to keep synchronised with the primary server. We should also have better mechanisms, than enumerating all servers and components, to identify the faulty one. In the best case, the outage can be much shortened. We will definitely learn from all events and keep our service stable. In any case, you can rest assured, that your data are still here, though it may take some more time for them to be recovered.

You may have heard the buzz. The second Office 2.0 Conference is going to be held on 5th - 7th September, 2007 in St. Regis Hotel, San Francisco — and we are going there. While it will be a lonely 12-hour trip from Hongkong to San Francisco for us, we will find familiar company at the conference — CentralDesktop, ThinkFree as well as our friends in the OpenSAM consortium are among some of the partners on our growing partner list that we’ll be meeting there. Some of them we’ll be meeting in person for the first time. We’re looking forward to that.

At the conference we will launch the EditGrid iPhone edition. The iPhone has taken mobile productivity to the next level, and we will demonstrate how EditGrid can help our users take advantage of that to get real-time updated data on their fingertip. This is also to celebrate the iPhone experiment in the Office 2.0 Conference.

In case you don’t know it yet, the second Office 2.0 Conference, organised by Ismael Ghalimi of IT|Redux, is not only a conference to explore the future of online productivity and collaboration but also an annual gathering for Office 2.0 folks to meet, learn and have fun. The conference line-up is impressive. It follows the tantalising success of the first Office 2.0 Conference. Follow the buzz here.

This will be the most important event EditGrid has ever participated in after the Under the Radar: Why Office 2.0 Matters conference. We’d love to reach out to more partners and customers and we hope that you’ll join us there.

Although great standards like (X)HTML and CSS defined by W3C already work smoothly on the Web, we still find them a little bit insufficient for presenting effectively graphical and tabular data, which is common with such as statistical and analytical information.

Tables are natively supported by HTML, but they are far from being convenient for non-technical people to use; charts are not easy either — most probably you’ll have to extract a chart from your desktop spreadsheet application, save it as an image file and upload it to somewhere before you can really display it on your web page. Worst of all, the tables and charts are static.

We therefore make EditGrid best for publishing live tables and charts. Charts for stock analysis can now be generated dynamically from real time data, and you get 30+ different kinds of useful charts to choose. Neat and tidy statistical tables can be embedded in your online experiment report, or you can update scores in your online tournament tables directly from your EditGrid spreadsheet. All you need to do is just getting a permalink for a chart, and copying and pasting a piece of readily available code for a table.

What if you’ve already drawn up a nice illustration and would like to post it on EditGrid? We also support inserting custom (static) images into EditGrid spreadsheets. Simply paste your link and everything is done! (Question: What if you’d like to insert a dynamic image that depends on parameters and, better still, these parameters are on your EditGrid spreadsheets? Watch this space )

Chart Gallery 1

Chart Gallery 2
We believe sooner or later you will find more tables and charts on the Internet. They are live, and generated by EditGrid.

What would EditGrid mean to you other than a spreadsheet application? Our smart users have demonstrated some of the great uses in the financial world.

This spreadsheet makes good use of the power of remote data. It fetches stock quotes of the Hong Kong Hang Seng Index companies and incorporates them in conjunction with the latest prices of the corresponding ADRs. In this way it compares the stocks’ performance in the two markets and, owing to the 13-hour time difference between New York and Hongkong, predicts the resulting impact to the Hang Seng Index on the next day.

Additionally, its second sheet comprises a comparison between prices of H-shares listed in Hong Kong and the corresponding prices of A-shares in Shanghai. You may probably discover that the prices in Hong Kong are to some extent influenced by the A-shares.

The spreadsheet has become one of the most popular spreadsheet on editgrid.com. Many visitors keep their eyes on the ADRs every night (in Hong Kong time) and the A-shares every morning. It really showcases EditGrid’s capabilities to manage and structure financial data (and most importantly the data is live!) in order to create useful presentations of instant market trends, analysis and even charts .

Hong Kong ADR prices to affect HSI and H-shares vs A-shares


Live currencies exchange table

Our team has also made a number of spreadsheets that contain the quotes of the component stocks of the world major indexes, including DJI and GSPC of US, HSI and HSCEI of Hongkong, and a live currencies exchange table, etc.

Similar usage can be found in this spreadsheet, where world major indexes quotes are centralized to produce a nice watch-list for busy investors.

Have you thought of any other clever usage of EditGrid to manage your portfolio?

Right after David wrote that EditGrid is no longer under-the-radar, we have received another accolade. EditGrid is selected by Rafe Needleman et al as one of the Webware 100 Finalists. We are in the Productivity category among many other great webwares.And that’s not the end of story – voting has just commenced. Voting closes on 11th June, and on 18th June the top 10 webwares in the 10 categories will be announced as winners of the Webware 100 contest. Let’s all vote for your favourite productivity webware now!

David and I will be attending the upcoming Under the Radar: Why Office 2.0 Matters conference on 23rd March (that’s this Friday!) in Mountain View, CA. We are excited by the prospect of meeting new partners and customers in the one-day conference.

Drop me a note if you are attending the conference as well! We’d be thrilled to meet you out there.

Nielson//NetRatings released a report (PDF) on 20th February which concluded that Google Docs & Spreadsheets has dominated the online productivity tool market. According to the report, Google D&S attracted 445,762 unique visitors in October 2006, which amounts to 92% of unique visitors in the market. The remaining 8% market share is shared by other players in the market, EditGrid included. The news was quickly picked up by the blogosphere at large on the eve of the official announcement of Google Apps + Google D&S integration and its subsequent spins.

Our internal statistics, however, paint a rather different picture. As you can see in the following chart…

… EditGrid was visited by 56,944 — 70,092 unique visitors over the same period, amounting to around 1/6 of Google D&S’s figure in December 2006. That alone makes up 14% of a market comprising only EditGrid and Google D&S — and needless to say there are a few more out there. Furthermore, while Google D&S’s market share has largely flattened out, EditGrid’s market share has been on the rise in 2006 Q4. It’s also of note that EditGrid’s trend of growth in December 2006 was in fact cut short by the Taiwan earthquake aftermaths.

Other stats show that we do lose out to Google D&S on average visit duration. EditGrid’s 2006 average…

… of 586 s (or roughly 9.76 min) per visit falls slightly behind Google D&S’s October 2006 figure at 10 minutes. Having said that, both figures are still far from impressive — an average productivity tool user would stay on the application for far longer than 10 or even 20 minutes. This is something that both Google D&S and EditGrid will have to work on.

I’m not suggesting that the Nielson//NetRatings figures are unreliable — the two sets of statistics may be compiled on different basis that makes it difficult to compare likes with likes and, after all, there are lies, damned lies, and statistics — but am merely providing some perspective on this piece of news. It is good news to the whole market that online productivity applications are gaining traction. It is, however, pre-mature to jump to the conclusion that Google D&S is the undisputed champion in a sure and settled contest. It’s still early stage and everyone is playing catch up to the desktop giant(s). The race is on — and wide open.

In this spirit we are making available more stats from EditGrid for public inspection. In a typical “eat our dogfood” move I’ve compiled selected statistics onto an online spreadsheet (some of which courtesy of Alexa and Quantcast). I hereby call for our partners and competitors in this field to do the same and publish comparable statistics. Google D&S’s statistics, if available, may still dwarf our statistics combined — yet I’m confident that the competition is nowhere near as one-sided as has been depicted elsewhere.

And, lastly… a belated Kung Hei Fat Choy to you all!

Update (2007-03-02): Ismael Ghalimi of IT|Redux apparently got my message and published comparable statistics from the ThinkFree and Zoho online office suites. These re-enforced our message — the market is still wide open and up for grabs.

More Update (2007-03-06): With kind permission from Ismael I’ve updated the statistics comparison spreadsheet to include the ThinkFree and Zoho figures.

Quoting ecmanaut:

“EditGrid does lots of things right, and in great style. They even care about making your URLs look good. Their developer interfaces are paid equal amounts of love and attention as are their user interfaces, which in this case is great praise for both.”

This is where work satisfaction comes from! We are so pleased to hear users’ response like that. Especially when this highlights a lot of our efforts that may have gone unnoticed.

We have spent lots of efforts in making EditGrid developer-friendly. We are no big brand - not yet anyway - and therefore the only way to get developers’ attention is to make good stuffs for them. And that’s why it’s particularly pleasing to get some praise on this front. Thank you. Thank you so much.

At the same time we feel ashamed. We remember our long to-do list with many more interesting and useful features in store but not yet put into reality. We are not there yet. There’s still a lot to do. Thanks ecmanaut for his compliments and reminder.

Re: full support of JSONP as recommended by ecmanaut. It is issue #6170 on our issue list. We’ll get that done soon.

In 2006, we brought EditGrid to life. In 2007, we will continue to innovate in the online spreadsheet domain.

What can you expect from us? Hmm… a lot.

Here’s an outline of our vision for 2007. The following diagram is what we think the current situation of enterprise software market is (Inspired by Rod Boothby).

Large vendors are still dominating a multi-billion market of software spending where the needs are sophisticated and the stake is high. Software-as-a-Service (”SaaS”) is emerging in the form of web apps, grasping a sizable market share. If we telescope into the complete landscape of enterprise software needs, we can see that a large chunk of business needs is still being served by millions of spreadsheets built by the users themselves. A good deal of simple and specific needs are, well, unserved.

What do the vast amount of users in this long tail really need? They need, first of all, simplicity. The conventional paradigm of “application + database” combo is an overkill. In most cases, what is required is simply a user-interface to manipulate structured or semi-structured data — preferrably with primitive programming support.

The other major need in the long tail is collaboration. The collaborative spreadsheet problem, as we name it, is a common phenomenon. Sending spreadsheets by email and then manually merge different sources into the “final version” is too painful a process to go through — but it was a must for multi-user collaboration.

We see high potential in the field of online spreadsheet. Spreadsheets are simple and flexible. A spreadsheet application is almost the simplest way you can get to manage structured data. It is easy to use and learn. It can even manage semi-structured data. What’s more, it supports a good degree of programmability through formula and functions.

And then you add the missing piece, online collaboration. This compliments the traditional spreadsheet’s single biggest failure, its largely single-user mode of operation. With online collaboration features, spreadsheets can easily become the medium of choice for many kinds of data management and manipulation.

This is what we think future landscape will look like.

EditGrid is not there yet. We’re still assembling a few missing pieces. This is our innovation focus in 2007.

Stay tuned.

This is a late post. We’ve had some busy days, thanks to the Taiwan earthquake. Anyway, late is better then never.

Much happened in 2006. It was when EditGrid was launched. When Google joined and brought lots of attention to this market. When Google acquired Jotspot and hired iRows founders - taking away two players in the trade. It was also when Num Sum, the first online spreadsheet in market, ceased developing and Numbler went open-sourced.

Now the online spreadsheet offerings that remain are (in no particular order): Google Spreadsheets (as part of Google Docs & Spreadsheets), Zoho Sheet (as part of Zoho Office Suite), WikiCalc (as part of SocialText), ThinkFree Calc (as part of Thinkfree Office) and, of course, EditGrid. It seems that we are the only standalone offering here.

Cool, we are lonely now. I think it is good to be focused, for 2 reasons:

1. As an start-up, we never want to be everything to compete with everybody.
2. We still have lots to work to do in this technology domain. The basics are not good enough. We want to make EditGrid faster, capable to load larger spreadsheet, etc. It is too early to diversify.

“Nice, but… you are not a complete solution!” Oh, yes, may be. We have a growing community of loyal users who use EditGrid everyday. I don’t know whether they see us as a complete solution. There may be customers on the street who would come in and say, “Hey, I need a wiki, too!” I can’t tell him/her to open a spreadsheet and put all their wiki article in one cell, can I?

Besides serving our users at the EditGrid.com, we are serving as the technology provider of online spreadsheets, too. Say, if a wiki service come to us and would like to integrate with EditGrid, we’ll work our socks off to support them to make a seamless integration. With partnership, we make EditGrid available to a wider audience, and in the process make both us and our partner a more complete solution. Some active integrations are under way, and don’t be surprised if you find EditGrid available in other online services soon.

By forming partnership, we are making lots of friends. We can focus to deliver a better technology and make it easier to be integrated. If you want EditGrid power in your site, the entry point is this integration guide. I will be waiting for you at my email/IM/skype/phone…

All the best in 2007!