Recent experiences with SlickEdit 2009 have reminded me that there are no small changes. Even when you think you understand the nature of a change, severe side-effects can pop up in complex software.

This was the case when we put out our first release of SlickEdit 2009, version 14.0.0.5. We had completed our usual beta cycle before releasing this version, which included a number of drops to correct reported problems. After the last beta drop, we made a few more changes, all of which seemed totally risk-free.

Within two hours of releasing SlickEdit 2009, we had reports of severe problems with the File tabs and the File list. These problems trace back to one of these simple, risk-free changes. We had done our normal testing after these changes, and we hadn’t discovered any problems. But this problem was so severe we had to pull the release until it could be fixed.

So, what did we learn from this?

First, there are no small changes. Even when you think you know the effect a change will have, you could be wrong.

Should we have tested more? Sure, you can always test more, but that is no guarantee that you will find a problem. Testing suffers from a horizon effect: you can only see as far as you test. The problem you need to find could lay just beyond or miles beyond. You don’t know until you find it.

Second, testing is different than using. When you are testing, you click all the buttons and try all the functions, but you are rarely able to perform all combinations of operations. Some problems depend on the order in which operations are invoked. Others require multiple operations before the problem surfaces, as was the case with the problem we missed.

It should be no surprise that the parts of SlickEdit that work the best are the things we use everyday. But there are many languages we don’t use on a daily basis. Even if we did, with a product like SlickEdit, our testing cannot hope to replicate the variety and complexity of our customers’ use patterns.

We’ve determined that prior to shipping any release, we need help validating it. So, we’ll create a Release Candidate when we have finished our testing. We’ll make the Release Candidate available for early adopters to test, and if all goes well, it will be used for the General Availability release. In this way, we hope to catch more problems before they are set free to roam the countryside and terrorize the populace.

Q&A

Does that mean you’ve given up on testing? No, we plan to continue testing just as we always have. We just need help with final validation.

What if I don’t have time to help you catch problems? Then wait until a release is no longer a release candidate. At that point, others will have helped to validate it, and you will have less likelihood of experiencing issues with a new version.

If an unladen swallow flies west from New York…

When I was a kid, I was raised on Sesame Street.  I learned some counting and some reading, but most importantly, I learned how to pay attention in 3 minute chunks of time.   Over the years, that time has been cut down to somewhere around 30 seconds, which means that the advertising industry has done their job well.  Unfortunately, there’s a large group of people that share my short attention span… end users.

Read The Friendly Manual… or not

Many software products have changed over the years to cater to end users who don’t want to spend a lot of time learning how to use new software.  I first saw this in the game industry, where people can’t seem to be bothered to read a manual at all.  These days, games don’t even come with instructions.  They provide a tutorial level or integrated help that watches what you do and introduces you to the things you need to know as you play.  It’s brilliant, really.  The users don’t have to read a thing before playing, and who’d want to?  When I get a new game, I don’t want to waste time reading when I could be already up to level 10.  The game companies win too, because they don’t have to pay for the printed materials.  Everyone’s happy.

But how does this translate outside the game industry?  It’s been tried before, with much less success.  Clippy was the spokesman for the first wave of learn-as-you-go user experiences.  We all know how that turned out.  “One of the worst software design blunders in the annals of computing,” wrote Smithsonian Magazine.

So users didn’t like being told how to use the software while they used it.  Another form of learning that is not always well received is the tip of the day.  I know my first order of business after installing new software is checking the “Never, never, ever show me this again” checkbox on the tip of the day dialog.

Wizards also tried to ease the user into getting up and going without having to wade through options pages and understanding all of the menus and toolbars.  Although wizards have had some success, it’s still a paradigm that’s seems old.  When I think of wizards, I feel like I’m flashing back on an episode of “I Love the 90s“.

At the very least , there hasn’t been a lack of effort to try to help users figure out how to use their software without the manual.  Yet, most of these attempts seem to fail.  It’s as if users don’t want to read how to use the software and can’t be bothered being shown how to use it by the software itself.  Part of me wants to rant about how idiotic that is and the other half of me recognizes that I fall into that group.

Done programming?  Great, now get back to work.

I’ve spent most of my programming career writing programs for in-house use, or writing programs that were bought by large companies to be rolled out in-house.  The thing that all of my work had in common was that the end users were not end users by choice.  The software I helped write was all rolled out in some fashion to a large group of people whose jobs depended on them learning the software.

Here at SlickEdit, I’m no longer writing software that users must adapt to.  Instead, I’m writing software that users  make a conscious choice to use or not use.  The choice to use it hinges heavily on the trial, and if the user can’t get the trial installed and doing spectacular feats in less than five minutes, then the opportunity is completely lost. The bottom line is that the most important software performance metric is not how fast my code works, but how fast the user can use it meaningfully.  If that doesn’t happen, then the user loses interest and all of the great work we’ve done will never be seen.

That’s quite a psychological transition for me, because I’ve never focused before on instant user gratification.  One of our strategies involves the software helping users through an optional tool window, called the Tools Assistant,  which is part of the “Tools for Visual Studio” product.  It stays off to the side or can be closed if the user doesn’t want the help.  But there’s also a lot more to this effort than just adding up-front convenience to the software.  We also have to think up new ways to get users to see the product in action with very little effort on their part, often without even downloading it and installing it.  This involves a lot of non-programming work, like video script writing, screen shots, articles, etc…  It’s certainly an interesting change of pace, but it really pushes the limits outside of my programming turf.  I took a break the other day to fix bugs and it was refreshing just to see code for a while.

I love a change of pace… it keeps things interesting.   However, trying to figure out how to pitch a software product may be one of the biggest challenges I’ve had as part of a software development team.  I certainly never took any marketing classes in college, and at this point they’d all be irrelevant anyway.  So I’m no longer focused on reducing the time it takes to perform complex operations… my focus now is on reducing the amount of time it takes a new user to say, “wow, that is so cool!”  So far I can’t find any way to calculate the big O for that, but I’m trying.

Core 3.4.0 introduced the Slick-C function _eclipse_execute_command which allows you to execute Eclipse commands from the SlickEdit environment.  The documentation for this function shows you how you can search the Plug-in Registry to find Eclipse command names and ids, but there is a much better way documented in the following blog post:  http://evans-stuff.blogspot.com/2007/10/it-seems-that-im-not-only-one-still.html . 

Unfortunately I discovered this technique after we finished the documentation and shipped the plug-in, but hopefully me pointing it out here may help some users. 

Also, don’t forget the “Activate Editor” command in Eclipse (F12 by default).  If you are using _eclipse_execute_command to jump into different Eclipse Views and search around mouse-less, hitting F12 will jump you back into the last active editor.

<update> 

Check out this thread of the SlickEdit Forums for adding a generic Eclipse Pass-through Command: http://community.slickedit.com/index.php?topic=3549.0

From the post:
If you are evaluating 3.4.0 for your company, you can make this happen easily.  One of the best features in Core 3.4.0 is the introduction of _eclipse_execute_command, which is discussed in the user guide/readme, and also a little more in my post here: http://community.slickedit.com/index.php?topic=3816.0 .  This is the idea that Rob suggests in this thread.

I’ll take this opportunity to give a brief tutorial on how to integrate that “Activate Task” shortcut into the SlickEdit environment.

First you need to know the command id of what you are trying to execute.  I used the method described http://evans-stuff.blogspot.com/2007/10/it-seems-that-im-not-only-one-still.html , in which you create a Simple Cheat Sheet and use the Command Browse button (see first screenshot).

Then I just used the search field to find the command (see second screenshot).

Now that I had the command/parameter info, I opened up the Slick-C macro file where I write my custom commands (ryan.e), and added functions for Activate Task and Deactivate Task using _eclipse_execute_command (see third screenshot).  Remember to load your macro file with F12 after writing the functions.

Then I used the SlickEdit Keybindings dialog to bind eclipse_activate_task to Ctrl + F9 (see fourth screenshot), and voila.  Hitting Ctrl + F9 in a SlickEdit editor will launch the Mylyn Activate Task dialog.

It may seem like a lot of steps, but it’s really not.  This whole process took me less than 10 minutes including the time I took to capture/crop/annotate the screenshots.  You can use this method to execute almost any Eclipse command from any plug-in from the SlickEdit environment.

- Ryan