Nobody likes bugs. They're annoying, messy, unprofessional and embarrassing. Although, when it's someone else's bugs, I do find it amusing. I admit I love it when I see Google or Yahoo throw an error, and the best one yet was a 10ft high by 20ft wide video display in a New York City Sprint store that had a big empty black screen and this lovely message facing Broadway for everyone to see:
 
"Windows could not start because the following file is missing..."
 
So, what's the best way to catch these bugs (or defects) and eliminate them, before they become those irritating problems that frustrate end users? The general strategy needs to be test early and test often. But 'test' doesn't just mean the programmer completes development of the new feature, moves it to a testing site and then the QA team jumps on it and runs their tests. The idea is illustrated beautifully here, with a chart that I've become very fond of and used in an earlier post:
 

This chart, produced by Construx Software, shows the stages in the project where defects are introduced, and the cost of repairing defects at each stage of the project. The color scheme and curves remind me of flames, and if you think about it that way then the later you get in your project the bigger your fires get and the harder (and more expensive) they are to put out. The chart shows that most projects devote their resources to repairing defects at the very late stages of the project, in system test. Repairing defects at the system test stage, or the post launch stage is considerably more difficult and time consuming then if the defect were caught earlier in the project and repaired at that point. Part of the challenge of raising the level of quality and finding and reparing defects early is realizing the potential for defects to be introduced early. It's too easy to blame a bug on a programmer's bad code or missing code (not that the Flightpath programmers ever produce any bad code, that's a hypothetical software firm I am referring to), but what Quality Assurance engineers need to realize is that many defects can be created in the earliest stages of a project, before a programmer even gets involved. Designers and Technical Architects can make mistakes too (but of course Project Managers never make mistakes), and the more of those mistakes that can be caught, the healthier the project will be. Having grown up playing goalkeeper on my soccer team, I like to use an analogy from the game - everyone likes to blame the goalie when the team gets scored on, but people easily forget that there were 10 other people on the field that could have made mistakes before the ball got back to the net. Similar to many software project defects, they can be created anywhere along the process.

 
So, test early and test often:
 

• Review the requirements document with the customer, to confirm that all are on exactly the same page with the goals of the project
  
• Involve the QA engineers early on in the development of the specifications document and testing scripts, to verify that the requirements are met, and as many issues as possible are considered and properly handled
  

 
Frederick Brooks, in The Mythical Man Month, writes:
 
"Long before any code exists, the specification must be handed to an outside testing group to be scrutinized for completeness and clarity. As [V.A.] Vyssotsky [of Bell Telephone Laboratories' Safeguard Project] says, the developers themselves cannot do this: 'They won't tell you they don't understand it; they will happily invent their way through the gaps and obscurities.' "
 
When the project is in the development stages, continue to test early and often:
 

• Incorporate regular programmer peer reviews of code and functionality
  
• Schedule feature reviews & tests whenever possible, for both programming team and QA engineers
  

Ideally if all of these steps are taken (and I'm sure there's much more that can be done that I haven't mentioned), then when the project is ready for system test many of the issues have already been identified and resolved. This is not to say that no defects will be found in system test, but hopefully many less than if these steps weren't taken.
 
A couple of things to keep in mind while in full system test:

• Test the pieces isolated and then test them together. If a piece passed an earlier feature review, test it again and don't just assume that it will work fine with the other pieces
  
• Test thoroughly within the mainline scenarios and don't forget to hit the edges and the outside, as Frederick Brooks writes in his book when discussing test cases for software:
  
  1. Mainline cases that test the program's chief functions for commonly encountered data (the most time will be spent here).
     
  2. Barely legitimate cases that probe the edge of the input data domain, ensuring that largest possible values, smallest possible values, and all kinds of valid exceptions work.
     
  3. Barely legitimate cases that probe the domain boundary from the other side, ensuring that invalid inputs raise proper diagnostic messages.


• Once a defect has been fixed and verified in the test system, review all other areas tested and confirm they still work properly. This can be the most time consuming task, but critical. When this task can be automated, even better.

I'd be lying if I said that all of my projects launch bug free, but I am trying to incorporate more and more of these processes into my projects and will continue to strive to keep the bug count down.

Happy Bug Hunting!

 

In case you haven't heard the news, there's a new search engine in town, and they are gunning for Google. It's called Cuil (http://www.cuil.com), and the company claims to "search more pages on the Web than anyone else—three times as many as Google and ten times as many as Microsoft." More about Cuil straight from the horse's mouth ("About" section):

Rather than rely on superficial popularity metrics, Cuil searches for and ranks pages based on their content and relevance. When we find a page with your keywords, we stay on that page and analyze the rest of its content, its concepts, their inter-relationships and the page’s coherency.

Then we offer you helpful choices and suggestions until you find the page you want and that you know is out there. We believe that analyzing the Web rather than our users is a more useful approach, so we don’t collect data about you and your habits, lest we are tempted to peek. With Cuil, your search history is always private.

Others have tried, unsuccessfully, to challenge Google's dominance in the search arena. Is bigger always better? Cuil will certainly return more results from more sources, but the question and primary concern as a searcher remains, will they be relevant? Make no mistake. relevance will always be THE key. I spent ten minutues searching for Whitney Houston's latest leaked single, and I can't tell you how frusterating it is to have to weed through pages of reviews of her "new" track, from 2003.

Google may be THE current undisputed search leader, but the company that remains there will ultimately be the one that adapts to the ways in which people will come to utilize search in the future. The key to success is evolving with your user or customer. At the very least, companies like Cuil keep Google on their toes (as well as keeping Google's research and development department's budget inflated). At best, they provide us with a more pleasurable search experience and a reminder that like tennis great Roger Federer, nothing or no one is infallible.

 

 

 

 

This is the third installment of a three part blog about hiring.  Click on the staffing link to view all three entries.

PART III: The Show and the Pudding

I have mentioned a phrase we kick around here a lot: Trust But Verify. But there is an even better phrase to describe what we are doing here. We have all heard the old proverb the proof is in the pudding. Well it turns out that as I prepared for this blog, I learned something.  The correct proverb is actually: The proof of the pudding is in the eating. It means that the true value or quality of something can only be judged when it's put to use or tried and tested.

This is my favorite part of the interview process, because at this point we have done our due diligence on the candidates, so we know if they make it through the gauntlet of the application and the phone interviews they are more or less hire-able. So what do we do?

We test the candidates….again! That’s right. Some would say that this may be a waste of time or insulting to a candidate.  I would say, if you know your stuff, you shouldn’t be offended if someone who is going to pay you to do something asks you to demonstrate your ability to do that work right in front of him.  I think this is especially true in our industry which lacks standards that you find in many other disciplines.  And what’s more, if a candidate doesn’t want to take a simple test, I think we have a right to be skeptical.

After the live testing is done, we begin a traditional face to face interview process that includes a twist. As we interview, the phone in the conference room rings and we instruct the candidate to take the call and role play a practical phone session with a live person on the other end of the phone as best as they can. That’s right; we ask them to make some pudding right there in front of us.

Then we eat the pudding. We watch and listen to see how instinctive the candidate is in managing this spontaneous situation. Our customer service guru, Cavol Forbes listens and critiques technique of the candidate, as Dina Garfinkel, PMP and lead project manager for UJC, pretends to be a frustrated admin who has been instructed to update a web page but doesn’t even know what internet explorer is or the url for the website she is supposed to make changes to.

This kind of pudding really helps us evaluate the candidates. It also incorporates 2 more staff members into the process bring the exposure of the candidate to the staff up to 4 people. This contributes to a community sense of ownership in the final hiring decision as well as gives the candidate more of a sense of the kind of people he or she would be working with. I don’t underestimate the power of how important that is for everyone involved.

In the end it is usually a fun process for everyone involved. The candidates are usually flabbergasted at our audacity at first but they typically say something like, ”that was the best interview ever” or “that was a fun interview.” It leaves them wanting a little more Flightpath.  I think that comes into play when we move into the offer phase and a candidate has to choose among several offers. 

 

The smart people over at Disney.com – the top internet destination for children’s entertainment – are in the midst of their second major site overhaul in a relatively short amount of time.

 

According to the New York Times, staid, old-school primary navigational options like “Movies,” “TV,” and “Live Events” are being phased out in favor of more action-oriented choices like “Games,” “Videos,” and “Characters.”

 

That’s not to say that Movies will no longer be represented at the site.  Actually, it’s the opposite… instead of unsatisfying short movie clips, the site will present full-length theatrical releases – cementing the sites’ role as a bona fide entertainment destination and migrating away from its former role, primarily serving as a crass promotional portal.

 

There’ll also be virtual worlds, sophisticated games, and plenty of ways to interact with your cell phones. 

 

All of these new offerings, combined with Disney’s embrace of its position as a bona fide web entertainment destination, help illustrate a seismic shift in the way that young people are spending their time.  This also heralds an important shift in diversions and amusements for the rest of us.

 

Steve Wadsworth, president of the Walt Disney Internet Group explained “our initial instincts were right, we just need to take it much further.” – There’s little doubt that the general public is eager to come along for the ride.

 

- Jon Fox

 

Hal R. Varian, Chief Economist at Google was recently quoted saying “The source of Google’s competitive advantage is learning by doing.”  Granted they’ve got a gigantic team of engineers and access to practically unlimited resources, but I think that this is a mindset that can effectively be employed by all kinds of organizations, including a mid-size interactive agency like Flightpath.

 

It’s apparent to Google that (a) interactive technology is evolving at a dizzying pace, and (b) the ways that consumers are utilizing interactive technology are changing just as rapidly.  Hence, in order to maintain their leadership position, it’s important that Google continuously tests, refines, and improves its offerings. 

 

Within the body of the primary Flightpath web site, you’ll find an illustration that looks like this:

 

 

This diagram intends to show a continuous cycle of evolution baked directly into Flightpath’s service offerings, keeping in mind the end goal of improving the performance of our clients’ businesses  (good news).

 

For better or for worse, this diagram doesn’t directly account for changes in interactive technologies and way people use them (bad news).  Also, what happens if a client or prospect of ours needs help with business challenges that we haven’t dealt with before?  This is where an organization like ours can also gain competitive advantage by learning by doing. 

 

In the past year, Flightpath has significantly grown its competency in things like online video, blogging, search marketing and optimization, social media, email, intranets, CMS deployment, widgets, adoption of open source technologies and more.  This was the direct result of learning by doing – many times via internal projects, sometimes (transparently) in partnership with clients.  Apart from these realms, other immediate challenges where we can continue to learn by doing include developing iPhone applications, mobile web offerings, SMS, ERP, CRM integrations and more.

 

Continuous experimentation and practice in new practice areas is rapidly being ingrained into the Flightpath culture.  Likewise, “learning by doing” is a mantra that can be adopted effectively by all kinds of organizations (and individuals too).  This is one more important lesson that we can thank our friends at Google for reaffirming.

 

- Jon Fox

    

On July 2, there was an interesting article in the New York Times, that discussed a new wrinkle in the successful design of some new shopping centers in Texas and other Southwestern destinations – the developers asked women what they wanted to see.  Here’s a quick excerpt:

The women weighed in on dozens of features, like the center’s layout, landscaping, parking options, pedestrian walkways and outdoor art. The developers “asked us about every detail, and then they listened.”…

Listening to women shoppers may seem like an entirely logical thing to do, yet many retail developers and consultants say such participation is often missing during the early stages of shopping center development.

I was immediately struck with the parallels when considering web site development – these two short paragraphs provoke four important corollaries to the web site design process:

1)   Identify your audience: I haven’t seen the stats, but I’m sure that the core constituency of suburban shopping centers is women, including a sizable number of mothers with children.  In the world of web site design, this should be an initial factor that you’d want to clearly identify.

2)   Drill down to the details: In designing a web site, it’s often the use of specific words, icons, visual cues, or interface elements that will define a positive experience vs. a negative experience for the user.  An example used in the shopping center analogy was walkways that could readily accommodate multiple baby strollers side-by-side for a hassle-free experience.

3)   Listen to user feedback: If you’re going to take the time to solicit these ideas, they should likely be heeded, rather than tossed out in the name of expediency or professional expertise.

4)   Get user participation early: True, it may be easier to re-engineer a web site post-launch compared to a shopping center, but by gathering user input early in the process, you’ll have a valuable set of assumptions to build upon which can minimize project swirling and ultimately lead to a better product.

A final point in the article was that instead of relying on formal questionnaires and focus groups, the developers chose to engage their consumer advisors with informal box lunch get-togethers.   This kind of approach buttresses a regular Flightpath principle which is that you don’t necessarily need/want formal (read: expensive) consumer testing in order to gather valuable user input.

- Jon Fox

On this Independence Day weekend we remember the struggles of our forefathers (and mothers), the separation from Britain and the Declaration of Independence that brought it.

Ok, let's be honest...we really spend the days thinking about grilling, fireworks, getting in on the big sales or getting away somewhere interesting. Me, I'll be spending the time off trying out my new inflatable kiddie pool with canopy and sprinkler (I won't actually be in the pool, the kids will of course).

But, on the topic of independence, liberty and justice, I’ve recently been reading about the Customer and Project Team’s Bill of Rights in the book ‘Software Project Survival Guide’, by Steve McConnell. He introduces this idea by comparing a poorly run software project to 17th century life under mob rule. This experience can be “solitary, poor, nasty, brutish, and hardly ever short enough.” In order for a software project to survive, all parties need to agree to treat each other in a civilized way. These rules of civilization he summarizes first in the Customer’s Bill of Rights:

CUSTOMER'S BILL OF RIGHTS
I have the right:
1. To set objectives for the project and have them followed
2. To know how long the software project will take and how much it will cost
3. To decide which features are in and which are out of the software
4. To make reasonable changes to requirements throughout the course of the project and to know the costs of making those changes
5. To know the project's status clearly and confidently
6. To be apprised regularly of risks that could affect cost, schedule, or quality, and to be provided with options for addressing potential problems
7. To have ready access to project deliverables throughout the project

But, this is a two-way street. On a software project, customers must pay for their rights by respecting the project team's rights, which he lists below:

PROJECT TEAM’S BILL OF RIGHTS
I have the right:
1. To know the project objectives and to clarify priorities.
2. To know in detail what product I'm supposed to build and to clarify the product definition if it is unclear.
3. To have ready access to the customer, manager, marketer, or other person responsible for making decisions about the software's functionality.
4. To work each phase of the project in a technically responsible way, especially to not be forced to start coding too early in the project.
5. To approve effort and schedule estimates for any work that I will be asked to perform, This includes the right to provide only the kinds of cost and schedule estimates that are theoretically possible at each stage of the project; to take the time needed to create meaningful estimates; and to revise estimates whenever the project's requirements change.
6. To have my project's status reported accurately to customers and upper management.
7. To work in a productive environment free from frequent interruptions and distractions, especially during critical parts of the project.

I would love to see a project kickoff meeting where all parties gather around a large table to read through the Bill of Rights document together and clients, managers, senior management, project team all take their turn to sign the document. This will affirm their commitment to upholding their responsibilities (although it seems to be mainly the responsibility of the project manager to make sure all of these rights are upheld). Who knows, maybe some project teams out there go through that exact process, or maybe this could be included as a part of the project charter. It’s an interesting idea, but at the very basic level I agree with the author that everything listed here is extremely important for the happiness of the client and project team, and the health and success of the project. I hope to find a way to incorporate this into my projects going forward, so if you’re my client or on my team, better keep a quill handy!