Testing the Claims of Test Automation

** For more stories on test automation, read the full eBook “Uptime Guarantees Begin with Upfront Quality Assurance,” sponsored by Mu Dynamics, at www.xchangemag.com/mudynamics. **

One of the core tenets of the scientific method is that experiments and tests must be repeatable. That holds true whether testing takes place in the labs of the University of New Hampshire or Verizon Communications Inc., AT&T Inc. or Juniper Networks, Joliet Junior College or One Communications Inc. Preferably, when running millions of iterations and variations, the testing must be something else as well: automated.

Test automation is a no-brainer as ideas go. However, it takes an awful lot of brainpower to bring such an idea to fruition. So far, it is still a work in progress and always may be as network technologies change or one company’s innovation trumps another.

Elisabeth Rainge, director of NGN Operations at IDC, said much skepticism surrounds the expectations of test automation and that it will continue to struggle for acceptance unless network equipment manufacturers and service providers can eliminate the people problem by removing them from the process and convince engineers they need to focus on achieving business goals and not just efficiency and design goals.

In terms of spending too much time on testing, “automation is essential,” Rainge said. “However, what automation is about is turning the process from one driven by technology to one driven by business goals.”

Still, technology must drive the capabilities and technology companies aren’t thinking small when it comes to automation. Two of the larger vendors in network testing, Spirent Communications Inc. and Ixia Communications, recently formed alliances around developing frameworks for test integration and automation. And companies such as Mu Dynamics, Fanfare, EdenTree Technologies (now Gale Technologies), Apcon, Shunra and others all are contributing to the effort.

The Test Automation Alliance (TAA) was announced by Spirent in June this year. The TAA will be an ecosystem of testing and lab management companies that will form a framework made up of open standards to make test automation a reality. In addition to EdenTree, Fanfare and MRV Communications Inc., the alliance will leverage the Eclipse community. Eclipse is an open-source community whose projects are focused on building an open development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle. The Eclipse Project was founded in 2001 by IBM Corp.

“Our customers kept saying there was added value in common integration. So we are setting up common integration standards that will help us pull our values together so we achieve integration and so the customer doesn’t have to,” said Patrick Johnson, director at Spirent.

This is easier said than done. Johnson acknowledges that vendors are challenged to work together because “we all want customers to use our platforms and innovations,” he said. “But unless it is all integrated into a [common] environment, customers don’t get the experience they are seeking.”