WLAN vendors have long relied on VeriWave for both closed (chambered) and open air (lab or field) tests, ranging from early development and QA verification to small proof-of-concepts and RFP bake-offs. However, as 802.11n becomes a mainstream technology, a more daunting demand has emerged: large-scale, real-world-loaded, in-situ interoperability testing by enterprises and their solution providers.
This week, VeriWave announced a new software package, aimed squarely at this nascent market: WaveAgent 2.0 for Interoperability Testing. This tiny (<64Kb) cross-platform utility lets organizations run rigorous, repeatable open-air tests by using a VeriWave WaveTest 90/20 platform to drive and measure a large number of third-party, mobile 802.11 clients, operating in their native habitat.
Hitting the pavement
Performance testing in a controlled environment is tricky, but can be tackled in a rigorous, methodical manner by using RF traffic generation and low-level measurement systems like VeriWave’s WaveTest hardware or WaveClient test bench software. While one could always take a portable WaveTest 20 on-site to conduct a proof-of-concept, running a well-engineered open-air test with a large, heterogeneous client population has been very difficult. And yet, that’s precisely what enterprises now really want to see.
For example, Jake Woodhams, Senior Manager of Technical Marketing at Cisco Systems, says that customers want to see that Cisco’s 802.11n APs really meet the demands imposed by increasingly diverse clients and applications. “The WaveAgent 2.0 interoperability testing solution allows us to conduct verification of real-world-loaded, large networks,” he said. “WaveAgent 2.0 allows us to scale from one AP to multiple APs permitting true system level assessments that better mirror customers’ networks.”
Walk softly and leave no footprints
Not only are large uncontrolled open-air tests hard to analyze and repeat, but customers may impose conditions that complicate more formal test tools. For example, customers may not have a solid handle on their client population or application mix, making it hard to script representative traffic. Moreover, customers may not let testers install control or measurement programs on live client devices.
WaveAgent 2.0 addresses these challenges by extending VeriWave’s open-air tests to incorporate diverse third-party clients, without imposing chipset limitations or leaving a persistent footprint. WaveAgent 2.0 runs at the sockets level as an install-free executable on Windows XP/Vista/Mobile 6, and Linux 2.6. VeriWave staff report using WaveAgent on Windows 7, as well as systems booted from Linux CDs and USB drives. If you can get to a terminal or shell window on a Wi-Fi client and have 200 Kbytes of free memory, you can run WaveAgent. However, WaveAgent does not yet run on iPhone or BlackBerry.
WaveAgent does not control the client’s WLAN connection, so an established (or auto-(re)established) association is required. This low-touch approach lets WaveAgent run on many 802.11 clients, using any combo of standard or proprietary options. However, this puts the onus on staff to configure and/or document every WaveAgent client before each run—for example, to enable WMM or disable power save everywhere.
With or without ecosystem traffic
Each test must be orchestrated using VeriWave’s WaveDynamix or WaveQoE application. These control and reporting tools can be used to plot low-level WaveAgent measurements like round-trip forwarding rate, loss, latency, and jitter. WaveQoE can also model complex application traffic scenarios to report on high-level “quality of experience,” observed from the client’s perspective.
For example, WaveQoE can start with a supplied vertical market traffic profile (e.g., enterprise office, retail, hospital, or a custom profile. For each included client device and application, a relevant Service Level Agreement (SLA) is defined (e.g., MOS and R-Value for VoIP, MDI for RTP video, duration and goodput for FTP). WaveQoE then generates modeled traffic, using WaveAgent to take and report client-side measurements, then determining success based on SLA (e.g., was my average MOS score across all VoIP clients high enough?).
Of course it’s one thing to meet SLAs in a lab and another to meet SLAs in a large-scale live network impacted by RF interferers, other clients, and mobility-induced factors like roaming. To measure this, WaveAgent is first used alone to establish a baseline. Next, one or more WaveTest platforms can be added to generate ecosystem traffic that simulates a large number of clients, operating at varied distances, transmitting from one or more directions.
In this way, VeriWave can create a much larger and more heavily-loaded scenario than otherwise feasible using a handful of live clients. But with WaveAgent, desired metrics can now be observed and reported on real-world clients, in their native habitat.
Charting a path through turbulent waters
WaveAgent 2.0 for Interoperability Testing will be released by VeriWave on August 26, 2009. To conduct in-situ tests using the WaveAgent, customers will need at least one WaveTest 20 or 90 system and one WaveDynamix or WaveQoE application (installed on a Windows PC with 1 GHz CPU and 256 MB RAM).
VeriWave expects to see WaveAgent 2.0 used in many 802.11n proof-of-concept, interoperability, and certification tests—activities that will ramp up in late 2009 with ratification of the final 802.11n standard.
“The floodgates will open on new deployments and the introduction of Wi-Fi devices used in mission-critical environments,” said Eran Karoly, VeriWave’s VP of Marketing. “The ability to precisely measure how a new device will impact and perform in a particular user’s ecosystem benefits both manufacturers and enterprise IT professionals.” In particular, Karoly predicts that WaveAgent will help enterprises “decide which products to buy and how to configure the network for maximum performance and reliability.”
Coming in September: a Wi-Fi Planet Industry Insiders interview with Karoly.
Lisa Phifer owns Core Competence, a consulting firm focused on business use of emerging network and security technologies. A 28-year networking industry veteran, Lisa has been involved in the design, implementation, and testing of wireless products and services since 1996.
