Spirent, SRG Offer Smartphone Performance Reality Check

Based on new data from Signals Research Group, developed using Spirent Communications’ 8100 Mobile Device Test System, operators should be taking a closer look at the handsets they send to market, because not all 3G chipsets perform the same.

When it comes to performance issues with today’s advanced mobile handsets, problems may be miniscule at the chip level, but that doesn’t make them insignificant. SRG concluded that there remains a wide variation of data performance capability across the HSPA chipset supplier base, in some cases up to 40 percent.

The study found that industry-standard conformance testing for call reliability may not go far enough. Dramatic differences were found in the ability to reliably make and maintain voice calls by devices that had passed such testing.

Michael Thelander, CEO of Signals Research Group, said the results clearly illustrate that testing chipset and device performance beyond minimum certification requirements uncovers performance differences, which can significantly impact a subscriber’s satisfaction with a particular device.

The device testing from SRG and Spirent included Apple’s 3G iPhone and Research in Motion’s BlackBerry Bold, and other 3G handsets from LG, Motorola, and Sony Ericsson, which all had passed conformance testing. The study also looked at the performance of 3G chipsets from Ericsson Mobile Platforms, Icera, Infineon, InterDigital, Motorola, NEC, Nokia and Qualcomm.

Performance throughput results indicate that there are still major differences in the amount of throughput (data rate) that a chipset can achieve for a given, real-world, network condition. In some cases the difference was in excess of 30 percent. However, this does indicate an improvement over tests done last year.

The best performing 3G chipset in 10 of 12 scenarios, including those under challenging network conditions, was Icera’s.

Somewhat surprising, said Nigel Wright, vice president of product management for Spirent Communications, were the differences in call reliability across commercially available handsets.

“You’d think that six years into 3G deployments you wouldn’t even have to question whether a device can set up a call reliably and not drop it under regular network conditions, but a lot of them do,” Wright said. “When calls drop, subscribers blame the network coverage, but this report highlights that in many cases it is the device itself that can’t cope.”

The iPhone 3G received plenty of negative feedback on its call performance when launched, but testing shows that not only has new firmware made a meaningful improvement, but that other phones on the market have call reliability problems that far surpass those of the iPhone.