Strategy and Execution: The BellSouth Cellular Billing Migration

In this industry, CIO is said to mean “career is over” instead of chief information officer. It’s not true when production costs drop from as high as $7 per bill to about $1 per bill over a six-year period. The information technology team at BellSouth Cellular, led by Russ Akins, Vice President and CIO, accomplished this goal, among others, as they performed a sweeping and staged architecture swap-out and billing migration. The team has moved BellSouth Cellular to a billing system environment where 75% of 692 new pricing plan and 2,086 new feature requests in 4Q98 were implemented in two weeks or less. While other CIOs might be green with envy, the BellSouth Cellular team is green with the financial success of a new billing environment.

Background and Pre-Migration Environment

BellSouth Cellular covers its centralized business in the Southeastern states and its out-of-region decentralized business in areas such as Los Angeles, Houston, Honolulu and Indianapolis. It offers billing services in 27 markets, runs 126 billing cycles, processes 300 million messages per month, and produces 4.5 million bills.

The billing migration was a calculated and planned upgrade from multiple and outmoded billing platforms, allowing all of its markets to share a common billing system. As Akins candidly admits, “Our IT infrastructure and capability was a roadblock for the business as a whole – soon to become a stumbling block if not remedied.”

In BellSouth Cellular’s service areas, growth has been high, and capacity had become a major operational issue. Bill processing time was approaching critical mass and processing time was beyond online downtime windows. With BellSouth’s acquisition of American Cellular in 1990, the overall system architecture had become costly and redundant.

Before the migration, all billing and online customer information ran on IBM and Amdahl mainframes. BellSouth Mobility had bought and modified an earlier CBIS (now Convergys) package with an option for source code. But operating in multiple environments meant change management was a headache, and enhancements were slow and expensive with the demands and different process approaches of multiple wireless markets (see Figures 1 and 2).

The switching environment was already tough, with Ericsson, Lucent, Hughes and Alcatel switches interfacing with the home location register. Even before the billing system overhaul, Akins and team had developed SwitchMaster, a processing interface for each switch that formats messages regardless of air time protocol and technology before sending them to the billing system.

Billing and customer care processing support was moved to BellSouth Information Systems, which had been established to manage affiliate processing on the mainframes. “The long-term goal was to move out of the mainframe environment,” says Akins, “because of cost, turnaround time for change, and all the issues with legacy.”

Akins’ team undertook a customer care capabilities project known as Facelift, aimed at reducing the number of screens to get customer care information, training IT staff on development in a non-dumb terminal environment, and exposing users to the look and feel of graphical user interfaces. Facelift, as the name implies, was a GUI front-end to the legacy systems.

“This approach proved valuable because we spent very little time having to train people on PCs and Windows,” says Akins. “Zero technical computer training was required once we migrated to the new billing system because of the early exposure. This is always a main challenge for big companies evolving from legacy systems.”

Key Drivers, Overall Strategy and Billing Objectives

With the mergers and acquisitions occurring at BellSouth Cellular, and system implications every time, the cost of running multiple systems was escalating. BellSouth also had to drive to one system and a more common approach in billing to get a grip on costs, especially with an annual growth rate of over 40%. At the time, it was costing BellSouth $4 to $7 a month per bill.

The ability to be consistent in business process application, in customer service, and across the board was essential to developers. “If we wanted to implement something in south Florida and the same thing in Indianapolis,” recalls Akins, “the systems were different, the processes were different, and it drove the highest level management team crazy.” The competition in the customer service arena drove the need for capturing and improving performance characteristics such as abandon rate, response time and quality assurance.

While the business drove the technology through its priorities, it was IT’s bailiwick to determine what to build, and in what order - usually. “We live in a world of exceptions,” says Akins. “While I would have replaced the batch billing process first as many other carriers do, because it costs so most to run, the first business priority for the marketing groups was a new rating and pricing module.”

From an IT perspective, the billing migration strategy focused on a staged architecture approach delivering incremental improvements rather than on a big-bang replacement effect. “We actually started in late ’95 with early scoping,” says Akins, “and it was defined as a three-year strategy plan from cradle to grave.” In order to get funded at $10-$15 million a year, Akins had to sell the vision internally. The presidents of the BellSouth Cellular operating companies signed off on three-year strategy, but the overall system replacement and migration had to be paid for out of net income.

The strategy took a little more than a year to fully define. “Different vendors were looked at,” says Parisian, “as was partnering with different vendors. But the business needs for new rating functionality could not wait, so this was part of the decision to move the project in-house.”

Architecturally, it was decided to use Unix for pricing and rating and AS/400 for customer records. From Akins’ viewpoint, the AS/400 was the closest to a mainframe out there that offered a relational database and had a “never breaks” mentality.

While the AS/400 is not viewed as an open system like Unix, with Unix’s versatility, flexibility and dynamic capability, it is the next best thing and is a workhorse when it comes to handling customer records. Because customer care systems must be up all the time, BellSouth Cellular chose not to risk them with Unix. “We could not afford frequent crashes,” Akins says. With advances in Unix, and a tenable history of greater operating stability, BellSouth is reviewing its opportunities to convert data from AS/400 to Unix. “We have ample capacity in the AS/400 environment which is very reliable, it runs all the time, we experience no inability to do any type of record transfers, or sharing of data, zero integration problems. However, our team continues to access and utilize the new technology, including Internet, that can handle large scale enterprise requirements.”

Based on the staged approach and the marketing needs, a set of fairly universal billing objectives fell into line. IT needed to deliver a more flexible rating and pricing capability, provide tools for improving customer service while enhancing productivity to reduce operating costs, implement a scaleable architecture with common systems, and reduce billing cost per unit. But IT could not put the business in a holding pattern while awaiting change.

“For the migration itself,” points out Parisian, “we had to convert more than four million customer records from the old COBOL environment.” The actual customer migration strategy was to move customers by market and leave them in their existing billing cycles. BellSouth Cellular started with a small market, then two medium markets, then recognizing the need for greater pay out, and having gotten its feet wet, they migrated a major market next.

The strategy was executed as planned over a four-year period.

The New Blueprint

IT established a blueprint before it went shopping and knew it wanted open systems, client-server, graphical user interfaces, and a common data repository, along with functional modules for varying applications serving different needs of the business. After BellSouth was finished, very little of its existing business systems architecture would be left untouched (see Figure 3).

After considering its existing vendor solutions, reviewing new vendor solutions, and attempting to partner, the company decided to take the best they had and develop in-house to fill gaps. As a result, BellSouth was driven to the staged approach, which has allowed IT to develop and implement functional modules and to build on each success, using an iterative approach now finding favor with development executives. Payouts are realized sooner, the business gains progressively, and no one is left awaiting a very risky flash-cut.

Developing the Migration Baseline

BellSouth Cellular implemented its middle stage architecture, thus effectively leveraging existing investment and capability, while development ensued. (See Figure 4.) Nearly every system ancillary to billing was touched. The blueprint roadmap to today used as a springboard the Facelift project in 1993 that established the GUI customer service front end to existing applications to improve customer service information access and response time. An inexpensive first step, it met immediate needs of the business. This was evolved further with the CARE project that began being deployed in 1997, and gave sorely needed enhancements to the existing customer database and customer operations online system. In addition, CARE was constructed with standard APIs for subsystem integration.

IT also developed a new pricing module and a rating engine for all message types. With table-driven, flexible pricing, changes such as new calling plans could be made virtually overnight. This allowed the company to meet competitive threats such as roam with home pricing, toll service, first incoming minute free, mobile to mobile pricing, and other rating methods and calling plans. The new pricing and rating capability was rolled out to all BellSouth Cellular markets between 1995 and 1997 in a phased deployment, providing payback in the first year of use. It eliminated significant time from the mainframe batch processing, reducing costs from BellSouth Information Services.

Concurrently, IT was developing its bill generation module, WISDOM, which takes data from the customer database, rated message data from pricing, and applies discounts, taxes, adjustments, and monthly charges. Allowing for flexible bill formats and bill messages, it creates a bill transmission file for bill presentation services (in this case IBS). Upon deployment of the bill generation module, all mainframe bill-processing costs were eliminated (see Figure 5 for current architecture view).

Although numerous contractors and vendors were used, the project was self-directed, with the internal IT leadership team serving as the systems integrator. “We did not hire a big consulting firm or product offering group because we wanted to instill a greater sense of urgency,” says Akins. “Sort of the old adage, ‘You make your bed, you sleep in it.’” The team used IBM and Sun on the platforms and hardware; technical operations were handled by BellSouth Information Services.

Methodology

BellSouth Cellular’s business strategies group serves as the point of contact for the various business entities during development and deployment. It focuses on areas such as engineering - where the switching environment and network fabric affects provisioning and other interfaces - in addition to billing captures. “The business strategy group is our sanity check between the traditional IT tendency to say ‘everything is great’” says Akins, “and the business people screaming, ‘no it’s not.’”

BellSouth Cellular, in its normal business milieu, has an R&D group for new product offerings, and a segment of the business strategies group stays involved. Using MIS Project Authority (MPA) to start a project and then evaluate it at the end, BellSouth Cellular has characteristically used its Quest product development methodology, which is being refined now. “MPA allows us to loop back around to see whether initial funding premises held,” says Akins. “We look at return on investment, such as a retail stores efficiency project that allowed us to reduce support costs.”

Customer Migration Planning and Development

IT tried not to affect customer services, although there were some differences. The business area with the most re-engineering was collections. The collections transition had already started, and a lot of what the Customer Operations group wanted was included in the new customer interface system, CARE (the customer online piece). The old system relied on monthly/cycle aging, while the new system bases aging on a daily basis – so bad debts estimates were different.

The actual data conversion required specific code. IT wrote programs to go from the source system to target system. Customer movement was based on a market-by-market approach, concurrent with the implementation of the new customer facing module and the backend bill calculation module. The first market BellSouth migrated was small; then, two mid-size markets and a large market were tackled. Then, migration progressed by state, because some states have centralized call centers and IT wanted to make sure customer service reps were not referencing two systems to take calls.

During the development of the baseline architecture, the modular approach had to shift when one module got too large. Not obvious to part of the early development team was the need to have rating and bill calculation as separate modules. Once this was realized, the resource was re-deployed from one group to the next.

The modular approach was aligned to the business strategy with the rating engine project first, because BellSouth knew that with the new architecture it could add incremental value along the way. Many development executives now favor this approach with incremental builds.

IT had a master project schedule once they started deployment, but the time between the first small market to the large market (with two mid-sized markets between) was longer than originally anticipated, due to a continuing enhancement of the legacy systems to meet immediate business needs. Regulatory issues also surfaced. BellSouth Cellular assessed the situation late last year and put together an accelerated deployment plan. They met their new schedule, and the migration is 100% complete.

Bill verification procedures changed, but IT had already laid the foundation by developing a web-based (Intranet) bill verification tool suite. By the time customer migrations began, the tool suite was usable for the old and new systems, and interfaced to different reports and data specific for each system. The tool allows sampling of calling plans - most frequently used, toll calls – and on request would bring back customer samples, from either the new or the old system.

Execution

Data conversion between source and target included having the markets (Customer Operations) help with address cleansing. The old system did not have fixed field addresses and could be inputted in any format, but the new system required a translation to fixed fields. During conversion dry runs, reports listed addresses and other data that did not translate correctly. The market scrubbed the addresses before the live conversion.

A typical migration by market, once fine-tuned, resembled this: first a market kick-off meeting was held by IT to explain the market’s role, the training process (not to train too far in advance), what would happen on conversion weekend, procedures, and so on. Closer to conversion, weekly meetings were held and a team was formed with an IT project manager and market lead, who would coordinate all activities with CSRs, sales, and other groups. Next, IT ran the process approximately 50 times, processing it through dry-run billings and parallel billings. Snapshots of data were taken at certain points, and IT would run this data through the new billing system with the converted data in a parallel test environment, comparing the output of the billings. Two to three weeks before “conversion weekend,” the market group completed some controlled data entry to get used to the process and view the parallel billings. Finally, the process would be executed.

IT retained all old data for 60 days. If something looked strange in the new system, the team could go back and scrutinize the old system. All customer histories – notes, changes, etc. – were converted and migrated to the new system. Twelve months of bill images were converted on the first migration; on later conversions, with an optical system in place, three months of images were converted.

Every week the migration team would do a checkpoint review. With so many files to move around, there were times where updates were not applied during conversion weekend, but caught early the next week as the checklist was reviewed. It is important to consider things that happened before and after in the old environment, the predecessors and triggers, such as an extract 10 days later going to another system.

Lessons Learned, Results and the Future

Were there bumps in the road? Absolutely. Initially, the company was doing up to 100 dry runs to prove the process capable. By the end, they ran about 20.

“That no calls were dropped during conversion and migration is a credit to our reconciliation process,” says BellSouth’s Pam Parisian. “Strong revenue assurance processes are critical to migration, and to daily production.” Checking the number of messages that get off the switches, and that they all end up in a billed bucket, or an unbilled file, or rejected for some valid reason, is part of the balancing and audit procedure in place at most carriers, but not all. Early on, BellSouth found that unguided/rejected calls were higher, due to the way some data were converted in certain scenarios. “Without good controls this stuff could go unnoticed,” Parisian says.

“In the beginning, we had a hard time getting the right leadership team in place to handle the daily migration, who had the vision and endurance to see it through, recalls Parisian. “Or someone who wanted to build a space shuttle or something, too far off track.” Once the right team was pulled together, including team leader Laynne Holloway, they stuck with it for four years.

The main lesson learned, according to Parisian? “The more the market groups were involved, the more successful we were,” she says. “You just cannot have enough involvement and leadership. It can’t be IT driven, or not completely IT driven.” Parisian also cautions not to use all of the new billing functionality right away – to move to platforms that allow you to do more, but don’t roll it all out one week after conversion. The result of the infrastructure change and billing migration is that no system in the BellSouth Cellular inventory is older than three years, which makes future prospects bright. In 1996 the retail store systems had to be put up - 59 stores late in the fourth quarter alone - and there was no off-the-shelf retail system available to fit the full requirements. Although the project started with a vendor, it was brought in-house. The deployment of StoreMaster and the highly touted compensation system in 1998 never jeopardized the billing migration schedule.

The last conversion on the initial BellSouth Cellular systems was on April 18th, but BellSouth Cellular has acquired new GSM markets and their systems since then, so migration will continue.

Frank Slavick is a telecommunications consultant based in Boulder, Colorado, specializing in product development, new business development, and billing and customer care. He can be reached at 303/554-0958, or at fslavick@earthlink.net.