Take a Proactive Approach to Data Integrity

By Michael Haggerty

Data integrity has been defined many ways, but it ultimately refers to the accuracy of the data itself. The most difficult question regarding network data integrity at this point is: What is the database of record for each data element?

Even with that question answered, there still is a good chance that not even that data is 100 percent accurate. So, striving for that unreachable goal of complete data accuracy is the best anyone can do in order to minimize the downstream impact. And in the world of communications, that impact often has a trickle down effect that crosses operational, financial and customer systems.

In order to proactively deal with data integrity issues it is important to understand how they manifest themselves in the first place. The most common ways are:

• Manual interaction
• System design and changes
• Systems integration
• Business rule and code changes
• Data migrations

Manual interaction with systems is probably the most common cause of data corruption because it is not completely controllable and it is inconsistent from person to person. By maintaining consistent processes, providing ongoing reinforcement via clear communications and leveraging greater levels of system input validation, you can help minimize data corruption.

Systems and code-level elements make up the next greatest cause of data corruption issues. This usually happens less frequently but when it does happen, it can be harder to detect. This is based on the assumption that all carriers are engaged in a tight Quality Assurance (QA) process when dealing with any kind of system, code, or business rule changes. While this may be the case, it is critical to leverage some form of quality testing after any release to production of any system, code, business rule or associated changes. This will at least provide a way to quickly address any issue that may have gotten through the QA process.

Data migrations can occur for a number of reasons: System replacements, system consoldation, M&A activity, etc. Any time large amounts of data need to be transformed data corruption is likely, even when the destination and origination locations have identical structures. If the data was corrupt to start with it will still be corrupt when migrated.

But let’s assume the originating data was accurate, transformations to new schemas along with applied business rules could cause data integrity issues. So in this case, cleaning the data in its current location is the best place to start so you do not propagate data integrity issues. Combined with a rigorous QA and validation of the final transformation process, this is the best way to minimize data corruption during a migration.

When this proactive approach is taken, data integrity issues can be minmized, but it will never eliminate them all together. Ongoing testing and analysis is required to maintain the maximum level of data integrity. But more important, action must be taken when issues are identified. Often CSPs are more interested in correcting the results of data integrity issues as opposed to what caused them. For example, they may feel that it is more important to fix the billing portion ASAP so that revenue can be realized rather than fixing the issues upstream that were responsible for billing issues in the first place.

So in summary, a proactive approach to data integrity combined with a reactive discipline can not only maximize the level of data integrity, it can also maintain it.

Michael Haggerty is director of business assurance at Vertek Corp. He has over 8 years of Telecommunication industry experience, specifically in the core area of Financial and Revenue Assurance. Mike as lead over 40 complex Revenue Assurance audits for Vertek, which required an in-depth knowledge of billing systems and processes. His expertise in the Financial Assurance area includes Project Management, Process Optimization, Business Requirements and Operational Management. He can be reached at: MHaggerty@Vertek.com