We all know what happens to the plumbing in an old house: When it is first installed, it’s state of the art. But over time, as the needs of the occupants grow and the house is expanded, more plumbing is grafted onto the old. Then building codes change, so some of the pipes have to be refitted. As the house ages, some pipes begin to leak and more plumbing is replaced. Then there are the old lead pipes, which constrains the use of certain taps. The kitchen and the bathrooms are updated, so a new hookup to the water main is necessary for the increased demand. Eventually, almost all of the pipes have been impacted in one way or another.

If the data coursing through the veins of your business were like this, you would certainly say “enough!” and order a rearchitecture and reimplementation. Or would you? Sadly, if you are a network operator in the mobile telecom industry, instead you probably spent your time “bolting on” one new system after another to keep up with mergers and market forces. You would have ended up with a whole range of different and incompatible technologies: multiple network devices, billing, CRM, and reporting systems... to name just a few. And the complexity, inefficiency and cost would be eating you alive today. Just like the old house with leaky plumbing.

“But does it have to be like this?” a large mobile operator asked of Ab Initio. “Yes” if you work with standard technologies and methodologies. “No” if you work with Ab Initio. So this customer gave Ab Initio the green light to dig into its systems.

GETTING TO THE FOUNDATION

Getting started, Ab Initio uncovered the usual smorgasbord of technologies. Holding the whole thing together was a large amount of handcrafted C++ code. Ultimately, in this case, the root of the problem was the mediation architecture—its lack of flexibility meant that changing upstream and downstream systems would require even more custom C++ code, and this encouraged all kinds of work-around solutions. The mediation engine was also incapable of keeping up with the data volumes, hundreds of millions of call detail records (CDRs) per day, which meant that it took days for the data to wind its way through the environment.

So Ab Initio set the following architectural principles for the redesign:

• Decode CDRs once only, saving processing time and resources
• Write data to disk once only
• Promote reuse of application code (develop once, use many times)
• Future-proof against record format and application changes
• Provide faster and more reliable application delivery time
• Ensure metadata availability covering end-to-end processing
• Deliver much lower latency from network switch to target systems

If you are not a geek, you don’t need to read further—the project was a success! Of course, Ab Initio wouldn’t have it any other way. From inception through pre-production testing to go-live took less than 12 months for this business critical system. Today, the mediation system not only processes the full daily volume on a fraction of the previous hardware, the latency is measured in minutes rather than days—which means that the operational and reporting systems are tracking the business in near-real-time. When the management decides it needs the data in real-real-time, the system as is will support that too. Finally, due to the “metadata-driven” flexibility of the new mediation engine, the bottlenecks in the upstream and downstream systems were quickly resolved and much of the custom C++ code was thrown away.

THE NITTY GRITTY

Now, for the geeks amongst you, here are some of the technical details.

Ab Initio approached this challenge by designing a single generic CDR mediation system that is able to take the numerous different input stream formats (such as voice calls, data records, multimedia, etc.) and produce decoded, de-duplicated and enriched CDRs. These CDRs are then published in Ab Initio^® queues, ready for consumption by the various downstream systems.

By publishing decoded and de-duplicated data in Ab Initio queues, the team met the architectural principles of decoding data once and writing data once. These queues are low-latency, parallel devices that support multiple independent subscribers (readers). As a result, data is retained in the queue until all subscribers have consumed the records, thus ensuring that only one copy of the data is required to be stored in recoverable media.

The mediation system now handles changes to data formats and processing logic for each input data stream without requiring large development projects. This was achieved by designing the mediation system to be “metadata-driven.” As a result, the specific requirements for each data stream are decoupled from the generic mediation processing flow, which means all data streams are processed in a consistent manner—and specific formats and rules for a data stream are loaded dynamically when required.

NO NEED TO COMPROMISE

Traditionally, there is a compromise between flexibility and performance, and between speed of implementation and implementation longevity. The Ab Initio solution, built on top of Ab Initio’s Co>Operating System^®, defies these traditions. Volumes are on their way to doubling, and yet upstream and downstream systems have the flexibility they need to adapt. And new development efforts are being undertaken at speeds not previously seen, while simplifying and reducing maintenance.

At Ab Initio, innovation trumps tradition.