If the data coursing through the veins of your business were like this, you would certainly say “enough!” and order a rearchitecture and reimplementation. Or would you? Sadly, if you are a network operator in the mobile telecom industry, instead you probably spent your time “bolting on” one new system after another to keep up with mergers and market forces. You would have ended up with a whole range of different and incompatible technologies: multiple network devices, billing, CRM, and reporting systems... to name just a few. And the complexity, inefficiency and cost would be eating you alive today. Just like the old house with leaky plumbing.
“But does it have to be like this?” a large mobile operator asked of Ab Initio. “Yes” if you work with standard technologies and methodologies. “No” if you work with Ab Initio. So this customer gave Ab Initio the green light to dig into its systems.
Getting started, Ab Initio uncovered the usual smorgasbord of technologies. Holding the whole thing together was a large amount of handcrafted C++ code. Ultimately, in this case, the root of the problem was the mediation architecture—its lack of flexibility meant that changing upstream and downstream systems would require even more custom C++ code, and this encouraged all kinds of work-around solutions. The mediation engine was also incapable of keeping up with the data volumes, hundreds of millions of call detail records (CDRs) per day, which meant that it took days for the data to wind its way through the environment.
So Ab Initio set the following architectural principles for the redesign:
If you are not a geek, you don’t need to read further—the project was a success! Of course, Ab Initio wouldn’t have it any other way. From inception through pre-production testing to go-live took less than 12 months for this business critical system. Today, the mediation system not only processes the full daily volume on a fraction of the previous hardware, the latency is measured in minutes rather than days—which means that the operational and reporting systems are tracking the business in near-real-time. When the management decides it needs the data in real-real-time, the system as is will support that too. Finally, due to the “metadata-driven” flexibility of the new mediation engine, the bottlenecks in the upstream and downstream systems were quickly resolved and much of the custom C++ code was thrown away.
Now, for the geeks amongst you, here are some of the technical details.
Ab Initio approached this challenge by designing a single generic CDR mediation system that is able to take the numerous different input stream formats (such as voice calls, data records, multimedia, etc.) and produce decoded, de-duplicated and enriched CDRs. These CDRs are then published in Ab Initio queues, ready for consumption by the various downstream systems.
By publishing decoded and de-duplicated data in Ab Initio queues, the team met the architectural principles of decoding data once and writing data once. These queues are low-latency, parallel devices that support multiple independent subscribers (readers). As a result, data is retained in the queue until all subscribers have consumed the records, thus ensuring that only one copy of the data is required to be stored in recoverable media.
The mediation system now handles changes to data formats and processing logic for each input data stream without requiring large development projects. This was achieved by designing the mediation system to be “metadata-driven.” As a result, the specific requirements for each data stream are decoupled from the generic mediation processing flow, which means all data streams are processed in a consistent manner—and specific formats and rules for a data stream are loaded dynamically when required.
Traditionally, there is a compromise between flexibility and performance, and between speed of implementation and implementation longevity. The Ab Initio solution, built on top of Ab Initio’s Co>Operating System, defies these traditions. Volumes are on their way to doubling, and yet upstream and downstream systems have the flexibility they need to adapt. And new development efforts are being undertaken at speeds not previously seen, while simplifying and reducing maintenance.
At Ab Initio, innovation trumps tradition.