Ab Initio makes extensive use of algorithms and machine learning techniques to simplify data governance and data quality.
Ab Initio technology can discover datasets and generate data lineage views across cloud, on-premises, and legacy sources, enabling compliance and impact analysis on source data assets. Datasets are integrated into the larger metadata management and governance system, simplifying governance tasks such as assigning data stewards and identifying PII. Data quality rules can be automatically generated from the enriched metadata.
Users with appropriate permissions can view data quality results, profiles, and virtualized data from across the enterprise. Data specialists can find, vet, and curate data to be used in specific projects, campaigns, and applications. Curated data is tagged, shared, and made available for use in applications.
Ab Initio uses a combination of algorithms and machine learning to detect data anomalies and provides extensive automation capabilities to generate data quality controls at appropriate control points. Our technical data lineage capabilities support business-friendly views of data lineage, giving the business greater transparency and control of data processing. Data and metadata are up to date and at your fingertips.
When the best case is that you can run your audit rules only once a year, it’s time to find a better case.
Waiting six months to determine if you have a discrepancy in your data is a problem, particularly when regulators come to visit.
For one major financial firm, a six-month delay would have been wonderful — they faced twelve-month delays for their most critical reports, and less critical reports might run only once every two years. Their systems could not support running their audit rules more frequently than once every 12 to 24 months. They didn’t have the capacity to access the data they needed in a timely manner.
Developing new audit rules and reports was a very slow, time-consuming process. To develop new rules, a business expert would specify them and then a developer would code them. If the implementation had problems — for example, if the expert didn’t clearly specify the rule or the developer misunderstood the specification — debugging was slow and torturous. By the time a business expert determined that the rule was incorrect, the developer was already assigned to another task. As a result, it typically took six months to create and debug new rules and reports.
Further complicating the process, auditors needed to be able to review the rules. Since the auditors couldn’t read the code, the financial institution had to maintain detailed documentation for each rule. Because they had thousands of rules, this task became very large and prone to error. Of course, errors in the documentation then led to more scrutiny from the auditors, creating a vicious circle. This was not a good situation.
Fortunately, they asked Ab Initio for help.
By using Ab Initio, the financial services company completely transformed rule development. With Ab Initio, they could easily define rules to extract data from virtually any type of data store, transform it into the form they needed, and load it into any destination. This simple process enabled business experts to write, test, and debug the new rules without involving developers. In the first month, a handful of non-technical experts put a dozen new rules into production, a feat that would previously have been impossible.
Ab Initio provided significant improvements in data access and performance. Using Ab Initio software, the company could run their rules daily rather than once every one to two years. Business and regulatory exposure plunged. To top it all off, Ab Initio’s intuitive interface made the rules self-documenting. Rather than trying to maintain reams of documentation about the rules, the company could just let the auditors read the rules.
In the end, the company went from running its rules yearly to daily and decreased the time required to create new rules from months to days. The savings in terms of reduced financial exposure were huge. The savings in terms of not having auditors breathing down their necks were incalculable.