Most of a data scientist’s day is not spent training the machine learning models — it is spent finding and preparing the data.
Ab Initio makes it easy to catalog your data, cleanse it, and identify appropriate data subsets. Ab Initio software also simplifies the organization of data into a big wide record containing the necessary inputs to the machine learning models. We help you add new data, understand your data, and join datasets together into the big wide record.
When the best case is that you can run your audit rules only once a year, it’s time to find a better case.
Waiting six months to determine if you have a discrepancy in your data is a problem, particularly when regulators come to visit.
For one major financial firm, a six-month delay would have been wonderful — they faced twelve-month delays for their most critical reports, and less critical reports might run only once every two years. Their systems could not support running their audit rules more frequently than once every 12 to 24 months. They didn’t have the capacity to access the data they needed in a timely manner.
Developing new audit rules and reports was a very slow, time-consuming process. To develop new rules, a business expert would specify them and then a developer would code them. If the implementation had problems — for example, if the expert didn’t clearly specify the rule or the developer misunderstood the specification — debugging was slow and torturous. By the time a business expert determined that the rule was incorrect, the developer was already assigned to another task. As a result, it typically took six months to create and debug new rules and reports.
Further complicating the process, auditors needed to be able to review the rules. Since the auditors couldn’t read the code, the financial institution had to maintain detailed documentation for each rule. Because they had thousands of rules, this task became very large and prone to error. Of course, errors in the documentation then led to more scrutiny from the auditors, creating a vicious circle. This was not a good situation.
Fortunately, they asked Ab Initio for help.
By using Ab Initio, the financial services company completely transformed rule development. With Ab Initio, they could easily define rules to extract data from virtually any type of data store, transform it into the form they needed, and load it into any destination. This simple process enabled business experts to write, test, and debug the new rules without involving developers. In the first month, a handful of non-technical experts put a dozen new rules into production, a feat that would previously have been impossible.
Ab Initio provided significant improvements in data access and performance. Using Ab Initio software, the company could run their rules daily rather than once every one to two years. Business and regulatory exposure plunged. To top it all off, Ab Initio’s intuitive interface made the rules self-documenting. Rather than trying to maintain reams of documentation about the rules, the company could just let the auditors read the rules.
In the end, the company went from running its rules yearly to daily and decreased the time required to create new rules from months to days. The savings in terms of reduced financial exposure were huge. The savings in terms of not having auditors breathing down their necks were incalculable.