Oswald Regular
OpenSans Regular
Business Rules Environment and the Express>It® Web Application

Ab Initio serves a wide range of users. At one end of the spectrum are professional application developers. These folks understand how to design and build sophisticated systems that process large amounts of data in complex environments. For these users, Ab Initio provides the Graphical Development Environment (GDE®) for graphically building complex processes and business logic. The resulting applications run on the extremely robust and scalable Co>Operating System®.

At the other end of the Ab Initio spectrum are business users. These people are not skilled at or even interested in the mechanics of processing terabytes of data or thousands of messages per second. However, they do know what their systems are supposed to accomplish and often best know the details of the data and the rules that drive the business.

Over the years, a division – sometimes an adversarial one – has evolved between IT teams and the business users they serve. The business users know what they want done and what it means to the business, and the IT professionals know how to do it. The IT shop asks for specs and the business users do their best to provide them. But there are inevitable ambiguities and mistakes – mistakes in the specs and mistakes in the translation of the specs into code. The process – from spec to code to test to release – never works as cleanly or as quickly as everyone hopes. At each stage, one human has to figure out what another human meant. That takes time and, worse, introduces big opportunities for errors. And each error (of course) takes more time to find, revise in the spec, and change in the code. Productivity suffers. Correctness is never completely assured. Sometimes the cycle never converges – and projects simply fail.

What if business users had a way to control the specific parts of the systems that implemented their business logic? What if their business rules could be expressed in a way that they understood, and that they could even write, and that could be automatically converted to code that ran inside a larger system? Sounds pretty good? That’s the whole idea behind the Business Rules Environment.

The Ab Initio® Business Rules Environment (BRE®) allows business analysts to specify business rules in a form that is very familiar and comfortable to them: grid-like spreadsheets. In the BRE, the rules are specified in business terms, not technical terms, and with expressions that are clear to anyone who has worked with Microsoft Excel. As a consequence, not only can rules be specified quickly and accurately, they are also easily understood by other business people.

Furthermore, the BRE puts the business users in the driver’s seat when it comes to verifying the business rules. The business users are not only able to put the rules directly into the system, they are also able to immediately see the results of applying those rules to test data. If they don’t like what they see, they can change the rules on the spot. The savings in time are enormous.

The Ab Initio® Business Rules Environment is not a standalone tool

Traditional “rules engine” products are standalone tools. They require work to interface to other products and the rest of the computing infrastructure. They have performance and scalability limitations. They can process only simple data structures. Their perspective is limited to the rules they process, so they cannot trace lineage across entire systems.

The Ab Initio BRE, in the Express>It® web application, solves all these problems and limitations, and many more, because it is tightly coupled with Ab Initio’s Co>Operating System and metadata technologies. It was architected and implemented that way – this is not an afterthought or the result of a typical software company’s marketing strategy.

  • The same BRE rules can be run without reimplementation in batch, continuous, web services, and real-time applications.
  • BRE rules run extremely efficiently and can be scaled to run across many CPUs and many servers. There is no limit to the scalability of systems built with Ab Initio.
  • BRE rules can process complex legacy hierarchical data. Whatever may exist in a legacy environment (EBCDIC, packed decimal, international formats, XML, Cobol Copybooks, …) can be handled natively by the BRE in conjunction with the Co>Operating System.
  • The same BRE rules run identically on all platforms supported by the Co>Operating System (Unix, Linux, Windows, z/OS, and z/Linux).
  • The BRE benefits from all of the Co>Operating System’s robustness features, such as error handling, checkpointing, and so on.

What are “business rules”?

The BRE supports three different styles of business rules: decision rules, validation rules, and mapping rules. While they are fundamentally similar, business users are comfortable thinking of rules as belonging in one of these categories.

As required by the nature of the business, rules can be simple and short or extremely long and complex. The BRE handles both extremes. Currently, the largest BRE rulesets in production have more than 50,000 rules.

Here is an example of a very simple decision-style business rule specified in the BRE. This is the tax computation for US income taxes (form 1040):

This rule has two inputs, Filing status and Taxable income line 43, and computes a single output named Tax (line 44). In the BRE, there is an implicit AND operator between columns and an implicit ELSE operator between rows. So the first few rows of this rule read as follows:

IF the Filing status is Single AND the Taxable income line 43 is less than or equal to 100000, THEN the Tax (line 44) is equal to looking up the Taxable income line 43 in the Tax from Table and returning the amount for Single filers.

ELSE IF the Filing status is Single AND the Taxable income line 43 is greater than 100000 and less than or equal to 171550, THEN the Tax (line 44) is equal to the Taxable income line 43 times 0.28 minus 6280.00.

ELSE IF the Filing status is Single AND the Taxable income line 43 is greater than 171550 …

… and so on …

Next is an example of a validation rule. It validates multiple inputs in the same rule, and sets multiple outputs in each case:

And here is an example of a source-to-target mapping rule. For this style of rule, the BRE displays on the left a list of the potential input values (the “Inputs” column) such as fields, variables, constants and calculated values. In the middle is a list of the target fields (“Output Name” column). To the right of each target field is a column (“Expression/Rule”) where the user can drag in values from the “Inputs” column or create an expression for calculating that output. (We’ll see what the “Computed Value” column is all about later.)

BRE® rules can include arbitrarily complex logic

The logic inside BRE rules can be simple, as in the examples above. However, in real life business users often have very complex rules. With non-Ab Initio technologies, those rules cannot be implemented in the business rules product. Instead, they require hand-coding in programming languages such as C++ or Java. This can have a huge negative impact on usability and productivity. It reintroduces the whole spec-interpret-code-test-fix cycle that dooms projects to failure.

Not so with the Ab Initio BRE. The BRE inherits the complete data processing capability of the Co>Operating System. This means that all Co>Operating System functions – and there are hundreds – are available to the BRE, as well as complex logic structures.

As a result, there is no limit to the size or complexity of the expressions within rules, or the size or complexity of the rules, or the number of rules within a ruleset.

Built-in testing allows business users to verify rules – by themselves

A key advantage to using the Ab Initio Business Rules Environment to develop business rules is the BRE’s built-in testing capability: rules developed within the BRE can be executed against sample data without ever leaving the editing environment. During testing, the BRE reports the outputs computed for each test record, as well as the states of every intermediate variable and calculation. A BRE user can inspect output records, and then click on any computed value to see exactly how and why that value was computed:

For each test record, the BRE shows the case(s) that triggered (in yellow), and also darkens any cells that resulted in a failed comparison. The BRE also calculates counts of the number of times each particular rule is triggered (“Times Fired” in screen shot). This information can be used to find missing test cases or invalidly constructed rules (if a case never triggers).

Testing can be run at any time during rule development, allowing for an incremental approach to the creation of a large set of rules. There is no need to wait until the rules are all entered before starting to evaluate behavior.

The output of a test run can be saved as a benchmark against which later test runs are automatically compared. When rules are changed, either during development or when modifications are being designed, it is possible to see all the differences between the current behavior of the modified rules and the saved benchmarks results. This feature is invaluable when evaluating the impact of a proposed change. Without the Ab Initio BRE, business users are often “flying blind” when changing their rules.

Furthermore, rule execution details are also available from rulesets running in production. Users can configure the amount of tracking information that is saved, ranging from just information about which rule cases (rows) are triggered for every input record to all input, lookup, intermediate, and final values. This allows analysts to study the behavior of the rules over time, often leading to improvements in results by adjusting the business logic. These details can also be critical to answering questions about how specific decisions were made in the past.

Transparency – Business logic exposed, at last

Large, complex business rules systems can consist of many rules that have been implemented on top of other rules. Understanding how these rules interrelate and how data flows through them is critical for business users. Unfortunately, typical business users can rarely see how their systems work.

With the BRE, business users have complete visibility into their systems, because the BRE displays diagrams of how data flows through interconnected rules, irrespective of how large and complex those rules are. Furthermore, graphical lineage can be displayed for applications that consist of many rulesets that may be distributed across an application, or even across multiple applications.

The following figure shows a simple lineage example for the ruleset shown above that calculates income taxes based on the US 1040 EZ form. Green ovals represent rules (calculations) and rounded rectangles represent variables. In testing mode, sample test values are shown below each variable name, along with all intermediate calculations. Below the full lineage diagram is a zoomed section showing how the calculation of the deductible (line 5) impacts the final refund or amount owed.

The lineage ability of the BRE allows business users to truly understand how large sets of complex rules work.

No “RETE algorithm”

What’s RETE? Don’t worry – with the BRE, you don’t have to know. With traditional rules products, however, rules are executed in an order determined by something called the “RETE algorithm”. This means that a “business” person has to understand a fairly complex computer-science / artificial intelligence concept (see http://en.wikipedia.org/wiki/Rete_algorithm if you really want to know).

In practice, the RETE approach makes it very difficult to understand the consequences of a change to a rule, since it may impact other rules that are very far away in the specifications. Performance is also very difficult to tune, since small changes may have big, unintended consequences.

In the BRE, rules run in the order in which you specified them. Performance is not only substantially higher, it’s predictable! You don’t have to be a computer scientist to figure out how rules work.

How does the BRE work with the Co>Operating System®?

It’s straightforward: the BRE takes the rules created by the user and puts them into a component in a graph run by the Co>Operating System. Learn more about components, graphs and the Co>Operating System.

Because business rules run in the Co>Operating System, they inherit all the advantages and strengths of the Co>Operating System. Rules can interface to any data source or target, can process any kind of data, can handle any volume of data, can run batch and/or real-time, can run distributed across multiple servers running different operating systems, all in a completely robust way. The Co>Operating System includes everything for building and running robust mission-critical applications, and the BRE inherits all of this.

Below is an example of a Co>Operating System graph (application) in which a key component contains rules specified with the BRE:

Complete rules management and governance

All rules are stored in the Ab Initio Enterprise Meta>Environment® (EME®), which has complete source-code management capabilities, including versioning. As a consequence, BRE users have the choice of two deployment methodologies:

  • Treat rules like other application elements. This means they must go through all the standard code promotion stages: development, testing, and finally promotion to the production environment. While this robust methodology is very familiar to IT development staff, the number of steps can take a lot of time. Nevertheless, the BRE’s integrated testing streamlines and shortens the overall process.
  • Treat rules more like reference-code data in an operational database. Business users are often able to make changes (add, updates, and deletes) to reference codes without running those changes through the complete application-code promotion process, thereby allowing them to respond quickly to business needs.

Ab Initio does not promote one approach over the other; we just make it possible for our users to choose the approach that best fits their needs.

Finally, because of the EME’s strong support for versioning, users can easily configure their environment to enable “champion/challenger” deployment of rules. The idea is to run two versions of the same rules side by side to compare and contrast their outputs. Users then establish processes to review these differences and improve their rules until they get the desired results.

Uniting the enterprise with unified technology

With integrated, interactive testing and automatic generation of production-ready, executable modules, the BRE allows business users to participate directly in the development of applications in a way that complements the expertise of IT staff. By presenting business logic in a way that business users can express and understand, the BRE removes huge amounts of uncertainty and risk surrounding the development and deployment of new applications.

BRE rules and the applications driven by them are executed by the Co>Operating System, which means they can be arbitrarily complex, yet run extremely efficiently, and with complete scalability. And the same rules can be run in batch applications and web service applications without recoding. Every benefit of the Co>Operating System, in fact, is realized through the BRE. Why? Because the BRE was built with the same, unified technology as the Co>Operating System. The BRE and Co>Operating system work together, seamlessly, to unite the enterprise.