Support Configurable Exit Codes for chunking & streaming batches to communicate higher number of skips with a configurable exit code

Our application has several hundreds Curam batches and most of the batches use Chunking & Streaming framework. The current Curam batch framework provides two exit codes the batches 1) successful 2) failed. The exit codes do not provide a way for the enterprise scheduler/operations to be notified in case of a large number of skipped records.

We have a variety of batches. Some batches are very critical in nature where even a single record being skipped is an operational support activity (not blocking for our nightly batch), some other where beyond a certain number of records skipped should result in batch being deemed as "operational failure" and requiring immediate intervention. Some batches process 100s of records a night whereas others process 100's of thousands of records.

Keeping above in mind, we propose that Curam extends its chunking and streaming framework to:

- support different exit code concept. It can be done in a completely flexible way (different exit code definitions for x skips or y% skips) or in a prescriptive way like below:

- Define "Successful run" - e.g. <0 skipped records or <1% skipped records) and actual skipped records meeting the definition. Exit code is 0.

- Define "Successful with skips" e.g. <10 skipped records or <5% skipped records and actual skipped records meeting the definition. Exit code is 200.

- Define "Failed due to high skips" e.g. >10 skipped records or >5% skipped records and actual skipped records meeting the definition. Exit code is 201.

- Current defintion of catastrophic batch failures (should not require newcode) in case of catastrophic error in launcher, exit code 1 is automatically returned.

- use a batch specific exit code configuration with

- % based data or hard number based configuration

- support definition of "Successful" execution

- support definition of "Successful with skips" execution

- support definition of "Failed due to skips" execution

- status - active or disabled indicating whether the configuration is to be used

- record the batch outcome in a database table (in addition to using exit codes). This would help in auditing batch executions (severly lacking in current Curam infrastructure irrespective of this enhancement request)

- executionStartDateTime

- executionFinishDateTime,

- totalRecords,

- processedRecords,

- skippedRecords,

- exitCode