Merative Ideas Portal

Shape the future of Merative!

We invite you to shape the future of Merative, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:

Post your ideas

Start by posting ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,

  1. Post an idea

  2. Upvote ideas that matter most to you

  3. Get feedback from the Merative team to refine your idea

Help Merative prioritize your ideas and requests

The Merative team may need your help to refine the ideas so they may ask for more information or feedback. The offering manager team will then decide if they can begin working on your idea. If they can start during the next development cycle, they will put the idea on the priority list. Each team at Merative works on a different schedule, where some ideas can be implemented right away, others may be placed on a different schedule.

Receive notification on the decision

Some ideas can be implemented at Merative, while others may not fit within the development plans for the product. In either case, the team will let you know as soon as possible. In some cases, we may be able to find alternatives for ideas which cannot be implemented in a reasonable time.


Merative External Privacy Statement: https://www.merative.com/privacy

Add a new idea Subscribe
Status Future consideration
Categories Social Program Management Platform
Created by Guest
Created on Aug 17, 2020

RELATED IDEAS

Lack of foreign key can allow incremental workflow deployments to silently fail

The ProcessDefinition and ProcessInstance database tables are intimately linked in which the former contains definitions of all the workflows in the system while the latter contains data for specific executions of the workflows. Curam OOTB expects all entries in ProcessInstance to be linked to entries in ProcessDefinition (see DeadlineScanner.deliverReminders()). However, no foreign key exists to enforce the relationship between ProcessDefinition and ProcessInstance. This becomes an issue if an incremental deployment of workflows fails such that the keys in ProcessDefinition change. The lack of a foreign key to ensure each ProcessInstance record is associated with the ProcessDefinition record which defines its workflow can allow ProcessInstance records to become disassociated with any ProcessDefinition records or associated with wrong ProcessDefinition records in such a situation. A foreign key like the following is needed between these two tables:

ALTER TABLE ProcessInstance ADD CONSTRAINT PROCESS_ID_VERSION_FK FOREIGN KEY (processID, processVersion) REFERENCES ProcessDefinition (processID, processVersion);

Support Case ID is WH00010739; field above did not properly accept the number

Customer Name Missouri
  • Attach files
  • Guest
    Reply
    |     Oct 16, 2020

    Hi John/Jason,

    We have reviewed your enhancement suggestion. Based on the information provided, our understanding of your request is as follows:
    Address the general problem with Curam's lack of an incremental deployment capability for workflows by removing the requirement to back up and re-apply workflow process definition identifiers during a product upgrade.
    If we solved this, workflows would follow the same process as other metadata records during the upgrade process making this whole process much easier. Once we implemented these changes, the addition of the foreign key could also be implemented then to close this whole issue out.
    The workflow process identifiers in both the process definition metadata table and in the runtime workflow instance data tables would be updated to a set specific unique value and would never change again for the lifetime of that workflow (and SPM versions). Therefore, regardless of the type of deployments (upgrade, continuous deployment etc.), the identifier would remain the same and any current unique processing related to the workflow process definition table and related runtime tables would not be required anymore.

    The theme is aligned with our current multi-year strategy for our product and we have accepted your suggestion as a consideration for a future release. Our roadmap focus on Caseworker Experience will allow us to explore topics such as this one, with sponsor user input. Not all items under consideration will make it into a release. As plans are confirmed, you will be notified when a specific release includes this enhancement.

    Thank you for taking the time to share your ideas with us. We are committed to involving our users in building our product roadmap and appreciate your suggestions.

    Regards,
    Shane McFadden, SPM Offering Management team
    You can find more information on the request process here.

  • Jason Blackerby
    Reply
    |     Oct 9, 2020

    Hi Shane,

    This other item you wrote about does seem to address part of our concerns, and if the foreign key could be added at that time then we could accept this issue as addressed. However, please note that the issue is not limited to product upgrades but rather any time Curam is deployed. If the solution to the existing backlog item could address this more general use case rather than the much more narrowly scoped issue around product upgrades then the result would be far more useful. If not, so long as the foreign key is added, then we can continue using our custom deployment scripts with the knowledge that the foreign key is in place to keep us from encountering another ID misalignment issue as we have before.

    Best regards,
    Jason

  • Guest
    Reply
    |     Oct 8, 2020

    Hi Jason,

    With the extra information you provided we understand that the main issue you are having is "a general problem with Curam's lack of an incremental deployment capability for workflows."

    We have another item in our backlog which we believe would address your requirement: "Workflow Upgrades - remove the requirement to back up and re-apply workflow process definition identifiers during a product upgrade".
    If we solved this, workflows would follow the same process as other metadata records during the upgrade process making this whole process much easier.
    Once we implemented these changes, the addition of the foreign key could also be implemented then to close this whole issue out.

    Let me know if our understanding is correct and you agree with this approach.

    Thank you,
    Shane McFadden, SPM Offering Management team
    You can find more information on the request process here.

  • Guest
    Reply
    |     Sep 30, 2020

    Hi Jason,

    Thank you for providing the additional information requested. Within the next 30 days we will review the details you have provided and inform you of our analysis and decision.

    Regards,
    Shane McFadden, SPM Offering Management team
    You can find more information on the request process here.

  • Jason Blackerby
    Reply
    |     Sep 29, 2020

    Hi Shane,

    As for an example of an instance when failure actually arose, I have already provided all the details I have on when the previous failure occurred and orphaned ProcessInstance records resulted. I am not sure what more you are asking for here.

    Yes, we could implement this foreign key ourselves. However, we raised this issue because it is not a problem specific to Missouri but rather a general problem with Curam's lack of an incremental deployment capability for workflows. This foreign key does not fix the underlying issue of a lack of incremental deployment capability OOTB. It only puts up guardrails to prevent clients from careening off the cliff when they are forced to develop their own incremental deployment capability due to the lack of support OOTB. If this modest support for addressing a fundamental gap in the deployment of the Curam product can not be accommodated please let us know so we can take corrective actions ourselves.

    Best regards,
    Jason

  • Guest
    Reply
    |     Sep 28, 2020

    Hi John, Jason,

    We would like to get further information from you explaining the issue you have/are experiencing to allow us to determine the seriousness of the situation for you due to the lack of this FK OOTB.

    Our position on FKs has always been that we don't implement all FKs - customers can add to what we do. Have you considered adding this FK yourselves?
    The statement in the request indicates that it could result in a failure because the FK isn't enforced, could you provide an example of an instance when a failure actually arose?

    Please provide the requested information within 30 days so we may proceed with our evaluation. If we do not hear from you within that timeframe we will have to close the request due to insufficient information.

    Thank you,
    Shane McFadden, SPM Offering Management team
    You can find more information on the request process here.

  • Jason Blackerby
    Reply
    |     Sep 8, 2020

    Hi Shane,

    Please find the requested information below.

    Best regards,
    Jason


    1. Deployment of workflows - does this mean the insertion of the workflow process definitions onto the ProcessDefinition table?

    Yes, we mean the insertion of the work flow process definitions into the ProcessDefinition table.

    2. If so, what mechanism is being used to perform this action?

    I cannot speak to the process that was used previously that failed resulting in the issue reported here, but the process that has been developed since and that we are still using today is to first make a copy of the ProcessDefinition table, allow the OOTB deployment scripts to insert records into ProcessDefinition, then update the ProcessID values in ProcessDefinition using the copy of ProcessDefinition. The current process also handles updating the definition field to use the correct ProcessID value in the ID attribute of workflow-process element.

    3. What errors are you seeing and what are the failures? Are the process identifiers changing and hence becoming out of sync with the runtime instance data?

    Yes, exactly, the ProcessIDs changed at some point due to an error in the old deployment process leaving ProcessInstance records orphaned from their parent ProcessDefinition records. This was allowed to occur due to the lack of a foreign key constraint to enforce the relationship. While the issue was eventually discovered and the deployment process refined, some orphaned records were never re-associated with their parents. We can work to relink the orphaned records. What we would like from IBM is to put in place a foreign key constraint to make this sort of orphaning an impossibility in future.

  • Guest
    Reply
    |     Sep 3, 2020

    Hi John,

    We have reviewed your enhancement suggestion and require more information to properly understand the issue and the business scenario you are trying to support.

    Based on your following explanation of the issue "This becomes an issue if an incremental deployment of workflows fails such that the keys in ProcessDefinition change."

    We would like to get further information from you on what this incremental deployment of workflows involves and the failures you are seeing.

    1. Deployment of workflows - does this mean the insertion of the workflow process definitions onto the ProcessDefinition table?
    2. If so, what mechanism is being used to perform this action?
    3. What errors are you seeing and what are the failures? Are the process identifiers changing and hence becoming out of sync with the runtime instance data?

    Please provide the requested information within 30 days so we may proceed with our evaluation. If we do not hear from you within that timeframe we will have to close the request due to insufficient information.

    Thank you,
    Shane McFadden, SPM Offering Management team
    You can find more information on the request process here.

  • Guest
    Reply
    |     Aug 18, 2020

    Hi John,

    Thank you for taking the time to share your ideas with us. We are committed to involving our users in building our product roadmap and appreciate your suggestions.

    We will review the information you have provided and get back to you within 30 days. If additional details are required in order to complete our evaluation, we will send you a request for more information.

    Thank you,
    Shane McFadden, SPM Offering Management team
    You can find more information on the request process here.