Total Pageviews

Tuesday, December 25, 2012

Openstack Orchestration?

Orchestration is a critical piece of Cloud Infrastructure.   Automation and workflows across different domains (system level, enterprise level) need to be executed in a coordinated, consistent and reliable manner. They also need to be predictable and auditable to show who did exactly what, when and if possible why.  They must also show the state of each request and deliver metrics related to the progress or otherwise of the execution of those processes.    This is not just simple automation, but equally applies to higher level business processes for managing the cloud infrastructure. such as customer onboarding, billing or provisioning etc.     Openstack utilises to some extent, tools such as Puppet and Chef to perform single domain automation but is this really enough?  

This blog post is a thought experiment.   The thought experiment consists of how BPMN 2.0 could be used to improve the management and capability of openstack clouds.

BPMN for those interested in learning more is Business Process Modelling Notation and to quote   “A standard Business Process Modeling Notation (BPMN) will provide businesses with the capability of understanding their internal business procedures in a graphical notation and will give organizations the ability to communicate these procedures in a standard manner. Furthermore, the graphical notation will facilitate the understanding of the performance collaborations and business transactions between the organizations. This will ensure that businesses will understand themselves and participants in their business and will enable organizations to adjust to new internal and B2B business circumstances quickly.”

To me, this seems like a worthy addition to openstack as it allows a standardised way of providing business rigour that is relatively easy to build on.   It also allows business process specialists to design the way they want their cloud to work and communicate that effectively with engineers who need to build and deliver it.

For the practical side of this thought experiment I chose to use the JBPM5  JBOSS based software from RedHat.  It is a comprehensive and well thought out piece of software and has the necessary tooling to deliver valuable business results.   It’s also Enterprise ready, though it appears the current commercial implementation of JBPM doesn’t yet include the BPMN functionality - this is just a matter of time for this to appear as is the strategy going forward.    

So, how does this thought experiment start.   In our hypothetical private cloud we’d like to have a provisioning process where internal teams can request virtual instances,  but if they require instances with 4 vcpu’s (or more) then approval is required from the cloud administrator.   Additionally instances with >=32Gb of memory AND > 4vcpu require internal financial approval from a designated finance role.   In all cases an external CMDB must be updated and the approvals stored for audit purposes.

In this hypothetical world, the Horizon web application or other third party web applications exploiting the openstack API’s would collect the required provisioning request data and then execute the business process.

For our first hypothetical provisioning request the input parameters are 1 vCPU, 4Gb Ubuntu system.

From the diagram above the workflow executes and passes all the business controls areas due to the request being under the limits.  The ‘provision machine sub-process utilises the openstack API to perform the provisioning request.  Depending on the level of complexity required this can recover from error events or simply pass the error condition along.  

For a second hypothetical provisioning request the user requests a 4vCPU system with 4Gb memory.   In this case we’re required to execute the human task ‘Cloud Admin Approval’.   As this is a human approval task the Cloud Administrator role (could be a team) is notified that the request needs to be reviewed.  At this point the process stops and waits for that approval.  It should be noted that escalation points can also be managed.   The Cloud Administrator either approves or rejects the request and the process continues to either performs the provisioning or notify the requester of the rejection.

For a third hypothetical request we have 4 vCPU AND 32Gb of memory.   In this case and according to our hypothetical business rules, both the Finance approver AND the Cloud Administrator MUST approve the request before the provisioning commences.  Note:  The Cloud Administrator doesn’t get notified until the Finance Department has approved in this process flow.   Again, both of these approval steps are human tasks and the process is suspended while the approvals are sought.

Could you write this logic into straight python code within the Horizon web application.   Yes, of course you could, but it should be readily apparent that using BPMN provides at the simplest level wonderful documentation of the process that is followed.   In addition to that it completely allows the web presentation logic or the provisioning automation to be decoupled from the business or implementation logic.   This is a significant benefit.    Furthermore it allows for dynamic changes to the business logic being applied - with little to no coding effort required.   Not only that, but you can view any request and see where it currently resides within the process.  You may see that many requests are waiting for approvals from finance and perhaps changes to spend limits may be required.   

Now a logical extension of this thought experiment is to encapsulate the openstack API as ‘reusable sub processes’ within the BPMN framework.  This would allow BPMN designer tools to orchestrate any part of openstack as well as managing approval processes etc. This allows complex internal logic to be expressed using a consistent and standard notation which provides valuable benefits.

As you can imagine, these types of business processes are very common.  They’re currently not functions typically performed by openstack deployments out of the box.  The addition of BPMN based orchestration makes all this (and more) possible .

Next steps for me would be to put this into a blueprint on launchpad and then make some reusable processes/code available, but before I do that, does anyone think the addition of BPMN based orchestration into openstack is a good idea?   Thoughts appreciated.

1 comment:

  1. I'm thinking of this, too. Especially I'm thinking of who is calling whom...

    If you call the provisioning from your business process (which would be the straight-forward way), how will your UI look like?

    If you call the business process just before execution of the specific actions by some kind of hooks in the horizon dashboard, you could stay with horizon as UI