A Change Approval Process at DevOps Speed

NOTE: This post was originally published on djgoosen.blogspot.com Monday, December 21, 2015.

Nobody launches a startup thinking about change management.
Everybody eventually realizes they needed it yesterday.
Still, change management can be a formidable topic. If and when we figure out what we want to do (drawing from the best parts of ITIL, Agile/Lean, etc.), it can be hard to champion it, because even in the most supportive of environments, we’re asking DevOps engineers to add to their already-overloaded plates, and we’re asking business stakeholders to be patient with us while we adopt and build it into existing workflows.

Trying to Talk Ourselves Out of Change Management is Easy

We may procrastinate or downright talk ourselves out of doing it. In fact, let’s try:
·             Managing change takes time. Sometimes managing a change takes longer than the change itself. 
·             And… engineers usually make safe and successful changes. Managers usually approve their engineers’ changes somehow. The business usually knows what’s going on. 
·             And… the change management processes that some companies implement are basically, um, not great. They’re often a grab-bag of intranet docs, post-it notes, “institutional knowledge” and mental reminders. They’re basically not even processes per se, as much as they are proclamations and wishful thinking on the part of management. 

Our Tools Should Enforce the Team Processes We Want to Have

Today we’re going to take it as a given that we’ve already had our organizational change management “epiphany”, and focus on building a lightweight yet effective change approval process, which is probably the first thing we can start doing if we aren’t doing much else to formally manage change yet.  
Ground rules for having change approval process that goes at DevOps speed:

  1. Don’t reinvent wheels. If we already have a solution like JIRA in place, we’ll use that.
  2. Only managers should be able to approve or deny a request.
  3. Auto-emails should happen, but be sent to only the people who actually need to see them.
  4. Changes involving product downtime should probably be subject to multiple approvals in sequence (e.g., DevOps manager à Product Manager à CTO).
  5. If we have to document much beyond stating, “This is where you submit change requests for approval”, we probably want to iterate over how the tool enforces the process again. Because a good tool will funnel the engineer to the desired result every time.

Again, JIRA can handle most of this pretty trivially in a given project’s workflow. 
OK, so now we have the beginnings of a change approval process. But is it one that lets our DevOps go sufficiently fast? Our engineers are here to solve problems, and as managers we’re here to solve theirs. One problem we can largely solve for them, is reducing a simple change request (e.g., routine tasks or anything low-risk, zero-downtime and self-explanatory) to a one-liner Python script.
The simpler the process, the more often it will be followed consistently.
So is this bash command simple enough? For zero-downtime changes at least, we’ll hope so:
$ ./create_change_req.py -u <my username> -a <my mgr’s username> -s “Upgrading server1” -de “More details about upgrading server1” -dt 2016-01-31T21:30:00.0-0800 -du 2h
^—- This will live in our engineers’ bash histories. It’s not the prettiest string, but it’s definitely effective at requesting approval for a change with one line.
^—- BTW, the datetime format above is Atlassian’s default, given here with the Pacific Standard Time Zone offset.


How Much of the Request Process Can We Automate?

Here’s how that Python script might look (let’s treat this as the example code it is, OK?):
#!/usr/bin/env python
# create_change_req.py – create a JIRA Change Request
from jira import JIRAimport argparse, getpass
# Gather values from args
parser = argparse.ArgumentParser()parser.add_argument(‘-u’, ‘–username’, help=’Username for authentication and reporter fields’)parser.add_argument(‘-a’, ‘–assignee’, help=’Manager username for assignee field’)parser.add_argument(‘-s’, ‘–summary’, help=’Change summary’)parser.add_argument(‘-l’, ‘–link’, help=’Optional: Related JIRA/Confluence URL’)parser.add_argument(‘-de’, ‘–description’, help=’Optional: Change description’)parser.add_argument(‘-dt’, ‘–datetime’, help=’format: 2015-12-31T21:30:00.0-0800′)parser.add_argument(‘-du’, ‘–duration’, help=’format: 2h’)args = parser.parse_args()
# Authenticate to Jira
jira = JIRA(‘https://<JIRA FQDN>,basic_auth=(‘%s’ % args.username,getpass.getpass()))
# Create issue# Fields are visible by looking at https://<JIRA FQDN>/rest/api/2/issue/CM-1/editmeta
issue = jira.create_issue(    project = ‘CM’,    issuetype = {‘name’:’Change Request’},    customfield_11203 = {‘id’: ‘11739’}, # 11739 means ‘Infrastructure Change’. This could be another arg if we had several types and wanted to add it.    reporter = {‘name’:’%s’ % args.username},    assignee = {‘name’:’%s’ % args.assignee},    summary = ‘%s’ % args.summary,    description = ‘%s’ % args.description,    customfield_11201 = ‘%s’ % args.link,    customfield_11200 = ‘%s’ % args.datetime,    customfield_11204 = ‘%s’ % args.duration    )jira.transition_issue(issue, ‘151’) # change status to Approval Requested. Transition id’s are visible by editing your workflow.print(“\nChange Request: https://<JIRA FQDN>/browse/” + issue.key)


Wrapping Up

Running this Python script might not be the most wildly-popular practice our DevOps engineers will ever adopt. But after they’ve used this script for a while, they won’t be able to imagine not having it. Teams never want less automation, and we’re probably not going to be asking anyone to slow down anytime soon.