These are the news items I've curated in my monitoring of the API space that have some relevance to the API governance converstation. A converstion that is still growing, but has been getting more roots lately as mainstream companies push to adopt APIs.20 Dec 2017
I am working with a group that has begun defining their API governance strategy. We’ve discussed a full spectrum of API lifecycle capabilities that need to be integrated into their development practices, and CI/CD workflow, as well as eventually their API governance documentation. However, they are just getting going with the concept of API governance, and I want to make sure they don’t get ahead of themselves and start piling in too much into their API governance documentation, before they can get buy in, and participation from other groups.
We are approaching the first draft of an API governance document for the organization, and while it has lofty aspirations, the first draft is really nothing more than some basic API design guidelines. It is basically a two-page document that explains why REST is good, provides guidance on naming paths, using your verbs, and a handful of other API design practices. While I have a much longer list of items I want to see added to the document, I feel it is much more important to get the basic first draft up, circulated amongst groups, and establishing feedback loops, than making sure the API governance document is comprehensive. Without buy-in from all groups, any API governance strategy will be ignored, and ultimately suffocated by teams who feel like they don’t have any ownership in the process.
I am lobbying that the API governance strategy be versioned and evolved much like any other artifact, code, or documentation applied across API operations. This is v1 of the API governance, and before we can iterate towards v2, we need to get feedback, accept issues, comments, and allow for pull requests on the strategy before it moves forward. It is critical that ALL teams feel like they have been part of the conversation from day one, otherwise it can be weakened as a strategy, and any team looking to implement, coach, advise, report on, and enforce will be hobbled. API governance advocates always wish for things to move forward at a faster speed, but the reality within large organizations will require more consensus, or at least involvement, which will move forward at a variety of speeds depending on the size of the organization.
This process has been a reminder for me, and hopefully for my readers who are looking to get started on their API governance strategy. Always start small. Get your first draft up. Start with the basics of how you define and design your APIs, and then begin to flesh out the finer details of design, deployment, management, testing, and the other stops along your lifecycle. Just get your basic version documentation and guidance published. Maybe even consider calling it something other than governance from day one. Come up with a much more friendly name, that might not turn your various teams off, and then once it matures you can call it what it is, after everyone is participating, and has buy-in regarding the overall API governance strategy for your platform.
I am evaluating an existing continuous integration and deployment workflow to make recommendations regarding how they can evolve to service their growing API lifecycle. This is an area of my research that spans multiple areas of my work, but I tend to house under what I call API orchestration. I try to always step back and look at an evolving area of the tech space as part of the big picture, and attempt to look beyond any individual company, or even the wider industry hype in place that is moving something forward. I see the clear technical benefits of CI/CD, and I see the business benefits of it as well, but I haven’t always been convinced of it as a standalone thing, and have spent the last couple of years trying understand how it fits into the bigger picture.
As I’ve been consulting with several enterprise groups working to adopt a CI/CD mindset, and having similar conversations with government agencies, I’m beginning to see the bigger picture of “continuous”, and starting to decouple it from just deployment and even integration. The first thing that is assumed, not always evident for newbies, but is always a default–is testing. You alway test before you integrate or deploy, right? As I watch groups adopt I’m seeing them struggle with making sure there are other things I feel are an obvious part of the API lifecycle, but aren’t default in a CI/CD mindset, but quickly are being plugged in–things like security, licensing, documentation, discovery, support, communications, etc. In the end, I think us technologists are good at focusing on the tech innovations, but often move right past many of the other things that are essential for the business world. I see this happening with containers, microservices, Kubernetes, Kafka, and other fast moving trends.
I guess the point I want to make is that there is more to a pipeline than just deployment, integration, and testing. We need to make sure that documentation, discovery, security, and other essentials are baked in by default. Otherwise us techies might be continuously forgetting about these aspects, and the newbies might be continuously frustrated that these things aren’t present. We need to make sure we are continuously documenting, continuously securing, and continuously communicating around training, and our continuously evolving (and sharing) our road maps. I’m sure what I’m saying isn’t anything new for the CI/CD veterans, but I’m trying to onboard new folks with the concept, and as with most areas of the tech sector I find the naming and on-boarding materials fairly deficient in possessing all the concepts large organizations are needing to make the shift.
I’m thinking I’m going to be merging my API orchestration (CI/CD) research with my overall API lifecycle research, thinking deeply about how everything from definition to deprecation fits into the pipeline. I feel like CI/CD has been highly focused on the technology of evolving how we deploy and integrate (rightfully so) for some time now, and with adoption expanding we need to zoom out and think about everything else organizations will need to be successful. I see CI/CD as being essential to decoupling the monolith, and changing culture at some of the large organizations I’m working with. I want these folks to be successful, and not fall into the trappings of only thinking about the tech, but also consider the business and political implications involved with being able to move from annual or quarterly deployments and integrations, to where they can do things in weeks, or even days.
People have been asking me for more stories on API governance. Examples of how it is working, or not working at the companies, organizations, institutions, and government agencies I’m talking with. Some folks are looking for top down ways of controlling large teams of developers when it comes to delivering APIs consistently across large disparate organizations, while others are looking for bottom ways to educate and incentivize developers to operate APIs in sync, working together as a large, distributed engine.
I’m approach my research into API governance as I would any other area, not from the bottom up, or top down. I’m just assembling all the building blocks I come across, then began to assemble them into a coherent picture of what is working, and what is not. One example I’ve found of an approach to helping API providers across the federal government better implement consistent API patterns is out of the General Services Administration (GSA), with the Prototype City Pairs API. The Github repository is a working API prototype, documentation and developer portal that is in alignment with the GSA API design guidelines, providing a working example that other API developers can reverse engineer.
The Prototype City Pairs API is a forkable example of what you want developers to emulate in their work. It is a tool in the GSA’s API governance toolbox. It demonstrates what developers should be working towards in not just their API design, but also the supporting portal and documentation. The GSA leads by example. Providing a pretty compelling approach to model, and a building block any API provider could add to their toolbox. I would consider a working prototype to be both a bottom up approach because it is forkable, and usable, but also top down because it can reflect wider organizational API governance objectives.
I could see mature API governance operations having multiple API design and deployment templates like the GSA has done, providing a suite of forkable, reusable API templates that developers can put to use. While not all developers would use, in my experience many teams are actually made up of reverse engineers, who tend to emulate what they know. If they are exposed to bad API design, they tend to just emulate that, but if they are given robust, well-defined examples, they will just emulate healthy patterns. I’m adding API deployment templates to my API governance research, and will keep rounding off strategies for successful API governance, that can work at a wide variety of organizations, and platforms. As it stands, there are not very many examples out there, and I’m hoping to pull together any of the pieces I can find into a coherent set of approaches folks can choose from when crafting their own approach.
I’m still kicking around my API Transit strategy in my head, trying to find a path forward with applying to API governance. I started moving it forward a couple years ago as a way to map out the API lifecycle, but in my experience, managing APIs are rarely a linear lifecycle. I have been captivated by the potential of the subway map to help us map out, understand, and navigate complex infrastructure since I learned about Harry Beck’s approach to the London Tube map which has become the standard for quantifying transit around the globe.
I am borrowing from Beck’s work, but augmenting for a digital world to try and map out the API practices I study in my research of the space in a way that allow them to be explored, but also implemented, measured, and reported upon by all stakeholders involved with API operations. While I’m still pushing forward this concept in the safe space of my own API projects, I’m beginning to dabble with applying at the industry level, by applying to PSD2 banking, and seeing if I can’t provide an interactive map that helps folks see, understand, and navigate what is going on when it comes to banking APIs.
An API Transit map for PSD2 would build upon the framework I have derived from my API research, applied specifically for quantifying the PSD2 world. Each of the areas of my research broken down into separate subway lines, that can be plotted along the map with relative stops along they way:
- Definition - Which definitions are used? Where are the OpenAPI, schema, and other relevant patterns.
- Design - What design patterns are in play across the API definitions, and what is the meaning behind the design of all APIs.
- Deployment - What does deployment look like on-premise, in the cloud, and from region to region.
- Portals - What is the minimum viable standard for an API portal presence with any building blocks.
- Management - Quantify the standard approaches to managing APIs from on-boarding to analysis and reporting.
- Plans - How are access tiers and plans defined, providing 3rd party access to APIs, including that of aggregators and application developers.
- Monitoring - What does monitoring of web APIs look like, and how is data aggregated and shared.
- Testing - What does testing of web APIs look like, and how is data aggregated and shared.
- Performance - What does performance evaluation of web APIs look like, and how is data aggregated and shared.
- Security - What are the security practices in place for the entire API stack?
- Breaches - When there is a breach, what is the protocol, and practices surrounding what should happen–where is the historical data as well.
- Terms of Service - What does terms of service across many APIs look like?
- Support - What are all the expected support channels, and where are they located?
- Road Map - What is expected, and where do we find the road map and change log for the platform?
These are just a handful of the lines I will be laying out as part of my subway map. I have others I want to add, but this provides a nice version of what I”d like to see as an API Transit map of the PSD2 universe. Each line would have numerous stops that would provide resources and potentially tooling to help educate, quantify, and walk people through each of these areas in detail, but in the context of PSD2, and the banking industry. This where I’m beginning to push the subway map context further to help make work in a virtualized world, and augmenting with some concepts I hope will add new dimensions to how we understand, and navigate our digital worlds, but using the subway map as a skeuomorph.
To help make the PSD2 landscape I’m mapping out more valuable I am playing with adding a “tour” layer, which allows me to craft tours that cover specific lines, hitting only the stops that matter, bridges multiple lines, and creates a meaningful tour for a specific audience. Here are a handful of the tours I’m planning for PSD2:
- Introduction - A simple introduction to the concepts at play when it comes to the PSD2 landscape.
- Provider Training - A detailed training walk-through for anyone looking to provide a PSD2 compliant platform.
- Provider Certification - A detailed walkthrough that gathers information and detail to map out, quantity, and assess a specific PSD2 API / platform.
- Executive - A robust walk-through of the concepts at play for an executive from the 100K view, as well as those of their own companies PSD2 certified API, and possibly those of competitors.
- Regulator - A comprehensive walk through the entire landscape, including what is required, as well as the certification of individual PSD2 API platforms, with real-time control dashboard.
These are just a few of the areas I’m looking to provide tours through this quantified PSD2 API Transit landscape. I am using Github to deploy, and evolve my maps, which leverages Jekyll as a Hypermedia client to deliver the API Transit experience. While each line of the API Transit map has it’s own hypermedia flow for storing and experiencing each stop along the line, the tours also have its own hypermedia flows which can augment existing lines and stops, as well as inject their own text, images, audio, video, links and tooling along the way.
The result will be a single URL which anyone can land on for the PSD2 API Transit platform. You can choose from any of the pre-crafted tours, or just begin exploring each line, getting off at only the stops that interest you. Some stops will be destinations, while others will provide transfers to other lines. I’m going to be investing some cycles into my PSD2 API Transit platform over the holidays. If you have any questions, comments, input, or would like to invest in my work, please let me know. I’m always looking for feedback, as well as interested parties to help fund my work and ensure I can carve out the time to make them happen.
API evangelism and even advocacy at many organizations has always been a challenge to introduce, because many groups aren’t really well versed in the discipline, and often times it tends to take on a more marketing or even sales like approach, which can hurt its impact. I’ve worked with groups to rebrand, and change how they evangelize APIs internally, with partners, and the public, trying to ensure the efforts are more effective. While I still bundle all of this under my API evangelism research, I am always looking for new approaches that push the boundaries, and evolve what we know as API evangelism, advocacy, outreach, and other variations.
I was introduced to a new variation of the internal API evangelism concept a few weeks back while at Capital One talking with my friend Matthew Reinbold(@libel_vox) about their approach to API governance. His team at the Capital One API Center of Excellence has the concept of the API coach, and I think Matt’s description from his recent API governance blueprint story sums it up well:
_At minimum, the standards must be a journey, not a destination. A key component to “selective standardization” is knowing what to select. It is one thing for us in our ivory tower to throw darts at market forces and team needs. It is entirely another to repeatedly engage with those doing the work.
Our coaching effort identifies those passionate practitioners throughout our lines of business who have raised their hands and said, “getting this right is important to my teams and me”. Coaches not only receive additional training that they then apply to their teams. They also earn access to evolving our standards.
In this way, standards aren’t something that are dictated to teams. Teams drive the standards. These aren’t alien requirements from another planet. They see their own needs and concerns reflected back at them. That is an incredibly powerful motivator toward acceptance and buy-in._
A significant difference here between internal API evangelism and API coaching is you aren’t just pushing the concept of APIs (evangelizing), you are going the extra mile to focus on healthy practices, standards, and API governance. Evangelism is often seen as an API provider to API consumer effort, which doesn’t always translate to API governance internally across organizations who are developing, deploying, and managing APIs. API coaches aren’t just developing API awareness across organizations, they are cultivating a standardized, bottom up, as well as top down awareness around providing and consuming APIs. Providing a much more advanced look at what is needed across larger organizations, when it comes to outreach and communication.
Another interesting aspect of Capital One’s approach to API coaching, is that this isn’t just about top down governance, it has a bottom up, team-centered, and very organic approach to API governance. It is about standardizing, and evolving culture across many organizations, but in a way that allows team to have a voice, and not just be mandated what the rules are, and required to comply. The absence of this type of mindset is the biggest contributor to a lack of API governance we see across the API community today. The is what I consider the politics of APIs, something that often trumps the technology of all of this.
API coaching augments my API evangelism research in a new and interesting way. It also dovetails with my API design research, as well as begins rounding off a new area I’ve wanted to add for some time, but just have not see enough activity in to warrant doing so–API governance. I’m not a big fan of the top down governance that was given to us by our SOA grandfathers, and the API space has largely been doing alright without the presence of API governance, but I feel like it is approaching the phase where a lack of governance will begin to do more harm than good. It’s a drum I will start beating, with the help of Matt and his teams work at Capital One. I’m going to reach out to some of the other folks I’ve talked with about API governance in the past, and see if I can produce enough research to get the ball rolling.
I am still working through my notes from a recent visit to Capital One, where I spent time talking with Matthew Reinbold (@libel_vox) about their API governance strategy. I was given a walk through their approach to defining API standards across groups, as well as how they incentivize, encourage, and even measure what is happening. I’m still processing my notes from our talk, and waiting to see Matt publish more on his work, before I publish too many details, but I think it is worth looking at from a high level view, setting the bar for other API governance conversations I am engaging in.
First, what is API governance. I personally know that many of my readers have a lot of misconceptions about what it is, and what it isn’t. I’m not interesting in defining a single definition of API governance. I am hoping to help define it so that you can find it a version of it that you can apply across your API operations. API governance is at its simplest form, about ensuring consistency in how you do API across your development groups, and a more robust definition might be about having an individual or team dedicated to establishing organization-wide API standards, helping train, educate, enforce, and in the case of capital one, measure their success.
Before you can begin thinking about API governance, you need to start establishing what your API standards are. In my experience this usually begins with API design, but should also quickly also be about consistent, API deployment, management, monitoring, testing, SDKs, clients, and every other stop along the API lifecycle. Without well-defined, and properly socialized API standards, you won’t be able to establish any sort of API governance that has any sort of impact. I know this sounds simple, but I know more API providers who do not have any kind of API design, or other guide for their operations, than I know API providers who have consistent guides to design, and other stops along their API lifecycle.
Many API providers are still learning about what consistent API design, deployment, and management looks like. In the API industry we need to figure out how to help folks begin establishing organizational-wide API design guides, and get them on the road towards being able to establish an API governance program–it is something we suck at currently. Once API design, then deployment and management practices get defined we can begin to realize some standard approaches to monitoring, testing, and measuring how effective API operations are. This is where organizations will begin to see the benefits of doing API governance, and it not just being a pipe dream. Something you can’t ever realize if you don’t start with the basics like establishing an API design guide for your group. Do you have an API design guide for your group?
While talking with Matt about their approach at Capital One, he asked if it was comparable to what else I’ve seen out there. I had to be honest. I’ve never come across someone who had established API design, deployment, and management practices. Were actively educating and training their staff. Then actually measuring the impact and performance of APIs, and the teams behind them. I know there are companies who are doing this, but since I tend to talk to more companies who are just getting started on their API journey, I’m not seeing anyone organization who is this advanced. Most companies I know do not even have an API design guide, let alone measuring the success of their API governance program. It is something I know a handful of companies would like to strive towards, but at the moment API governance is more talk than it is ever a reality.
If you are talking API governance at your organization, I’d love to learn more about what you are up to. No matter where you are at in your journey. I’m going to be mapping out what I’ve learned from Matt, and compare with I’ve learned from other organizations. I will be publishing it all as stories here on API Evangelist, but will also look to publish a guide and white papers on the subject, as I learn more. I’ve worked with some universities, government agencies, as well as companies on their API governance strategies. API governance is something that I know many API providers are working on, but Capital One was definitely the furthest along in their journey that I have come across to date. I’m stoked that they are willing to share their story, and don’t see it as their secret sauce, as it is something that doesn’t just need sharing, it is something we need leaders to step up and show everyone else how it can be done.
I was reading a story on GraphQL this weekend which I won’t be linking to or citing because that is what they want, and they do not deserve the attention, that was just (yet) another hating on REST post. As I’ve mentioned before, the GraphQL’s primary strength seems to be they have endless waves of bros who love to write blog posts hating on REST, and web APIs. This particular post shows it’s absurdity by stating that HTTP is just a bad idea, wait…uh what? Yeah, you know that thing we use for the entire web, apparently it’s just not a good idea when it comes to exchanging data. Ok, buddy.
When it comes to GraphQL, I’m still watching, learning, and will continue evaluating it as a tool in my API toolbox, but when it comes to the argument of GraphQL vs. Web APIs I will just be waiting out the current assault as I did with all the other haters. The link data haters ran out of steam. The hypermedia haters ran out of steam. The GraphQL haters will also run out steam. All of these technologies are viable tools in our API toolbox, but NONE of them are THE solution. These assaults on “what came before” is just a very tired tactic in the toolbox of startups–you hire young men, give them some cash (which doesn’t last for long), get them all wound up, and let them loose talking trash on the space, selling your warez.
GraphQL has many uses. It is not a replacement for web APIs. It is just one tool in our toolbox. If you are following the advice of any of these web API haters you will wake up in a couple of years with a significant amount of technical debt, and probably also be very busy chasing the next wave of technology be pushed by vendors. My advice is that all API providers learn about the web, gain several years of experience developing web APIs, learn about linked data, hypermedia, GraphQL, and even gRPC if you have some high performance, high volume needs. Don’t spend much time listening to the haters, as they really don’t deserve your attention. Eventually they will go away, find another job, and technological kool-aid to drink.
In my opinion, there is (almost) always a grain of usefulness with each wave of technology that comes along. The trick is cutting through the bullshit, tuning out the haters, and understanding what is real and what is not real when it comes to the vendor noise. You should not be adopting every trend that comes along, but you should be tuning into the conversation and learning. After you do this long enough you will begin to see the patterns and tricks used by folks trying to push their warez. Hating on whatever came before is just one of these tricks. This is why startups hire young, energetic, an usually male voices to lead this charge, as they have no sense of history, and truly believe what they are pushing. Your job as a technologist is to develop the experience necessary to know what is real, and what is not, and keep a cool head as the volume gets turned up on each technological assault.
I’ve been reading and curating information on GraphQL as part of my regular research and monitoring of the API space for some time now. As part of this work, I wanted to take a moment and revisit my earlier thoughts about GraphQL, and see where I currently stand. Honestly, not much has changed for me, to move me in one direction or another regarding the popular approach to providing API access to data and content resources.
I still stand by my cautionary advice for GraphQL evangelist regarding not taking such an adversarial stance when it comes to the API approach, and I feel that GraphQL is a good addition to any API architect looking to have a robust and diverse API toolbox. Even with the regular drumbeat from GraphQL evangelists, and significant adoption like the Github GraphQL API I am not convinced it is the solution for all APIs and is a replacement for simple RESTful web API design.
My current position is that the loudest advocates for GraphQL aren’t looking at the holistic benefits of REST, and too wrapped in ideology, which is setting them up for similar challenges that linked data, hypermedia, and even early RESTafarian practitioners have faced. I think GraphQL excels when you have a well educated, known and savvy audience, who are focused on developed web and mobile applications–especially the latest breed of single page applications (SPA). I feel like in this environment GraphQL is going to rock it, and help API providers reduce friction for their consumers.
This is why I’m offering advice to GraphQL evangelists to turn down the anti-REST, and complete replacement/alternative for REST–it ain’t helping your cause and will backfire for you. You are better to off educating folks about the positive, and being honest about the negatives. I will keep studying GraphQL, understanding the impact it is making, and keeping an eye on important implementations. However, when it comes to writing about GraphQL you are going to see me continuing to hold back, just like I did when it came to hypermedia and linked data because I prefer not to be in the middle of ideological battles in the API space. I prefer showcasing the useful tools and approaches that are making a significant impact across a diverse range of API providers–not just echoing what is coming out of a handful of big providers, amplified by vendors and growth hackers looking for conversions.
If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.