Automation enables you to stay ahead of the data deluge

Anyone reading this would probably agree that nearly all enterprises are in an endless struggle to maintain their operational data. This is not new, in fact it’s something that has been an ongoing challenge since the first companies began to operate at scale. In the very early days there were people whose actual job title was “Computer” (that is, one who computes). Even back then employees were a key element of data entry and correction and they did an acceptable job with it.

Of course that was then, and this is now. The problem is that when data volumes increased employees were no longer able to keep up with the data deluge and accuracy (that is, quality) began to suffer. This problem is growing exponentially, and there is realistically no end in sight.  In order to survive (or thrive, which would be better), organizations need to develop new methodologies that enable them to stay ahead of the effort to maintain quality data, and as we see in the press nearly every day, that isn’t happening as often as it should.

Organizations have always needed high quality, accurate data, whether it was delivered in the 1800s by a “Computer” named Sears Cook Walker (the first person who actually had that title) or whether it’s delivered now by a cloud based AWS server farm. What’s been an interesting side effect is that advances in technology have not only brought more capabilities, they’ve brought a steady increase in the ability to measure and track pretty much every detail of the operation. With this increase in measurement capability paralleled by an increase in data, this process is no longer manageable by humans (sorry Sears). Even with blindingly fast machines doing the work, there are increasing gaps in what can be validated. The pressure to process more and do it faster tends to increase the likelihood of mistakes. Mistakes that become endemic to business processes are unacceptable for an organization relying on accurate operational data to make important business decisions.

The good news is that the continuous advancement of automation offer opportunities to address the issues of volume, speed and accuracy. As with any technology, (or actually anything, really), the right tool must be selected and it must then be implemented properly to realize the benefits. Automation also offers process consistency, which under the current workload is well beyond the scope of humans to manage. As a side benefit, automation tools also address resources issues where individuals don’t want to or can’t perform mundane and repetitive data quality tasks.

All individuals want to feel that they are positively contributing to the growth and success of the organization.  Working to enter and correct data is simply a task that most employees would rather not do on a regular basis, and it’s a poor use of resource dollars. Automation technologies can help in that they can take away some of these lower end tasks and perform them better, faster and more accurately.

Automation technologies continue to mature but many offer considerable enhanced capabilities which are unrealistic to expect from individuals doing so manually. Even the most simple aggregation and normalization of data from two sources is an immense effort for an individual.  For automation technology however, this is just a starting point and better, it can do it 24/7 with consistent results every time. The benefit grows considerably when we consider that the aggregation is typically across far more than 2 data sources, something that is not realistic for an individual to perform on a regular basis and definitely not with any acceptable speed, consistency or quality.

Companies will continue to struggle with the aggregation and normalization of data which is why they need to look deeper into automation tools that deliver higher quality consistently and faster. They will also get the added benefit of relieving their employees of the mundane tasks and allow them to reallocate those resources to analyze and assess the results rather than collect and sort the raw data.

In order to thrive, enterprises need to take a step back and look at the longer term patterns that are driving their operational decisions. Anytime there is a disconnect, anytime something happens that makes you want to hit the pause button, chances are there is an underlying data issue. More specifically, there is an underlying data quality issue. The fact that we live in a fully-connected, mobile, social, cloud-base environment should be seen as the massive opportunity it is, but that opportunity comes with data issues that need to be addressed head on.

Quality data is the key ingredient to better business outcomes

Quality inputs are a core requirement for a quality output, and this is true for nearly any profession, regardless of whether you’re a chef, or an IT professional. Your service delivery organization needs high quality reliable data before it can assemble it properly for the decision makers, in order to ensure positive business outcomes. You need to not only find those reliable data sources, but also put in place the necessary quality controls and checks. Those high quality data elements are the vital ingredients that make up the products and services your customers consume.

If you ever speak to chefs as they describe the dishes they provide, you will hear them talk about the same things in nearly the same terms (obviously adjusted for domain). They believe that the only way they can deliver the best quality product for their customers is to ensure the ingredients they put into their dishes are consistently of the highest quality. It’s simply a philosophy of the output will never be better than the input, or in the common vernacular, garbage in, garbage out. Improve the input quality and the end result also is improved.

This concept isn’t exclusive to food products; it also applies to data used in any sort of business making decision. It’s especially applicable when we’re talking about aggregation of data from various sources or migrating data from one database to another. It makes no sense to migrate data to a new source when its quality level is unknown or worse, is known to be poor and unreliable. Yet, we see attempts to simply lift and move data to a new source all the time. What is shocking is that people are surprised the poor quality data wasn’t miraculously corrected in the migration. Data doesn’t magically get better on its own, and in fact a migration runs a real risk of making things worse without a directed effort to improve the data as part of the migration process.

Every employee makes decisions based on the data that is available to them, so it’s important that they have the best quality data available. This is similar to the chef who carefully picks only the best leaves from their personal herb garden, because these are the only ones they would include in their special gourmet dish. As IT professionals, we also need to carefully evaluate our data sources as high quality ingredients. We need to decide which sources are best to use for our systems and which data elements within them we can count on to help us improve business outcomes. For long term success, we also need to determine which processes will be necessary to maintain the desired quality levels. It is Important to keep in mind that not all of them have to be used, some may only need to be used in a specific context, and some may be unable to provide the data quality we seek.

Every product or service created by an organization can only be as good as those elements that go into it. IT organizations need to employ more rigorous data quality controls and audit checks to ensure that those elements are of the very best quality. When sources or element data is not of the optimum quality, an alternative has to be used or considered. Regular checks and balances must be in place between systems to ensure that they don’t begin to diverge and potentially corrupt one another.

If the effort is intended for just a one-time migration, it is even more important to have a rigorous mechanism to normalize and cross reference the data for quality and completeness. This is because there will be no second chance to identify discrepancies and the end product will be negatively impacted. Bottom line? Quality will always matter, and the more critical the end product, the more important quality becomes. Getting an ingredient in a gourmet dish wrong will have a small, localized effect, but migrating bad data in a health care application can be disastrous. Bad data and its effects are entirely manageable; the tools are available, and the best practices are well established.

Bringing on the Pain: Majority of C-Suite Not Involved with Data Quality Management Initiatives

[this is a reprint from Inside Big Data, dated July 5, 2016, link at the bottom]

It is a broadly accepted fact that the volume of data entering any enterprise has skyrocketed, and is going to continue to accelerate at a staggering rate. So while the good news is that enterprises have more fodder than ever before to make decisions, the challenge is that data fodder needs to be converted to actionable intelligences, and there is a massive disconnect between the people making the decisions (the C-Suite), and the folks who provide them with the information needed to make those decisions (the IT function).

The data quality divide

According to a new Blazent survey which polled the opinions of IT directors/VPs live on the exhibition floor at ServiceNow’s annual Knowledge16 conference (effectively, a big box full of real experts), nearly all (98 percent) indicated data quality management was either very important or critical to IT operations, yet nearly half (48 percent) would give their organization a grade of C or lower in terms of data quality driving business decisions, and over two thirds are nominally confident in the quality of their organization to drive IT decisions.

Couple that with the fact that 56 percent of execs cited that the C-suite isn’t involved in daily data quality management (DQM) conversations or the technology selection, yet still expect data to positively impact ROI. Then, putting the icing on the cake, nearly half of all execs are still relying on spreadsheets (1980s technology) or manual processes (15th century technology) to perform analytics within their organization – an inherently time consuming process, using the wrong tools, and saddled with an uncomfortably large margin for error.

So what does this mean?

  • All of IT admits that data quality is important although half of IT admits to not doing it correctly
  • A majority of IT execs don’t trust themselves to manage the data quality within their organization
  • The C-suite/senior management is not involved with data quality initiatives half the time
  • Half the time the tools being used to making data true, accurate and actionable are antiquated or even worse, non-existent

Clearly there is a significant disconnect. If data quality is that important, how do these companies rationalize the dismal implications of their other responses? The key response is the lack of participation of the C-Suite – these are the people sitting at the top of the mountain with the long-range view. Anyone lower than that is usually in fire-fighting mode, and it’s hard for that group to look down the road. Data quality is a strategic enabler and whoever has better data is in a stronger position to make better decisions than their competitors. The tools are available, the talent is in place, and the payoff is significant and quantifiable. This is a matter of focus and will; every day we see evidence of decisions based on bad data, and every day is another day where the majority of the C-suite is not doing anything about it.

What’s Next? 

It’s clear that data quality remains a huge priority for IT, yet it’s current state of implementation (or lack thereof) is genuinely alarming. In order to clear this data quality hurdle, organizations need to educate from top down on the importance of having a consolidated view of the information associated with quality data and being able to leverage the proper automated processes to minimize human error – all with the goal of enabling an enterprise-wide strategic value-add. The companies who figure this out first will have a massive competitive advantage, those who don’t will become a footnote in case studies of what to avoid.

The original article can be found here:

Bringing on the Pain: Majority of C-Suite Not Involved with Data Quality Management Initiatives



Using data quality to fight fires in IT

You would think that the unending scramble to resolve IT emergencies and put out fires would get old, but for some reason, it seems to be standard operating procedure at most organizations. What’s even worse is that you routinely hear people say that they are so busy that other aspects of their jobs are simply not getting done. In some cases, they even acknowledge that there would be significant benefits in terms of efficiencies and effectiveness if they could just go back and correct data issues, or what my kid refers to as a “do-over”. I pointed out to him that in the real world no one gets do-overs, and IT is no exception. Given that little reality check, what is the closest we can come to a legitimate do-over?

In an age where budgets and resources are constantly being stretched or cut, an immense strain is put on employees to get things done, but only the highest priority things are ever addressed (not unlike Agile software development). In the heat of fire fighting, people naturally do whatever seems to be easiest at that time to address the immediate issue. There is no time to think about fixing things long term, so they focus on the current burning issue (and let’s be fair, no one is going to think about designing a fire sprinkler system while their house is on fire). Because most IT fire drills are by nature intended to address what’s in the enterprises collective face, everything else keeps smoldering and eventually develops into another fire elsewhere in the organization. The collective waste of energy and resources to repeatedly fight data fires is staggering.

A secondary issue is that while all this time is being spent fire fighting, taking time away from performing the normal day to day activities, things naturally slip through the cracks or drop down the priority list. For example, when bad data goes uncorrected, it offers an opportunity for evil-doers who might want to do harm to the organization to mask their efforts within the bad data. They could easily modify IDs, account numbers and maybe even credentials with some confidence that the organization isn’t analyzing or correcting poor quality data unless it is causing an outage.

A firefighting mentality is not a sustainable framework for the enterprise. Unfortunately, most businesses simply can’t seem to take a step back far enough to understand the overall dynamic of the situation. Many times, it may feel like it is a small, localized data quality issue when in fact it’s a systemic issue across the organization. When we consider the efforts of individuals trying to resolve high priority outages, it is understandable that for them to look deeper into potential data quality issues is asking too much. The issues, however, are that there never seem to be any follow-up or follow-on action to investigate what happens afterwards, because it is incredible easy for IT to become interrupt-driven. It is this mentality that fuels future fires.

Eliminating or at least minimizing the need to firefight has a multitude of benefits to both the organization and the firefighters, your employees. Constantly putting out fires and running from emergency to emergency is exhausting and employees don’t want to work that way. They want to permanently fix things rather than put patches over them, because they know they’ll be impacted again by it. Organizations cannot continue on this track of inefficient and ineffective activities rooted in poor data quality. Organizations need to acknowledge this road they’re on and step up to the challenge with strong leadership, enhanced processes and reliable technologies. It’s actually not that difficult to minimize the need for firefighting when you focus and fix the bad data which is causing the fires.

So what do you do? Take a BIG step back, grab the people who understand what’s going on (and not just management, get the grunts who are on the front lines as well) and look at the longer term patterns that are driving your organization and the functions within them. What you’re seeing are the effects that define your day to day. Now you need to determine the cause, and we can guarantee you’ll find bad data is a core enabler. Bad data can trigger a nearly endless stream of problem, but data can also be cleaned up, aligned, remediated, normalized, contextualized, etc. Pushing data through a stringent Data Quality filter will make an enormous and immediate difference in your day to day, and this is not just hype.

We’ve seen multiple organizations take their game to a whole new level by addressing the root cause of data fire-fighting, and because its data-centric, it lends itself very nearly to quantifiable analysis.


The potential perils of software asset management

Consider the following case study:

A large enterprise had a long-standing relationship with a major software vendor, who provided a critical software product used widely for many purposes across the enterprise.

The price for this product was set based on the power of the computer running it. A license would cost less for computer with 4 cores and 1 gigabyte of RAM, than it would for a computer with 16 cores and 8 gigabytes of RAM. The largest computers required the most expensive licenses.

The goal of virtualization is to use one powerful physical computer to consolidate more lightly-loaded computers as “virtual machines”. This can provide significant savings, which the enterprise in question was seeking.

Over the course of 3 years, the enterprise described here virtualized about 5,000 formerly physical computers, each of which had been running the vendor’s software.

However, a deadly wrinkle emerged in the software vendor’s licensing terms. The formerly physical computers were smaller machines. The new virtual farms were clusters of 16 of the most powerful computers available on the market. The vendor held that EACH of the 5,000 instances of its software running in the virtual machines was liable for the FULL licensing fee applicable to the most powerful machine!

Even though each of the 5,000 virtual machines could not possibly have been using the full capacity of the virtual farm simultaneously, the vendor insisted (and was upheld) that the contract did not account for that, and there was no way of knowing whether any given VM had been using the full capacity of the farm at some point.

The dispute escalated to the CEOs of each company, but the vendor held firm. The enterprise was obliged to pay a “true-up” charge of over $100 million (9 figures).[1]

[1] Adapted from Agile IT Management: From Startup to Enterprise, by Charles Betz © 2016

This is not an isolated instance. Major software vendors have earned billions in such charges and continue to audit aggressively for these kinds of scenarios. Software licensing audits have become a major source of revenue for companies whose licensing sales are down. These firms are experiencing competitive pressure from Cloud alternatives, and stories such as the above are increasingly common throughout the industry.

This is why contracts and licenses should never be taken lightly. Even startups could be vulnerable, if licensed commercial software is used in un-authorized ways in a Cloud environment, for example.

How can data quality help?

Without a thorough understanding of the IT asset base (physical and virtual servers in particular) the organization risks licensing non-compliance. Software vendors expect the customer’s records to be accurate. When they see discrepancies, the vendor’s auditors become more aggressive and dig further into the usage of their software. As Stephanie Overby notes, “Anything from use of software on non-named servers to lack of centralized software asset management processes to inadvertent including of software on a base image can raise red flags.”

Inaccurate records make it even more difficult to contest a vendor’s claims that you have improperly used their software. On the other hand, if everything they see in their tests is also available in your system, that starts to build confidence that your records are sufficient. You need a clear understanding of licenses, virtual machines, physical machines and the supply chain that delivered them.

Of course, data quality all by itself is not going to solve your software licensing problem. You also need controls and processes to ensure compliance. The change management, asset management, procurement, request management and provisioning processes need to work well together. But, in the final analysis, effective process and quality data are two sides of the same coin. You need both to save the much larger quantities of coin that may otherwise be at risk with poorly managed software licensing.

Is your data perception 20/20?

The rate at which things change can sometimes be so gradual that we barely notice it. Like that one day when we casually grab for and put on someone else’s glasses, only to realize how our vision has changed and how nearsighted we’ve become. Or, you head to the closet to find last year’s golf shorts have mysteriously “shrunk” around the waistline (still can’t figure that one out). The situations are numerous where the slow progression of change goes essentially unnoticed.  It’s only when an event causes us to compare the current situation to one of a significant time before that we can see how far off path we have drifted.

Sadly, many organizations are in this situation with regard to the quality of their data. Data which is slowly becoming unusable for decision making and will eventually kill their operations. First it’s just a few minutes of hands-on ‘tweaking’ to make it look good, but over time it becomes a whole team which ‘tweaks’ the data to make it presentable and useable. Before they realize it, the team is expending the bulk of its time manually cleansing and manipulating data rather than doing their actual jobs. The result in the end is the same as our vision and waistline. Our inability to recognize the slow progression of change in our environment has caused an unsustainable situation that now requires a major correction to get back into line.

Successful organizations grow, and so to must the way they generate, aggregate and maintain their operational data. Early on, spreadsheets are perfectly acceptable and satisfy the need, but as the volume and complexity of what they‘re tracking grows, the practicality of using spreadsheets fades. The recognition that the practicality has faded, however, lags well behind the reality. Everyone has seen those spreadsheets with external links, pivot tables, macros, VB scripts and whatever else can be packed into spreadsheet these days. It works today, so nobody wants to change it and it never seems like that one little, extra feature is asking too much. The problem is that continually doing so over the years is where this approach meets its ill-fated end.

We have all seen, heard of or actually been in situations where organizations are floundering and struggling to make decisions. They talk about how the data needed to support decisions is not available or takes too much effort to acquire and manipulate. Then, a new employee comes into the team and is shocked by the situation. They quickly point out how so many sources are bloated with features and enhancements to address oddities in the data that have occurred over the years. After further analysis, they expose even more little tweaks and adjustments that have been put in place at what appears to be exorbitant cost. Worse is that the source is still not providing the data quality desired.

Our vision was blurring and we never recognized it. The unintended use of someone else’s glasses was the wake-up call that was needed, and allowed us to immediately realize our eyesight had drifted so far off course from our childhood 20/20 vision. In this case, it was the new employee who was able to immediately recognize that the data quality wasn’t sufficient to continue running operations and was already undergoing immense manipulation at considerable costs. Neither of which was recognized by the existing employees or management.

There comes a time when you need to take a step back and take a fresh look at what you’re doing. Maybe it takes bringing someone in from outside to look at your data or a new tool to analyze it differently. The bottom line is that the pattern needs to be broken and it may take new eyes or tools to provide that fresh look at your data. Having grown and been involved with the current environment makes it that much more difficult for you to recognize how far it may have drifted from where you started and need it to be. The old saying “if it ain’t broke, don’t fix it” is dogging a lot of enterprises today. It is, in fact, “broke”, but people are so close to the problem they don’t realize it. The better maxim might be “If it ain’t broke, break it, and make a better one.”

The fundamentals of Configuration Management Database (CMDB) with the objective of mapping business to technology

As the world of technology changes, many new technologies and techniques complicate the traditional IT Service Management vision. For example, the Agile/DevOps community embraces the concept of Infrastructure as Code. What is that? Wikipedia defines it as “the process of managing and provisioning computing infrastructure (processes, bare-metal servers, virtual servers, etc.) and their configuration through machine-processable definition files, rather than physical hardware configuration or the use of interactive configuration tools.”
Infrastructure as code is a two-edged sword when it comes to the traditional CMDB. Machine definition files would be an excellent source of data for a federated CMDB, but if one is sufficiently mature with infrastructure as code, who needs a CMDB at all?
Other modern trends include increasingly volatile virtual machines, and now ephemeral containers. The ultimate vision is one of infrastructure on demand – for example, the Amazon Web Services (AWS) Lambda computing platform, a “compute service that runs code in response to events and automatically manages the compute resources required by that code.”
With infrastructure so completely driven by need, and spinning up and down at lightning speed, what will be the role of the CMDB?
To answer this, we need to get back to basics. In my experience, the CMDB is most valuable as a sense-making tool that helps people understand the business value of technology. What is the minimum data required for this? I believe it is:
* The lowest level business concept
o Mapped to
* The highest level technical concept
This may be puzzling terminology, but bear with me.
Most businesses understand their highest levels very well: major value chains, product families, channels, and P&Ls. They may map value chains into enterprise processes and lower level workflows. They can break down product families into products, channels into customers, and P&Ls into budgetary accounts. But, what they struggle with is mapping the IT resource back up into this familiar language. (That’s why the IT Financial Management Association has existed for so many years, with its successful ongoing conference series.)
On the other hand, technologists live in their details – IP addresses, ports, virtual machines, databases, firewalls, load balancers, software products, the list goes on. One of the initial motivations for IT Service Management was to get technologists out of this technical swamp and challenge them to define their activities in a more business-centric manner. Thus was born the “IT Service”, a wrapper to contain various technical resources and give them a more business-friendly identity. Many organizations have explicitly embraced the “IT Service terminology”. Other organizations found they could achieve the same result by elevating the term “Application” to the same level. Whether “Service” or “Application”, these terms became the highest level technical category.
And, mapping the lowest level of the business to this highest stable level of the technology is, in my view, the fundamental CMDB goal. At the end of the day, it doesn’t matter what concepts you choose, as long as you are bridging the gap.
One of the most frequently seen mappings historically has been “Application to Server”. This was the state-of-the-art for many companies. It was an evolutionary step, because previously these companies only knew they had thousands of servers. Determining what was running on any given server might require expensive research and, of course, accounting for the server’s costs was challenging as well.
Sometimes it might be known what cost center or project funded the server, but this was often unsatisfactory; servers could be re-allocated according to demand and the original acquirer might no longer be responsible.
Other companies might reject the term “Application” (frequently because of the perception that it was equivalent to Software). Instead, a concept of IT Service would be mapped to the Servers.
Notice also that the Server itself is a “highest level” concept for much other IT technical data, including network dependencies, databases and more. The IP address resolved via the Domain Name system is an essential point of coupling that decodes much else in the environment.
But, these are only the best known mappings. What if you no longer care about servers, or applications?
You probably still have concepts of business and technology. On the business side, a very popular new concept is “product”. You have a digital product portfolio and the products have owners. These start to become management points. You’ll find yourself wanting that normalized, high-quality list of official Products just as you needed a high-quality Application portfolio ten years ago, and for all the same reasons.
On the technology side, even in a virtual or full Cloud environment, and even if you believe “servers are cattle not pets”, you need to have some ability to trace those cattle to their origins. Why was the server initialized? Why is it showing up on the bill? If it’s running and starts throwing errors, who is responsible? These questions don’t go away just because you’re operating in the Cloud.
In closing, the more things change, the more they stay the same. The important thing is to understand why you want your CMDB or repository and the value you hope to derive from it.  We’ll talk next about how to quantify this value a bit better.

The repository evangelist

Let’s talk about the Repository Evangelist. (Repository being CMDB, CMS, ITAM, or take your pick.)

The repository evangelist believes, PASSIONATELY, that “Knowledge is Power”. By putting as much IT management data as possible into a single system, “Miraculous Things Will Happen”. They say things like,

“We need to Run IT Like a Business!”

“The CMDB is the single source of truth!”

The trouble with the statements like that is that they are true and yet misleadingly incomplete.

Yes, it is true that “business” domains such as ERP, CRM and supply chain management require large and sophisticated systems that aggregate much diverse data. And yes, it is true that IT process is often under-automated.

But, simply putting data into a database for its own sake is misguided. It is dangerous to envy business systems just because they are large scale and seemingly sophisticated. These systems were not built to scale just for the sake of scale, nor for hoarding data like some pack rat. There’s a name for engaging in something because someone you admire does it, without understanding why they do it. It’s called Cargo Cult Thinking (Wikipedia – Cargo Cult Science).

The point of any production system is to deliver business results. If business results can be supported just as effectively with data in a well-maintained spreadsheet, then do that! Production systems are expensive to acquire, implement and run.

And, we need to be clear on what we mean by business results. “Integrate data” is not a business result. Even “better enabled IT service management processes” are not really a result.  But, “reduced mean time to recovery” (MTTR) might be meaningful, especially in environments where outages cost real money.

The evangelist has passion. They KNOW that there has to be value somewhere in all that data. But sometimes an over-zealous supporter can cause you more harm than someone opposed to your program.

In order to have credibility, you need to be open about the capability of your systems. What specific business problems are you trying to solve? Fortunately, IT repositories do have many potential value propositions.

For example, if your intent is to reduce the risk of change by communicating planned changes to appropriate stakeholders, a repository containing servers, applications and support groups all correctly related may be essential.

Similarly, if you want to reduce Mean Time to Recovery, and know you have a problem with getting the right people on a conference bridge quickly, the repository again may be essential. But you should have some evidence that you are solving a real problem!

In the final analysis, there are only a few ways to truly add value:

Grow & protect revenue

  • Cut costs
  • Reduce risk

You need to be able to draw a clear line of sight from your data repository to one of those goals. How exactly are you reducing risk? Cutting cost?

One commonly overlooked value proposition for repositories is reducing the waste of re-capturing the same data over and over again. More than one CMDB has built a business case on the fact that the same people (often, application and infrastructure managers) were being surveyed over and over again. This re-surveying might be driven by:

  • Server consolidation efforts
  • Portfolio rationalization programs
  • Compliance initiatives
  • Technology refreshes
  • And many more

Complex IT organizations with many moving parts are managed along multiple dimensions, but one thing they all have in common is a thirst for up-to-date data! And, after you’ve gone chasing after the same data over and over again, doesn’t it start to seem like a waste?

So where does this leave your repository evangelist?

Your repository evangelist has the passion and motivation to do the right thing. Help them understand that there needs to be a clear line of sight from data to business value. For example, “Up-to-date data means”:

  • That we will spend less time identifying impacts and owners, which should reduce “Mean Time to Recovery.”
  • That we reduce the risk of a multi-million-dollar licensing “true-up”
  • Up-to-date consolidated data means that these three projects can each save $50,000 in discovery and analysis costs.

Help your evangelist frame the repository value proposition in these more concrete terms, and more people may start to believe in them!

The Data Quality Skeptic

As a Configuration Management Database (CMDB) or IT Asset Management (ITAM) repository owner or sponsor, it’s only a matter of time until you meet The Data Quality Skeptic within your organization.

The Data Quality Skeptic has the following characteristics:

  • Does not believe that the repository can be kept up to date
  • Refuses to use repository data and disparages it at every opportunity. “I saw bad data in there! You can’t trust it.”
  • May be in a position to direct an entire team to avoid the repository, reducing your value proposition
  • Often has their own data management capability or process they favor

In the worst cases, The Skeptic can derail your entire CMDB or ITAM repository initiative.

How do you deal with this person?

The first thing is to accept that their concerns are valid. A CMDB or ITAM repository with poor data quality may be worse than no CMDB at all. Discounting data quality as an issue will not help you!

Instead, you need to change the terms of the debate. Fortunately, while data quality is often an unfamiliar topic to IT service management, it has its own community who have developed useful guidance over the years.

The Data Management Body of Knowledge is a rich and detailed survey of data management best practices. You can start with the concept of a data strategy:

Typically, a data strategy is a data management program strategy – a plan for maintaining and improving data quality, integrity, security and access. However, a data strategy may also include business plans to use information to competitive advantage and support enterprise goals.

Data Management Body of Knowledge

A Data strategy for a CMDB must place data quality front and center. Data quality typically includes the following:

  • Quality indicators and controls (what is the data expected to look like)
  • Exception reports
  • Audits
  • Trending (display/use for Organizational Change Management purposes)
  • Continuous improvement effort that addresses the causes of quality issues (not just fixing defects)

Perhaps the biggest challenge with data quality is that perfect data quality is not possible, any more than 100% uptime is possible. This makes dealing with The Skeptic difficult, as they will use any instance of data inaccuracy to discredit the entire repository effort. The best response you can have is again to change the debate from one of “managing by anecdote” to one of “continuous improvement”.

This requires measurement. You must track your exceptions (whether manually or automatically identified) and be able to present data such as this:

Percentage of Data Quality Exceptions


What does the above diagram represent? It is tracking two kinds of data quality exceptions:

  1. Hosts with no services (e.g. applications) – we know the asset is there, but we do not know what it is doing or how it is providing value. Is it the Accounts Receivable System? Or the employee potluck signup?
  2. Hosts in the IT Asset Management system but not in the Fixed Asset system.

It is telling us that both of these exceptions have trended down over the year (perhaps as the result of some focused cleanup programs).

With data like this, you change the conversation:You cant trust it-01


The question is, how much data quality do you need and can you afford? “No errors” is unrealistic. Instead, you need to have a business conversation about the risk that bad data quality presents and understand those risks in terms of all the other risks your organization must respond to.  For example, it is very difficult to fully reconcile IT Asset with Fixed Asset systems (even 10% might be unrealistic). Your company might decide that 20% exceptions are OK and the risks of those exceptions will be handled via other controls.

The important point is to focus on the business needs that drive your need for quality IT management data. By keeping this as your priority and basing the conversation on “continuous improvement” rather than “unattainable perfection,” your CMDB/ITAM effort will be on solid footing.

Blazent’s critical role within Technology Business Management

I recently joined Blazent as their VP of Marketing. My first day on the job I flew to Chicago to attend the Technology Business Management (TBM) Conference sponsored by Apptio. On your first day on the job, you are, of course, supposed to be pumped (particularly if you’re a VP of Marketing). I have however, also been in technology marketing a long time, and a certain amount of cynicism is inherent after a while. In spite of this, the difference on my first day with Blazent was immediate and very apparent. When we described what we did and why it mattered, people understood right away (no justification, no rationalization), and the level of interest I saw from both prospects and partners was stunning. So why did this happen?

For those who have not had exposure to TBM, it is the next phase of evolution for the IT ecosystem as technology infrastructure and its management race to keep pace with accelerating business requirements. Similar to ITIL, which defined and contextualized the alignment process of IT with business services, TBM defines the standard elements of managing the business of IT, focusing on organizational considerations, core disciplines, value optimizing decisions, and requirements for a performance based culture that drives an effective TBM program.

Technology, like the business it supports, is (in reality) completely dependent on accurate, current, and contextualized data for making informed decisions at both the tactical and strategic levels. There are multiple forces hitting the business of IT, and the implications are game changing. Billions of people are uploading the minutiae of their lives on a continuous basis; there were actually a few minutes recently where there were one billion people on Facebook at the same time. And that’s just people; you start adding in machines (e.g. sensors, RFID tags, telemetry systems), the number of input sources jumps to the trillions. The volume of data is already beyond huge, and is nowhere near where it’s going to be. Same exact thing with Velocity and Variability of data. All of this impacts the business of technology, which is why TBM is getting so much traction. Industry leaders are seeing what’s happening, and an updated operational framework is needed.

The challenge with managing all the Vs (volume, velocity, variability) falls directly into IT’s lap. There are indescribable amounts of data entering the system, and all of this data is being used to make decisions – data drives information, information drives business. The real problem is actually a matter of alignment; data coming in at high volume from so many different sources is bound to have inconsistencies, and inconsistent data leads to inaccurate information, which can lead to poor business decisions. Which bring us to the other core element (also a V), Validation. Validating the data by contextualizing it and aligning it for consistency is a foundational requirement for business success, and for the success of TBM.

TBM is a great operational framework that is getting a lot of traction across a broad range of industries. It is the business of IT. Business relies on information, which relies on data, which relies on Validity. This is an area where Blazent’s value-add has a direct and immediate impact on TBM and the companies that are increasingly depending on it for the success of their business. By correlating and validating data across all the disparate data sources that drive business (humans, machines, siloed data, all sorts of processes, etc.), we focus on the foundational element that changes everything – Validated, and hence, accurate data. Valid data leads to valid information, which leads to valid business decisions, which is the core premise associated with Technology Business Management.