Consultancy's Dirty Little Secret
Shuusshh! Close the door and come in closer to the screen, for what I'm about to tell you may cause the bulk of IT consultants to deny me the secret handshake... even vote to revoke my IT consultancy decoder ring. But here it is:
When it comes to Information Technology, data management is not that difficult.
That does it, now I'm cursed to be chased endlessly through the rows of server racks - by zombies who make tons of money by living the myth that data management is an incredibly complex task that should only be done by a super society - who promote expensive licensed "solutions".
Nope. Sorry. While the data itself is very important, data systems should be very easy.
First, consider these basics:
- Back up your data regularly. Secure your data against theft.
- Validate the data that people provide, to prevent errors.
- The best way to organize your data has already been thought out.
Folks like Ed Codd and Ray Boyce established database normalization rules years ago.
- A data storage solution should store the data you give it, and then give it back when you ask for it. Anything else is frosting.
- Storing the same data in more than one place is costly.
- Running many different data management systems is costly.
None of the above concepts represent anything worthy of hundreds of hours in billable consultation. Seems like a few good database and network administrators would do the trick. And yet, large and financially healthy companies will often find themselves awash in consultants, analysts and managers - all tasked with the important task of managing and synchronizing the firm's vital data. So, what happened?
The problem is not with the data itself. The issue is likely human, and can often be traced to one of the contributing factors below. Please bear in mind, I'm not saying that any of these conditions mark the traits of poor management or evil people. These factors are mostly caused by innocent human behavior over time, and they are not too hard to correct:
Underestimating data's role in compliance
Management fails to realize the importance of keeping all it's data in as few places as possible. This is the easiest rule to establish during start-up, and the hardest to reconcile after-the-fact. Without this core tenet, the firm risks a huge resource drain - or asset loss - when the first security or Sarbanes-Oxley audit comes along. Meanwhile, data can be lost to attrition, as employees leave - taking knowledge of random data locations with them.
Underestimating your own staff's data & application needs
The I.T. department of a large firm needs to do much more than set up computers and networks. They should be the in-house "technology services vendor" with regard to business users and the data they inevitably create and consume. If employees don't feel welcome asking I.T. for the proper software or system to use (or aren't reminded to ask), they're likely to develop a business data system of their own... perhaps on their own console or laptop.
Growth with little architectural oversight
A large enterprise will have many business centers and departments. They can be separated by walls, time zones or even just philosophies. If these disparate centers don't understand compliance, and if they are not provided the technical services they need, then they will develop or purchase solutions without knowledge of the big picture. Managers or directors might simply select a solution that worked at their last job. Rather than being involved in pre-purchase review, the I.T. department is saddled with integrating a less-compatible system afterward. Incompatible systems often means storing the same data in multiple places.
Invasion of the consulting behemoths
Large consulting firms make the most money by billing for each person, and then placing more of their people at the site; either as additional consultants or as employees (who then retain the consultancy for more work, or hire even more consultancy alumni as employees). This crabgrass-like growth is allowed to thrive only when the consultee thinks that the work is very technical and complicated, a misperception the consultant is only happy to allow - if not promote.
Bottom-Heavy in Applied Knowledge
One of the few ways to guard against a voracious growth of data management technologies - and people - is to have someone with years of hands-on data management experience (a 'coder') in a position of authority. These skills are important to architecture development and policy creation, they help debunk myths and dispel fears.
Unfortunately, principals can have a preconceived bias of coders as introverted 'cave geeks' with little business acumen or management skills. This bias is somewhat founded, as it's true that there are fewer people who posses both the skills of a coder and communicator.
But possession of at least one such person is virtually mandatory, whether recruited from outside or raised from within. Without the co-membership of an experienced coder, leadership often develops "analysis paralysis"; finding itself unwilling to shut down obsolete or redundant systems - for fear of some unknown systems failure. The addition of a new data system appears easier than learning and leveraging the existing one, because most other tangible maintenance problems can be fixed through increased expenditure.
But more spending, more systems and more people will only make data management more difficult, and it was so easy to begin with.