What constitutes data worth collecting?
This is something that can only be decided in advance. At the highest possible level, hopefully Board level. Data-driven business (or Big Data, if you prefer to focus on raw materials rather than process) is a systemic activity. A systemic activity does not take place without serious managerial endorsement and planning.
I hear you say: sure, and what about digital illiteracy, volatile management requests, contradictory expectations, non-existent budget, and the general ICT scapegoating?
CIOs should stop whining about Board decisions and start deserving their place there
My reply is this. Do CIOS and people in ICT believe other functions do not face the very same obstacles? Do you really believe the typical CMO doesn’t get his budget by tooth and nail (yes, yes, unwarranted, undeserved, bloated, etc. Still, CMOs get the money the ask.)? If you really believe this we are definitely not worth C-level. Which is OK, so just stop whining and head back to your
cage where you can play BOFH, read Dilbert and rant about lusers between tech support tickets until someone finds a way to outsource you. (And someone will: outsourcing non-core IT is my specialty, support tickets being totally non-core).
So, for those of us who believe there’s life and work in IT beyond tech support, let’s get back to business.
What constitutes data worth collecting?
This is a strategic question: we need to plan for what we intend to do with data.
Of course, there’s NoSQL and all sorts of other wonderful technologies to deal with unstructured data. But remember that even the NSA that in the world of his boss Gen. Alexander “collect all” needs to discriminate what needs to be paid attention and what doesn’t.
And let’s be frank: your company may love NSA’s attitude but will never have NSA’s budget.
So, focusing on which data needs collecting is paramount.
You want to get into “you may also like” suggestions? Then you need to store each client’s buying history: it requires aligning databases in sales, logistics, production, and marketing. In several companies I know there are multiple CRMs, ERPs, DMSs, BPMs, BIs, and they are made to communicate by… Excel spreadsheets, elaborated by functions owners and joyfully re-input by IT personnel.
Alignment is a nightmare, for two main reasons:
- data is stored in different formats across different systems
- each function will have a different sense of what specific data are important, so fields will be empty or full depending on that.
- many fields will contain dummy values, whenever IT felt a field was required and the end-user didn’t.
You can only work today on the data you decided yesterday you needed
And alignment is not enough. You can only align the data you have. But if you never decided, say, client’s addresses, or Twitter accounts, or country of origin, or date and time of purchase, or preferred payment method were data to be gathered, you won’t have those data. (You don’t think all commerce is e-commerce, do you?)
Let’s face it: you can only work today on the data you decided yesterday you needed.
You want to investigate what makes your customers decide between different product models? You can of course use focus groups. Marketing will be happy for the budget increase. But then, you will have gorgeous, colored, powerpointed reports telling you what customers say they do. Which is not even close to what they actually do.
If you want to assess how in-store product placement, tryouts, sales personnel ability and reaction times, store and product lighting, and a hundred other sales-environment variables, you need to decide they are variables in the first place.
Or you may never discover that what keeps down sales of product X, is that it is placed too close to the second-best option for the customer to be lured into buying. Or that the customer can have access to five models at a time but trying out all of them takes too long, so he’ll naturally avoid extremes.
So, here’s your to-do list to insure your business can start working with data:
- get relevant function owners to indicate what they want to do with data
- make a list of the data needed to perform those actions
- check existing databases to see if those data are being collected, and where
- sample-check those data for validity.
The first step will be the hardest, and if you do not have good working relationships with function owners, you’ll understand why you should.
At this point you are ready to appear as Creator Of Difficulties (or Preventer of Information Services –yes, I read Dilbert too) which is not the right position to start negotiating from.
In the next step of the Eightfold Way, Right Collection, we’ll see what you can do to move on from the role of Creator of Difficulties (the standard role for in-house ICT).