Making these connections requires an integrated view across observations from many different sources, including structured, semi-structured, and unstructured data, and even advanced sources such as video with facial recognition. Jeff pointed out that one flaw in conventional business intelligence tools is that they require smart people to ask smart questions, and only then can these tools give answers. There's no way your organisation has enough smart people to ask all the right questions all the time, so you need analytics that find relevant connections and bring them to your attention, telling you things you would otherwise never have known, such as the connection between the arrest record and the croupier's job application. Entity Analytics are also quite valuable for developing richer "views of the customer" as well as for householding and other techniques crucial to success in the era of Digital Disruption.
Jeff used a story about jigsaw puzzle pieces to convey a powerful metaphor for linking information and observations. He has used groups of people assembling jigsaw puzzles to conduct experiments that reveal important insights about the way humans' analytical thinking enables them to link pieces together to make a picture, just as analysts want to link disparate observations together to form a cohesive picture of an intelligence threat, to find a perpetrator after a bombing, or even to learn enough about you to make you offers that you just can't refuse. But Jeff's presentation happened the week before the Boston Marathon bombing, and when that happened I wondered what role NORA's descendants might have played in analysing video feeds and finding the bombers.
Unfortunately, Jeff sees many organisations getting dumber about their data - the algorithms they have developed to help them make sense of their data are not growing and innovating fast enough to keep up with the flood of new data from new sources, such as location data, which is a potential source of deep new insights. He calls this gap "enterprise amnesia," and told the story of retailers that have been known to hire associates who were previously arrested for shoplifting - from the same store location.
Lessons Jeff Learned From A Lifetime Of Linking Data Together
- Data is often imperfect - and that's usually a good thing! You don't need perfect information to find interesting relationships in the data - in fact, counter-intuitively, "dirty" data is sometimes better for finding relationships, because cleansing may remove the very attribute that enables matching. On the other hand, some information is a lie, as "bad guys" will intentionally try to fool you, or to separate their interactions with your firm into different channels (web, mobile, store) to avoid detection. You should assign a trust level to "known" information, and it rarely approaches 100%.
- Your data can make you smarter as time passes. As new observations continue to accumulate, they enable you to refine your understanding, and even to reverse earlier assertions of your analysis based on what you knew at the time. Therefore, be sure to rerun earlier analyses over the full dataset, and don't assume the conclusions of your previous analysis were correct.
- Partial information is often enough. It's surprising how soon you can start to see a picture emerge - with puzzles, the picture can often be identified with only 50% of the pieces, and this aspect of human cognition often applies to machine learning, too. Once the picture starts to emerge, you can more quickly understand each new puzzle piece (observation) by seeing it in the context it occupies among those around it
Sign up for MIS Asia eNewsletters.