Monday, July 14, 2014

Big Data is still a big idea


The age of Big Data has been talked about for some time now and I’m sure you’re all familiar with the buzzword. But I’ll venture to guess that it means something different to everyone, like those words that are overused to the point where they mean both everything and nothing.

I used to believe that Big Data meant vast stores of corporate data about customers and related topics (mostly sales data but also competitive intelligence, market information, product specs and so on). When I worked at Verizon many years ago, we even talked about the mountains of data we had about telephone calls, and yearned for the day when we had systems (and computer power) capable of doing something with it. (The NSA must have made a note.)

I’ve been reading a new book by Patrick Tucker called “The Naked Future” that has taken my dim view of Big Data and enlarged my thoughts about it in, well, a very Big Way. The subtitle of the book – “What happens in a world that anticipates your every move?” – says a lot about Tucker’s POV. He talks about powerful and sophisticated predictive software that, when harnessed with the data that all of us as individuals are generating, can allow others or ourselves to eerily predict the future.

An example he gives is a scenario in which an app on your smartphone cheerily predicts that on your way to lunch today you’re going to pass an old girlfriend on the street, and by the way, she’s just got engaged to be married so try to act surprised. Sounds wild, but all it takes is (1) lots of information, in this case about your past friends and your plans for lunch, plus more information about your friends and their plans for lunch (and marriage), and (2) some smart software to connect the dots for you. Sound far-fetched? Given that just about everything on our smart phones now was far-fetched only 10 years ago, this scenario becomes a lot more plausible.

Tucker emphasizes the terabytes of data that we create and record about ourselves today: not just Point Of Sale transactions (the old basis of the Big Data idea), but a million other recorded data points: Facebook posts, photos, tags and check-ins; browsing histories; texts, tweets, calls and emails; Google search queries; physical monitoring data from wearable devices like Fitbit; plus a continuous stream of GPS location data of where we are at all times. Telemetric data already senses huge data about the world around us (and the functioning of our own bodies) and the internet of things – the term coined for all the networked devices and data points that are on-line and growing – can only get bigger and bigger. All of this is being recorded and is, in varying ways, preserved and accessible by devices and entities known and unknown. Try to imagine how this information can be used to know us, track us and predict our next move.

I’m reminded of a true story that you may already know, since it’s been around a while; I use it in my Ethics class at Immaculata. A retail firm used clever programming to spot hidden meanings in on-line behavior and purchases. One prediction it was able to make was for women who had just become pregnant (partly, they switched from scented to unscented skin cream); the company then sent direct mail sales pitches to the women in hopes of selling some baby food and Pampers. A father of an unwed teenage daughter saw this and was outraged enough to send a scathing letter to the store CEO, which he also made public. Much to his shock, of course, his daughter later told him that she was in fact pregnant. The software spotted what the father had not. 

Big Data has a reputation of being the next big thing in how companies will manipulate and sell to us, and I admit that I had this negative view myself. “The Naked Future” goes well beyond that, showing how predictive systems can be used in a variety of ways, from solving crimes to helping us find our mates. Many of these ways, incidentally, put us in control of our data and in the driver’s seat when it comes to benefiting from what the systems can do. His final chapter sounds the death knell for privacy, but also for hidden persuaders and totalitarian manipulation, since the Big Data tools will be in our hands too. Let’s hope so.

As a buzzword, Big Data has been around since about 2000. But as force in our daily lives, Big Data is an idea whose time has come, and is coming still.

3 comments:

David D said...

The concept of predictive software was dramatized in the film Minority Report, loosely based on Philip K. Dick's short story written in 1956. In it, instead of by computers, the predictions are made by mutant psychics, but the fundamental similarity is striking. Predicting that a person is going to commit a crime, and arresting him preemptively, does still seem a wee bit, uh, unconstitutional. But computers are something like mutations of ourselves, and we are creatures of habit, predictable more than we want to believe. While we all think we possess free will, Big Data might -- god-like -- snatch that away from human kind.

DK said...

I could see big data being a course changer - similar to the way the smart phone, or, the computer in your pocket has. That being said, I don't think it will be an irrefutable predictive device. There's no way a computer program of any time (at this time) can account for the human chaos factor. The "I'm not going to work today, because" factor.

Still, it will be a big deal, and that's a crazy scenario that book paints out. Are we moving towards a hive mind of some sort?

Lainey said...

In agreement with DK. I do think there are inherent limitations to how accurately computers can predict human behavior. However, that limitation is also the scariest bit. As the software becomes more refined, the accuracy percentage will increase. If the assumption then becomes that a certain percentage of fallout is acceptable, the concern becomes which sectors of our society will have access to customized versions that can be applied in a less than ethical fashion. Who suffers unjustly from decisions made by government/legal institutions based on "mostly accurate" predictive models? We know that people in decision making positions are not infallible, but it can be harder to refute hard data. Another unattractive scenario is the probability that the software will find its way into the hands of criminal or terrorist organizations. Much to ponder about this very "Big" subject. I will add Patrick Tucker's book to my reading list.