IBM’s purchase of SPSS (for $1.2 Billion!) was the initial spark to my thinking. After all, IBM is already heavily engaged in business consulting. And everyone has heard of the Smarter Planet concept. Most observers took the SPSS purchase as a bold move into predictive analytics, but I believe this was a more transformative purchase and a signal of where business consultancies are headed.

An even clearer signal is the formation of NM Incite, a joint venture between Nielsen and McKinsey. When Nielsen and McKinsey join forces, it is worth examining. Nielsen has the social media measurement prowess (BuzzMetrics) and McKinsey obviously has the business consulting brand.

A pattern is emerging.

Management-business consulting is clearly moving into the insights ecosystem in order to strengthen its core consulting offering by gaining proficiency with the analysis of new information flows (whether it is existing company data or open source social media data).

I cant think of a reason why this pattern won’t continue. The data inside and around companies continues to expand geometrically. There is strategic value in this data.

Insights ecosystem beware. Resistance (may) be futile. You (may) be assimilated.

Then again, the insights ecosystem may just take Ian Lewis’ good advice and move into a more consultative role themselves.

Rubinson takes a look at the evolution of the ad agency and applies this evolutionary footpath to MR with media buying the analog to the data collection side of the business and creative the analog to the full service, analytics side of the business. It’s worth a read.

In my thinking about the future of the MR agency, I have developed at least five (5) plausible futures (not mutually exclusive):

1. Army of Davids: In this scenario, the larger firms keep consolidating and attempting to buy their future by acquiring hot new companies, but the systemic change and “free agent nation” overwhelms them. The goliaths are killed off by the Davids. The carriagemakers can’t adjust to the age of the automobile, and they are overhwelmed by new entrants, new technologies, new approaches (social media listening, etc.) and agile specialty firms.

2. Whuffie-Driven Free Agency: A second and more extreme “Army of Davids” scenario, Whuffie-driven free agency develops when individual researchers band together under a kind of digital guild in which each is rated by his-her clients. The resulting client score, their Whuffie (hat tip to Cory Doctorow’s “Down and Out in the MagicKingdom”) roughly determines an individual researcher’s compensation if and when they are chosen to do the work. Sound fantastical? Hardly. Consider: (a) eBay feedback profiles and (b) the mechanical turk. Now assume that a community of credentialed researchers (think PRC or similar) build an elance site on these principles. Insights rockstars, as opposed to large companies or even specialty firms, dominate the scene.

3. Convergence: Under the convergence scenario the MR industry is renamed, rebranded and redefined by a much larger collection of data-driven strategic consultants, including: management consulting, social media listening and analytics firms, predictive markets, game modelled consumer behavior research, foresight practices and IBM. Under this scenario, the first to combine all fields in an integrated way gains a significant advantage. The deepest pockets are the most favored here.

4. Incredible Shrinking MR: In this scenario the market research industry retains its current, distinctive, identity. It is not reinvented or transformed. Plenty of new innovations come along, but they’re not part of MR. MR labors on, like the dinosaurs after the first asteroid strike, oblivious of their doomed future.

5. DIY: Google, Facebook and future social media platforms enter the consumer research business by offering unimaginably massive panels of their users-members. With MR now a simple page on these social media sites, the era of DIY research is fully unleashed. Need a quick, global survey of fashion conscious, well-educated women? No problem. GoogleSurvey will collect data for you in 30 minutes after you have posted your (mandated) 5 questions or less survey. Need some collage-based qual done in 20 countries on a new tourist destination? No problem. 3 hours. Survey quality suffers, but the data is abundant. Insights? Well, those are a bit tougher to come by.

Marshall Toplansky, President of WiseWindow, explores this question and scans the horizon of market research in a new MRA Alert article. For those MRA members, I highly recommend reading Toplansky’s article titled “Measuring the Future of Market Research.”

Reading this article reminded me very much of the things I’ve been writing on this platform and in my white paper (“Insight’s Future; From Market Research to Strategic Insight”) and I’m glad to see others thinking along similar lines.

One area of considerable agreement is what he titled “the new expectations.” Here Toplansky is, at base, writing about organizations’ increasing “need for speed” and how this is impacting traditional market research. Think James Gleick (of Faster book fame) meets Simon Chadwick.

As Toplansky colorfully puts it:

“To today’s CEO, a six week qualitative study to find a simple answer is a slow moving elephant in a race for gazelles.”

I believe that clients now have 8 demands of market research:

1. Strategic Recommendations
2. Concise Deliverables
3. Deeper Insight into the Whole Consumer
4. Speed
5. An Integrated Understanding of the “Infoverse”
6. Truly Understanding the Role of Emotion in Human Behavior
7. Insights Management
8. Value

And speed is #4 on my list.

Given the compressed decision making cycle in corporations, there is certainly a need for faster data collection, analysis and reporting of insights. Of course, “fast” and “deep thinking” are difficult to combine. And, there are limitations on how quickly good research can be conducted. But, there is strong client demand for speedier delivery of research based insights and firms that can work quickly, using a 24 hour “global clock”, will be at a significant long term advantage.

As I noted in my white paper, this threatens the large traditional suppliers who do not appear to be built for speed. For them it will be imperative to argue for a deliberative process that tests assumptions as well as hypothesis. But, when it comes to speed, these large suppliers will be at a disadvantage relative to smaller, nimbler firms.

Of course, Toplansky takes his speed argument further and merges it with the emergence of what some have called social media “listening posts.” Here he rightly argues that mass analysis of unstructured, “unsolicited and unexpurgated” comments across social media platforms can provide organizations with a new kind of real-time tracking system – a trends and insights stream.

I generally agree with Toplansky’s argument. We have certainly seen a rapid development of social media research tools, and in time these will become incredibly powerful. To my thinking there are two key hinge points in the development of social media listening. The first is improvements in the analysis of unstructured text. The progress being made here is impressive, but the human element is still needed. The second is the representativeness of social media engagement. We assume that social media usage will continue to explode and eventually become ubiquitous across generational, gender and SES lines. This seems to be a solid assumption, but we’re not there yet and participation frequency rates can differ dramatically. I freely admit that projectability in the survey sense may not be a valid critique in this instance. My second concern about participation rates is the privacy issue. In my scenario building for the futures of market research, at least one scenario has privacy concerns reversing some of these basic assumptions and challenging the development of “listening posts” altogether. In fact, Jeffrey Henning is giving a speech on this very issue today at the CASRO Technology Conference. The title of this speech is “Um, We Didn’t Know You Were Listening.” As Henning puts it in his synopsis, “Unlike the traditional ethnography, consumers have not given researchers explicit permission to study them online. What do individuals think about this?” Good question.

Toplansky outlines the three “seminal” methodologies that he sees driving a “new era” of market research. I’m glad he uses this “era” terminology.

In my thinking on the futures (thanks to Peter Bishop at the University of Houston) of market research I have segmented market research into historical and evolving eras and epochs. The “Data Collection Epoch” which we are now exiting, began with face-to-face interviewing, advanced to telephone and then advanced again to online. I have also named this epoch the “Asking Epoch” because it was defined by the utilization of the structured survey instrument. But, there are now two epochs on the market research horizon. The first is what I call the “Listening Epoch” and the next I have termed the “Simulation Epoch.” The “Listening Epoch” is defined by observational analytics, a movement away from the survey instrument as the primary research vehicle and a significant shift to social media analytics and other observational technologies (such as fMRI, eye tracking ,etc.). The “Simulation Epoch” is defined by anticipatory research. It is this market research epoch that I am most excited about. I see the “Simulation Epoch” as one defined by mass simulation gaming, predictions markets like those designed by Inkling, MROC Delphi panels and strategic foresight. In fact, when market research enters this era, I believe that the survey instrument will be replaced by the online game and that market research game designers will replace today’s survey writers. This may sound a bit strange, but consider that this would mesh with the gaming behaviors of younger people today and would be more observational and less intrusive. One company that may epitomize this new Simulation Epoch is Simulex.

In essence, the “Asking Epoch” was about the survey instrument. The “Listening Epoch” is about real-time observation, and the “Simulation Epoch” is about modeling future behavior. One could argue that this progression takes us from a focus on the past (reported behavior in surveys) to the present (observed behavior and social media sentiment in real-time) and on to the future (gaming, prediction markets and scenario building).

Finally, Toplanksy discusses marketing eras. His thinking very much tracks with that of noted futurist Jim Dator and author Virginia Postrel. I think his taxonomy (industrial to information to relationship) is excellent, although I would not name the current marketing era the “relationship era.” Instead, I would christen it the “design”, “dream” or “experience” era. Market research, neuromarketing, fMRI and behavioral economics are significantly chipping away at the notion of the “rational man” and instead revealing the power of the creative and emotional elements of the human psyche. Toplansky gets at this in his piece in a shorthand way.

By this Gordon means to ask which players will dominate the market research industry.

His thesis is that the answer changes based on the year, but that one probable future is the rise, and ultimate dominance of the sub-contracting firms upon which full-service shops often depend. Several years ago the answer would have been the big advertising and communications holding companies (Aegis, WPP, etc.) More recently one answer might have been private equity. But, Gordon now thinks that the sub-contracting firms that provide full-service shops with data collection, data processing, sample, etc. are positioned to overtake the traditional MR firms.

As Gordon puts it:

“Just as researchers wonder how marketing consultants can justify their fees, increasingly the “servants” of market research are feeling that maybe their “masters” don’t deserve their profits.”

This is an interesting proposition. How will these sub-contracting firms break through? Gordon sees several different paths:

1. Out-sourcers grow up and offer more advanced services until they become full-service shops themselves.
2. Technical firms will gain entry via DIY solutions on the desktop and become data mining and integration experts who stay sticky with a portal or other GUI.
3. A final scenario, and one that feels a bit out of place with the article, is the rise of the external consulting company (think IBM, McKinsey, etc.) within the classic MR space. This has been well discussed on this and other venues.

While only time will tell, I have several thoughts on the issue.

1. This article assumes the “market research industry” will be recognizable in 2020. I believe that the industry will have undergone so much change by 2020 that (a) it might not even be called market research, (b) the players will be radically different (c) the tools will be dramatically enhanced-evolved and (d) the newer employees in this field may not think of themselves as “market researchers.”

2. Assuming that the trend toward commoditization of insights continues, these subcontracting firms may indeed be the major players of 2020.

3. But, I think more likely scenarios are that:

3a. New entrants from analytics firms, consulting firms (McKinsey, BCG, etc.), MROCs, neuromarketing and social media agencies transform the business and marginalize traditional players.

3b. Google, Facebook or other large online players become the sample provider of choice and then develop their own DIY and custom research offerings.

3c. The industry undergoes radical “free-agentism” with large numbers of current employees taking advantage of their skill sets and low barriers to entry and going out on their own. With each one touting their unique sector or research expertise, client side buyers begin employing a short term army of Davids via a MR version of the mechanical turk.

Is the term advantageous for the industry?

I have written about this before as what I describe as the “linguistic” challenge to the industry.

This is why it was so good to read Andrew Jeavons’ piece in the February edition of Research World (page 11). Jeavons is NOT arguing that the market research industry is dead or dying, but he is arguing that the term feels a bit like antiquated terms such as “VHS”, “fountain pen”, etc. He argues that we think more about owning the term “customer” (the people we generally study) and focus less on the “market” or the process by which we gather the date. I think he is completely right.

Many in the industry have already shifted toward the term “insight”, “insights” or “customer insights”. As I’ve written previously, I’m not sure that’s the best language. I think we can do better, but I think the associations need to take a leadership role in rebranding “market research”, and I think a name change is in order. As a researcher working in Washington, we test new language constantly, and more often than not counsel significant changes in our clients’ lexicon. I’m certainly not alone in doing this. It’s time we take our own advice.

As I have noted before, I believe that the term “market research” has been incredibly detrimental to the industry, because it focuses on the data collection process and NOT the value of data-driven strategy. This is a significant error. The term “market research” has minimized the industry’s influence within the corporate decision making structure by reinforcing the idea that the industry is all about the process and data dumps and not about the strategy that flows from the data. In my view it has put the industry into a cul-de-sac. Moreover, focusing on the physical act of the “research”, but not the value that is delivered, puts the industry into a commoditized data collection box. When the industry’s core strength is data collection in a world where data has become plentiful, then we have positioned ourselves poorly.

This gets at a much wider point. When I started in “market research” and polling, data was scarce, difficult to obtain, and difficult to analyze in anything remotely approaching a fast manner. This is manifestly NOT the case today. In the past, when data was scarce, mastering the process of data collection and being the expert in data collection methods was a wonderful niche. But, data is now abundant. It is so abundant that most of our clients feel that they are awash in it. The Economist ran as its cover story recently the title “The data deluge.”

But, what should we call ourselves?

This is a difficult question.

The “market research” industry is at a significant crossroads. There are so many new entrants (listening experts, management consultants, etc.) that I’m not sure many of the new players would consider themselves to be involved in “market research.” This is a telling sign. If the future of the industry isn’t likely to think of itself as part of the industry, then we’re seeing a quantum shift. This is why I think we’re likely to see a larger number of people analyzing customer centric data in order to build strategies that advantage their firms or clients, but these people may not think of themselves as “market researchers.”

So, again, what do we call ourselves?

“Insight” or “insights” is workable, but omits what will be done with this insight, is silent on strategic implications and does not place the focus on the future. Perhaps we should modify “insight” by adding “strategic”.

“Fact-Based Consulting/Strategy” is better, but feels a bit cheeky, since it implies strongly that everyone else is just going on gut (which they are). Still, I view this as a strong option and it is the positioning GfK is going for.

“Data-Driven Strategy” is one of my favorites, and feels stronger and more active than “fact based consulting.”

Another option is “Strategic Foresight”. Of course, this technically means something historically different from market research. But, if I’m correct, what we call “market research” today will over time merge with what we think of today as “strategic foresight.”

Of course, any real rebranding effort would utilize research among our buyers in the c-suite.

But, I’m interested in your thoughts.

What should we call ourselves?

Ideally, it should be read in tandem with his January Quirk’s piece titled “Ten keys to increasing the impact of research” and with his comments on this website.

Although I highly recommend that you read (and re-read) Ian’s piece in Research World and draw your own conclusions, there are several things that I think are worth amplifying.

1. His concept of a “management contract” for MR is new to me and seems challenging, but novel.
2. Ian’s emphasis on “future-focused knowledge” that is “above the brand” makes two points simultaneously. First, that MR needs to inhabit a more strategic space with fewer “validation” surveys and second, that MR will naturally over time begin to blend with strategic foresight.
3. His thinking around future leadership in MR is dead on. Until recently mastery of MR meant mastery of a process that yielded reltaively scarce data. Data is now abundant. The process is being automated and simplified. Creativity is the scarce item, not data.
4. His listing of mindsets and skills is useful for anyone in MR tasked with hiring and managing. It’s an excellent shortlist.

“High Impact Research” is an exellent piece, and I highly recommend it.

For those interested in prediction markets I strongly recommend David Rothschild’s article in this month’s Public Opinion Quarterly (AAPOR’s publication).

Although Rothschild’s article focuses on “comparing prediction markets, polls and their biases”, the implications of his article stretch well beyond the political world. In fact, from one vantage point, elections are just a useful space to study these markets since they happen at frequent and predictable intervals. But any future event with these similar parameters would do.

One critique of market research is that it is a snapshot of the present and is rooted in present attitudes and behaviors.

This is why I believe strategic foresight and market research will merge (or at least cross-pollinate) in the near future.

Prediction markets are just one example of an area in which this cross-pollination will occur.

BrainJuicer already appears to be experimenting with prediction markets.

Some possible (as opposed to probable or preferable) futures:

1. The commoditized part of the business is called “market research” and it loses talent and ages dramatically. The non-commoditized part of the business is called “strategic foresight” (see BCG report), “data-driven consulting”, or “customer community.” The industry becomes fragmented, but many of those working in the industry evolve with society and technology into new roles.

For some more thinking on this:

http://www.futureofinsight.com/2010/01/strategic-foresight/

http://www.futureofinsight.com/2009/12/professional-line-blurring/

2. Google and Facebook become massive market research suppliers/panel managers.

http://www.futureofinsight.com/2010/01/new-entrants/

3. Millennials working in a number of other fields reinvent the industry, but it isn’t called “market research.”

http://www.futureofinsight.com/2010/01/mr-millennials/

4. Participant engagement drops to the point where survey research is simply undoable. Researchers turn their attention to simulations (games) that participants enjoy playing. These games replace traditional survey-based research.

http://www.futureofinsight.com/2010/01/lightening-the-respondent-burden-making-research-fun/

5. Online and community research coupled with the decline of the landline kills off traditional phone-based research. But, the brave new world of online surveys hits a big speedbump as (1) the data isn’t projectable and (2) privacy concerns and legislation increase the cost. The industry then goes “back to the future”, interviewing people face-to-face. Walmart and Starbucks become (through a JV) a participant intercept network that everyone tries to leverage.

“Are the Kids Alright?”

What are they thinking about the future of market research?

Why aren’t we hearing from them?

My take:

I don’t think we’re asking their opinion and I’m not sure the industry is listening.

But, I would add another option somewhat related to “they’re too busy doing amazing stuff.”

“They’re in industries, businesses and functions that are not classified as ‘market research’ now, but will replace traditional ‘market research’ in a few years.”

Theoretical examples:

1. A MashUp whiz who is a baseball stats maven, has a boring job in accounting now, but will create a killer DIY research tool in 2012.

2. A gifted PHD candidate in anthropology that studies how women relate to food in their kitchens. She’ll go on in 2013 to build a stunning new “deep qual” firm.

3. A 25 year old Google math whiz that has never taken a survey course in her life, but will be tasked with architecting Google’s survey sampling methodology and DIY survey platform next year.

4. The 27 year old Boston Consulting Group wunderkind that is a virtuoso at synthesizing multiple data sets in his consulting gigs, but would never identify himself as a “market researcher” or in “market research”.

5. The 26 year old online gaming programmer that loves building simple simulation games for his buddies, but in 2013 will have the hottest new research interface on the planet. His company (Simusurveys) won’t ask respondent questions at all. Instead, it will have them play fun simulations. In fact, many people will play these surveys willingly as entertainment knowing that they are actually part of niche research efforts.

6. A 22 year old self-described “fashionista” in Miami that will create a global, invitation only, communnity for only the most cutting edge hipsters. When a high fashion corporation realizes she is sitting on a gold mine, they’ll buy her community for top dollar and hire her to maintain it. She will become their lead trends expert among the trendiest trendsetters. Overnight she’ll be a market research executive, but she’ll never use the term.

I could go on, but I think this is what part of the transformation will look like.

But, the two that have most interested me are (a) consulting firms like BCG or McKinsey and (b) large digital platforms that could become uber-panel companies of the future. These two interest me because they occupy the two extremes of the industry: the data-driven strategy providers (high value, non-commoditized) and the data collectors (commoditized but VERY important).

I’ve written and talked about the former quite a bit, but not so much about the latter.

This is why Stan Sthanunathan’s thoughts (click here) on the matter have so intrigued me. I met Stan at TMRE, and he’s pleasure to talk with.

Stan, insights leader at Coca-Cola, poses two questions:

What if…

1. Facebook becomes an insights provider?
2. Google creates a MASSIVE insights panel company?

Thinking about each of these, and looking at the first possibility, it seems obvious to me that Facebook will become an insights provider of some kind. It also seems obvious to me that there will eventually be a privacy backlash of some kind. (As an aside, if Facebook entered the market research supplier market in a big way, it could complicate things dramatically for the MROC builders. In fact, if I were Facebook, I would buy the largest MROC builder. It is a natural fit.)

But what about Google as an uber-panel company? LinkedIn is already far down this path on the B2B side (click here). It doesnt seem to be much of a leap for Google to get heavily in the game.

What if Google entered the market?

1. It would enter in a BIG way.
2. It could offer online sample dramatically larger than the current providers.
3. It might mix in online behavioral data (triggering an eventual privacy backlash).
4. It could create easy-to-use, online, DIY survey tools. This would make surveys much more easy to execute, but would geometrically expand the volume of truly awful research.
5. Assuming this tool could manage the physical geographies of participants (setting geographic quotas), it could kill off phone polling quickly.
6. Tracking surveys could be conducted in realtime, continuously throughout an advertising or political campaign.

Someone at Google has to be thinking about this.

One of the people that will think about it will be the person with this job.

Food for thought.