IBM’s purchase of SPSS (for $1.2 Billion!) was the initial spark to my thinking. After all, IBM is already heavily engaged in business consulting. And everyone has heard of the Smarter Planet concept. Most observers took the SPSS purchase as a bold move into predictive analytics, but I believe this was a more transformative purchase and a signal of where business consultancies are headed.

An even clearer signal is the formation of NM Incite, a joint venture between Nielsen and McKinsey. When Nielsen and McKinsey join forces, it is worth examining. Nielsen has the social media measurement prowess (BuzzMetrics) and McKinsey obviously has the business consulting brand.

A pattern is emerging.

Management-business consulting is clearly moving into the insights ecosystem in order to strengthen its core consulting offering by gaining proficiency with the analysis of new information flows (whether it is existing company data or open source social media data).

I cant think of a reason why this pattern won’t continue. The data inside and around companies continues to expand geometrically. There is strategic value in this data.

Insights ecosystem beware. Resistance (may) be futile. You (may) be assimilated.

Then again, the insights ecosystem may just take Ian Lewis’ good advice and move into a more consultative role themselves.

This is a noteworthy development.

The course will focus on social media, online communities and virtual research platforms.

Until now, the Burke Institute was fairly orthodox and conservative in its offerings. Their courses, all very solid, focused on surveys, focus groups and analytics – the traditional tools of the “Asking Epoch” of market research.

But, times change and most of market research is now behind advances of the wider insights industry. The market research industry may be struggling, but the wider insights industry is doing quite well and innovating rapidly.

The Burke Institute is a good proxy for tracking how traditional market research catches up.

I’m not criticizing them. They’re great folks that do great work. But they are playing catch up.

I anticipate that the Institute will roll out the following classes in the near future:

1. Mobile research tools and techniques
2. Insights Communities, UGC and Co-Creation (this deserves its own class)
3. Social media (again deserving its own class)
4. Neuromarketing or Biological Response Research
5. Eye Tracking in Ad, Packaging, Retail Environment and Menu Testing
6. Prediction Markets

Tracking the rapidity with which they roll out these new classes will be a good gauge of industry adoption.

In the meantime, market research professionals should support Burke’s evolution and attend the Sep. 30 – Oct. 1 class or send their junior staff. Send the market a signal. If the class oversubscribes dramatically, then the Burke Institute will react to the signal, offering more classes in this area andexpanding their new offerings.

Tom Anderson promos the RIT a bit over at Next Gen Market Research. It’s worth a look.

The top two concerns are:

1. The non-representative nature of online samples
2. Quality of online sample is worse than most believe

Clearly these two concerns complement each other.

Although Jeffrey Henning is right about random samples vs. so-called “convenience samples”, the industry has clearly embraced “convenience samples” as the next best thing to purely random studies.

Stay tuned.

This year’s RIT study promises much more.

(Of course, I should disclose that my firm, StrategyOne, and I were pleased to be included in the survey design discussions for the RIT, and so I do have some “skin in the game.”)

Marshall Toplansky, President of WiseWindow, explores this question and scans the horizon of market research in a new MRA Alert article. For those MRA members, I highly recommend reading Toplansky’s article titled “Measuring the Future of Market Research.”

Reading this article reminded me very much of the things I’ve been writing on this platform and in my white paper (“Insight’s Future; From Market Research to Strategic Insight”) and I’m glad to see others thinking along similar lines.

One area of considerable agreement is what he titled “the new expectations.” Here Toplansky is, at base, writing about organizations’ increasing “need for speed” and how this is impacting traditional market research. Think James Gleick (of Faster book fame) meets Simon Chadwick.

As Toplansky colorfully puts it:

“To today’s CEO, a six week qualitative study to find a simple answer is a slow moving elephant in a race for gazelles.”

I believe that clients now have 8 demands of market research:

1. Strategic Recommendations
2. Concise Deliverables
3. Deeper Insight into the Whole Consumer
4. Speed
5. An Integrated Understanding of the “Infoverse”
6. Truly Understanding the Role of Emotion in Human Behavior
7. Insights Management
8. Value

And speed is #4 on my list.

Given the compressed decision making cycle in corporations, there is certainly a need for faster data collection, analysis and reporting of insights. Of course, “fast” and “deep thinking” are difficult to combine. And, there are limitations on how quickly good research can be conducted. But, there is strong client demand for speedier delivery of research based insights and firms that can work quickly, using a 24 hour “global clock”, will be at a significant long term advantage.

As I noted in my white paper, this threatens the large traditional suppliers who do not appear to be built for speed. For them it will be imperative to argue for a deliberative process that tests assumptions as well as hypothesis. But, when it comes to speed, these large suppliers will be at a disadvantage relative to smaller, nimbler firms.

Of course, Toplansky takes his speed argument further and merges it with the emergence of what some have called social media “listening posts.” Here he rightly argues that mass analysis of unstructured, “unsolicited and unexpurgated” comments across social media platforms can provide organizations with a new kind of real-time tracking system – a trends and insights stream.

I generally agree with Toplansky’s argument. We have certainly seen a rapid development of social media research tools, and in time these will become incredibly powerful. To my thinking there are two key hinge points in the development of social media listening. The first is improvements in the analysis of unstructured text. The progress being made here is impressive, but the human element is still needed. The second is the representativeness of social media engagement. We assume that social media usage will continue to explode and eventually become ubiquitous across generational, gender and SES lines. This seems to be a solid assumption, but we’re not there yet and participation frequency rates can differ dramatically. I freely admit that projectability in the survey sense may not be a valid critique in this instance. My second concern about participation rates is the privacy issue. In my scenario building for the futures of market research, at least one scenario has privacy concerns reversing some of these basic assumptions and challenging the development of “listening posts” altogether. In fact, Jeffrey Henning is giving a speech on this very issue today at the CASRO Technology Conference. The title of this speech is “Um, We Didn’t Know You Were Listening.” As Henning puts it in his synopsis, “Unlike the traditional ethnography, consumers have not given researchers explicit permission to study them online. What do individuals think about this?” Good question.

Toplansky outlines the three “seminal” methodologies that he sees driving a “new era” of market research. I’m glad he uses this “era” terminology.

In my thinking on the futures (thanks to Peter Bishop at the University of Houston) of market research I have segmented market research into historical and evolving eras and epochs. The “Data Collection Epoch” which we are now exiting, began with face-to-face interviewing, advanced to telephone and then advanced again to online. I have also named this epoch the “Asking Epoch” because it was defined by the utilization of the structured survey instrument. But, there are now two epochs on the market research horizon. The first is what I call the “Listening Epoch” and the next I have termed the “Simulation Epoch.” The “Listening Epoch” is defined by observational analytics, a movement away from the survey instrument as the primary research vehicle and a significant shift to social media analytics and other observational technologies (such as fMRI, eye tracking ,etc.). The “Simulation Epoch” is defined by anticipatory research. It is this market research epoch that I am most excited about. I see the “Simulation Epoch” as one defined by mass simulation gaming, predictions markets like those designed by Inkling, MROC Delphi panels and strategic foresight. In fact, when market research enters this era, I believe that the survey instrument will be replaced by the online game and that market research game designers will replace today’s survey writers. This may sound a bit strange, but consider that this would mesh with the gaming behaviors of younger people today and would be more observational and less intrusive. One company that may epitomize this new Simulation Epoch is Simulex.

In essence, the “Asking Epoch” was about the survey instrument. The “Listening Epoch” is about real-time observation, and the “Simulation Epoch” is about modeling future behavior. One could argue that this progression takes us from a focus on the past (reported behavior in surveys) to the present (observed behavior and social media sentiment in real-time) and on to the future (gaming, prediction markets and scenario building).

Finally, Toplanksy discusses marketing eras. His thinking very much tracks with that of noted futurist Jim Dator and author Virginia Postrel. I think his taxonomy (industrial to information to relationship) is excellent, although I would not name the current marketing era the “relationship era.” Instead, I would christen it the “design”, “dream” or “experience” era. Market research, neuromarketing, fMRI and behavioral economics are significantly chipping away at the notion of the “rational man” and instead revealing the power of the creative and emotional elements of the human psyche. Toplansky gets at this in his piece in a shorthand way.

By this Gordon means to ask which players will dominate the market research industry.

His thesis is that the answer changes based on the year, but that one probable future is the rise, and ultimate dominance of the sub-contracting firms upon which full-service shops often depend. Several years ago the answer would have been the big advertising and communications holding companies (Aegis, WPP, etc.) More recently one answer might have been private equity. But, Gordon now thinks that the sub-contracting firms that provide full-service shops with data collection, data processing, sample, etc. are positioned to overtake the traditional MR firms.

As Gordon puts it:

“Just as researchers wonder how marketing consultants can justify their fees, increasingly the “servants” of market research are feeling that maybe their “masters” don’t deserve their profits.”

This is an interesting proposition. How will these sub-contracting firms break through? Gordon sees several different paths:

1. Out-sourcers grow up and offer more advanced services until they become full-service shops themselves.
2. Technical firms will gain entry via DIY solutions on the desktop and become data mining and integration experts who stay sticky with a portal or other GUI.
3. A final scenario, and one that feels a bit out of place with the article, is the rise of the external consulting company (think IBM, McKinsey, etc.) within the classic MR space. This has been well discussed on this and other venues.

While only time will tell, I have several thoughts on the issue.

1. This article assumes the “market research industry” will be recognizable in 2020. I believe that the industry will have undergone so much change by 2020 that (a) it might not even be called market research, (b) the players will be radically different (c) the tools will be dramatically enhanced-evolved and (d) the newer employees in this field may not think of themselves as “market researchers.”

2. Assuming that the trend toward commoditization of insights continues, these subcontracting firms may indeed be the major players of 2020.

3. But, I think more likely scenarios are that:

3a. New entrants from analytics firms, consulting firms (McKinsey, BCG, etc.), MROCs, neuromarketing and social media agencies transform the business and marginalize traditional players.

3b. Google, Facebook or other large online players become the sample provider of choice and then develop their own DIY and custom research offerings.

3c. The industry undergoes radical “free-agentism” with large numbers of current employees taking advantage of their skill sets and low barriers to entry and going out on their own. With each one touting their unique sector or research expertise, client side buyers begin employing a short term army of Davids via a MR version of the mechanical turk.

Ideally, it should be read in tandem with his January Quirk’s piece titled “Ten keys to increasing the impact of research” and with his comments on this website.

Although I highly recommend that you read (and re-read) Ian’s piece in Research World and draw your own conclusions, there are several things that I think are worth amplifying.

1. His concept of a “management contract” for MR is new to me and seems challenging, but novel.
2. Ian’s emphasis on “future-focused knowledge” that is “above the brand” makes two points simultaneously. First, that MR needs to inhabit a more strategic space with fewer “validation” surveys and second, that MR will naturally over time begin to blend with strategic foresight.
3. His thinking around future leadership in MR is dead on. Until recently mastery of MR meant mastery of a process that yielded reltaively scarce data. Data is now abundant. The process is being automated and simplified. Creativity is the scarce item, not data.
4. His listing of mindsets and skills is useful for anyone in MR tasked with hiring and managing. It’s an excellent shortlist.

“High Impact Research” is an exellent piece, and I highly recommend it.

I’m inclined to believe that major innovation will not come from the large players and will come mostly from outside what we think of today as “the market research industry.”

As Jack Honomichl noted in his 2008 report on the industry:

“Of special note regarding list turnover is the growth of startups, many in recent years rooted in online data collection. They demonstrate how the US research industry revitalizes itself via innovation…Then when a newer firm grows to an appetizing size, it usually goes public or is acquired by a larger, well-established research conglomerate. So the industry evolves, picks up energy and adapts to technological changes.”

This suggests to me that the larger players will struggle to innovate, holding on to their size through their control of large, institutional tracking programs and strategic purchases. In his 2009 report, Honomichl referred to this as “their long-time strategy of buying revenue growth, even in a slumping market.”

But why is this the case?

Perhaps the best answer I have found is in Richard Watson’s intriguing book, “Future Files.” In this book Watson states that:

“According to McKinsey only 0.5% of all companies perform well over several decades, so there is every reason to believe that the majority of companies around today won’t exist in the future. The primary reason seems to be the need for them to perform two seemingly contradictory tasks to survive. First, they must execute flawlessly in the present. This requires strict control and tight hierarchies that reward individuals with extensive skills and experience. However, this experience and expertise can create barriers that prevent an organization from adapting to changed circumstances in the future. Organizations are disabled by their own experience and success. In addition, senior executives develop mental models about what is and what works based on historical experience. Moreover, successful organizations tend to evolve into large networks that become gridlocked; innovation and change are resisted because they inevitably have a negative effect on someone, somewhere. This in-built corporate immune system partly explains why most radical innovations don’t come from industry incumbents…”

This seems to describe the industry fairly well.

But where is all this innovation going to come from?

The ready answer seems to be that much of it will come from outside the traditional industry leaders. For a more granular look, see my interview with Simon Chadwick.

For those interested in prediction markets I strongly recommend David Rothschild’s article in this month’s Public Opinion Quarterly (AAPOR’s publication).

Although Rothschild’s article focuses on “comparing prediction markets, polls and their biases”, the implications of his article stretch well beyond the political world. In fact, from one vantage point, elections are just a useful space to study these markets since they happen at frequent and predictable intervals. But any future event with these similar parameters would do.

One critique of market research is that it is a snapshot of the present and is rooted in present attitudes and behaviors.

This is why I believe strategic foresight and market research will merge (or at least cross-pollinate) in the near future.

Prediction markets are just one example of an area in which this cross-pollination will occur.

BrainJuicer already appears to be experimenting with prediction markets.

His colleague’s frustrations:

1. Too much email communication rather than face to face.
2. Increased speed of communication and deadline expectations.
3. Non-traditional competition from “non-researchers purporting to do or know how to do MR”.

The first is a general societal frustration.

I’ll address the 2nd and 3rd.

Speed:
Like it or not, expectations for more rapid turnaround time are growing. This is simply a function of the business environment and will only increase. But, I sense that there are somewhat different time expectations for work that is tactical (and in which the MR team is brought in late, usually for cover) and work that is of a more strategic nature. In my experience, the more strategic the work, the greater the understanding that time to contemplate is needed. Garbage in, garbage out. This is an advantage that the consulting community (McKinsey, BCG) have over traditional market research players. They may wind up using the same research tools, but they are brought in at a higher level and have the luxury of a greater time horizon for project completion.

I also strongly believe that one future of MR is the globally distributed research team. These teams will be able to leverage the 24 hour global clock in ways that could dramatically accelerate our delivery times. Right now most MR is regionally based with some offshoring of functions. The real change will come when the analytical and strategic teams are globally distributed.

Non Traditional Competition:
Get ready. Most of the non-traditional competitors will be in qualitative methods like communities, etc.

“Are the Kids Alright?”

What are they thinking about the future of market research?

Why aren’t we hearing from them?

My take:

I don’t think we’re asking their opinion and I’m not sure the industry is listening.

But, I would add another option somewhat related to “they’re too busy doing amazing stuff.”

“They’re in industries, businesses and functions that are not classified as ‘market research’ now, but will replace traditional ‘market research’ in a few years.”

Theoretical examples:

1. A MashUp whiz who is a baseball stats maven, has a boring job in accounting now, but will create a killer DIY research tool in 2012.

2. A gifted PHD candidate in anthropology that studies how women relate to food in their kitchens. She’ll go on in 2013 to build a stunning new “deep qual” firm.

3. A 25 year old Google math whiz that has never taken a survey course in her life, but will be tasked with architecting Google’s survey sampling methodology and DIY survey platform next year.

4. The 27 year old Boston Consulting Group wunderkind that is a virtuoso at synthesizing multiple data sets in his consulting gigs, but would never identify himself as a “market researcher” or in “market research”.

5. The 26 year old online gaming programmer that loves building simple simulation games for his buddies, but in 2013 will have the hottest new research interface on the planet. His company (Simusurveys) won’t ask respondent questions at all. Instead, it will have them play fun simulations. In fact, many people will play these surveys willingly as entertainment knowing that they are actually part of niche research efforts.

6. A 22 year old self-described “fashionista” in Miami that will create a global, invitation only, communnity for only the most cutting edge hipsters. When a high fashion corporation realizes she is sitting on a gold mine, they’ll buy her community for top dollar and hire her to maintain it. She will become their lead trends expert among the trendiest trendsetters. Overnight she’ll be a market research executive, but she’ll never use the term.

I could go on, but I think this is what part of the transformation will look like.