The 100-year history of market research – 1920 to 2020

The arrival of the new decade marks 100 years in the history of market research – and what a rollercoaster of a time it’s been!

It’s taken us from laborious door-to-door surveying projects that took weeks to complete through to today’s digital tools that can deliver results in hours. Your job as a marketing or insights professional today has been made unimaginably easier thanks to the pioneers who went before you.

In this article, we’re going to celebrate the vision, tenacity and dedication of those individuals and look at just how far the market research industry has come. 

History of market research timeline 

1920s – The birth of creative testing

American psychologist Daniel Starch wasn’t the first person to conduct market research, but he was the first to professionalise it by developing methodologies for market research in advertising. He invented the Starch Test, which attempted to measure the effectiveness of ads in newspapers and magazines.

This involved door-to-door canvassing to find readers of particular magazines or newspapers. These people would then be asked if they remembered any of the ads from these publications and what impression the ads had made on them. 

An effective ad, Starch wrote, has to be seen, read, believed and acted upon. And the Starch Test enabled companies to know for the first time if this had happened. They could also compare the effectiveness of their ads with rivals’.

1930s – Public polling gets underway

Founder of the American Institute of Public Opinion (later known as Gallup), George Gallup was a pioneer in scientific polling techniques and is a legend in the history of market research.

While teaching journalism at university, Gallup found that small samples of the populace could predict general attitudes. In 1932, with a presidential election approaching, he decided to see if he could use his statistical method to assess the state of politics.

Gallup’s faith in numbers allowed him to break down the US electorate into a precise set of demographic groups, which he then represented proportionally in his sample group of 3,000 people. 

In 1936, he helped to advance the public’s trust in survey research when he correctly predicted that Franklin Roosevelt would beat out Alfred Landon for the US presidency. This prediction directly contradicted The Literary Digest, the poll of record at the time. The Gallup poll soon became a household name, influencing everything from the titles of Hollywood movies to book choices for the Book of the Month Club.

1940s – Focus groups are formed

The first focus groups arose to test reactions to anti-Nazi radio broadcasts on behalf of the Office of War Information. The US government aimed to counter Nazi propaganda by explaining the war to Americans, but it also wanted to influence them. At the time, the public wasn’t eager to go to war.

The idea of the ‘focused interview’ was developed by Paul Lazarsfeld, director of The Bureau of Applied Social Research at Columbia University and his colleague, sociologist Robert Merton. Small test audiences were recruited to listen to broadcasts and press buttons that indicated ‘like’ or ‘dislike’. The results were recorded on an early computer called the Lazarsfeld-Stanton Program Analyzer. 

Afterwards, the participants were interviewed to find out why they pressed the buttons. Lazarsfeld and Merton’s methodology allowed them to “test preliminary analysis and hunches by direct observation of human beings in action.” But more importantly, it provided them with the insights of qualitative data.

Through their research, the pair discovered that the radio broadcasts were not having the intended effect on the American people, in fact, quite the opposite. It was making them less enthusiastic about going to war. From the button pressing alone they would not have understood why, but when they questioned the participants, they found people were frightened because of how bloodthirstily the Nazis had been presented.

1950s – Motivational research is invented

Austrian-born psychologist Ernest Dichter is known as the father of motivational research. He spent a lifetime studying human motivations and pioneered the application of Freudian psychoanalytic concepts to marketing.

Dichter was the first to present the idea that every product has a ‘soul’, and is bought not only for its purpose but for the values it embodies. Possessions, he said, serve as a “kind of mirror that reflects our own image.” 

As post-war consumerism thrived, Dichter’s research into consumer behaviour changed the way that brands like Chrysler, Procter & Gamble, Exxon, General Mills, and DuPont advertised products. 

Through in-depth consumer interviews, Dichter said brands could unlock the hidden desires within consumers’ minds and gain the ability to “mobilise and manipulate human needs.” Although Dichter had a Freudian-like obsession with sex, which alienated some clients, his ideas continue to influence advertising today and undoubtedly shaped the history of market research.

1960s –  A revolution in predictive statistical techniques

Conjoint analysis is what allows marketing managers to make accurate decisions about what products and services to sell. This powerful predictive statistical technique was developed by marketing professor Paul Green.

After earning a bachelor’s degree in mathematics, Green embarked on a career in marketing, working at companies including  Sun Oil, Lukens Steel and DuPont.

In 1962, Green left DuPont to work in the  Marketing Department at Wharton Business School (part of the University of Pennsylvania) and it was here he came up with the idea for conjoint analysis.

He was inspired by a research article from a mathematical psychology journal that provided a new system to measure rank order data and decided to try applying it to marketing. 

Green realised he could predict what people would do in the future based on how they answered questions about likes and dislikes. His statistical modelling technique has become a backbone of market research and earned him a Lifetime Achievement Award from the American Marketing Association.

1970s – A new model for market segmentation 

In 1974, Jerry Yoram Wind and Richard Cardozo published their seminal article ‘Industrial Market Segmentation’. It detailed a new model for categorising customers to guide strategic and tactical decision-making.

Their two-step market segmentation model, based on macro-segmentation and micro-segmentation, is one of the most common methods applied in industrial markets.

The pair defined consumer segments as “a group of present and potential customers with some common characteristic (s) which is relevant in explaining (and predicting) their response to a supplier’s marketing stimuli.” 

In other words, consumer segments should organise current and previous customers into groups while identifying the most likely candidates for future consumers. They argued that the predictive value of consumer segments should have a profound impact on the way they are marketed to. Targeting different consumer groups with different messaging has become an integral part of modern marketing.

1980s – Theories linking attitudes and behaviours are published

The theory of reasoned action (TRA) and the theory of planned behaviour (TPB) – theories pioneered by Martin Fishbein and Icek Ajzen – have formed the conceptual framework for predicting, explaining, and changing human social behaviour. 

The theory of reasoned action is used to predict how individuals will behave based on their pre-existing attitudes and behavioural intentions. Meanwhile, the theory of planned behaviour expands the theory further, to consider the role of perceived behavioural control (a person’s belief that a behaviour is under his or her control). 

According to these theories, there is a strong relationship between attitudes, intentions and behaviour, and that changes in attitudes and intentions are followed by changes in behaviour –  a pivotal finding for marketers. 

To give brands the ability to measure consumer attitudes and gain insight into the factors driving consumer behaviour, Fishbein developed a multi-attribute attitude model. The model uses three components of attitude—salient beliefs, object-attribute linkages, and evaluation—for determining a measurable score representing a consumer’s attitude.

1990s – Web analytics become a part of the mix

No history of market research would be complete without considering the impact the arrival of the World Wide Web had on the industry. The first website went live on the internet in 1991 and it soon became possible to start tracking consumer behaviour online.

The third-party cookie was launched in 1992, allowing advertisers to track web users across the internet, building up a profile of everything they interacted with. This made it possible to serve highly targeted advertising and carry out retargeting (until the death of the third-party cookie in 2024).

Later, in 1995, Analog was launched, billed as the first web log analysis software to show website owners the usage patterns on their web server. The software counted website visitors and page views and could even show where in the world website hits originated from. 

Previously, access to this type of data was restricted to computer scientists but now marketing teams could join in on the action too.

Launched in the same year as Amazon, Craigslist and eBay, Dr. Stephen Turner’s software paved the way for more advanced web analytics systems like Google Analytics, which debuted in 2005. 

2000s – The Net Promoter Score is invented

The Net Promoter Score (NPS) is a measurement commonly used by brands to quantify how satisfied their customers are. It also allows brands to benchmark their performance against competitors. 

The metric was invented by Fred Reichheld, a partner at management consulting firm Bain & Company.  Reichheld shared his system with the world in 2003 via his Harvard Business Review article, “The One Number You Need To Grow.”

To measure NPS, people are asked just one question – how likely are you to recommend this company? They answer using a scale of 1-10. Depending on what number they choose, the individual is designated either a ‘promoter’ of the company, a ‘passive’ or a ‘detractor’.

While the Net Promoter Score itself has its detractors, it is still one of the most widely-used customer experience measures, relied on by companies both big and small. 

2010s – On-demand consumer insight become a reality

The last decade has seen perhaps the most exciting development in the history of market research – the arrival of on-demand insights.

Thanks to the rise in smartphones, access to consumers has been transformed. Attest was built to capitalise on this by opening up an audience of 150+ million people in 59 markets to brands in need of consumer data.

Founded by Jeremy King (pictured, middle) in 2015, Attest quickly became a leader in its field, attracting big brand names like Deliveroo, Fever-Tree, Uber, TransferWise and Samsung.

Prior to the arrival of on-demand insights, any brand wanting to conduct a large-scale consumer survey would need to work with a market research agency. Getting results often took a lot of time and money. 

Today, marketers can get answers to their burning questions in a matter of hours, not weeks or months, meaning important decisions can be always be grounded in reliable data.

2020s – Market research gets faster and easier

A hundred years ago, consumer market research was a massive undertaking. Today, it couldn’t be easier. Thanks to the hard work of market research pioneers like those profiled above, modern solutions like Attest are able to combine all the learnings of the last 10 decades.

Market research no longer has to be a long and costly process carried out by academics. These new tools mean any person, in any brand, can ask their target customers market research questions and get actionable insights in the space of a few clicks. With so much power to know, some might say there’s never been a more exciting time to be in marketing?