Jump to the text version

Sometimes data challenges us. Analysis often reveals surprising, and sometimes not so surprising, patterns. As we dig deeper and learn more, the findings start to make sense.

But when the data just doesn’t make sense, like that time a children’s museum survey showed the majority of respondents were men without children, we knew immediately what had happened…

They got botted.

Each year, the Annual Survey of Museum-Goers is distributed by museums mostly via their contact lists, as well as social media. Social media platforms allow museums to reach respondents that their contact lists may lack, but this can open up the potential for “botting.”

What’s a “bot”?

Unfortunately, sometimes malicious web-crawlers comb social media for online surveys, spamming them with fake responses. These fake respondents or “bots” use automated tools to complete surveys many, many times and often very quickly.

Why spam a museum survey?

Honestly, we’re not entirely sure. A few years ago a large percentage of bots appeared to be coming from Russia, In that case, we assumed it was intended to indiscriminately mess with any U.S. based survey they could find.

Now, the raw data says most bots are coming from within the United States. That could be the case, or it could also be evidence of more sophisticated tactics of the people behind the bots.

We have found that the small prize drawing incentive offered for completing the survey attracts bots, just like bees to honey. And this is what web-crawlers are hoping to benefit from when they spam a survey on a large scale. Essentially, they’re trying to stack the odds in their favor!

Either way, bots are more than a nuisance…and they can skew your data!

So how do we know which responses are legit or not?

Here are some of the red flags we look for when we suspect bots have infiltrated a survey:

• A cluster of surveys that were started and/or completed at the same time or within seconds of each other
• A large number of responses from the same IP address
• A larger than normal number of responses from out of state and/or country
• Email addresses that consist of jumbled letters and numbers or have names/numbers in a pattern
• Inconsistent and/or contradictory answers to questions
• Ratio of completed surveys and partially completed surveys is out of normal range (it is normal to have between 15%-20% of your surveys only partially completed)
• Repetitive, patterned, nonsensical, irrelevant, and/or less than genuine responses to open-ended questions
• Results that don’t make sense for some museum types (like our children’s museum example above!)
• Surveys that were taken very quickly, sometimes in seconds

Some of these potential red flags on their own may be normal and not necessarily a bot, which is why we conduct thorough data cleaning.

Because participation in the survey is voluntary and questions are not required, data cleaning becomes a subjective process. Therefore, we use our brains (and not computers!) to pre-sort suspicious responses and to clean data. That’s because our brains are now trained in the nuances of responses to make the final decision on the quality of each response and how many red flags is enough to identify the response as a bot.

While we do implement many automatic strategies (sorry, we’re keeping those confidential because we don’t want botters to know our secrets!) to help stop bots from attacking, our experience suggests that they will still find ways to sneak into a survey.

But rest assured, we’ll continue to diligently comb through the data to spot bots—looking for those red flags and then cleaning up the data—to make sure that collected responses are reliable and accurate!

Annual Survey of Museum-Goers Data Stories are created by Wilkening Consulting on behalf of the American Alliance of Museums. Sources include:
• 2017 – 2024 Annual Survey of Museum-Goers

*Data Stories share research about both frequent museum-goers (typically visit multiple museums each year) and the broader population (including casual, sporadic, and non-visitors to museums).

More Data Stories can be found at wilkeningconsulting.com/data-stories.

Jump to the text version

When we look at visitation gaps among the broader population, there are two big things to consider: incidence and frequency.

Incidence is a binary yes/no question: has someone been to a museum (of any type) in the past year? We measure incidence by doing broader population sampling among US adults.

Frequency is different, and looks at how often someone has been to museums in a defined time period.

When we field the Annual Survey of Museum-Goers, the respondents are overwhelmingly frequent museum-goers.

Our simplified definition of these individuals is that they are engaged enough with one (or more) museums to be on a communications list (email, social media, etc.) and then complete a survey on a museum’s behalf. We do assess their frequency, and virtually all respondents are what we would define as frequent museum-goers.

When we plot out certain demographic characteristics, we find that who visits museums frequently (and the gaps we discover) can be very different than who visits museums at all (the incidence). That is, the demographic profile of frequent museum-goers is a bit different than the profile of all museum-goers, especially casual and sporadic visitors.

Understanding these gaps (and where gaps don’t exist) is important, because it gives us a much more nuanced understanding of who museums are reaching, and how deeply.

There are three demographic factors that we want to examine more completely.

Educational Attainment

Among frequent museum-goers, 83% of respondents have a college degree; that’s 2.3x the rate of college degrees among US adults (36%).

But when we look at incidence among the broader population, we see a much smaller gap. Among all casual and sporadic museum goers, 51% have a college degree, and 49% do not.

That does suggest that people with college degrees are more likely to visit museums than those without college degrees. The chart below shows this to be true.

Percent of each population segment who visited a museum:

College degree 51%

Some college/technical school/ associate degree 31%

High school diploma/GED or less 18%

Race and Ethnicity

Among frequent museum-goers, 84% say that they (or a member of their household) identify as white. In contrast, 59% of people living in the United States identify as white (not Hispanic or Latine).

Again, it would be easy to make a big assumption here. But the accurate conclusion from this is that yes, frequent museum-goers are significantly more likely to identify as white than the broader population.

Incidence tells a very different story.

Since 2023, African American and Hispanic or Latine incidence has held steady, while incidence has increased slightly among white people and Asian or Asian Americans. In fact, for the third year in a row, Asian or Asian Americans have been the most likely racial or ethnic group to have visited a museum.

Percent of each population segment who visited a museum:

White (not Hispanic or Latine) 35%
African American or Black 28%
Hispanic or Latine 31%
Asian or Asian American 46%

(Responses by other racial and ethnic groups were too small to be stable.)

Additionally, and this has been true for the past several years, white people are the most likely to say they “never” visit museums, while people of color are more likely to say they visit museums at least occasionally.

Surprised? We were too when we first saw a closing, and then erasure of the incidence gap a few years ago, and we wondered if it was a fluke. But three years of consistent data? This is a pattern.

A victory lap? Not so fast!

Incidence gaps do still persist by museum type and for specific museums.

By museum types, historic sites in particular have persistently high incidence and frequency gaps, with audiences still, overall, skewing disproportionately white.

Additionally, this still varies widely for individual museums. Your museum may still be facing an incidence gap, and only research specific to your museum can help you understand where your gaps are (or are not).

The frequency gap we see by race and ethnicity is incredibly persistent, but we have some good news here as well. Since 2017, it has been decreasing, albeit slowly.

Frequent Museum-Goers from the Annual Survey of Museum-Goers:

2017: 92%
2018: 87%
2019: 88%
2020: 88%
2021: 87%
2022: 84%
2023: 84%
2024: 84%

(From 2017 to 2022, the white [not Hispanic or Latine] population of the US decreased from 62% to 58%)

Age and Life Stage

Some museum types deliberately focus on families with minor children (children’s museums and, to a lesser extent, zoos, aquaria, and science centers), and these museums don’t tend to see frequency gaps (or incidence gaps) by age that are of significant concern…though they often see frequency gaps with young adults without children.

But it is a different story for art museums, history organizations, and botanical gardens. It is incredibly common for the majority of their frequent museum-goers to be over the age of 60.

This, understandably, can cause a great deal of concern for people working in these types of museums. And that’s why looking at incidence is so important.

It turns out, adults over 60 are the least likely segment of the population to visit museums (a pattern we have seen pretty consistently over the past several years). We actually underserve them.

Percent of each population segment who visited a museum:

Young adults (under 40, no children) 34%
Parents/guardians of minor children 40%
40 – 59, no minor children 31%
60 or older 27%

Additionally, those casual and sporadic young adult visitors visit most museum types, lagging behind the overall average only for children’s museums and zoos/aquaria (largely because, well, they don’t have children; this shifts when some of them start families, of course!).

But overall, at this time, we don’t have significant concerns about the long-term pipeline of future visitors to museums of any type.

Annual Survey of Museum-Goers Data Stories are created by Wilkening Consulting on behalf of the American Alliance of Museums. Sources include:
• 2024 Annual Survey of Museum-Goers, n = 90,178; 202 museums participating
• 2024 Broader Population Sampling, n = 2,154
• 2017 – 2023 Annual Surveys of Museum-Goers

U.S. demographic data from the U.S. Census Bureau.

*Data Stories share research about both frequent museum-goers (typically visit multiple museums each year) and the broader population (including casual, sporadic, and non-visitors to museums). See the Purpose and Methodology (Update) Data Story from September 5, 2024 for more information on methodology.

More Data Stories can be found at wilkeningconsulting.com/data-stories.