If you follow generation Z news closely, you might have noticed a sudden increase in headlines that projects Generation Z as being more “socially conservative” than older generations. The finding, which claims that Generation Z are less open to same-sex marriage, gender fluidity and body adornments such as tattoos go back to a survey (actually a quiz) carried out by a British marketing company called The Gild. As many as 59% of respondents born after 2001 reported to belong to the ‘conservative’ or the ‘moderate’ categories against only 15 to 16 percent of Millennials and Generation X.
Having followed generational attitudes for almost a decade and a half myself, the findings fly in the face of most surveys and observations I have seen about Generation Z’s open-mindedness. I was curious and decided to find out how they got these results.
As soon as I landed on The Gild’s website I was invited to partake in the very same survey. No filter question to make sure I met sampling criteria or any other requirements were needed to participate. The Gild prepped my motivation by telling me: “Big Business and Governments are stereotyping you into one of six generational groups” and, “At The Gild, we think that’s wrong and needs to stop, so we’ve designed this Generation Quiz to prove or disprove our theory, and we need your help.” So as the freedom-loving, justice-seeking arm-chair activist I am, I was ready! I’d show those Big Businesses and Governments they can’t pigeonhole me or create mind-controlling Newspeak based on my generation or my preferences!
First I was asked to compare statements and pick the one from each category I agreed the most with. Since the statements were double-barreled and categories were overlapping, I had a hard time finding a statement that fit me. None of the 5 answer alternatives managed to capture my preferences and idiosyncrasies. But the biggest puzzle to me was: How do they get teenagers with 8-second attention spans to read through questions with five long winded answer alternatives like: “My main focus is on being fit and healthy, enjoying life and spending quality time with my family.”? Having designed a few nation-wide surveys aimed at this age cohort myself, I’ve learned to keep the butcher knife close, chopping up and eliminating even the tiniest part of bad meat from my sentences to accommodate this easily distracted target group.
As for the results? Of course my answers supported their claim that generational research is all bovine feces! I came out as a textbook Generation Z despite the fact that I am born right smack in the middle of Generation X. I guess I’m work-damaged from spending so much time trying to figure these kids out that I’ve become the pathetic mid-lifer who is reliving her own adolescence to the inevitable embarrassment of her own children.
But the funniest part was, when the survey was over I could just refresh the page and do it all over again! No IP address detector to block me from taking it twice. Even more, I was encouraged to share it over my social networks, ensuring that my echo-chamber could eliminate any tiny element of random selection mechanism that might have otherwise existed.
Garbage in, garbage out
They say you can lie with statistics, but I’d rather say, “Garbage in, garbage out”. Here’s why self-respecting media outlet wary of being associated with fake news might want to have a second look before they label this as “research”:
- If you make biased claims in the introduction, you will get biased answers. Any chance a typical respondent would want to prove the researchers right after their quite pointed intro? If you could stop the “Evil Empire” in their tracks by giving them slanted information, wouldn’t you? Can you envision a bunch of 12-year old pranksters who decided to use this survey to stir the fake news pot? I mean, especially since they were encouraged to go against the generational stereotypes in the introduction. Let’s toss objectivity out the window.
- Open surveys on the internet are not representative. Ensuring representative samples from a population is a professional skill. Only the people that landed on The Gild’s website or were recruited by their like-minded brethren were represented, meaning there are dark, black holes of skewness involved. If you can take the survey multiple times and are encouraged to share the quiz with your friends in social media? Well, now also representativeness gets tossed out the window.
- Double-barreled questions and overlapping answer categories. The issue of double-barreled questions refers to the fallacy of including more than one argument in the same statement. In other words, you have to agree to all the suggestions in the sentence, which will yield inaccurate answers. So to answer “How would you generally describe your views on issues such as same-sex marriage, transgender rights and marijuana legalisation?” you’d better agree or disagree on all accounts! Given that so many of the answer alternatives were overlapping, this problem could probably have been avoided by making sure all statements were mutually exclusive. So let survey design join the two aforementioned bullet points and get a good taste of the hard concrete outside the window!
And while we’re about it, let me repeat a premise I always mention to my clients and audiences: A single survey is never enough to assess a generational mindset. For one thing, there is no way to control cohort effects from age effects – a distinction you might remember from Introduction to Sociology classes. Because to discern whether an age difference is due to age or generation you have to compare longitudinal data. There are many other reasons, which I won’t mention here.
I can’t help being reminded of those internet quizzes that tell you if your subconscious personality matches that of a golden retriever or a trapeze artist, and so I doubt if this particular survey was ever meant to be taken seriously. Andrew Mulholland, managing director at The Gild seems to make the argument that generational surveys in general are bogus, so it is hard to imagine they put in a lot of time with the survey design or took it very seriously. But seriously is how it has been received, even by the “non-fake” media. The Times, The Telegraph and The Daily News are just some of print news media that buy into this new “fact” that Generation Z is more conservative. Who knows? Maybe they are, but this internet quiz doesn’t prove it.
Generational differences do exist as an independent variable and is at least partially responsible for differences between people. And this is important because of the sheer volume of people who are going through the same life changes at the same time. But to get trustworthy answers you have to consider your sources. This is why there are consultants who have spent more than the famous 10,000 hours to become experts in this field. It is THAT complex, and a simple internet quiz cannot change it.