The Times Australia
The Times World News

.
The Times Real Estate

.

Studying social media can give us insight into human behaviour. It can also give us nonsense

  • Written by Jason Burton, PhD researcher, Birkbeck, University of London

Since the early days of social media, there has been excitement[1] about how data traces left behind by users can be exploited for the study of human behaviour. Nowadays, reseachers who were once restricted to surveys or experiments in laboratory settings have access to huge amounts of “real-world” data from social media.

The research opportunities enabled by social media data are undeniable. However, researchers often analyse this data with tools that were not designed to manage the kind of large, noisy observational sets of data you find on social media.

We explored problems that researchers might encounter due to this mismatch between data and methods.

What we found[2] is that the methods and statistics commonly used to provide evidence for seemingly significant scientific findings can also seem to support nonsensical claims.

Absurd science

The motivation for our paper comes from a series of research studies that deliberately present absurd scientific results.

One brain imaging study[3] appeared to show the neural activity of a dead salmon tasked with identifying emotions in photos. An analysis of longitudinal statistics from public health records[4] suggested that acne, height, and headaches are contagious. And an analysis of human decision-making[5] seemingly indicated people can accurately judge the population size of different cities by ranking them in alphabetical order.

Read more: One reason so many scientific studies may be wrong[6]

Why would a researcher go out of their way to explore such ridiculous ideas? The value of these studies is not in presenting a new substantive finding. No serious researcher would argue, for example, that a dead salmon has a perspective on emotions in photos.

Rather, the nonsensical results highlight problems with the methods used to achieve them. Our research explores whether the same problems can afflict studies that use data from social media. And we discovered that indeed they do.

Positive and negative results

When a researcher seeks to address a research question, the method they use should be able to do two things:

  • reveal an effect, when there is indeed a meaningful effect

  • show no effect, when there is no meaningful effect.

For example, imagine you have chronic back pain and you take a medical test to find its cause. The test identifies a misaligned disc in your spine. This finding might be important and inform a treatment plan.

However, if you then discover the same test identifies this misaligned disc in a large proportion of the population who do not have chronic back pain, the finding becomes far less informative for you.

Studying social media can give us insight into human behaviour. It can also give us nonsense Like a spinal test that can’t tell the difference between people with back pain and people without, much social media research isn’t using the right tools for the job. Shutterstock

The fact the test fails to identify a relevant, distinguishing feature of negative cases (no back pain) from positive cases (back pain) does not mean the misaligned disc in your spine is non-existent. This part of the finding is as “real” as any finding. Yet the failure means the result is not useful: “evidence” that is as likely to be found when there is a meaningful effect (in this case, back pain) as when there is none is simply not diagnostic, and, as result, such evidence is uninformative.

XYZ contagion

Using the same rationale, we evaluated commonly used methods for analysing social media data — called “null hypothesis significance testing” and “correlational statistics” — by asking an absurd research question.

Past and current studies have tried to identify what factors influence Twitter users’ decisions to retweet other tweets. This is interesting both as a window into human thought and because resharing posts is a key mechanism by which messages are amplified or spread on social media.

So we decided to analyse Twitter data using the above standard methods to see whether a nonsensical effect we call “XYZ contagion” influences retweets. Specifically, we asked

Does the number of Xs, Ys, and Zs in a tweet increase the probability of it being spread?

Upon analysing six datasets containing hundreds of thousands of tweets, the “answer” we found was yes. For example, in a dataset of 172,697 tweets about COVID-19, the presence of an X, Y, or Z in a tweet appeared to increase the message’s reach by a factor of 8%.

Needless to say, we do not believe the presence of Xs, Ys, and Zs is a central factor in whether people choose to retweet a message on Twitter.

However, like the medical test for diagnosing back pain, our finding shows that sometimes, methods for social media data analysis can “reveal” effects where there should be none. This raises questions about how meaningful and informative results obtained by applying current social science methods to social media data really are.

As researchers continue to analyse social media data and identify factors that shape the evolution of public opinion, hijack our attention, or otherwise explain our behaviour, we should think critically about the methods underlying such findings and reconsider what we can learn from them.

What is a ‘meaningful’ finding?

The issues raised in our paper are not new, and there are indeed many research practices that have been developed to ensure results are meaningful and robust.

For example, researchers are encouraged to pre-register their hypotheses and analysis plans before starting a study to prevent a kind of data cherry-picking called “p-hacking”[7]. Another helpful practice is to check whether results are stable after removing outliers and controlling for covariates[8]. Also important are replication studies[9], which assess whether the results obtained in an experiment can be found again when the experiment is repeated under similar conditions.

These practices are important, but they alone are not sufficient to deal with the problem we identify. While developing standardised research practices is needed, the research community must first think critically about what makes a finding in social media data meaningful.

Read more: Predicting research results can mean better science and better advice[10]

References

  1. ^ excitement (www.nature.com)
  2. ^ found (www.nature.com)
  3. ^ One brain imaging study (www.psychology.mcmaster.ca)
  4. ^ analysis of longitudinal statistics from public health records (www.bmj.com)
  5. ^ analysis of human decision-making (link.springer.com)
  6. ^ One reason so many scientific studies may be wrong (theconversation.com)
  7. ^ “p-hacking” (www.wired.com)
  8. ^ covariates (journals.sagepub.com)
  9. ^ replication studies (www.sciencedirect.com)
  10. ^ Predicting research results can mean better science and better advice (theconversation.com)

Read more https://theconversation.com/studying-social-media-can-give-us-insight-into-human-behaviour-it-can-also-give-us-nonsense-163000

The Times Features

Itinerary to Maximize Your Two-Week Adventure in Vietnam and Cambodia

Two weeks may not seem like much, but it’s just the right time for travelers to explore the best of Vietnam and Cambodia. From the bustling streets of Hanoi to the magnificent te...

How to Protect Your Garden Trees from Wind Damage in Australia

In Australia's expansive landscape, garden trees hold noteworthy significance. They not only enhance the aesthetic appeal of our homes but also play an integral role in the local...

Brisbane Homeowners Warned: Non-Compliant Flexible Hoses Pose High Flood Risk

As a homeowner in Brisbane, when you think of the potential for flood damage to your home, you probably think of weather events. But you should know that there may be a tickin...

Argan Oil-Infused Moroccanoil Shampoo: Nourish and Revitalize Your Hair

Are you ready to transform your hair from dull and lifeless to vibrant and full of life? Look no further than the luxurious embrace of Argan Oil-Infused Moroccanoil Shampoo! In a...

Building A Strong Foundation For Any Structure

Building a home or commercial building can be very exciting. The possibilities are endless and the future is interesting. You can always change aspects of the building to meet the ...

The Role of a Family Dentist: Why Every Household Needs One

source A family dentist isn’t like your regular dentist who may specialise in a particular age group and whom you visit only when something goes wrong. A family dentist takes proa...

Times Magazine

"Eternal Nurture" by Cara Barilla: A Timeless Collection of Wisdom and Healing

Renowned Sydney-born author and educator Cara Barilla has released her latest book, Eternal Nurture, a profound collection of inspirational quotes designed to support mindfulness, emotional healing, and personal growth. With a deep commitment to ...

How AI-Driven SEO Enhancements Can Improve Headless CMS Content Visibility

Whereas SEO (search engine optimization) is critical in the digital landscape for making connections to content, much of it is still done manually keyword research, metatags, final tweaks at publication requiring a human element that takes extensiv...

Crypto Expert John Fenga Reveals How Blockchain is Revolutionising Charity

One of the most persistent challenges in the charity sector is trust. Donors often wonder whether their contributions are being used effectively or if overhead costs consume a significant portion. Traditional fundraising methods can be opaque, with...

Navigating Parenting Arrangements in Australia: A Legal Guide for Parents

Understanding Parenting Arrangements in Australia. Child custody disputes are often one of the most emotionally charged aspects of separation or divorce. Parents naturally want what is best for their children, but the legal process of determining ...

Blocky Adventures: A Minecraft Movie Celebration for Your Wrist

The Minecraft movie is almost here—and it’s time to get excited! With the film set to hit theaters on April 4, 2025, fans have a brand-new reason to celebrate. To honor the upcoming blockbuster, watchfaces.co has released a special Minecraft-inspir...

The Ultimate Guide to Apple Watch Faces & Trending Wallpapers

In today’s digital world, personalization is everything. Your smartwatch isn’t just a timepiece—it’s an extension of your style. Thanks to innovative third-party developers, customizing your Apple Watch has reached new heights with stunning designs...

LayBy Shopping