The Times Australia

The Times World News
The Times

8 surprising things data science has revealed about us over the past decade

  • Written by Paul X. McCarthy, Adjunct Professor, UNSW Sydney
8 surprising things data science has revealed about us over the past decade

Big data analysis has long supported major feats[1] in physics and astronomy. But more recently we’ve seen it underpin breakthroughs in the social sciences and humanities.

Since the landmark paper Computational Social Science[2] was published in 2009, a new generation of data analytics tools has given researchers insight into fundamental questions about how we communicate, who we are and what we value.

For instance, by analysing the relative frequency of certain words in historical texts, researchers can identify important changes in our use of language over time.

In some cases these shifts will be obvious, such as the use of archaic words being replaced by more contemporary words. But in other cases, they may reflect more subtle but widespread social and cultural changes. Below are some of the most influential data-centric discoveries from the past 10 years.

How we communicate

Over the past decade, a growing number of global open data sources have helped researchers reveal patterns in what we read, write and pay attention to. Google Books, Worldcat[3] and Project Gutenberg[4] are just some examples.

The release of the Google Books n-gram viewer[5] in the early 2010s was a game changer on this front. Using the entire Google Books database, this tool shows you the relative frequency of a specific term or phrase as it has been used over hundreds of years. Researchers[6] have used this data to explore the systematic suppression of the mention of Jewish painters, such as Marc Chagall, in German books during World War II.

Data analysis can also reveal patterns in the expression of human emotions over time. CSIRO’s We Feel[7] tracks emotions in communities around the world. It does this by analysing the language people are using on social media in real time and mapping it out.

The tool can be used to determine the general mood over time (hour by hour, day by day) within particular cities and countries. Patterns in these data can then be explored in association with other information, such as weather, holidays and economic fluctuations.

Some research findings even claim to represent fundamental changes in humans’ social values, community sentiment and how we think (for example, the rise and fall of words associated with rationality such as “method”, “analysis” and “determine”).

Here are some key findings in this space:

  • Cultural turnover is accelerating

    A Harvard University-led analysis[8] of more than a century of data from millions of books provides evidence that society’s attention span for historical events is declining, as appetite for new material grows.

    In other words, we are forgetting the past faster. You can see this in the graph below, which tracks how often three specific years are mentioned across a vast range of literature through time. As time passes, the “half-life” of each year (the point at which it receives just half the attention it had at its peak) comes quicker.

    Counts of mentions of the years 1883, 1910 and 1950 in all books for the past 200 years.
    Our collective attention for historical events has shrunk over the past century. Michel et al., Science 2010[9]
  • Human language diversity and biodiversity are correlated

    By mapping linguistic diversity and the diversity of animal species, researchers have shown[10] these two worlds are correlated geographically – both increasing with temperature and proximity to the equator. So the closer to the equator you get, the more variation there is in spoken language and the greater the variety of species there is.

    The authors propose this is due to heat near the equator producing greater productivity and variety in plant life, which in turn provides more complex and interactive environments for both animals and humans alike – feeding into a cycle whereby “diversity begets more diversity”.

    Three figures showing diversity distributions of language and animals and their relation to geography.
Researchers have shown both linguistic diversity and species diversity increase exponentially with temperature and proximity to the equator. Hamilton, Walker & Kempes, Scientific Reports 2020[11] There have been society-wide shifts in language use over the past century

In an article published[12] in December researchers used machine learning to show long-term, consistent changes in our use of language. Specifically, they reveal an inflection point in the 1980s where there is a shift towards more egocentric, emotional and supposedly less rational language.

The authors suggest (although not without contest[13]) this could signal the beginning of a “post-truth era”.

Who we are

In the field of psychology, the same data analytics tools have shown that people’s personalities can be measured using the “Big 5” traits, which largely become stable in adulthood[14].

This was possible thanks to extensive data sets such as HILDA in Australia, the German Socio-Economic Panel in Germany and the British Household Panel Survey in the UK.

Robust studies have also demonstrated that personality traits can be reliably and accurately predicted from a variety of data sources including voice recordings[15], mobile phone usage patterns[16] and even portrait photographs[17].

In turn, there have been some remarkable associations found at scale between personality and:

  • Elevation

    A study published in 2020, and based on more than three million people’s data, shows[18] mountain-dwelling people tend to have different personality traits than those who live at sea level. They are generally more open to new experiences and more emotionally stable.

  • Location

    Another earlier study shows people who live in the United States can be divided into three clear and measurable clusters[19] of personality types, linked with associated geographic footprints. New Yorkers and Texans (who are in the same cluster) are more likely to be temperamental and uninhibited.

  • Occupation

    In our own research published with colleagues in 2019, we analysed the personality features of people in more than 1,000 different occupations. We found[20] people in the same role share similar traits. Scientists are more open to new ideas yet ready to argue[21], whereas tennis professionals tend to be friendly and outgoing.

    The research used machine learning to infer the personality features of more than 100,000 people, based on language used on social media.

Read more: Robot career advisor: AI may soon be able to analyse your tweets to match you to a job[22]

What we value

In economics, we’re seeing major research frontiers being opened up thanks to data analysis, including in:

  • Network science

    When it comes to success, we’ve learnt that performance matters most when it can be measured (like in sport). But in other fields where it can’t be measured easily (like in the art world), networks matter[23] most[24].

  • Behavioural economics

    We can now see how we behave as individuals en masse, unveiling valuable clues for effective policy interventions around employment, taxation and education. For instance, one large-scale study[25] revealed those quickest to re-enter the workforce displayed certain key behaviours. These included being an early riser and being geographically mobile (perhaps meaning they’re more willing to travel further, or relocate, for work).

Post-theory science?

Some have argued data science poses a fundamental challenge to the traditional sciences, with the emergence of “post-theory science[26]”. This is the concept that machines are better at understanding the relationship between data and reality than the traditional scientific method of hypothesise, predict and test.

However, reports of the death of theory[27] are perhaps greatly exaggerated. Data are not perfect. And data science based on incomplete or biased data has the potential to miss, or mask, important patterns in human activity. This can only be addressed by critical thinking and theory.

Read more: Nobel economics prize winners showed economists how to turn the real world into their laboratory[28]


  1. ^ major feats (
  2. ^ Computational Social Science (
  3. ^ Worldcat (
  4. ^ Project Gutenberg (
  5. ^ n-gram viewer (
  6. ^ Researchers (
  7. ^ We Feel (
  8. ^ analysis (
  9. ^ Michel et al., Science 2010 (
  10. ^ shown (
  11. ^ Hamilton, Walker & Kempes, Scientific Reports 2020 (
  12. ^ published (
  13. ^ without contest (
  14. ^ stable in adulthood (
  15. ^ voice recordings (
  16. ^ mobile phone usage patterns (
  17. ^ portrait photographs (
  18. ^ shows (
  19. ^ three clear and measurable clusters (
  20. ^ We found (
  21. ^ ready to argue (
  22. ^ Robot career advisor: AI may soon be able to analyse your tweets to match you to a job (
  23. ^ matter (
  24. ^ most (
  25. ^ large-scale study (
  26. ^ post-theory science (
  27. ^ death of theory (
  28. ^ Nobel economics prize winners showed economists how to turn the real world into their laboratory (

Read more

Times Lifestyle

Choosing the Perfect Parka: Expert Tips for Beating the Freeze in Style

As the cold weather approaches, the right parka can protect you from the cold while keeping you stylish. Like the choice of a custom letterman jacket, the choice of a custom parka is based on both the aesthetic and practical considerations. In this a...

Wooden Name Trains: A Personalized Gift for Modern Kids

Are you looking for a personalised and at the same time an educational gift for the toddler in your life? Then a wooden name train should be on your shopping list! This unique toy is not only fun to play with but comes with an educational element...

Choosing A Commercial Bar Fridge: Essential Tips For Restaurant Owners 

Your restaurant's bar is the heart of your establishment. It's where customers gather, socialize, and enjoy your carefully crafted drinks. Your commercial bar fridge ensures your beverages are perfectly chilled, ready to delight your guests. A poo...

Times Magazine

Uniden Adds Three New Baby Monitors to Award-Winning BabyWatch Range

Uniden has introduced three new models to its award-winning BabyWatch baby monitor range, offering parents a variety of high-tech features at an affordable price point, to keep an eye on newborns and toddlers from anywhere around the home. The th...

Take The Plunge, Elevate Your Personal Health: P3 Recovery Opens In Port Melbourne

World leaders in wet and dry therapy make wellbeing even more accessible for Melbournians  Ice baths, infrared saunas, IV therapy, breathwork. Just some of the latest wellness therapies that happen to be housed inside P3 Recovery centres emergin...

Prestons ranked Australia’s worst suburb for parcel theft

Shocking new data reveals that parcel theft claims have more than doubled this year, with Prestons in New South Wales named the worst suburb. This year there’s been a 59% increase in claims for parcel loss with a wider range of people lodging ...

Business Marketing