Bootstrap

RIP Australian Twittersphere

Published on Monday, 17 July 2023 by QUT Digital Observatory

As Twitter began restricting free research access to their data, our collectors for the Australian Twittersphere were officially cut off on 13 July 2023.

In this eulogy for the Australian Twittersphere, we will trace back its evolution over the years, the contributions it has made to the research community, and what comes next for researchers and the Digital Observatory.

About the Australian Twittersphere

The Australian Twittersphere is a longitudinal collection of tweets from a periodically updated list of Twitter accounts that are identified as Australian (i.e., have a stated connection to Australia in the free text fields of the account profile).

Since the Twittersphere's establishment six years ago, over 1.9 billion tweets have been collected from more than 1 million Twitter accounts considered to be Australian. The earliest tweet captured was posted on 9 January 2007.

Historical background

The Australian Twittersphere began as the TrISMA project, during which tweets were collected from roughly 530,000 Twitter accounts deemed as "Australian". There was a gap in tweet collection between May 2017 and early 2018 which coincided with the TrISMA project ending and the QUT Digital Observatory being established.

It was at this time that we (the QUT Digital Observatory) took over the tweet collection and made improvements to secure it as a stable longitudinal collection. Subsequently, we built another collector and developed a method to identify more Australian accounts. The method allowed us to periodically update the list of accounts and collect more tweets from them. Updates were completed in late 2020 and 2021, resulting in twice the number of accounts compared to the original list.

More details on this can be found in the Australian Twittersphere factsheet.

Research impacts

The Australian Twittersphere was our first major resource as a research infrastructure facility. It provided historial, Australia-specific data to researchers at a time when it was not possible to geo-locate Twitter data in significant volumes (which is still the case: roughly 1% of tweets include precise geolocation information). At the time, so much of our work was related to the Australian Twittersphere that the Digital Observatory became synonymous with it.

The Australian Twittersphere has helped inform impactful research into education, health, business, law, and the humanities and social sciences. A non-exhaustive list of publications arising from this databank is listed below.

What's next for researchers

Twitter/X is only one of many cases where the gates to research-friendly, free-to-access APIs are gradually closing. Other platforms have also begun to gatekeep their data by charging for API access.

For example, researchers working with Twitter data are now having to bear the extra costs of the API. It's not cheap: a reasonable amount of 1 million tweets costs no less than $5,000 USD per month. These costs mean that further collection for the Australian Twittersphere is no longer feasible, but we will continue to make the existing data available to researchers. There are also new limitations on the Twitter/X platform that researchers should be aware of. This article gives a good overview of how to navigate the new Twitter/X.

The good news is that not all doors are closed. Platforms such as YouTube and Tiktok still provide free API access for researchers (albeit with some limitations). Indeed, the Digital Observatory has been working on different platforms and alternatives for researchers for several years. Our youte tool is a fantastic example of this work. Researchers can also leverage other maturing methods of collecting Web data, such as web archiving and data donation. The Digital Observatory is working on resources for web archiving, so watch this space!

The transition away from Twitter will be uncomfortable for some, but it is important to remember that other sources of data still exist on the Web.

Tell us your challenges at our Research Data Roundtable

For us to navigate the path forward and work out how best to support researchers in this "post-API" world, we would appreciate your feedback and input. Join us at our Research Data Roundtable and share your research problems, the data you need for your research, and what data-related challenges you are encountering. Express your interest here via this form: https://bit.ly/research-data-roundtable.

Publications using the Australian Twittersphere

Publications that have used the Australian Twittersphere include: