Published on Monday, 17 July 2023 by QUT Digital Observatory
As Twitter began restricting free research access to their data, our collectors for the Australian Twittersphere were officially cut off on 13 July 2023.
In this eulogy for the Australian Twittersphere, we will trace back its evolution over the years, the contributions it has made to the research community, and what comes next for researchers and the Digital Observatory.
The Australian Twittersphere is a longitudinal collection of tweets from a periodically updated list of Twitter accounts that are identified as Australian (i.e., have a stated connection to Australia in the free text fields of the account profile).
Since the Twittersphere's establishment six years ago, over 1.9 billion tweets have been collected from more than 1 million Twitter accounts considered to be Australian. The earliest tweet captured was posted on 9 January 2007.
The Australian Twittersphere began as the TrISMA project, during which tweets were collected from roughly 530,000 Twitter accounts deemed as "Australian". There was a gap in tweet collection between May 2017 and early 2018 which coincided with the TrISMA project ending and the QUT Digital Observatory being established.
It was at this time that we (the QUT Digital Observatory) took over the tweet collection and made improvements to secure it as a stable longitudinal collection. Subsequently, we built another collector and developed a method to identify more Australian accounts. The method allowed us to periodically update the list of accounts and collect more tweets from them. Updates were completed in late 2020 and 2021, resulting in twice the number of accounts compared to the original list.
More details on this can be found in the Australian Twittersphere factsheet.
The Australian Twittersphere was our first major resource as a research infrastructure facility. It provided historial, Australia-specific data to researchers at a time when it was not possible to geo-locate Twitter data in significant volumes (which is still the case: roughly 1% of tweets include precise geolocation information). At the time, so much of our work was related to the Australian Twittersphere that the Digital Observatory became synonymous with it.
The Australian Twittersphere has helped inform impactful research into education, health, business, law, and the humanities and social sciences. A non-exhaustive list of publications arising from this databank is listed below.
Twitter/X is only one of many cases where the gates to research-friendly, free-to-access APIs are gradually closing. Other platforms have also begun to gatekeep their data by charging for API access.
For example, researchers working with Twitter data are now having to bear the extra costs of the API. It's not cheap: a reasonable amount of 1 million tweets costs no less than $5,000 USD per month. These costs mean that further collection for the Australian Twittersphere is no longer feasible, but we will continue to make the existing data available to researchers. There are also new limitations on the Twitter/X platform that researchers should be aware of. This article gives a good overview of how to navigate the new Twitter/X.
The good news is that not all doors are closed. Platforms such as YouTube and Tiktok still provide free API access for researchers (albeit with some limitations). Indeed, the Digital Observatory has been working on different platforms and alternatives for researchers for several years. Our youte tool is a fantastic example of this work. Researchers can also leverage other maturing methods of collecting Web data, such as web archiving and data donation. The Digital Observatory is working on resources for web archiving, so watch this space!
The transition away from Twitter will be uncomfortable for some, but it is important to remember that other sources of data still exist on the Web.
Researchers can refer to our guide to the Australian Twittersphere for more information on the Australian Twittersphere and how to access it. For a more in-depth technical overview, please read the Australian Twittersphere technical fact sheet.
Publications that have used the Australian Twittersphere include:
