The Superdiversity dataset includes the
Superdiversity Index (SI) calculated on the diversity of the emotional content expressed in texts of different communities. The emotional valences of words used by a community are extracted from Twitter data produced by that specific community. The Superdiversity dataset includes the SI built on
Twitter data and
lexicon-based Sentiment Analysis. In addition, the dataset comprises other possible diversity measures calculated from the same data from which the SI is calculated, such as the number of tweets in the community language and the Type-Token Ratio, the number of languages in a community. The SI ranges in [0, 1]:
- a value of 0 means an emotional content very close between the computed valences and a standard emotional lexicon.
- a value of 0.5 indicates no correlation between the emotional content of words used by the community on Twitter and the standard emotional content.
- a value of 1 would correspond to the use of terms with the opposite emotional content compared to the standard.
Data is computed at three different geographical scales based on the
Classification of Territorial Units for Statistics (NUTS), i.e., NUTS1, NUTS2, and NUTS3, for two different nations Italy and the United Kingdom. The untagged Twitter dataset is composed of just under 73,175,500 geolocalised tweets gathered for 3 months, from the 1st August to the 31st October of 2015.