What makes a trophy hunter? An empirical analysis of Reddit

What makes a trophy hunter? An empirical analysis of

Reddit discussions

Chien Lu

, Jaakko Peltonen

, Timo Nummenmaa

, Xiaozhou Li

, and Zheying

Zhang

Tampere University, Kalevantie 4, 33100 Tampere, Finland

Abstract. In this paper, an empirical data-driven analysis of online discussions

of meta-game reward systems is carried out. The data is collected from one of

the biggest online discussion forums called Reddit and a text-mining technique

called topic modeling is employed. Over 46000 discussion threads from the two

most relevant subreddits /r/xboxachievements and /r/Trophies are analyzed and

the results of topic modeling shows not only interesting topics but also the

(dis)similarity between two text sources the temporal trends of topics. We have

found that the volume of related discussions shows an ongoing trend. The topic

model results have also revealed that some game genres are more prevalent than

others and the (dis)similarities between text sources have been also discovered.

Keywords: trophy hunting, Reddit, topic modeling

1 Introduction

It has been noted that games can also be gamified [10], which has attracted research-

ers’ attention [9,14]. Early work on applying badge systems to non-game contexts

have also used game achievement systems as a source of inspiration [19]. One of the

gamification design implementations is a meta-game reward system, which awards

visual indicators for completion of tasks and acts as an overarching game in which

players obtain rewards through playing across different games [5]. In game industries,

meta-game reward systems have been widely used such as badges (STEAM),

achievement points (Xbox) or trophies (PlayStation), etc. [12, 15, 30].

Reddit, a major participatory culture platform [17], wh has more than 138,000 ac-

tive subreddits (each represents a focused sub-community) and more than 330 million

users. Due to the number of discussions, the text content has been a valuable resource

in various fields [16, 22]. There are two subreddits related to our interests:

/r/xboxachievements, a place to “show off achievements” has more than 3 thousand

users and /r/Trophies has more than 25 thousand “trophy hunters”. They both yield an

abundance of discussions that reflect the gamers or potential gamers’ perception.

However, despite containing valuable information, they have not been well explored.

This work performs a large-scale data-driven analysis of gamers’ interests and atti-

tudes toward two reward systems with comprehensive data collection of discussion

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

146

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

threads in the above described two sources. We use topic modeling to computational-

ly extract themes of discussion from the huge corpus. How the prevalence of the ex-

tracted themes varies over the two subreddits and over time are further estimated.

The rest of the paper is structured as follows. Section 2 describes related work.

Section 3 describes data collection and text mining methods. Section 4 describes the

resulting extracted themes. Section 5 provides analysis and discussions. Conclusions,

limitations, and opportunities can be found in Section 6.

2 Related work

Cruz et al. [5] investigated meta-reward systems for Xbox and Playstation and how

they affect game players’ experience. They collected discussions from focus groups

of 36 console players and used Self-determination theory (SDT), Organismic integra-

tion theory (OIT), Cognitive evaluation theory (CET) and Signaling theory (ST) as

the framework to analyze the discussions. SDT, OIT, CET disclose an individual’s

motivations [26] and ST provides a framework to understand how the information is

sent and interpreted when communicating [6]. Many studies shed light on importance

of game reward systems towards players’ enjoyment and motivation. Jakobsson,

based on study of the Xbox 360 achievement system, indicates it not only works as a

reward system but also as a multiplayer online game in which all Xbox players partic-

ipate compulsorily [11]. Wang and Sun suggest players enjoy game rewards and react

to the motivation they provide while indicating that reward mechanisms foster intrin-

sic motivation and give extrinsic rewards based on relevant psychological theories

[27]. By studying the reward system of World of Warcraft, Rapp emphasizes im-

portance of rewards in shaping player experiences but suggests to consider reward

designs carefully to “surpass the exclusive employment of extrinsic motivators in

gamification design” [24]. Many studies obtained positive outcomes adopting design

similar to game reward systems in gamification and serious games [8, 20].

One example of using empirical data is the work of Wells et al. [28] who analyzed

30,227 PlayStation user profiles to explore how factors including, PlayStation Plus

subscriptions, player regions, etc., influence earning achievements. Using data from

Steam, O'Neill indicates achievements may incentivize more playtime from players

who would have otherwise played less [21]. Players' achievements data and data min-

ing methods are also used in analyzing their performance in serious games towards

enhancement of players' motivation and engagement [2]. Many studies have chosen

Reddit as their data source when researching users' opinions on various topics [13,

29]. We didn’t find works analyzing text of online discussion regarding meta-game

reward systems, perhaps due to difficulties when it comes to a large amount of data.

We tackle the issue by a computer-aided solution explained in Section 3.

We aim to answer three research questions. RQ1. Is discussion of the meta-game

reward system an ongoing trend? RQ2. What kind of games are more mentioned

concerning meta-game reward systems? RQ3. What are similarities & dissimilarities

of discussion between Xbox and PS about the meta-game reward system?

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

147

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

3 Method

Data Collection and Processing. Reddit threads from the two subreddits between

2014 and 2019 (until the 17th of October) were collected via the pushshift API

. The

text was lemmatized and stop words removed. Each submission with its comments

was aggregated as a document. We eliminated submissions without comments as they

did not evoke discussions. Documents with less than ﬁve words were ﬁltered out after

processing. In total, we obtained 1384 documents from /r/xboxachievements and

45145 documents from /r/trophies, more details shown in Table 1.

Text Mining. We used a text mining technique called topic modeling to analyze the

collected online discussions. It is a probabilistic model that represents each document

as a mixture of several latent topics. Each topic speaks for a speciﬁc semantic theme

within the data collection. The topics are not provided by a human expert but are

learned by an algorithm in a data-driven manner. Topic models such as Latent Di-

richlet Allocation [3] have been used in domains including game studies (e.g. [7]).

The basic representations found by a topic model are: 1) The strength (prevalence)

of the latent topics; each topic has a prevalence in each document, represented as

proportions that sum to one, and can be summarized over diﬀerent data subsets. 2)

The vocabulary of the topics, represented as a distribution over typical words in each

topic. In this work, a more advanced approach called Structural Topic Model (STM,

[25]) is employed. It is a topic model where the strength of topics within a document

can depend on document-level covariates, enabling us to model both text content of

the documents, and how they are aﬀected by annotations. We take the document

source (subreddit) and submission time (initial time of the discussion series) as docu-

ment-level covariates. The two covariates are taken to model (dis)similarity between

two subreddits and the temporal dynamics. Compared to conducting topic modeling

in two sources separately (e.g. [1]), this provides an integrated solution via directly

treating the eﬀect of the source as a learned parameter in the model.

We use held-out likelihood as the criterion to decide the number of topics K, the

user-speciﬁed parameter of STM. For eﬃciency we ﬁrst searched from K = 10 to 500

with an interval of 5; the best setting was K = 355. We investigated exhaustively from

K = 351 to 359 and K = 355 was still the best. We set up 10 initializing models with K

= 355; the model with best semantic coherence [18] was adopted as the ﬁnal model.

4 Results

Table 1 shows the volume of collected data. There are 1384 submissions in

/r/xboxachievements and 45145 in /r/trophies. In both subreddits, the number of sub-

missions has grown over time. After topic model training, found topics were labeled

under a practice similar to the Thematic analysis [4] but without reading though all

the text manually. Labels were decided with authors of this paper examining seman-

https://github.com/pushshift/api

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

148

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

tics of the top words and example documents of the topic (having high prevalence of

the topic). Topics whose top words prominently contained words for specific game

titles or characters were denoted as specific topics about those games; such topics

might concern multiple games, such as a game series, if often discussed together.

While separation of topics on specific games into genres cannot be made fully conclu-

sively, each topic was tagged with one or more genres. Non-game-specific topics

were labeled according to semantic themes they contained. Topics with no clear se-

mantic theme were labeled as linguistic topics that provide nuance to the Reddit dis-

cussion but which are not a focus of our analysis. Note that each topic may weakly

contain other thematic content besides the main one marked as their label.

Table 1. Number of documents of each year

Subreddit

2014

2015

2016

2017

2018

2019

Total

xboxachievements

154

141

274

294

495

1384

trophies

2645

4153

6044

7797

11307

13197

45145

The /r/Trophies subreddit introduced a bot ‘PSNTrophyBot’ in 2018 which de-

scribes a game’s trophies, for example, “God of War has 37 trophies: 22 bronze, 8

silver, 6 gold, and platinum...”. The bot adds ‘I am a bot’ to the end of its posts; thus

topics where ‘bot’ was among the top words were considered related to bot activity

and are shown in the online full table but not focused on in later analysis. A selection

of found topics are in Tables 2 - 3

. We discuss selected examples next.

Game related topics. 105 of the 355 topics (30% of the topics) turned out to be

game-specific (see online full table). The most discussed game-specific topics were

about games PlayerUnknown’s Battlegrounds, the ‘Infamous’ series, point and click

adventure games from Telltale Games, the ‘Far Cry’ series, Uncharted the Lost Lega-

cy and Conan Exiles, the ‘Assassin’s Creed’ series, the Lego video games series, the

‘God of War’ series for Playstation portable, Demon’s Souls and Bloodborne, and

Rocket League and Teslagrad. We do not present their top words in Tables 2-3 as

they are game-specific: for example, the Telltale Games topic contained words like

‘walk’, ‘telltale’, ‘episode’, ‘dead’, ‘tale’, ‘batman’, ‘borderland’, and ‘season’ refer-

encing games like ‘The Walking Dead’ and ‘Tales from the Borderlands’. However,

examples of game-specific time courses are shown in Fig. 1 to demonstrate the rise

and fall of a game’s popularity for trophy-hunting: PlayerUnknown’s Battlegrounds

(Plot (a), Fig.1) is slowly declining, whereas Grim Fandango and Day of the Tentacle

(Plot (b), Fig.1) have a strong Playstation peak in 2017. See the full set of plots for

other games: e.g. Tomb Raider has little change on Playstation but has a peak around

2016 on Xbox, and Uncharted a broad Playstation peak around 2017. The broader the

peaks, the more enduring the game and its reward mechanism are for trophy hunters;

the strongest ones can serve as examples for future design.

Full table at shorturl.at/hrtH5, all time plots at shorturl.at/ t/uFGN.

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

149

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

Table 2. Top 8 words of selected topics, part 1, continued with Table 3

Topic

Pr(%)

Top 8 Words

Deciding to try a platinum trophy

1.76

platinum decide tricky huh get trophy twenty

gioco

Time Allocation to Play

1.62

take hour long around say roughly half clock

Achieving trophies after long

time

1.12

get year finally ago almost trophy platinum

lanuary

Easiness and achievability

1.02

easy pretty simple platinum achievable trophy

play noby

Wastefulness and pointless

1.01

like feel matter waste pointless play trophy

think

Games becoming boring or stale

0.95

play everything bore clean throughs normally

get stale

Requesting and appreciating help

0.93

help someone appreciate please able can ad-

vance thank

Congratulations

0.85

congrats thank man congratulation dude jeal-

ous madman good

Earning trophies and happiness

0.69

trophy week earn happy special date favorite

talk

Fun of playing, esp. platformers

0.65

fun blast play platformer platinum definitely

cfxtwywmlx batrlefield

Difficulty of getting trophies

0.61

hard say trophy require definitely difficulty

platinum take

Enjoyment and recommendation

0.57

enjoy definitely recommend highly play thor-

oughly entertain reasonably

Patches and glitches

0.57

trophy pop problem patch glitched happen

glitchy nothing

Messaging and invites

0.49

send message add request will tomorrow to-

night available

Hunting for trophies

0.38

hunt usually hunter tend generally interest

always

Seek information about a trophy

0.36

else trophy question anyone share hide answer

requirement

Downloadable content and ex-

pansions

0.33

dlc base main trophy additional expansion

atleast ons

Progress, corruptions and back-

ups

0.30

save load file progress reload corrupt backup

datum

Dedication required from players

0.28

player require single active dedicate somewhat

trophy exclusively

Relationships

0.26

right wife play girlfriend local trust overcook

couch

Buying in packs for discounts

0.26

buy purchase pack discount platd separately

payday consider

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

150

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

Table 3. Top 8 words of selected topics, part 2

Topic

Pr(%)

Top 8 Words

Boss fights and weapon upgrades

0.26

boss fight weapon upgrade defeat fully

secret phase

Shooting and kills

0.26

kill shoot floor bullet shotgun headshots

aim rifle

Fan hopes for sequels

0.25

hope fan sequel disappoint previous quali-

ty plot twist

Purchasing modes and preordering

0.25

version stack digital disc physical cost

confuse preorder

Getting trophies

0.24

another several platted ready chore gratz

someday tear

Milestone

0.22

hit moment milestone onto aswell trophys

hot aha

Frustration and perseverance

0.16

tough frustrate bitch definitely get say

perseverance dear

Enjoyment and motivation

0.16

gaming sometimes always accomplishment

enjoyment trophy motivate love

Multiplayer gaming

0.13

group team pvp randoms chat host team-

mate invite

Bugs in games

0.10

bug encounter buggy remix legacy laggy

thanatophobia khidr

Memories and nostalgia

0.09

memory nostalgia hilarious childhood

history nostalgic muscle lay

Attempting and memorizing

moves and routes

0.05

try attempt jump second route pause mem-

orize shave

Reward related topics include going for certain types of rewards, earning rewards,

reward difficulties, congratulates others, and discussing rewards themselves. Some

share feelings of accomplishment, topics Earning trophies and happiness, Mile-

stone, Enjoyment and motivation indicate pride and celebration. In Milestone, a

user stated: “...I’d share my 100% of this as I hit level 100 yesterday, the game is

consuming my life at the moment!”. Some topics imply the achievement difficulty

level, e.g. Easiness and achievability and Difficulty of getting trophies; topics like

Achieving trophies after long time and Dedication required from players imply

perseverance and commitment. In Easiness and achievability, a user stated: “...I just

kept at it until I got em. Otherwise, the trophies in both Pac-Man games are pretty

simple...”. Topics about feelings (Fun of playing) and fandom (Memories and nos-

talgia indicate players’ memorable experiences of the games, and lifestyle (Time

allocation to playing, relationship) indicates players’ commitment to playing as part

of their life. Wastefulness and pointlessness expresses a feeling of time waste, and

Frustration with difficulty and perseverance, Games becoming boring or stale,

and Grinding and monotony indicate mixed feelings for the reward system. A user

stated in Wastefulness and pointlessness: “...played both games over 200 hours and

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

151

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

fully enjoyed them but can never waste time collecting pointless...”. Seek infor-

mation about a trophy reflects information sharing among users.

Gameplay related topics. These topics indicate players hunting for trophies aim to

ﬁnd information to excel and to be aware of the games’ problems and updates, includ-

ing gameplay experiences (Multiplayer gaming, Fighting moves: combos, counter-

ing and blocking and Boss ﬁghts and weapon upgrades) and technical aspects

(Patches and glitches, Bugs in games, Saving and restoring progress, corruptions

and backups). Enjoyment and recommendation of entertainment and Fan hopes

for sequels link individual games to larger entertainment culture. Buying in packs

for discounts, Downloadable content and expansions and Purchasing modes and

pre-ordering indicate ﬁnancial planning of trophy hunters.

Temporal Dynamics. Trends of selected topics are in Fig 1. E.g. Grinding and

monotony on Playstation peaks in 2016 but is rising on Xbox in 2018-2019; Decid-

ing to try a platinum trophy peaks on Playstation in 2017. Dotted lines represent

95% confidence intervals; non-overlapping intervals of two curves suggest signifi-

cantly different prevalences. Some topics are Domain-speciﬁc. Game-specific topics

PlayerUnknown’s BattleGrounds and the “Tomb Raider” series are more preva-

lent in /r/xboxachievements whereas Grim Fandango, Day of the Tentacle, “Spi-

der-Man” series and Uncharted The Lost Legacy, Conan Exiles are more preva-

lent in /r/Trophies. Besides, topics Messaging and invites, Completion percentage,

Requesting and appreciating help, are found to be more mentioned in

/r/xboxachievements whereas Games becoming boring or stale, Grinding and mo-

notony, Time allocation to playing, Deciding to try a platinum trophy and Con-

gratulations are more mentioned in /r/Trophies. Attempting moves and routes has

an attention shift, as the trends have opposite directions in the two sources.

5 Discussion

As shown in Table 1, the data volume is drastically growing indicating growth of the

communities and impact of meta-game reward systems on gameplay and social me-

dia, so we answer our RQ1: the trend is clearly ongoing.

A sizable proportion of topics were game-specific indicating which games are most

discussed, answering RQ2. Action-adventure games were the most discussed genre

(topics with that genre tag had a total prevalence 0.069), followed by first-person

shooters (0.022), role-playing games (0.022) and platformers (0.020). The rest had a

prevalence of under 0.010. Discussions on multi-player games (e.g. PlayerUnknown’s

Battlegrounds) can be connected to the statement by Jakobsson [11] that the reward

system can itself be a massive multiplayer online game. Topics such as Requesting

and appreciating help, Congratulations are related to the relatedness in SDT.

The (dis)similarity of focus in the two subreddits is learned by STM and can be

evaluated from its parameters, answering RQ3. Topics that are more prevalent in

/r/xboxachivements such as Messaging and invites, Completion percentage and

Request, appreciate help are more related to pragmatic means of achieving scores.

While /r/Trophies has a wider variety of discussions, from perceived emotions

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

152

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

(Games becoming boring or stale, Grinding and monotony and Congratulations)

to decision making (Time allocation to playing, Deciding whether to try a plati-

num trophy and Difficulty of getting trophies, note that, although the last two topics

contain PlayStation-specific words such as “platinum” and “trophy”, other terms

(“decide”, “tricky”) can convey general semantic meanings so that they may also

appear in /r/xboxachievements threads). This could be due to that PlayStation has

different levels of rewards from Bronze to Platinum. This layered design enhances the

complexity and potentially enrichs the online discussions. The r/xboxachievements

subreddit is also smaller by comparison which means some of the discussion on Xbox

achievements has spread out more to other venues, such as r/xbox, a more general

discussion venue on Xbox related topics.

Fig. 1. Selected topic prevalence over time. Blue: /r/xboxachievements; Red: /r/Trophies; Dot

lines: 95% interval.

6 Conclusions, limitations, and opportunities

We presented a data-driven topic modeling study of player’s interests and attitudes to

reward systems based on two subreddit corpora. Our qualitative analysis of text con-

tent was assisted by computational text mining. The results were both interesting on

their own and supported human interpretation and efficient analysis of a large amount

of text data. We found semantically meaningful topics on game series, trophy-hunting

techniques, appreciation of achievements, dissatisfaction with difficulty, grinding, and

time use, and their temporal fluctuation. Appreciation or frustration revealed by the

topics can help promote a similar appreciation and avoid frustration in future gamifi-

cation design. The topics provide an understanding of players’ attitudes and opinions

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

153

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

regarding gamification (i.e., badge and achievement systems) of games. The topics

show players commit game achievement-related behaviors, e.g., expressing frustra-

tion, asking for help, appreciating help, and etc. via online forums. Benefits of such

behaviors towards continuous engagement in achievement pursuing and game playing

can be useful for other gamification applications.

This work used only Reddit discussions; other large-volume venues include the

Playstation Forum and Xbox achievement discussion forum. This can lead to a bias as

most Reddit users are from the United States. More categorical analysis can be con-

ducted in future by combining data sources, e.g., Twitter, Steam Community, etc., or

comparing to other methods, e.g., Non-Negative Matrix Factorization (NMF) [23].

Deepened understanding of player behavior regarding game trophies and achieve-

ments can facilitate the design of personalized gamification, and application of such

computational methods towards the understanding of player communities can enrich

game studies. We expect the computational approach will inspire similar studies,

especially in game studies. The discovered topics and temporal trends can further be

used to establish research questions or directions in related fields.

Acknowledgments

The work was supported by Academy of Finland decisions 312395, 313748 and

327352.

References

1. Ammari, T., Schoenebeck, S., Romero, D.M.: Pseudonymous parents: Comparing parent-

ing roles and identities on the mommit and daddit subreddits. In: Proceedings of the 2018

CHI Conference on Human Factors in Computing Systems. pp. 1–13 (2018)

2. Benmakrelouf, S., Mezghani, N., Kara, N.: Towards the identification of players’ profiles

using game’s data analysis based on regression model and clustering. In: Proc. ASONAM

2015. pp. 1403–1410. ACM (2015)

3. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. In: Advances in neural in-

formation processing systems. pp. 601–608 (2002)

4. Cooper, H.E., Camic, P.M., Long, D.L., Panter, A., Rindskopf, D.E., Sher, K.J.: APA

handbook of research methods in psychology, Vol 1: Foundations, planning, measures, and

psychometrics. American Psychological Association (2012)

5. Cruz, C., Hanus, M.D., Fox, J.: The need to achieve: Players’ perceptions and uses of ex-

trinsic meta-game reward systems for video game consoles. Computers in Human Behav-

ior 71, 516–524 (2017)

6. Donath, J.: Signals in social supernets. Journal of Computer-Mediated Communication

13(1), 231–251 (2007)

7. Faisal, A., Peltoniemi, M.: Establishing video game genres using data-driven modeling

and product databases. Games and Culture 13(1), 20–43 (2018)

8. Goh, D.H.L., Pe-Than, E.P.P., Lee, C.S.: Perceptions of virtual reward systems in

crowdsourcing games. Computers in Human Behavior 70, 365–374 (2017)

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

154

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

9. Hamari,J.,Eranti,V.:Frameworkfordesigningandevaluatinggameachievements. In: Digra

conference. Citeseer (2011)

10. Huotari, K., Hamari, J.: Gamification” from the perspective of service marketing. In: Proc.

CHI 2011 Workshop Gamification (2011)

11. Jakobsson, M.: The achievement machine: Understanding xbox 360 achievements in gam-

ing practices. Game Studies 11(1), 1–22 (2011)

12. Jakobsson, M., Sotamaa, O., Moore, C., Begy, J., Consalvo, M., Gazzard, A., Medler, B.,

Williams, P., Nesbitt, K.V., Eidels, A., et al.: game reward systems. Game Studies 11(1)

(2011)

13. Kilgo, D.K., Yoo, J.J., Sinta, V., Geise, S., Suran, M., Johnson, T.J.: Led it on reddit: An

exploratory study examining opinion leadership on reddit. First Monday 21(9) (2016)

14. Kominiarczuk,M.:Gamifiedgameplay:Gamificationingamedesign.Gamification. Critical

Approaches (2015)

15. Lewis, I., de Salas, K., Wells, L.: Features of achievement systems. In: Proc.

CGAMES’2013 USA. pp. 66–73. IEEE (2013)

16. Lu, J., Sridhar, S., Pandey, R., Hasan, M.A., Mohler, G.: Investigate transitions into drug

addiction through text mining of reddit data. In: Proc. KDD. pp. 2367– 2375. ACM (2019)

17. Massanari, A.: Playful participatory culture: Learning from reddit. AoIR Selected Papers

of Internet Research 3 (2013)

18. Mimno, D., Wallach, H.M., Talley, E., Leenders, M., McCallum, A.: Optimizing semantic

coherence in topic models. In: Proc. EMNLP. pp. 262–272 (2011)

19. Montola, M., Nummenmaa, T., Lucero, A., Boberg, M., Korhonen, H.: Applying game

achievement systems to enhance user experience in a photo sharing service. In: Proc.

MindTrek. pp. 94–97. ACM (2009)

20. Nagle, A., Wolf, P., Riener, R., Novak, D.: The use of player-centered positive reinforce-

ment to schedule in-game rewards increases enjoyment and performance in a serious

game. International Journal of Serious Games 1(4), 35–47 (2014)

21. O’Neill, M., Vaziripour, E., Wu, J., Zappala, D.: Condensing steam: Distilling the diversi-

ty of gamer behavior. In: Proc. 2016 internet measurement conference. pp. 81–95. ACM

(2016)

22. Park, A., Conway, M., Chen, A.T.: Examining thematic similarity, difference, and mem-

bership in three online mental health communities from reddit: a text mining and visualiza-

tion approach. Computers in human behavior 78, 98–112 (2018)

23. Pauca, V.P., Shahnaz, F., Berry, M.W., Plemmons, R.J.: Text mining using non- negative

matrix factorizations. In: Proceedings of the 2004 SIAM International Conference on Data

Mining. pp. 452–456. SIAM (2004)

24. Rapp, A.: From games to gamification: A classification of rewards in world of warcraft

for the design of gamified systems. Simulation & Gaming 48(3), 381–401 (2017)

25. Roberts, M.E., Stewart, B.M., Airoldi, E.M.: A model of text for experimentation in the

social sciences. J Am Stat Assoc 111(515), 988–1003 (2016)

26. Ryan, R.M., Deci, E.L.: Self-determination theory and the facilitation of intrinsic motiva-

tion, social development, and well-being. American psychologist 55(1), 68 (2000)

27. Wang, H., Sun, C.T.: Game reward systems: Gaming experiences and social meanings. In:

DiGRA Conference. vol. 114 (2011)

28. Wells, L., Cauchi-Saunders, A., Lewis, I., Monsif, L., Geelan, B., de Salas, K.: Mining for

gold (and platinum): Playstation network data mining. In: Proc. 2016 Annual Symposium

on Computer-Human Interaction in Play. pp. 304–312. ACM (2016)

29. Weninger, T.: An exploration of submissions and discussions in social news: Mining col-

lective intelligence of reddit. Soc Netw An Mining 4(1), 173 (2014)

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

155

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

30. Zulkifly, A., Lewis, I., de Salas, K.: Do achievement systems undermine intrinsic player

motivation? In: Proc. CGAMES’2013 USA. pp. 19–22. IEEE (2013)

GamiFIN Conference 2020, Levi, Finland, April 1-3, 2020 (organized online)

156

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).