PHILOSOPHY OF RESEARCH

Abstract:

Gamification is a term used to describe the concept of non-game type tasks being altered, making for a more amusing and engaging experience for the participants involved in said activity. [1]

In the educational setting of computer science, due to the instructional content and gamification present in educational computer games, these platforms have been shown to improve motivation, retention and speed of completion within activities. [3]

Large failure rates in computer science courses are one of the dilemmas that educators face in undergraduate programs, especially with initial programming. Although concepts such as pair programming have been formed to help weaker students perform at a higher level academically, “Game based learning may resolve some of these challenges” [2] of failure and premature drop outs. [2]

This paper discusses the design principals of gamification, and where it can directly be applied regarding the process of software development, also weighing the positives and negatives of the gamification of software development in the workplace/classroom. The focus of this literature review is to discover a research gap in existing studies in the gamification of software development in education, with a brief pilot study on the research question founded.

Gamification in Software Development: Overview

Gamification is the process of applying elements of games into non-game contexts, which intends to provoke positivity in the engagement of the player who partake in the game. [1]

Teachers and academics have often used games as a platform for educating students, because games possess learning techniques not found in traditional teaching methods. Due to the nature of games altering the state of reality of the players, a game can create a compelling alternative using narrative’s and free experimentation. [2]

The aim of gamification is to engage users, to increase retention, knowledge, and alter behaviour when it comes to learning content. the purpose of which is to “create a sense of playfulness in non-game environments so that participation becomes enjoyable and desirable”. [1]

As a learning platform within the software development realm, gamification has the potential to motivate software developers to discover new learning opportunities’ in the form of new programming languages and improve the quality of the work conducted by the developer. [4]

It is crucial that the designers of these games have the utmost respect of the motivational construct of the elements within the game, “Gamified systems which demonstrate a clear understanding of the psychological needs of their proposed participants can be utilized to engender a sense of intrinsic motivation in these participants”. [1]

Although there are criticisms surrounding the gamification concept, being quoted as “exploitionware”, described as a business orientated way of exploiting people in the form of clever marketing, [5] there are many studies being conducted on its the uses and benefits.

Gamification in an educational setting has been well documented and studied, as well as frameworks created by game designers and academics for its successful execution, involving motivational theories in aid of its formation. [1]

Design Principals of Gamification in Software Development

Motivation:

“Research shows that learners may feel more engaged when culturally relevant tools are harnessed for education.” [6]

The players of games are often highly engaged and are attracted by the elements that the game provides, such as leader boards, personalized avatars and achievement’s. Through the act of interaction, players can be motivated to execute tasks within the game, due to the reward mechanisms provided. [1]

It is apparent that traditional teaching methods fail to provide this motivation that gamification Is based upon.[7] Games designed to inhibit learning outcomes are often motivationally analysed and built upon using frameworks based on theories of motivation, such as the Self-Determination-Theory (SDT), which have been academically and empirically tested, arguing that if the 3 constructs are met, Competence, Autonomy, and Relatedness, then the player will feel motivated to undertake a task. Researchers created a taxonomy of 18 different game elements, such as Achievements, Avatars, and Boss fights. [1]

 The average level of agreement for Competence was 70.9%, Autonomy had a percentage of 68%, and 57.2% for Relatedness. This provided a link between 18 different game elements and the motivational constructs of Competence, Autonomy, and Relatedness. [1]

Learning outcomes:

Establishing learning goals for the player when designing a gamification system is important, as well as defining a set of essential questions for achieving these goals. Identifying the tasks that will be performed in the game to provide the evidence needed to ascertain the players understanding of the learning material. Identifying other evidence that should be collected to determine if the desired results have been met. Include playing time in each level, incorrect and correct moves etc. Provide a manner for the players to reflect on their learning experiences. Research target users, Narrative, fellowship. Sensory aesthetics, reward systems. 

Benefits of Gamification in Software Development

Increases completion speed:

In gamification platforms geared towards education, assessments have shown positive results on learning objectives among students. Research was carried out whether assessments within a game, designed to teach computer programming concepts would increase the engagement and the completion speed of levels within the game. It was found that students who were given assessment levels successfully completed more game levels and played the game a greater length of time then those who were given non-assessment levels. [3]

Engagement:

 Engagement is a crucial aspect in gamification, and the elements within the game can potentially impact the motivational participation in its users. A study was conducted on the use of badge elements within a gamification platform, “When it comes to our first hypothesis, the results showed that gamification lead to a significantly higher level of engagement.” [4]

Collaboration:

Gamification has been successful in not just in the solving of course content individually, but also has been shown to aid collaboration and communication efficiency among developers/students.

Gamification can be used as a motivator for team participation and incentivise collaborative behaviour among a team of software developers or a group of students. “Besides motivation, another human factor that is important to achieve success in a software development process is collaboration.” [8]

Students or software developer’s working on a project within a group, feel motivated to collaborate more successfully due to the gamification elements within a platform, for example, a case study which tested a gamified, collaborative discussion group forum which was introduced to 249 students in their first semester. The case study proved successful, increasing student collaboration, and increased the communication of course content among students by 88%, also allowing for less resources to be used regarding lecturers replying to individual queries. The discussion group increased engagement, as well as collaborative behaviour using gamification methods. [9]

Challenges in Gamification

It has been argued that people who have the desire to design games to be of benefit in an educational setting fall under the scrutiny of “pointsification”, a term used to describe the use of badges, trophy’s and leader boards being used as  outer layer to the game, where no initial thought was put into how these are of any use to the player regarding participation or effectiveness, where the educational aspect just seems like an afterthought. [1]

In a study done by a research group, 120 games were analysed which were advertised as educational aids with initial programming modules. The findings of the analysis were that little games within the 120 analysed did not mention their learning objectives or motives, and the lack of assessment’s in the games. [2]

Achievement alone within a game may not transfer over to the grasp of the content when it comes to initial programming and computer science, educational games may fail due to the “The Vegas effect, the learning that happens in games, stays in games”. [2]

The main setback for gamification is the lack of structure and motivation, where the know how logic of a game designer can meet the psychological requirements needed to improve engagement and retention in the user. Gamification requires an empirical motivational theory in its construct, such as the self-determination theory, avoiding the “pontification” concept regarding the design of a gaming platform. For the game to be successful in an educational setting, assessing the participants correctly throughout the experience is essential. [3]

Although assessments may increase the speed of completion, assessments may also hinder the engagement of the users. assessments can lead to test anxiety, and poor attendance, and within a game environment may lead to poor flow of mechanics, causing the user to become demotivated and to shut the game off. [3]

Research question identified

From the state of the art conducted, a vast amount of research has been pursued in the use of gamification in education, targeted at individual performances.  There is also data available on the effects of gamification among large classes and teams of software developers, focussing more on motivational design frameworks, collaboration and communication.

The research question that arose from the state of the art is as follows:

What effect does gamification have on the software development process among a team of software developers using pair programming?

Empirical Study Design:

To conclude what effect gamification has on the software development process among developers using pair programming, an experiment needs to be designed.

The experiment will observe the workings of a group of software developers, who are participating in pair-programming (PP), using the gamification methodology.

Another pair of software developers who are participating in PP will be gathered, who will be using the non-gamification methodology in contrast.

Both parties will be asked to complete a work sheet, containing programming assessment questions ranging in length and intensity, in the Java programming language.

The gamification system will follow the work of Buckley et al [1], who created a gamification-motivation design framework. The gamification system will use elements such as levels, quests, teams, achievements, and other elements that call on the constructs of Competence, Autonomy, and Relatedness.

Figure 1

The motivational design framework GaMDeF [1]

The initial phase of the empirical study will involve gathering subjects willing to participate in the experiment. 

A demographic analysis will be recorded regarding the participants, such as gender and programming ability.

Expected difficulties will be designing a suitable gamification system that does the GaMDeF sufficient justice, using the resources available to the conductor of the experiment.

The results of experiment will compare and contrast the speed of completion of the Java lab sheet among the PP non-gamification participants and the PP gamification participants.

To complete the experiment, the data will be compiled in order to distinguish the effects of gamification on software developers using the pair programming method of development.

A survey will be conducted and completed by the participants, giving data on the outcome of the experiment, regarding engagement, collaboration, and quality of work done among the pairs.   

Pilot study:

A preliminary pilot study was conducted on the topic, where 6 participants in a survey where asked the following questions:

Figure 2:

Figure 3:

Figure 3:

Figure 4

References

[1] Jim Buckley, T. D. (2018). Journal of Educational Technology Systems. A Gamification–Motivation Design Framework for Educational Software Developers, 3. Retrieved November 21, 2018, from https://ulir.ul.ie/bitstream/handle/10344/6911/Murray_2018_Gamification.pdf?sequence=4

[2] Monica M. McGill. Durell Bouchard, C. J. (2018, 01 30). If Memory Serves: Towards Designing and Evaluating a Game for Teaching Pointers to Undergraduate StudentsIf Memory Serves: Towards Designing and Evaluating a Game for Teaching Pointers to Undergraduate Students. ITiCSE-WGR ’17, 2, 4. Retrieved November 21, 2018, from https://dl.acm.org/citation.cfm?id=3174783&dl=ACM&coll=DL

[3] Michael J. Lee, A. J. (2014, August 12). In-Game Assessments Increase Novice Programmers’Engagement and Level Completion Speed. ,7. Retrieved November 28, 2018, from https://www.pixel42.com/cv/publications/Lee2013_GidgetAssessments.pdf

[4] Margarita Ortiz, K. C. (2017). Gamification in Computer Programming: Effects on learning, engagement, self-efficacy and intrinsic motivation. Conference: 11th European Conference on Games Based Learning, At Graz, Austria. Austria: Research Gate. Retrieved November 29, 2018, from https://www.researchgate.net/publication/320372920_Gamification_in_Computer_Programming_Effects_on_learning_engagement_self-efficacy_and_intrinsic_motivation

[5] BOGOST, I. (2011, August 9). ‘Gamification Is Bullshit’. Retrieved from : http://www.theatlantic.com/author/ian-bogost/: http://creativegames.org.uk/modules/Gamification/Bogost_Gamification_is_Bullshit-2012.pdf

[6] Chris Johnson, M. K. (2016, November 11). Game Development for Computer Science Education. Retrieved December 1, 2018, from http://repository.falmouth.ac.uk/1724/1/iticse16_wg_gamedev.pdf

[7] Joey Lee, J. H. (2011). Gamification in Education: What, How, Why Bother? Academic Exchange Quarterly, 2. Retrieved November 22, 2018, from https://www.researchgate.net/publication/258697764_Gamification_in_Education_What_How_Why_Bother

[8] Flavio Steffens, F. F.-A. (2015). Using Gamification as a Collaboration Motivator for Software Development Teams: A Preliminary Framework. Conference: SBSC ’15: Proceedings of the Brazilian Symposium in Collaborative Systems (p. 2). Research gate. Retrieved November 28, 2018, from https://www.researchgate.net/publication/288833904_Using_Gamification_as_a_Collaboration_Motivator_for_Software_Development_Teams_A_Preliminary_Framework

[9] Antti Knutas, J. I. (2014). Increasing Collaborative Communications in a Programming Course with Gamification: A Case Study. International Conference on Computer Systems and Technologies – CompSysTech’14 (p. 6). Bulgaria: ACM. Retrieved December 1, 2018, from http://delivery.acm.org.proxy.lib.ul.ie/10.1145/2660000/2659620/p370-knutas.pdf?ip=193.1.100.108&id=2659620&acc=ACTIVE%20SERVICE&key=846C3111CE4A4710%2E2C2349ADB1824AA1%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&__acm__=1544417220_ce2e5ada1a7dab3e000bda1e40deb