A NOVEL METHODOLOGY OF COMPLETING CODING EXERCISES ON OBJECT ORIENTED PROGRAMMING LANGUAGES BY LLMS TO DEVELOP SOFTWARE

Abstract

By dynamically proposing the most probable next action within a program's logic, based on the current contextual flow, Integrated Development Environments significantly accelerate the construction of software. The efficacy of three pre-trained language models in tackling multiple-choice assessments pertaining to introductory and intermediate computer science coursework was investigated. Short instructional sequences (instead of "codings snippets") were commonly included in these assessments, offered at the post-secondary level. The emergence of this novel technology has sparked widespread discourse surrounding its potential benefits, such as the generation of practice exercises and explanations of written instructions, in the context of computer science education. Concerns regarding its potential misutilization, such as facilitating fraudulent behaviour, have also been raised within this domain. Despite limited knowledge concerning their effectiveness in analyzing instructional sequences within educational contexts, research into the reasoning capabilities of GPT models remains sparse. An investigation was conducted utilizing various OpenAI models to assess their performance on multiple-choice evaluations (formative and summative) drawn from three introductory computer science courses employing Java, encompassing a total of 530 questions.