Stylometry for real-world expert coders: a zero-shot approach - Télécom Paris
Journal Articles PeerJ Computer Science Year : 2024

Stylometry for real-world expert coders: a zero-shot approach

Abstract

Code stylometry is the application of stylometry techniques to determine the authorship of software source code snippets. It is used in the industry to address use cases like plagiarism detection, code audits, and code review assignments. Most works in the code stylometry literature use machine learning techniques and (1) rely on datasets coming from in vitro coding competition for training, and (2) only attempt to recognize authors present in the training dataset (in-distribution authors). In this work we give a fresh look at code stylometry and challenge both these assumptions: (1) we recognize expert authors who contribute to real-world open-source projects, and (2) we show how to accurately recognize authors not present in the training set (out-distribution authors). We assemble a novel open dataset of code snippets for code stylometry tasks consisting of 114,400 code snippets, authored by 104 authors having contributed 1,100 snippets each. We develop a K-nearest neighbors algorithm (k-NN) classifier for the code stylometry task and train it on the dataset. Our system achieves a top accuracy of 69% among five randomly selected in-distribution authors, thus improving state of the art by more than 20%. We also show that when moving from in-distribution to outdistribution authors, the classification performances of the k-NN classifier remain the same, achieving a top accuracy of 71% among five randomly-selected out-distribution authors.
Fichier principal
Vignette du fichier
stylometry-zero-shot.pdf (758.39 Ko) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

hal-04793178 , version 1 (20-11-2024)

Licence

Identifiers

Cite

Andrea Gurioli, Maurizio Gabbrielli, Stefano Zacchiroli. Stylometry for real-world expert coders: a zero-shot approach. PeerJ Computer Science, 2024, 10, pp.e2429. ⟨10.7717/peerj-cs.2429⟩. ⟨hal-04793178⟩
0 View
0 Download

Altmetric

Share

More