This repository contains the replication package, analysis scripts, and additional information on our paper accepted at ICSE 2021.

Publication: Norman Peitek, Sven Apel, Chris Parnin, André Brechmann, and Janet Siegmund. Program Comprehension and Code Complexity Metrics: An fMRI Study. In Proceedings of the International Conference of Software Engineering (ICSE) 2021, pages 524-536. ACM SIGSOFT Distinguished Paper Award. (preprint pdf, Digital Library)

In /replication, we provide:

• all used stimuli for the comprehension, control condition, and distractor tasks (/tasks-*) either as image and/or text file. We used our CodeImageGenerator to create image files for our Java code snippets.
• experiment protocol for the fMRI session
• the meta protocol for the post-session interview

In /data, we share all raw and preprocessed data that we can. Due to our local privacy law, we cannot publicly provide the fMRI data at this time. Please contact us for individual solutions.

In /analysis, we share our analysis scripts that process the input data, compute the results, create plots, and run statistics.

To run the script yourself, you will need:

• Python 3.x
• matplotlib
• numpy
• pandas
• pytest
• scipy
• seaborn
• statsmodels

You can install all dependencies via:

pip install -r requirements.txt

Once your system is ready, start the pipeline:

python main.py ## or pytest

The analysis output is located in /analysis/output.

If you want to test your own metric and evaluate whether it better fits our captured data, please add an additional column with the metric values for each snippet in data/metrics/SnippetComplexityMetricsValues.csv. Then, start a full pipeline run as described above. Please contact us if you have any problems.

For convenience, we provide all output that the analysis script yields for our data in /analysis/output.

In addition to the selected plots presented in the paper, we added all correlation plots in /analysis/output.

We explored overall 41 additional metrics. In /data/metrics/Metrics.md, we provide a description of each metric.

In the paper, we provide a shortened overview. Here is the full correlation table:

This repository is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

If you have questions, please contact me directly: peitek@cs.uni-saarland.de. Thank you!