Performance Evolution Matrix: Visualizing Performance Variations along Software Versions

Abstract

# Performance Evolution Matrix<br> This repository contains the artifacts needed to replicate our experiment in the paper "Performance Evolution Matrix". # Video Demo<br> [download](https://github.com/jpsandoval/PerfEvoMatrix/blob/master/MatrixMovie.mp4)<br> <br> # XMLSupport and GraphET Examples To open the XMLSupport and GraphET Examples (which appears in the paper) execute the following commands in a Terminal. **MacOSX.** We do all the experiments in a Mac Book Pro. To open the Matrix execute the following command in the folder where this project was downloaded. ```<br> ./Pharo-OSX/Pharo.app/Contents/MacOS/Pharo Matrix.image<br> ``` **Windows.**<br> You may also run the experiment in Windows, but depending on the windows version you have installed it may be some some UI bugs.<br> ```<br> cd Pharo-Windows<br> Pharo.exe ../XMLSupportExample.image<br> ``` **Open the Visualization.**<br> Please select the following code, then execute it using the green play button (at the top right of the window).<br> ```<br> ToadBuilder xmlSupportExample.<br> ```<br> or <br> ```<br> ToadBuilder graphETExample.<br> ```<br> **Note.** There are two buttons at the panel top left In (zoom in) and Out (zoom out). To move the visualization just drag the moves over the panel. # Experiment<br> This subsection describe how to execute the tools, for replicating our experiment. ## Baseline<br> The baseline contains the tools and the project-dataset to realize the tasks described in the paper (identifying and understanding performance variations). ## Open the Baseline **MacOSX.** We do all the experiments in a Mac Book Pro. To open the Baseline execute the following command in the folder where this project was downloaded. ```<br> ./Pharo-OSX/Pharo.app/Contents/MacOS/Pharo Baseline.image<br> ``` **Windows.**<br> You may also run the experiment in Windows, but depending on the windows version you have installed it may be some some UI bugs.<br> ```<br> cd Pharo-Windows<br> Pharo.exe ../Baseline.image<br> ``` ## Open a Project There are three projects under study, depending on the project you wanna use for the task, you may execute one of the following scripts. For executing a script press Cmd-d or right-click and press do it. **Roassal**<br> ```<br> TProfileVersion openRoassal.<br> ``` **XML**<br> ```<br> TProfileVersion openXML.<br> ```<br> **Grapher**<br> ```<br> TProfileVersion openGrapher.<br> ``` ## Baseline Options<br> For each project, we provide a UI which contains all the tools we use as a baseline. Each item in the list is a version of the selected project. &lt;img src="images/baseline.png" width="300"&gt; - Browse: open a standard window to inspect the code of the project in the selected version.<br> - Profile: open a window with a call context tree for the selected version.<br> - Source Diff: open a window with the code differences between the selected version and the previous one.<br> - Execution Diff: open a window with the merge call context tree gathered from the selected version and the previous one. **Note.** All these options require you select first a item in the list. # Matrix ## Open Matrix Image. **MacOSX.** We do all the experiments in a Mac Book Pro. To open the Matrix execute the following command in the folder where this project was downloaded. ```<br> ./Pharo-OSX/Pharo.app/Contents/MacOS/Pharo Matrix.image<br> ``` **Windows.**<br> You may also run the experiment in Windows, but depending on the windows version you have installed it may be some some UI bugs.<br> ```<br> cd Pharo-Windows<br> Pharo.exe ../Matrix.image<br> ``` ## Open a project There are three projects under study, depending on the project you wanna use for the task, you may execute one of the following scripts. For executing a script press Cmd-d or right-click and press do it. **Roassal**<br> ```<br> ToadBuilder roassal.<br> ``` **XML**<br> ```<br> ToadBuilder xml.<br> ```<br> **Grapher**<br> ```<br> ToadBuilder grapher.<br> ``` # Data Gathering Before each participant starts a task we execute the following script in Smalltalk. For executing a script press Cmd-d or right-click and press do it. It allows us to track the time that a user starts the experiment and how many mouse clicks, movements.<br> ```<br> UProfiler newSession.<br> UProfiler current start.<br> ``` After finishing the task we executed the following script. It stop recording the mouse events and save the stops time.<br> ```<br> UProfiler current end.<br> ``` The last script generates a file with the following information: start time, end time, number of clicks, number of mouse movements, and the number of mouse drags (we do not use this last one).<br> ```<br> 11:34:52.5205 am,11:34:56.38016 am,14,75,0 ```<br> # Quit<br> To close the artifact, just close the window or press click in any free space of the window and select quit.