RefactoringMiner is a library/API written in Java that can detect refactorings applied in the history of a Java project.
Currently, it supports the detection of the following refactorings:
supported by RefactoringMiner 1.0 & 2.0
supported by RefactoringMiner 2.0
In order to build the project, run ./gradlew jar (or gradlew jar, in Windows) in the project's root directory.
Alternatively, you can generate a complete distribution zip including all runtime dependencies running ./gradlew distZip.
You can also work with the project with Eclipse IDE. First, run ./gradlew eclipse to generate Eclipse project metadata files. Then, import it into Eclipse using the Import Existing Project feature.
In order to use RefactoringMiner as a maven dependency in your project, run ./gradlew install to add RefactoringMiner.jar into your local mvn repository, and then add the following dependency to your project:
<dependency>
<groupId>org.refactoringminer</groupId>
<artifactId>RefactoringMiner</artifactId>
<version>1.0</version>
</dependency>If you want to get refactoring information when inspecting a commit on GitHub, you can install our Refactoring Aware Commit Review Chrome Extension.
The Chrome extension can detect refactorings for public projects and commits matching the following URL patterns:
https://github.com/user/project/commit/idhttps://github.com/user/project/pull/id/commits/id
If you are using RefactoringMiner in your research, please cite the following paper:
Nikolaos Tsantalis, Matin Mansouri, Laleh Eshkevari, Davood Mazinanian, and Danny Dig, "Accurate and Efficient Refactoring Detection in Commit History," 40th International Conference on Software Engineering (ICSE 2018), Gothenburg, Sweden, May 27 - June 3, 2018.
@inproceedings{Tsantalis:ICSE:2018:RefactoringMiner,
author = {Tsantalis, Nikolaos and Mansouri, Matin and Eshkevari, Laleh M. and Mazinanian, Davood and Dig, Danny},
title = {Accurate and Efficient Refactoring Detection in Commit History},
booktitle = {Proceedings of the 40th International Conference on Software Engineering},
series = {ICSE '18},
year = {2018},
isbn = {978-1-4503-5638-1},
location = {Gothenburg, Sweden},
pages = {483--494},
numpages = {12},
url = {http://doi.acm.org/10.1145/3180155.3180206},
doi = {10.1145/3180155.3180206},
acmid = {3180206},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {Git, Oracle, abstract syntax tree, accuracy, commit, refactoring},
}RefactoringMiner has been used in the following studies:
The code in package **gr.uom.java.xmi.*** is developed by Nikolaos Tsantalis.
The code in package **org.refactoringminer.*** was initially developed by Danilo Ferreira e Silva and later extended by Nikolaos Tsantalis.
RefactoringMiner can automatically detect refactorings in the entire history of git repositories, between specified commits or tags, or at specified commits.
In the code snippet below we demonstrate how to print all refactorings performed in the toy project https://github.com/danilofes/refactoring-toy-example.git.
GitService gitService = new GitServiceImpl();
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
Repository repo = gitService.cloneIfNotExists(
"tmp/refactoring-toy-example",
"https://github.com/danilofes/refactoring-toy-example.git");
miner.detectAll(repo, "master", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});You can also analyze between commits using detectBetweenCommits or between tags using detectBetweenTags. RefactoringMiner will iterate through all non-merge commits from start commit/tag to end commit/tag.
// start commit: 819b202bfb09d4142dece04d4039f1708735019b
// end commit: d4bce13a443cf12da40a77c16c1e591f4f985b47
miner.detectBetweenCommits(repo,
"819b202bfb09d4142dece04d4039f1708735019b", "d4bce13a443cf12da40a77c16c1e591f4f985b47",
new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});// start tag: 1.0
// end tag: 1.1
miner.detectBetweenTags(repo, "1.0", "1.1", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});It is possible to analyze a specifc commit using detectAtCommit instead of detectAll. The commit
is identified by its SHA key, such as in the example below:
miner.detectAtCommit(repo, "05c1e773878bbacae64112f70964f4f2f7944398", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
});You can get the churn of a specific commit using churnAtCommit as follows:
Churn churn = miner.churnAtCommit(repo, "05c1e773878bbacae64112f70964f4f2f7944398", handler);There is also a lower level API that compares the Java files in two directories containing the code before and after some changes:
UMLModel model1 = new UMLModelASTReader(new File("/path/to/version1")).getUmlModel();
UMLModel model2 = new UMLModelASTReader(new File("/path/to/version2")).getUmlModel();
UMLModelDiff modelDiff = model1.diff(model2);
List<Refactoring> refactorings = modelDiff.getRefactorings();To use this API, please provide a valid OAuth token in the github-oauth.properties file.
If you don't want to clone locally the repository, you can use the following code snippet:
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
miner.detectAtCommit("https://github.com/danilofes/refactoring-toy-example.git",
"36287f7c3b09eff78395267a3ac0d7da067863fd", new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
}, 10);If you want to analyze all commits of a pull request, you can use the following code snippet:
GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
miner.detectAtPullRequest("https://github.com/apache/drill.git", 1807, new RefactoringHandler() {
@Override
public void handle(String commitId, List<Refactoring> refactorings) {
System.out.println("Refactorings at " + commitId);
for (Refactoring ref : refactorings) {
System.out.println(ref.toString());
}
}
}, 10);All classes implementing the Refactoring interface include refactoring-specific location information.
For example, ExtractOperationRefactoring offers the following methods:
getSourceOperationCodeRangeBeforeExtraction() : Returns the code range of the source method in the parent commitgetSourceOperationCodeRangeAfterExtraction() : Returns the code range of the source method in the child commitgetExtractedOperationCodeRange() : Returns the code range of the extracted method in the child commitgetExtractedCodeRangeFromSourceOperation() : Returns the code range of the extracted code fragment from the source method in the parent commitgetExtractedCodeRangeToExtractedOperation() : Returns the code range of the extracted code fragment to the extracted method in the child commitgetExtractedOperationInvocationCodeRange() : Returns the code range of the invocation to the extracted method inside the source method in the child commitEach method returns a CodeRange object including the following properties:
String filePath
int startLine
int endLine
int startColumn
int endColumnAlternatively, you can use the methods List<CodeRange> leftSide() and List<CodeRange> rightSide() to get a list of CodeRange objects for the left side (i.e., parent commit) and right side (i.e., child commit) of the refactoring, respectively.
All method-related refactoring (Extract/Inline/Move/Rename/ExtractAndMove Operation) objects come with a UMLOperationBodyMapper object, which can be obtained by calling method getBodyMapper() on the refactoring object.
Let's consider the Extract Method refactoring in commit JetBrains/intellij-community@7ed3f27
ExtractOperationRefactoring refactoring = ...;
UMLOperationBodyMapper mapper = refactoring.getBodyMapper();
List<StatementObject> newLeaves = mapper.getNonMappedLeavesT2(); //newly added leaf statements
List<CompositeStatementObject> newComposites = mapper.getNonMappedInnerNodesT2(); //newly added composite statements
List<StatementObject> deletedLeaves = mapper.getNonMappedLeavesT1(); //deleted leaf statements
List<CompositeStatementObject> deletedComposites = mapper.getNonMappedInnerNodesT1(); //deleted composite statementsFor the Extract Method Refactoring example shown above mapper.getNonMappedLeavesT2() returns the following statements:
final String url = pageNumber == 0 ? "courses" : "courses?page=" + String.valueOf(pageNumber);
final CoursesContainer coursesContainer = getFromStepic(url,CoursesContainer.class);
return coursesContainer.meta.containsKey("has_next") && coursesContainer.meta.get("has_next") == Boolean.TRUE;ExtractOperationRefactoring refactoring = ...;
UMLOperationBodyMapper mapper = refactoring.getBodyMapper();
for(AbstractCodeMapping mapping : mapper.getMappings()) {
AbstractCodeFragment fragment1 = mapping.getFragment1();
AbstractCodeFragment fragment2 = mapping.getFragment2();
Set<Replacement> replacements = mapping.getReplacements();
for(Replacement replacement : replacements) {
String valueBefore = replacement.getBefore();
String valueAfter = replacement.getAfter();
ReplacementType type = replacement.getType();
}
}For the Extract Method Refactoring example shown above mapping.getReplacements() returns the following AST node replacement for the pair of matched statements:
final List<CourseInfo> courseInfos = getFromStepic("courses",CoursesContainer.class).courses;
final List<CourseInfo> courseInfos = coursesContainer.courses;Replacement: getFromStepic("courses",CoursesContainer.class) -> coursesContainer
ReplacementType: VARIABLE_REPLACED_WITH_METHOD_INVOCATION
ExtractOperationRefactoring refactoring = ...;
UMLOperationBodyMapper mapper = refactoring.getBodyMapper();
Set<Refactoring> overlappingRefactorings = mapper.getRefactorings();For the Extract Method Refactoring example shown above mapper.getRefactorings() returns the following refactoring:
Extract Variable coursesContainer : CoursesContainer in method
private addCoursesFromStepic(result List<CourseInfo>, pageNumber int) : boolean
from class com.jetbrains.edu.stepic.EduStepicConnector
because variable coursesContainer = getFromStepic(url,CoursesContainer.class) has been extracted from the following statement of the original method by replacing string literal "courses" with variable url:
final List<CourseInfo> courseInfos = getFromStepic("courses",CoursesContainer.class).courses;When you build a distributable application with ./gradlew distZip, you can run Refactoring Miner as a command line application. Extract the file under build/distribution/RefactoringMiner.zip in the desired location, and cd into the bin folder (or include it in your path). Then, run RefactoringMiner -h to show its usage:
> RefactoringMiner -h
-h Show tips
-a <git-repo-folder> <branch> Detect all refactorings at <branch> for <git-repo-folder>. If <branch> is not specified, commits from all branches are analyzed.
-bc <git-repo-folder> <start-commit-sha1> <end-commit-sha1> Detect refactorings Between <start-commit-sha1> and <end-commit-sha1> for project <git-repo-folder>
-bt <git-repo-folder> <start-tag> <end-tag> Detect refactorings Between <start-tag> and <end-tag> for project <git-repo-folder>
-c <git-repo-folder> <commit-sha1> Detect refactorings at specified commit <commit-sha1> for project <git-repo-folder>
-gc <git-URL> <commit-sha1> <timeout> Detect refactorings at specified commit <commit-sha1> for project <git-URL> within the given <timeout> in seconds. All required information is obtained directly from GitHub using the OAuth token in github-oauth.properties
-gp <git-URL> <pull-request> <timeout> Detect refactorings at specified pull request <pull-request> for project <git-URL> within the given <timeout> in seconds for each commit in the pull request. All required information is obtained directly from GitHub using the OAuth token in github-oauth.propertiesWith a locally cloned repository, run:
> git clone https://github.com/danilofes/refactoring-toy-example.git refactoring-toy-example
> ./RefactoringMiner -c refactoring-toy-example 36287f7c3b09eff78395267a3ac0d7da067863fdIf you don't want to clone locally the repository, run:
> ./RefactoringMiner -gc https://github.com/danilofes/refactoring-toy-example.git 36287f7c3b09eff78395267a3ac0d7da067863fd 10In both cases, you will get the output in JSON format:
{
"commits": [{
"repository": "https://github.com/danilofes/refactoring-toy-example.git",
"sha1": "36287f7c3b09eff78395267a3ac0d7da067863fd",
"url": "https://github.com/danilofes/refactoring-toy-example/commit/36287f7c3b09eff78395267a3ac0d7da067863fd",
"refactorings": [{
"type": "Pull Up Attribute",
"description": "Pull Up Attribute private age : int from class org.animals.Labrador to class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Labrador.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "original attribute declaration",
"codeElement": "age : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "pulled up attribute declaration",
"codeElement": "age : int"
}]
},
{
"type": "Pull Up Attribute",
"description": "Pull Up Attribute private age : int from class org.animals.Poodle to class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Poodle.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "original attribute declaration",
"codeElement": "age : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 5,
"endLine": 5,
"startColumn": 14,
"endColumn": 21,
"codeElementType": "FIELD_DECLARATION",
"description": "pulled up attribute declaration",
"codeElement": "age : int"
}]
},
{
"type": "Pull Up Method",
"description": "Pull Up Method public getAge() : int from class org.animals.Labrador to public getAge() : int from class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Labrador.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "original method declaration",
"codeElement": "public getAge() : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "pulled up method declaration",
"codeElement": "public getAge() : int"
}]
},
{
"type": "Pull Up Method",
"description": "Pull Up Method public getAge() : int from class org.animals.Poodle to public getAge() : int from class org.animals.Dog",
"leftSideLocations": [{
"filePath": "src/org/animals/Poodle.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "original method declaration",
"codeElement": "public getAge() : int"
}],
"rightSideLocations": [{
"filePath": "src/org/animals/Dog.java",
"startLine": 7,
"endLine": 9,
"startColumn": 2,
"endColumn": 3,
"codeElementType": "METHOD_DECLARATION",
"description": "pulled up method declaration",
"codeElement": "public getAge() : int"
}]
}
]
}]
}