Inclusion of Post-Commit Test Cases in Defects4J

[JyotirS20]

Hi,
I was wondering whether Defects4J includes test cases added after the commit that fixed the bug. The Defects4J paper does not explicitly mention this, so are any such test cases available? If yes, could someone please provide links or more information about their inclusion?
Thanks a lot.

[mernst]

The bug-mining README describes how many of the bugs in Defects4J have been collected. It finds a bug-fix commit such that at least one test fails before the commit. In other words, it does not include test cases added after the commit that fixed the bug.

However, there is no guarantee that all bugs were created using that process. In other words, some bugs in Defects4J might include test cases added after the commit that fixed the bug.

We would welcome additions to Defects4J of the form you suggested.

[rjust]

@JyotirS20,

Figure 2 in the Defects4J paper visualizes the bug-mining process.

I believe what you are asking is whether any of the bugs in Defects4J may have tests added in V_n+1 (or V_n+m in general).

Since all bugs in Defects4J are tied to a particular fix commit (the fixed version) and each buggy version is constructed by reintroducing the bug into this fixed version without any changes to the tests, no buggy or fixed version will have tests added after the fix commit.

If you wanted to include additional tests, added after the fix commit, you can use git in the checked-out working directory to do so. Each working directory is a clone of the project repository, so you have access to the entire commit history and can use git to patch the working directory.

To what extent back-ported tests added (much) later after a fix commit would compile and run is probably a question to answer in order to determine whether a naive git checkout or git cherrypick approach is a viable solution.