JustinStr

JustinStr is a tool for automatically generating string test data.

To get the latest version the tool, please go to the the JustinStr-New project under development.

1. How to use JustinStr

• We support running the tool in the command line. After downloading the jar package (JustinStr/tool/JustinStr_jar), you can run that with

java -jar JustinStr.jar "jrePath" "targetPath"

​ The "jrePath" is the default java environment on your machine, such as "/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre".

​ The "targetPath" is the path to the jar or class file you will analyze.

• JustinStr will output two json files.

  The file "regex.json" expresses the constraints of the string parameters with regular expressions.

  And the file "stringData.json" generates a string according to the regular expression in "regex.json".

• In addition, we are only running in Mac and Ubuntu environment, and the java version used is 1.8. Therefore, we cannot guarantee that it will work in other environments.

2. Overview

3. API-Regex Mapping

We map a regular expression for each of the 48 string APIs and their conditional expressions.

All the specific correspondence can be viewed in the table(JustinStr/API-Regex-Mapping.pdf).

The table below shows only part of the mapping table.

4. Experiment

The benchmark information can be avaiable in the directory "JustinStr/experiment/".

4.1 DataSets

4.2 RQ1: How effectively does JustinStr characterize the input string parameter?

In summary, JustinStr can effectively characterize the input string parameter with a regular expression based on its API-Regex mapping and the inference algorithm for both single string API invocation and their combination usage.

4.3 RQ2: Is the string data generated by JustinStr helpful to improve the branch coverage in programs?

JustinStr is effective for the improvement of the branch coverage rate by about 22%(40/186) of the methods, which can be improved by up to 57%, and 17% on average.

4.4 RQ3: How does JustinStr perform on finding bugs?

classification	IssueID(Commit ID) in JDK
ArrayIndexoutofBoundsException	I4MWI1, 8279422
StringIndexOutOfBoundsException	8278186, 8279129, 8279128, 8279198, 8279218, 8279336, 8279342, 8279341, 8279362, 8279423, 8279424, 0ccabf89898e216c0bd828bfde840338baba7d11(Commit ID), 96a1333c77a8473d9ae06f304b9fdbe21212bd18(Commit ID), e67420cefce8c5c26274b02a5430d31411a404a9(Commit ID)
Infinite Loop	8278993

Details for the Commit ID:

• https://github.com/openjdk/nashorn/commit/0ccabf89898e216c0bd828bfde840338baba7d11

  

• https://github.com/openjdk/jdk/commit/96a1333c77a8473d9ae06f304b9fdbe21212bd18

  The origin test case is:

  

  

• https://github.com/openjdk/jdk/commit/e67420cefce8c5c26274b02a5430d31411a404a9

  

Our approach can effectively characterize string parameters with regular expressions. The test cases generated based on it has efficiently found 81(+28%) more defects with an extreme low cost of time. Besides, the first two test cases in JustinStr have triggered 74% of the bugs, while in EvoSuite have triggered 36%. JustinStr has found 14 new bugs on JDK which are all confirmed by the JDK devepolers.