Skip to content

Commit

Permalink
Continuing README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
jaltmayerpizzorno authored Jan 31, 2024
1 parent 88df126 commit b14211b
Showing 1 changed file with 12 additions and 2 deletions.
14 changes: 12 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ CoverUp is available on PyPI, so you can install simply with
python3 -m pip install coverup
```

CoverUp currently requires an [OpenAI account](https://platform.openai.com/signup) to run.
While work is underway to support local LLMs, CoverUp currently requires an [OpenAI account](https://platform.openai.com/signup) to run.
Your account will also need to have a [positive balance](https://platform.openai.com/account/usage).
Create an [API key](https://platform.openai.com/api-keys) and store its "secret key" (usually a
string starting with `sk-`) in an environment variable named `OPENAI_API_KEY`:
Expand Down Expand Up @@ -57,11 +57,21 @@ to fail and disabled it.
That test remains as `disabled_test_coverup_19.py`, where it can be reviewed for the cause
and possibly re-added to the suite.

### Best Used With Docker
### Better With Docker
To evaluate the tests generated by the LLM, CoverUp must execute them.
To minimize security concerns, including the risk of damage to your system, we recommend
that you run CoverUp using [Docker](https://www.docker.com/).

## Evaluation
<img src="images/comparison.png?raw=True" align="right" width="65%"/>

The graph shows CoverUp in comparison to the state-of-the-art [CodaMosa](https://www.carolemieux.com/codamosa_icse23.pdf),
which improves upon the [Pynguin](https://github.com/se2p/pynguin) test generator by incorporating LLM queries.
The bars show the difference in coverage percentage between CoverUp and CodaMosa for various Python modules;
green bars, above 0, indicate that CoverUp achieved a higher coverage.
As the graph shows, CoverUp achieves better coverage in almost every case.
Both CoverUp and CodaMosa created tests "from scratch", that is, ignoring any existing test suite.

## Work In Progress
Please enjoy this early release of CoverUp, and pardon any disruptions while we actively develop it.

Expand Down

0 comments on commit b14211b

Please sign in to comment.