Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for data structure concurrentHashmap which results in flaky-tests #1

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

219sansim
Copy link
Owner

Overview:
Change concurrentHashMap to Collections.synchronizedMap(new HashMap<>()) as it results in flaky tests for the test com.cerner.common.kafka.consumer.ProcessingKafkaConsumerTest.nextRecord_fairProcessing

Command to reproduce:
mvn -pl common-kafka edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=com.cerner.common.kafka.consumer.ProcessingKafkaConsumerTest#nextRecord_fairProcessing

Result:
The test fails on 4/5 iterations of the Nondex tool

Reason:
According to concurrentHashMap documentation,
"Because the elements of a ConcurrentHashMap are not ordered in any particular way, and may be processed in different orders in different parallel executions, the correctness of supplied functions should not depend on any ordering"

About the tool used to detect the Error:
This test is detected as flaky by the NonDex tool developed by researchers at UIUC.
Flaky tests are tests in software development that produce inconsistent or unreliable results, which can lead to false non-deterministic outcomes of the test case, fixing them is important to both reliable testing and fixing vulnerabilities in the code.

@bjchris32
Copy link

Could you explain why you modified CUT instead of modifying the test to make the assertions pass?
I think it would be better to just modify the assertions, so that it would not affect the original use cases, such as performance in the program.

@219sansim
Copy link
Owner Author

The test depends heavily on ordering and in effect it would be checking nothing (it would destroy the complete logic, in effect we would have to delete the test), we would need to modify code under test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants