Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: Add tests to increase coverage for ksample.py and npc.py files. #221

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

TST: Add tests to increase coverage for ksample.py and npc.py files. #221

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
903fdf7
TST: Add tests to increase coverage for ksample.py and npc.py files.
aayshahwork Jul 10, 2023
07aa1f5
Merge branch 'main' into coverage_1
aayshahwork Jul 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions permute/tests/test_ksample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ def test_worms_ksample():
res = k_sample(worms.x, worms.y, stat='one-way anova', reps=1000, seed=1234)
np.testing.assert_array_less(0.006, res[0])
np.testing.assert_array_less(res[0], 0.02)
res2 = k_sample(worms.x, worms.y, stat=one_way_anova, reps=1000, keep_dist=True, seed=1234)
Copy link
Author

@aayshahwork aayshahwork Jul 30, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For testing, I added prints for value of res from line 15 (printed as "Res1:") , and res2 (printed as "Res2:") from line 18. Values show up as following:

Res1: (0.01098901098901099, 17.58271428571427)

Res2: (0.01098901098901099, 17.58271428571427, array([9.28809524e+00, 3.30785714e+00, 1.67714286e-01, 7.86523810e-01,
       1.88938095e+00, 1.10714286e-01, 2.46271429e+00, 5.63095238e-01,
       5.70752381e+00, 1.53693810e+01, 1.91438095e+00, 7.90309524e+00,
       6.19509524e+00, 4.54752381e+00, 4.09342857e+00, 3.04938095e+00,
       1.65152381e+00, 1.15303810e+01, 1.31738095e+00, 7.42752381e+00,
       4.84538095e+00, 6.41071429e+00, 6.76771429e+00, 4.37800000e+00,
       1.02115238e+01, 2.26942857e+00, 1.53510952e+01, 1.22423810e+01,
          :
          :

So assert added in:
line 19: length is 1000 of array being printed in res2
line 20: res and res2 are same.
line 21: see values of array printed in res2.

Also the 'one_way_anova' method is the default method, so res and res2 have same results.

assert len(res2[2]) == 1000
np.testing.assert_array_equal([res[0], res[1]], [res2[0], res2[1]])
np.testing.assert_array_less([8, 2], [res2[2][0], res2[2][1]])


def test_one_way_anova():
Expand Down Expand Up @@ -52,3 +56,7 @@ def test_testosterone_ksample():
assert len(x) == 55
res = bivariate_k_sample(x, group1, group2, reps=5000, seed=5)
np.testing.assert_array_less(res[0], 0.0002)
res2 = bivariate_k_sample(x, group1, group2, reps=5000, stat=two_way_anova, keep_dist=True, seed=5)
Copy link
Author

@aayshahwork aayshahwork Jul 30, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For testing, I added prints of res variable (printed as "Res1: ....") and res2 variable (printed as "Res2: .....").

Res1: (0.0001999600079984003, 0.012142004602000753)
Res2: (0.0001999600079984003, 0.012142004602000753, array([0.00314784, 0.00107214, 0.00423246, ..., 0.00011875, 0.00354231,
       0.00287068]))
.

Value printed are like above. So the asserts work fine that are added below at:
line 60: length of array is 5000.
line 61: values of res and res2 are same.
line 62: assert also works fine.

Also, the stat method 'two_way_anova' is the default method. So res2 has same result.

assert len(res2[2]) == 5000
np.testing.assert_array_equal([res[0], res[1]], [res2[0], res2[1]])
np.testing.assert_array_less([0.002, 0.0001], [res2[2][0], res2[2][1]])
26 changes: 19 additions & 7 deletions permute/tests/test_npc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ def test_sim_npc():
# test Y always greater than X so p-value should be 1
responses = np.array([[0, 1], [0, 1], [1, 2], [1, 2]])
group = np.array([1, 1, 2, 2])
my_randomizer = Experiment.Randomizer(randomize = randomize_group, seed = prng)
my_randomizer = Experiment.Randomizer(randomize=randomize_group, seed=prng)
data = Experiment(group, responses)

# create median test statistic to apply to every column
Expand All @@ -137,22 +137,34 @@ def med_diff(data, resp_index):
test_array = Experiment.make_test_array(med_diff, [0, 1])
res = sim_npc(data, test_array, combine="fisher", seed=None, reps=int(1000))
np.testing.assert_almost_equal(res[0], 1)

res = sim_npc(data, test_array, combine="liptak", seed=1234, reps=int(1000))
np.testing.assert_almost_equal(res[0], 1)
res = sim_npc(data, test_array, combine=tippett, in_place=True, seed=None, reps=int(1000))
np.testing.assert_almost_equal(res[0], 1)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In above code, I added prints for fisher, liptak, tippet combine methods.
So res in line 138 is printed as "Res1:".
res in line 140 is printed as "Res2:".
res in line 142 is printed as "Res3:".

Res1: (1.0, {0: -1.0, 1: -1.0}, {0: 1.0, 1: 1.0})
Res2: (1.0, {0: -1.0, 1: -1.0}, {0: 1.0, 1: 1.0})
Res3: (1.0, {0: -1.0, 1: -1.0}, {0: 1.0, 1: 1.0})
.

Have added the code that I used to print:

    print("")
    print("Res1: {}".format(res))
    np.testing.assert_almost_equal(res[0], 1)
    res = sim_npc(data, test_array, combine="liptak", seed=1234, reps=int(1000))
    print("Res2: {}".format(res))
    np.testing.assert_almost_equal(res[0], 1)
    res = sim_npc(data, test_array, combine=tippett, in_place=True, seed=None, reps=int(1000))
    print("Res3: {}".format(res))
    np.testing.assert_almost_equal(res[0], 1)

# test X = Y so p-value should be 1
responses = np.array([[0, 1], [0, 1], [0, 1], [0, 1]])
group = np.array([1, 1, 2, 2])
data = Experiment(group, responses, randomizer = my_randomizer)
res = sim_npc(data, test = Experiment.make_test_array(Experiment.TestFunc.mean_diff, [0, 1]),
data = Experiment(group, responses, randomizer=my_randomizer)
res = sim_npc(data, test=Experiment.make_test_array(Experiment.TestFunc.mean_diff, [0, 1]),
combine="fisher", seed=None, reps=int(1000))
np.testing.assert_almost_equal(res[0], 1)

# test stat for cat_1 is smaller if X all 0s which about 0.015 chance so pvalue should be about 0.985
responses = np.array([[0, 1], [1, 1], [0, 1], [0, 1], [1, 1], [1, 1], [1, 1], [0, 1]])
group = np.array([1, 1, 1, 1, 2, 2, 2, 2])
data = Experiment(group, responses, randomizer = my_randomizer)
res = sim_npc(data, test = Experiment.make_test_array(Experiment.TestFunc.mean_diff, [0, 1]),
data = Experiment(group, responses, randomizer=my_randomizer)
res = sim_npc(data, test=Experiment.make_test_array(Experiment.TestFunc.mean_diff, [0, 1]),
combine="fisher", seed=None, reps=int(1000))
np.testing.assert_almost_equal(res[0], 0.985, decimal=2)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only white-space changes in this block of code. No real code change.


# test stat for cat_1 is smaller if X all 0s which about 0.015 chance so pvalue should be about 0.0209
responses = np.array([[0, 1], [1, 1], [0, 1], [0, 1], [1, 1], [1, 1], [1, 1], [0, 1]])
group = np.array([1, 1, 1, 1, 2, 2, 2, 2])
data = Experiment(group, responses, randomizer=my_randomizer)
res = sim_npc(data, test=Experiment.make_test_array(Experiment.TestFunc.one_way_anova, [0, 1]),
combine="fisher", seed=None, reps=int(1000))
np.testing.assert_almost_equal(res[0], 0.985, decimal = 2)
np.testing.assert_almost_equal(res[0], 0.0209, decimal=2)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, at line 165 - have changed the test function to use 'one_way_anova' method. And I printed the value of 'res' as returned from 'sim_npc()' function.

Res:(0.02097902097902098, {0: 0.5, 1: 0.0}, {0: 0.48151848151848153, 1: 1.0})

Code used to print above value:

    res = sim_npc(data, test=Experiment.make_test_array(Experiment.TestFunc.one_way_anova, [0, 1]),
                  combine="fisher", seed=None, reps=int(1000))
    print("")
    print("Res:{}".format(res))
    np.testing.assert_almost_equal(res[0], 0.0209, decimal=2)



def test_westfall_young():
Expand Down