[Possible Flaky] Run ftrepo vis_builder test will fail even tho 2.4.0 enable vis by default

[peterzhuamazon]

Hi,

Without stating vis_builder.enabled: true in the osd yml, sometimes we see vis_builder fail the second ftrepo test.

  Running:  core-opensearch-dashboards/opensearch-dashboards/apps/vis_builder/da           (2 of 57)
            shboard.spec.js

  Visualization Builder Dashboard Tests
    ✓ Should have valid visualizations (12857ms)
    1) Should be able to add a visualization

I dont have a full log yet but I do see this issue happens on x64 linux tar, possibly flaky test.

Using this:
opensearch-project/opensearch-build#2649 (comment)

Thanks.

[ashwin-pc]

@peterzhuamazon a test log or any other information here would be useful. I've pushed a fix right after the release to fix a flakey test but based on what i'm seeing here, its hard to tell what the flakey test is.

[peterzhuamazon]

@peterzhuamazon a test log or any other information here would be useful. I've pushed a fix right after the release to fix a flakey test but based on what i'm seeing here, its hard to tell what the flakey test is.

Right now I specifically enabled that setting in config and i dont see any failure yet.
I only see the failure if I dont specify the setting, and it is not always happening.

I will see if I can get a test log next time. Thanks.

[CCongWang]

Hi @peterzhuamazon , do you still see the flakiness? Do we still need to work on this issue?

[CCongWang]

removing v2.5.0 label for now, feel free to add proper labels

[xluo-aws]

@peterzhuamazon Hi, Peter, is this still a problem in main?

[peterzhuamazon]

Hi @xluo-aws

I am not sure about main, but I personally havent seen this for a while in 2.x.

Probably because I specifically enable it here already: https://github.com/opensearch-project/opensearch-build/blob/main/manifests/2.12.0/opensearch-dashboards-2.12.0-test.yml#L14

Anyway, I do not think this is a blocker.

Will close issue.

Thanks!