[BUG] Flaky test org.opensearch.remotestore.RemoteStoreIT.testRemoteStoreRestoreFromRemoteSegmentStore

[sachinpkale]

          > ### Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/11025/
• CommitID: a781dc0
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

#6248

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.remotestore.RemoteStoreIT.testRemoteSegmentStoreRestore" -Dtests.seed=6C75DC5D95F409A0 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ar-LY -Dtests.timezone=Asia/Chungking -Druntime.java=17

org.opensearch.remotestore.RemoteStoreIT > testRemoteSegmentStoreRestore FAILED
    java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([6C75DC5D95F409A0:F88BD9C4AFBD1A70]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:1007)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:938)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:927)
        at org.opensearch.remotestore.RemoteStoreIT.testRemoteSegmentStoreRestore(RemoteStoreIT.java:165)

Originally posted by @dreamer-89 in #6288 (comment)

[sachinpkale]

@dreamer-89 Created new issue to track the test failure.

[sachinpkale]

Not able to reproduce on local with the same seed. Going ahead with muting the test.

[sachinpkale]

Fixed in #6375