[updatecli] update elastic stack version for testing 8.13.0-ybxdr713 #37502

apmmachine · 2023-12-25T15:09:04Z

Generated automatically with https://github.com/elastic/beats/actions/runs/7411199577

Bump elastic-stack to latest snapshot version

Update snapshot.yml

1 file(s) updated with "$1:8.13.0-p7cya7tg-SNAPSHOT": * testing/environments/snapshot.yml

	Created automatically by Updatecli Options: Most of Updatecli configuration is done via its manifest(s). If you close this pull request, Updatecli will automatically reopen it, the next time it runs. If you close this pull request and delete the base branch, Updatecli will automatically recreate it, erasing all previous commits made. Feel free to report any issues at github.com/updatecli/updatecli. If you find this tool useful, do not hesitate to star our GitHub repository as a sign of appreciation, and/or to tell us directly on our chat!

Made with ❤️️ by updatecli

elasticmachine · 2023-12-25T17:34:54Z

💚 Build Succeeded

Expand to view the summary

Build stats

Start Time: 2023-12-25T15:09:19.290+0000
Duration: 145 min 20 sec

Test stats 🧪

Test	Results
Failed	0
Passed	28732
Skipped	2015
Total	30747

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

mergify

Automatically approving mergify

Made with ❤️️ by updatecli

elasticmachine · 2023-12-26T17:22:05Z

💚 Build Succeeded

Expand to view the summary

Build stats

Start Time: 2023-12-26T15:09:11.768+0000
Duration: 133 min 38 sec

Test stats 🧪

Test	Results
Failed	0
Passed	28732
Skipped	2015
Total	30747

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Made with ❤️️ by updatecli

elasticmachine · 2023-12-27T17:24:14Z

💚 Build Succeeded

Expand to view the summary

Build stats

Duration: 134 min 25 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Made with ❤️️ by updatecli

elasticmachine · 2023-12-28T17:22:03Z

💚 Build Succeeded

Expand to view the summary

Build stats

Duration: 132 min 5 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Made with ❤️️ by updatecli

elasticmachine · 2023-12-29T17:20:24Z

💚 Build Succeeded

Expand to view the summary

Build stats

Start Time: 2023-12-29T15:09:17.402+0000
Duration: 131 min 52 sec

Test stats 🧪

Test	Results
Failed	0
Passed	28732
Skipped	2015
Total	30747

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Made with ❤️️ by updatecli

elasticmachine · 2024-01-01T15:18:30Z

❕ Build Aborted

Expand to view the summary

Build stats

Start Time: 2024-01-01T12:07:18.453+0000
Duration: 191 min 56 sec

Test stats 🧪

Test	Results
Failed	12
Passed	28720
Skipped	2015
Total	30747

Test errors

Expand to view the tests failures

> Show only the first 10 test failures

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}}, full object: 
  {'log.offset': 974, 'log.level': 'informational', 'destination.geo.continent_name': 'Europe', 'destination.geo.region_iso_code': 'GB-ENG', 'destination.geo.city_name': 'London', 'destination.geo.country_iso_code': 'GB', 'destination.geo.country_name': 'United Kingdom', 'destination.geo.location.lon': -0.0931, 'destination.geo.location.lat': 51.5142, 'destination.geo.region_name': 'England', 'destination.address': '81.2.69.142', 'destination.port': 62409, 'destination.ip': '81.2.69.142', 'source.address': 'exp-wait', 'source.port': 62409, 'source.domain': 'exp-wait', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'network.transport': 'udp', 'network.iana_number': 17, 'input.type': 'log', 'observer.ingress.interface.name': 'SERVERS', 'observer.hostname': '216.160.83.56', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'observer.egress.interface.name': 'outside', 'related.hosts': ['216.160.83.56', 'exp-wait'], 'related.ip': ['81.2.69.142'], 'service.type': 'cisco', 'host.hostname': '216.160.83.56', 'event.severity': 6, 'event.code': 305012, 'event.original': '%ASA-6-305012: Teardown dynamic UDP translation from SERVERS:exp-wait/62409 to outside:81.2.69.142/62409 duration 0:00:41', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-07-15T15:38:06.000Z', 'event.type': ['connection', 'end'], 'event.duration': 41000000000, 'event.action': 'flow-expiration', 'event.end': '2024-07-15T13:38:47.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.destination_interface': 'outside', 'cisco.asa.source_interface': 'SERVERS', 'cisco.asa.message_id': '305012'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06...'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_290_cisco>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_290_cisco>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cisco/asa/test/non-canonical.log'
objects = [{'@timestamp': '2024-07-15T13:38:14.000-02:00', 'agent': {'ephemeral_id': '7f85ed35-8b99-4130-9c11-eeb7518d4e29', 'id...ource_interface': 'outside'}}, 'destination': {'address': 'sh-mailgw1', 'domain': 'sh-mailgw1', 'port': 25}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}}, full object: 
E             {'log.offset': 974, 'log.level': 'informational', 'destination.geo.continent_name': 'Europe', 'destination.geo.region_iso_code': 'GB-ENG', 'destination.geo.city_name': 'London', 'destination.geo.country_iso_code': 'GB', 'destination.geo.country_name': 'United Kingdom', 'destination.geo.location.lon': -0.0931, 'destination.geo.location.lat': 51.5142, 'destination.geo.region_name': 'England', 'destination.address': '81.2.69.142', 'destination.port': 62409, 'destination.ip': '81.2.69.142', 'source.address': 'exp-wait', 'source.port': 62409, 'source.domain': 'exp-wait', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'network.transport': 'udp', 'network.iana_number': 17, 'input.type': 'log', 'observer.ingress.interface.name': 'SERVERS', 'observer.hostname': '216.160.83.56', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'observer.egress.interface.name': 'outside', 'related.hosts': ['216.160.83.56', 'exp-wait'], 'related.ip': ['81.2.69.142'], 'service.type': 'cisco', 'host.hostname': '216.160.83.56', 'event.severity': 6, 'event.code': 305012, 'event.original': '%ASA-6-305012: Teardown dynamic UDP translation from SERVERS:exp-wait/62409 to outside:81.2.69.142/62409 duration 0:00:41', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-07-15T15:38:06.000Z', 'event.type': ['connection', 'end'], 'event.duration': 41000000000, 'event.action': 'flow-expiration', 'event.end': '2024-07-15T13:38:47.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.destination_interface': 'outside', 'cisco.asa.source_interface': 'SERVERS', 'cisco.asa.message_id': '305012'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06...'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}}, full object: 
  {'log.offset': 481, 'log.level': 'debug', 'source.address': '192.168.2.2', 'source.ip': '192.168.2.2', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'input.type': 'log', 'observer.ingress.interface.name': 'net', 'observer.hostname': 'dev01', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'related.hosts': ['dev01'], 'related.ip': ['192.168.2.2'], 'service.type': 'cisco', 'host.hostname': 'dev01', 'event.severity': 7, 'event.code': 609002, 'event.original': '%FTD-7-609002: Teardown local-host net:192.168.2.2 duration 0:00:00', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-05-05T19:51:17.000Z', 'event.type': ['connection', 'end'], 'event.duration': 0, 'event.action': 'flow-expiration', 'event.end': '2024-05-05T17:51:17.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.source_interface': 'net', 'cisco.asa.message_id': '609002'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17...'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_294_cisco>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_294_cisco>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cisco/asa/test/additional_messages.log'
objects = [{'@timestamp': '2024-05-05T17:51:17.000-02:00', 'agent': {'ephemeral_id': '8fb6d86c-5af4-40e7-a3e1-9b43e718dac1', 'id...': '175.16.199.1', 'message_id': '302020'}}, 'destination': {'address': '10.10.10.10', 'ip': '10.10.10.10'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}}, full object: 
E             {'log.offset': 481, 'log.level': 'debug', 'source.address': '192.168.2.2', 'source.ip': '192.168.2.2', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'input.type': 'log', 'observer.ingress.interface.name': 'net', 'observer.hostname': 'dev01', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'related.hosts': ['dev01'], 'related.ip': ['192.168.2.2'], 'service.type': 'cisco', 'host.hostname': 'dev01', 'event.severity': 7, 'event.code': 609002, 'event.original': '%FTD-7-609002: Teardown local-host net:192.168.2.2 duration 0:00:00', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-05-05T19:51:17.000Z', 'event.type': ['connection', 'end'], 'event.duration': 0, 'event.action': 'flow-expiration', 'event.end': '2024-05-05T17:51:17.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.source_interface': 'net', 'cisco.asa.message_id': '609002'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17...'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_305_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_305_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/357_monitor_fw_rules_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'c15213a4-4317-43d5-83e5-b97b471656d8', 'id': 'e...tor FW rules start', 'issuer': 'Batch', 'message': 'Monitor FW rules start', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}}, full object: 
  {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_313_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_313_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/59_clear_safe_history.log'
objects = [{'@timestamp': '2021-03-04T19:25:02.000Z', 'agent': {'ephemeral_id': '9661ac82-300d-4973-a775-e981c9a12f3a', 'id': 'e...r Safe History', 'iso_timestamp': '2021-03-09T09:00:47Z', 'issuer': 'Batch', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}}, full object: 
E             {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_325_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_325_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/310_monitor_dr_replication_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'ceba0996-faab-4ba1-b1d8-5b744f361dec', 'id': 'e...cation start', 'issuer': 'Batch', 'message': 'Monitor DR Replication start', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_332_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_332_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/358_monitor_fw_rules_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'edb97f9f-4354-4f16-a6d6-b6e4a19a653f', 'id': 'e...Monitor FW Rules end', 'issuer': 'Batch', 'message': 'Monitor FW Rules end', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}}, full object: 
  {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_335_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_335_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/479_security_warning_the_signature_hash_algorithm_of_the_vault_certificate_is_sha1.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'e42ad4ec-1413-4da6-bb55-3ada12cd1055', 'id': 'e...y warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}}, full object: 
E             {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}}, full object: 
  {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_346_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_346_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/legacysyslog.log'
objects = [{'@timestamp': '2024-03-08T03:41:01.000-02:00', 'agent': {'ephemeral_id': '60ca615b-7586-479e-94df-41d3330568ee', 'id...: 'Root\\Policies\\Policy-BusinessWebsite.ini', 'issuer': 'PasswordManager', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}}, full object: 
E             {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_351_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_351_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/311_monitor_dr_replication_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '84fd50b9-c12e-44e2-b8f0-3e0a29892d4f', 'id': 'e...eplication end', 'issuer': 'Batch', 'message': 'Monitor DR Replication end', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}}, full object: 
  {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_353_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_353_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-6-30280c8d-713f-44f8-a229-7dbc3416b26c/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/88_set_password.log'
objects = [{'@timestamp': '2021-03-04T19:16:19.000Z', 'agent': {'ephemeral_id': 'b67badd6-0a14-4075-9a39-ffbe2566c8e4', 'id': 'e...mp': '2021-03-10T09:11:52Z', 'issuer': 'PSMPApp_localhost.localdomain', ...}}, 'ecs': {'version': '1.12.0'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}}, full object: 
E             {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

Steps errors

Expand to view the steps failures

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 104 min 57 sec . View more details here
Description: mage pythonIntegTest

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'org.jenkinsci.plugins.workflow.steps.FlowInterruptedException'

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-01-01T20:44:28Z

💔 Tests Failed

Expand to view the summary

Build stats

Start Time: 2024-01-01T15:09:25.750+0000
Duration: 335 min 45 sec

Test stats 🧪

Test	Results
Failed	12
Passed	28720
Skipped	2015
Total	30747

Test errors

Expand to view the tests failures

> Show only the first 10 test failures

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}, "root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}}}, full object: 
  {'log.offset': 974, 'log.level': 'informational', 'destination.geo.continent_name': 'Europe', 'destination.geo.region_iso_code': 'GB-ENG', 'destination.geo.city_name': 'London', 'destination.geo.country_iso_code': 'GB', 'destination.geo.country_name': 'United Kingdom', 'destination.geo.location.lon': -0.0931, 'destination.geo.location.lat': 51.5142, 'destination.geo.region_name': 'England', 'destination.address': '81.2.69.142', 'destination.port': 62409, 'destination.ip': '81.2.69.142', 'source.address': 'exp-wait', 'source.port': 62409, 'source.domain': 'exp-wait', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'network.transport': 'udp', 'network.iana_number': 17, 'input.type': 'log', 'observer.ingress.interface.name': 'SERVERS', 'observer.hostname': '216.160.83.56', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'observer.egress.interface.name': 'outside', 'related.hosts': ['216.160.83.56', 'exp-wait'], 'related.ip': ['81.2.69.142'], 'service.type': 'cisco', 'host.hostname': '216.160.83.56', 'event.severity': 6, 'event.code': 305012, 'event.original': '%ASA-6-305012: Teardown dynamic UDP translation from SERVERS:exp-wait/62409 to outside:81.2.69.142/62409 duration 0:00:41', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-07-15T15:38:06.000Z', 'event.type': ['connection', 'end'], 'event.duration': 41000000000, 'event.action': 'flow-expiration', 'event.end': '2024-07-15T13:38:47.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.destination_interface': 'outside', 'cisco.asa.source_interface': 'SERVERS', 'cisco.asa.message_id': '305012'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38...00-02:00'}, "root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_290_cisco>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_290_cisco>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cisco/asa/test/non-canonical.log'
objects = [{'@timestamp': '2024-07-15T13:38:14.000-02:00', 'agent': {'ephemeral_id': '42c8cb7c-9160-4a1c-b6ee-bd442677e6a0', 'id...ource_interface': 'outside'}}, 'destination': {'address': 'sh-mailgw1', 'domain': 'sh-mailgw1', 'port': 25}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}, "root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}}}, full object: 
E             {'log.offset': 974, 'log.level': 'informational', 'destination.geo.continent_name': 'Europe', 'destination.geo.region_iso_code': 'GB-ENG', 'destination.geo.city_name': 'London', 'destination.geo.country_iso_code': 'GB', 'destination.geo.country_name': 'United Kingdom', 'destination.geo.location.lon': -0.0931, 'destination.geo.location.lat': 51.5142, 'destination.geo.region_name': 'England', 'destination.address': '81.2.69.142', 'destination.port': 62409, 'destination.ip': '81.2.69.142', 'source.address': 'exp-wait', 'source.port': 62409, 'source.domain': 'exp-wait', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'network.transport': 'udp', 'network.iana_number': 17, 'input.type': 'log', 'observer.ingress.interface.name': 'SERVERS', 'observer.hostname': '216.160.83.56', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'observer.egress.interface.name': 'outside', 'related.hosts': ['216.160.83.56', 'exp-wait'], 'related.ip': ['81.2.69.142'], 'service.type': 'cisco', 'host.hostname': '216.160.83.56', 'event.severity': 6, 'event.code': 305012, 'event.original': '%ASA-6-305012: Teardown dynamic UDP translation from SERVERS:exp-wait/62409 to outside:81.2.69.142/62409 duration 0:00:41', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-07-15T15:38:06.000Z', 'event.type': ['connection', 'end'], 'event.duration': 41000000000, 'event.action': 'flow-expiration', 'event.end': '2024-07-15T13:38:47.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.destination_interface': 'outside', 'cisco.asa.source_interface': 'SERVERS', 'cisco.asa.message_id': '305012'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38...00-02:00'}, "root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}, "root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}}}, full object: 
  {'log.offset': 481, 'log.level': 'debug', 'source.address': '192.168.2.2', 'source.ip': '192.168.2.2', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'input.type': 'log', 'observer.ingress.interface.name': 'net', 'observer.hostname': 'dev01', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'related.hosts': ['dev01'], 'related.ip': ['192.168.2.2'], 'service.type': 'cisco', 'host.hostname': 'dev01', 'event.severity': 7, 'event.code': 609002, 'event.original': '%FTD-7-609002: Teardown local-host net:192.168.2.2 duration 0:00:00', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-05-05T19:51:17.000Z', 'event.type': ['connection', 'end'], 'event.duration': 0, 'event.action': 'flow-expiration', 'event.end': '2024-05-05T17:51:17.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.source_interface': 'net', 'cisco.asa.message_id': '609002'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51...00-02:00'}, "root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_294_cisco>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_294_cisco>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cisco/asa/test/additional_messages.log'
objects = [{'@timestamp': '2024-05-05T17:51:17.000-02:00', 'agent': {'ephemeral_id': '6ec4b916-3958-476a-bad3-08ff8dd68cd3', 'id...': '175.16.199.1', 'message_id': '302020'}}, 'destination': {'address': '10.10.10.10', 'ip': '10.10.10.10'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}, "root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}}}, full object: 
E             {'log.offset': 481, 'log.level': 'debug', 'source.address': '192.168.2.2', 'source.ip': '192.168.2.2', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'input.type': 'log', 'observer.ingress.interface.name': 'net', 'observer.hostname': 'dev01', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'related.hosts': ['dev01'], 'related.ip': ['192.168.2.2'], 'service.type': 'cisco', 'host.hostname': 'dev01', 'event.severity': 7, 'event.code': 609002, 'event.original': '%FTD-7-609002: Teardown local-host net:192.168.2.2 duration 0:00:00', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-05-05T19:51:17.000Z', 'event.type': ['connection', 'end'], 'event.duration': 0, 'event.action': 'flow-expiration', 'event.end': '2024-05-05T17:51:17.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.source_interface': 'net', 'cisco.asa.message_id': '609002'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51...00-02:00'}, "root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_305_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_305_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/357_monitor_fw_rules_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '97e80034-0926-40da-a442-4dc1512ff437', 'id': 'a...tor FW rules start', 'issuer': 'Batch', 'message': 'Monitor FW rules start', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}}, full object: 
  {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_313_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_313_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/59_clear_safe_history.log'
objects = [{'@timestamp': '2021-03-04T19:25:02.000Z', 'agent': {'ephemeral_id': 'b20e2f26-c32c-4e70-a5cf-769ea74c3d9b', 'id': 'a...r Safe History', 'iso_timestamp': '2021-03-09T09:00:47Z', 'issuer': 'Batch', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}}, full object: 
E             {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_325_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_325_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/310_monitor_dr_replication_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'c11b509c-48b4-4372-b8a1-5631c96d4255', 'id': 'a...cation start', 'issuer': 'Batch', 'message': 'Monitor DR Replication start', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_332_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_332_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/358_monitor_fw_rules_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '0c137355-40b1-4421-94a0-5159289344f9', 'id': 'a...Monitor FW Rules end', 'issuer': 'Batch', 'message': 'Monitor FW Rules end', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}}, full object: 
  {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_335_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_335_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/479_security_warning_the_signature_hash_algorithm_of_the_vault_certificate_is_sha1.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'c5378701-15fe-4ada-8f7b-6120149a52c0', 'id': 'a...y warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}}, full object: 
E             {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}}, full object: 
  {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_346_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_346_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/legacysyslog.log'
objects = [{'@timestamp': '2024-03-08T03:41:01.000-02:00', 'agent': {'ephemeral_id': '935e6ad0-f360-47ba-921c-76df4b465d00', 'id...: 'Root\\Policies\\Policy-BusinessWebsite.ini', 'issuer': 'PasswordManager', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}}, full object: 
E             {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_351_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_351_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/311_monitor_dr_replication_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': 'e422b03d-365e-4fcb-812c-bca7e74540f5', 'id': 'a...eplication end', 'issuer': 'Batch', 'message': 'Monitor DR Replication end', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}}, full object: 
  {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_353_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_353_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-7-0c9b003b-22ab-49e1-90fb-d30f5a10fc21/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/88_set_password.log'
objects = [{'@timestamp': '2021-03-04T19:16:19.000Z', 'agent': {'ephemeral_id': 'e7cbd49e-53dd-467b-97e0-1422616dfa03', 'id': 'a...mp': '2021-03-10T09:11:52Z', 'issuer': 'PSMPApp_localhost.localdomain', ...}}, 'ecs': {'version': '1.12.0'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}}, full object: 
E             {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

Steps errors

Expand to view the steps failures

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 104 min 49 sec . View more details here
Description: mage pythonIntegTest

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 100 min 18 sec . View more details here
Description: mage pythonIntegTest

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 100 min 22 sec . View more details here
Description: mage pythonIntegTest

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_354_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_379_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Made with ❤️️ by updatecli

elasticmachine · 2024-01-02T20:44:41Z

💔 Build Failed

Expand to view the summary

Build stats

Duration: 335 min 11 sec

Pipeline error

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-01-03T02:27:41Z

💔 Tests Failed

Expand to view the summary

Build stats

Start Time: 2024-01-02T20:49:56.024+0000
Duration: 337 min 28 sec

Test stats 🧪

Test	Results
Failed	12
Passed	27332
Skipped	2004
Total	29348

Test errors

Expand to view the tests failures

> Show only the first 10 test failures

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}}, full object: 
  {'log.offset': 974, 'log.level': 'informational', 'destination.geo.continent_name': 'Europe', 'destination.geo.region_iso_code': 'GB-ENG', 'destination.geo.city_name': 'London', 'destination.geo.country_iso_code': 'GB', 'destination.geo.country_name': 'United Kingdom', 'destination.geo.location.lon': -0.0931, 'destination.geo.location.lat': 51.5142, 'destination.geo.region_name': 'England', 'destination.address': '81.2.69.142', 'destination.port': 62409, 'destination.ip': '81.2.69.142', 'source.address': 'exp-wait', 'source.port': 62409, 'source.domain': 'exp-wait', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'network.transport': 'udp', 'network.iana_number': 17, 'input.type': 'log', 'observer.ingress.interface.name': 'SERVERS', 'observer.hostname': '216.160.83.56', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'observer.egress.interface.name': 'outside', 'related.hosts': ['216.160.83.56', 'exp-wait'], 'related.ip': ['81.2.69.142'], 'service.type': 'cisco', 'host.hostname': '216.160.83.56', 'event.severity': 6, 'event.code': 305012, 'event.original': '%ASA-6-305012: Teardown dynamic UDP translation from SERVERS:exp-wait/62409 to outside:81.2.69.142/62409 duration 0:00:41', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-07-15T15:38:06.000Z', 'event.type': ['connection', 'end'], 'event.duration': 41000000000, 'event.action': 'flow-expiration', 'event.end': '2024-07-15T13:38:47.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.destination_interface': 'outside', 'cisco.asa.source_interface': 'SERVERS', 'cisco.asa.message_id': '305012'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06...'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_290_cisco>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_290_cisco>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cisco/asa/test/non-canonical.log'
objects = [{'@timestamp': '2024-07-15T13:38:14.000-02:00', 'agent': {'ephemeral_id': 'c2304c6e-2985-47db-a670-3f7decff7a3d', 'id...ource_interface': 'outside'}}, 'destination': {'address': 'sh-mailgw1', 'domain': 'sh-mailgw1', 'port': 25}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06.000Z'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}}, full object: 
E             {'log.offset': 974, 'log.level': 'informational', 'destination.geo.continent_name': 'Europe', 'destination.geo.region_iso_code': 'GB-ENG', 'destination.geo.city_name': 'London', 'destination.geo.country_iso_code': 'GB', 'destination.geo.country_name': 'United Kingdom', 'destination.geo.location.lon': -0.0931, 'destination.geo.location.lat': 51.5142, 'destination.geo.region_name': 'England', 'destination.address': '81.2.69.142', 'destination.port': 62409, 'destination.ip': '81.2.69.142', 'source.address': 'exp-wait', 'source.port': 62409, 'source.domain': 'exp-wait', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'network.transport': 'udp', 'network.iana_number': 17, 'input.type': 'log', 'observer.ingress.interface.name': 'SERVERS', 'observer.hostname': '216.160.83.56', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'observer.egress.interface.name': 'outside', 'related.hosts': ['216.160.83.56', 'exp-wait'], 'related.ip': ['81.2.69.142'], 'service.type': 'cisco', 'host.hostname': '216.160.83.56', 'event.severity': 6, 'event.code': 305012, 'event.original': '%ASA-6-305012: Teardown dynamic UDP translation from SERVERS:exp-wait/62409 to outside:81.2.69.142/62409 duration 0:00:41', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-07-15T15:38:06.000Z', 'event.type': ['connection', 'end'], 'event.duration': 41000000000, 'event.action': 'flow-expiration', 'event.end': '2024-07-15T13:38:47.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.destination_interface': 'outside', 'cisco.asa.source_interface': 'SERVERS', 'cisco.asa.message_id': '305012'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-07-15T15:38:06.000Z', 'old_value': '2023-07-15T15:38:06...'}, "root['event.end']": {'new_value': '2024-07-15T13:38:47.000-02:00', 'old_value': '2023-07-15T13:38:47.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}}, full object: 
  {'log.offset': 481, 'log.level': 'debug', 'source.address': '192.168.2.2', 'source.ip': '192.168.2.2', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'input.type': 'log', 'observer.ingress.interface.name': 'net', 'observer.hostname': 'dev01', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'related.hosts': ['dev01'], 'related.ip': ['192.168.2.2'], 'service.type': 'cisco', 'host.hostname': 'dev01', 'event.severity': 7, 'event.code': 609002, 'event.original': '%FTD-7-609002: Teardown local-host net:192.168.2.2 duration 0:00:00', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-05-05T19:51:17.000Z', 'event.type': ['connection', 'end'], 'event.duration': 0, 'event.action': 'flow-expiration', 'event.end': '2024-05-05T17:51:17.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.source_interface': 'net', 'cisco.asa.message_id': '609002'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17...'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_294_cisco>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_294_cisco>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cisco/asa/test/additional_messages.log'
objects = [{'@timestamp': '2024-05-05T17:51:17.000-02:00', 'agent': {'ephemeral_id': 'a94a7b57-bf99-4cdf-8f68-147d94a42c21', 'id...': '175.16.199.1', 'message_id': '302020'}}, 'destination': {'address': '10.10.10.10', 'ip': '10.10.10.10'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17.000Z'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}}, full object: 
E             {'log.offset': 481, 'log.level': 'debug', 'source.address': '192.168.2.2', 'source.ip': '192.168.2.2', 'fileset.name': 'asa', 'tags': ['cisco-asa', 'forwarded'], 'input.type': 'log', 'observer.ingress.interface.name': 'net', 'observer.hostname': 'dev01', 'observer.product': 'asa', 'observer.vendor': 'Cisco', 'observer.type': 'firewall', 'related.hosts': ['dev01'], 'related.ip': ['192.168.2.2'], 'service.type': 'cisco', 'host.hostname': 'dev01', 'event.severity': 7, 'event.code': 609002, 'event.original': '%FTD-7-609002: Teardown local-host net:192.168.2.2 duration 0:00:00', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cisco', 'event.start': '2024-05-05T19:51:17.000Z', 'event.type': ['connection', 'end'], 'event.duration': 0, 'event.action': 'flow-expiration', 'event.end': '2024-05-05T17:51:17.000-02:00', 'event.category': ['network'], 'event.dataset': 'cisco.asa', 'cisco.asa.source_interface': 'net', 'cisco.asa.message_id': '609002'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['event.start']": {'new_value': '2024-05-05T19:51:17.000Z', 'old_value': '2023-05-05T19:51:17...'}, "root['event.end']": {'new_value': '2024-05-05T17:51:17.000-02:00', 'old_value': '2023-05-05T17:51:17.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_305_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_305_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/357_monitor_fw_rules_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '7a89e0c9-05db-4e98-ba8a-a496e2ded004', 'id': '4...tor FW rules start', 'issuer': 'Batch', 'message': 'Monitor FW rules start', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 580, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW rules start', 'cyberarkpas.audit.message': 'Monitor FW rules start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW rules start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '357', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}}, full object: 
  {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_313_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_313_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/59_clear_safe_history.log'
objects = [{'@timestamp': '2021-03-04T19:25:02.000Z', 'agent': {'ephemeral_id': 'a4092ec4-93c7-44cc-9f97-d616a36be3e7', 'id': '4...r Safe History', 'iso_timestamp': '2021-03-09T09:00:47Z', 'issuer': 'Batch', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}}, full object: 
E             {'log.offset': 604, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:10:31.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.safe': 'PasswordManager_workspace', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Clear Safe History', 'cyberarkpas.audit.message': 'Clear Safe History', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.desc': 'Clear Safe History', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '59', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'clear safe history', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:10:31.000-02:00', 'old_value': '2023-03-08T03:10:31.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_325_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_325_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/310_monitor_dr_replication_start.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '3c9a448b-c00c-4e01-bb9e-75058feb48e3', 'id': '4...cation start', 'issuer': 'Batch', 'message': 'Monitor DR Replication start', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 598, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication start', 'cyberarkpas.audit.message': 'Monitor DR Replication start', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication start', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '310', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication start', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
  {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_332_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_332_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/358_monitor_fw_rules_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '68e91ba0-79f1-4ecb-9df8-e0ca02a51cc3', 'id': '4...Monitor FW Rules end', 'issuer': 'Batch', 'message': 'Monitor FW Rules end', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}}, full object: 
E             {'log.offset': 574, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:32:56.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor FW Rules end', 'cyberarkpas.audit.message': 'Monitor FW Rules end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor FW Rules end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '358', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor fw rules end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:32:56.000-02:00', 'old_value': '2023-03-08T02:32:56.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}}, full object: 
  {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_335_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_335_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/479_security_warning_the_signature_hash_algorithm_of_the_vault_certificate_is_sha1.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '36b69e2a-18b1-4950-a4e5-4e73ef20ffe9', 'id': '4...y warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}}, full object: 
E             {'log.offset': 760, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T07:46:54.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Error', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.message': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'cyberarkpas.audit.issuer': 'Builtin', 'cyberarkpas.audit.desc': 'Security warning - The Signature Hash Algorithm of the Vault certificate is SHA1.', 'host.name': 'VAULT', 'event.severity': 7, 'event.code': '479', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'security warning - the signature hash algorithm of the vault certificate is sha1.', 'event.type': 'error', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T07:46:54.000-02:00', 'old_value': '2023-03-08T07:46:54.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}}, full object: 
  {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_346_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_346_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/legacysyslog.log'
objects = [{'@timestamp': '2024-03-08T03:41:01.000-02:00', 'agent': {'ephemeral_id': '2131ad9d-9ecf-4f40-97e2-c78bc6f3baa0', 'id...: 'Root\\Policies\\Policy-BusinessWebsite.ini', 'issuer': 'PasswordManager', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}}, full object: 
E             {'log.offset': 0, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T03:41:01.000-02:00', 'file.path': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.message': 'Retrieve File', 'cyberarkpas.audit.issuer': 'PasswordManager', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.file': 'Root\\Policies\\Policy-BusinessWebsite.ini', 'cyberarkpas.audit.safe': 'PasswordManagerShared', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Retrieve File', 'cyberarkpas.audit.desc': 'Retrieve File', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '51', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'retrieve file', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T03:41:01.000-02:00', 'old_value': '2023-03-08T03:41:01.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
  {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_351_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_351_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/311_monitor_dr_replication_end.log'
objects = [{'@timestamp': '2021-03-04T19:10:01.000Z', 'agent': {'ephemeral_id': '3cdf9818-c7cc-4229-9fa8-df727af3ed06', 'id': '4...eplication end', 'issuer': 'Batch', 'message': 'Monitor DR Replication end', ...}}, 'ecs': {'version': '1.12.0'}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}}, full object: 
E             {'log.offset': 592, 'source.address': '0.0.0.0', 'source.ip': '0.0.0.0', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:48:07.000-02:00', 'related.ip': ['0.0.0.0'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '0.0.0.0', 'cyberarkpas.audit.action': 'Monitor DR Replication end', 'cyberarkpas.audit.message': 'Monitor DR Replication end', 'cyberarkpas.audit.issuer': 'Batch', 'cyberarkpas.audit.desc': 'Monitor DR Replication end', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '311', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'monitor dr replication end', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:48:07.000-02:00', 'old_value': '2023-03-08T02:48:07.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:
   Diff:
  {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}}, full object: 
  {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
assert 1 == 0
 +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_353_cyberarkpas>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../../../../../python-env/build/ve/linux/lib/python3.10/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:150: in test_fileset_file
    self.run_on_file(
../../filebeat/tests/system/test_modules.py:247: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_353_cyberarkpas>
test_file = '/var/lib/jenkins/workspace/PR-37502-9-0e1ffcc2-72cd-41df-b85d-2fe3edfb8834/src/github.com/elastic/beats/x-pack/filebeat/module/cyberarkpas/audit/test/88_set_password.log'
objects = [{'@timestamp': '2021-03-04T19:16:19.000Z', 'agent': {'ephemeral_id': '48b97476-9cd3-43d9-9785-346f2e594d0c', 'id': '4...mp': '2021-03-10T09:11:52Z', 'issuer': 'PSMPApp_localhost.localdomain', ...}}, 'ecs': {'version': '1.12.0'}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
                    for key in objects[k].keys():
                        if isinstance(objects[k][key], list) and key not in array_fields_dont_sort:
                            objects[k][key].sort(key=str)
    
                json.dump(objects, f, indent=4, separators=(
                    ',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        # Do not perform a comparison between the resulting and expected documents
        # if the TESTING_FILEBEAT_SKIP_DIFF flag is set.
        #
        # This allows to run a basic check with older versions of ES that can lead
        # to slightly different documents without maintaining multiple sets of
        # golden files.
        if os.getenv("TESTING_FILEBEAT_SKIP_DIFF"):
            return
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(
                d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}}, full object: 
E             {'log.offset': 1113, 'source.address': '10.0.1.20', 'source.ip': '10.0.1.20', 'fileset.name': 'audit', 'tags': ['cyberarkpas.audit', 'forwarded'], 'input.type': 'log', 'observer.product': 'Vault', 'observer.hostname': 'VAULT', 'observer.vendor': 'Cyber-Ark', 'observer.version': '11.7.0000', '@timestamp': '2024-03-08T02:54:46.000-02:00', 'related.ip': ['10.0.1.20'], 'service.type': 'cyberarkpas', 'cyberarkpas.audit.rfc5424': False, 'cyberarkpas.audit.severity': 'Info', 'cyberarkpas.audit.station': '10.0.1.20', 'cyberarkpas.audit.action': 'Set Password', 'cyberarkpas.audit.message': 'Set Password', 'cyberarkpas.audit.issuer': 'PVWAGWUser', 'cyberarkpas.audit.desc': 'Set Password', 'host.name': 'VAULT', 'event.severity': 2, 'event.code': '88', 'event.timezone': '-02:00', 'event.kind': 'event', 'event.module': 'cyberarkpas', 'event.action': 'set password', 'event.dataset': 'cyberarkpas.audit'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['@timestamp']": {'new_value': '2024-03-08T02:54:46.000-02:00', 'old_value': '2023-03-08T02:54:46.000-02:00'}}})

../../filebeat/tests/system/test_modules.py:292: AssertionError

Steps errors

Expand to view the steps failures

`isToolInstalled`

Took 1 min 59 sec . View more details here
Description: tar --version >/dev/null

`Cleanup`

Took 0 min 0 sec . View more details here
Description: rm source.tgz

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 106 min 51 sec . View more details here
Description: mage pythonIntegTest

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 101 min 24 sec . View more details here
Description: mage pythonIntegTest

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

Took 101 min 54 sec . View more details here
Description: mage pythonIntegTest

`Error signal`

Took 0 min 0 sec . View more details here
Description: untar: step failed with error Remote call on beats-ci-immutable-ubuntu-2204-1704229240382145939 failed

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_354_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_379_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Made with ❤️️ by updatecli

elasticmachine · 2024-01-03T20:51:05Z

💔 Build Failed

Expand to view the summary

Build stats

Duration: 341 min 53 sec

Pipeline error

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Made with ❤️️ by updatecli

elasticmachine · 2024-01-04T17:23:59Z

💚 Build Succeeded

Expand to view the summary

Build stats

Start Time: 2024-01-04T15:09:44.567+0000
Duration: 134 min 57 sec

Test stats 🧪

Test	Results
Failed	0
Passed	28732
Skipped	2015
Total	30747

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-01-04T18:00:23Z

❕ Build Aborted

Expand to view the summary

Build stats

Start Time: 2024-01-04T17:25:49.595+0000
Duration: 34 min 17 sec

Test stats 🧪

Test	Results
Failed	10
Passed	9875
Skipped	136
Total	10021

Test errors

Expand to view the tests failures

`Build&Test / metricbeat-windows-2022-windows-2022 / [empty] – TEST-go-unit.xml`

no error details

Expand to view the stacktrace

 Test report file C:\Users\jenkins\workspace\PR-37502-12-8e70f64d-595b-4a4a-9049-c3937e571588\src\github.com\elastic\beats\build\metricbeat\build\TEST-go-unit.xml was length 0

`Build&Test / heartbeat-windows-2022-windows-2022 / [empty] – TEST-go-unit.xml`

no error details

Expand to view the stacktrace

 Test report file C:\Users\jenkins\workspace\PR-37502-12-c877a5b8-731f-4407-9883-d0dda85a7ea7\src\github.com\elastic\beats\build\heartbeat\build\TEST-go-unit.xml was length 0

`Build&Test / filebeat-windows-2016-windows-2016 / TestLogFileTimedClosing – github.com/elastic/beats/v7/filebeat/input/filestream`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestLogFileTimedClosing
=== RUN   TestLogFileTimedClosing/read_from_file_and_close_after_interval

`Build&Test / filebeat-windows-2016-windows-2016 / TestLogFileTimedClosing/read_from_file_and_close_after_interval – github.com/elastic/beats/v7/filebeat/input/filestream`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestLogFileTimedClosing/read_from_file_and_close_after_interval

`Build&Test / libbeat-unitTest / [empty] – TEST-go-unit.xml`

no error details

Expand to view the stacktrace

 Test report file /var/lib/jenkins/workspace/PR-37502-12-6be233cb-12ee-4ed3-913f-51f588b6fe63/src/github.com/elastic/beats/build/libbeat/build/TEST-go-unit.xml was length 0

`Build&Test / x-pack/filebeat-unitTest / TestSQSS3EventProcessor – github.com/elastic/beats/v7/x-pack/filebeat/input/awss3`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestSQSS3EventProcessor
=== RUN   TestSQSS3EventProcessor/s3_events_are_processed_and_sqs_msg_is_deleted
--- PASS: TestSQSS3EventProcessor/s3_events_are_processed_and_sqs_msg_is_deleted (0.00s)
=== RUN   TestSQSS3EventProcessor/invalid_SQS_JSON_body_does_not_retry
    sqs_s3_event_test.go:79: failed processing SQS message (message was deleted): non-retryable error: failed to decode SQS message body as an S3 notification: json: cannot unmarshal string into Go value of type awss3.s3EventsV2
--- PASS: TestSQSS3EventProcessor/invalid_SQS_JSON_body_does_not_retry (0.00s)
=== RUN   TestSQSS3EventProcessor/zero_S3_events_in_body
--- PASS: TestSQSS3EventProcessor/zero_S3_events_in_body (0.00s)
=== RUN   TestSQSS3EventProcessor/visibility_is_extended_after_half_expires

`Build&Test / x-pack/filebeat-unitTest / TestSQSS3EventProcessor/visibility_is_extended_after_half_expires – github.com/elastic/beats/v7/x-pack/filebeat/input/awss3`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestSQSS3EventProcessor/visibility_is_extended_after_half_expires

`Build&Test / x-pack/metricbeat-unitTest / [empty] – TEST-go-unit.xml`

no error details

Expand to view the stacktrace

 Test report file /var/lib/jenkins/workspace/PR-37502-12-3299a5c9-3a99-4463-8b8e-0546a07076dc/src/github.com/elastic/beats/build/x-pack/metricbeat/build/TEST-go-unit.xml was length 0

`Build&Test / libbeat-goIntegTest / [empty] – TEST-go-integration.xml`

no error details

Expand to view the stacktrace

 Test report file /var/lib/jenkins/workspace/PR-37502-12-c4b287c7-08b2-42aa-a6fd-390a79198101/src/github.com/elastic/beats/build/libbeat/build/TEST-go-integration.xml was length 0

`Build&Test / filebeat-goIntegTest / TestParsersCloseTimeoutWithMultiline – github.com/elastic/beats/v7/filebeat/input/filestream`

Expand to view the error details

 Failed

Expand to view the stacktrace

 === RUN   TestParsersCloseTimeoutWithMultiline

Steps errors

Expand to view the steps failures

`Error signal`

Took 0 min 0 sec . View more details here
Description: Error 'org.jenkinsci.plugins.workflow.steps.FlowInterruptedException'

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-01-04T20:05:42Z

💚 Build Succeeded

Expand to view the summary

Build stats

Duration: 135 min 9 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-01-04T22:21:02Z

💚 Build Succeeded

Expand to view the summary

Build stats

Duration: 134 min 1 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

…lastic#37502) * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli * chore: Update snapshot.yml Made with ❤️️ by updatecli --------- Co-authored-by: apmmachine <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Craig MacKenzie <[email protected]>

chore: Update snapshot.yml

a2a3347

Made with ❤️️ by updatecli

apmmachine requested a review from a team as a code owner December 25, 2023 15:09

apmmachine requested review from ycombinator and rdner December 25, 2023 15:09

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Dec 25, 2023

apmmachine added automation build-monitoring Issues created as part of Build Monitoring dependency backport-skip Skip notification from the automated backport with mergify Team:Beats-On-Call Label for the On Call team labels Dec 25, 2023

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Dec 25, 2023

mergify bot assigned apmmachine Dec 25, 2023

mergify bot approved these changes Dec 25, 2023

View reviewed changes

chore: Update snapshot.yml

d12079e

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-p7cya7tg~~ [updatecli] update elastic stack version for testing 8.13.0-qvykcic3 Dec 26, 2023

chore: Update snapshot.yml

ddf6ed4

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-qvykcic3~~ [updatecli] update elastic stack version for testing 8.13.0-pjcmn9wb Dec 27, 2023

chore: Update snapshot.yml

afaceed

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-pjcmn9wb~~ [updatecli] update elastic stack version for testing 8.13.0-4aavtlfc Dec 28, 2023

chore: Update snapshot.yml

eb01ea5

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-4aavtlfc~~ [updatecli] update elastic stack version for testing 8.13.0-4f8l51qr Dec 29, 2023

pierrehilbert approved these changes Jan 1, 2024

View reviewed changes

mergify bot and others added 2 commits January 1, 2024 12:06

Merge branch 'main' into updatecli_bump-elastic-stack-snapshot-main

6fd263a

chore: Update snapshot.yml

8cb73d2

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-4f8l51qr~~ [updatecli] update elastic stack version for testing 8.13.0-p45l43ib Jan 1, 2024

chore: Update snapshot.yml

0d0389f

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-p45l43ib~~ [updatecli] update elastic stack version for testing 8.13.0-42wme37p Jan 2, 2024

Merge branch 'main' into updatecli_bump-elastic-stack-snapshot-main

058a831

chore: Update snapshot.yml

bd8b011

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-42wme37p~~ [updatecli] update elastic stack version for testing 8.13.0-toucn398 Jan 3, 2024

chore: Update snapshot.yml

20319e2

Made with ❤️️ by updatecli

apmmachine changed the title ~~[updatecli] update elastic stack version for testing 8.13.0-toucn398~~ [updatecli] update elastic stack version for testing 8.13.0-ybxdr713 Jan 4, 2024

mergify bot added 2 commits January 4, 2024 17:25

Merge branch 'main' into updatecli_bump-elastic-stack-snapshot-main

2aadb0d

Merge branch 'main' into updatecli_bump-elastic-stack-snapshot-main

ec83484

Merge branch 'main' into updatecli_bump-elastic-stack-snapshot-main

ad3115a

mergify bot merged commit 7a3a9f9 into main Jan 4, 2024
7 checks passed

mergify bot deleted the updatecli_bump-elastic-stack-snapshot-main branch January 4, 2024 22:21

[updatecli] update elastic stack version for testing 8.13.0-ybxdr713 #37502

[updatecli] update elastic stack version for testing 8.13.0-ybxdr713 #37502

Conversation

apmmachine commented Dec 25, 2023 • edited Loading

Bump elastic-stack to latest snapshot version

elasticmachine commented Dec 25, 2023

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

mergify bot left a comment

Choose a reason for hiding this comment

elasticmachine commented Dec 26, 2023

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Dec 27, 2023

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

elasticmachine commented Dec 28, 2023

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

elasticmachine commented Dec 29, 2023

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Jan 1, 2024

❕ Build Aborted

Build stats

Test stats 🧪

Test errors

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Steps errors

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

Error signal

🤖 GitHub comments

elasticmachine commented Jan 1, 2024

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Steps errors

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

x-pack/filebeat-pythonIntegTest - mage pythonIntegTest

Error signal

🐛 Flaky test report

Genuine test errors

🤖 GitHub comments

elasticmachine commented Jan 2, 2024

💔 Build Failed

Build stats

apmmachine commented Dec 25, 2023 •

edited

Loading

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`Error signal`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`Error signal`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_290_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_294_cisco – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_305_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_313_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_325_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_332_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_335_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_346_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_351_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`Build&Test / x-pack/filebeat-pythonIntegTest / test_fileset_file_353_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

`isToolInstalled`

`Cleanup`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`x-pack/filebeat-pythonIntegTest - mage pythonIntegTest`

`Error signal`

`Build&Test / metricbeat-windows-2022-windows-2022 / [empty] – TEST-go-unit.xml`

`Build&Test / heartbeat-windows-2022-windows-2022 / [empty] – TEST-go-unit.xml`

`Build&Test / filebeat-windows-2016-windows-2016 / TestLogFileTimedClosing – github.com/elastic/beats/v7/filebeat/input/filestream`

`Build&Test / filebeat-windows-2016-windows-2016 / TestLogFileTimedClosing/read_from_file_and_close_after_interval – github.com/elastic/beats/v7/filebeat/input/filestream`

`Build&Test / libbeat-unitTest / [empty] – TEST-go-unit.xml`

`Build&Test / x-pack/filebeat-unitTest / TestSQSS3EventProcessor – github.com/elastic/beats/v7/x-pack/filebeat/input/awss3`

`Build&Test / x-pack/filebeat-unitTest / TestSQSS3EventProcessor/visibility_is_extended_after_half_expires – github.com/elastic/beats/v7/x-pack/filebeat/input/awss3`

`Build&Test / x-pack/metricbeat-unitTest / [empty] – TEST-go-unit.xml`

`Build&Test / libbeat-goIntegTest / [empty] – TEST-go-integration.xml`

`Build&Test / filebeat-goIntegTest / TestParsersCloseTimeoutWithMultiline – github.com/elastic/beats/v7/filebeat/input/filestream`

`Error signal`