SMTP TLS Report Fails To Save To Opensearch

[brycied00d]

The save_smtp_tls_report_to_opensearch method is looking for a field named org_name in the TLS-RPT object, which is not what's described by RFC8460. Meanwhile, the Elasticsearch storage's save_smtp_tls_report_to_elasticsearch correctly looks for organization_name. It appears that the fixes in f3206dc/#477 did not get mirrored over to opensearch.py, and the two modules have diverged a little bit.

Backtrace:

parsedmarc-1  |     INFO:cli.py:1204:Starting parsedmarc
parsedmarc-1  |    DEBUG:graph.py:140:Folder Archive already exists, skipping creation
parsedmarc-1  |    DEBUG:graph.py:140:Folder Aggregate already exists, skipping creation
parsedmarc-1  |    DEBUG:graph.py:140:Folder Forensic already exists, skipping creation
parsedmarc-1  |    DEBUG:graph.py:140:Folder SMTP-TLS already exists, skipping creation
parsedmarc-1  |    DEBUG:graph.py:140:Folder Invalid already exists, skipping creation
parsedmarc-1  |    DEBUG:__init__.py:1562:Found 1 messages in Inbox
parsedmarc-1  |    DEBUG:__init__.py:1569:Processing 1 messages
parsedmarc-1  |    DEBUG:__init__.py:1573:Processing message 1 of 1: UID AAMkAGNlNjAwZDRjLThiZDgtNDQzOS1iNWY5LTljMTEwMjM3NTI2MgBGAAAAAACYCGJapYKkQLwhWo72KQqjBwCrxeGclh7KRIksfmGJHwADAAAAAAEMAACrxeGclh7KRIksfmGJHwADAAAB627MAAA=
parsedmarc-1  |     INFO:__init__.py:1194:Parsing mail from [email protected] on 2024-11-09 02:43:35-08:00
parsedmarc-1  |    DEBUG:__init__.py:1682:Moving SMTP TLS report messages from Inbox to Archive/SMTP-TLS
parsedmarc-1  |    DEBUG:__init__.py:1691:Moving message 1 of 1: UID AAMkAGNlNjAwZDRjLThiZDgtNDQzOS1iNWY5LTljMTEwMjM3NTI2MgBGAAAAAACYCGJapYKkQLwhWo72KQqjBwCrxeGclh7KRIksfmGJHwADAAAAAAEMAACrxeGclh7KRIksfmGJHwADAAAB627MAAA=
parsedmarc-1  |     INFO:opensearch.py:687:Saving aggregate report to OpenSearch
parsedmarc-1  | {
parsedmarc-1  |   "aggregate_reports": [],
parsedmarc-1  |   "forensic_reports": [],
parsedmarc-1  |   "smtp_tls_reports": [
parsedmarc-1  |     {
parsedmarc-1  |       "organization_name": "Google Inc.",
parsedmarc-1  |       "begin_date": "2024-11-08T00:00:00Z",
parsedmarc-1  |       "end_date": "2024-11-08T23:59:59Z",
parsedmarc-1  |       "contact_info": "[email protected]",
parsedmarc-1  |       "report_id": "2024-11-08T00:00:00Z_redacted.com",
parsedmarc-1  |       "policies": [
parsedmarc-1  |         {
parsedmarc-1  |           "policy_domain": "redacted.com",
parsedmarc-1  |           "policy_type": "sts",
parsedmarc-1  |           "policy_strings": [
parsedmarc-1  |             "version: STSv1",
parsedmarc-1  |             "mode: enforce",
parsedmarc-1  |             "mx: redacted.mail.protection.outlook.com",
parsedmarc-1  |             "max_age: 2419200"
parsedmarc-1  |           ],
parsedmarc-1  |           "successful_session_count": 31,
parsedmarc-1  |           "failed_session_count": 0
parsedmarc-1  |         }
parsedmarc-1  |       ]
parsedmarc-1  |     }
parsedmarc-1  |   ]
parsedmarc-1  | }
parsedmarc-1  | 
parsedmarc-1  | Traceback (most recent call last):
parsedmarc-1  |   File "/opt/pypy/bin/parsedmarc", line 8, in <module>
parsedmarc-1  |     sys.exit(_main())
parsedmarc-1  |              ^^^^^^^
parsedmarc-1  |   File "/opt/pypy/lib/pypy3.10/site-packages/parsedmarc/cli.py", line 1563, in _main
parsedmarc-1  |     process_reports(results)
parsedmarc-1  |   File "/opt/pypy/lib/pypy3.10/site-packages/parsedmarc/cli.py", line 309, in process_reports
parsedmarc-1  |     opensearch.save_smtp_tls_report_to_opensearch(
parsedmarc-1  |   File "/opt/pypy/lib/pypy3.10/site-packages/parsedmarc/opensearch.py", line 688, in save_smtp_tls_report_to_opensearch
parsedmarc-1  |     org_name = report["org_name"]
parsedmarc-1  |                ^^^^^^^^^^^^^^^^^^
parsedmarc-1  | KeyError: 'org_name'

[seanthegeek]

@Szasza Can you look into this please?

[Szasza]

@seanthegeek sure, on it.