test for spark streaming to make sure it uses merger when configured

[jonvex]

Change Logs

Describe context and summary for this change. Highlight if any code was copied.

Impact

Describe any public API or user-facing feature change or any performance impact.

Risk level (write none, low medium or high below)

If medium or high, explain what verification was done to mitigate the risks.

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

• The config description must be updated if new configs are added or the default value of the configs are changed
• Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
  ticket number here and follow the instruction to make
  changes to the website.

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[codope]

Is this especially needed for structured streaming?

[jonvex]

No, but I think it will be useful for additional testing as we switch over to merge mode on the write side in 1.1.

[codope]

Can we parameterze over all merge modes? This test only handles custom merge mode and I guess that's why you had to add DeleteRecordMerger, which is fine. But, let's also test other merge modes in streaming (maybe event time merge mode is getting tested due to default).

[jonvex]

I don't think it's worth the extra CI runtime. This isn't to really test out all the merge modes. It is to test that we can do structured streaming with custom merge mode and the configs are passed through correctly. I parameterized it for mor/cow now because we needed to test the merger for mor as well in the reader.