Support partitionID in Azure eventhub input

[Martin-Kemp]

Describe the enhancement:

Currently partitionID is not supported in Azure eventhub input. It's commented out in the code because Azure SDK apparently doesn't support it. See commented out code here: https://github.com/elastic/beats/blob/main/x-pack/filebeat/input/azureeventhub/input.go#L197
and here: https://github.com/elastic/beats/blob/main/x-pack/filebeat/input/azureeventhub/eph.go#L77

Is this still the case?

As far as I can see partitionID is supported: https://github.com/Azure/azure-event-hubs-go

Describe a specific use case for the enhancement or feature:

PartitionID is very useful for downstream applications consuming eventhub data, we use MQTT topic strings in our partitionID and use this to route data.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[Martin-Kemp]

I can confirm that the new SDK works. I tested this basic example: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/messaging/azeventhubs/example_consuming_events_test.go
and followed this documentation: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/messaging/azeventhubs#section-readme

I sent a message to an eventhub and specified the partitionKey then read the message and got the partitionKey:

Received message: Your message content, PartitionKey: %!d(string=your-partition-key), sequence number: 0, enqueued time: 2023-10-27 10:34:41.948 +0000 UTC

By the way, it turns out I was looking for PartitionKey, not PartitionID. But both should be available with the new SDK.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!