How to extract TAG components and add them as attributes to the record?

[sfc-gh-aivanou]

Hello! I searched in github and surprisingly I did not find this pattern, that should be pretty common.

I have a tag that has the following pattern:

my_prefix.<PROCESS_NAME>

How do I add <PROCESS_NAME> as attribute to the record?

E.g. I want to produce the following record:

my_prefix.<PROCESS_NAME>.data.log: [[1712008024.991679581, {}], {"message"=>"my-message", "process_name": "<PROCESS_NAME>"} 

[Rodeoclash]

I also seem to be having a bit of trouble finding where in the documentation I should be looking for this. In my case, I'm using the fluentd logging engine in Docker to ship my logs to a fluent-bit instance:

Terraform config

  log_configuration = {
    logDriver = "fluentd"
    options = {
      "fluentd-address" = "localhost:24224",
      "tag"             = "xonboard_backend2"
    }
    secretOptions = null
  }

I then want to extract the tag that's been passed and set it as the service.name field:

[FILTER]
    Name   modify
    Match  *
    Set    service.name test

In fluentd this was as possible using ${tag} but I'm having a bit more trouble in fluent-bit land. @sfc-gh-aivanou did you find a solution to this? I feel like we're in a similar spot 🙂

[sfc-gh-aivanou]

hi.
I used lua script to achieve my goal:

  filters.conf: |
    [FILTER]
        Name lua
        Match *
        script filters.lua
        call  my_script

  filters.lua: |
    function my_script(tag, timestamp, record)
        local parts = {}
        for part in tag:gmatch("[^.]+") do
            table.insert(parts, part)
        end
        record.my_tag = parts[0]
        return 2, timestamp, record
    end