uninitialized constant OpenSearch::Transport::Transport::Serializer (NameError) include OpenSearch::Transport::Transport::Serializer::Base

[sreekanth-h]

(check apply)

• [ Y] read the contribution guideline
• (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.

Steps to replicate

@type dummy tag dummy dummy {"hello":"world"}

<match .>
@type opensearch
host host1
port 9200
user ***
password ****

@type json

Expected Behavior or What you need to ask

fluentd service needs to run and able to connect to Opensearch
...

Using Fluentd and OpenSearch plugin versions

• OS version Ubuntu 22.04

• Bare Metal or within Docker or Kubernetes or others? VM

• Fluentd v1.0 or later
  fluentd 1.16.3

• OpenSearch plugin version - 1.1.4

  • paste boot log of fluentd or td-agent

  2024-01-15 08:25:21 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
  2024-01-15 08:25:21 +0000 [info]: parsing config file is succeeded path="/etc/fluent/fluentd.conf"
  2024-01-15 08:25:21 +0000 [info]: gem 'fluentd' version '1.16.3'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-calyptia-monitoring' version '0.1.3'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.4.2'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-flowcounter-simple' version '0.1.0'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-kafka' version '0.19.2'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-metrics-cmetrics' version '0.1.2'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-opensearch' version '1.1.4'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.1.0'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.1.1'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.1'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-s3' version '1.7.2'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-sd-dns' version '0.1.0'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-td' version '1.2.0'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-utmpx' version '0.5.0'
  2024-01-15 08:25:21 +0000 [info]: gem 'fluent-plugin-webhdfs' version '1.5.0'
  /opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-opensearch-1.1.4/lib/fluent/plugin/oj_serializer.rb:33:in `class:Oj': uninitialized constant OpenSearch::Transport::Transport::Serializer (NameError)

  include OpenSearch::Transport::Transport::Serializer::Base
                                                      ^^^^^^
  

  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-opensearch-1.1.4/lib/fluent/plugin/oj_serializer.rb:32:in <module:Serializer>' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-opensearch-1.1.4/lib/fluent/plugin/oj_serializer.rb:30:in module:Plugin'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-opensearch-1.1.4/lib/fluent/plugin/oj_serializer.rb:29:in <top (required)>' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-opensearch-1.1.4/lib/fluent/plugin/out_opensearch.rb:54:in require_relative'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluent-plugin-opensearch-1.1.4/lib/fluent/plugin/out_opensearch.rb:54:in <top (required)>' from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in require'
  from internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb:85:in require' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:103:in block in search'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in each' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:100:in search'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/registry.rb:44:in lookup' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:169:in new_impl'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/plugin.rb:114:in new_output' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:130:in add_match'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:74:in block in configure' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in each'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/agent.rb:64:in configure' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/root_agent.rb:149:in configure'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:105:in configure' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/engine.rb:80:in run_configure'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/supervisor.rb:571:in run_supervisor' from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/lib/fluent/command/fluentd.rb:352:in <top (required)>'
  from internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb:85:in require' from <internal:/opt/fluent/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in require'
  from /opt/fluent/lib/ruby/gems/3.2.0/gems/fluentd-1.16.3/bin/fluentd:15:in <top (required)>' from /opt/fluent/bin/fluentd:25:in load'
  from /opt/fluent/bin/fluentd:25:in <top (required)>' from /usr/sbin/fluentd:15:in load'
  from /usr/sbin/fluentd:15:in `

  '
  • paste result of fluent-gem list, td-agent-gem list or your Gemfile.lock

abbrev (default: 0.1.1)
addressable (2.8.5)
async (1.31.0)
async-http (0.61.0)
async-io (1.38.0)
async-pool (0.4.0)
aws-eventstream (1.2.0)
aws-partitions (1.785.0)
aws-sdk-core (3.178.0)
aws-sdk-kms (1.71.0)
aws-sdk-s3 (1.129.0)
aws-sdk-sqs (1.61.0)
aws-sigv4 (1.6.0)
base64 (0.2.0, default: 0.1.1)
benchmark (default: 0.2.1)
bigdecimal (default: 3.1.3)
bindata (2.4.15)
bundler (default: 2.4.10, 2.3.26)
cgi (default: 0.3.6)
cmetrics (0.3.3)
concurrent-ruby (1.2.2)
console (1.23.2)
cool.io (1.8.0)
csv (default: 3.2.6)
date (default: 3.3.3)
debug (1.7.1)
delegate (default: 0.3.0)
did_you_mean (default: 1.6.3)
digest (default: 3.1.1)
digest-crc (0.6.5)
digest-murmurhash (1.1.1)
drb (default: 2.1.1)
elastic-transport (8.3.0)
elasticsearch (8.8.0)
elasticsearch-api (8.8.0)
elasticsearch-transport (7.17.10)
english (default: 0.7.2)
erb (default: 4.0.2)
error_highlight (default: 0.5.1)
etc (default: 1.4.2)
excon (0.109.0)
faraday (2.7.12, 1.10.3)
faraday-em_http (1.0.0)
faraday-em_synchrony (1.0.0)
faraday-excon (2.1.0, 1.1.0)
faraday-httpclient (1.0.1)
faraday-multipart (1.0.4)
faraday-net_http (3.0.2, 1.0.1)
faraday-net_http_persistent (1.2.0)
faraday-patron (1.0.0)
faraday-rack (1.0.0)
faraday-retry (1.0.3)
faraday_middleware-aws-sigv4 (1.0.1, 0.6.1)
fcntl (default: 1.0.2)
ffi (1.15.5)
fiber-annotation (0.2.0)
fiber-local (1.0.0)
fiddle (default: 1.1.1)
fileutils (1.7.2, default: 1.7.0)
find (default: 0.1.1)
fluent-config-regexp-type (1.0.0)
fluent-diagtool (1.0.3)
fluent-logger (0.9.0)
fluent-plugin-calyptia-monitoring (0.1.3)
fluent-plugin-elasticsearch (5.4.2)
fluent-plugin-flowcounter-simple (0.1.0)
fluent-plugin-kafka (0.19.2)
fluent-plugin-metrics-cmetrics (0.1.2)
fluent-plugin-opensearch (1.1.4)
fluent-plugin-prometheus (2.1.0)
fluent-plugin-prometheus_pushgateway (0.1.1)
fluent-plugin-record-modifier (2.1.1)
fluent-plugin-rewrite-tag-filter (2.4.0)
fluent-plugin-s3 (1.7.2)
fluent-plugin-sd-dns (0.1.0)
fluent-plugin-systemd (1.0.5)
fluent-plugin-td (1.2.0)
fluent-plugin-utmpx (0.5.0)
fluent-plugin-webhdfs (1.5.0)
fluentd (1.16.3)
forwardable (default: 1.3.3)
getoptlong (default: 0.2.0)
hirb (0.7.3)
http_parser.rb (0.8.0)
httpclient (2.8.3)
io-console (default: 0.6.0)
io-nonblock (default: 0.2.0)
io-wait (default: 0.3.0)
ipaddr (default: 1.2.5)
irb (default: 1.6.2)
jmespath (1.6.2)
json (default: 2.6.3)
linux-utmpx (0.3.0)
logger (default: 1.5.3)
ltsv (0.1.2)
matrix (0.4.2)
mini_portile2 (2.8.2)
minitest (5.16.3)
msgpack (1.7.2)
multi_json (1.15.0)
multipart-post (2.3.0)
mutex_m (default: 0.1.2)
net-ftp (0.2.0)
net-http (default: 0.3.2)
net-imap (0.3.4)
net-pop (0.1.2)
net-protocol (default: 0.2.1)
net-smtp (0.3.3)
nio4r (2.6.1)
nkf (default: 0.1.2)
observer (default: 0.1.1)
oj (3.16.1)
open-uri (default: 0.3.0)
open3 (default: 0.1.2)
opensearch (0.0.1)
opensearch-ruby (3.1.0)
opensearch-transport (2.1.0)
openssl (default: 3.1.0)
optparse (default: 0.3.1)
ostruct (default: 0.5.5)
parallel (1.20.1)
pathname (default: 0.2.1)
power_assert (2.0.3)
pp (default: 0.4.0)
prettyprint (default: 0.1.1)
prime (0.1.2)
prometheus-client (2.1.0)
protocol-hpack (1.4.2)
protocol-http (0.25.0)
protocol-http1 (0.16.0)
protocol-http2 (0.15.1)
pstore (default: 0.1.2)
psych (default: 5.0.1)
public_suffix (5.0.4)
racc (default: 1.6.2)
rake (13.1.0, 13.0.6)
rbs (2.8.2)
rdkafka (0.12.0)
rdoc (default: 6.5.0)
readline (default: 0.0.3)
readline-ext (default: 0.1.5)
reline (default: 0.3.2)
resolv (default: 0.2.2)
resolv-replace (default: 0.1.1)
rexml (3.2.6, 3.2.5)
rinda (default: 0.1.1)
rss (0.2.9)
ruby-kafka (1.5.0)
ruby-progressbar (1.13.0)
ruby2_keywords (default: 0.0.5)
rubyzip (1.3.0)
securerandom (default: 0.2.2)
serverengine (2.3.2)
set (default: 1.0.3)
shellwords (default: 0.1.0)
sigdump (0.2.5)
singleton (default: 0.1.1)
stringio (default: 3.0.4)
strptime (0.2.5)
strscan (default: 3.0.5)
syntax_suggest (default: 1.0.2)
syslog (default: 0.1.1)
systemd-journal (1.4.2)
td (0.17.1)
td-client (1.0.8)
td-logger (0.3.28)
tempfile (default: 0.1.3)
test-unit (3.5.7)
time (default: 0.2.2)
timeout (default: 0.3.1)
timers (4.3.5)
tmpdir (default: 0.1.3)
traces (0.11.1)
tsort (default: 0.1.1)
typeprof (0.21.3)
tzinfo (2.0.6)
tzinfo-data (1.2023.3)
un (default: 0.2.1)
uri (0.12.2, default: 0.12.1)
weakref (default: 0.1.2)
webhdfs (0.10.2)
webrick (1.8.1)
yajl-ruby (1.4.3)
yaml (default: 0.2.1)
zip-zip (0.3)
zlib (default: 3.0.0)

• OpenSearch version (optional)
• OpenSearch template(s) (optional)

[finityrjames]

I have the same issue. It's working fine on one server, on another it's giving this error.
fluent-package 5.0.5 fluentd 1.16.6 on Ubuntu Jammy.
Supposedly this was fixed some time back, but somehow it's popping its head up here and there.

edit: full fluentd-package purge and reinstall with fluent-plugin-opensearch fixed the issue