[Question] Why is opensearch-env different between rpm and tar builds?

[peterzhuamazon]

I tried to compare rpm and tar builds from the files.
And they apparently have many differences within opensearch-env.

The rpm even hard code to source from /etc/sysconfig/opensearch for env vars

38,42c38,39
< # now set the path to java: OPENSEARCH_JAVA_HOME -> JAVA_HOME -> bundled JDK
< if [ ! -z "$OPENSEARCH_JAVA_HOME" ]; then
<   JAVA="$OPENSEARCH_JAVA_HOME/bin/java"
<   JAVA_TYPE="OPENSEARCH_JAVA_HOME"
< elif [ ! -z "$JAVA_HOME" ]; then
---
> # now set the path to java
> if [ ! -z "$JAVA_HOME" ]; then
84c81
< source /etc/sysconfig/opensearch
---
> if [ -z "$OPENSEARCH_PATH_CONF" ]; then OPENSEARCH_PATH_CONF="$OPENSEARCH_HOME"/config; fi
94c91
< OPENSEARCH_DISTRIBUTION_TYPE=rpm
---
> OPENSEARCH_DISTRIBUTION_TYPE=tar

[peterzhuamazon]

This kinda pose a problem when producing bundled OpenSearch RPM with these files.
As you need to first remove the sourcing (The file only exist after you install the rpm on customer host).
Then define OPENSEARCH_PATH_CONF so that opensearch-plugin can run properly to install all the plugin zip.

Then add back the sourcing and then repackage into rpm.

However, in practice, even without sourcing /etc/sysconfig/opensearch the variables will still be sourced by systemctl service file with EnvironmentFile=-/etc/sysconfig/opensearch.

Unless someone manually run opensearch bin without using service file for RPM, then it would pose an issue.

Thanks.

[peterzhuamazon]

Any thoughts, @dblock ?

[reta]

@peterzhuamazon could you please point out which opensearch-env is used for to build rpm? I suspect this is not the same one (copy?) as used by archive distributions.

[peterzhuamazon]

@reta I run distribution:packages:rpm:build then compare the opensearch-env from rpm-extracted with the one from distribution:archives:linux-tar:build, then get the diffs you see in description.

[dblock]

I don't know the history of one vs. the other, we inherited this from the fork.

[reta]

@dblock @peterzhuamazon I will take a look shortly

[reta]

@peterzhuamazon I just did a fresh build of the distributions, there are only 2 differences in opensearch-env between archive and RPM (and DEB as well):

• OPENSEARCH_DISTRIBUTION_TYPE=rpm vs OPENSEARCH_DISTRIBUTION_TYPE=tar
• and indeed OPENSEARCH_PATH_CONF

if [ -z "$OPENSEARCH_PATH_CONF" ]; then
  source /etc/sysconfig/opensearch
fi

vs

if [ -z "$OPENSEARCH_PATH_CONF" ]; then
  if [ -z "$OPENSEARCH_PATH_CONF" ]; then OPENSEARCH_PATH_CONF="$OPENSEARCH_HOME"/config; fi
fi

It comes from the build script [1] and @dblock is right, we just inherited it from the fork, it seems to be convention the original project has followed for years.

[1] https://github.com/opensearch-project/OpenSearch/blob/main/distribution/build.gradle#L536

[peterzhuamazon]

@reta I am using 1.2.0 for the build, sorry for not mentioning that.
I clearly see more changes as in my description.
Not sure if newer versions have less changes tho.

Thanks.

[reta]

@peterzhuamazon oh ... thanks for clarifying, will rebuild in a sec, the OPENSEARCH_JAVA_HOME presence is very suspicious in 1.2.x though - we did not merge this change [1]. I will come back with the update shortly.

[1] #2042

[reta]

@peterzhuamazon I built fresh 1.2.5-SHAPSHOTs

-rw-rw-r-- 1 user user 223M Feb 14 15:06 opensearch-min-1.2.5-SNAPSHOT.x86_64.rpm  
-rw-rw-r-- 1 user user 223M Feb 14 15:05 opensearch-min-1.2.5-SNAPSHOT-linux-x64.tar.gz  

And the diff is exactly the same, I suspect the archive is not for 1.2.x (the OPENSEARCH_JAVA_HOME should not be present there in any form), could you please double check (or even better, share links to these distributions)?

Thank you!

[peterzhuamazon]

@reta I use tags/1.2.0 to build but will test again.

[peterzhuamazon]

Tar: distribution/archives/linux-tar/build/tar-extracted/opensearch-1.2.0/bin/opensearch-env
Rpm: distribution/packages/rpm/build/extracted/rpm-extracted/usr/share/opensearch/bin/opensearch-env

diffs:

% diff -bi ~/env-tar ~/env-rpm
81c81
< if [ -z "$OPENSEARCH_PATH_CONF" ]; then OPENSEARCH_PATH_CONF="$OPENSEARCH_HOME"/config; fi
---
> source /etc/sysconfig/opensearch
91c91
< OPENSEARCH_DISTRIBUTION_TYPE=tar
---
> OPENSEARCH_DISTRIBUTION_TYPE=rpm

[peterzhuamazon]

Weird enough the diffs are different but anyway the source still present......

[peterzhuamazon]

I think the source one is understandable as if anyone installed RPM but not using systemd, then there needs to be a way to source the sysconfig.

The OPENSEARCH_PATH_CONF one seems needs to stay for now, as security plugin has some hard dependencies on the config folder. If the folder/link present in OPENSEARCH_HOME/config, then demo installer will use tarball mode, despite you install a version of rpm, else it will link to /etc/opensearch hardcoded.

https://github.com/opensearch-project/security/blob/main/tools/install_demo_configuration.sh#L159-L173

[reta]

Yeah, the source is "inherited", you definitely have better assumptions why it is there ...

[peterzhuamazon]

Seems like expected behavior at this point, close issue.