Model deployment support for model observability

[tiopramayudi]

What this PR does / why we need it:

In the previous PR we introduce PyFuncV3Model to support model observability use case. What missing from previous PR is deployment process of PyFuncV3Model, hence this PR is raised.

Modification:

• api/config/config.go - Adding PyFuncPublisherConfig new configuration for pyfunc publisher
• api/config/environment.go - Update parsing DeploymentConfig to add relevant config to the deployment config
• db-migrations/34_version_endpoints_enable_observability.*.sql - Adding model observability enable flag to version endpoint
• api/models/model.go - Adding pyfunc_v3 model type
• api/cluster/resource/templater.go
  • All deployment config will be part of deploymentConfig including standard transformer configuration
  • Add required env vars to be passed for PyFuncV3Model
• api/api/version_endpoints_api.go - Adding PyFuncV3Model as supported model type for UPI
• python/sdk/test/pyfunc_integration_test.py - Adding integration test for PyFuncV3Model

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Checklist

• Added unit test, integration, and/or e2e tests
• Tested locally
• Updated documentation
• Update Swagger spec if the PR introduce API changes
• Regenerated Golang and Python client if the PR introduce API changes

[ghost]

👇 Click on the image for a new way to code review

Legend

[krithika369]

Thanks for this PR and the detailed description, @tiopramayudi! Left some small comments - the rest LGTM.

[khorshuheng]

The e2e test failed because of:

Merlin log
  {"level":"panic","ts":1701181719.603602,"caller":"api/main.go:101","msg":"Failed validating config: Key: 'Config.StandardTransformerConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag\nKey: 'Config.PyFuncPublisherConfig.Kafka.Brokers' Error:Field validation for 'Brokers' failed on the 'required' tag\nKey: 'Config.PyFuncPublisherConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag","stacktrace":"main.main\n\t/src/api/cmd/api/main.go:101\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}
  panic: Failed validating config: Key: 'Config.StandardTransformerConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag
  Key: 'Config.PyFuncPublisherConfig.Kafka.Brokers' Error:Field validation for 'Brokers' failed on the 'required' tag
  Key: 'Config.PyFuncPublisherConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag

[tiopramayudi]

The e2e test failed because of:

Merlin log
  {"level":"panic","ts":1701181719.603602,"caller":"api/main.go:101","msg":"Failed validating config: Key: 'Config.StandardTransformerConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag\nKey: 'Config.PyFuncPublisherConfig.Kafka.Brokers' Error:Field validation for 'Brokers' failed on the 'required' tag\nKey: 'Config.PyFuncPublisherConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag","stacktrace":"main.main\n\t/src/api/cmd/api/main.go:101\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}
  panic: Failed validating config: Key: 'Config.StandardTransformerConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag
  Key: 'Config.PyFuncPublisherConfig.Kafka.Brokers' Error:Field validation for 'Brokers' failed on the 'required' tag
  Key: 'Config.PyFuncPublisherConfig.Kafka.Acks' Error:Field validation for 'Acks' failed on the 'required' tag

Need to merge this PR first