Support additional metadata via Client_Info with BigQuery Storage Read & Write (gRPC) APIs

[deathstarjanitor]

Determine this is the right repository

• I determined this is the correct repository in which to report this feature request.

Summary of the feature request

I would like to be able to provide a user_agent (or alternatively "request_reason" ) metadata string to BigQuery REST APIs as well as BigQuery gRPC APIs, and have them propagated to Cloud Logging.

Ideally, the metadata specification should be identical across the API surfaces and clients.

Desired code experience

An example of how this is done in BigQuery client is available here:

file: bigquery_rest.py

from google.cloud import bigquery
from google.api_core import client_info

def create_dataset() -> None:
  custom_agent = client_info.ClientInfo(
      user_agent = "test_A/1.0 (lineage_id:baz;user_id:lem)"
  )
  client = bigquery.Client(client_info=custom_agent)

  dataset_id = "{}.test_python_user_agent".format(client.project)
  dataset = bigquery.Dataset(dataset_id)
  dataset = client.create_dataset(dataset, timeout=30)
  print("Created dataset {}.{}".format(client.project, dataset.dataset_id))

if __name__ == "__main__":
  create_dataset()

However, this does not work with BigQuery Storage Read / Write APIs in the same way.

Expected results

protoPayload.requestMetadata.callerSuppliedUserAgent = test_A/1.0 (lineage_id:baz;user_id:lem) ...

API client name and version

google-cloud-bigquery_storage_v1

Use case

I want to specify additional metadata that represents the workload behind my BigQuery API calls.

Additional context

The following workaround exists for using the bigquery_storage_v1 grpc api, but it is less intuitive than the Client_Info method above.

from google.cloud import bigquery_storage_v1, bigquery
from google.cloud.bigquery_storage_v1 import types, writer
from google.cloud.bigquery_storage import BigQueryReadClient

…

user_agent = "Test_Z/1.0 (lineage_id:foo,user_id:bar)" 

write_transport_cls = bigquery_storage_v1.BigQueryWriteClient.get_transport_class()
options = [('grpc.primary_user_agent', user_agent)]
write_channel = write_transport_cls.create_channel(options=options)
write_transport = write_transport_cls(channel=write_channel)

…

def append_rows_pending(project_id: str, dataset_id: str, table_id: str):
    """Create a write stream, write some sample data, and commit the stream."""
    write_client = bigquery_storage_v1.BigQueryWriteClient(transport=write_transport)
    parent = write_client.table_path(project_id, dataset_id, table_id)
    write_stream = types.WriteStream()