DOC-1014 update BYOVPC and PSC (#204)

* DOC-1014 update BYOVPC and PSC

* incorporate review feedback

* fix link, clarify text for consistency

* update <shared-vpc-name>, <host-project-id>

* style edits

* export CLUSTER_ID=<cluster-id>

* Add to what's new & link to support

* style edit

* add step to disable PSC in API

modules/get-started/pages/cloud-overview.adoc

@@ -248,7 +248,7 @@ either a public endpoint or a VPC peering network connection. Sensitive data and
 
 image::shared:d_c_plane.png[Data plane and control plane]
 
-include::partial$no-access.adoc[]
+include::get-started:partial$no-access.adoc[]
 
 A BYOC cluster is initially set up from the control plane. This is a two-step process performed by `rpk cloud byoc apply`:
 

modules/get-started/pages/cluster-types/byoc/aws/create-byoc-cluster-aws.adoc

@@ -44,7 +44,7 @@ Optionally, click *Advanced settings* to specify up to five key-value custom tag
 +
 As part of agent deployment, Redpanda assigns the permission required to run the agent. For details about these permissions, see xref:security:authorization/cloud-iam-policies.adoc[AWS IAM policies].
 
-include::partial$no-access.adoc[]
+include::get-started:partial$no-access.adoc[]
 
 == Next steps
 

modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc

@@ -6,7 +6,7 @@
 
 include::shared:partial$feature-flag-rpcn.adoc[]
 
-This topic explains how to create a Bring Your Own Virtual Private Cloud (BYOVPC) cluster. This setup allows you to deploy the Redpanda glossterm:data plane[] into your existing VPC and take full control of managing the networking lifecycle. Compared to a standard Bring Your Own Cluster (BYOC) setup, where Redpanda manages the networking lifecycle for you, this option provides more security.
+This topic explains how to create a Bring Your Own Virtual Private Cloud (BYOVPC) cluster. This setup allows you to deploy the Redpanda glossterm:data plane[] into your existing VPC and take full control of managing the networking lifecycle. Compared to a standard Bring Your Own Cloud (BYOC) setup, where Redpanda manages the networking lifecycle for you, this option provides more security.
 
 When you create a BYOCVPC cluster, you specify your VPC and service account. The Redpanda Cloud agent doesn't create any new resources or alter any settings in your account. With BYOVPC: 
 
@@ -387,4 +387,4 @@ After that completes, run:
 rpk cloud byoc aws destroy --redpanda-id ${REDPANDA_ID}
 ```
 
-include::partial$no-access.adoc[]
+include::get-started:partial$no-access.adoc[]

modules/get-started/pages/cluster-types/byoc/azure/create-byoc-cluster-azure.adoc

@@ -144,7 +144,7 @@ Replace the following placeholders:
 - `<cluster-id>`: Enter the cluster ID listed in the Redpanda Cloud UI. Go to the Cluster Overview page, and look in the Details section.
 - `<token>`: Enter the API token you received in step 1.
 
-include::partial$no-access.adoc[]
+include::get-started:partial$no-access.adoc[]
 
 == Next steps
 

modules/get-started/pages/cluster-types/byoc/azure/vnet-azure.adoc

@@ -4,7 +4,7 @@
 
 include::shared:partial$feature-flag-rpcn.adoc[]
 
-This topic explains how to create a Bring Your Own Virtual Private Cloud (BYOVPC) cluster. This setup allows you to deploy the Redpanda glossterm:data plane[] into your existing virtual network (VNet) and take full control of managing the networking lifecycle. Compared to a standard Bring Your Own Cluster (BYOC) setup, where Redpanda manages the networking lifecycle for you, this option provides more security.
+This topic explains how to create a Bring Your Own Virtual Private Cloud (BYOVPC) cluster. This setup allows you to deploy the Redpanda glossterm:data plane[] into your existing virtual network (VNet) and take full control of managing the networking lifecycle. Compared to a standard Bring Your Own Cloud (BYOC) setup, where Redpanda manages the networking lifecycle for you, this option provides more security.
 
 When you create a BYOVPC cluster, you specify your VNet and managed identities. The Redpanda Cloud agent doesn't create any new resources or alter any settings in your account. With a customer-managed VNet: 
 
@@ -459,4 +459,4 @@ After that completes, run:
 rpk cloud byoc azure destroy --redpanda-id ${REDPANDA_ID}
 ```
 
-include::partial$no-access.adoc[]
+include::get-started:partial$no-access.adoc[]

modules/get-started/pages/cluster-types/byoc/gcp/create-byoc-cluster-gcp.adoc

@@ -32,7 +32,7 @@ Optionally, click *Advanced settings* to specify up to five key-value custom lab
 +
 Note that `rpk` configures the permissions required by the agent to provision and actively maintain the cluster. For details about these permissions, see xref:security:authorization/cloud-iam-policies-gcp.adoc[GCP IAM permissions].
 
-include::partial$no-access.adoc[]
+include::get-started:partial$no-access.adoc[]
 
 == Next steps
 

modules/get-started/pages/cluster-types/byoc/gcp/vpc-byo-gcp.adoc

@@ -4,9 +4,9 @@
 
 include::shared:partial$feature-flag-rpcn.adoc[]
 
-This topic explains how to create a Bring Your Own Virtual Private Cloud (BYOVPC) cluster. This setup allows you to deploy the Redpanda glossterm:data plane[] into your existing VPC and take full control of managing the networking lifecycle. Compared to a standard Bring Your Own Cluster (BYOC) setup, where Redpanda manages the networking lifecycle for you, this option provides more security.
+This topic explains how to create a Bring Your Own Virtual Private Cloud (BYOVPC) cluster. This setup allows you to deploy the Redpanda glossterm:data plane[] into your existing VPC and take full control of managing the networking lifecycle. Compared to a standard Bring Your Own Cloud (BYOC) setup, where Redpanda manages the networking lifecycle for you, this option provides more security.
 
-When you create a BYOCVPC cluster, you specify your VPC and service account. The Redpanda Cloud agent doesn't create any new resources or alter any settings in your account. With BYOVPC: 
+When you create a BYOVPC cluster, you specify your VPC and service account. The Redpanda Cloud agent doesn't create any new resources or alter any settings in your account. With BYOVPC: 
 
 * You provide your own VPC in your Google Cloud account.
 * You maintain more control of your Google Cloud account, because Redpanda requires fewer permissions than standard BYOC clusters.
@@ -73,7 +73,7 @@ gcloud compute routers nats create <nat-config-name> \
 ```bash
 gcloud compute firewall-rules create redpanda-ingress \
   --description="Allow access to Redpanda cluster" \
-  --network="<vpc-name>" \
+  --network="<shared-vpc-name>" \
   --project="<host-project-id>" \
   --direction="INGRESS" \
   --target-tags="redpanda-node" \
@@ -86,7 +86,7 @@ gcloud compute firewall-rules create redpanda-ingress \
 ```bash
 gcloud compute firewall-rules create gke-redpanda-cluster-webhooks \
   --description="Allow master to hit pods for admission controllers/webhooks" \
-  --network="<vpc-name>" \
+  --network="<shared-vpc-name>" \
   --project="<host-project-id>" \
   --direction="INGRESS" \
   --source-ranges="<gke-master-cidr-range>" \
@@ -149,7 +149,7 @@ gcloud storage buckets create gs://<management-storage-bucket-name> \
 gcloud storage buckets update gs://<management-storage-bucket-name> --versioning
 ``` 
 +
-* Redpanda uses the Tiered Storage bucket for writing log segments. This should not be versioned.
+* Redpanda uses the tiered storage bucket for writing log segments. This should not be versioned.
 * Redpanda uses the management storage bucket to store cluster metadata. This can have versioning enabled.
 
 . Create service accounts with necessary permissions and roles.
@@ -169,7 +169,7 @@ cat << EOT > redpanda-agent.role
   "title": "Redpanda Agent Role",
   "description": "A role comprising general permissions allowing the agent to manage Redpanda cluster resources.",
   "includedPermissions": [
-    "compute.firewalls.get",
+   "compute.firewalls.get",
     "compute.globalOperations.get",
     "compute.instanceGroupManagers.get",
     "compute.instanceGroupManagers.delete",
@@ -208,6 +208,42 @@ cat << EOT > redpanda-agent.role
     "serviceusage.services.list",
     "storage.buckets.get",
     "storage.buckets.getIamPolicy",
+    "compute.subnetworks.use",
+    "compute.instances.use",
+    "compute.networks.use",
+    "compute.regionOperations.get",
+    "compute.serviceAttachments.create",
+    "compute.serviceAttachments.delete",
+    "compute.serviceAttachments.get",
+    "compute.serviceAttachments.list",
+    "compute.serviceAttachments.update",
+    "compute.forwardingRules.use",
+    "compute.forwardingRules.create",
+    "compute.forwardingRules.delete",
+    "compute.forwardingRules.get",
+    "compute.forwardingRules.setLabels",
+    "compute.forwardingRules.setTarget",
+    "compute.forwardingRules.pscCreate",
+    "compute.forwardingRules.pscDelete",
+    "compute.forwardingRules.pscSetLabels",
+    "compute.forwardingRules.pscSetTarget",
+    "compute.forwardingRules.pscUpdate",
+    "compute.regionBackendServices.create",
+    "compute.regionBackendServices.delete",
+    "compute.regionBackendServices.get",
+    "compute.regionBackendServices.use",
+    "compute.regionNetworkEndpointGroups.create",
+    "compute.regionNetworkEndpointGroups.delete",
+    "compute.regionNetworkEndpointGroups.get",
+    "compute.regionNetworkEndpointGroups.use",
+    "compute.regionNetworkEndpointGroups.attachNetworkEndpoints",
+    "compute.regionNetworkEndpointGroups.detachNetworkEndpoints",
+    "compute.disks.list",
+    "compute.disks.setLabels",
+    "compute.instanceGroupManagers.update",
+    "compute.instances.delete",
+    "compute.instances.get",
+    "compute.instances.setLabels"
   ],
 }
 EOT
@@ -330,7 +366,7 @@ cat << EOT > redpanda-gke.role
   "title": "Redpanda cluster utility node role",
   "description": "Redpanda cluster utility node role",
   "includedPermissions": [
-    "artifactregistry.dockerimages.get",
+   "artifactregistry.dockerimages.get",
     "artifactregistry.dockerimages.list",
     "artifactregistry.files.get",
     "artifactregistry.files.list",
@@ -402,6 +438,14 @@ cat << EOT > redpanda-gke.role
     "stackdriver.resourceMetadata.write",
     "storage.objects.get",
     "storage.objects.list"
+    "compute.instances.use",
+    "iam.serviceAccounts.getAccessToken",
+    "compute.regionNetworkEndpointGroups.create",
+    "compute.regionNetworkEndpointGroups.delete",
+    "compute.regionNetworkEndpointGroups.get",
+    "compute.regionNetworkEndpointGroups.use",
+    "compute.regionNetworkEndpointGroups.attachNetworkEndpoints",
+    "compute.regionNetworkEndpointGroups.detachNetworkEndpoints"
   ],
 }
 EOT
@@ -470,6 +514,18 @@ gcloud iam service-accounts add-iam-policy-binding <gke-service-account-name>@<s
 ```
 ====
 
+* Private Service Connect Controller service account
++
+.Show commands
+[%collapsible]
+====
+```
+gcloud iam service-accounts add-iam-policy-binding <gke-service-account-name>@<service-project-id>.iam.gserviceaccount.com \
+    --role roles/iam.workloadIdentityUser \
+    --member "serviceAccount:<service-project-id>.svc.id.goog[redpanda-psc/psc-controller]"
+```
+====
+
 == Create cluster
 
 Log in to the https://cloud.redpanda.com[Redpanda Cloud UI^], and follow the steps to xref:get-started:cluster-types/byoc/gcp/create-byoc-cluster-gcp.adoc[create a BYOC cluster], with the following exceptions:
@@ -523,7 +579,7 @@ export GOOGLE_APPLICATION_CREDENTIALS=<keyfile for service account>
 gcloud config set account $SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com
 ```
 
-. Run the following `rpk` command to validate your configuration: 
+. To validate your configuration, run: 
 +
 ```bash
 rpk cloud byoc gcp apply --redpanda-id='<redpanda-id>' --project-id='<service-project-id>' --validate-only
@@ -533,4 +589,8 @@ rpk cloud byoc gcp apply --redpanda-id='<redpanda-id>' --project-id='<service-pr
 
 . On the *Deploy* page, similar to standard BYOC clusters, log in to Redpanda Cloud and deploy the agent.
 
-include::partial$no-access.adoc[]
+include::get-started:partial$no-access.adoc[]
+
+== Next steps
+
+xref:networking:byoc/gcp/index.adoc[Configure private networking]

modules/get-started/pages/whats-new-cloud.adoc

@@ -9,6 +9,10 @@ This page lists new features added in Redpanda Cloud.
 
 == February 2025
 
+=== Improved Private Service Connect support with AZ affinity
+
+xref:networking:gcp-private-service-connect.adoc[GCP Private Service Connect] now provides the ability to allow requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs. To upgrade, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda Support^].
+
 === Serverless Pro usage limits increased
 
 xref:get-started:cluster-types/serverless-pro.adoc[Usage limits for Serverless Pro] clusters increased to: ingress = 100 MBps, egress = 300 MBps, partitions = 5000, and topics = 3000.

modules/networking/pages/configure-private-service-connect-in-cloud-ui.adoc

@@ -4,7 +4,14 @@
 
 include::shared:partial$feature-flag.adoc[]
 
-NOTE: This guide is for configuring GCP Private Service Connect using the Redpanda Cloud UI. To configure and manage Private Service on an existing public cluster, you must use the xref:networking:gcp-private-service-connect.adoc[Redpanda Cloud API].
+[NOTE]
+====
+
+* This guide is for configuring GCP Private Service Connect using the Redpanda Cloud UI. To configure and manage Private Service on an existing public cluster, you must use the xref:networking:gcp-private-service-connect.adoc[Redpanda Cloud API].
+* As of Februrary 2025, the Redpanda GCP Private Service Connect service supports AZ affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs. To upgrade, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda Support^].
+==== 
+
+
 
 The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your own VPC. Traffic over Private Service Connect does not go through the public internet because these connections are treated as their own private GCP service. While your VPC has access to the Redpanda VPC, Redpanda cannot access your VPC.
 
@@ -16,39 +23,51 @@ Consider using the endpoint services if you have multiple VPCs and could benefit
 
 == Requirements
 
-* Use https://cloud.google.com/sdk/docs/install[gcloud^] to create the consumer-side resources, such as a client VPC and forwarding rule, or modify existing resources to use the Private Service Connect service attachment created for your cluster.
+* Use the https://cloud.google.com/sdk/docs/install[gcloud^] command-line interface (CLI) to create the consumer-side resources, such as a client VPC and forwarding rule, or to modify existing resources to use the Private Service Connect service attachment created for your cluster.
 * The client VPC must be in the same region as your Redpanda cluster.
 
 == Enable endpoint service for existing clusters
 
 . In the Redpanda Cloud UI, open your https://cloud.redpanda.com/clusters[cluster^], and click **Cluster settings**.
 . Under Private Service Connect, click **Enable**. 
-. For xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[BYOVPC clusters], you need a NAT subnet with the *Purpose* set to `PRIVATE_SERVICE_CONNECT`. You can create the subnet using the `gcloud` command-line interface (CLI):
+. For xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[BYOVPC clusters], you need a NAT subnet with `purpose` set to `PRIVATE_SERVICE_CONNECT`. You also need to create VPC firewall rules to allow Private Service Connect traffic. You can use the `gcloud` CLI:
++
+NOTE: The firewall rules support up to 20 Redpanda brokers. If you have more than 20 brokers, or for help enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^].
 +
 [,bash]
 ----
 gcloud compute networks subnets create <subnet-name> \
-    --project=<project> \
-    --network=<network-name> \
+    --project=<host-project-id> \
+    --network=<shared-vpc-name> \
     --region=<region> \
     --range=<subnet-range> \
     --purpose=PRIVATE_SERVICE_CONNECT
 ----
 +
+[,bash]
+----
+gcloud compute firewall-rules create redpanda-psc \
+  --description="Allow access to Redpanda PSC endpoints" \
+  --network="<shared-vpc-name>" \
+  --project="<host-project-id>" \
+  --direction="INGRESS" \
+  --target-tags="redpanda-node" \
+  --source-ranges="10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10" \
+  --allow="tcp:30181,tcp:30282,tcp:30292,tcp:31004,tcp:31082-31101,tcp:31182-31201,tcp:31282-31301,tcp:32092-32111,tcp:32192-32211,tcp:32292-32311"
+----
++
 Provide your values for the following placeholders:
 +
 - `<subnet-name>`: The name of the NAT subnet.
-- `<project>`: The **host** GCP project ID.
-- `<network-name>`: The name of the VPC being used for your Redpanda Cloud cluster.
+- `<host-project-id>`: The host GCP project ID.
+- `<shared-vpc-name>`: The name of the VPC being used for your Redpanda Cloud cluster.
 - `<region>`: The region of the Redpanda Cloud cluster.
 - `<subnet-range>`: The CIDR range of the subnet. The mask should be at least `/29`. Each Private Service Connect connection takes up one IP address from the NAT subnet, so the CIDR must be able to accommodate all projects from which connections to the service attachment will be issued.
 +
-See the GCP documentation for https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#add-subnet-psc[creating a subnet for Private Service Connect^].
+See the GCP documentation for https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#add-subnet-psc[creating a subnet for Private Service Connect^]. 
 . For the accepted consumers list, you need the GCP project IDs from which incoming connections will be accepted.
 . It may take several minutes for your cluster to update. When the update is complete, the Private Service Connect status in **Cluster settings** changes from **In progress** to **Enabled**.
 
-NOTE: For help with issues when enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^].
-
 === Deploy consumer-side resources
 
 For each VPC network, you must complete the following steps to successfully connect to the service and use Kafka API and other Redpanda services such as HTTP Proxy.
@@ -59,14 +78,14 @@ For each VPC network, you must complete the following steps to successfully conn
 +
 [,bash]
 ----
-gcloud dns --project=<GCP Project ID> managed-zones create <DNS zone name> --description="<description>" --dns-name="<DNS Zone from the UI>" --visibility="private" --networks="<list of fully-qualified name of networks where the DNS zone will be visible>"
+gcloud dns --project=<gcp-project-id> managed-zones create <dns-zone-name> --description="<description>" --dns-name="<dns-zone-from-the-ui>" --visibility="private" --networks="<list-of-fully-qualified-names-of-networks-where-the-dns-zone-will-be-visible>"
 ----
 
 . In the newly-created DNS zone, create a wildcard DNS record using the cluster **DNS record** value. 
 +
 [,bash]
 ----
-gcloud dns --project=rp-byoc-juan-0e38 record-sets create '*.<DNS Zone from the UI>' --zone="<DNS zone name>" --type="A" --ttl="300" --rrdatas="<PSC endpoint IP>"
+gcloud dns --project=<gcp-project-id> record-sets create '*.<dns-zone-from-the-ui>' --zone="<dns-zone-name>" --type="A" --ttl="300" --rrdatas="<psc-endpoint-ip>"
 ----
 
 . Confirm that your GCP VPC firewall allows traffic to and from the Private Service Connect forwarding rule IP address, on the expected ports.
@@ -85,8 +104,4 @@ include::networking:partial$private-links-test-connection.adoc[]
 
 == Disable endpoint service
 
-In **Cluster settings**, click **Disable**. Existing connections are closed after GCP Private Service Connect is disabled. To connect using Private Service Connect again, you must re-enable the service.
-
-include::shared:partial$suggested-reading.adoc[]
-
-* xref:networking:gcp-private-service-connect.adoc[]
+In **Cluster settings**, click **Disable**. Existing connections are closed after GCP Private Service Connect is disabled. To connect using Private Service Connect again, you must re-enable the service.