BucketV2 to Bucket aliasing is no longer working

[t0yv0]

Describe what happened

BucketV2 is aliased to Bucket but is not able to seamlessly read its state.

References: #4470

Sample program

Starting from this program:

name: y1
runtime: yaml
resources:
  my-bucket:
    type: aws:s3:Bucket
    properties:
      bucket: "my-bucket-26224917"

      serverSideEncryptionConfiguration:
        rule:
          applyServerSideEncryptionByDefault:
            sseAlgorithm: "AES256"
      lifecycleRules:
         - enabled: true
           expiration:
             days: 30
      policy:
        fn::toJSON:
          Version: "2012-10-17"
          Id: "PutObjPolicy"
          Statement:
            - Sid: "DenyObjectsThatAreNotSSEKMS"
              Principal: "*"
              Effect: "Deny"
              Action: "s3:PutObject"
              Resource: "arn:aws:s3:::my-bucket-26224917/*"
              Condition:
                "Null":
                  "s3:x-amz-server-side-encryption-aws-kms-key-id": "true"
      tags:
        Environment: "Dev"
      objectLockConfiguration:
        objectLockEnabled: "Enabled"
      versioning:
        enabled: true

And going to this program:

name: y1
runtime: yaml
resources:
  my-bucket:
    type: aws:s3:BucketV2
    properties:
      bucket: "my-bucket-26224919"
      serverSideEncryptionConfigurations:
        - rules:
          - applyServerSideEncryptionByDefaults:
              - sseAlgorithm: "AES256"
      lifecycleRules:
         - enabled: true
           expirations:
             - days: 30
      policy:
        fn::toJSON:
          Version: "2012-10-17"
          Id: "PutObjPolicy"
          Statement:
            - Sid: "DenyObjectsThatAreNotSSEKMS"
              Principal: "*"
              Effect: "Deny"
              Action: "s3:PutObject"
              Resource: "arn:aws:s3:::my-bucket-26224919/*"
              Condition:
                "Null":
                  "s3:x-amz-server-side-encryption-aws-kms-key-id": "true"
      tags:
        Environment: "Dev"
      objectLockEnabled: true
      versionings:
        - enabled: true

The alias kicks in and sends the Bucket state to the BucketV2 resource to process. However due to the accumulated schema drift, the reading of the Bucket state no longer works. Instead it generates this error:

    warning: urn:pulumi:t2::y1::aws:s3/bucketV2:BucketV2::my-bucket verification warning: Use the aws_s3_bucket_policy resource instead
    warning: urn:pulumi:t2::y1::aws:s3/bucketV2:BucketV2::my-bucket verification warning: Use the aws_s3_bucket_server_side_encryption_configuration resource instead
    warning: urn:pulumi:t2::y1::aws:s3/bucketV2:BucketV2::my-bucket verification warning: Use the aws_s3_bucket_versioning resource instead
    error: diffing urn:pulumi:t2::y1::aws:s3/bucketV2:BucketV2::my-bucket: 1 error occurred:
        * [server_side_encryption_configuration] missing expected [

Log output

See above.

Affected Resource(s)

Bucket
BucketV2

Output of pulumi about

CLI          
Version      3.130.0
Go Version   go1.22.6
Go Compiler  gc

Plugins
KIND      NAME  VERSION
resource  aws   unknown
language  yaml  unknown

Host     
OS       darwin
Version  14.6.1
Arch     arm64

This project is written in yaml

Current Stack: anton-pulumi-corp/y1/t2

TYPE                  URN
pulumi:pulumi:Stack   urn:pulumi:t2::y1::pulumi:pulumi:Stack::y1-t2
pulumi:providers:aws  urn:pulumi:t2::y1::pulumi:providers:aws::default
aws:s3/bucket:Bucket  urn:pulumi:t2::y1::aws:s3/bucket:Bucket::my-bucket


Found no pending operations associated with t2

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/anton-pulumi-corp
User           anton-pulumi-corp
Organizations  anton-pulumi-corp, moolumi, demo, pulumi
Token type     personal

No dependencies found

Pulumi locates its logs in /var/folders/gd/3ncjb1lj5ljgk8xl5ssn_gvc0000gn/T/com.apple.shortcuts.mac-helper// by default

Additional context

We must have regressed this somewhere but failed to call out in https://www.pulumi.com/blog/announcing-6-0-of-the-pulumi-aws-classic-provider/

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[dstockhammer]

We ran into the same issue. Fixed by manually editing the state via pulumi state edit (which was very scary - some kind of diff output would be nice).

In all resources of type aws:s3/bucketV2:BucketV2, replace the old singular properties with the new plural properties.

Old

"serverSideEncryptionConfiguration": {
  "rule": {
      "applyServerSideEncryptionByDefault": {
          "kmsMasterKeyId": "",
          "sseAlgorithm": "AES256"
      },
      "bucketKeyEnabled": true
  }
},
"versioning": {
    "enabled": false,
    "mfaDelete": false
},

New

"serverSideEncryptionConfigurations": [
  {
    "rules": [
      {
        "applyServerSideEncryptionByDefaults": [
          {
            "kmsMasterKeyId": "",
            "sseAlgorithm": "AES256"
          }
        ],
        "bucketKeyEnabled": true
      }
    ]
  }
],
"versionings": [
  {
    "enabled": false,
    "mfaDelete": false
  }
],

[corymhall]

@dstockhammer thanks for the info! It looks like in order for us to fix this we need to do some state transforms. Maybe something like this to convert all the objects to arrays.

TransformFromState: func(ctx context.Context, props resource.PropertyMap) (resource.PropertyMap, error) {
	if versioning, ok := props["versioning"]; ok && versioning.IsObject() {
		props["versionings"] = resource.NewArrayProperty([]resource.PropertyValue{versioning})
		delete(props, "versioning")
	}

It would be a bit verbose because there are some properties that have nested properties that need to be converted. Maybe there is a way to walk the schema and add a transform for every property that changed to an array.

[t0yv0]

There's a few more discrepancies between the schemas of Bucket vs BucketV2 that have accumulated over time, perhaps worth checking out in case we can program a complete migration method.