Importing CloudFormation Resources With CDK L2s

CloudFormation allows you to import existing resources into a stack. This is a great feature, but when you combine it with the abstractions that the CDK provides, you can find yourself in some hard-to-solve situations. We recently ran across one of these scenarios and I wanted to share it with you.

If you'd rather watch a video, I've got you covered here on YouTube.

While this is a real scenario we ran across recently at Defiance Digital, I've changed a few of the insignificant details to protect the innocent.

The Problem

A production environment needed a quick restoration of a database to get the system up and running. While this could (and should) have been done in the CDK, it wasn't at the time and the RDS instance was created manually from a snapshot. This was a one-off situation and the team was under pressure to get the system back up and running.

While we had the RDS instance up and running, we needed to import it into the CDK stack so that it could be managed through our normal IaC pipelines.

The Solution

We had a couple choices:

1. Retain and delete the old RDS instance from the CDK code. Add it back in and use CloudFormation's import to tie the 'new' RDS resource to the manually created instance.
2. Add a new CDK resource that would represent the manually created RDS instance and import it into the stack.

Option #1 wasn't really a good option. In this case we'd have to also delete all the supporting resources and re-import them and if there were any complications along the way we could end up in a worse situation than we started.

Option #2 felt like the safer choice. We could add a new CDK resource that represented the RDS instance and import it into the stack. However, there was a catch. The RDS instance that was restored manually was using the same Subnet Group as the old instance. So now we have this:

1. The original CDK RDS instance L2 (CDK1) that defines the original CFN RDS instance (DB1) (that we no longer need) and the CFN Subnet Group (SG1) (that we do need).
2. A new CDK RDS instance L2 (CDK2) that defines the CFN RDS instance (DB2) and re-uses the CFN Subnet Group (SG1). We can't let it create its own Subnet Group because that would require a replacement of the RDS instance.

The code looks something like this (simplified):

export class DatabaseStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const rdsInstance = new DatabaseInstance(this, 'CDK1', {
      // ...
    });

    const subnetGroup = rdsInstance.node.findChild('SubnetGroup') as SubnetGroup;
    const newRdsInstance = new DatabaseInstanceFromSnapshot(this, 'CDK2', {
      // ...
      subnetGroup, // Re-use the subnet group
    });
  }
}

With the CDK code set up, we can generate a template and then import the resources through CloudFormation.

But, a new problem happens when we later clean up the CDK code to remove the old RDS instance (CDK1). Because that L2 also defines the Subnet Group (SG1), then deleting the L2 will delete that Subnet Group, and then what does the CDK2 construct do?

Deleting the resource itself isn't a problem. The problem is that the CDK constructs are tied together in a way that deleting the CDK1 resource means there is no longer a Subnet Group for the CDK2 resource to use.

We can let the Subnet Group get created by the CDK2 construct, but the logical IDs won't align and CloudFormation will see this as a replacement operation. We clearly can't let that happen.

So we resolve this by overriding the logical ID of the Subnet Group in the CDK2 construct to match the logical ID of the CDK1's Subnet Group (we'll use my logical id remapper aspect to handle it). Although the original Subnet Group is deleted, the new Subnet Group created will have the same logical ID and CloudFormation will see this as a no-op. Of course, we need to make sure the CDK2's Subnet Group isn't materially different from the original, or else a replacement would be necessary, requiring a replacement of the instance and defeating the entire point. But, Subnet Groups are very simple definitons and making them match is trivial.

export class DatabaseStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const newRdsInstance = new DatabaseInstanceFromSnapshot(this, 'CDK2', {
      // allow the subnet group to be created by this construct
    });

    Aspects.of(newRdsInstance).add(new LogicalIdMapper({
      map: {
        CDK2SubnetGroupNewLogicalId: 'CDK1SubnetGroupOriginalLogicalId',
      }
    }));
  }
}

Recap

Alright, so I'll recap all the steps that went into this.

1. CDK code already existed that defined an RDS instance and a Subnet Group.
2. A new RDS instance was created manually outside of the CDK code that we wanted managed by the CDK code. It used the same Subnet Group as the existing database.
3. A new RDS instance was created in the CDK code and had it use the original construct's Subnet Group.
4. CloudFormation imported the new RDS instance into the stack.

-- At this point we had a stable stack, where there were two RDS instances properly managed by CloudFormation, and we had to clean up the original instance.

5. Remove the original RDS instance from the CDK code.
6. Remove the usage of the Subnet Group reference and allow the L2 construct to create a new Subnet Group.
7. Override the logical ID of the Subnet Group in the L2 construct to match the original Subnet Group's logical ID.
8. Diff & deploy.

Hope this helps you in the future!