fix(dynamodb): honor MICROSECOND ApproximateCreationDateTimePrecision for Kinesis destinations and streams false enablement - #3874
Conversation
… for Kinesis destinations Store the precision requested in EnableKinesisStreamingConfiguration, echo it in the enable response, and return it from DescribeKinesisStreamingDestination, so Terraform no longer destroys and recreates the destination on every apply. Stamp ApproximateCreationDateTime in epoch microseconds for MICROSECOND destinations, with a shared eventID per change across destinations. Reject precisions other than MILLISECOND and MICROSECOND with ValidationException. Add forwarder unit and end-to-end integration coverage, and document the behavior.
EnableKinesisStreamingDestination turned on DynamoDB Streams for the table. Kinesis forwarding does not need it, and Terraform saw stream_enabled drift on aws_dynamodb_table on every run. Leave the table's stream setting as the caller configured it.
…cision Apply a Kinesis streaming destination with MICROSECOND precision and check that it reads back, that a second plan is clean, and that a precision change replaces the destination once and then settles.
atosz33
changed the title
pgermosen
left a comment
pgermosen
left a comment
There was a problem hiding this comment.
Nice find, and good that you chased the second issue too. I went through it against the DynamoDB model and the provider source: the precision enum, the EnableKinesisStreamingConfiguration in the Enable response and the field in Describe all match, and the provider does mark approximate_creation_date_time_precision as ForceNew, so the replace-once assertion in the Terraform test is right. The forwarder handling is careful (a snapshot of each destination's precision, one eventID shared across precisions, exact microsecond math), and old persisted destinations read back as MILLISECOND. I also looked for anything that relied on enabling a Kinesis destination turning DynamoDB Streams on and found nothing, and you called out the behavior change in the description, which is what it needed. The Terraform test now runs and passes in CI.
I corrected the test path in the description, it pointed at compat-terrs_precision.bats and the file is compatibility-tests/compat-terraform/test/dynamodb_kinesis_precision.bats.
A few things that don't need to hold this up, for follow-ups:
- The real API also has UpdateKinesisStreamingDestination, which changes the precision on an active destination. Floci doesn't dispatch it yet. Terraform doesn't call it, so this PR isn't affected.
- The local test run box in the checklist is unticked, worth ticking if you ran it.
- Not about this PR, but the native image artifact in the compatibility workflow has a one day retention, so re-running just a failed job after a day fails at the download step. That's what tripped the CI here.
3 participants
Summary
Heey,
While testing my service, I found an issue with the
ApproximateCreationDateTimeprecision of Kinesis streaming destinations. My infra usesMICROSECONDinstead ofMILLISECOND, but Floci did not support it, so I added it.Floci ignored the precision and always reported
MILLISECOND. The Terraform provider marks this field asForceNew, so aMICROSECONDdestination was destroyed and recreated on every apply.While looking around, I also found a second issue: enabling a Kinesis streaming destination also turned on DynamoDB Streams for the table. On AWS these two features are separate, and Kinesis forwarding in Floci does not need DynamoDB Streams. Because of this, Terraform planned to turn the stream off on
aws_dynamodb_tableon every run.What changed
EnableKinesisStreamingDestinationstoresApproximateCreationDateTimePrecision, and returns it in its response and inDescribeKinesisStreamingDestination.MICROSECONDdestinations,ApproximateCreationDateTimein the Kinesis record is in epoch microseconds.MILLISECONDorMICROSECONDis rejected withValidationException.Type of change
fix:)feat:)feat!:orfix!:)AWS Compatibility
Incorrect behavior before this PR:
DescribeKinesisStreamingDestinationalways returnedApproximateCreationDateTimePrecision: MILLISECOND, even when the destination was enabled withMICROSECOND.EnableKinesisStreamingDestinationturned on DynamoDB Streams (NEW_AND_OLD_IMAGES) on the table.Behavior change
Enabling a Kinesis streaming destination no longer turns on DynamoDB Streams. If someone relied on this, they need to enable streams on the table themselves (
StreamSpecification).Tests
compatibility-tests/compat-terraform/test/dynamodb_kinesis_precision.bats): apply, second plan is clean, and aprecision change replaces the destination once and then settles. It fails on
mainand passes on this branch.Checklist
./mvnw testpasses locally