Backup-DbaDbCertificate: Don't use decryption password if certificate is encrypted by the database master key

[andreasjordan]

Currently we get this output in the AppVeyor tests:

To avoid this output we now create the certificate with a password.

[andreasjordan]

@claude Can you help me find a way to create the certificate that it can be used in the test for Backup-DbadbCerficifate and also in the AG related tests like Add-DbaAgDatabase which now fail with:

Running C:\github\dbatools\tests\Add-DbaAgDatabase.Tests.ps1 ...[07:25:43][New-DbaEndpoint] Failure | An exception occurred while executing a Transact-SQL statement or batch.The certificate 'dbatoolsci_AGCert' is not valid for endpoint authentication. The certificate must have a private key encrypted with the database master key and current UTC date has to be between the certificate start date and the certificate expiration date.
Completed C:\github\dbatools\tests\Add-DbaAgDatabase.Tests.ps1 in 10742ms

[claude]

Claude finished @andreasjordan's task in 9m 6s —— View job

---

Analyzing the Certificate Compatibility Issue

• Read changed files to understand current state
• Investigate AG certificate requirements vs backup requirements
• Find the root cause of the conflict
• Implement the fix

---

Root cause

Both InstanceSingle and InstanceHadr map to sql2022 in AppVeyor, so dbatoolsci_AGCert (created in master by appveyor.SQL2022.ps1) gets picked up by the "backup all certs on instance" test.

The conflict:

• AG endpoint auth requires the cert's private key to be encrypted by the database master key (DMK) — ENCRYPTION BY PASSWORD breaks this with: "The certificate must have a private key encrypted with the database master key"
• Backup-DbaDbCertificate was calling the 4-param $cert.export(..., encPass, decPass) even for DMK-encrypted certs, which SQL Server rejects with: "No decryption password should be provided because the private key of this certificate is encrypted by a master key"

Solution (3 files changed)

tests/appveyor.SQL2019.ps1 / tests/appveyor.SQL2022.ps1
Reverted dbatoolsci_AGCert to DMK encryption (remove ENCRYPTION BY PASSWORD). This restores AG endpoint authentication.

public/Backup-DbaDbCertificate.ps1
When both EncryptionPassword and DecryptionPassword are provided, check cert.PrivateKeyEncryptionType first:

• If MasterKey: use the 3-param export (DMK handles decryption automatically, DecryptionPassword is invalid/ignored with a Verbose log)
• Otherwise: use the 4-param export as before

tests/Backup-DbaDbCertificate.Tests.ps1
The "Backs up all db certs for an instance" test was expecting HaveCount 3 only because the DMK cert was failing and returning nothing. With the fix, dbatoolsci_AGCert succeeds and appears in results. Changed the assertion from an exact count to containment checks for the 3 expected certs — which is better test design regardless (an exact count is fragile if other certs exist on the instance).
|