Repository navigation
docs: add volunteer guide for AWS ML pipeline backup and cleanup - #329
Open
arnoldcastro5000 wants to merge 6 commits into
Open
arnoldcastro5000 wants to merge 6 commits into
arnoldcastro5000 wants to merge 6 commits into
Conversation
Add a plain-language guide, written in simplified technical English for volunteers, explaining how we back up an old, unused machine-learning system in the AWS account before deleting it to save cost. Includes: - docs/aws-ml-pipeline-backup/index.md - volunteer guide with a glossary - docs/aws-ml-pipeline-backup/runbook.md - detailed engineer runbook - docs/aws-ml-pipeline-backup/console-runbook.md - AWS console clickpaths - docs/aws-ml-pipeline-backup/data-loss-risk-register.md - safety reference - link from docs/index.md Documentation only. No infrastructure or code changes.
Remove the 12-digit AWS account ID from the volunteer guide and the runbooks, including where it appeared inside the archive bucket name and IAM ARNs. Replace every occurrence with <ACCOUNT_ID>. Resource IDs are kept. No functional change; documentation only.
Move all account-specific values into a 'Configuration' block at the top of each runbook and use tokens ($ACCOUNT_ID, $ARCHIVE_BUCKET, $UNLOAD_ROLE_ARN, $BREAKGLASS_ROLE_ARN) in the body. No IAM ARN now carries an account ID. The volunteer guide drops the account number and refers to the bucket generically. Also de-link two template files that are not part of this docs set, to avoid broken links.
arnoldcastro5000
marked this pull request as ready for review
September 15, 2026 01:01
A live re-check on 2026-09-23 found the inventory unchanged, but no source had deletion protection: - Add a SETUP step to enable RDS deletion protection on both databases, set 7-day retention on eu-north-1, set DeleteOnTermination=false on the 4 EBS root volumes, and enable termination protection on the On-Demand instances. - Make the stop of running instances i-04b and i-0e1 mandatory before the snapshot (both write 0.2-0.8 GB per day). - Warn that i-04b is a persistent Spot instance: cancelling its disabled Spot request while it is stopped terminates it. - Record the new vector in the data-loss risk register.
The archive relied on a Deny-delete bucket policy that any principal with s3:PutBucketPolicy can remove. An IAM scan on 2026-09-23 found 7 IAM users plus root with that permission, most without MFA. After the sources are deleted, the archive is the only copy. - Enable bucket versioning at creation (Object Lock stays off; it can be added later without a new bucket). - Transition noncurrent versions to Glacier Instant Retrieval too; keep zero Expiration and no NoncurrentVersionExpiration rules. - Note that MFA Delete cannot be used with a lifecycle rule. - Update the risk register and the volunteer guide.
IAM Access Analyzer found no syntax issue, but the Deny on the delete APIs alone does not stop deletes. A lifecycle Expiration rule deletes objects without those APIs, and AWS requires a Deny on s3:PutLifecycleConfiguration to block deletes. The policy, versioning and bucket-delete actions were also open, so an admin could remove the protection first. - Add a Deny on PutLifecycleConfiguration, PutBucketVersioning, PutBucketPolicy, DeleteBucketPolicy and DeleteBucket for every principal except the break-glass role. - Reorder SETUP: bucket, lifecycle rule, IAM roles, then the bucket policy last. - Document how to create the archive-breakglass role (MFA-only trust, root path). - Fix the DataSync step reference (4.2, not 3.2).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds documentation only. It explains, in plain language for volunteers, how we safely back up an old and unused machine-learning system in our AWS account before we delete it to save cost. No infrastructure or code changes.
What this adds
A new docs folder
docs/aws-ml-pipeline-backup/:index.md- the volunteer guide. Written in simplified technical English, with a glossary of every AWS term. Covers what we back up, where it goes, the safety rule, the cost, and who does what.runbook.md- the detailed engineer runbook (exact commands).console-runbook.md- the same steps as clicks in the AWS web console, validated against AWS docs.data-loss-risk-register.md- the safety reference: how backups can silently fail and how we prevent it.docs/index.md.Why
The AWS account holds a dormant tree-image ML pipeline, idle about 15 months, costing roughly $500/month. Deleting it saves about $350 to $425/month. We must copy any real data to safe, cheap storage first. This documentation records that backup plan so any volunteer can understand and help run it.
Scope and status
How to review
Read
docs/aws-ml-pipeline-backup/index.mdfirst (about 5 minutes). It links the detailed runbooks. Check that the language is clear for a non-expert volunteer.Notes
Account <ACCOUNT_ID>. Backup target: a new S3 bucket in eu-central-1, with versioning on, Glacier Instant Retrieval, kept indefinitely.
Console navigation was validated against AWS documentation on 2026-09-15.
Live re-check on 2026-09-23: the 18-resource inventory is unchanged. The re-check added a SETUP step to protect the sources (RDS deletion protection,
DeleteOnTermination=falseon the EBS root volumes, EC2 termination protection), made the stop of the 2 running instances mandatory, and added a Spot-request warning for i-04b.2026-09-23: the archive bucket now has versioning on (was off). Reason: the Deny-delete bucket policy can be removed by any of the 7 IAM users plus root that have
s3:PutBucketPolicy, and the archive becomes the only copy after the sources are deleted.2026-09-23: the archive bucket policy now also denies lifecycle, versioning, policy and bucket-delete changes (except the break-glass role), because a lifecycle Expiration rule can delete objects without the delete APIs. SETUP applies the policy last. Validated with IAM Access Analyzer (0 findings).