Skip to content

Add migrate-db command and support airflow downgrades#247

Merged
michaelfu1029 merged 5 commits intoaws:mainfrom
michaelfu1029:main
May 14, 2025
Merged

Add migrate-db command and support airflow downgrades#247
michaelfu1029 merged 5 commits intoaws:mainfrom
michaelfu1029:main

Conversation

@michaelfu1029
Copy link
Copy Markdown
Contributor

@michaelfu1029 michaelfu1029 commented Apr 24, 2025

Issue #, if available:

Description of changes:
Added a new command migrate-db for a special container that will run the same airflow db init/upgrade logic but will also check for an environment variable MWAA__CORE__TARGET_VERSION to downgrade. Existing inplace upgrade will not be changed for now.

The migrate_with_downgrade.py file is a copy or migrate.py but with an additional downgrade check

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Comment thread images/airflow/2.10.1/python/mwaa/database/update.py
Comment thread images/airflow/2.10.1/python/mwaa/database/update.py Outdated
Copy link
Copy Markdown
Contributor

@sudiptoghosh01 sudiptoghosh01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please include the corresponding unit test for your code changes. I see the quality checks are failing for this change.

@michaelfu1029
Copy link
Copy Markdown
Contributor Author

michaelfu1029 commented May 9, 2025

Unit tests are added, is only failing because did not add test for

if command == "migrate-db":
        await airflow_db_migrate(environ)
        print("Finished running db validations")
        return

but there is already a unit test for airflow_db_migrate() so I don't believe this is necessary. I believe we also don't have coverage tests for other commands

kuyperse
kuyperse previously approved these changes May 9, 2025
@sudiptoghosh01
Copy link
Copy Markdown
Contributor

Unit tests are added, is only failing because did not add test for

if command == "migrate-db":
        await airflow_db_migrate(environ)
        print("Finished running db validations")
        return

but there is already a unit test for airflow_db_migrate() so I don't believe this is necessary. I believe we also don't have coverage tests for other commands

I believe you should cover this branch as well. Similar to what we have for other commands, please refer to this code pointer.

kuyperse
kuyperse previously approved these changes May 13, 2025
Comment thread tests/images/airflow/2.9.2/python/mwaa/test_entrypoint_2_9_2.py Outdated
@michaelfu1029 michaelfu1029 merged commit 4ced7e1 into aws:main May 14, 2025
1 check passed
michaelfu1029 added a commit that referenced this pull request Jul 16, 2025
*Issue #, if available:*

*Description of changes:*
Porting this change into this 3.0.2 image
#247

The 3.0.2 image was copied off an old version so it does not have the
new required migrate-db command

Tested by running the run script with some modifications to go into the
migratedb flow

By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.

Co-authored-by: Michael Fu <miklfu@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants