Reproducible Builds

[IzzySoft]

It seems you're switching between building on Windows and building on Linux. Each time you do that, RBs fail, and we have to adjust the recipe manually (as when building on Windows, files in META-INF/services/* have Windows line endings we need to match – which are not there when building on Linux).

As that always means extra manual work, and delayed releases: could you decide to stick with one of the two? Ideally, with Linux, which the current release was built on 🙈 Thanks in advance!

[yamin8000]

It seems you're switching between building on Windows and building on Linux. Each time you do that, RBs fail, and we have to adjust the recipe manually (as when building on Windows, files in META-INF/services/* have Windows line endings we need to match – which are not there when building on Linux).

As that always means extra manual work, and delayed releases: could you decide to stick with one of the two? Ideally, with Linux, which the current release was built on 🙈 Thanks in advance!

Well that's odd. I'm using Windows since I remember. I never built this app using Linux. Maybe something in the build process of Android Studio and Gradle changed this behavior.

[IzzySoft]

That's really odd then, because it didn't happen only once, but multiple times now. Actually, from the history as it looks by the recipes: Win=>Linux (one build)=>Win=>Linux (current). Always multiple "Windows-like" builds, and from time to time a single "Linux-like" build. Difference is just the line endings in some files.

[yamin8000]

That's really odd then, because it didn't happen only once, but multiple times now. Actually, from the history as it looks by the recipes: Win=>Linux (one build)=>Win=>Linux (current). Always multiple "Windows-like" builds, and from time to time a single "Linux-like" build. Difference is just the line endings in some files.

I see. I just checked my Android Studio and line separators are set to CRLF (Windows style). But I assume this is set for files I will type and edit, not files that are put together using the build process such as the ones you mentioned.
Between current and the last version I updated some libraries and Gradle and AGP, if there's a change in them there will more cases like me which you will probably know about. I also updated part of my Git process specifically GPG tools (Kleopatra), but I don't think that's a factor because files you mentioned are generated using Android Studio and Gradle and are not part of git history.
Finally, I think the ultimate solution to this matter is that I should build my APK the way you and F-Droid are building. Can you share with me a guide for that? Since it appears to me that Android's build process is nondeterministic.

[IzzySoft]

But I assume this is set for files I will type and edit

and should be taken care for consistently by Git, per its configuration.

because files you mentioned are generated using Android Studio and Gradle and are not part of git history.

Correct. Which is why the developer building on Windows, cannot control this, and we have to take care for it with our recipes.

Can you share with me a guide for that?

I see you've already established a Github action. Those usually work fine concerning RB: the environment is well documented, so we can adjust our recipes easily to match that (OS, build paths, even CPU count if necessary). Automating Android APK Builds with GitHub Actions (The Sane Way) is a guide for this, should you need further orientation. We do not suggest to upload your keystore and credentials as Github Secrets, though (for the added risks: should someone manage to get access to your Github account, they'd then also have access to those) – but we do not "forbid" it either. Just make sure with your Signing Config, that signing is not "mandatory" (but e.g. will only be "enforced" if the keystore file exists), as we need to build unsigned APKs here.

[yamin8000]

@IzzySoft Based on guides you mentioned I made some changes, another action will create an unsigned debug APK. However, I think you need an unsigned release APK, right?
BTW I just realized the last version 1.7.9 was OK with F-Droid. Are you stricter on RB or something is different on machines used by you and F-Droid?

[IzzySoft]

I think you need an unsigned release APK, right?

From you, we'll need the signed release APK – because that's what will be distributed by our repo. Ourselves, we need to build an unsigned APK; we then use apksigcopier to copy over the signing part, to compare the resulting APK to yours (if both are identical, RB was confirmed). It's still your APK that will be distributed; ours is just thrown away at the end of the RB process.

I just realized the last version 1.7.9 was OK with F-Droid. Are you stricter on RB or something is different on machines used by you and F-Droid?

Well: 1.7.9 was RB with us as well. As were the previous versions. It's just that we have to adjust the recipe whenever the line endings change. F-Droid had to do the same, as their recipes show:

  - versionName: 1.7.3
    versionCode: 46
    commit: 259c26e39bf1745f1f9dbb33e5b92a58cbec8a05
    subdir: app
    gradle:
      - yes
    srclibs:
      - reproducible-apk-tools@v0.2.3
    postbuild: $$reproducible-apk-tools$$/inplace-fix.py --zipalign fix-newlines '$$OUT$$'
      'META-INF/services/*'

  - versionName: 1.7.5
    versionCode: 48
    commit: d3da2ccb330f2542bd024342d346f0d4ba0582f8
    subdir: app
    gradle:
      - yes
    srclibs:
      - reproducible-apk-tools@v0.3.0
    postbuild: $$reproducible-apk-tools$$/inplace-fix.py --zipalign --page-size 16
      fix-newlines $$OUT$$ 'META-INF/services/*'

  - versionName: 1.7.8
    versionCode: 51
    commit: f6573ff6735a30c938304697fb36d621aae4b953
    subdir: app
    gradle:
      - yes
    srclibs:
      - reproducible-apk-tools@v0.3.0
    postbuild: $$reproducible-apk-tools$$/inplace-fix.py --zipalign --page-size 16
      fix-newlines $$OUT$$ 'META-INF/services/*'

  - versionName: 1.7.9
    versionCode: 52
    commit: c69e71540c4eaf2cecad7bbe3769a8f62afa7bfb
    subdir: app
    gradle:
      - yes

And they skipped 1.7.4, as you can see – that was the other release without the Windows line endings. Was most likely not repaired before 1.7.5 kicked in, having them again.

[yamin8000]

@IzzySoft Sorry for the huge delay, I can't properly function and respond due to Internet blackout. I'll try fixing this as soon as possible with your help.

[IzzySoft]

Totally understood! I should have looked up your location… IIRC, you were in the process of having release APKs built bei GitHub Actions. At the point where that's completed, the "line endings" should be consistent (\n, and never Windows ones \r\n) – and thus the problem gone.

But you indeed have much more pressing and important things to deal with. Wishing you and your people to have peace and decent conditions soon, inshallah!