add http post build event uploader #nonprod

[KlotzAndrew]

this is an alternative to using the manifold client to upload post-build events

[buck2]
log_upload_url = http://localhost:8080

currently post build event are not able to be pushed to a log/metrics store for open source projects. buck2 uses a ManifoldClient (internal meta blob storage) with a hardcoded fbinternal address to push this data on post build

this adds a config [log_upload_url] to the .buckconfig that enables a http client to put the event blob to a configurable URL, instead of the manifold client to a fbinternal url

this change only updates a single manifold client instantiation for post-build event uploads, there are other paths that use the manifold client that could be upgraded to also use the http client if configured. they are being left out of this change for now to keep the scope smaller

some notes:
this http uploader does not support chunked uploads, while the manifold client does support that. these event blobs seem to be a few mb for large builds so the oss path might not have as much of a need for chunked uploads as meta, keeping a simple upload is likely a feature we want for easy out-of-the-box integration with metrics collection

should_upload_log was previously guarding against oss builds from publishing logs, with this change that should no longer be a restriction. it was only being used in 2 locations, that oss condition is lifted to the 2nd location not covered by this http uploader

[thoughtpolice]

Interesting. Recently I actually solved this a different way in my fork – see 3e23cac and 25a0062; the net effect is that I can put this in my .buckconfig.local:

[buck2]
log_upload_cmd   = buck_log_uploader --trace-id $TRACE_ID
log_download_cmd = buck_log_downloader $TRACE_ID $OUT

these two commands are located here, they are built on top of my custom REAPI server: https://github.com/thoughtpolice/a/tree/canon/src/tools/cache-server/client

The idea here is just to completely reuse the existing Remote Cache server you hopefully have available; this tool will upload the logfile into a CAS blob, and then it will use the Remote Asset API to give it a tag like buck2-logs://$TRACE_ID and then when you do buck log replay --trace-id xyz it will try fetching buck2-logs://xyz in turn.

I've found this to be a really good approach since it reuses my existing infrastructure naturally and been meaning to propose it; but I've only been working with it for about 2 weeks now.

[KlotzAndrew]

Oh that is super cool, I've been deliberating on how to make the protocol generic. Have you thought about how to make the command support running on multiple platforms?

I wonder if running bxl would solve that problem