Port pony to Haiku

[ahwayakchih]

Hi,

This is my attempt to port Pony to Haiku operating system (https://www.haiku-os.org/).

It passes all the tests (but i had to force regression-1118 to use just 2 threads on Haiku, otherwise it times out, and disable error same way as Windows has it disabled), all examples work:
pony-test.log

Runtime tracing works, except for crash when actor behavior category is enabled (i'm attaching crash log, maybe you could help me fix that?):
net-5509-debug-17-05-2026-16-58-34.report.txt.

I did get some crashes with systematic_testing, but i have to retest it yet.

I did not test LTO, pony-lint and pony-lsp (i mean, besides checking if executable works).
pony-doc seems to work too.

libponyc and libponyrt benchmarks work, but they warn about not being able to set thread affinity (but i think that may be because of benchmark library?):
pony-benchmark.txt

If you think this has a chance to be merged, i can try to prepare CI config for running Haiku, but it would have to be a bit... hacky. Haiku is not Linux or BSD, so running it means running QEmu, and building pony requires over 10 GB of space on disk. My idea is to prepare install image, and then separate data disk (qcow) for pony. Disks could be cached between CI runs. First build run would be horribly long (it took over half a day on my laptop ;).

[redvers]

I will tell you that there are BeOS fans among the ponyc maintainers ;)

I found inspiration in https://www.haiku-os.org/legacy-docs/bebook/TheInterfaceKit_Overview_Introduction.html

[SeanTAllen]

Hi. Thanks for taking this on. I haven't looked at all. Some general comments.

I have a couple of large changes coming that I would want to land before these and would then ask you to incorporate and update again.

We need to talk about if we want to take this on as platform we support in the codebase. I love BeOS dearly. I learned to "really C++" on it. You will find my name in the old BeOS newsletter. But I am hesitant.

I'd like to see things like:

        ifdef haiku then
          // Workaround haiku bug with parsing "nan(123)": https://dev.haiku-os.org/ticket/20092
          if (errno == 0) and (endp == ptr.offset(_size - 1)) and (this.at_offset(_size.isize() - 1)? == 0x29) then
            res

addressed upstream and not carry the cost of this in the codebase if we were to accept it.

I notice there are at least a few such things.

---

Linking would need to be made to work with the new embedded linker that we are moving all platforms to before merging. I'm not going to take on doing that work on Haiku. It would need to be done before merging. All the Linuxes, MacOS, and Windows currently use it. The BSDs have yet to be ported over.

---

I note that there are comment like "under pressure example consistently crashes". This and other issues would need to be resolved before merging.

---

Lastly, it would need a tier 3 CI job like what OpenBSD, FreeBSD, and DragonFly have.

Before you undertake any work related to the above, I'd suggest waiting for our decision if we would want to take this on to maintain long term.

[SeanTAllen]

I will tell you that there are BeOS fans among the ponyc maintainers ;)

I found inspiration in https://www.haiku-os.org/legacy-docs/bebook/TheInterfaceKit_Overview_Introduction.html

Sylvan was also a BeOS developer. Both of us were.

[ahwayakchih]

Thanks for checking and answering :).

I will tell you that there are BeOS fans among the ponyc maintainers ;)

I found inspiration in https://www.haiku-os.org/legacy-docs/bebook/TheInterfaceKit_Overview_Introduction.html

Great! BeOS was my favorite OS, unfortunately i learned about it only around BeOS R5 PE, so not long before the end. But i was lucky enough to use it as my one and only OS for a few years, before i had to change hardware :).

Sylvan was also a BeOS developer. Both of us were.

Even better :D! Thanks!
Now i know why, when i've read about pony, i thought "this could be perfect for Haiku, with all it's multi-threading wrapped by actors, it could make writing UI easier for newcommers (or lazy people like me ;)".

I have a couple of large changes coming that I would want to land before these and would then ask you to incorporate and update again.
addressed upstream and not carry the cost of this in the codebase if we were to accept it.

Sure, i already saw change about strtof/d, so the ugly workaround i did was mainly to pass the tests and i did plan to remove it eventually. I guess it would be better to just ifdef the only two test cases that fail on Haiku? I reported the problem, but i don't think they'll work on this anytime soon.

The other workaround is for socket shutdown, and i plan to prepare test case and report it (unless it's just something that i messed up in asio implementation and i'll be able to fix it myself).

Two smaller things are in tests:

• one in regression-1118 (with more threads it times out when tests run, but it does not fail or crash when i run it separately, it's just slow),
• error.cc test, but that's already ifdefed for Windows too.

I was trying to keep changes minimal, so most of it is just adding "or haiku" or "and not haiku" or error/signal constants in places where platform is already checked in code.

Two "bigger" chunks of code are in:

• cpu.c: i would prefer to add some platform-specific file and call to it from cpu.c, but i was not sure if you'd like more files or not (Linux code there is also "big" for that file, but i understand that's one of main platforms, so it has "privileges" :),
• wfo.c: new file for asio on Haiku, but other implementations have own files, so i suspect that's ok.
• edited genexe.cc: forgot about this one: bigger chunk of code is to support rdef files, but i kept it in separate functions, so hopefully it's not too annoying for others.

I plan to remove custom logging from wfo.c (LOG and LOGLIST stuff). It's suboptimal, it just helped me a lot while implementing asio. I was thinking about adding stuff to runtime tracing, but decided to postpone it, because that could mess with other platforms (either it would be strange that there are functions that only one platform uses, or i would have to implemented it for all of them, some of which i would not have a way to test besides waiting for CI).

There is kevent implementation for Haiku, but it's missing some of the stuff used in pony. I wanted to see if i can write Haiku-specific implementation anyway ;).

I was also thinking about implementing threading using native threads and ports, but that would require more changes, because pthread* stuff is not contained in a single file. Also messaging could be slower then, because it would need to pass through Haiku kernel then, and AFAIK it's all on pony side now.

Linking would need to be made to work with the new embedded linker that we are moving all platforms to before merging. I'm not going to take on doing that work on Haiku. It would need to be done before merging. All the Linuxes, MacOS, and Windows currently use it. The BSDs have yet to be ported over.

Hmm... i could try it, but i never wrote any linking (or compiler, for that matter) code before, so... i would need some pointers regarding what's needed and where, or maybe i'll wait for BSDs and follow their steps?

BTW i'm not sure yet about static libs. BeOS and Haiku are all for shared libraries, and i did not have a look yet to see how to make sure pony stuff links dynamically and then generate ponylang package with correct content (to add it to HaikuDepot).

I note that there are comment like "under pressure example consistently crashes". This and other issues would need to be resolved before merging.

Yes, of course. But under pressure works fine now. Actually nothing crashes AFAIK, except for runtime tracing actor behaviors. Not sure why. It looks like the tracing thread calls a function chain that ends up checking if tracing is enabled and then it crashes. It's kinda similar to how timers were crashing, only there thread ctx (like on Windows) was needed, and it looked like thread local variable access was crashing. My guess is that, it's because it's Haiku's thread that calls timer function. Once i changed it to just "ping" asio thread, which in turn sends event, timers started working ok 100% of time.

Lastly, it would need a tier 3 CI job like what OpenBSD, FreeBSD, and DragonFly have.

Yeah, that's why i was already thinking about how to implement that.

Before you undertake any work related to the above, I'd suggest waiting for our decision if we would want to take this on to maintain long term.

OK, thanks for at least considering it anyway. I know Haiku is way smaller in popularity than other platforms, and i understand well that maintaining project takes precious time. So, if it means too much of a burden for pony team (for now?), i can keep fork updated from time to time and maybe you can reconsider merging in the future :).

[SeanTAllen]

I'm completely ok if we decide to move forward with Haiku support of treating it as a real option and leaning into its own APIs rather than doing things like riding on a kqueue compatibility layer. So long as those Haiku APIs dont bring an oversized burden for their relative merit. Just had to caveat that. If we take on Haiku, we should take on Haiku.

I was also thinking about implementing threading using native threads and ports, but that would require more changes, because pthread* stuff is not contained in a single file. Also messaging could be slower then, because it would need to pass through Haiku kernel then, and AFAIK it's all on pony side now.

I'm not sure where this would fall on the line I put above. I would encourage doing a rough and dirty spike for what it would look like to get early feedback before investing a ton of time. That would probably be the answer with everything "like this". "Can you show us what it might look like and we can tell you how likely we would be to accept?"

I really feel bad when people put a lot of work into something and then we reject it. I'd much rather they talk with us early on and rejections come with much less effort that won't get merged (I dont want to say the effort is wasted as people often learn a ton in the process).

Re the linking: you dont need to write linker. You "just" need to use the embedded LLD linker like the MacOS, Windows, and Linux versions are. Basically, instead of calling out to an external linker, you call into the "lld as a library" that exists in the pony compiler.

BTW i'm not sure yet about static libs. BeOS and Haiku are all for shared libraries, and i did not have a look yet to see how to make sure pony stuff links dynamically and then generate ponylang package with correct content (to add it to HaikuDepot).

And again, thank you for doing this. BeOS is still my favorite OS. I still have 2 early rev BeBoxes and I still check in on Haiku once a year to see if there is any way I could use it as a daily driver. It's a ritual I have been undertaking for at least 10 years and almost assuredly longer.

I would model after MacOS on this front as it "doesn't really do static libs".

[ahwayakchih]

I would encourage doing a rough and dirty spike for what it would look like to get early feedback before investing a ton of time. That would probably be the answer with everything "like this". "Can you show us what it might look like and we can tell you how likely we would be to accept?"

Hmm... then maybe i'll try it, after i'm sure everything else is 100% OK. I know that sending a message to native thread/port means it's copied, so i guess i could send pointer to pony message data instead of whole data... anyway, i suspect it will be slower than just passing pointer all on pony side. Still, it could be interesting to see how it would look like anyway.

I really feel bad when people put a lot of work into something and then we reject it. I'd much rather they talk with us early on and rejections come with much less effort that won't get merged (I dont want to say the effort is wasted as people often learn a ton in the process).

No worry :). It's on me - i got curious and carried away ;). And i did learn stuff and got reminded about a lot more (i'm not working with C/C++, and never really did professionally, so my knowledge is... spotty and/or rusty).

Basically, instead of calling out to an external linker, you call into the "lld as a library" that exists in the pony compiler.

OK, i'll try that :).

BeOS is still my favorite OS. I still have 2 early rev BeBoxes and I still check in on Haiku once a year to see if there is any way I could use it as a daily driver. It's a ritual I have been undertaking for at least 10 years and almost assuredly longer.

Nice! I'm checking Haiku from time to time too, but less regularly (usually when i'm between work/projects or on holidays) and in meantime i just try to be up to date on the stuff happening there.

I would model after MacOS on this front as it "doesn't really do static libs".

OK, thanks!

[SeanTAllen]

@ahwayakchih please proceed if you want. our caveat would be. we don't see ourselves supporting haiku going forward but we are happy to take into mainline with all the previous caveats if you are willing to commit to doing ongoing support and if breakages start piling up and you are no longer around, we would almost assuredly remove support.

Sound good?

[ahwayakchih]

@SeanTAllen that's great! and very fair, thanks!