Fix rustdoc warnings and test cargo doc in CI

[arpad-m]

Problem

cargo +nightly doc is giving a lot of warnings: broken links, naked URLs, etc.

Summary of changes

• update the proc-macro2 dependency so that it can compile on latest Rust nightly, see Delete use of proc_macro::Span::before/after dtolnay/proc-macro2#391 and error[E0635]: unknown feature proc_macro_span_shrink dtolnay/proc-macro2#398
• allow the private_intra_doc_links lint, as linking to something that's private is always more useful than just mentioning it without a link: if the link breaks in the future, at least there is a warning due to that. Also, one might enable --document-private-items in the future and make these links work in general.
• fix all the remaining warnings given by cargo +nightly doc
• make it possible to run cargo doc on stable Rust by updating opentelemetry and associated crates to version 0.19, pulling in a fix that previously broke cargo doc on stable: fix cargo doc failing on stable Rust open-telemetry/opentelemetry-rust#904
• Add cargo doc to CI to ensure that it won't get broken in the future.

Fixes #2557

Future work

• Potentially, it might make sense, for development purposes, to publish the generated rustdocs somewhere, like for example how the rust compiler does it. I will file an issue for discussion.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

1032 tests run: 981 passed, 0 failed, 51 skipped (full report)

---

Flaky tests (1)

Postgres 14

• test_threshold_based_eviction: debug

[vadim2404]

Should we incorporate this check into CI?

[koivunej]

I think these are good changes.

[vadim2404]

related to #2557

[arpad-m]

Should we incorporate this check into CI?

I think so. Originally this PR didn't have such a CI check as nightly was required to run cargo doc and CI currently only has one stable Rust version. But then I changed the PR to update opentelemetry to 0.19 to fix the stable cargo doc build, so I've now added it to CI.

[arpad-m]

Okay, a trial edit to add a broken link to the code is showing a failure as expected, very nice. I've removed that edit from the CI now.

[koivunej]

Still looking good.

I don't think we have any tests for the otel stuff, the parts that we use. They don't publish a clear changelog so ... aah they have per crate changelog:

• http 0.7 -> 0.8, looks quite boring
• otlp 0.11 -> 0.12, similarly boring, deps (within and outside otel)
• reqwest-tracing bump, for otel
• otel semantic conventions, deps
• proc_macro2 was for latest nightlies I understood?

I think these are all looking very mundane version bumps, and thus we could merge this even today but perhaps it's best to still consider if we release and what we release. So let's hold merging until that has been clarified.