Recover S3 stream after skip failure#29419
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the S3 filesystem input stream behavior to be more resilient to transient S3 failures when performing forward seeks, which benefits lakehouse connectors that rely on forward seeking while reading object data.
Changes:
- Retry forward-seek operations by reopening the S3 stream when
skip()fails with anIOException. - Remove the previous zero-length response workaround tied to an AWS SDK issue.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
92cb049 to
396dc34
Compare
|
I addressed the Copilot comment to handle |
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
|
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR refactors Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
S3 cloud tests saw connection resets while skipping within an open response stream. Treat that path like read failures so the next read can reopen at the requested position.
Older AWS SDK versions returned a checksum byte instead of EOF for empty S3 objects. The current SDK returns EOF directly, so the local replacement stream is no longer needed.
396dc34 to
b9e1530
Compare
Release notes
(x) Release notes are required, with the following suggested text: