Skip to content

redb support#4

Draft
cryptoquick wants to merge 1 commit intoStorm-WG:masterfrom
cryptoquick:redb
Draft

redb support#4
cryptoquick wants to merge 1 commit intoStorm-WG:masterfrom
cryptoquick:redb

Conversation

@cryptoquick
Copy link
Copy Markdown

@casey gave me the advice that redb performs better than sled. In the hopes that it's more stable and user-friendly than sled, I investigated adding redb support to stored, and things initially went pretty well for a bit, but then I ran into a few lifetimes errors I couldn't think of a good way to resolve.

@dr-orlovsky, would you be interested in something like this? If so, I'm not sure if there's a good way to resolve the lifetimes issues, but maybe @casey might have some ideas. If we can get redb support, maybe it'll resolve the issue I reported in #3, by, uh, supporting something other than sled.

@casey
Copy link
Copy Markdown

casey commented Sep 14, 2022

Can you post some of the errors? Definitely note that redb is not yet stable. @cberner (the author) is working on stabilizing the file format, but you should expect at least a few breaking changes until then.

@cryptoquick
Copy link
Copy Markdown
Author

@casey There were so many errors that I just got overwhelmed. They'd also make more sense if you checked out the repo, since they'd be in context. I did reference some of your code in ord as an example of how to use redb, that was helpful. Although the implementation and use-cases are similar, there are definitely some major differences between the sled and redb interfaces that make switching between one and the other more difficult. Being explicit about when writes are to occur, the different datatypes supported, and the <k', v'> database lifetimes make the transition more difficult.

Also, I think stored is best relied on as a cache and not a more permanent contract storage solution, so the file format isn't a big deal-breaker for us, but it certainly would be for those already using stored. I don't think it should default to redb, I only did that for convenience in development. It would be very helpful for my use-case, however, since sled just won't quit.

@casey
Copy link
Copy Markdown

casey commented Sep 14, 2022

I'm a little backed up, so I probably won't have time to check out the repo to try out for a bit. Maybe start with code that produces a single error, post that, and then work from there?

@cryptoquick
Copy link
Copy Markdown
Author

Totally fine, I'm also investigating alternative approaches

@dr-orlovsky
Copy link
Copy Markdown
Member

I think these experiments are cool and should be continued. Unfortunately due to a big time problems on my side I can't really participate them - but would be happy to merge once a better solution than sled will be found and working.

@cryptoquick
Copy link
Copy Markdown
Author

@dr-orlovsky Awesome! I'm also working on a new, similar project, if you find it valuable. It came to mind because changing database formats could be a problem; it might be nice to have a flat file backup format for replication and checkpointing. I'm still working out a few details, but overall it mostly seems to work. Check it out:
https://github.com/diba-io/carbonado

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants