Skip to content

Asynchronous Inbox + asynchronous house saving - #4561

Open
gunzino wants to merge 29 commits into
otland:masterfrom
gunzino:async-inbox
Open

Asynchronous Inbox + asynchronous house saving#4561
gunzino wants to merge 29 commits into
otland:masterfrom
gunzino:async-inbox

Conversation

@gunzino

@gunzino gunzino commented Nov 8, 2023

Copy link
Copy Markdown
Contributor
  • I have followed [proper The Forgotten Server code styling][code].
  • I have read and understood the [contribution guidelines][cont] before making this PR.
  • I am aware that this PR may be closed if the above-mentioned criteria are not fulfilled.

Changes Proposed

I'm happy to share a contribution to the TFS which consists of async, zero-blocking Inbox implementation to prevent abusing the current sync system with large Inboxes and causing lags due to database IO like mentioned here #1431. The similar implementation is rotating between server owners but does not include complete zero-blocking IO for both saving and loading and parcel/item delivery to Inbox when player is offline. The implementation can be further used for more components like depots, store inbox etc. but those are currently either limited or controlled by server. The basic concept was created by kondra (otclient@otclient.ovh) few years ago but the code is already widespread across server owners which some of them resell for real money.

The implementation highlights:

  • The database IO and some of the serialization is done in separate thread (secondary Dispatcher g_dispatcherInbox) with dedicated database connection
  • Data structures protected by mutex
  • Async zero-blocking delivery of parcels and items from market/houses

Items delivery to inbox currently done for:

  • Houses
  • Parcels
  • Market

EDIT (21th November): Added asynchronous houses saving

I would be really happy for your hints/recommendations regarding so this contribution can be pushed to the master branch.

Comment thread src/ioinbox.cpp Fixed
@ghost
ghost requested review from DSpeichert, MillhioreBT, nekiro and ranisalt November 8, 2023 17:54
Comment thread src/container.cpp Outdated

@ramon-bernardo ramon-bernardo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I liked the PR, it looks promising.

I made some improvements, removed duplicate code, see
https://github.com/ramon-bernardo/forgottenserver/pull/2

I'm still reviewing IOInbox!

Comment thread src/player.cpp Outdated
{
inbox = _inbox;
if (depotLocker) {
depotLocker->internalAddThing(2, inbox);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the index here could be a const?

Comment thread src/player.cpp Outdated
Comment thread src/game.cpp Outdated
Comment thread src/mailbox.cpp Outdated
@ghost ghost added the enhancement Increase or improvement in quality, value, or extent label Nov 9, 2023
…ng inbox, the delivery should now be guaranteed and IOInbox::flush is now only necessary when g_dispatcherInbox does not accept new tasks, needs further research if IOInbox::flush() can be removed
@gunzino gunzino changed the title [WIP] Asynchronous Inbox implementation Asynchronous Inbox implementation Nov 10, 2023
@gunzino
gunzino marked this pull request as ready for review November 11, 2023 15:50
@gunzino gunzino changed the title Asynchronous Inbox implementation Asynchronous Inbox + Asynchronous house saving Nov 21, 2023
@gunzino

gunzino commented Nov 21, 2023

Copy link
Copy Markdown
Contributor Author

@ramon-bernardo
I've added async house saving today as well which uses similar approach. Can you review?

@gunzino gunzino changed the title Asynchronous Inbox + Asynchronous house saving Asynchronous Inbox + asynchronous house saving Nov 21, 2023
Comment thread src/database.h
{
static Database asyncInstance;
return asyncInstance;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC getInstance already returns an "async" instance, used for other purposes but should be suitable. Why having another instance?

On that note, singleton sucks, but a "doubleton" is even worse 😆 let's just make it not a global and pass it around

@gunzino gunzino Dec 2, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean byt getInstance returns "async" instance? Is Database class thread-safe? Can database class execute multiple parallel queries from several threads? The asyncInstance is there only to be used by g_asyncTasks thread for async database IO- in this PR it is now only House and Inbox.

Comment thread src/ioinbox.h
std::map<uint32_t, PlayerDBEntryPtr> inboxCache;
std::map<uint32_t, std::list<ItemBlockList>> pendingItemsToSave;
std::set<uint32_t> pendingPlayerSet;
};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it a namespace and move everything private to the .cpp file only as this is not used anywhere else

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there similar approach somewhere else in the code? What about recursive mutex?

@gunzino
gunzino requested a review from ranisalt December 6, 2023 07:51
@Codinablack

Copy link
Copy Markdown
Contributor

This looks like a very promising PR, what is the status of this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Increase or improvement in quality, value, or extent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants