Skip to content

Allow OtherDoi, Event, and Activity shard configuration - #1605

Merged
jrhoads merged 2 commits into
masterfrom
cursor-index-shard-settings-2a61
Oct 5, 2026
Merged

jrhoads merged 2 commits into
masterfrom
cursor-index-shard-settings-2a61

Conversation

@jrhoads

@jrhoads jrhoads commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Purpose

Make OtherDoi, Event, and Activity OpenSearch shard counts configurable, using the same pattern as DataciteDoi.

Approach

Add an environment variable for each index, defaulting to 5, and apply that value as number_of_shards on the model settings used when the index or template is created. Shared Doi settings stay without a shard override, and each model keeps its own settings.

Key Modifications

  • Default NUMBER_OF_SHARDS_OTHER_DOI, NUMBER_OF_SHARDS_EVENT, and NUMBER_OF_SHARDS_ACTIVITY to "5" in config/application.rb, alongside NUMBER_OF_SHARDS_DATACITE_DOI.
  • Set OtherDoi index settings from the shared Doi index settings plus NUMBER_OF_SHARDS_OTHER_DOI.
  • Point the OtherDoi index template at OtherDoi.settings so the shard count is included when that template is applied.
  • Set Event and Activity index settings to number_of_shards from their own environment variables, leaving their mappings in place.
  • Add shard specs for OtherDoi, Event, and Activity, and update the DataciteDoi isolation spec so shared Doi settings are still checked for the absence of a shard count.

Important Technical Details

  • number_of_shards applies only when an index is created. Existing OtherDoi, Event, and Activity indices keep their current shard count until they are recreated (for example via upgrade_index or a new index and alias switch).
  • Each variable is set in config/application.rb with ||= "5" and converted with .to_i when the model settings are defined, matching NUMBER_OF_SHARDS_DATACITE_DOI.
  • Doi.settings still has no number_of_shards. DataciteDoi continues to use NUMBER_OF_SHARDS_DATACITE_DOI. OtherDoi copies the Doi index analysis settings and adds its own shard count, so the Datacite and Other DOI indices can be sized independently.
  • Event and Activity do not inherit Doi settings. Their templates use each model's own settings and mappings.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Reviewer, please remember our guidelines:

  • Be humble in the language and feedback you give, ask don't tell.
  • Consider using positive language as opposed to neutral when offering feedback. This is to avoid the negative bias that can occur with neutral language appearing negative.
  • Offer suggestions on how to improve code e.g. simplification or expanding clarity.
  • Ensure you give reasons for the changes you are proposing.
Open in Web Open in Cursor 

cursoragent and others added 2 commits October 5, 2026 08:34
Add NUMBER_OF_SHARDS_OTHER_DOI, NUMBER_OF_SHARDS_EVENT, and
NUMBER_OF_SHARDS_ACTIVITY, each defaulting to 5. Apply them on the
model settings used when those indices are created, and keep the shared
Doi settings free of a shard override.

Co-authored-by: Joseph Rhoads <jrhoads@users.noreply.github.com>
Mirror the DataciteDoi shard specs: default of 5, isolation from shared
Doi settings, and index template settings for each model.

Co-authored-by: Joseph Rhoads <jrhoads@users.noreply.github.com>
@jrhoads
jrhoads marked this pull request as ready for review October 5, 2026 08:46
@jrhoads
jrhoads requested review from a team and digitaldogsbody October 5, 2026 08:57
@jrhoads
jrhoads merged commit 0baaec8 into master Oct 5, 2026
19 checks passed
@jrhoads
jrhoads deleted the cursor-index-shard-settings-2a61 branch October 5, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants