-
Notifications
You must be signed in to change notification settings - Fork 393
VUFIND-1210: Use Solr JSON APIs #4991
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Changes from 6 commits
48a8396
caae5eb
8e80568
e82a8e9
27c2e6d
2a56ef9
f6d550a
e35bfe1
6206ef4
5db7650
09be7e1
2f2d089
d943442
9451ac2
3a51fa7
af4ee52
5cc2f9f
990c9ed
1faec2b
cf8e76d
9d45b06
d3f0dde
7d2d26b
6e84cf5
4ff1d6e
e285203
9828993
db3abd5
31e30c8
c08ae52
c16edfc
0e44e67
b34c2ee
5cea405
7398011
e2cb768
e73f2ef
67ba9b5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -32,6 +32,7 @@ | |
| use Laminas\EventManager\EventInterface; | ||
| use Laminas\EventManager\SharedEventManagerInterface; | ||
| use VuFindSearch\Backend\BackendInterface; | ||
| use VuFindSearch\ParamBagBag; | ||
| use VuFindSearch\Service; | ||
|
|
||
| use function in_array; | ||
|
|
@@ -126,26 +127,28 @@ public function onSearchPre(EventInterface $event) | |
| $command = $event->getParam('command'); | ||
| if ($command->getTargetIdentifier() === $this->backend->getIdentifier()) { | ||
| $params = $command->getSearchParameters(); | ||
| $params = ParamBagBag::from($params); | ||
| $allShardsContexts = ['retrieve', 'retrieveBatch']; | ||
| if (in_array($command->getContext(), $allShardsContexts)) { | ||
| // If we're retrieving by id(s), we should pull all shards to be | ||
| // sure we find the right record(s). | ||
| $params->set('shards', implode(',', $this->shards)); | ||
| $params->setNested('params', 'shards', implode(',', $this->shards)); | ||
| } else { | ||
| // In any other context, we should make sure our field values are | ||
| // all legal. | ||
|
|
||
| // Normalize array of strings containing comma-separated values to | ||
| // simple array of values; check if $params->get('shards') returns | ||
| // an array to prevent invalid argument warnings. | ||
| $shards = $params->get('shards'); | ||
| $shards = $params->getNested('params', 'shards'); | ||
| $shards = explode( | ||
| ',', | ||
| implode(',', (is_array($shards) ? $shards : [])) | ||
| ); | ||
| $fields = $this->getFields($shards); | ||
| $specs = $this->getSearchSpecs($fields); | ||
| $this->backend->getQueryBuilder()->setSpecs($specs); | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The logic below is just a first draft, I don't like how it breaks the ParamBag abstraction completely in order to use array_filter. Either have to build some sort of filtering into ParamBag itself or just create a new ParamBag on the fly instead. Also, I will need assistance actually testing this with shards.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You should be able to test it with any two instances of Solr running the same schema. Let me know if you need more specific help on setting up a test environment; I haven't done it in a few years, but I don't recall it being too difficult.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok thanks...will try pointing it at our test and prod environments...
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That should work as long as they contain different record IDs (otherwise it will be hard to tell if the sharding is working).
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have (mostly) overlapping record IDs, so the sharding doesn't really work. But the faceting logic that I'm implementing does seem to -- I'm able to remove a facet group with [StripFields]. I suppose I can create a few fresh schemas and do partial indexes into each, but if someone already has sharding set up that will be a plus.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One simple approach could be to spin up a standard test instance using
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (If you can't easily do this, I certainly can -- just have to find time, since I remain buried under a large review backlog right now).
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry I'm still not having any luck with testing the shards config...if/when you have time, I'd appreciate it. |
||
| // TODO This will need an update for the JSON Facet API | ||
| $facets = $params->get('facet.field') ?: []; | ||
| $params->set('facet.field', array_diff($facets, $fields)); | ||
| } | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.