You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current importer doesn't seem to support the bulk DataImportHandler method to add data to a Solr index in bulk. Pushing data record by record is a slow, error-prone process since it seems to re-trigger the indexing process each time a new record is pushed and committed to the index. The DataImportHandler method circumvents this.
We've implemented this method of indexing in the Datahub::Factory application (which is heavily based on the Catmandu architecture)
Would it be viable to reuse this code in this module as a separate importer?
The local location of the JSON file which contains data to be uploaded.
The URL defined by the DataImportHandler in the Solr configuration.
Implementation looks like this:
my filename = "/tm/bulk.json"
my $requestHandler = "http://localhost:8983/solr/blacklight-core/update/json"
my $indexer = Datahub::Factory->indexer('Solr')->new(
'file_name' = $filename,
'request_handler' => $requestHandler
);
$indexer->import();
$indexer->commit();
Both methods will return the response of the handler API as a perl hash.
Both methods throw a Catmandu::HTTP:Error at the moment if something goes wrong.
The text was updated successfully, but these errors were encountered:
The current importer doesn't seem to support the bulk DataImportHandler method to add data to a Solr index in bulk. Pushing data record by record is a slow, error-prone process since it seems to re-trigger the indexing process each time a new record is pushed and committed to the index. The DataImportHandler method circumvents this.
We've implemented this method of indexing in the Datahub::Factory application (which is heavily based on the Catmandu architecture)
Would it be viable to reuse this code in this module as a separate importer?
See: https://github.com/thedatahub/Datahub-Factory/blob/master/lib/Datahub/Factory/Indexer/Solr.pm
The above module expects two inputs:
Implementation looks like this:
Both methods will return the response of the handler API as a perl hash.
Both methods throw a Catmandu::HTTP:Error at the moment if something goes wrong.
The text was updated successfully, but these errors were encountered: