solr

Indexing SOLR Using Data from Google’s BigQuery

For this entry I assume you already know how to configure SOLR’s Data Import Handler as that is how we’ll configure SOLR to use BigQuery: https://wiki.apache.org/solr/DataImportHandler Steps Google’s Service Account File Download the service account file as described here: https://cloud.google.com/docs/authentication/getting-started  I used the JSON version of the file. For the Read more…

hadoop

Using Hadoop to Create SOLR Indexes

One of the most challenging projects I faced at work recently was to create a Apache SOLR index consisting of approx 15 million records. This index had been created once in the history of the company using a MySQL database and SOLR’s Data Import Handler (DIH). It had not been attempted since then because the original indexing process was time consuming (12-14 hours), required human supervision, and on failure had to be restarted from the very beginning.
(more…)

%d bloggers like this: