hadoop

Using Hadoop to Create SOLR Indexes

One of the most challenging projects I faced at work recently was to create a Apache SOLR index consisting of approx 15 million records. This index had been created once in the history of the company using a MySQL database and SOLR's Data Import Handler (DIH). It had not been attempted since then because the original indexing process was time consuming (12-14 hours), required human supervision, and on failure had to be restarted from the very beginning. (more…)

By dan, ago
maven

OutOfMemoryException in Maven

I came across a situation today where Maven threw an OutOfMemoryException. I didn't think the process would have taken that much memory but it clearly did. I was trying to deploy. I then tried it again but skipped the tests. No good. I found out that by setting the MAVEN_OPTS environment variable to something like -Xmx512m I was good to go.

By dan, ago