Apache Lucene Basics

Apache Lucene
Presenter – Anirudh Sharma

What we’ll cover?
 What is Lucene?
 Basic Concept
 Indexing
 Documents
 Searching
 Code Examples

What is Lucene?
 Full-text search library in Java.
 Adds content to a full-text index.
 Allows you to perform queries on this index and then results.
 Content can be from various sources, like an SQL/NoSQL database, a
filesystem, or even from websites.

Basic Concept
Raw
Content
Acquire
content
Build
document
Analyze
document
Index
document
Index
Users
Search UI
Build
query
Render
results
Run query

Core Inndexing Classes
 IndexWriter
 Central component that allows you to create a new index, open an existing one,
and add, remove, or update documents in an index
 Built on an IndexWriterConfig and a Directory
 Directory
 Abstract class that represents the location of an index
 Analyzer
 Extracts tokens from a text stream

Creating an IndexWriter
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.store.Directory;
...
private IndexWriter writer;
public Indexer(String dir) throws IOException {
Directory indexDir = FSDirectory.open(new File(dir));
Analyzer analyzer = new StandardAnalyzer();
IndexWriterConfig cfg = new IndexWriterConfig(analyzer);
cfg.setOpenMode(OpenMode.CREATE);
writer = new IndexWriter(indexDir, cfg)
}

Core indexing classes (contd.)
 Document
 A Document is the unit of search and index
 Represents a collection of named Fields. Text in these Fields are indexed.
 Field
 A Document consists of one or more Fields.
 A Field is simply a name-value pair. For example, a Field commonly found in
applications is title. In the case of a title Field, the field name is title and the
value is the title of that content item.

Creating a Document
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
...
protected Document getDocument(Integer id, String firstName, String lastName,
String website) throws Exception {
Document document = new Document();
document.add(new StringField("id", id.toString(),Field.Store.YES));
document.add(new TextField("firstName", firstName, Field.Store.YES));
document.add(new TextField("lastName", lastName, Field.Store.YES));
document.add(new TextField("website", website, Field.Store.YES));
return document;
}

Core Searching Classes
 IndexSearcher
 Central class that exposes several search methods on an index
 Accessed via an IndexReader
 Query
 Abstract query class. Concrete subclasses represent specific types of queries,
e.g., matching terms in fields, boolean queries, phrase queries etc.
 QueryParser
 Parses a textual representation of a query into a Query instance

IndexSearcher
IndexSearcher
IndexReader
Directory
Query TopDocs

Creating an IndexSearcher
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.store.Directory;
import org.apache.lucene.index.IndexReader;
...
public static void search(String indexDir,String q)throws IOException{
private static IndexSearcher createSearcher() {
Directory directory = FSDirectory.open(Paths.get(indexDir));
IndexReader reader = DirectoryReader.open(directory);
IndexSearcher searcher = new IndexSearcher(reader);
return searcher;
}

Query and Query Parser
import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.search.Query;
private static TopDocs search(String keyword, IndexSearcher searcher)
throws Exception {
QueryParser qp = new QueryParser(“keyword", new
StandardAnalyzer());
Query keywordQuery = qp.parse(firstName);
TopDocs hits = searcher.search(keywordQuery, 10);
return hits;
}

Core searching classes (contd.)
 TopDocs
 Contains references to the top documents returned by a search
 ScoreDoc
 Represents a single search result

TopDocs and ScoreDoc
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TopDocs;
…..
TopDocs foundDocs = searchById(1, searcher);
System.out.println("Total results: " + foundDocs1.totalHits);
for(ScoreDoc doc : foundDocs.scoreDocs) {
Document d = searcher.doc(doc.doc);
System.out.println(String.format(d.get("firstName")));
}

Resources
 Complete code used in this session - https://github.com/ani03sha/lucene-
starter
 Further reading - https://lucene.apache.org/core/quickstart.html

Apache Lucene Basics

More Related Content

What's hot (6)

Similar to Apache Lucene Basics (20)

Recently uploaded (20)

Apache Lucene Basics