Running faster dtSearches

dtSearch works in two stages. The first step, looking up a string or word, is very fast. The second step controls the enumeration of the search hits. dtSearch performance can be optimized by either restricting the total number of hits, or limiting the amount of time allowed for a search. The time of the second stage increases in proportion to the number of hits. A search with 10,000 hits will take ten times as long as a search using dtSearch that gets 1,000 hits.

A wildcard search in dtSearch will create a separate word match for each request. So, a search for ACME* (in a case where this is used a Bates number prefix) will take much longer than a search for just ACME -- assuming there is a space between the prefix and the production number.

The xfilter option can be used in a dtSearch as a condition, that will not generate hits, and hence allow dtSearch to run faster. So, for example a search for:

(dog AND cat) AND xfilter(word "date::20171201~~20171231")

. . . will only find documents with the terms 'dog' and 'cat' that are dated between 12/1/2017 and 12/31/2017 in the 'date' field.

The xfilter option is not available in Relativity, but if you're running a complex dtSearch in a workspace and you want to have it run faster, you may want to consider having the admin change the dtSearchWorkerProgressThreshold instance setting. This controls how many documents will be sent to the dtSearch engine before it sends back information on its progress to the database. Changing the instance setting CreateZSRTablesForSTRs to false will remove these tables and allow dtSearch index to be built faster. ZSR tables are created for search term reports.

The instance setting dtSearchStreamThresholdInBytes controls the maximum file size of documents allowed to be processed directly into dtSearch. Documents of greater file size must be converted to bytes before they are fed to dtSearch. Increasing the value of this setting can allow dtSearch to run faster.