Optimizing a training set

When setting up an analytics index, the option to optimize the training set will perform the following operations:

1. remove conceptually irrelevant documents.

2. remove documents which are too long or too short to serve as good examples.

3. remove spreadsheets or documents which consist of predominately numeric data.

4. remove log files.

5. remove documents with text resulting from processing errors.

Word count, word uniqueness, punctuation, and words with a high character count are evaluated to determine what must be removed.

Other Languages in Relativity

After running the structured analytics set for language identification in Relativity, you may see many documents for which the Docs_Languages::Language field is set to 'Other'.

There are several reasons why this can happen. The analyzed document may consist entirely of numeric data:

. . . documents may be in a language other than the three most common languages Relativity analytics identified, or documents may be in a language which is not one of the 173 languages that Relativity can identify.

My own testing has shown that Relativity will list the language as 'Other' for documents which consist of long lists of names:

. . . or entirely of graphics:

Rootless threads

Note the line in a Relativity email thread summary which lists email threads for which no root was found.

This indicates that there are differences in the extracted text for some email threads. Don’t overlook these when focusing on inclusive emails. They may be an indication that an email recipient altered someone’s message - whether to answer a serious of questions with an inline response, or for some nefarious purpose.