Fisher Yates Shuffle

When generating random samples, Relativity uses an algorithm called the Fisher-Yates shuffle. See the documentation of the shuffling process on the kCura website here. The Fisher-Yates shuffle creates a random sequence of items in a closed set by picking a random number, n, and then removing the nth occurrence of that number (counting from the end) in the original set and moving it to a new list. This process is repeated until all of the numbers in the set have been removed from the original set. See this example from the Wikipedia page for the Fisher-Yates algorithm:

So after the first selection, we pick a random number N between 1 and 7 (we 'roll' the dice so to speak), 1 to 7 being the count of the remaining number of items in the set, and then take out the nth occurrence of that number counting from the end of the list.

Random sampling can be performed in Excel using the method described in this blog post. In this example we put the numbers in our set from 1 to 8 in column A. Then we use this formula:

=RANDBETWEEN($A$2,$A$9)

to generate random selections from the set in column B. But the trouble is that the numbers repeat. Proceed by entering the numbers from the set in first row beginning in column C. Then enter this formula in C2:

=IF(C$1=$A2,INDEX(IF($A2=1,$C$1:$J$1,$C1:$J1),$B2),IF(C$1=$B2,INDEX(IF($A2=1,$C$1:$J1,$C1:$J1),$A2),IF($A2=1,C$1,C1)))

. . . then drag it to the right and down. The numbers in red at C3:J9 are our random set of numbers.

Getting Rid of NBSP Codes

In Relativity under the Imaging tab and the Imaging Profiles subtab if you select the 'Native' radial button in the Imaging Method section, you'll see the options for HTML files appear. Here you have the choice to 'Remove non-breaking space (NBSP) codes.

What are such codes? In HTML a blank space will not be interpreted as actual space. You need to enter a special code, &nbsp , in order to signify that an empty space should appear.

See this example where the html text just has multiple white spaces between the words, "Litigation Support Tip of the Night"

. . . the html code is displayed in IE without the multiple spaces:

However when html code is entered this way:

. . . IE will show the additional spaces.

Relativity removes &nbsp codes so that they don't prevent the text from wrapping correctly. In a text editor these codes will show up as a hallow circle - something you have have come across before . . .

Image Dithering

Image dithering involves altering the appearance of black and white images by changing the arrangement of pixels to make it seem as though there are more variations in the shades of colors.

In Relativity if you go to Imaging . . . Imaging Profiles . . . in the Native Imaging Engine Options sections, you'll see that you can select between 8 different dithering algorithms. These are employed when rendering documents as the black and white tiff images that the database works with.

The Purdue College of Engineering has a guide on its site which shows the basic differences between these dithering options. A threshold image is the simplest kind, and is one in which a pixel is compared against a fixed threshold. As you can see it's largely a black and white image, and there isn't a lot of detail.

Clustered dithering produces an image by varying the size of dots - light areas are filled by small dots, and in dark areas large dots are close to overlapping with adjacent dots.

Dispersed dithering, of which a Bayer matrix is a particular sort, has a distinctive cross hatched pattern and is based on the precise placement of pixels.

Floyd-Steinberg dithering uses a technique called error diffusion, which has a tendency to produce images with sharper edges. For this reason it may be the best dithering technique for producing readable text.

The above images are used to make very clear the distinctions between the various techniques. kCura shows samples of images of text and a diagram produced with various dithering algorithms on this its site where the differences are much subtler.