How to avoid duplicate data when collecting from a folder multiple times

SYMPTOMS

There is no need to collect a document again when it is already collected unless somebody changed it.
There are two ways to determine if a file has changed:

1. When compliance mode is disabled, the File system Collector checks on every field that is specified in the ZyLAB.EDiscovery.Collectors.FileSystem.dll.xml. If one of the properties has changed, it should be recollected. (e.g. modified date)
2. When compliance mode is enabled (default), the File System Collector still checks the specified fields in the File System Collector config file but on top of that, it will calculate a hash value over the contents of the document. When this is changed, it will also recollect regardless of the fields in the config file. There are ways to change files without changing the modification date etc. When this is the case, the files are probably modified and the metadata is intentionally set back to the original value to cover up changes to the document.

CAUSE

Not applicable

RESOLUTION

Running in Compliance mode is the most secure option to collect changed/modified files.

APPLIES TO

3.0; 3.1; 3.2; 3.3; 3.4