Reveal Review Publication

Cycle: Current cycle number, or Pre-Training.

Progress: Number of positive samples identified for the current cycle so far.

Classifier Status: Current status of the classifier.

Classifier Detailed Status: Status of classifier with any available details.

Current Stability Status: Unknown, instable, or stable.

Referenced Models: If the storybook uses any external model, it will show up in the “Referenced Models” line.

Training Mode: Active Learning or Infinite Learning.

Model Type: Identifies where model is assigned.

Last Refreshed: The last time the classifier was run.

High (60% to 100%)

Medium (40% to 60%)

Low (0% to 40%)

Scored (Total) (0 to 100%)

Uncertain – No Model Features (-500): When a document does have some metadata or text features but doesn’t have those features in the current run of the COSMIC classification model; in other words, if the model doesn’t have the set of features that a document has, we cannot score the document against the model so it gets marked as Empty.

Unclassified (-100): These are the documents that get a probability score of 50 from the COSMIC classifier. This means they are classified as neither positive nor negative as per the current round of COSMIC classification.

No Score – Errored Documents (-200): An error has occurred during the 2nd pass of the processing for these documents.

No Score – Empty Documents (-300): When a document is found to have no text and metadata representation in the form of vectors; this almost never happens for emails as there is metadata in the form of the fields From, Sent, Subject, etc… But this can happen for an attachment that has a unique filename.

No Score – Missing Text Vectors (-400): this number reflects the number of documents for which our system cannot locate Text Vectors. This can occur if something abnormal happened during the processing or if any vector files have changed location at any point.

Run Full Process: Use this function to force the system to re-classify the whole document set at any time.

Run Warmup Process: Send a request to the COSMIC service to warmup the target storybook. Administrators should run this once at the beginning of COSMIC. This step will reduce the time required when you run an initial classification.

Reset COSMIC Group: Sends a request to COSMIC service to clean current COSMIC tags and scores assigned to the documents.

Update Metadata Vectors: Allows you to update metadata vectors for this storybook. This button gives the user the option to incorporate all new entity examples into the COSMIC metadata vector, even if they are not in a model that has been run.

Name: Labeled Name of COSMIC Tag set.

Training Mode: Active Learning or Infinite Learning. Infinite Learning reduces the need to review non-relevant documents. Active Learning aims at limiting the human coding of even relevant documents.

Checkout Size: Number of documents sent to reviewers at one time when entering the queue.

Retraining Interval: The amount of non-control set documents that must be coded for re-classification in the document universe.

Training Queue Size: The approximate number of documents to be added to the training queue after reclassification.

Minimum Positive Examples: The minimum number of documents tagged as Yes before a reviewer can start training COSMIC.

Control Set Percentage: The portion of random sample documents to be set aside per training queue for the purpose of statistical measurement. The Percentage can range from 0% to 100% of the training queue.

Is Inclusive Only: Tells the classifier to classify and select for training All documents or Inclusive (Inclusive emails, attachments, loose eFiles) documents only.

Autotune Enabled: Allows the system to automatically adjust weights based on current results to achieve best results (by default: On).

Autotune Cutoff Threshold: Maximum number of documents that will be used for autotuning (by default: 1000).

Stability Threshold: Minimum number of document tags added or deleted that will trigger stability to recalculate.

Auto Submit Status:

Training Queue Tag Name: Name of the training queue to be used for COSMIC.

Positive Tag Name: This label is for the Responsive coding button.

Negative Tag Name: This is label is for the not-responsive coding button.

Skip Tag Name: This label is for the skip button to allow reviewers to temporarily “pass” on a document.

The Classifier Configuration Weights table displays the weights given to each feature (beyond the word content as such) of each document segment. The features include enriched natural language processing (NLP) and metadata often implying dynamics outside the literal content of the documents themselves – such as emotionally charged communications, indications of pressure applied or endured, or the over-use of capitalization or personal pronouns. Also, the feature list includes both standard and custom entities.

The weights range from 0 to 1. By default, each feature is given a 0.1 weight. If a feature is given a higher weight, it will have more impact on the classifier. The Autotune feature may assign a different weight to the feature if it is enabled, or a user can manually adjust the weights.

Yes: system agrees with the tag assigned.

No: system does not agree with the tag assigned.

Added: tagged Yes.

Removed: tagged No.

Removed (Auto added to Control set): when a document was first tagged as a seed document and later selected as a Control Set document, the system automatically removes the document from training.

Included (Exclude tag removed): User tagged a previously Excluded document as Included.

Excluded (Exclude tag added): User tagged a document as Excluded.

Yes: tagged from within any COSMIC queue.

No: not tagged from within a COSMIC queue.

Not in COSMIC Queue: when tags are assigned from outside a COSMIC queue.

[COSMIC GROUP]: primary COSMIC Group when tagging occurred.