Skip to main content

Reveal Review Publication

Reviewing & Tagging Documents in COSMIC

You can use the COSMIC panel to review and tag documents. Numbered sections identified below.

603e7410675d0.png

Notice for documents not qualified for COSMIC, a warning message will appear on the top of the document:

603e741281529.png
1. Primary COSMIC Group Selector

Click the dropdown to select the primary COSMIC group when you have more than one COSMIC group in the current storybook.

2. Primary COSMIC Group Tags

Sets of tags used for the primary COSMIC Group, based on the labels set for the COSMIC group. Note these tags are at a document level.

Review the documents and tag each document as Yes, No or Skip (based on labels set for the COSMIC group). Notice that when a reviewer tags a segment as Yes at a segment level, the system automatically assigns Yes at a document level. It is recommended that reviewers use a segment tag to identify Yes documents, if possible.

603e74141c884.png

If the document is selected as a control set (see Control Set Percentage setting for more details), the Primary COSMIC Group Selector button will be labelled as COSMIC Group (Queue) CS:

603e7415ab263.png

User can also turn COSMIC models on or off, or add new COSMIC models on-the-fly, by clicking the Models drop down below:

603e7417527cd.png

Note

By default, AutoSave feature is on. Click ”“ to turn on or off the AutoSave feature. When AutoSave is on, clicking Yes, No or Skip automatically saves the choice and moves to next document in the queue:

603e7418e69ce.png

If AutoSave is off, when a reviewer selects one of the choices for the primary COSMIC Group, the reviewer will need to click Save or Save + next to save the choice.

603e741acc0af.png
3. Secondary COSMIC Groups

In the next row(s) appear tagging buttons for the other available COSMIC groups. These allow you to set tags to the current document without switching the focus of the primary COSMIC group.

Click any tags in the secondary tagging panel to tag for a secondary COSMIC model. The color of the tag automatically shows Green (Yes) when clicked once, Red (No) when clicked twice, or White (not tagged) when the tag button is left alone or clicked three times.

For example, the picture below shows a document being coded for the Control Set Queue for the Financial Strategy COSMIC Group. At the top level this document has been tagged Yes for for this group. At the secondary level below, HPC Overview has been clicked once to set the green Yes for this COSMIC model, and HPC_Overview2 has been clicked twice to set the red No for this COSMIC model to further define the training offered by this document:

603e741c6a58f.png
4. Include/ Exclude Selector

By default, all documents are included in a user’s model. However, if the user believes a document or a set of documents are irrelevant in building a model, they may choose to exclude documents from the model.

603e741e0c72d.png
603e741fa2698.png
5. Segment Level COSMIC Tags

Reviewers can also assign a tag at the segment level for the primary COSMIC Group. The segment level COSMIC tag is located on the top right corner of each segment. Notice the system only allows a Yes tag. When a reviewer tags a segment as Yes, the whole document will also be automatically tagged as Yes.

The Reset button is used to clear segment level COSMIC tagging.

603e74211dbe6.png

The right bottom of the segment shows if the same segment has been tagged in another document. For example, the segment below has been tagged as Yes in enron000003 and No in enron0000011.

603e7422aaea8.png
6. High Precision Classification

Models using High Precision Classification (HPC) train the model by a user highlighting relevant text within the document to focus on the relevant text. Models created in COSMIC Mission Control as High Precision or Autodetect to be used in this way. Users can select the classification view here, and then highlight and select relevant text as a Yes or No for the model.

Select_HPC_3_00.png

See High Precision Classification for further details.

7. COSMIC Cycle Information

By clicking the COSMIC SETTINGS tab you can review the current status of the COSMIC projects.

603e7424651c8.png
  • Cycle:

    • Pre-Training: no cycle has started.

    • Cycle #: The current cycle number.

    • Post-Training: This appears when the COSMIC group is in Disabled or Standby status.

  • Progress: The progress section shows number of documents tagged in current cycle/total required documents for current cycle.

  • Classifier Status:

    • Idle: The classifier is waiting for documents.

    • Submitted: Tagged documents have been submitted to the classifier.

    • Thinking: The classifier is classifying documents.

    • Disabled: Occurs when COSMIC group is set to Disabled.

    • Standby: Occurs when COSMIC group is set to Standby.

    • Error-Service Stopped: Service is stopped.

    • Error: Service is Errored.

    • Not Enough Data Provided: the COSMIC group runs without enough positive and negative samples. This only happens when user kicks off Run Full Process without enough samples.

8. COSMIC Notifications

When a reviewer finishes the last document in the current queue, the system will display a notification based on the readiness of the model:

  1. The model has completed its initialization phase and COSMIC AI Queue is ready:

    603e74260f3b3.png
  2. The model is still being trained, and has not reached stability:

    603e74274f3f2.png
  3. Training has completed and stability has been reached:

    603e7428d3f87.png