Skip to main content

Reveal Review Publication

Reveal AI Processing Stages

The steps involved in the Reveal AI ingestion process are set out in the table below to offer a better understanding of the activity taking place and troubleshooting any potential issues with Support.

ID

Display

Explanation

1

Preparing processing jobs to start

Create jobs in the SQL ProcessingQueue table

2

Pre-process

Java processing initialization

3

Pre-process - checking source dataset

Status pass

Compare documents that are sent to processing vs. existing documents

4

Computer-generated content detection

Zero pass

Detect computer generated content

5

Ingestion

1st pass

Email segmentation

Metadata parsing

6

Name resolution and threading - entity update

Entity update based on 1st pass processing

7

Name resolution and threading - thread creation

Threading creation

8

Natural language pipeline

2nd pass

Entity extraction

Sentiment detection

Text vector generation

9

Data finalization - entity update

Entity update based on 2nd pass processing

10

Data finalization - network-based merge

Network-based entity merging

11

Data finalization - segment count update

Combine Id

Segment Counts

Thread Properties

Relationship Counts

Random Id

Language

12

Data finalization - populating denormal tables

SQL Sync (Populating denormal tables)

(2.50) Submit clustering job, create numeric vector and other vectors

13

Pattern Detection

Pattern Detection

14

Near Duplicate Document Detection

Near Duplicate Document Detection

15

Post-process

Java processing steps to stop things such as the JVM monitor

(2.80) Submit clustering job, create numeric vector and other vectors

16

Warmup

Run Queries to cache server

17

Populate copy thread extended

Create thread extended table for exporting purposes later