Abstract

In this work we implement Bayesian surprise as a method to sift through sequences of discrete patterns and identify any unusual or interesting patterns that deviate from known sequences. Surprise is a biological trait inherent in humans and animals and is essential for many creative acts and efforts of discovery. Numerous technical domains are comprised of discrete elements in sequences such as e-commerce transactions, genome data searching, online financial transactions of many types, criminal cyber-attacks and life-course data from sociology. In addition to the complexity and computational burden of this type of problem is the issue of their rarity. Many anomalies are infrequent and may defy categorization therefore not suited to classification solutions. We test our methods on four discrete datasets (Hospital Sepsis patients, Chess Moves, the Wisconsin Card Sorting Task and BioFamilies) consisting of discrete sequences. Probabilistic Suffix Trees are trained on this data which maintain each discrete symbols location and position in a given sequence. The trained models are exposed to ``new'' data where any deviations from learned patterns either in location on the sequence or frequency of occurrence will denote patterns that are unusual compared with the original training data. To assist in the identification of new patterns and to avoid confusing old patterns as new or novel we use Bayesian surprise to detect the discrepancies between what we are expecting and actual results. We can assign the degree of surprise or unexpectedness to any new pattern and provides an indication of why certain patterns are deemed novel or surprising and why others are not.