Use Cases

Functional Connectivity and Computational Neuroscience

The advent of neuroimaging modalities, notably functional MRI, has marked the dawn of a new era in computational neuroscience allowing for non-invasive observation of brain activity in real-time, enabling us to understand connections between disparate brain regions as a function of activity.  My work has focused on representing brain activity as a real-time network of connected micro-regions, and tracking that activity in real-time using data analysis on fMRI signals. This was joint work with researchers from Imperial College and Kings College London. The real-time aspect of this work offers the unique advantage of enabling neurofeedback: the ability to train the brain by voluntary activity or via electrical stimulation in response to observed activity. This real-time control loop has never before been possible and holds promise for future medical interventions.

Statistical Cybersecurity

Cybersecurity may be simplistically defined as the defence of a set of information assets that lie within a perimeter from malicious agents outside it. A common example is the protection of enterprise data from exfiltration. Although dominated by rules-based systems, recently the cybersecurity community has recognised that ubiquitous digital data collection renders it a dreamland for statistical machine learning. Nevertheless, obtaining labelled datasets in cybersecurity is exceptionally difficult, as it requires manual labelling by security experts, whose time is scarce and invaluable. The problem is aggravated by a dramatic imbalance in the relative frequency of legitimate and illegitimate traffic: although estimates vary depending on the enterprise in question, attacks can be as rare as one a month, or less, pitted against terabytes of legitimate daily traffic. Consequently, techniques that rely on semi-supervised, weakly supervised or fully unsupervised (e.g., anomaly detection) technology are of great interest, and the focus of my work in this area.
  • Poorly Supervised Learning: how to engineer labels for Cybersecurity. Christoforos Anagnostopoulos, Statistical Cybersecurity Workshop, Imperial College London, September 2017, United Kingdom [slides]

Adaptive Scorecards in Retail Banking

Scorecards are in widespread use in the retail banking industry, primarily for determining the creditworthiness of individuals or companies, but also, in modified forms, for determining the likelihood that a certain transaction was part of a money laundering effort. They are similar to probabilistic classifiers, but are subject to additional constraints because of regulatory issues. As a result, building a scorecard (which is the term used for the process of feature extraction, model selection and parameter estimation) is a costly exercise that involves human supervision, and retail banks are reluctant to repeat that process often. This results in scorecards whose parameters can often become obsolete because of population drift, i.e., because of demographic or other changes in the population of interest. Especially in anti-money laundering, the arms race between regulators and launderers results in frequent behavioural changes of both parties, which would render any scorecard built on historical data possibly irrelevant to current practice. My group at Imperial College spent considerable energy evangelising the use of continual scorecard updating instead of periodic rebuilding, and the use of principled exponential forgetting techniques to gradually downweight the importance of historical data and enable the scorecard to remain up-to-date. 
  • Temporally-Adaptive Linear Classification for Handling Population Drift in Credit Scoring. Adams, N.M., Tasoulis, D.K., Anagnostopoulos, C., and Hand, D.J., COMPSTAT, 2010

Predictive Maintenance and Next-Generation Fault Prevention

A general trend in predictive analytics is to replace modelling assumptions, largely driven by mathematical descriptions of a given phenomenon, with data-driven techniques, driven by data crunching and trial and error. This same trend is about to revolutionise industrial manufacturing, as the introduction of numerous high-frequency sensors on industrial components enables the use of data-hungry machine learning algorithms, that can anticipate the behaviour of the IoT device with comparable accuracy to physical models of the device involving complex mathematics and possibly years of study.  By the same token, AI can detect when the behaviour of a component is outside its normal bounds, and raise an early warning for a potential catastrophic fault. The ease of use and general applicability of AI means that such capabilities will soon be rolled out across all IoT industrial devices -a massive market, with huge potential gains. For more, please read the blog post cited below which describes our work with Mentat in predicting faults of pneumatic valves. 
  • https://blog.ment.at/machine-learning-for-predictive-maintenance-from-physics-to-data-ae1a094b3669

THEORY AND METHODOLOGY

Measuring Classification Performance

Much of the recent wave of AI success stories is down to a specific type of tool: a supervised classifier, whereby an algorithm seeks to identify the relationship between a description of an object and the object's label, via a large number of labelled examples. A vast literature on this topic has been largely driven by performance comparisons of different classifiers on benchmark datasets. As a result, agreeing on sensible, coherent, and useful performance metrics is more critical than ever. As it turns out this is a highly non-trivial matter, and one that crucially controls the match between results in silico vs real life. For more information,  please visit www.hmeasure.net
  • When is the area under the receiver operating characteristic curve an appropriate measure of classifier performance? Hand, D.J., and Anagnostopoulos, C.  Pattern Recognition Letters, 34, pp 492--495, 2012
  • CRAN package averaging over 800 downloads per month

Streaming Correlation Analysis

Although most monitoring systems focus on changes in the average value of a sensor, it can be just as important to identify any changes in the dependency structure across different sensors. For example, we expect the gas pedal position to be highly correlated with the engine torque in a smart car use case, and if that correlation suddenly breaks down, this might indicate a fault in the gearbox. The tracking of dynamic covariance matrices is a particularly challenging statistical problem, especially in cases where the number of variables is large.
  • Streaming covariance selection with applications to adaptive querying in sensor networks, Anagnostopoulos, C., Adams, N.M.and Hand, D.J.,  The Computer Journal, doi: 10.1093/comjnl/bxp123, 2010

Anomaly and Change Detection

Anomaly detection is the original "needle in a haystack" problem: detect any abnormal patterns in a sea of data. Detecting anomalies is often reliant on a solid model of what "normal" behaviour looks like. Alternatively, one can instead define meaningful metrics of what it means for two objects to be "similar", and then rely on clever use of algorithms to detect objects that are "unlike" everything else.  

Streaming Classification

Classification technology typically expects the availability of a labelled dataset which is explored to tune the parameters of the model. Often, however, labelled examples continue to arrive over time, introducing a need to continually update classifiers without the computational burden of having to store or revisit the entire data history. Moreover, in real-world streaming data settings, it is often the case that the data generating process can change, either suddenly or abruptly, rendering models that were trained on old data obsolete. My work in this area in collaboration with both UK and US academics has produced a number of successful algorithms. 
  • Online Linear and Quadratic Discriminant Analysis with adaptive forgetting for streaming classification. Anagnostopoulos, C.,and Tasoulis, D.,and Adams, N.M.,and Hand, D.J., Statistical Analysis and Data Mining, 5(2), pp 139--166, 2012
  • Information-theoretic data discarding for dynamic trees on data streams. Anagnostopoulos, C., and Gramacy, R.B, Entropy, 15(12), 5510--5535, 2013
  • Temporally adaptive estimation of logistic classifiers on data streams, Adams, N.M., Anagnostopoulos, C., Hand, D.J., and Tasoulis, D.,  Journal of Advances in Data Analysis and Classification (ADAC), doi:10.1007/s11634-009-0051-x, 2009

Alternatives to Supervised Learning

Perhaps the foremost complaint of data scientists working in industry is the poor state of labelled datasets or altogether lack thereof. Indeed, in many settings such as cybersecurity and healthcare, labels are typically produced manually by human experts who are understandably reluctant to spend much of their time labelling data examples for use by machine learning algorithms. This situation calls for a better understanding of the relatively unexplored space between supervised and unsupervised learning, that represents more faithfully the label-generating process and its constraints. A recent framework of great interest is weak supervision, wherein labelling heuristics are employed in lieu of actual labels, to allow for programmatic, though possibly fairly noisy, labelling. Weak supervision was used by Stanford's Snorkel project for image classification and natural language processing, and I have been working towards its adoption in medical diagnosis and statistical cybersecurity, via a number of well-received seminars, a book chapter and a CRAN package underway.
  • In print: Weakly Supervised Learning. Christoforos Anagnostopoulos, Data Science for Cybersecurity, https://doi.org/10.1142/9781786345646_010, 2018