Computational Neuroscience:Disenchanted curves

Bochum-based research duo uncovers weaknesses in the interpretation of ROC curves – Questioning the established measurement method due to the high complexity of processes in recognition memory

How do I know this face? A question that surely everyone has asked themselves at some point in time. Neuroscientists in memory research describe the processes that help in the search of the answer as familiarity or recollection. The hypothesis is that both mechanisms differ significantly in their neuronal processes. A research duo from Bochum has now taken a closer look at the common method of measuring these processes – the so-called Receiver Operating Characteristic Curves or ROC curves. They found that the assessment of memory processes on the basis of ROC curves should be reappraised, since the curves cannot draw a line between the assumed memory processes. The duo published the results of their work in the renowned Journal of Cognitive Neuroscience at the beginning of May.

Established tool evaluating brain injury

“ROC curves are currently considered the standard scientific method to represent neural processes of familiarity and recollection,” said Olya Hakobyan, lead author of the study. “Familiarity is about analysing, for example, ‘Have I seen this face before or not?’ Recollection is about answering ‘Where have I seen this face before?’ and, if applicable, ‘Who does it belong to?'” “The curves are often used to gain insights into memory processes, for example, whether a particular brain injury has affected the familiarity or recollection component,” Prof. Dr Sen Cheng continues. “However, in literature we came across many divergent results on recognition memory. This made us curious.”

The Bochum neuroinformatics team suspected that the discrepancies were caused by the enormously complex interplay between different brain regions and thought processes in recognition memory. Hakobyan and Cheng based their research on the thesis that memory processes are too complex to be uniquely determined from an ROC curve. In the analysis, the focus is often on single processes, which then yields misleading results.

Computational model for replicating memory impairments

The computational model developed by the Bochum research duo explicitly shows how the performance of recognition memory is influenced by the interaction of various factors. “Although our model is abstract, it can be linked to brain regions involved in the corresponding memory functions,” said Hakobyan. “In this way, we use computer simulations to reproduce and interpret different memory impairments that occur in patients with frontal and hippocampal lesions.”

In their study, the Bochum duo found that ROC curves – in contrast to previous assumptions – do not allow a clear demarcation between familiarity and recollection processes. Rather, it shows an interplay of different factors such as memory processing, decision-making and stimulus statistics.

High sensitivity of ROC curves to small modifications

“We were surprised how sensitive the ROC curves are to small parameter changes in the model,” said Cheng. “This shows that we should be very cautious about classifying different memory processes based on ROC curves and calls into question the extent to which such a distinction represents real memory processes.”