Study Reconstructs Letters from Brain Scan Data

Study Reconstructs Letters from Brain Scan Data

Researchers in the Netherlands have taken another step towards that future by using brain scans to successfully identify what a person is perceiving, specifically the letters that they are seeing.

By the specific use of a combination of shape recognition and training an algorithm to understand the changes in functional magnetic resonance imaging (fMRI) signals caused by viewing a letter, a team from Radboud University Nijmegen was able to decode sensory information inside the brains of the study participants.

In simple terms, the pixels on your retina maintain their relative positions at the occipital lobe.

Their technique relies on intercepting visual stimuli at the occipital lobe, the visual processing centre at the back of the brain, where there is still a one-to-one correlation with the data at the retina.

Study participants were made to watch a series of letters flash on a screen while undergoing a fMRI scan. Different handwritten versions of the letters B, R, A, I, N and S were shown and the fMRI response at the occipital lobe monitored.

The researchers were able to teach their algorithm to recognise and recognise the shapes the participants had viewed by watching how small cubes of space -- voxels -- at the occipital lobe responded to the visual stimuli.

Once you've taught the algorithm how the voxels respond to stimulus, "you could reconstruct any possible inputs," says van Geven.

He is careful to point out that this isn't the same as reading a person's thoughts. While it's well-known that the occipital lobe responds to external stimuli, and therefore perception can be decoded there, it's not known if imagining the same stimuli also creates an effect at the occipital lobe.

The 2006 Inria study hinted that this might be the case, showing that imagined patterns could be reconstructed using data from the occipital lobe.