Clinical Applications 
 

Projects: 

Brain Flattening 

Brain Mapping 

Brain Movies 
 

1. tomographic reconstruction of  structure and hemodynamic/metabolic processes; and 
2. recordings of electromagnetic fields. 

Examples of Anatomically-Constrained MEG (aMEG) 

 Median nerve stimulation 
One method of empirically validating the aMEG methodology is to compare the localizations that it derives in situations where the generator of the brain activity is thought to already be known from previous intracranial studies in humans as well as animal studies. By these criteria, the early (~20ms) response to median n. stimulation is generated in the posterior and anterior banks of the central sulcus of the contralateral hemisphere. aMEG assigns activation to the predicted areas (Figure 1). 

 
Figure 1. Snapshots of brain movies at 20 ms after median nerve stimulation. Rolandic cortex in the left hemisphere (left 2 images)  responds only to right median n. stimulation (second image). Similarly, Rolandic cortex in the right hemisphere (right 2 images)  responds only to left median n. stimulation (third image). 

Epilepsy 
   An important clinical application of the proposed software is in localizing the origin and propagation of interictal epileptiform spikes. aMEG of two spikes with propagation from the frontal pole to the inferior frontal gyrus pars orbitalis is shown in Figure 2. This subject was subsequently examined with subdural grids and strips, which confirmed the aMEG localization (Figure 3, and movie spike). Removal of this area resulted in complete abolition of seizures. 
Movie 1: Activity from an epileptiform spike is seen to spread from the right frontal pole at 360 ms to the right ventrolateral prefrontal cortex 16 milliseconds later.  This pattern of spread was confirmed by direct intracranial recordings and surgical removal of these areas has cured the patient's seizures.  (spike) 
 
Figure 2. Snapshots from aMEG movies of two epileptiform discharges. In both cases, the spike begins in the right frontal pole and spreads 20ms later to involve the right inferior frontal gyrus. 
Movie 1- To see a movie of spike 2 click here: spike 

 
Figure 3. Intracranial exploration of the patient shown in Figure 2 and Movie 1. Intraoperative photos were taken of the cortical surface before and after placement of the grid electrodes (white circles), and again after the cortectomy (hatched area). These images are superimposed on the reconstructed cortical surface. Note that subdural strips also sampled the frontal polar, orbital, medial frontal, and anterior temporal cortex. 

Combined MEG/EEG/fMRI studies of motion processing 
  We have collected fMRI, 32 channel EEG and 122 channel MEG data in the same subjects and with similar stimuli , in an attempt to characterize the spatiotemporal pattern of activity in human cortex in response to visual motion. In order to test the overall consistency of the MEG activations with the corresponding fMRI activations, we first estimated the activity at each point on the cortical surface independent of the fMRI data (that is, using 0% fMRI weighting). The estimated activity map over time evoked by motion onset is shown on the left in Figure 4. Comparison with the fMRI activation to moving vs. stationary visual stimuli (shown on the right) reveals a general similarity between the two maps, suggesting a general agreement between neural activity, as measured by MEG, and the hemodynamic response, as measured by fMRI. This also suggests that our linear estimation approach provides a reasonably accurate localization of MEG activity, even in the absence of fMRI data. However, closer examination shows that the MEG-based solution is slightly displaced anterodorsally to the fMRI measurement, suggesting that MEG with anatomical constraints alone is not sufficient for a high-accuracy spatiotemporal map. 
 
Figure 4. Brain activation to low-contrast moving stimuli: anatomically-constrained MEG (left), fMRI (right). 

   In a further experiment, we investigated the differential response to coherent and incoherent motion. The subjects viewed two types of rotation/dilation random dot flow field stimuli. In one condition (coherent motion) each dot moved as part of the same flow field, while in a other condition (incoherent motion) each dot moved independently, but with the same local statistics as in the coherent case. The time course of electrical activity for each point on the cortical surface was estimated using our fMRI-weighted anatomically constrained linear estimation approach. The solutions were weighted towards all visual areas identified as motion specific for each subject using fMRI, including V3A, MT, PSVA, and SPO. The estimated time-courses of activity within the different visual areas are shown in a flattened representation of the occipital lobe in Figure 5 for both coherent (red/purple) and incoherent (green/cyan). (Waveforms are plotted separately for high and low random dot density stimuli (hd and ld)).  
 
Figure 5. Estimated Activity of Multiple Visual Areas to Coherent and Incoherent Motion.  Both coherent and incoherent motion onset produce a complex sequence of activation within the different visual areas. 

The MEG response to coherent and incoherent motion is estimated by the linear approach to be quite similar in all areas, except in area SPO, where the response to coherent motion is significantly greater than that to incoherent motion. Interestingly, this is also the area which has been shown to be selective for coherent motion by fMRI . Since the localization of the MEG motion coherence effect was carried out independently of the fMRI motion coherence effect, the fact that hey both localized to SPO provides further converging evidence for the anatomical accuracy of the linear estimation approach. 

Spatiotemporal imaging of semantic processing and word-repetition effects 
   In semantic judgment tasks involving words, repetition of a given word will result in a large change in the cerebral activity that it evokes. According to invasive EEG recordings as well as PET studies, this ‘repetition-effect’ is large and involves many extended cortical areas. Thus, the ‘repetition-effect’ provides a difficult but realistic case for testing the anatomically-constrained fMRI-weighted linear estimation procedure from EEG/MEG data.  
    MEG/EEG recordings were obtained  in 4 normal subjects by reading a briefly-appearing word from a monitor, and deciding if the object or animal that it represents is more than a foot long in any dimension. In half of the blocks, all of the words were novel, and in half the same set of words were repeatedly presented.  aMEG demonstrated a progression of activation to novel words from primary visual cortex to inferotemporal, anterior temporal, and finally ventrolateral prefrontal cortices. The same progression of activation was found to repeated words, only it faded more quickly. In the movie of novel minus repeated words, the repetition effect is seen to involve many parts of the cortex, with the highest levels (ie Broca's area) involved at least as early as the more perceptual levels. These findings offer new insights into the progression of cortical events in memory.  Snapshots of this activation are shown in Figure 6 and a movie in size.  Movies of other verbal paradigms yield similar movies (see movie rhyme). 
    The identical task was given during whole-head  fMRI  (Figure 7a).  fMRI activation was predominantly in the left hemisphere. Novel and repeated words activated the retinotopic visual areas (V1-V4) and primary somatosensory and motor areas approximately equally. Three bands of cortex consistently responded more to novel than to repeated words: the occipitotemporal junction (area 37); intraparietal sulcus including superior angular and supramarginal gyrus (areas 7, 39, and 40); and postero-ventral prefrontal cortex. 
    Using this differential fMRI response to novel and repeated stimuli to bias the anatomically constrained inverse solution, we were able to estimate a spatiotemporal map of the repetition effect. Figure 8 shows snapshots of the estimated activity maps at 335, 370, and 510 ms post stimulus onset. By comparing the fMRI biased solution shown in the top row (90% fMRI weighting) with the independent anatomically constrained solution (0% fMRI weighting) shown in the bottom row, we notice a strong overall similarity, again providing further evidence for a general correspondence between electrical activity and hemodynamic response. However, it is worth noting that the independent anatomically constrained activations appear more spread out, presumably reflecting the limited spatial resolution of MEG by itself. Note also that the activity in the intraparietal sulcus is poorly localized unless fMRI-weightings are added, and that the activity in Broca’s area is partially mis-allocated to the anterior superior temporal plane in the independent anatomically constrained solution. Adding fMRI constraints corrects this apparent misallocation. 

 
Figure 6. Snapshots of brain activation estimated from anatomically-constrained MEG. 

 
Figure 7a. Above. fMRI responses to Novel-minus-Fixation (left) and Novel-minus-Repeated words in a Size judgement task. 
Figure 7b. Below. Comparison of the aMEG (left), and iEEG (right) responses to words in the Size task. 

 
Figure 8. Brain areas differentially activated by novel vs. repeated words in a size-judgement task. 

Movie 2: To see a movie of the response to novel words in a single subject in this task, click here: size. 
   In this movie, the responses of four subjects' brains during a size judgement task are averaged using surface morphing.  The subjects read words that referred to objects or animals.  They pressed a key if the item was larger than 1 foot in its longest dimension (e. g., elephant or house) and withheld their response if it was not (e.g. fly or pin).  The earliest response is in the visual areas at the very back of the brain (occipital pole).  However, almost immediately, the activity goes forward to the temporal pole, known to be inportant for semantic knowledge-- the sorts of facts that this task uses.  By 170 ms, the whole base of the occipital lobe (used for higer visual processing of objects and patterns,) as well as the temporal lobe, are are very active.  At 230 ms, the temporal activity is even more intense, but the occipital activity has greatly decreased.  By 365 ms, the activity has shifted even further anteriorly, and now heavily involves the ventral posterior frontal cortex, again as an area intimately involved in semantic knowledge and comparisons.  Activation remains strong for another 400 ms.  However, as words are repeatedly processed (Figure 7), the activation dies away much more quickly. 

Movie 3: To see a movie of the response to novel words in a different subject in a rhyming task, click here: rhyme. 
   The study of behavior after strokes or other brain injury led to the discovery about 100 years ago of two brain areas that are essential for language:  Wernicke's area, in the posterior superior temporal lobe, and Broca's area, in the posterobasal prefrontal cortex.  This movie was made of a subject who read words and decided if they thymed with "bay."  The first significant activation arrives in the Wernicke's area at about 100 ms after the word exposure onset, only 30 ms after the first activity in the primary visual cortex (hidden on the medial cortical surface).  Gradually, this activity intensifies and involves Broca's area and the temporal pole.  The temporal activity is maximal at about 265 ms after word onset and the frontal activity at about 380 ms.  By 440 ms, the activity is mainly frontal, and by 560 ms, it has faded away. 

Validation of noninvasive estimates by comparison with intracranial recordings 
   A more direct test of the accuracy of the noninvasive activity estimates comes from a comparison anatomically constrained MEG (aMEG) with invasive recordings in patients (iEEG). Figure 7b (left) shows the time-averaged anatomically constrained estimate for the word repetition effect discussed above (here averaged across four subjects), displayed on a folded brain. The generators of potentials evoked by novel and repeated words, summarized from a previous study of about 2500 iEEG recordings , are shown on the right. Such recordings are only performed in epileptics and thus may be affected by pathology and limited sampling. However, they can identify generators without the ambiguities associated with all extracranial electromagnetic measures. Note the close correspondence is seen between the iEEG-identified generators of the components that change with word repetition (N4 and P3b), and those estimated to change with repetition using aMEG. 
   As a further test of the accuracy of the noninvasive estimates, the aMEG waveforms were compared to ones actually measured in similar locations with intracranial electrodes (Figure  9) from a previously published iEEG study . (Note that there is no iEEG recording from the early visual cortex.) Thick lines show the response to novel words, and thin lines the response to repeated words. Although the general similarity between the aMEG and iEEG waveforms provide further evidence for the accuracy of the noninvasive estimates, further studies are needed in order to assess the accuracy within-subject, with higher anatomical precision. 

 
Figure 9. Comparison of anatomically constrained MEG (aMEG) and invasive recordings (iEEG).