Add images

A ggbrain plot must have at least one 3D image stored in NIfTI format. These images are read into ggbrain using the RNifti package. All images for a given plot must have the same image dimensions (otherwise, an error will occur). At present, ggbrain provides no internal support for interpolation to combine images with different resolutions. We encourage you to use 3dresample from AFNI for this purpose (see details in the ‘Other considerations’ section below).

The file names containing relevant images can be passed in as arguments to ggbrain or they can be appended using images() In either, it is important to provide a name for the image that is used internally to refer to the image in contrasts, labels, or other operations. If you do not provide a name, you will see a warning:

Warning message:
In private$set_images(images, fill_holes, clean_specks) :
  The images vector does not contain any names. This may lead to weird behaviors downstream if 'underlay' and 'overlay' are requested.

gg_obj <- gg_obj + images(c(underlay = underlay_3mm))

Note that the image name of ‘underlay’ has a special meaning in the package. Specifically, using this name will set a default color scale that is grayscale and will not show the legend of this scale in the plot. You don’t have to use ‘underlay’ but it can make things a bit simpler if you have a conventional plot that shows an anatomical scan underneath functional activations.

You can add many images at once by providing a named vector:

gg_obj <- ggbrain() + 
  images(c(underlay = underlay_3mm, echange = echange_overlay_3mm))

Or you can add many images in independent images steps. Note that the order of addition is not important.

gg_obj <- ggbrain() +
  images(c(underlay = underlay_3mm)) +
  images(c(echange = echange_overlay_3mm))

Add slices

After adding images to the ggbrain object, the next step is to specify which slices should be displayed on the plot. This is achieved using the slices() function. Following standard neuroimaging terminology, we can add axial, sagittal, and coronal slices.

The order of the addition operations, whether within a single slices() call or in a sequence of calls like slices("x=12") + slices(c("y=15", "z=10")), determines the order of panels on the rendered plot from upper left to lower right (following the standard facet_wrap() conventions in ggplot2). Thus, consider the order in which you wish the panels to appear so that the slice additions follow this order.

gg_obj <- gg_obj + slices(c("x = 10", "y = 70", "z = 15"))

gg_obj <- gg_obj + slices(c("x = 25%", "y = 50%", "z = 90%"))

gg_obj <- gg_obj +
  slices(
    c("x = 25%", "x = 50%"),
    bg_color = "gray25", xlab = "x axis", theme_custom = theme(axis.text.x = element_text(size = 24))
  )

gg_obj <- gg_obj + slices(montage("axial", 10, min = 0.1, max = 0.8))

gg_obj <- gg_obj + slices(montage("sagittal", 5, min_coord = -10, max_coord = 10))

Add brain overlay layers

Now that we have defined the images to read and the slices to render, we need to map the numeric values in the slices to the points on the plot. This is achieved by geom_brain(), which is essentially an extension of geom_raster() from ggplot2. The numeric values in the slices fill squares on the plot, essentially forming ‘pixels’.

As with standard ggplot2 objects, the fill color of these pixels is determined by a scale_fill_* scale, allowing you to have fine control over the coloration.

Extending geom_raster(), the geom_brain() layer supports the following standards:

• interpolate: if TRUE, geom_raster smooths the layer, rather than rendering discrete squares.
• alpha: A number between 0 and 1 controlling the alpha transparency of the layer
• show_legend: If TRUE (default), a legend mapping the numeric values to a color scale is included in the panel
• fill: A color to be used to fill all non-NA pixels. In ggplot2 terms, this is ‘setting’ the color of all pixels, as opposed to ‘mapping’ the color to the numeric value via a scale.
• mapping: The aesthetic mapping between the data and the raster layer. In general, the default of aes(fill=value) should be retained for parametric layers (i.e., where the color of pixels should map to a continuous scale). But as detailed in a subsequent section, the fill can mapped to label columns in a categorical image (e.g., atlas).

Relative to standard ggplot2 conventions, the use of geom_brain() requires that you specify details about the fill color scale when the layer is added. This results from the fact that ggbrain objects re-use the ‘fill’ geometry when layers are overlaid. For example, if you have an anatomical underlay with a grayscale fill and a statistical overlay with a color fill, each layer needs to have its own fill scale (with a scale_fill_* object). This is achieved internally using the ggnewscale package.

The ‘take-home message’ is that you need to specify fill scale information as part of the geom_brain() call. Here are the key arguments:

• limits: a two-element vector that if provided, sets the upper and lower bounds on the scale
• breaks: a function to draw the breaks on the scale. The default is integer_breaks(), which draws a few integer-valued breaks (usually, 5) between the min and max of the scale.
• unify_scale: if TRUE (default), the color scale across panels (slices) will be equated. If FALSE, each panel will apply the color scale independently such that the min and max are slice-dependent.
• fill_scale: a scale_fill_* object use for mapping the numeric values in the data to the fill channel on the plot. Standard scales such as scale_fill_viridis_c() can be used for continuous layers. Note that the package also provides a scale_fill_bisided() option that, if added, creates separate color scales for the positive and negative pixels on the plot. This is a conventional approach in displays of fMRI activation maps where, for example, positive modulation is mapped to warm colors (reds) while negative modulation is mapped to cool colors (blues). Note that scale_fill_bisided() has arguments of neg_scale and pos_scale that allow the scales used for negative and positive values, respectively, to be specified directly.

# define a reusable object that has the same images and slices
gg_base <- ggbrain(bg_color = "gray80", text_color = "black") +
  images(c(underlay = underlay_3mm, overlay = echange_overlay_3mm)) +
  slices(c("x = 10", "y = 70", "z = 15"))

gg_obj <- gg_base +
  geom_brain(definition = "underlay", fill_scale = scale_fill_gradient(low = "grey8", high = "grey62"), show_legend = FALSE) +
  geom_brain(definition = "overlay", fill_scale = scale_fill_bisided(), show_legend = TRUE)

gg_obj$render()

## Warning: `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.

gg_obj <- gg_base +
  geom_brain(definition = "overlay[abs(overlay) > 2.5]", fill_scale = scale_fill_bisided(), show_legend = TRUE)

plot(gg_obj)

## Warning: `guide_colourbar()` cannot be used for fill_ggnewscale_1.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_1.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_1.
## ℹ Use one of fill or fill_new instead.

gg_obj <- gg_base +
  geom_brain(definition = "underlay") +
  geom_brain(definition = "overlay[underlay > 10]", fill_scale = scale_fill_bisided(), show_legend = TRUE)

plot(gg_obj)

## Warning: `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.

gg_obj <- gg_base +
  geom_brain(definition = "underlay") +
  geom_brain(
    definition = "overlay[abs(overlay) > 2.5 & underlay > 20]",
    fill_scale = scale_fill_bisided("z"), show_legend = TRUE
  )

plot(gg_obj)

## Warning: `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.

gg_obj <- ggbrain(bg_color = "gray80", text_color = "black") +
  images(c(underlay = underlay_3mm, pe = pe_overlay_3mm, abspe = abspe_overlay_3mm)) +
  slices(c("x = 10", "y = 50", "z = 15")) +
  define("diff := pe - abspe") +
  geom_brain(definition = "underlay") +
  geom_brain(definition = "diff", fill_scale = scale_fill_bisided(symmetric = FALSE), show_legend = TRUE)

plot(gg_obj)

## Warning: `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.
## `guide_colourbar()` cannot be used for fill_ggnewscale_2.
## ℹ Use one of fill or fill_new instead.

gg_obj <- ggbrain(bg_color = "gray80", text_color = "black") +
  images(c(underlay = underlay_3mm, pe = pe_overlay_3mm, abspe = abspe_overlay_3mm)) +
  slices(c("x = 10", "y = 50", "z = 15")) +
  geom_brain(definition = "diff := pe - abspe", fill_scale = scale_fill_bisided(symmetric = FALSE), show_legend = TRUE)

gg_obj <- ggbrain(bg_color = "gray80", text_color = "black") +
  images(c(underlay = underlay_3mm, pe = pe_overlay_3mm, abspe = abspe_overlay_3mm)) +
  slices(c("x = 10", "y = 50", "z = 15")) +
  define("diff := pe - abspe") +
  geom_brain(definition = "underlay") +
  geom_brain(definition = "diff[diff > 3]", fill_scale = scale_fill_distiller("PE > absPE", palette="Reds"), show_legend = TRUE)

plot(gg_obj)

Add outlines

In addition to filling in regions with colors, brain plots often benefit from being able to outline, rather than fill, regions. This is accomplished using geom_outline(). This geometry finds the boundaries of distinct clusters in the image/contrast and traces the boundaries with a line of some thickness (controlled by size).

For continuous images (e.g., z-statistics), one typically sets the outline color using geom_outline(outline="cyan") or whatever color is desired. This draws the outlines for non-NA voxels in a single color. For categorical images (e.g., anatomical or functional atlases), it may be useful to map the color of the outlines to a categorical label such as network or region. This is accomplished with geom_outline(mapping=aes(outline=roi_name)) or whatever column in the data corresponds to the categories one wishes to draw with different colors.

Here, we might wish to outline voxels whose z-statistics exceed |z| > 3 for the decision onset regressor. These voxels represent areas that are sensitive to the onset of the decision phase of the reinforcement learning trial.

gg_obj <- ggbrain() +
  slices(c("x = -2", "x=2", "y = 40", "z = 15")) +
  images(c(underlay = underlay_3mm, onset = decision_onset_3mm)) +
  geom_brain(definition = "underlay") +
  geom_outline(definition = "onset[abs(onset) > 3]", size = 1, outline = "cyan")

plot(gg_obj)

Converting to a ggplot object and displaying the plot

For reasons that are somewhat boring (having to do with how R handles S3 operators like +), the brain-related addition steps for ggbrain object must come before generic ggplot2 modifications of the plot. For your ggbrain object to be a ggplot object, it must be rendered using the render() function. This function compiles the images, layers, and slices into the requested plot and returns it as a patchwork object. The patchwork package is extremely useful for combining ggplot objects in different layouts. More details can be found here: https://patchwork.data-imaginist.com.

In ggbrain, each panel is ggplot object and the panels are composited using wrap_plots in patchwork.

The key message here is: place ggbrain-specific additions before render() and ggplot-specific additions afterwards.

# without a render step
gg_obj <- ggbrain() +
  slices(c("x = -2", "x=2", "y = 40", "z = 15")) +
  images(c(underlay = underlay_3mm, onset = decision_onset_3mm)) +
  geom_brain(definition="underlay") +
  geom_outline(definition = "onset[abs(onset) > 3]", size=1, outline="cyan")

class(gg_obj)

## [1] "ggb" "R6"

# note the different class after render
gg_obj <- ggbrain() +
  slices(c("x = -2", "x=2", "y = 40", "z = 15")) +
  images(c(underlay = underlay_3mm, onset = decision_onset_3mm)) +
  geom_brain(definition="underlay") +
  geom_outline(definition = "onset[abs(onset) > 3]", size=1, outline="cyan") +
  render()

class(gg_obj)

## [1] "patchwork" "gg"        "ggplot"

Using the general approach provided by patchwork, we can add annotations and aesthetic modifications to the overall plot using the plot_annotation. Likewise, you can modify the overall theme information using the theme() function, as articulated here.

# add a different theme to all panels and add an overall title
gg_obj + plot_annotation(title="Overall title") & theme_minimal()

Note that you don’t necessarily need to use render() at all. When you call plot() on a ggbrain object, the render happens automatically behind the scenes, allowing calls like:

gg_obj <- ggbrain() +
  slices(c("x = -2", "x=2", "y = 40", "z = 15")) +
  images(c(underlay = underlay_3mm, onset = decision_onset_3mm)) +
  geom_brain(definition="underlay") +
  geom_outline(definition = "onset[abs(onset) > 3]", size=1, outline="cyan")

plot(gg_obj) + plot_annotation(title = "Overall")

Another point is that by keeping your plots as ggbrain objects – that is not adding render() – you can continue to add brain-related calls like geom_brain() or annotate_coordinates() to them. Thus, you may wish to use the plot(gg_obj) + [ggplot2 stuff here] approach above so that the ggbrain object remains in its original form and can be modified prior to display.