wiki.wtex

<latex>
\documentclass{scrartcl}
%\documentclass[a4paper]{article}
%\usepackage{mathpazo,helvet}
\usepackage[T1]{fontenc}

\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsrefs}
\usepackage{subfig}
\usepackage{funkey}
\usepackage{notation}
\usepackage{array}
\usepackage{menukeys}
\usepackage{hyperref}
\input{tikzstyles}

\begin{document}

\title{BigDataViewer Documentation}
\author{Tobias Pietzsch}
\maketitle
\newpage
\tableofcontents
\newpage

%\newcommand\menu[1]{\textit{#1}}
\newcommand\button[1]{\textit{#1}}
\newcommand\key[1]{\texttt{#1}}
\newcommand\screenshotA[1]{\centerline{\includegraphics[width=.8\textwidth]{figures/#1}}}
\newcommand\screenshotB[1]{\centerline{\includegraphics[width=.8\textwidth]{figures/#1}}}
\newcommand\screenshotTikz[1]{\centerline{\includetikz{figures/#1}}}
\newcommand\coloredlink[1]{\textcolor{blue!75!black}{\underline{\smash{#1}}}}
</latex>
<wiki>
{{Infobox
| name                  = BigDataViewer
| software              = Fiji
| author                = [[User:Pietzsch|Tobias Pietzsch]]
| maintainer            = [mailto:pietzsch@mpi-cbg.de Tobias Pietzsch]
| source                = {{GitHub|org=bigdataviewer|repo=bigdataviewer_fiji}}
| latest version        = 1.0.9
| status                = stable
| category              = [[:Category:Visualization|Visualization]], [[:Category:Transform|Transform]]
}}

__TOC__
</wiki>

\section{Description}
%--------------------
The \bdv is a re-slicing browser for terabyte-sized multi-view image sequences.
\Bdv was developed with multi-view light-sheet microscopy data in mind and integrates well with Fiji's SPIMage processing pipeline.

Conceptually, the visualized data comprises multiple \emph{data sources}.
Each source provides one 3D image (for each time-point in the case of a time-lapse sequence).
For example, in a multi-angle SPIM sequence, each angle is a source.
In a multi-angle, multi-channel SPIM sequence, each channel of each angle is a source.

\Bdv comes with a custom data format that is is optimized for fast random access to very large data sets.
This permits browsing to any location within a multi-terabyte recording in a fraction of a second.

The file format is based on XML and HDF5<latex>~\cite{hdf5}</latex>.
Images are represented as tiled multi-resolution pyramids, and stored in HDF5 chunked multi-dimensional arrays.
The XML file contains metadata, for example the registration of sources to the global coordinate system.

\section{Installation}
%---------------------
The BigDataViewer comes with Fiji.
You should have a sub-menu \menu{Plugins > BigDataViewer}.

\section{Usage}
%--------------

\subsection{Opening a Dataset}
%---------------------------
To use the BigDataViewer we need some example dataset to browse.
There are various options:

\subsubsection{Multi-view data converted to XML/HDF5}
%---------------------------------------------------------
A special purpose of BigDataViewer is to visualise multi-view light sheet microscopy datasets.
Our custom XML/HDF5  is a special purpose hierarchical data format that optimises access to any part of large-multi view datasets using ImgLib2.
You can download a small dataset from
<latex>\url{http://fly.mpi-cbg.de/~pietzsch/bdv-example/}</latex>
<wiki>[http://fly.mpi-cbg.de/~pietzsch/bdv-example/ here]</wiki>
, comprising two views and three time-points.
This is an excerpt of a 6 angle 715 time-point sequence of \emph{drosophila melanogaster} embryonal development, imaged with a Zeiss Lightsheet Z.1.
Download both the XML and the HDF5 file and place them somewhere next to each other.

Alternatively, you can create a dataset by exporting your own data as described below.

To start \bdv, select \menu{Plugins > BigDataViewer > Open XML/HDF5} from the Fiji menu.
This brings up a file open dialog. Open the XML file of your test dataset.

\subsubsection{Standard microscopy data}
%---------------------------------------
The BigDataViewer can be used to visualise and navigate any image that can be opened in Fiji.
Since Fiji relies of LOCI Bioformats library that means essentially all know microscopy file formats.

For example, open the sample image \menu{File > Open Samples > Mitosis (26MB, 5D stack)} from the Fiji menu.
Subsequently select \menu{Plugins > BigDataViewer > Open Current Image} which will launch the BigDataViewer with the sample image, for navigation by arbitrary re-slicing.

\subsubsection{Imaris files}
%---------------------------------------
<latex>\href{http://www.bitplane.com/imaris/imaris}{\coloredlink{Imaris (Bitplane)}}</latex>
<wiki>[http://www.bitplane.com/imaris/imaris Imaris (Bitplane)]</wiki>
 uses a hierarchical data format (similar to \bdv's XML/HDF5 format).
In order to open Imaris \texttt{.ims} files select \menu{Plugins > BigDataViewer > Open Imaris (experimental)} from the Fiji menu.
Please note, that support for the Imaris format is still experimental.

\subsection{Basic Navigation}
%----------------------------
Assuming you downloaded the \emph{drosophila melanogaster} example dataset, you should see something like this:
\\
\screenshotA{bdv-start.png}
\\
On startup, the middle slice of the first source (angle) is shown.
You can browse the stack using the keyboard or the mouse.
To get started, try the following:
\begin{itemize}
  \item Use the mouse-wheel or \keys{<} and \keys{>} keys to scroll through z slices.
  \item \keys{right-click + drag} anywhere on the canvas to translate the image.
  \item Use \keys{\ctrl + \shift + mouse-wheel}, or \keys{\arrowkeyup} and \keys{\arrowkeydown} keys to zoom in and out.
  \item \keys{left-click + drag} anywhere on the canvas to rotate (reslice) the image.
\end{itemize}

The following table shows the available navigation commands using the mouse:
<latex>
\vspace{-2mm}
\newcommand{\specialcell}[2][c]{%
  \begin{tabular}[#1]{@{}l@{}}#2\end{tabular}}
\newcommand\tablecell[2][&]{#2 #1}
</latex>

<wiki>
{| border="1" style="border-collapse:collapse;"
</wiki>%
<latex>
{
%\setlength{\extrarowheight}{5pt}
\renewcommand{\arraystretch}{1.5}
\begin{center}
\begin{tabular}{ | p{.3\textwidth - 2\tabcolsep} | p{.7\textwidth - 2\tabcolsep} |}
    \hline
</latex>
  \tablecell{\keys{left-click + drag}}
  \tablecell[\\]{Rotate (pan and tilt) around the point where the mouse was clicked.}
  \hline
%
  \tablecell{\specialcell{\keys{right-click + drag} or\\ \keys{middle-click + drag}}}
  \tablecell[\\]{Translate in the XY-plane.}
  \hline
%
  \tablecell{\keys{mouse-wheel}}
  \tablecell[\\]{Move along the z-axis.}
  \hline
%
  \tablecell{\specialcell{\keys{\cmd + mouse-wheel} or\\ \keys{\shift + \ctrl + mouse-wheel}}}
  \tablecell[\\]{Zoom in and out.}
<latex>
  \hline
\end{tabular}
\end{center}
}
</latex>
<wiki>
|}

</wiki>

The following table shows the available navigation commands using keyboard shortcuts:
<wiki>
{| border="1" style="border-collapse:collapse;"
</wiki>%
<latex>
{
%\setlength{\extrarowheight}{5pt}
\renewcommand{\arraystretch}{1.5}
\begin{center}
\begin{tabular}{ | p{.3\textwidth - 2\tabcolsep} | p{.7\textwidth - 2\tabcolsep} |}
    \hline
</latex>
  \tablecell{\keys{X}, \keys{Y}, \keys{Z}}
  \tablecell[\\]{Select keyboard rotation axis.}
  \hline
%
  \tablecell{\keys{\arrowkeyleft}, \keys{\arrowkeyright}}
  \tablecell[\\]{Rotate clockwise or counter-clockwise around the choosen rotation axis.}
  \hline
%
  \tablecell{\keys{\arrowkeyup}, \keys{\arrowkeydown}}
  \tablecell[\\]{Zoom in or out.}
  \hline
%
  \tablecell{\keys{,}, \keys{.}}
  \tablecell[\\]{Move forward or backward along the Z-axis.}
  \hline
%
  \tablecell{\keys{\shift + X}}
  \tablecell[\\]{Rotate to the ZY-plane of the current source. (Look along the X-axis of the current source.)}
  \hline
%
  \tablecell{\keys{\shift + Y} or \keys{\shift + A}}
  \tablecell[\\]{Rotate to the XZ-plane of the current source. (Look along the Y-axis of the current source.)}
  \hline
%
  \tablecell{\keys{\shift + Z}}
  \tablecell[\\]{Rotate to the XY-plane of the current source. (Look along the Z-axis of the current source.)}
  \hline
%
  \tablecell{\keys{[} or \keys{N}}
  \tablecell[\\]{Move to previous timepoint.}
  \hline
%
  \tablecell{\keys{]} or \keys{M}}
  \tablecell[\\]{Move to next timepoint.}
<latex>
  \hline
\end{tabular}
\end{center}
}
</latex>
<wiki>
|}

</wiki>

\noindent
For all navigation commands you can hold \keys{\shift} to rotate and browse
<latex>$10\times$ </latex>
<wiki>10x </wiki>
faster, or hold \keys{\ctrl} to rotate and browse
<latex>$10\times$ </latex>
<wiki>10x </wiki>
slower.
For example, \keys{\arrowkeyleft} rotates by
<latex>$1^\circ$ </latex>
<wiki>1° </wiki>
clockwise, while \keys{\shift + \arrowkeyleft} rotates by
<latex>$10^\circ$</latex>
<wiki>10°</wiki>
, and \keys{\ctrl + \arrowkeyleft} rotates by
<latex>$0.1^\circ$</latex>
<wiki>0.1°</wiki>
.

The axis-rotation commands (\eg, \keys{\shift + X}) rotate around the current mouse location.
That is, if you press \keys{\shift + X}, the view will pivot such that you see a ZY-slice through the dataset (you look along the X-axis).
The point under the mouse will stay fixed, \ie, the view will be a ZY-slice through that point.

<latex>\newpage</latex>
\subsection{Interpolation Mode}
%------------------------------
Using \keys{I} you can switch between nearest-neighbor and trilinear interpolation schemes.
The difference is clearly visible when you zoom in such that individual source pixels are visible.
\\
\screenshotA{interpolation.png}
\\
Trilinear interpolation results in smoother images but is a bit more expensive computationally.
Nearest-neighbor is faster but looks more pixelated.

%
<latex>\newpage</latex>
\subsection{Displaying Multiple Sources}
%---------------------------------------
\Bdv datasets typically contain more than one source.
For a SPIM sequence one usually has multiple angles and possibly fused and deconvoled data on top.

Select \menu{Settings > Visibility \& Grouping} from the \bdv menu to bring up a dialog to control source visibility.
You can also bring up this dialog by the shortcut \keys{F6}.
\\
\screenshotB{visibility.png}
\\
Using the current source checkboxes (A in the figure above), you can switch between available sources.
The first ten sources can also be made current by the number keys \keys{1} through \keys{0} in the main \bdv window.
%In normal display mode only the \emph{current source} is visible.

To view multiple sources overlaid at the same time, switch to \emph{fused mode} using the checkbox (B).
You can also switch between normal and fused mode using the shortcut \keys{F} in the main window.
In fused mode individual sources can be turned on and off using the checkboxes (C) or shortcuts \keys{\shift + 1} through \keys{\shift + 0} in the main window.

Whether in normal or fused mode, the (unselectable) boxes (D) provide feedback on which sources are actually currently displayed.
Also the main window provides feedback:
\\
\screenshotB{overlays-1.png}
\\
In the top-left corner an overview of the dataset is displayed (E).
Visible sources are displayed as green/magenta wireframe boxes, invisible sources are displayed as grey wireframe boxes.
The dimensions of the boxes illustrate the size of the source images. The filled grey rectangle illustrates the screen area, \ie, the portion of the currently displayed slice.
For the visible sources, the part that is in front of the screen is green, the part that is behind the screen is magenta.

At the top of the window, the name of the current source is shown (F).

Note, that also in fused mode there is always a \emph{current source}, although this source may not even be visible.
Commands such as \keys{\shift + X} (rotate to ZY-plane) refer to the local coordinate system of the current source.

%
\subsection{Grouping Sources}
%----------------------------
Often there are sets of sources for which visibility is logically related.
For example, in a multi-angle, multi-channel SPIM sequence, you will frequently want to see all channels of a given angle, or all angles of a given channel.
If your dataset contains deconvolved data, you may want to see either all raw angles overlaid, or the deconvolved view, respectively.
You want to be able to quickly switch between those two views.
Turning individual sources on and off becomes tedious in these situations.
Therefore, sources can be organized into \emph{groups}.
All sources of a \emph{group} can be activated or deactivated at once.

Source grouping is handled in the visibility and grouping dialog, too
(menu \menu{Settings > Visibility \& Grouping} or shortcut \keys{F6}).
\\
\screenshotB{grouping.png}
\\
The lower half of the dialog is dedicated to grouping.
There are 10 groups available.
They are named ``group 1'' through ``group 10'' initially, but the names can be edited (A).

Sources can be assigned to groups using the checkboxes (B).
In every line, there are as many checkboxes as there are sources.
Sources corresponding to active checkboxes are assigned to the respective group.
For example, in the above screenshot there are two sources and therefore two ``assigned sources'' checkboxes per line
The first source is assigned to groups 1 and 2, the second source is assigned to groups 2 and 3.
Group 2 has been renamed to ``all sources''.

\emph{Grouping} can be turned on and off by the checkbox (C) or by using the shortcut \keys{G} in the main window.
If grouping is enabled, groups take the role of individual sources:
There is one \emph{current group} which is visible in normal mode (all individual sources that are part of this group are overlaid).
Groups can be activated or deactivated to determine visibility in fused mode (all individual sources that are part of at least one active group are overlaid).

Groups can be made current and made active or inactive using the checkboxes (D).
Also, if grouping is enabled the number key shortcuts in the main \bdv window act on groups instead of individual sources.
That is, groups 1 through 10 can be made current by keys \keys{1} through \keys{0}.
Similarly, shortcuts \keys{\shift + 1} through \keys{\shift + 0} in the main window activate or deactivate groups 1 through 10 for visibility in fused mode.

If grouping is enabled, the name of the current group is shown at the top of the main window.
\\
\screenshotB{overlays-2.png}

%
\subsection{Adjusting Brightness and Color}
%------------------------------------------
To change the brightness, contrast, or color of particular sources select \menu{Setting > Brightness \& Color} or press the shortcut \keys{S}.
This brings up the brightness and color settings dialog.
\\
\screenshotB{brightness-1.png}
\\
The \emph{min} and \emph{max} sliders (A) can be used to adjust the brightness and contrast.
They represent minimum and maximum source values that are mapped to the display range.
For the screenshot above, this means that source intensity 200 (and everything below) is mapped to black. Source intensity 862 (and everything above) is mapped to white.

When a new dataset is opened, \Bdv tries to estimate good initial \emph{min} and \emph{max} settings by looking at the first image of the dataset.

\Bdv datasets are currently always stored with 16 bits per pixel, however the data does not always exploit the full value range 0 \dots 65535.
The example drosophila dataset uses values in the range of perhaps 0 \dots 1000, except for the much brighter fiducial beads around the specimen.
The \emph{min} and \emph{max} sliders in this case are a bit fiddly to use, because they span the full 16 bit range with the interesting region squeezed into the first few pixels.
This can be remedied by adjusting the range of the sliders.
For this, click on the <latex>$>>$</latex><wiki>>></wiki> dialog button (B).
This shows two additional input fields, where the range of the sliders can be adjusted.
In the following screenshot, the leftmost value of the slider range has been set to 0 and the rightmost value to 2000, making the sliders much more useful.
\\
\screenshotB{brightness-2.png}
\\
So far, all sources share the same \emph{min} and \emph{max} settings.
However, these can also be adjusted for each individual source or for groups of sources.
The checkboxes (C) assign sources to \emph{min-max-groups}.
There is one checkbox per source.
In the example drosophila dataset there are two sources, therefore there are two checkboxes.
The active checkboxes indicate for which sources the \emph{min} and \emph{max} values apply.

If you uncheck one of the sources, it will move to its own new \emph{min-max-group}.
Now you can adjust the values for each source individually.
The sliders of new group are initialized as a copy of the old group.
\\
\screenshotB{brightness-3.png}
\\
Sources can be assigned to \emph{min-max-groups} by checking/unchecking the checkboxes.
The rule is that every source is always assigned to exactly one min-max-group.
Thus, if you activate an unchecked source in a min-max-group, this will remove the source from its previous min-max-group and add it to the new one.
Unchecking a source will remove it from its min-max-group and move it to a new one.
Min-max-groups that become empty are removed.
To go back to a single min-max-group in the example, you would simply move all sources to the same group.

Finally, at the bottom of the dialog (D) colors can be assigned to sources.
There is one color button per source (two in the example).
Clicking a button brings up a color dialog, where you can choose a color for that particular source.
In the following screenshot, the sources have been colored magenta and green.
\\
\screenshotB{brightness-4.png}

%
\subsection{Bookmarking Locations and Orientations}
%--------------------------------------------------
\Bdv allows to bookmark the current view.
You can set bookmarks for interesting views or particular details of your dataset to easily navigate back to those views later.

Each bookmark has an assigned shortcut key, \ie, you can have bookmarks ``a'', ''A'', ''b'', \dots, ``1'', ``2'', \etc.
To set a bookmark for the current view, press \keys{\shift + B} and then the shortcut you want to use for the bookmark.
To recall bookmark, press \keys{B} and then the shortcut of the bookmark.

\Bdv provides visual feedback for setting and recalling bookmarks.
When you press \keys{\shift + B}, the message ``\textbf{set bookmark:}'' appears in the lower right corner of the main window, prompting to press the bookmark shortcut next.
\\
\screenshotB{set-bookmark.png}
\\
Now press the key you want to use as a shortcut, for example \keys{A}.
The promt message will change to ``\textbf{set bookmark: a}'' indicating that you have set a bookmark with shortcut \keys{A}.
Instead of pressing a shortcut key you can abort using \keys{esc}.

Similarly, when you press \keys{B} to recall a bookmark, the prompt message ``\textbf{go to bookmark:}'' appears.
Now press the shortcut of the bookmark you want to recall, for example \keys{A}.
The promt message will change to ``\textbf{go to bookmark: a}'' and the view will move to the bookmarked location.
Instead of pressing a shortcut key you can abort using \keys{esc}.

Note, that bookmark shortcuts are case-sensitive, \ie, \keys{A} and \keys{\shift + A} refer to distinct bookmarks ``a'' and ``A'' respectively.

The bookmarking mechanism can also be used to bookmark and recall orientations.
Press \keys{O} and then a bookmark shortcut to recall only the orientation of that bookmark.
This rotates the view into the rotation of the bookmarked view (but does not zoom or translate to the bookmarked location).
The rotation is around the current mouse location (\ie, the point under the mouse stays fixed).


%
\subsection{Loading and Saving Settings}
%---------------------------------------
Organizing sources into groups, assigning appropriate colors, adjusting brightness correctly, and bookmarking interesting locations is work that you do not want to repeat over and over every time you re-open a dataset.
Therefore, \Bdv allows to save and load these settings.

Select \menu{File > Save settings} from the menu to store settings to an XML file, and \menu{File > Load settings} to load them from an XML file.

When a dataset is opened, \bdv automatically loads an appropriately named settings file if it is present.
This settings file must be in the same directory as the dataset's XML file, and have the same filename with \emph{.settings} appended.
For example, if the dataset's XML file is named \emph{drosophila.xml}, the settings file must be named \emph{drosophila.settings.xml}.
(If you select \menu{File > Save settings}, this filename is already suggested in the Save File dialog.)

Settings files assume that a specific number of sources are present, therefore settings are usually not compatible across different datasets.

%
\subsection{Opening \Bdv Datasets as ImageJ Stacks}
%--------------------------------------------------
\Bdv may be great for looking at your data, but what if you want to apply other ImageJ algorithms or plugins to the images?
You can open individual images from a dataset as ImageJ stacks using \menu{File > Import > BigDataViewer...} from the Fiji menu.
\\
\screenshotB{import.png}
\\
Select the XML file of a dataset, then choose the time-point and source (setup) index of the image you want to open.
If you enable the \emph{``open as virtual stack''} checkbox the image will open as an ImageJ \emph{virtual stack}.
This means that the opened image is backed by \bdv's cache and slices are loaded on demand.
Without \emph{``open as virtual stack''}, the full image will be loaded into memory.
Virtual stacks will open a bit faster but switching between slices may be less instantaneous.

Note that the import function is macro-recordable.
Thus, you can make use of it to batch-process images from \bdv datasets.


%\section{Experimental features}
%\subsection{Recording a movie}
%\subsection{Manually adjusting transformations}
%\subsection{Exporting a cropped dataset}
%
%
\section{Exporting Datasets for the \Bdv}
%----------------------------------------

BigDataViewer uses a custom file-format that is optimized for fast arbitrary re-slicing at various scales.
This file format is build on open standards <latex>XML\cite{xml}</latex><wiki>[http://www.w3.org/XML/ XML]</wiki> and <latex>HDF5\cite{hdf5}</latex><wiki>[http://www.hdfgroup.org/HDF5/ HDF5]</wiki>.
XML is used to store meta-data and HDF5 is used to store image volumes.
(Actually, we support several ways to store the image volumes besides HDF5. For example, the volume data can be provided by a web service for remote access.)

Before we discuss how to export your data to this XML/HDF5 format, we provide a rough overview for some background, rationale, and terminology.
The format is explained in detail in our paper<latex>~\cite{arxiv-bdv}</latex><wiki> [http://arxiv.org/abs/1412.0488 BigDataViewer: Interactive Visualization and Image Processing for Terabyte Data Sets]</wiki>.

%
\subsection{About the \Bdv data format}
%--------------------------------------
Each \bdv dataset contains a set of 3D grayscale image volumes organized by \emph{timepoints} and \emph{setups}.
In the context of lightsheet microscopy, each channel or acquisition angle or combination of both is a \emph{setup}.
\Eg, for a multi-view recording with 3 angles and 2 channels there are 6 \emph{setups}.
Each \emph{setup} represents a visualisation data source in the viewer that provides one image volume per \emph{timepoint}.
We refer to each combination of \emph{setup} and \emph{timepoint} as a \emph{view}.
Each \emph{view} has one corresponding grayscale image volume.

A dataset comprises an XML file to store meta-data and one or more HDF5 files to store the raw images.
Among other things, the XML file contains
\begin{itemize}
  \item the path of the HDF5 file(s),
  \item a number of \emph{setups},
  \item a number of \emph{timepoints},
  \item the registration of each \emph{view} into the global coordinate system.
\end{itemize}

Each \emph{view} has one corresponding image volume which is stored in the HDF5 file.
Raw image volumes are stored as multi-resolution pyramids:
In addition to the original resolution, several progressively down-scaled resolutions (mipmaps) are stored.
This serves two purposes.
First, using mipmaps minimizes aliasing effects when rendering a zoomed-out view of the dataset~\cite{Williams:1983it}.
Second, and more importantly, using mipmaps reduces data access time and thus increases the perceived responsiveness for navigation.
Low-resolution mipmaps take up less memory and therefore load faster from disk.
New chunks of data must be loaded when the user browses to a part of the dataset that is not currently cached in memory.
In this situation, \bdv can rapidly load and render low-resolution data, filling in high resolution detail later as it becomes available.
This multi-resolution pyramid scheme is illustrated in the following figure.
\\
\screenshotTikz{pyramidblocks}
\\
Each raw image volume is stored in multiple resolutions, the original resolution (left) and successively smaller, downsampled versions (right).
Each resolution is stored in a chunked representation, split into small 3D blocks.


Each level of the multi-resolution pyramid is stored as a \emph{chunked multi-dimensional array}.
Multi-dimensional arrays are the standard way of storing image data in HDF5.
The layout of multi-dimensional arrays on disk can be configured.  We use a \emph{chunked} layout which means that the 3D image volume is split into several chunks (smaller 3D blocks).
These chunks are stored individually in the HDF5 file, which optimizes performance for our use-case where fast random access to individual chunks is required.

Rendering a virtual slice requires data contained within a small subset of chunks.
Only chunks that touch the slice need to be loaded, as illustrated in the following Figure.
\\
\screenshotTikz{sliceblocks}
\\
When rendering a slice (schematically illustrated by the blue line) the data of only a small subset of blocks is required.
In the original resolution 5 blocks are required, while only 2, respectively 1 block is required for lower resolutions.
Therefore, less data needs to be loaded to render a low-resolution slice.
This allows low-resolution versions to be loaded and rendered rapidly.
High-resolution detail is filled in when the user stops browsing to view a certain slice for an extended period of time.

Each of these chunks, however, is loaded in full, although only a subset of voxels in each chunk is required to render the actual slice.
Loading the data in this way, aligned at chunk boundaries, gurantees optimal I/O performance.

All loaded chunks are cached in RAM.
During interactive navigation, subsequent slices typically intersect with a similar set of chunks because their pose has changed only moderately, \ie. cached data are re-used.
Only chunks that are not currently in the cache need to be loaded from disk, as illustrated in the following Figure.
\\
\screenshotTikz{cacheblocks}
\\
For rendering the slice indicated by the red line, only the red blocks need to be loaded.
The blue blocks are already cached from rendering the blue slice before.
Combined with the multi-resolution mipmap representation, this chunking and caching scheme allows for fluid interactive browsing of very large datasets.

The parameters of the mipmap and chunking scheme are specific to each dataset and they are fully configurable by the user.
In particular, when exporting images to the \bdv format, the following parameters are adjustable:
\begin{itemize}
  \item the number of mipmap levels,
  \item the subsampling factors in each dimension for each mipmap level,
  \item the chunk sizes in each dimension for each mipmap level.
\end{itemize}
\Bdv suggests sensible parameter settings, however, for particular applications and data properties a user may tweak these parameters for optimal performance.


%
\subsection{Exporting from ImageJ Stacks}
%----------------------------------------
You can export any dataset to \bdv format by opening it as a stack in Fiji and then selecting \menu{Plugins > BigDataViewer > Export Current Image as XML/HDF5} from the Fiji menu.
If the image has multiple channels, each channel will become one \emph{setup} in the exported dataset.
If the image has multiple frames, each frame will become one \emph{timepoint} in the exported dataset.
Of course, you may export from virtual stacks if your data is too big to fit into memory.

To get started, let's open one of the ImageJ sample images by \menu{File > Open Samples > T1 Head (2.4M, 16-bits)}.
Selecting \menu{Plugins > BigDataViewer > Export Current Image as XML/HDF5} brings up the following dialog.
\\
\screenshotB{export-stack.png}
\\
Parts (A) and (C) of the dialog are optional, so we will explain (B) and (D) first.

At the bottom of the dialog (D), the export path is defined.
Specify the path of the XML file to which you want to export the dataset.
The HDF5 file for the dataset will be placed into the same directory under the same name with extension ``.h5''.
If the \emph{``use deflate compression''} checkbox is enabled, the image data will be compressed using HDF5 built-in DEFLATE compression.
We recommend to use this option.
It will usually reduce the file size to about 50\% with respect to uncompressed image size.
The performance impact of decompression when browsing the dataset is negligible.

In part (B) of the dialog the \emph{value range} of the image must be specified.
\Bdv always stores images with 16-bit precision currently, while the image you want to export is not necessarily 16-bit.
The value range defines the minimum and maximum of the image you want to export.
This is mapped to the 16-bit range for export.
\Ie, the minimum of the value range will be mapped to the minimum of the unsigned 16-bit range (0).
The maximum of the value range will be mapped to the maximum of the unsigned 16-bit range (65535).
In the drop-down menu you can select one the following options to specify how the value range should be determined:
\begin{itemize}
  \item \emph{``Use ImageJ's current min/max setting''}. The minimum and maximum set in ImageJ's Brightness\&Contrast are used. Note, that image intensities outside that range will be clipped to the minimum or maximum, respectively.
  \item \emph{``Compute min/max of the (hyper-)stack''}. Compute the minimum and maximum of the stack and use these. Note, that this may take some time to compute because it requires to look at all pixels of the stack you want to export.
  \item \emph{``Use values specified below''}. Use the values specified in the \emph{Min} and \emph{Max} fields (B) of the export dialog. Note, that image intensities outside that range will be clipped to the minimum or maximum, respectively.
\end{itemize}

After you have specified the value range and selected and export path, press \emph{OK} to export the dataset.
Messages about the progress of the operation are displayed in the ImageJ Log window.
\\
\screenshotB{export-stack-log.png}
\\
When the export is done you can browse the dataset in the \bdv by opening the exported XML file.

The optional parts (A) and (C) of the export dialog provide further options to customize the export.
If the checkbox \emph{``manual mipmap setup''} is enabled, you can customize the multi-resolution mipmap pyramid which stores your image stacks.
You can specify the number of resolution levels used, and their respective down-scaling factors, as well as the chunk sizes into which each resolution level is subdivided.

The \emph{``Subsampling factors''} field specifies a comma-separated list of resolution levels, formatted as <latex>\texttt{\{level, \dots, level\}}</latex><wiki><tt>{level, ..., level}</tt></wiki>.
Each \texttt{level} is a list of subsampling factors in $X, Y, Z$, formatted as <latex>\texttt{\{x-scale, y-scale, z-scale\}}</latex><wiki><tt>{x-scale, y-scale, z-scale}</tt></wiki>.
For example consider the specification <latex>\texttt{\{\{1,1,1\}, \{2,2,1\}, \{4,4,2\}\}}</latex><wiki><tt>{{1,1,1}, {2,2,2}, {4,4,2}}</tt></wiki>.
This will create a resolution pyramid with three levels.
The first level is the full resolution -- it is scaled by factors $1$ in all axes.
The second level is down-scaled by factors $2, 2, 1$ in $X, Y, Z$ respectively.
So it has half the resolution in $X$ and $Y$, but full resolution in $Z$.
The third level has half the resolution of the second in all axes, \ie, it is down-scaled by factors $4, 4, 2$ in $X, Y, Z$ respectively.
Note, that you should always order levels by decreasing resolution like this.
Also note, that in the above example we down-scale by different factors in different axes.
One may want to do this if the resolution of the dataset is anisotropic.
Then it is advisable to first downscale only in the higher-resolved axes until approximately isotropic resolution is reached.

The \emph{``Hdf5 chunk sizes''} specifies the chunk sizes into which data on each resolution level is sub-divided.
This is again formatted as <latex>\texttt{\{level, \dots, level\}}</latex><wiki><tt>{level, ..., level}</tt></wiki>, with the same number of levels as supplied for the \emph{Subsampling factors}.
Each \texttt{level} is a list of sizes in $X, Y, Z$, formatted as <latex>\texttt{\{x-size, y-size, z-size\}}</latex><wiki><tt>{x-size, y-size, z-size}</tt></wiki>.
For example consider the specification <latex>\texttt{\{\{16,16,16\}, \{16,16,16\}, \{16,16,16\}\}}</latex><wiki><tt>{{16,16,16}, {16,16,16}, {16,16,16}}</tt></wiki>.
This will sub-divide each resolution level into chunks of $16\times 16\times 16$ pixels.

It is usually not recommended to specify subsampling factors and chunk sizes manually.
When browsing a dataset, the mipmap setup determines the loading speed and therefore the perceived speed of browsing to data that is not cached.
With \emph{``manual mipmap setup''} turned off, reasonable values will be determined automatically depending on the resolution and anisotropy of your dataset.

Finally, in part (C) of the export dialog, you may choose to split your dataset into multiple HDF5 files.
This is useful in particular for very large datasets.
For example when moving the data to a different computer, it may be cumbersome to have it sitting in a single 10TB file.
If the checkbox \emph{``split hdf5''} is enabled the dataset will be split into multiple HDF5 \emph{partition} files.
The dataset can be split along the \emph{timepoint} and \emph{setup} dimensions.
Specify the number of \emph{timepoints per partition} and \emph{setups per partition} in the respective input fields.

For example, assume your dataset has 4 setups and 10 timepoints.
Setting \emph{timepoints per partition = 5} and \emph{setups per partition = 2} will result in 4 HDF5 partitions:
\begin{itemize}
  \item setups 1 and 2 of timepoints 1 through 5,
  \item setups 3 and 4 of timepoints 1 through 5,
  \item setups 1 and 2 of timepoints 6 through 10, and
  \item setups 3 and 4 of timepoints 6 through 10.
\end{itemize}
Setting \emph{timepoints per partition = 0} or \emph{setups per partition = 0} means that the dataset is not split in the respective dimension.

Note, that splitting into multiple HDF5 files is transparent from the viewer side.
There is still only one XML file that gathers all the partitions files into one dataset.

%
\subsection{Integration with Fiji's SPIMage Processing Tools}
%------------------------------------------------------------

\Bdv seamlessly integrates with the ``Multiview Reconstruction'' plugins that Fiji offers for registration and reconstruction of lightsheet microscopy data.
Recent versions of these tools build on the same XML format as \bdv itself.
In addition to HDF5, ``Multiview reconstruction'' supports a backend for datasets that store individual views as TIFF files,
because unprocessed data from lightsheet microscopes is often available in this format.

In principle, \bdv is able to display a TIFF dataset as is.
However, for quick navigation this is not the ideal format:
When navigating to a new timepoint, \bdv needs to load all TIFF files of that timepoint into memory, suffering a delay of tens of seconcds.
Therefore, it is beneficial to convert the TIFF dataset to HDF5.
Note, that this can be one at any point of the processing pipeline (\ie, before registration, after registration, after multiview fusion or deconvolution, \etc)
because the ``Multiview reconstruction'' plugins can operate on HDF5 datasets as well.

<latex>
A discussion of the ``Multiview reconstruction'' plugins is beyond the scope of this document.
We assume that the user has already created an XML/TIFF dataset, and refer to the description on the Fiji wiki,
  \href{http://fiji.sc/Multiview-Reconstruction}{\url{http://fiji.sc/Multiview-Reconstruction}}, for details.
</latex><wiki>
We assume that the user has already created an XML/TIFF dataset, as explained in [[Multiview-Reconstruction]].
</wiki>

To convert the dataset to HDF5, select \menu{Plugins > Multiview Reconstruction > Resave > As HDF5} form the Fiji menu.
This brings up the following dialog.
\\
\screenshotB{mvr-export1b.png}
\\
At the top of the dialog, select the XML file of the dataset you want to convert to HDF5.
In the lower part of the dialog, you can select which parts of the dataset you want to resave.
For example, assume that the source dataset contains the raw image stacks from the microscope, as well as deconvolved versions.
You might decide that you do not need the raw data as HDF5, so you can select only the deconvolved channels.
Once you have determined what you want to convert press \emph{OK}.

This brings up the next dialog, in which you need to specify the export path and options.
\\
\screenshotB{mvr-export2.png}
\\
These parameters are the same as discussed in the previous section:
If you want to specify custom mipmap settings, you can do so in the top part of the dialog.
Below that, choose whether you want to compress the image data.
For the export path, specify the XML file to which you want to export the dataset.
Press \emph{OK} to start the export.

%
\subsection{Cluster processing of XML/HDF5 SPIM data}
%----------------------------------------------------

Converting very large datasets into the HDF5 file format adds a significant overhead.
In order to speed up this conversion, we have developed a image processing pipeline that enables parallelisation of the process on a HPC cluster.

We have documented the steps and software needed to execute the above mentioned Fiji plugin on a cluster computer on
<latex>\href{http://fiji.sc/SPIM_Registration_on_cluster}{\coloredlink{the Fiji wiki}}</latex>
<wiki>this [[SPIM_Registration_on_cluster|'''wiki page''']]</wiki>
. Specifically, the sections
<latex>\href{http://fiji.sc/SPIM_Registration_on_cluster#Define_XML}{\coloredlink{Define XML}}</latex>
<wiki>[[SPIM_Registration_on_cluster#Define_XML|'''Define XML''']]</wiki>
 and
<latex>\href{http://fiji.sc/SPIM_Registration_on_cluster#Re-save_as_HDF5}{\coloredlink{Re-save as HDF5}}</latex>
<wiki>[[SPIM_Registration_on_cluster#Re-save_as_HDF5|'''Re-save as HDF5''']]</wiki>
 deal with the conversion process.

First it is necessary to define an XML file that describes the parameters of the dataset (i.e. number of time points, angles, illuminations and channels).
Subsequently, we launch one HDF5 re-save job per time point which generates as many \texttt{.h5} files as there are time points.
An additional master \texttt{.h5} file and the updated XML allow seamless navigation of such dataset with BigDataViewer.
Since the dataset will typically reside on a cluster filesystem without a graphical user interface, it is advisable to register it with the
<latex>\href{http://fiji.sc/BigDataServer}{\coloredlink{BigDataServer}}</latex>
<wiki>[[BigDataServer|'''BigDataServer''']]</wiki>
 and examine it remotely.
All subsequent processing steps of SPIM registration only modify the XML and thus it is necessary to perform the re-saving only once, usually as the first step of the SPIMage processing pipeline.

A multi-view dataset consisting of 715 six angle time points (altogether 2.1 Terabytes) converts to HDF5 with compression in 65 minutes using about 200 processors working in parallel.

<latex>
\newpage

\bibliographystyle{plain}
\bibliography{references}

\end{document}
</latex>
% vim::set expandtab tabstop=4 softtabstop=2 shiftwidth=2 ft=tex: