Below is the original announcment with download links.- Andrew Collette Announcing HDF5 for Python (h5py) 1.0. What is h5py?-HDF5 for Python (h5py) is a general-purpose Python interface to the Hierarchical Data Format library, version 5. HDF5 is a versatile, mature scientific software library designed for the fast, flexible. Liberating Real-Time Data via HDF5: The Fastest Approach for Exposing Embedded Data for Analysis, Machine Learning, and Cloud-Enabled Services December 8, 2020 2020 HDF. HDF5DotNet wraps a subset of the HDF5 library API in a.NET assembly for consumption by.NET applications. The wrapper is written in C/CLI and uses the.NET P/Invoke mechanism to call native code from managed code which facilitates multi-language development in other.NET languages such as C#, VB.NET, and IronPython (or Windows PowerShell).
It is highly recommended that you use a pre-built version of h5py, either from aPython Distribution, an OS-specific package manager, or a pre-built wheel fromPyPI.
Be aware however that most pre-built versions lack MPI support, and that theyare built against a specific version of HDF5. If you require MPI support, ornewer HDF5 features, you will need to build from source.
After installing h5py, you should run the tests to be sure that everything wasinstalled correctly. This can be done in the python interpreter via:
Download HDF5 and zlib from ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4. Create a directory that will hold the installation libs and include files. For this demonstration we use /opt/cdo-install (make sure that the directory is created). Liberating Real-Time Data via HDF5: The Fastest Approach for Exposing Embedded Data for Analysis, Machine Learning, and Cloud-Enabled Services December 8, 2020 2020 HDF Users Group Recordings and Slide Decks.
Pre-built installation (recommended)¶
Pre-build h5py can be installed via many Python Distributions, OS-specificpackage managers, or via h5py wheels.
Python Distributions¶
If you do not already use a Python Distribution, we recommend eitherAnaconda/MinicondaorEnthought Canopy, both of whichsupport most versions of Microsoft Windows, OSX/MacOS, and a variety of LinuxDistributions. Installation of h5py can be done on the command line via:
for Anaconda/MiniConda, and via:
for Canopy.
Wheels¶
If you have an existing Python installation (e.g. a python.org download,or one that comes with your OS), then on Windows, MacOS/OSX, andLinux on Intel computers, pre-built h5py wheels can be installed via pip fromPyPI:
Additionally, for Windows users, Chris Gohlke provides third-party wheelswhich use Intel’s MKL.
OS-Specific Package Managers¶
On OSX/MacOS, h5py can be installed via Homebrew,Macports, or Fink.
The current state of h5py in various Linux Distributions can be seen athttps://pkgs.org/download/python-h5py, and can be installed via the packagemanager.
As far as the h5py developers know, none of the Windows package managers (e.g.Chocolatey, nuget)have h5py included, however they may assist in installing h5py’s requirementswhen building from source.
Source installation¶
To install h5py from source, you need three things installed:
- A supported Python version with development headers
- Cython >=0.29
- HDF5 1.8.4 or newer with development headers
- A C compiler
On Unix platforms, you also need pkg-config
unless you explicitly specifya path for HDF5 as described in Custom installation.
OS-specific instructions for installing HDF5, Python and a C compiler are in the next fewsections.
Additional Python-level requirements should be installed automatically (whichwill require an internet connection).
The actual installation of h5py should be done via:
or, from a tarball or git checkout
If you are working on a development version and the underlying cython files changeit may be necessary to force a full rebuild. The easiest way to achieve this is
from the top of your clone and then rebuilding.
Source installation on OSX/MacOS¶
HDF5 and Python are most likely in your package manager (e.g. Homebrew,Macports, or Fink).Be sure to install the development headers, as sometimes they are not includedin the main package.
XCode comes with a C compiler (clang), and your package manager will likely haveother C compilers for you to install.
Source installation on Linux/Other Unix¶
HDF5 and Python are most likely in your package manager. A C compiler almostdefinitely is, usually there is some kind of metapackage to install thedefault build tools, e.g. build-essential, which should be sufficient for ourneeds. Make sure that that you have the development headers, as they areusually not installed by default. They can usually be found in python-dev
orsimilar and libhdf5-dev
or similar.
Source installation on Windows¶
Installing from source on Windows is a much more difficult prospect thaninstalling from source on other OSs, as not only are you likely to need tocompile HDF5 from source, everything must be built with the correct version ofVisual Studio. Additional patches are also needed to HDF5 to get HDF5 and Pythonto work together.
We recommend examining the appveyor build scripts, and using those to build andinstall HDF5 and h5py.
Custom installation¶
Important
Remember that pip installs wheels by default.To perform a custom installation with pip, you should use:
or build from a git checkout or downloaded tarball to avoid gettinga pre-built version of h5py.
You can specify build options for h5py as environment variables when you buildit from source:
The supported build options are:
- To specify where to find HDF5, use one of these options:
HDF5_LIBDIR
andHDF5_INCLUDEDIR
: the directory containing thecompiled HDF5 libraries and the directory containing the C header files,respectively.HDF5_DIR
: a shortcut for common installations, a directory withlib
andinclude
subdirectories containing compiled libraries and C headers.HDF5_PKGCONFIG_NAME
: A name to querypkg-config
for.If none of these options are specified, h5py will querypkg-config
bydefault forhdf5
, orhdf5-openmpi
if building with MPI support.
HDF5_MPI=ON
to build with MPI integration - see Building against Parallel HDF5.HDF5_VERSION
to force a specified HDF5 version. In most cases, you don’tneed to set this; the version number will be detected from the HDF5 library.H5PY_SYSTEM_LZF=1
to build the bundled LZF compression filter(see Filter pipeline) against an external LZF library, rather thanusing the bundled LZF C code.
Building against Parallel HDF5¶
If you just want to build with mpicc
, and don’t care about using ParallelHDF5 features in h5py itself:
If you want access to the full Parallel HDF5 feature set in h5py(Parallel HDF5), you will further have to build in MPI mode. This can be doneby setting the HDF5_MPI
environment variable:
You will need a shared-library build of Parallel HDF5 as well, i.e. built with./configure--enable-shared--enable-parallel
.
HDF5 is a data format for storing extremely large and complex data collections. For more information see the official website http://hdf.ncsa.uiuc.edu/HDF5/. The plugin uses the jhdf5 library from ETH SIS for reading and writing HDF5 files.
Features
The HDF5 plugin for ImageJ and Fiji provides The following features:
- Loading 2D - 5D datasets
- Loading and combining mulitple 2d/3D datasets to 3D/4D/5D Hyperstacks
- Writing Hyperstacks to multiple 3D datasets
- scriptable load and save commands
Change Log
- Version 2014-08-27:
- Now displays 'element size [um]' in load dialog
- Version 2014-08-22:
- Complete rewrite of the plugin. Now based on the jhdf5 library from ETH SIS
- older versions of this plugin.
Requirements
Download Hdf5 For Mac Windows 10
- ImageJ, plugins tested with Version 1.38 and newer. Or Fiji
- Linux, Mac OS X, or Windows, 32bit or 64bit
Download and Install
Hdf5 Download Linux
For ImageJ: Download the plugin and the jhdf5 library and put both files into the plugin-folder of your ImageJ installation
- HDF5_Vibez.jar (version 2014-08-27)
- cisd-jhdf5-batteries_included_lin_win_mac.jar (version SNAPSHOT-r30323 from Bernd Rinn from 29.12.2013)
For Fiji:
- Click on 'Help--Update...' (the last entry in the help-menu) and wait until the popup appears
- Click on the 'Manage update sites' button
- Enable the 'HDF5' update site, and click the 'Close' button
- Click the 'Apply changes' button to install the plugin
- Restart Fiji
Updgrading from an older version. If you have an older version of this plugin installed. Please delete all files (esp. the platform dependend libraries), before installing the new one.
Example Data Sets
- pollen.h5 (3D confocal data of a pollen grain: 8 bit gray, approx. 16MB
- e098.h5 3D confocal raw data of a zebrafish embryo from our ViBE-Z project: 2 channels, 2 tiles, 2 laserintensities, and 2 recording directions, 8 bit gray, approx. 477MB)
- ViBE-Z_72hpf_v1.h5 Aligned Gene expression patterns from our ViBE-Z project: 8 bit gray, 4 anatomical chanels and 16 pattern channels., approx. 218MB
Usage
Load data sets
- Select 'File -- Import -- HDF5...'. The file selector will pop up. Double click the file you want to load.
- The 'Select data sets' dialog will open:
- select one or more datasets. Multiple selections can be done by
- mouseclick on first and Shift+mouseclick on last item
- mousedown on first and drag to last item
- CTRL+mouseclick to select / deselect individual items
- CTRL+A selects all items
- chose how they should be loaded or combined to a hyperstack.
- Load as ... individual stacks will create an individual window for each selected data set
- Load as ... individual hyperstacks (custom layout) will create a new hyperstack for each selected dataset. The data set layout has to be specified in the textfield below. HDF5 uses C-style / Java-style indexing of the array, i.e., the slowest changing dimension comes first (see size in the table). Typical storage orders are:
- 'yx': 2D image
- 'zyx': 3D image
- 'tyx': 2D movie
- 'tzyx': 3D movie
- 'cyx': 2D multi-channel image
- 'tczyx': 3D multi-channel move
- ...
- Combine to ... hyperstack (multichannel) loads the selected 2D/3D data sets and combines them to a multi-channel hyperstack
- Combine to ... hyperstack (time series) loads the selected 2D/3D data sets and combines them to a time-series hyperstack
- Combine to ... hyperstack (multichannel time series) loads the selected 2D/3D data sets and combines them to a multichannel time-series hyperstack. You have to specify the Number of channels of the resulting hyperstack. The number of time points is then determined from the number of selected data sets divided by the number of channels
Save data sets
- Select 'File -- Save As -- HDF5 (new or replace)...' to create a new HDF5 file or 'File -- Save As -- HDF5 (append)...' to append the dataset(s) to an existing HDF5 file. The file selector will pop up. Select the file name.
- The Save Dialog will open to select the data set layout
- Compression Level allow to select the compression of the data set. The compression is lossless, i.e. it works like a zip-archive. Possible compression levels are
- no compression,
- 1 (fastest, larger file)
- 2
- ...
- 9 (slowest, smallest file)
- Presets: allows to select presets for the data set layout. There is no official standard, how to name the datasets. For general purpose data we usually name it as '/t0/channel0', 't0/channel1', ... which is the 'Standard' Preset.
- Dataset Names Template specifies the template string for the data set names. The placeholders {t} and {c}> will be replaced for each timepoint/channel combination with the strings specified in the following two textfields.
- Replace {t} with: and Replace {c} with: specifies the enconding of time points and channels in the filename. Possible entries area printf-style format string or a list of strings (one entry per line), e.g.,
- %d for number style like 1,2,3,...
- %.03d for zero-padded-numbers with 3 digits: 001, 002, 003, ...
- nuclei
cellborder
pattern for named channels
- The Update Preview button shows the Resulting Mapping: of the hyperstack time points and channels to the HDF5 data set names
Internals
The HDF5 plugin saves and loads the pixel/voxel size in micrometer of the image in the attribute 'element_size_um'. It has always 3 components in the order z,y,x (accordingly to the c-style indexing). Other meta data is not saved/loaded
Source Code
The source Code is included in the .jar-File. Just use unzip to extract it.
Wish list for next version
- Support for single data sets with more than 2GB size (will require a slice-wise or block-wise loadin/saving)
- disable the Log Window
- load a sub cube of the data set (e.g. for large 5D arrays stored in a single dataset)
Other versions
See here for much older versions of this plugin.
Contact, Bug reports and Questions
Hdf5 Viewer Windows
We are always interested in bug reports and feedback. Please contact us via email to Olaf Ronneberger.
Olaf Ronneberger