15-clijpy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 15. clij on Colab\n",
    "\n",
    "One of the great advantages of Colab is the availability of GPUs. While that possibility is mainly exploited to run Tensorflow or PyTorch based deep learning algorithms, other software can benefit from it.\n",
    "\n",
    "[clij](https://clij.github.io/) is a library available on multiple platforms that implements a large collection of GPU-based image processing functions. Interestingly, while most GPU software is based on NVIDIA's CUDA library which limits its use to NVIDIA GPUs, clij is based on OpenCL, an open-source library for GPU computing. This makes GPU computing available to GPUs from virtually any manufacturer, and allows everyone to exploit that resource even on a laptop.\n",
    "\n",
    "clij is mainly centered on Fiji but is also available for other software such as Icy or Matlab. And there is also an experimental version for Python named [clijpy](https://github.com/clij/clijpy). In this notebooks we demonstrate how to use clij in a notebook and in particular on Colab where one can benefit from powerful GPUs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installing components\n",
    "\n",
    "We have seen in the previous notebook how it was possible to run Fiji and Python together thanks to PyImageJ. On a regular computer, this could be achieved by simply installing that package as well as Java via conda. This installation is more complicated on Colab which offers a more constrained workspace, e.g. where it is not very easy to use conda. The main part of the installation solution comes from [this post](https://forum.image.sc/t/pyimagej-on-google-colab/32804/8?u=guiwitz) on the image.sc forum and [this notebook](https://colab.research.google.com/drive/1zo7DZRDK5FJTfEj5gUe4iLl7V4_7r48T). **Note that you only have to do this installation when you start a runtime from scratch. If you installed the components, you can \"Runtime -> Restart runtime\" and they will still be available. In that case jump to the next [section](#clij-via-macro).**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing os to set environment variable\n",
    "import os\n",
    "\n",
    "# install openjdk for java and set ENV variable\n",
    "!apt-get install -y openjdk-8-jdk-headless -qq > /dev/null      #install openjdk\n",
    "os.environ[\"JAVA_HOME\"] = \"/usr/lib/jvm/java-8-openjdk-amd64\"     #set environment variable\n",
    "\n",
    "# install maven\n",
    "!sudo apt-get install maven\n",
    "\n",
    "# install imglyb\n",
    "!pip install imglyb==0.3.5\n",
    "\n",
    "# install pyimagej\n",
    "!pip install pyimagej\n",
    "\n",
    "# install pyjnius (python library to access java classes)\n",
    "!wget https://anaconda.org/conda-forge/pyjnius/1.2.0/download/linux-64/pyjnius-1.2.0-py36hc2cfd8d_0.tar.bz2 && tar -xjf pyjnius-1.2.0-py36hc2cfd8d_0.tar.bz2 -C /usr/local"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to be able to use clij, we need to specifically install that plugin into Fiji. A solution to do this is to locally install Fiji and then install plugins from the command line. First we download Fiji and unzip it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://downloads.imagej.net/fiji/latest/fiji-linux64.zip && unzip fiji-linux64.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we need to add the plugins. For this we first add the clij update sites (the addresses that one finds when manually adding plugins in Fiji). Then we update all plugins to install them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "cd Fiji.app/\n",
    "./ImageJ-linux64 --update add-update-site clij2 https://sites.imagej.net/clij2/\n",
    "./ImageJ-linux64 --update add-update-site clij https://sites.imagej.net/clij/\n",
    "./ImageJ-linux64 --update update"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## clij via macro\n",
    "\n",
    "Now that we have a local installation of Fiji and of PyImageJ, we can run the same kind of code as in the last notebook, i.e. we can create a Fiji-macro using clij and run it directly in the notebook. First we start Fiji and the WindowManager class. On Colab we also need to specify the location of Pyjnius to be able to access Java classes. **If you just performed the above installation, please \"Runtime -> Restart Runtime\" now.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"PYJNIUS_JAR\"] = \"/usr/local/share/pyjnius/pyjnius.jar\"\n",
    "\n",
    "import imagej;\n",
    "ij = imagej.init(\"Fiji.app\")\n",
    "\n",
    "from jnius import autoclass\n",
    "WindowManager = autoclass('ij.WindowManager')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, we use a very simple macro where we import an image and then filter it with a Gaussian. Only this time, the filtering is done via clij:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define the macro as text\n",
    "macro = \"\"\"\n",
    "#@ Float sigma\n",
    "run(\"Blobs (25K)\");\n",
    "input = getTitle();\n",
    "blurred = \"Blurred\";\n",
    "\n",
    "run(\"CLIJ Macro Extensions\", \"cl_device=\");\n",
    "Ext.CLIJ_clear();\n",
    "\n",
    "Ext.CLIJ_push(input);\n",
    "run(\"Close All\");\n",
    "\n",
    "Ext.CLIJ_blur2D(input, blurred, sigma, sigma);\n",
    "Ext.CLIJ_pull(blurred);\n",
    "\n",
    "Ext.CLIJ_clear();\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "args = {\n",
    "    'sigma': 20.0,\n",
    "}\n",
    "ij.py.run_macro(macro, args);\n",
    "\n",
    "# recover the current window containig the blurred image\n",
    "blurred = WindowManager.getCurrentImage()\n",
    "\n",
    "# plot it\n",
    "ij.py.show(blurred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using clijpy\n",
    "\n",
    "Instead of using a Fiji macro, we can directly use the clijpy library which has been designed to work as a direct Python interface for clij. Note that this demonstration is heavily based on [a notebook](https://github.com/clij/clijpy/blob/master/python/clijpy_demo.ipynb) available in the clijpy repository. First, we import the clij Java classes, and then get a Clij instance (similarly to what is done in the macro:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "CLIJx = autoclass('net.haesleinhuepf.clijx.CLIJx')\n",
    "clijx = CLIJx.getInstance();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can write the entire pipeline in Python. We only have to be careful about conversions of images between the Numpy and Java worlds. Remember that clijpy exploits PyImageJ and therefore we deal with the same concepts here. Let's first import an image as a numpy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from skimage import io\n",
    "import numpy as np\n",
    "\n",
    "np_arr = io.imread('https://samples.fiji.sc/blobs.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can convert that image to Java, push it to the GPU, create a \"target\" image in which the blurred version of the image will be placed, and do the filtering:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ij_img = ij.py.to_java(np_arr)\n",
    "\n",
    "# push the image to the GPU\n",
    "input8 = clijx.push(ij_img)\n",
    "\n",
    "# create a float image\n",
    "inputF = clijx.create(input8.getDimensions())\n",
    "\n",
    "# copy image to float image\n",
    "clijx.copy(input8, inputF)\n",
    "\n",
    "# reserve memory for output, same size and type as input\n",
    "blurred = clijx.create(inputF);\n",
    "\n",
    "# filter the image on the GPU\n",
    "clijx.blur(inputF, blurred, 5, 5, 0);\n",
    "\n",
    "# pull image out of GPU\n",
    "imageplus_result = clijx.pull(blurred);\n",
    "\n",
    "# display the result\n",
    "ij.py.show(imageplus_result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}