Research News

This sections contains quick summaries about selected research articles.


Multi-Scale Segmentation and Surface Fitting for Measuring 3D Macular Holes

Selected photos from the paper below:

Click to go to full publication on IEEE Explore.

Abstract

Macular holes are blinding conditions, where a hole develops in the central part of retina, resulting in reduced central vision. The prognosis and treatment options are related to a number of variables, including the macular hole size and shape. High-resolution spectral domain optical coherence tomography allows precise imaging of the macular hole geometry in three dimensions, but the measurement of these by human observers is time-consuming and prone to high inter- and intra-observer variability, being characteristically measured in 2-D rather than 3-D. We introduce several novel techniques to automatically retrieve accurate 3-D measurements of the macular hole, including: surface area, base area, base diameter, top area, top diameter, height, and minimum diameter. Specifically, we introduce a multi-scale 3-D level set segmentation approach based on a state-of-the-art level set method, and we introduce novel curvature-based cutting and 3-D measurement procedures. The algorithm is fully automatic, and we validate our extracted measurements both qualitatively and quantitatively, where our results show the method to be robust across a variety of scenarios. Our automated processes are considered a significant contribution for clinical applications.

Overview

Given a 3D OCT image of a Macular Hole, this paper presents a fast and accurate fully-automated 3D level set segmentation method.

The method automatically outputs several metrics and measurements using novel methods, and is shown to give state-of-the-art performance in comparrison to previous approaches.

Citation

Please cite the article with:

@article{nasrulloh2017multiscale,
  author   = {A. V. Nasrulloh and C. G. Willcocks and P. T. G. Jackson and C. Geenen and M. S. Habib and D. H. W. Steel and B. Obara},
  journal  = {IEEE Transactions on Medical Imaging},
  title    = {Multi-scale Segmentation and Surface Fitting for Measuring 3D Macular Holes},
  year     = {2018},
  month    = {2},
  volume   = {37},
  pages    = {580-589},
  doi      = {10.1109/TMI.2017.2767908},
  ISSN     = {0278-0062}
}
  


Extracting 3D Parametric Curves from 2D Images of Helical Objects

Chris Willcocks, Philip Jackson, Carl Nelson, and Boguslaw Obara

Click to go to full publication on IEEE Explore.

Abstract

Helical objects occur in medicine, biology, cosmetics, nanotechnology, and engineering. Extracting a 3D parametric curve from a 2D image of a helical object has many practical applications, in particular being able to extract metrics such as tortuosity, frequency, and pitch. We present a method that is able to straighten the image object and derive a robust 3D helical curve from peaks in the object boundary. The algorithm has a small number of stable parameters that require little tuning, and the curve is validated against both synthetic and real-world data. The results show that the extracted 3D curve comes within close Hausdorff distance to the ground truth, and has near identical tortuosity for helical objects with a circular profile. Parameter insensitivity and robustness against high levels of image noise are demonstrated thoroughly and quantitatively.

Overview

In this paper, we take a 2D image of a helical object and automatically fit a 3D curve to it. To do this, we first segment and straighten the helical object and then we find peaks on the boundaries of the helical object. The curve fits through these peaks.

Key Processes:

• Image segmentation
• Image straightening
• Curve fitting

Applications

The primary applications of this research are to retreive a 3D model of 2D images of helical objects, e.g. for collecting measurements, where 3D imaging is currently too expensive or impossible.

At a micro scale:

At a macro scale:

Additional output 3D metrics include:

• Tortuosity
• Radius
• Pitch
• Length

Areas for Future Research

To our knowledge, this is the first work that can automatically and robustly fit a 3D parametric curve to a single 2D image of a helical object. We believe there are several areas for improvement:

Code Usage

Our code is written in MATLAB. To use it, clone the github repository somewhere and add it to your MATLAB workspace, then simply edit the image, set the parameters and run the script main.m.

I = imread('leptospira.png');
if (size(I,3) > 1); I = rgb2gray(I); end;                                   fprintf('\tdone!\nSegmenting...');

% Adjust algorithm parameters (see paper for detailed explanations) straighten = true; sigma = 0.01; % smoothing amount, try 0.15 for 'licerasiae.png' d = max(size(I))/20; % minimum distance between peaks, set to 0 for images with lots of coils delta = 50; omega = max(size(I))/20; omicron = 30; push = true; % Run our algorithm steps [c,B] = part1_segment(I); fprintf('\t\tdone!\nStraightening... '); [Bp,T] = part2_straighten(B,c,delta,omega,omicron,straighten); fprintf('\tdone!\nFitting... '); [p] = part3_fitting(Bp,sigma,d,push); fprintf('\t\t\tdone!\nUndo transforms... '); [tp] = part4_undo_transforms(p,T,straighten); fprintf('\tdone!\n'); % Create our cubic spline (cs) through the transformed curve control points tp cs = cscvn(tp); cs = fnplotdense(cs); % We can also use the straight spline (ss) from control points p ss = cscvn(p); ss = fnplotdense(ss); % Example 3D metrics from the straight spline tortuosity = sum(sqrt(sum(diff(ss,1,2).^2,1))) / sqrt(sum((ss(:,end) - ss(:,1)).^2)); peaks = floor(length(p)/2);

Github:

Sources for this paper can be downloaded here. These are well-documented and reflect the paper's method.

Citation

Please cite our paper:

@article{willcocks-2016-extract-3d-curve,
  author   = {C. Willcocks and P. Jackson and C. Nelson and B. Obara},
  journal  = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  title    = {Extracting 3D Parametric Curves from 2D Images of Helical Objects},
  year     = {2016},
  volume   = {PP},
  number   = {99},
  pages    = {1-1},
  doi      = {10.1109/TPAMI.2016.2613866},
  ISSN     = {0162-8828},
  month    = {},
}
  


Sparse Volumetric Deformation

Selected photos from 2013 thesis below:

Click to go to full publication on Durham e-thesis Link.

Overview

This thesis was split into two sections: 1. my attempt to come up with a way to render large amounts of animated voxel models, and 2. my skeletonization work.

The renderer itself was somewhat unique and written mainly in CUDA with C++. Very briefly, the key ideas were using skeleton-mapped sparse voxel octrees for each animated model stored on disk, and paged through the memory-levels at resolutions of interest, then the renderer itself makes heavy use of instancing and top-down occlusion culling. Although it was reasonably efficient at rendering large scenes, the limitation was overly using parallel stream-compaction to discard unimportant chunks, and relying on forward transformations for the animation. It had a few key contributions, such as simple solutions to prevent holes/gaps with forward transformations by altering the rendered hierarchy (lowering the voxel resolution to fill gaps).

Future Work

I had more success using implicit surfaces, e.g. storing the distance transforms of 3D models in 3D texture memory at multiple resolutions. Animation is then achieved with traditional ray marching into the texture volumes (whose external bounds are defined by cube functions). This allows for cool tricks such as smooth implicit blending to get nice looking deformations, e.g. taking the smooth minimum between nearby/connected volumes. I started to investigate automatically splitting up models into smaller distance transform volumes for each animated bone, but sadly I wasn't able to publish these results into my thesis due to time constraints. For future researchers who stumble on this page with the goal of animating lots of volumetric content, I recommend looking into implicit surfaces above traditional sparse voxel octrees.

Citation

Please cite the thesis with:

@phdthesis{willcocks-2013-sparse-volumetric-deformation,
  title  = {Sparse Volumetric Deformation},
  author = {Chris G. Willcocks},
  year   = {2013},
  school = {Durham University},
  url    = {http://etheses.dur.ac.uk/8471/},
}
  


Feature-Varying Skeletonization

Intuitive control over the target feature size and output skeleton topology

Click to go to full publication on Springer Link.

Abstract

Current skeletonization algorithms strive to produce a single centered result which is homotopic and insensitive to surface noise. However, this traditional approach may not well capture the main parts of complex models, and may even produce poor results for applications such as animation. Instead, we approximate model topology through a target feature size ω, where undesired features smaller than ω are smoothed, and features larger than ω are retained into groups called bones. This relaxed feature-varying strategy allows applications to generate robust and meaningful results without requiring additional parameter tuning, even for damaged, noisy, complex, or high genus models.

Overview

Given an input dense 3D mesh, it presents an intuitive algorithm to iterateively contract the mesh into a skeleton.

Simply, you have two forces:

  1. You iteratively smooth the mesh, by moving the vertices to their average location in their original one-ring neighbourhood.

  2. You iterateively merge the mesh, by moving the vertices to the average position of any nearby vertices within some fixed Euclidean distance ω.

    • For very simple implementations, you can store vertex id's in a grid and fetch nearby grid cells; it works more-or-less the same.

The final force that contracts the mesh is a simple linear interpolation between 1. and 2. governed by the one-ring neighbourhood surface area.

While there are now better ways to generate curve skeletons, the nice little contribution with this at the time was its simplicity, and that it can make skeletons that are useful for animation with just a single parameter to adjust.

Key Advantages:

• Simplicity
• Speed
• Low parameters

It would be nice if this approach could be extended to the imaging domain; I briefly tried expressing similar concepts to the merging and smoothing with convolutions and got some interesting results, but I was unable to stop the skeleton overcontracting. The reason it stops overcontracting in the mesh geometry is because of perhaps an unexpected artifact whereby the vertices get pulled into groups > ω. This is also a limitation of the method, in that the vertices can contract in an unpredictable way impacting the skeleton quality. I haven't thought much about extending this.

Citation

Please cite the paper:

@article{willcocks-2012-feature-varying-skeletonization,
   author    = {Chris G. Willcocks and Frederick W. B. Li},
   title     = {Feature-varying skeletonization - Intuitive control over the target feature size and output skeleton topology},
   journal   = {The Visual Computer},
   volume    = {28},
   number    = {6-8},
   pages     = {775--785},
   year      = {2012},
   url       = {http://dx.doi.org/10.1007/s00371-012-0688-x},
   doi       = {10.1007/s00371-012-0688-x}
}

[return to top]