ORB Feature

Properties of Features

  1. Repeatability: same region found in different images
  2. Distinctiveness: distinguish different regions from one another
  3. Efficiency: feature points should be much less than num of image pixels
  4. Locality: only associate with local patch

Fast Key Points

A corner dectector: examines whether 12 consecutive points out of 16 points are brighter or dimmer than the reference pixel. More details. Since this feature often aggregates locally, non-maximal suppression is applied. In the original paper, the summation of intensity difference between reference point and query point is used. Wheares in the ORB, harris corner filter is applied to reject edges. conversion

Harris Corner

It’s desired that sum squared difference is large in all directions i.e. eigenvalue lambda 1 is roughly the same as lambda 2 for SSD distribution function. In the below equation, large R value is desirable rather than small (flat) or negative (edge). conversion

BRIEF Feature

BRIEF is a probablistic descriptor of a local patch, which randomly samples N pairs of (xi, yi) and (xj, yj) to compute a N bit descriptor with 1 indicates the former has larger intensity. Two descriptors are compared using hamming distance, which is the minimum steps needed to transform from 1 string to the other. For non-matching sequences, the distribution would look like a gaussian center at 128 (random guess). conversion

Oriented BREIF

BREIF has no rotational nor multi-scale invariance, to resolve this, orientation is computed from intensity centroid, and an image pyramid is applied. It’s interesting that orienting the feature to same direction actually damage the performance a lot – obviously the performance depends on random orientation of a feature. To recover this performance drop, a learning-based approach is adopted on pascal test patches to obtain good samples that could generate uncorrelated, high-variance descriptors i.e. descriptor of high discriminating power.

Remarks

  1. outdoor environment with varying light conditions can still pose challenge to low level descriptors like ORB, since it’s operating on per-pixel level after all.
  2. environment of repeative texture can be problematic as well
  3. it still surprise me that BREIF feature actually works: the order and distribution of the sampling could be very different for the same patch, but somehow such random sampling captures some underlying patterns that allow a feature to distinguish from one another. ORB makes more sense to me since it learnt a specific 256 pairs of samples which would be applied to all features.