Feature detection, descriptors and matching

Detailed Description

Collaboration diagram for Feature detection, descriptors and matching:

Classes
class	mrpt::vision::CFeature
	A generic 2D feature from an image, extracted with CFeatureExtraction Each feature may have one or more descriptors (see descriptors), in addition to an image patch. More...
class	mrpt::vision::CFeatureList
	A list of visual features, to be used as output by detectors, as input/output by trackers, etc. More...
class	mrpt::vision::CMatchedFeatureList
	A list of features. More...
class	mrpt::vision::CFeatureExtraction
	The central class from which images can be analyzed in search of different kinds of interest points and descriptors computed for them. More...
struct	mrpt::vision::TKeyPoint_templ< PIXEL_COORD_TYPE >
	Simple structure for image key points. More...
struct	mrpt::vision::TKeyPointTraits< FEATURE >
struct	mrpt::vision::TKeyPointTraits< TKeyPoint >
struct	mrpt::vision::TKeyPointTraits< TKeyPointf >
struct	mrpt::vision::TKeyPointList_templ< FEATURE >
	A list of image features using the structure TKeyPoint for each feature Users normally will use directly: TKeyPointList, TKeyPointfList. More...
struct	mrpt::vision::KeypointResponseSorter< FEATURE_LIST >
	A helper struct to sort keypoints by their response: It can be used with these types: More...
class	mrpt::vision::CFeatureListKDTree< FEAT >
	Helper class: KD-tree search class for vector<KeyPoint>: Call mark_as_outdated() to force rebuilding the kd-tree after modifying the linked feature list. More...

Typedefs
using	mrpt::vision::TKeyPoint = TKeyPoint_templ< mrpt::img::TPixelCoord >
	Simple structure for image key points. More...
using	mrpt::vision::TKeyPointf = TKeyPoint_templ< mrpt::img::TPixelCoordf >
	A version of TKeyPoint with subpixel precision. More...
using	mrpt::vision::TKeyPointList = TKeyPointList_templ< TKeyPoint >
	A list of image features using the structure TKeyPoint for each feature. More...
using	mrpt::vision::TKeyPointfList = TKeyPointList_templ< TKeyPointf >
	A list of image features using the structure TKeyPointf for each feature - capable of KD-tree computations. More...

Functions
template<class DESCRIPTOR_KDTREE >
size_t	mrpt::vision::find_descriptor_pairings (std::vector< std::vector< size_t >> *pairings_1_to_multi_2, std::vector< std::pair< size_t, size_t >> *pairings_1_to_2, const CFeatureList &feats_img1, const DESCRIPTOR_KDTREE &feats_img2_kdtree, const mrpt::vision::TDescriptorType descriptor=descSIFT, const size_t max_neighbors=4, const double max_relative_distance=1.2, const typename DESCRIPTOR_KDTREE::kdtree_t::DistanceType max_distance=std::numeric_limits< typename DESCRIPTOR_KDTREE::kdtree_t::DistanceType >::max())
	Search for pairings between two sets of visual descriptors (for now, only SURF and SIFT features are considered). More...

Static methods with low-level detector functionality
static void	mrpt::vision::CFeatureExtraction::detectFeatures_SSE2_FASTER9 (const mrpt::img::CImage &img, TKeyPointList &corners, const int threshold=20, bool append_to_list=false, uint8_t octave=0, std::vector< size_t > *out_feats_index_by_row=nullptr)
	A SSE2-optimized implementation of FASTER-9 (requires img to be grayscale). More...
static void	mrpt::vision::CFeatureExtraction::detectFeatures_SSE2_FASTER10 (const mrpt::img::CImage &img, TKeyPointList &corners, const int threshold=20, bool append_to_list=false, uint8_t octave=0, std::vector< size_t > *out_feats_index_by_row=nullptr)
	Just like detectFeatures_SSE2_FASTER9() for another version of the detector. More...
static void	mrpt::vision::CFeatureExtraction::detectFeatures_SSE2_FASTER12 (const mrpt::img::CImage &img, TKeyPointList &corners, const int threshold=20, bool append_to_list=false, uint8_t octave=0, std::vector< size_t > *out_feats_index_by_row=nullptr)
	Just like detectFeatures_SSE2_FASTER9() for another version of the detector. More...

Typedef Documentation

TKeyPoint

using mrpt::vision::TKeyPoint = typedef TKeyPoint_templ<mrpt::img::TPixelCoord>

Simple structure for image key points.

See also

TKeyPointf, CFeature, TKeyPointList

Definition at line 78 of file TKeyPoint.h.

TKeyPointf

using mrpt::vision::TKeyPointf = typedef TKeyPoint_templ<mrpt::img::TPixelCoordf>

A version of TKeyPoint with subpixel precision.

Definition at line 81 of file TKeyPoint.h.

TKeyPointfList

using mrpt::vision::TKeyPointfList = typedef TKeyPointList_templ<TKeyPointf>

A list of image features using the structure TKeyPointf for each feature - capable of KD-tree computations.

Definition at line 280 of file TKeyPoint.h.

TKeyPointList

using mrpt::vision::TKeyPointList = typedef TKeyPointList_templ<TKeyPoint>

A list of image features using the structure TKeyPoint for each feature.

• capable of KD-tree computations

Definition at line 276 of file TKeyPoint.h.

Function Documentation

detectFeatures_SSE2_FASTER10()

void CFeatureExtraction::detectFeatures_SSE2_FASTER10 ( const mrpt::img::CImage &  img, TKeyPointList &  corners, const int  threshold = 20, bool  append_to_list = false, uint8_t  octave = 0, std::vector< size_t > *  out_feats_index_by_row = nullptr  )

static

Just like detectFeatures_SSE2_FASTER9() for another version of the detector.

Definition at line 46 of file CFeatureExtraction_FASTER.cpp.

References ASSERT_, mrpt::vision::TKeyPointList_templ< FEATURE >::clear(), fast_corner_detect_10(), mrpt::img::SHALLOW_COPY, and THROW_EXCEPTION.

Referenced by mrpt::vision::CGenericFeatureTracker::internal_trackFeatures().

Here is the call graph for this function:

Here is the caller graph for this function:

detectFeatures_SSE2_FASTER12()

void CFeatureExtraction::detectFeatures_SSE2_FASTER12 ( const mrpt::img::CImage &  img, TKeyPointList &  corners, const int  threshold = 20, bool  append_to_list = false, uint8_t  octave = 0, std::vector< size_t > *  out_feats_index_by_row = nullptr  )

static

Just like detectFeatures_SSE2_FASTER9() for another version of the detector.

Definition at line 62 of file CFeatureExtraction_FASTER.cpp.

References ASSERT_, mrpt::vision::TKeyPointList_templ< FEATURE >::clear(), fast_corner_detect_12(), mrpt::img::SHALLOW_COPY, and THROW_EXCEPTION.

Referenced by mrpt::vision::CGenericFeatureTracker::internal_trackFeatures().

Here is the call graph for this function:

Here is the caller graph for this function:

detectFeatures_SSE2_FASTER9()

void CFeatureExtraction::detectFeatures_SSE2_FASTER9 ( const mrpt::img::CImage &  img, TKeyPointList &  corners, const int  threshold = 20, bool  append_to_list = false, uint8_t  octave = 0, std::vector< size_t > *  out_feats_index_by_row = nullptr  )

static

A SSE2-optimized implementation of FASTER-9 (requires img to be grayscale).

If SSE2 is not available, it gratefully falls back to a non-optimized version.

Only the pt.{x,y} fields are filled out for each feature: the rest of fields are left uninitialized and their content is undefined. Note that (x,y) are already scaled to the 0-level image coordinates if octave>0, by means of:

pt.x = detected.x << octave;
pt.y = detected.y << octave;

If append_to_list is true, the corners list is not cleared before adding the newly detected feats.

If a valid pointer is provided for out_feats_index_by_row, upon return you will find a vector with as many entries as rows in the image (the real number of rows, disregarding the value of octave). The number in each entry is the 0-based index (in corners) of the first feature that falls in that line of the image. This index can be used to fasten looking for correspondences.

Definition at line 30 of file CFeatureExtraction_FASTER.cpp.

References ASSERT_, mrpt::vision::TKeyPointList_templ< FEATURE >::clear(), fast_corner_detect_9(), mrpt::img::SHALLOW_COPY, and THROW_EXCEPTION.

Referenced by mrpt::vision::CGenericFeatureTracker::internal_trackFeatures().

Here is the call graph for this function:

Here is the caller graph for this function:

find_descriptor_pairings()

template<class DESCRIPTOR_KDTREE >

size_t mrpt::vision::find_descriptor_pairings	(	std::vector< std::vector< size_t >> *	pairings_1_to_multi_2,
		std::vector< std::pair< size_t, size_t >> *	pairings_1_to_2,
		const CFeatureList &	feats_img1,
		const DESCRIPTOR_KDTREE &	feats_img2_kdtree,
		const mrpt::vision::TDescriptorType	descriptor = descSIFT,
		const size_t	max_neighbors = 4,
		const double	max_relative_distance = 1.2,
		const typename DESCRIPTOR_KDTREE::kdtree_t::DistanceType	max_distance = std::numeric_limits< typename DESCRIPTOR_KDTREE::kdtree_t::DistanceType>::max()
	)

Search for pairings between two sets of visual descriptors (for now, only SURF and SIFT features are considered).

Pairings are returned in one of two formats (or both of them simultaneously), depending on the non-NULL output containers present in the call.

CFeatureList  feats1, feats2;
// Populate feature lists [...]

// Create kd-tree for SIFT features of "feats2":
TSIFTDescriptorsKDTreeIndex<double>  feats2_kdtree(feats2);

// Search correspondences:
std::vector<std::vector<size_t>>             pairings_1_to_multi_2;
std::vector<std::pair<size_t,size_t> > pairings_1_to_2;
mrpt::vision::find_descriptor_pairings(
   &pairings_1_to_multi_2,   // Can be set to nullptr if not needed
   &pairings_1_to_2,         // Can be set to nullptr if not needed
   feats1, feats2_kdtree,    // The two sets of features
   mrpt::vision::descSIFT    // Select descriptor to use
   // [further optional params]
   );

See also

TSIFTDescriptorsKDTreeIndex, TSURFDescriptorsKDTreeIndex

The expected data type of elements for the kd-tree

Definition at line 48 of file descriptor_pairing.h.

References ASSERT_, ASSERT_ABOVEEQ_, ASSERTMSG_, mrpt::vision::descSIFT, mrpt::vision::descSURF, min, MRPT_END, MRPT_START, mrpt::vision::CFeature::TDescriptors::SIFT, mrpt::vision::CFeatureList::size(), mrpt::vision::CFeature::TDescriptors::SURF, and THROW_EXCEPTION.

Here is the call graph for this function: