Detailed Description

A virtual interface for all feature trackers, implementing the part of feature tracking that is common to any specific tracker implementation.

This class provides a quite robust tracking of features, avoiding as many outliers as possible but not all of them: more robust tracking would require application-specific information and could be done in a number of very different approaches, so this class will not try to do any kind of RANSAC or any other advanced outlier rejection; instead, it should be done by the users or the classes that employ this class.

The basic usage of this class is as follows:

       CFeatureTracker_KL    tracker;  // Note: CFeatureTracker_KL is the
most robust implementation for now.
       tracker.extra_params["add_new_features"] = 1;  // Enable detection of
new features, not only tracking
       tracker.extra_params[...] = ...
       // ....
       CFeatureList     theFeats;  // The list of features
       mrpt::utils::CImage  previous_img, current_img;
       while (true) {
           current_img = ... // Grab new image.
           if ( previous_img_is_ok )
               tracker.trackFeatures(previous_img, current_img, theFeats);
           previous_img = current_img;
       }

Below follows the list of optional parameters for "extra_params" which can be set and will be understood by this base class for any specific tracker implementation. Note that all parameters are double's, but boolean flags are emulated by the values 0.0 (false) and 1.0 (true).

List of parameters:

Parameter name <td align="center"

Default value

Comments

add_new_features

0

If set to "1", the class will not only track existing features, but will also perform (after doing the actual tracking) an efficient search for new features with the FAST detector, and will add them to the passed "CFeatureList" if they fulfill a set of restrictions, as stablished by the other parameters (see add_new_feat_min_separation,add_new_feat_max_features,minimum_KLT_response_to_add).

add_new_feat_min_separation

15

If add_new_features==1, this is the minimum separation (in pixels) to any other (old, or new) feature for it being considered a candidate to be added.

desired_num_features_adapt

(img_width*img_height)/512

If add_new_features==1, the threshold of the FAST(ER) feature detector is dynamically adapted such as the number of raw FAST keypoints is around this number. This number should be much higher than the real desired numbre of features, since this one includes many features concentrated in space which are later discarded for the minimum distance.

desired_num_features

100

If add_new_features==1, the target number of the patch associated to each feature will be updated with every N'th frame.

add_new_feat_patch_size

11

If add_new_features==1, for each new added feature, this is the size of the patch to be extracted around the keypoint (set to 0 if patches are not required at all).

minimum_KLT_response_to_add

10

If add_new_features==1, this sets the minimum KLT response of candidate FAST features to be added in each frame, if they also fulfil the other restrictions (e.g. min.distance).

check_KLT_response_every

0

If >0, it will compute the KLT response at each feature point every N frames and those below minimum_KLT_response will be marked as "lost" in their "track_status" field.

minimum_KLT_response

5

See explanation of check_KLT_response_every.

KLT_response_half_win <td align="center"

4

When computing the KLT response of features (see minimum_KLT_response and minimum_KLT_response_to_add), the window centered at the point for its estimation will be of size (2*W+1)x(2*W+1), with W being this parameter value.

update_patches_every

0

If !=0, the patch associated to each feature will be updated with every N'th frame.

remove_lost_features

0

If !=0, out-of-bound features or those lost while tracking, will be automatically removed from the list of features. Otherwise, the user will have to manually remove them by checking the track_status field.

This class also offers a time profiler, disabled by default (see getProfiler and enableTimeLogger).

See also: CFeatureTracker_KL, the example application "track-video-features".

Definition at line 148 of file tracking.h.

#include <mrpt/vision/tracking.h>

Inheritance diagram for mrpt::vision::CGenericFeatureTracker:

Classes
struct	TExtraOutputInfo

Public Member Functions
	CGenericFeatureTracker ()
	Default ctor. More...

	CGenericFeatureTracker (mrpt::utils::TParametersDouble extraParams)
	Ctor with extra parameters. More...

virtual	~CGenericFeatureTracker ()
	Dtor. More...

void	trackFeatures (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeatureList &inout_featureList)
	Perform feature tracking from "old_img" to "new_img", with a (possibly empty) list of previously tracked features "inout_featureList". More...

void	trackFeatures (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeaturefList &inout_featureList)
	overload with subpixel precision More...

void	trackFeatures (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, CFeatureList &inout_featureList)
	overload This overload version uses the old (and much slower) CFeatureList More...

void	trackFeaturesNewList (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, const vision::CFeatureList &in_featureList, vision::CFeatureList &out_featureList)
	A wrapper around the basic trackFeatures() method, but keeping the original list of features unmodified and returns the tracked ones in a new list. More...

const mrpt::utils::CTimeLogger &	getProfiler () const
	Returns a read-only reference to the internal time logger. More...

mrpt::utils::CTimeLogger &	getProfiler ()
	Returns a reference to the internal time logger. More...

void	enableTimeLogger (bool enable=true)
	Returns a read-only reference to the internal time logger. More...

int	getDetectorAdaptiveThreshold () const
	Returns the current adaptive threshold used by the FAST(ER) detector to find out new features in empty areas. More...

Public Attributes
mrpt::utils::TParametersDouble	extra_params
	Optional list of extra parameters to the algorithm. More...

TExtraOutputInfo	last_execution_extra_info
	Updated with each call to trackFeatures() More...

Protected Member Functions
virtual void	trackFeatures_impl (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeaturefList &inout_featureList)
	The tracking method implementation, to be implemented in children classes. More...

virtual void	trackFeatures_impl (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeatureList &inout_featureList)=0
	The tracking method implementation, to be implemented in children classes. More...

virtual void	trackFeatures_impl (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, CFeatureList &inout_featureList)=0
	This version falls back to the version with TSimpleFeatureList if the derived class does not implement it. More...

void	updateAdaptiveNewFeatsThreshold (const size_t nNewlyDetectedFeats, const size_t desired_num_features)
	Adapts the threshold m_detector_adaptive_thres according to the real and desired number of features just detected. More...

Protected Attributes
mrpt::utils::CTimeLogger	m_timlog
	the internal time logger, disabled by default. More...

mrpt::vision::TSimpleFeatureList	m_newly_detected_feats
	This field is clared by trackFeatures() before calling trackFeatures_impl(), and can be filled out with newly defected FAST(ER) features in the latter. More...

Private Member Functions
template<typename FEATLIST >
void	internal_trackFeatures (const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, FEATLIST &inout_featureList)
	Perform feature tracking from "old_img" to "new_img", with a (possibly empty) list of previously tracked features "featureList". More...

Private Attributes
size_t	m_update_patches_counter
	for use when "update_patches_every">=1 More...

size_t	m_check_KLT_counter
	For use when "check_KLT_response_every">=1. More...

int	m_detector_adaptive_thres
	For use in "add_new_features" == true. More...

Constructor & Destructor Documentation

◆ CGenericFeatureTracker() [1/2]

mrpt::vision::CGenericFeatureTracker::CGenericFeatureTracker ( )

inline

Default ctor.

Definition at line 154 of file tracking.h.

◆ CGenericFeatureTracker() [2/2]

mrpt::vision::CGenericFeatureTracker::CGenericFeatureTracker ( mrpt::utils::TParametersDouble extraParams )

inline

Ctor with extra parameters.

Definition at line 162 of file tracking.h.

◆ ~CGenericFeatureTracker()

virtual mrpt::vision::CGenericFeatureTracker::~CGenericFeatureTracker ( )

inlinevirtual

Dtor.

Definition at line 171 of file tracking.h.

Member Function Documentation

◆ enableTimeLogger()

void mrpt::vision::CGenericFeatureTracker::enableTimeLogger ( bool enable = true )

inline

Returns a read-only reference to the internal time logger.

Definition at line 224 of file tracking.h.

References mrpt::utils::CTimeLogger::enable(), and m_timlog.

Here is the call graph for this function:

◆ getDetectorAdaptiveThreshold()

int mrpt::vision::CGenericFeatureTracker::getDetectorAdaptiveThreshold ( ) const

inline

Returns the current adaptive threshold used by the FAST(ER) detector to find out new features in empty areas.

Definition at line 231 of file tracking.h.

References m_detector_adaptive_thres.

◆ getProfiler() [1/2]

const mrpt::utils::CTimeLogger& mrpt::vision::CGenericFeatureTracker::getProfiler ( ) const

inline

Returns a read-only reference to the internal time logger.

Definition at line 217 of file tracking.h.

References m_timlog.

◆ getProfiler() [2/2]

mrpt::utils::CTimeLogger& mrpt::vision::CGenericFeatureTracker::getProfiler ( )

inline

Returns a reference to the internal time logger.

Definition at line 222 of file tracking.h.

References m_timlog.

◆ internal_trackFeatures()

template<typename FEATLIST >

void CGenericFeatureTracker::internal_trackFeatures	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		FEATLIST &	featureList
	)

private

Perform feature tracking from "old_img" to "new_img", with a (possibly empty) list of previously tracked features "featureList".

This is a list of parameters (in "extraParams") accepted by ALL implementations of feature tracker (see each derived class for more specific parameters).

"add_new_features" (Default=0). If set to "1", new features will be also added to the existing ones in areas of the image poor of features. This method actually first call the pure virtual "trackFeatures_impl" method, then implements the optional detection of new features if "add_new_features"!=0.

Definition at line 586 of file tracking.cpp.

References mrpt::vision::CFeatureExtraction::detectFeatures_SSE2_FASTER12(), mrpt::utils::FAST_REF_OR_CONVERT_TO_GRAY, mrpt::utils::CImage::getHeight(), mrpt::utils::CImage::getWidth(), mrpt::utils::CImage::KLT_response(), min, mrpt::math::square(), mrpt::vision::detail::trackFeatures_addNewFeats(), mrpt::vision::detail::trackFeatures_checkResponses(), mrpt::vision::detail::trackFeatures_deleteOOB(), and mrpt::vision::detail::trackFeatures_updatePatch().

Here is the call graph for this function:

◆ trackFeatures() [1/3]

void CGenericFeatureTracker::trackFeatures	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		TSimpleFeatureList &	inout_featureList
	)

Perform feature tracking from "old_img" to "new_img", with a (possibly empty) list of previously tracked features "inout_featureList".

This is a list of parameters (in "extraParams") accepted by ALL implementations of feature tracker (see each derived class for more specific parameters).

"add_new_features" (Default=0). If set to "1", new features will be also added to the existing ones in areas of the image poor of features. This method does:

Convert old and new images to grayscale, if they're in color.

Call the pure virtual "trackFeatures_impl" method.
Implement the optional detection of new features if "add_new_features"!=0.

Definition at line 789 of file tracking.cpp.

Referenced by trackFeaturesNewList().

Here is the caller graph for this function:

◆ trackFeatures() [2/3]

void CGenericFeatureTracker::trackFeatures	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		TSimpleFeaturefList &	inout_featureList
	)

overload with subpixel precision

Definition at line 796 of file tracking.cpp.

◆ trackFeatures() [3/3]

void CGenericFeatureTracker::trackFeatures	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		CFeatureList &	inout_featureList
	)

overload This overload version uses the old (and much slower) CFeatureList

Definition at line 783 of file tracking.cpp.

◆ trackFeatures_impl() [1/3]

void CGenericFeatureTracker::trackFeatures_impl	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		TSimpleFeaturefList &	inout_featureList
	)

protectedvirtual

The tracking method implementation, to be implemented in children classes.

Reimplemented in mrpt::vision::CFeatureTracker_KL.

Definition at line 563 of file tracking.cpp.

References MRPT_UNUSED_PARAM, and THROW_EXCEPTION.

◆ trackFeatures_impl() [2/3]

virtual void mrpt::vision::CGenericFeatureTracker::trackFeatures_impl	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		TSimpleFeatureList &	inout_featureList
	)

protectedpure virtual

The tracking method implementation, to be implemented in children classes.

Implemented in mrpt::vision::CFeatureTracker_KL.

◆ trackFeatures_impl() [3/3]

virtual void mrpt::vision::CGenericFeatureTracker::trackFeatures_impl	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		CFeatureList &	inout_featureList
	)

protectedpure virtual

This version falls back to the version with TSimpleFeatureList if the derived class does not implement it.

Implemented in mrpt::vision::CFeatureTracker_KL.

◆ trackFeaturesNewList()

void mrpt::vision::CGenericFeatureTracker::trackFeaturesNewList	(	const mrpt::utils::CImage &	old_img,
		const mrpt::utils::CImage &	new_img,
		const vision::CFeatureList &	in_featureList,
		vision::CFeatureList &	out_featureList
	)

inline

A wrapper around the basic trackFeatures() method, but keeping the original list of features unmodified and returns the tracked ones in a new list.

Definition at line 204 of file tracking.h.

References mrpt::vision::CFeatureList::begin(), mrpt::vision::CFeatureList::end(), and trackFeatures().

Here is the call graph for this function:

◆ updateAdaptiveNewFeatsThreshold()

void CGenericFeatureTracker::updateAdaptiveNewFeatsThreshold	(	const size_t	nNewlyDetectedFeats,
		const size_t	desired_num_features
	)

protected

Adapts the threshold m_detector_adaptive_thres according to the real and desired number of features just detected.

Definition at line 803 of file tracking.cpp.

References min.

Member Data Documentation

◆ extra_params

mrpt::utils::TParametersDouble mrpt::vision::CGenericFeatureTracker::extra_params

Optional list of extra parameters to the algorithm.

Definition at line 151 of file tracking.h.

◆ last_execution_extra_info

TExtraOutputInfo mrpt::vision::CGenericFeatureTracker::last_execution_extra_info

Updated with each call to trackFeatures()

Definition at line 246 of file tracking.h.

◆ m_check_KLT_counter

size_t mrpt::vision::CGenericFeatureTracker::m_check_KLT_counter

private

For use when "check_KLT_response_every">=1.

Definition at line 287 of file tracking.h.

◆ m_detector_adaptive_thres

int mrpt::vision::CGenericFeatureTracker::m_detector_adaptive_thres

private

For use in "add_new_features" == true.

Definition at line 289 of file tracking.h.

Referenced by getDetectorAdaptiveThreshold().

◆ m_newly_detected_feats

mrpt::vision::TSimpleFeatureList mrpt::vision::CGenericFeatureTracker::m_newly_detected_feats

protected

This field is clared by trackFeatures() before calling trackFeatures_impl(), and can be filled out with newly defected FAST(ER) features in the latter.

If it's not the case, feats will be computed anyway if the user enabled the "add_new_features" option.

Definition at line 276 of file tracking.h.

◆ m_timlog

mrpt::utils::CTimeLogger mrpt::vision::CGenericFeatureTracker::m_timlog

protected

the internal time logger, disabled by default.

Definition at line 268 of file tracking.h.

Referenced by enableTimeLogger(), and getProfiler().

◆ m_update_patches_counter

size_t mrpt::vision::CGenericFeatureTracker::m_update_patches_counter

private

for use when "update_patches_every">=1

Definition at line 285 of file tracking.h.

Detailed Description

Classes

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Constructor & Destructor Documentation

◆ CGenericFeatureTracker() [1/2]

◆ CGenericFeatureTracker() [2/2]

◆ ~CGenericFeatureTracker()

Member Function Documentation

◆ enableTimeLogger()

◆ getDetectorAdaptiveThreshold()

◆ getProfiler() [1/2]

◆ getProfiler() [2/2]

◆ internal_trackFeatures()

◆ trackFeatures() [1/3]

◆ trackFeatures() [2/3]

◆ trackFeatures() [3/3]

◆ trackFeatures_impl() [1/3]

◆ trackFeatures_impl() [2/3]

◆ trackFeatures_impl() [3/3]

◆ trackFeaturesNewList()

◆ updateAdaptiveNewFeatsThreshold()

Member Data Documentation

◆ extra_params

◆ last_execution_extra_info

◆ m_check_KLT_counter

◆ m_detector_adaptive_thres

◆ m_newly_detected_feats

◆ m_timlog

◆ m_update_patches_counter