Main MRPT website > C++ reference for MRPT 1.5.7
tracking.h
Go to the documentation of this file.
1 /* +---------------------------------------------------------------------------+
2  | Mobile Robot Programming Toolkit (MRPT) |
3  | http://www.mrpt.org/ |
4  | |
5  | Copyright (c) 2005-2017, Individual contributors, see AUTHORS file |
6  | See: http://www.mrpt.org/Authors - All rights reserved. |
7  | Released under BSD License. See details in http://www.mrpt.org/License |
8  +---------------------------------------------------------------------------+ */
9 
10 #ifndef mrpt_vision_tracking_H
11 #define mrpt_vision_tracking_H
12 
13 #include <mrpt/vision/types.h>
15 
16 #include <mrpt/vision/CFeature.h>
18 #include <mrpt/utils/CImage.h>
19 #include <mrpt/utils/CTimeLogger.h>
20 #include <mrpt/utils/TParameters.h>
21 
23 
24 #include <memory> // for auto_ptr, unique_ptr
25 
26 namespace mrpt
27 {
28  namespace vision
29  {
30  /** \addtogroup vision_tracking Feature detection and tracking
31  * \ingroup mrpt_vision_grp
32  * @{ */
33 
34  /** A virtual interface for all feature trackers, implementing the part of feature tracking that is common to any specific tracker implementation.
35  * This class provides a quite robust tracking of features, avoiding as many outliers as possible but not all of them:
36  * more robust tracking would require application-specific information and could be done in a number of very different approaches,
37  * so this class will not try to do any kind of RANSAC or any other advanced outlier rejection; instead, it should
38  * be done by the users or the classes that employ this class.
39  *
40  * The basic usage of this class is as follows:
41  * \code
42  * CFeatureTracker_KL tracker; // Note: CFeatureTracker_KL is the most robust implementation for now.
43  * tracker.extra_params["add_new_features"] = 1; // Enable detection of new features, not only tracking
44  * tracker.extra_params[...] = ...
45  * // ....
46  * CFeatureList theFeats; // The list of features
47  * mrpt::utils::CImage previous_img, current_img;
48  *
49  * while (true) {
50  * current_img = ... // Grab new image.
51  * if ( previous_img_is_ok )
52  * tracker.trackFeatures(previous_img, current_img, theFeats);
53  * previous_img = current_img;
54  * }
55  * \endcode
56  *
57  * Below follows the list of optional parameters for "extra_params" which can be set
58  * and will be understood by this base class for any specific tracker implementation.
59  * Note that all parameters are double's, but boolean flags are emulated by the values 0.0 (false) and 1.0 (true).
60  *
61  * List of parameters:
62  * <table border="1" >
63  * <tr><td align="center" > <b>Parameter name</b> </td> <td align="center" > <b>Default value</b> </td> <td align="center" > <b>Comments</b> </td> </tr>
64  * <tr><td align="center" > add_new_features </td> <td align="center" > 0 </td>
65  * <td> If set to "1", the class will not only track existing features, but will also perform (after doing the actual tracking) an efficient
66  * search for new features with the FAST detector, and will add them to the passed "CFeatureList" if they fulfill a set of restrictions,
67  * as stablished by the other parameters (see <i>add_new_feat_min_separation</i>,<i>add_new_feat_max_features</i>,<i>minimum_KLT_response_to_add</i>).
68  * </td> </tr>
69  * <tr><td align="center" > add_new_feat_min_separation </td> <td align="center" > 15 </td>
70  * <td> If <i>add_new_features</i>==1, this is the minimum separation (in pixels) to any other (old, or new) feature for it
71  * being considered a candidate to be added.
72  * </td> </tr>
73  * <tr><td align="center" > desired_num_features_adapt </td> <td align="center" > (img_width*img_height)/512 </td>
74  * <td> If <i>add_new_features</i>==1, the threshold of the FAST(ER) feature detector is dynamically adapted such as the number of
75  * raw FAST keypoints is around this number. This number should be much higher than the real desired numbre of features, since this
76  * one includes many features concentrated in space which are later discarded for the minimum distance.
77  * </td> </tr>
78  * <tr><td align="center" > desired_num_features </td> <td align="center" > 100 </td>
79  * <td> If <i>add_new_features</i>==1, the target number of the patch associated to each feature will be updated with every N'th frame. </td> </tr>
80  * <tr><td align="center" > add_new_feat_patch_size </td> <td align="center" > 11 </td>
81  * <td> If <i>add_new_features</i>==1, for each new added feature, this is the size of the patch to be extracted around the keypoint (set to 0 if patches are not required at all).
82  * </td> </tr>
83  * <tr><td align="center" > minimum_KLT_response_to_add </td> <td align="center" > 10 </td>
84  * <td> If <i>add_new_features</i>==1, this sets the minimum KLT response of candidate FAST features to be added in each frame, if they also fulfil the other restrictions (e.g. min.distance).
85  * </td> </tr>
86  * <tr><td align="center" > check_KLT_response_every </td> <td align="center" > 0 </td>
87  * <td> If >0, it will compute the KLT response at each feature point every <i>N</i> frames
88  * and those below <i>minimum_KLT_response</i> will be marked as "lost" in their "track_status" field.
89  * </td> </tr>
90  * <tr><td align="center" > minimum_KLT_response </td> <td align="center" > 5 </td>
91  * <td> See explanation of <i>check_KLT_response_every</i>.
92  * </td> </tr>
93  * <tr><td align="center" > KLT_response_half_win </td> <td align="center" > 4 </td>
94  * <td> When computing the KLT response of features (see <i>minimum_KLT_response</i> and <i>minimum_KLT_response_to_add</i>),
95  * the window centered at the point for its estimation will be of size (2*W+1)x(2*W+1), with <i>W</i> being this parameter value.
96  * </td> </tr>
97  * <tr><td align="center" > update_patches_every </td> <td align="center" > 0 </td>
98  * <td> If !=0, the patch associated to each feature will be updated with every N'th frame. </td> </tr>
99  * <tr><td align="center" > remove_lost_features </td> <td align="center" > 0 </td>
100  * <td> If !=0, out-of-bound features or those lost while tracking, will be automatically removed from the list of features.
101  * Otherwise, the user will have to manually remove them by checking the track_status field. </td> </tr>
102  * </table>
103  *
104  * This class also offers a time profiler, disabled by default (see getProfiler and enableTimeLogger).
105  *
106  * \sa CFeatureTracker_KL, the example application "track-video-features".
107  */
109  {
110  /** Optional list of extra parameters to the algorithm. */
112 
113  /** Default ctor */
114  inline CGenericFeatureTracker() : m_timlog(false), m_update_patches_counter(0),m_check_KLT_counter(0),m_detector_adaptive_thres(10)
115  { }
116  /** Ctor with extra parameters */
117  inline CGenericFeatureTracker(mrpt::utils::TParametersDouble extraParams) : extra_params(extraParams), m_timlog(false), m_update_patches_counter(0),m_check_KLT_counter(0),m_detector_adaptive_thres(10)
118  { }
119  /** Dtor */
121  { }
122 
123  /** Perform feature tracking from "old_img" to "new_img", with a (possibly empty) list of previously tracked features "inout_featureList".
124  * This is a list of parameters (in "extraParams") accepted by ALL implementations of feature tracker (see each derived class for more specific parameters).
125  * - "add_new_features" (Default=0). If set to "1", new features will be also added to the existing ones in areas of the image poor of features.
126  * This method does:
127  * - Convert old and new images to grayscale, if they're in color.
128  * - Call the pure virtual "trackFeatures_impl" method.
129  * - Implement the optional detection of new features if "add_new_features"!=0.
130  */
131  void trackFeatures(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeatureList &inout_featureList );
132 
133  /** overload with subpixel precision */
134  void trackFeatures(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeaturefList &inout_featureList );
135 
136  /** overload This overload version uses the old (and much slower) CFeatureList */
137  void trackFeatures(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,CFeatureList &inout_featureList );
138 
139  /** A wrapper around the basic trackFeatures() method, but keeping the original list of features unmodified and returns the tracked ones in a new list. */
140  inline void trackFeaturesNewList(
141  const mrpt::utils::CImage &old_img,
142  const mrpt::utils::CImage &new_img,
143  const vision::CFeatureList &in_featureList,
144  vision::CFeatureList &out_featureList
145  )
146  {
147  out_featureList = in_featureList;
148  std::for_each(
149  out_featureList.begin(),out_featureList.end(),
151  this->trackFeatures(old_img, new_img, out_featureList);
152  }
153 
154  /** Returns a read-only reference to the internal time logger */
155  inline const mrpt::utils::CTimeLogger & getProfiler() const { return m_timlog; }
156  /** Returns a reference to the internal time logger */
157  inline mrpt::utils::CTimeLogger & getProfiler() { return m_timlog; }
158 
159  /** Returns a read-only reference to the internal time logger */
160  inline void enableTimeLogger(bool enable=true) { m_timlog.enable(enable); }
161 
162  /** Returns the current adaptive threshold used by the FAST(ER) detector to find out new features in empty areas */
163  inline int getDetectorAdaptiveThreshold() const { return m_detector_adaptive_thres; }
164 
166  {
167  size_t raw_FAST_feats_detected; //!< In the new_img with the last adaptive threshold
168  size_t num_deleted_feats; //!< The number of features which were deleted due to OOB, bad tracking, etc... (only if "remove_lost_features" is enabled)
169  };
170 
171  TExtraOutputInfo last_execution_extra_info; //!< Updated with each call to trackFeatures()
172 
173  protected:
174  /** The tracking method implementation, to be implemented in children classes. */
175  virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeaturefList &inout_featureList );
176 
177  /** The tracking method implementation, to be implemented in children classes. */
178  virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeatureList &inout_featureList ) = 0;
179 
180  /** This version falls back to the version with TSimpleFeatureList if the derived class does not implement it. */
181  virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,CFeatureList &inout_featureList ) = 0;
182 
183  mrpt::utils::CTimeLogger m_timlog; //!< the internal time logger, disabled by default.
184 
185  /** This field is clared by \a trackFeatures() before calling \a trackFeatures_impl(), and
186  * can be filled out with newly defected FAST(ER) features in the latter.
187  * If it's not the case, feats will be computed anyway if the user enabled the "add_new_features" option.
188  */
190 
191  /** Adapts the threshold \a m_detector_adaptive_thres according to the real and desired number of features just detected */
192  void updateAdaptiveNewFeatsThreshold(
193  const size_t nNewlyDetectedFeats,
194  const size_t desired_num_features);
195 
196  private:
197  size_t m_update_patches_counter; //!< for use when "update_patches_every">=1
198  size_t m_check_KLT_counter; //!< For use when "check_KLT_response_every">=1
199  int m_detector_adaptive_thres; //!< For use in "add_new_features" == true
200 
201  template <typename FEATLIST>
202  void internal_trackFeatures(
203  const mrpt::utils::CImage &old_img,
204  const mrpt::utils::CImage &new_img,
205  FEATLIST &inout_featureList );
206  };
207 
208 #if MRPT_HAS_CXX11
209  typedef std::unique_ptr<CGenericFeatureTracker> CGenericFeatureTrackerAutoPtr;
210 #else
211  typedef std::auto_ptr<CGenericFeatureTracker> CGenericFeatureTrackerAutoPtr;
212 #endif
213 
214  /** Track a set of features from old_img -> new_img using sparse optimal flow (classic KL method).
215  *
216  * See CGenericFeatureTracker for a more detailed explanation on how to use this class.
217  *
218  * List of additional parameters in "extra_params" (apart from those in CGenericFeatureTracker) accepted by this class:
219  * - "window_width" (Default=15)
220  * - "window_height" (Default=15)
221  * - "LK_levels" (Default=3) Number of pyramids to build for LK tracking (this parameter only has effects when tracking with CImage's, not with CImagePyramid's).
222  * - "LK_max_iters" (Default=10) Max. number of iterations in LK tracking.
223  * - "LK_epsilon" (Default=0.1) Minimum epsilon step in interations of LK_tracking.
224  * - "LK_max_tracking_error" (Default=150.0) The maximum "tracking error" of LK tracking such as a feature is marked as "lost".
225  *
226  * \sa OpenCV's method cvCalcOpticalFlowPyrLK
227  */
229  {
230  /** Default ctor */
231  inline CFeatureTracker_KL() { }
232  /** Ctor with extra parameters */
234 
235  protected:
236  virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,vision::CFeatureList &inout_featureList ) MRPT_OVERRIDE;
237  virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeatureList &inout_featureList ) MRPT_OVERRIDE;
238  virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeaturefList &inout_featureList ) MRPT_OVERRIDE;
239 
240  private:
241  template <typename FEATLIST>
242  void trackFeatures_impl_templ(
243  const mrpt::utils::CImage &old_img,
244  const mrpt::utils::CImage &new_img,
245  FEATLIST &inout_featureList );
246 
247  };
248 
249 
250  /** Search for correspondences which are not in the same row and deletes them
251  * ...
252  */
253  void VISION_IMPEXP checkTrackedFeatures( CFeatureList &leftList,
254  CFeatureList &rightList,
255  vision::TMatchingOptions options);
256 
257 
258  /** Filter bad correspondences by distance
259  * ...
260  */
261  void VISION_IMPEXP filterBadCorrsByDistance( mrpt::utils::TMatchingPairList &list, // The list of correspondences
262  unsigned int numberOfSigmas ); // Threshold
263 
264 
265 
266  /** @} */ // end of grouping
267  }
268 }
269 
270 
271 #endif
void trackFeaturesNewList(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, const vision::CFeatureList &in_featureList, vision::CFeatureList &out_featureList)
A wrapper around the basic trackFeatures() method, but keeping the original list of features unmodifi...
Definition: tracking.h:140
An object for making smart pointers unique (ie, making copies if necessary), intended for being used ...
#define MRPT_OVERRIDE
C++11 "override" for virtuals:
A class for storing images as grayscale or RGB bitmaps.
Definition: CImage.h:101
mrpt::utils::CTimeLogger m_timlog
the internal time logger, disabled by default.
Definition: tracking.h:183
mrpt::utils::TParametersDouble extra_params
Optional list of extra parameters to the algorithm.
Definition: tracking.h:111
CFeatureTracker_KL(mrpt::utils::TParametersDouble extraParams)
Ctor with extra parameters.
Definition: tracking.h:233
int m_detector_adaptive_thres
For use in "add_new_features" == true.
Definition: tracking.h:199
std::auto_ptr< CGenericFeatureTracker > CGenericFeatureTrackerAutoPtr
Definition: tracking.h:211
void VISION_IMPEXP filterBadCorrsByDistance(mrpt::utils::TMatchingPairList &list, unsigned int numberOfSigmas)
Filter bad correspondences by distance ...
Definition: tracking.cpp:766
A list of TMatchingPair.
Definition: TMatchingPair.h:78
size_t raw_FAST_feats_detected
In the new_img with the last adaptive threshold.
Definition: tracking.h:167
A list of visual features, to be used as output by detectors, as input/output by trackers, etc.
Definition: CFeature.h:211
size_t num_deleted_feats
The number of features which were deleted due to OOB, bad tracking, etc... (only if "remove_lost_feat...
Definition: tracking.h:168
void VISION_IMPEXP checkTrackedFeatures(CFeatureList &leftList, CFeatureList &rightList, vision::TMatchingOptions options)
Search for correspondences which are not in the same row and deletes them ...
Definition: tracking.cpp:706
void enableTimeLogger(bool enable=true)
Returns a read-only reference to the internal time logger.
Definition: tracking.h:160
This is the global namespace for all Mobile Robot Programming Toolkit (MRPT) libraries.
Track a set of features from old_img -> new_img using sparse optimal flow (classic KL method)...
Definition: tracking.h:228
const mrpt::utils::CTimeLogger & getProfiler() const
Returns a read-only reference to the internal time logger.
Definition: tracking.h:155
A structure containing options for the matching.
mrpt::vision::TSimpleFeatureList m_newly_detected_feats
This field is clared by trackFeatures() before calling trackFeatures_impl(), and can be filled out wi...
Definition: tracking.h:189
A versatile "profiler" that logs the time spent within each pair of calls to enter(X)-leave(X), among other stats.
Definition: CTimeLogger.h:41
size_t m_check_KLT_counter
For use when "check_KLT_response_every">=1.
Definition: tracking.h:198
CFeatureTracker_KL()
Default ctor.
Definition: tracking.h:231
size_t m_update_patches_counter
for use when "update_patches_every">=1
Definition: tracking.h:197
TExtraOutputInfo last_execution_extra_info
Updated with each call to trackFeatures()
Definition: tracking.h:171
virtual ~CGenericFeatureTracker()
Dtor.
Definition: tracking.h:120
CGenericFeatureTracker()
Default ctor.
Definition: tracking.h:114
A virtual interface for all feature trackers, implementing the part of feature tracking that is commo...
Definition: tracking.h:108
mrpt::utils::CTimeLogger & getProfiler()
Returns a reference to the internal time logger.
Definition: tracking.h:157
int getDetectorAdaptiveThreshold() const
Returns the current adaptive threshold used by the FAST(ER) detector to find out new features in empt...
Definition: tracking.h:163
CGenericFeatureTracker(mrpt::utils::TParametersDouble extraParams)
Ctor with extra parameters.
Definition: tracking.h:117



Page generated by Doxygen 1.8.14 for MRPT 1.5.7 Git: 5902e14cc Wed Apr 24 15:04:01 2019 +0200 at lun oct 28 01:39:17 CET 2019