Detailed Description

Classes for computer vision, detectors, features, etc.

mrpt_vision_grp

Namespaces
	detail

	pinhole
	Functions related to pinhole camera models, point projections, etc.

	pnp
	Perspective n Point (PnP) Algorithms toolkit for MRPT mrpt_vision_grp.

Classes
class	CCamModel
	This class represent a pinhole camera model for Monocular SLAM and implements some associated Jacobians. More...

class	CDifodo
	This abstract class implements a method called "Difodo" to perform Visual odometry with range cameras. More...

class	CFeature
	A generic 2D feature from an image, extracted with CFeatureExtraction Each feature may have one or more descriptors (see descriptors), in addition to an image patch. More...

class	CFeatureExtraction
	The central class from which images can be analyzed in search of different kinds of interest points and descriptors computed for them. More...

class	CFeatureList
	A list of visual features, to be used as output by detectors, as input/output by trackers, etc. More...

class	CFeatureListKDTree
	Helper class: KD-tree search class for vector<KeyPoint>: Call mark_as_outdated() to force rebuilding the kd-tree after modifying the linked feature list. More...

struct	CFeatureTracker_KL
	Track a set of features from old_img -> new_img using sparse optimal flow (classic KL method). More...

struct	CGenericFeatureTracker
	A virtual interface for all feature trackers, implementing the part of feature tracking that is common to any specific tracker implementation. More...

class	CImagePyramid
	Holds and builds a pyramid of images: starting with an image at full resolution (octave=1), it builds a number of half-resolution images: octave=2 at 1/2 , octave=3 at 1/2^2, octave=N at 1/2^(N-1). More...

class	CMatchedFeatureList
	A list of features. More...

class	CStereoRectifyMap
	Use this class to rectify stereo images if the same distortion maps are reused over and over again. More...

class	CUndistortMap
	Use this class to undistort monocular images if the same distortion map is used over and over again. More...

class	CVideoFileWriter
	An output stream which takes a sequence of images and writes a video file in any of a given of compatible formats. More...

struct	JacData

struct	KeypointResponseSorter
	A helper struct to sort keypoints by their response: It can be used with these types: More...

struct	lm_stat_t

struct	TFeatureObservation
	One feature observation entry, used within sequences with TSequenceFeatureObservations. More...

struct	TImageCalibData
	Data associated to each image in the calibration process mrpt::vision::checkerBoardCameraCalibration (All the information can be left empty and will be filled up in the calibration method). More...

struct	TImageROI
	A structure for defining a ROI within an image. More...

struct	TImageStereoCalibData
	Data associated to each stereo image in the calibration process mrpt::vision::checkerBoardCameraCalibration (All the information can be left empty and will be filled up in the calibration method). More...

struct	TImageStereoCallbackData
	Params of the optional callback provided by the user. More...

struct	TMatchingOptions
	A structure containing options for the matching. More...

struct	TMultiResDescMatchOptions
	Struct containing the options when matching multi-resolution SIFT-like descriptors. More...

struct	TMultiResDescOptions
	Struct containing the options when computing the multi-resolution SIFT-like descriptors. More...

struct	TMultiResMatchingOutput
	Struct containing the output after matching multi-resolution SIFT-like descriptors. More...

struct	TRelativeFeaturePos
	One relative feature observation entry, used with some relative bundle-adjustment functions. More...

struct	TResidJacobElement
	Data associated to each observation in the Lev-Marq. More...

struct	TROI
	A structure for storing a 3D ROI. More...

struct	TSequenceFeatureObservations
	A complete sequence of observations of features from different camera frames (poses). More...

struct	TSIFTDescriptorsKDTreeIndex
	A kd-tree builder for sets of features with SIFT descriptors. More...

struct	TSimpleFeature_templ
	A simple structure for representing one image feature (without descriptor nor patch) - This is the template which allows you to select if pixels are represented as integers or floats. More...

struct	TSimpleFeatureList_templ
	A list of image features using the structure TSimpleFeature for each feature. More...

struct	TSimpleFeatureTraits

struct	TSimpleFeatureTraits< TSimpleFeature >

struct	TSimpleFeatureTraits< TSimpleFeaturef >

struct	TStereoCalibParams
	Input parameters for mrpt::vision::checkerBoardStereoCalibration. More...

struct	TStereoCalibResults
	Output results for mrpt::vision::checkerBoardStereoCalibration. More...

struct	TStereoSystemParams
	Parameters associated to a stereo system. More...

struct	TSURFDescriptorsKDTreeIndex
	A kd-tree builder for sets of features with SURF descriptors. More...

Typedefs
typedef std::map< std::string, TImageCalibData >	TCalibrationImageList
	A list of images, used in checkerBoardCameraCalibration. More...

typedef void(*	TSteroCalibCallbackFunctor) (const TImageStereoCallbackData &d, void *user_data)
	Prototype of optional user callback function. More...

typedef std::vector< TImageStereoCalibData >	TCalibrationStereoImageList
	A list of images, used in checkerBoardStereoCalibration. More...

typedef std::map< int, std::map< int, std::map< int, std::deque< std::pair< TFeatureID, double > > > > >	TQuantizationTable

typedef std::unique_ptr< CGenericFeatureTracker >	CGenericFeatureTrackerAutoPtr

typedef TSimpleFeature_templ< mrpt::utils::TPixelCoord >	TSimpleFeature
	A simple structure for representing one image feature (without descriptor nor patch). More...

typedef TSimpleFeature_templ< mrpt::utils::TPixelCoordf >	TSimpleFeaturef
	A version of TSimpleFeature with subpixel precision. More...

typedef TSimpleFeatureList_templ< TSimpleFeature >	TSimpleFeatureList
	A list of image features using the structure TSimpleFeature for each feature. More...

typedef TSimpleFeatureList_templ< TSimpleFeaturef >	TSimpleFeaturefList
	A list of image features using the structure TSimpleFeaturef for each feature - capable of KD-tree computations. More...

typedef uint64_t	TFeatureID
	Definition of a feature ID. More...

typedef uint64_t	TLandmarkID
	Unique IDs for landmarks. More...

typedef uint64_t	TCameraPoseID
	Unique IDs for camera frames (poses) More...

typedef mrpt::aligned_containers< TCameraPoseID, mrpt::poses::CPose3D >::map_t	TFramePosesMap
	A list of camera frames (6D poses) indexed by unique IDs. More...

typedef mrpt::aligned_containers< mrpt::poses::CPose3D >::vector_t	TFramePosesVec
	A list of camera frames (6D poses), which assumes indexes are unique, consecutive IDs. More...

typedef std::map< TLandmarkID, mrpt::math::TPoint3D >	TLandmarkLocationsMap
	A list of landmarks (3D points) indexed by unique IDs. More...

typedef std::vector< mrpt::math::TPoint3D >	TLandmarkLocationsVec
	A list of landmarks (3D points), which assumes indexes are unique, consecutive IDs. More...

typedef std::map< mrpt::vision::TFeatureID, TRelativeFeaturePos >	TRelativeFeaturePosMap
	An index of feature IDs and their relative locations. More...

typedef std::vector< mrpt::aligned_containers< TResidJacobElement >::vector_t >	TResidualJacobianList

Enumerations
enum	TListIdx { firstList = 0, secondList, bothLists }

enum	TFeatureType { featNotDefined = -1, featKLT = 0, featHarris, featBCD, featSIFT, featSURF, featBeacon, featFAST, featFASTER9, featFASTER10, featFASTER12, featORB, featAKAZE, featLSD }
	Types of features - This means that the point has been detected with this algorithm, which is independent of additional descriptors a feature may also have. More...

enum	TDescriptorType { descAny = 0, descSIFT = 1, descSURF = 2, descSpinImages = 4, descPolarImages = 8, descLogPolarImages = 16, descORB = 32, descBLD = 64, descLATCH = 128 }
	The bitwise OR combination of values of TDescriptorType are used in CFeatureExtraction::computeDescriptors to indicate which descriptors are to be computed for features. More...

enum	TFeatureTrackStatus { status_IDLE = 0, status_TRACKED = 5, status_OOB = 1, status_LOST = 10 }

Functions
bool	checkerBoardCameraCalibration (TCalibrationImageList &images, unsigned int check_size_x, unsigned int check_size_y, double check_squares_length_X_meters, double check_squares_length_Y_meters, mrpt::utils::TCamera &out_camera_params, bool normalize_image=true, double *out_MSE=nullptr, bool skipDrawDetectedImgs=false, bool useScaramuzzaAlternativeDetector=false)
	Performs a camera calibration (computation of projection and distortion parameters) from a sequence of captured images of a checkerboard. More...

bool	checkerBoardCameraCalibration (TCalibrationImageList &images, unsigned int check_size_x, unsigned int check_size_y, double check_squares_length_X_meters, double check_squares_length_Y_meters, mrpt::math::CMatrixDouble33 &intrinsicParams, std::vector< double > &distortionParams, bool normalize_image=true, double *out_MSE=nullptr, bool skipDrawDetectedImgs=false, bool useScaramuzzaAlternativeDetector=false)

bool	findChessboardCorners (const mrpt::utils::CImage &img, std::vector< mrpt::utils::TPixelCoordf > &cornerCoords, unsigned int check_size_x, unsigned int check_size_y, bool normalize_image=true, bool useScaramuzzaMethod=false)
	Look for the corners of a chessboard in the image using one of two different methods. More...

void	findMultipleChessboardsCorners (const mrpt::utils::CImage &img, std::vector< std::vector< mrpt::utils::TPixelCoordf >> &cornerCoords, unsigned int check_size_x, unsigned int check_size_y)
	Look for the corners of one or more chessboard/checkerboards in the image. More...

bool	checkerBoardStereoCalibration (TCalibrationStereoImageList &images, const TStereoCalibParams &params, TStereoCalibResults &out_results)
	Optimize the calibration parameters of a stereo camera or a RGB+D (Kinect) camera. More...

template<class DESCRIPTOR_KDTREE >
size_t	find_descriptor_pairings (std::vector< vector_size_t > pairings_1_to_multi_2, std::vector< std::pair< size_t, size_t >> pairings_1_to_2, const CFeatureList &feats_img1, const DESCRIPTOR_KDTREE &feats_img2_kdtree, const mrpt::vision::TDescriptorType descriptor=descSIFT, const size_t max_neighbors=4, const double max_relative_distance=1.2, const typename DESCRIPTOR_KDTREE::kdtree_t::DistanceType max_distance=std::numeric_limits< typename DESCRIPTOR_KDTREE::kdtree_t::DistanceType >::max())
	Search for pairings between two sets of visual descriptors (for now, only SURF and SIFT features are considered). More...

void	saveQTableToFile (const TQuantizationTable &qTable, const std::string &filename)

void	insertHashCoeffs (const CFeature::Ptr &feat, TQuantizationTable &qTable)

TMultiResMatchingOutput	relocalizeMultiDesc (const mrpt::utils::CImage &image, CFeatureList &baseList, CFeatureList &currentList, TQuantizationTable &qTable, const TMultiResDescOptions &desc_opts, const TMultiResDescMatchOptions &match_opts)

void	updateBaseList (CFeatureList &baseList, const CFeatureList &currentList, const std::vector< int > &idx)

void	checkScalesAndFindMore (CMatchedFeatureList &baseList, const CFeatureList &currentList, const mrpt::utils::CImage &currentImage, const TMultiResMatchingOutput &output, const TMultiResDescOptions &computeOpts, const TMultiResDescMatchOptions &matchOpts)

bool	computeGradient (const mrpt::utils::CImage &image, const unsigned int x, const unsigned int y, double &mag, double &ori)
	Computes the gradient of certain pixel within the image. More...

bool	computeMainOrientations (const mrpt::utils::CImage &image, const unsigned int x, const unsigned int y, const unsigned int patchSize, std::vector< double > &orientations, const double &sigma)
	Computes the main orientations (within 80% of the peak value of orientation histogram) of a certain point within an image (for using in SIFT-based algorithms) More...

void	interpolateHistEntry (std::vector< double > &hist, const double &cbin, const double &rbin, const double &obin, const double &mag, const int d, const int n)
	Inserts the orientation value of a certain pixel within the keypoint neighbourhood into the histogram of orientations. More...

void	computeHistogramOfOrientations (const mrpt::utils::CImage &image, const unsigned int x, const unsigned int y, const unsigned int patchSize, const double &orientation, std::vector< int32_t > &descriptor, const TMultiResDescOptions &opts, std::vector< int32_t > &hashCoeffs)
	Computes the SIFT-like descriptor of a certain point within an image at the base scale, i.e. More...

TMultiResMatchingOutput	matchMultiResolutionFeatures (const CFeatureList &list1, CFeatureList &list2, const mrpt::utils::CImage &rightImage, const TMultiResDescMatchOptions &matchOpts, const TMultiResDescOptions &computeOpts)
	Matches two CFeatureList containing mulit-resolution descriptors. More...

int	matchMultiResolutionFeatures (CMatchedFeatureList &mList1, CMatchedFeatureList &mList2, const mrpt::utils::CImage &leftImage, const mrpt::utils::CImage &rightImage, const TMultiResDescMatchOptions &matchOpts, const TMultiResDescOptions &computeOpts)
	Matches two CMatchedFeatureList containing mulit-resolution descriptors. More...

int	computeMoreDescriptors (const mrpt::utils::CImage &image, const CFeature::Ptr &inputFeat, CFeature::Ptr &outputFeat, const bool &lowerScales, const TMultiResDescOptions &opts)
	Computes more multi-resolution SIFT-like descriptors for a feature using its position in a new image. More...

void	setProperScales (const CFeature::Ptr &feat1, const CFeature::Ptr &feat2, int &firstScale, int &lastScale)
	Computes the initial and final scales where to look when finding a match between multi-resolution features. More...

void	computeMultiResolutionDescriptors (const mrpt::utils::CImage &imageLeft, const mrpt::utils::CImage &imageRight, CMatchedFeatureList &matchedFeats, const TMultiResDescOptions &opts)
	Computes the multi-resolution SIFT-like descriptor of a set of matched features. More...

bool	computeMultiResolutionDescriptors (const mrpt::utils::CImage &image, CFeature::Ptr &feat, const TMultiResDescOptions &opts)
	Computes the multi-resolution SIFT-like descriptor of a features. More...

std::vector< bool >	computeMultiResolutionDescriptors (const mrpt::utils::CImage &image, CFeatureList &list, const TMultiResDescOptions &opts)
	Computes the multi-resolution SIFT-like descriptor of a list of features. More...

void	computeMultiOrientations (const mrpt::utils::CImage &image, CFeatureList &list, const TMultiResDescOptions &opts)
	Computes the multi-resolution SIFT-like descriptor of a list of features. More...

void	checkTrackedFeatures (CFeatureList &leftList, CFeatureList &rightList, vision::TMatchingOptions options)
	Search for correspondences which are not in the same row and deletes them ... More...

void	filterBadCorrsByDistance (mrpt::utils::TMatchingPairList &list, unsigned int numberOfSigmas)
	Filter bad correspondences by distance ... More...

void	openCV_cross_correlation (const mrpt::utils::CImage &img, const mrpt::utils::CImage &patch_img, size_t &x_max, size_t &y_max, double &max_val, int x_search_ini=-1, int y_search_ini=-1, int x_search_size=-1, int y_search_size=-1)
	Computes the correlation between this image and another one, encapsulating the openCV function cvMatchTemplate This implementation reduced computation time. More...

void	flip (mrpt::utils::CImage &img)
	Invert an image using OpenCV function. More...

mrpt::math::TPoint3D	pixelTo3D (const mrpt::utils::TPixelCoordf &xy, const mrpt::math::CMatrixDouble33 &A)
	Extract a UNITARY 3D vector in the direction of a 3D point, given from its (x,y) pixels coordinates, and the camera intrinsic coordinates. More...

mrpt::math::CMatrixDouble33	buildIntrinsicParamsMatrix (const double focalLengthX, const double focalLengthY, const double centerX, const double centerY)
	Builds the intrinsic parameters matrix A from parameters: More...

mrpt::math::CMatrixDouble33	defaultIntrinsicParamsMatrix (unsigned int camIndex=0, unsigned int resolutionX=320, unsigned int resolutionY=240)
	Returns the stored, default intrinsic params matrix for a given camera: More...

void	deleteRepeatedFeats (CFeatureList &list)
	Explore the feature list and removes features which are in the same coordinates. More...

void	rowChecking (CFeatureList &leftList, CFeatureList &rightList, float threshold=1.0)
	Search for correspondences which are not in the same row and deletes them. More...

void	getDispersion (const CFeatureList &list, mrpt::math::CVectorFloat &std, mrpt::math::CVectorFloat &mean)
	Computes the dispersion of the features in the image. More...

double	computeMsd (const mrpt::utils::TMatchingPairList &list, const poses::CPose3D &Rt)
	Computes the mean squared distance between a set of 3D correspondences ... More...

void	cloudsToMatchedList (const mrpt::obs::CObservationVisualLandmarks &cloud1, const mrpt::obs::CObservationVisualLandmarks &cloud2, mrpt::utils::TMatchingPairList &outList)
	Transform two clouds of 3D points into a matched list of points ... More...

float	computeMainOrientation (const mrpt::utils::CImage &image, unsigned int x, unsigned int y)
	Computes the main orientation of a set of points with an image (for using in SIFT-based algorithms) More...

void	normalizeImage (const mrpt::utils::CImage &image, mrpt::utils::CImage &nimage)
	Normalizes the brigthness and contrast of an image by setting its mean value to zero and its standard deviation to unit. More...

size_t	matchFeatures (const CFeatureList &list1, const CFeatureList &list2, CMatchedFeatureList &matches, const TMatchingOptions &options=TMatchingOptions(), const TStereoSystemParams &params=TStereoSystemParams())
	Find the matches between two lists of features which must be of the same type. More...

void	generateMask (const CMatchedFeatureList &mList, mrpt::math::CMatrixBool &mask1, mrpt::math::CMatrixBool &mask2, int wSize=10)
	Calculates the Sum of Absolutes Differences (range [0,1]) between two patches. More...

double	computeSAD (const mrpt::utils::CImage &patch1, const mrpt::utils::CImage &patch2)
	Calculates the Sum of Absolutes Differences (range [0,1]) between two patches. More...

void	addFeaturesToImage (const mrpt::utils::CImage &inImg, const CFeatureList &theList, mrpt::utils::CImage &outImg)
	Draw rectangles around each of the features on a copy of the input image. More...

void	projectMatchedFeatures (const CMatchedFeatureList &matches, const mrpt::utils::TStereoCamera &stereo_camera, std::vector< mrpt::math::TPoint3D > &out_points)

void	projectMatchedFeatures (const CFeatureList &leftList, const CFeatureList &rightList, std::vector< mrpt::math::TPoint3D > &vP3D, const TStereoSystemParams &params=TStereoSystemParams())
	Computes the 3D position of a set of matched features from their coordinates in the images. More...

void	projectMatchedFeature (const CFeature::Ptr &leftFeat, const CFeature::Ptr &rightFeat, mrpt::math::TPoint3D &p3D, const TStereoSystemParams &params=TStereoSystemParams())
	Computes the 3D position of a particular matched feature. More...

void	projectMatchedFeatures (CMatchedFeatureList &mfList, const TStereoSystemParams &param, mrpt::maps::CLandmarksMap &landmarks)
	Project a list of matched features into the 3D space, using the provided parameters of the stereo system. More...

void	projectMatchedFeatures (CFeatureList &leftList, CFeatureList &rightList, const TStereoSystemParams &param, mrpt::maps::CLandmarksMap &landmarks)
	Project a pair of feature lists into the 3D space, using the provided options for the stereo system. More...

void	StereoObs2BRObs (const mrpt::obs::CObservationStereoImages &inObs, const std::vector< double > &sg, mrpt::obs::CObservationBearingRange &outObs)
	Converts a stereo images observation into a bearing and range observation. More...

void	StereoObs2BRObs (const CMatchedFeatureList &inMatches, const mrpt::math::CMatrixDouble33 &intrinsicParams, const double &baseline, const mrpt::poses::CPose3D &sensorPose, const std::vector< double > &sg, mrpt::obs::CObservationBearingRange &outObs)
	Converts a matched feature list into a bearing and range observation (some of the stereo camera system must be provided). More...

void	StereoObs2BRObs (const mrpt::obs::CObservationVisualLandmarks &inObs, mrpt::obs::CObservationBearingRange &outObs)
	Converts a CObservationVisualLandmarks into a bearing and range observation (without any covariances). More...

void	computeStereoRectificationMaps (const mrpt::utils::TCamera &cam1, const mrpt::utils::TCamera &cam2, const mrpt::poses::CPose3D &rightCameraPose, void outMap1x, void outMap1y, void outMap2x, void outMap2y)
	Computes a pair of x-and-y maps for stereo rectification from a pair of cameras and the relative pose of the second one wrt the first one. More...

template<bool POSES_ARE_INVERSE>
void	frameJac (const mrpt::utils::TCamera &camera_params, const mrpt::poses::CPose3D &cam_pose, const mrpt::math::TPoint3D &landmark_global, mrpt::math::CMatrixFixedNumeric< double, 2, 6 > &out_J)
	The projective camera 2x6 Jacobian $\frac{\partial h(f,p)}{\partial p}$ (wrt the 6D camera pose) More...

template<bool POSES_ARE_INVERSE>
void	pointJac (const mrpt::utils::TCamera &camera_params, const mrpt::poses::CPose3D &cam_pose, const mrpt::math::TPoint3D &landmark_global, mrpt::math::CMatrixFixedNumeric< double, 2, 3 > &out_J)
	Jacobians wrt the point. More...

template<bool POSES_ARE_INVERSE>
void	ba_compute_Jacobians (const TFramePosesVec &frame_poses, const TLandmarkLocationsVec &landmark_points, const mrpt::utils::TCamera &camera_params, mrpt::aligned_containers< JacData< 6, 3, 2 >>::vector_t &jac_data_vec, const size_t num_fix_frames, const size_t num_fix_points)

void	ba_build_gradient_Hessians (const TSequenceFeatureObservations &observations, const std::vector< std::array< double, 2 >> &residual_vec, const mrpt::aligned_containers< JacData< 6, 3, 2 >>::vector_t &jac_data_vec, mrpt::aligned_containers< mrpt::math::CMatrixFixedNumeric< double, 6, 6 >>::vector_t &U, mrpt::aligned_containers< CArrayDouble< 6 >>::vector_t &eps_frame, mrpt::aligned_containers< mrpt::math::CMatrixFixedNumeric< double, 3, 3 >>::vector_t &V, mrpt::aligned_containers< CArrayDouble< 3 >>::vector_t &eps_point, const size_t num_fix_frames, const size_t num_fix_points, const vector< double > *kernel_1st_deriv)
	Construct the BA linear system. More...

double	recompute_errors_and_Jacobians (const lm_stat_t &lm_stat, TResidualJacobianList &res_jac, bool use_robust_kernel, double kernel_param)

void	build_linear_system (const TResidualJacobianList &res_jac, const vector_size_t &var_indxs, Eigen::VectorXd &minus_g, Eigen::MatrixXd &H)

void	add_lm_increment (const Eigen::VectorXd &eps, const vector_size_t &var_indxs, lm_stat_t &new_lm_stat)

Bundle-Adjustment Auxiliary methods
void	ba_initial_estimate (const mrpt::vision::TSequenceFeatureObservations &observations, const mrpt::utils::TCamera &camera_params, mrpt::vision::TFramePosesVec &frame_poses, mrpt::vision::TLandmarkLocationsVec &landmark_points)
	Fills the frames & landmark points maps with an initial gross estimate from the sequence observations, so they can be fed to bundle adjustment methods. More...

void	ba_initial_estimate (const mrpt::vision::TSequenceFeatureObservations &observations, const mrpt::utils::TCamera &camera_params, mrpt::vision::TFramePosesMap &frame_poses, mrpt::vision::TLandmarkLocationsMap &landmark_points)

double	reprojectionResiduals (const mrpt::vision::TSequenceFeatureObservations &observations, const mrpt::utils::TCamera &camera_params, const mrpt::vision::TFramePosesVec &frame_poses, const mrpt::vision::TLandmarkLocationsVec &landmark_points, std::vector< std::array< double, 2 >> &out_residuals, const bool frame_poses_are_inverse, const bool use_robust_kernel=true, const double kernel_param=3.0, std::vector< double > *out_kernel_1st_deriv=nullptr)
	Compute reprojection error vector (used from within Bundle Adjustment methods, but can be used in general) See mrpt::vision::bundle_adj_full for a description of most parameters. More...

double	reprojectionResiduals (const mrpt::vision::TSequenceFeatureObservations &observations, const mrpt::utils::TCamera &camera_params, const mrpt::vision::TFramePosesMap &frame_poses, const mrpt::vision::TLandmarkLocationsMap &landmark_points, std::vector< std::array< double, 2 >> &out_residuals, const bool frame_poses_are_inverse, const bool use_robust_kernel=true, const double kernel_param=3.0, std::vector< double > *out_kernel_1st_deriv=nullptr)
	Compute reprojection error vector (used from within Bundle Adjustment methods, but can be used in general) See mrpt::vision::bundle_adj_full for a description of most parameters. More...

void	add_se3_deltas_to_frames (const mrpt::vision::TFramePosesVec &frame_poses, const mrpt::math::CVectorDouble &delta, const size_t delta_first_idx, const size_t delta_num_vals, mrpt::vision::TFramePosesVec &new_frame_poses, const size_t num_fix_frames)
	For each pose in the vector frame_poses, adds a "delta" increment to the manifold, with the "delta" given in the se(3) Lie algebra: More...

void	add_3d_deltas_to_points (const mrpt::vision::TLandmarkLocationsVec &landmark_points, const mrpt::math::CVectorDouble &delta, const size_t delta_first_idx, const size_t delta_num_vals, mrpt::vision::TLandmarkLocationsVec &new_landmark_points, const size_t num_fix_points)
	For each pose in the vector frame_poses, adds a "delta" increment to the manifold, with the "delta" given in the se(3) Lie algebra: More...

Bundle-Adjustment methods
using	TBundleAdjustmentFeedbackFunctor = std::function< void(const size_t cur_iter, const double cur_total_sq_error, const size_t max_iters, const mrpt::vision::TSequenceFeatureObservations &input_observations, const mrpt::vision::TFramePosesVec &current_frame_estimate, const mrpt::vision::TLandmarkLocationsVec &current_landmark_estimate)>
	A functor type for BA methods. More...

double	bundle_adj_full (const mrpt::vision::TSequenceFeatureObservations &observations, const mrpt::utils::TCamera &camera_params, mrpt::vision::TFramePosesVec &frame_poses, mrpt::vision::TLandmarkLocationsVec &landmark_points, const mrpt::utils::TParametersDouble &extra_params=mrpt::utils::TParametersDouble(), const mrpt::vision::TBundleAdjustmentFeedbackFunctor user_feedback=mrpt::vision::TBundleAdjustmentFeedbackFunctor())
	Sparse Levenberg-Marquart solution to bundle adjustment - optimizes all the camera frames & the landmark locations. More...

Typedef Documentation

◆ TBundleAdjustmentFeedbackFunctor

using mrpt::vision::TBundleAdjustmentFeedbackFunctor = typedef std::function<void( const size_t cur_iter, const double cur_total_sq_error, const size_t max_iters, const mrpt::vision::TSequenceFeatureObservations& input_observations, const mrpt::vision::TFramePosesVec& current_frame_estimate, const mrpt::vision::TLandmarkLocationsVec& current_landmark_estimate)>

A functor type for BA methods.

See also: bundle_adj_full

Definition at line 40 of file bundle_adjustment.h.

◆ TResidualJacobianList

typedef std::vector<mrpt::aligned_containers<TResidJacobElement>::vector_t> mrpt::vision::TResidualJacobianList

Definition at line 73 of file chessboard_stereo_camera_calib_internal.h.

Enumeration Type Documentation

◆ TListIdx

enum mrpt::vision::TListIdx

Enumerator
firstList
secondList
bothLists

Definition at line 26 of file CFeature.h.

Function Documentation

◆ add_lm_increment()

void mrpt::vision::add_lm_increment	(	const Eigen::VectorXd &	eps,
		const vector_size_t &	var_indxs,
		lm_stat_t &	new_lm_stat
	)

Definition at line 868 of file chessboard_stereo_camera_calib.cpp.

References mrpt::poses::CPose3D::composeFrom(), eps, mrpt::vision::lm_stat_t::left_cam_params, mrpt::vision::lm_stat_t::left_cam_poses, mrpt::vision::lm_stat_t::right2left_pose, mrpt::vision::lm_stat_t::right_cam_params, and mrpt::vision::lm_stat_t::valid_image_pair_indices.

Referenced by checkerBoardStereoCalibration().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ ba_build_gradient_Hessians()

void mrpt::vision::ba_build_gradient_Hessians	(	const TSequenceFeatureObservations &	observations,
		const std::vector< std::array< double, 2 >> &	residual_vec,
		const mrpt::aligned_containers< JacData< 6, 3, 2 >>::vector_t &	jac_data_vec,
		mrpt::aligned_containers< mrpt::math::CMatrixFixedNumeric< double, 6, 6 >>::vector_t &	U,
		mrpt::aligned_containers< CArrayDouble< 6 >>::vector_t &	eps_frame,
		mrpt::aligned_containers< mrpt::math::CMatrixFixedNumeric< double, 3, 3 >>::vector_t &	V,
		mrpt::aligned_containers< CArrayDouble< 3 >>::vector_t &	eps_point,
		const size_t	num_fix_frames,
		const size_t	num_fix_points,
		const vector< double > *	kernel_1st_deriv
	)

Construct the BA linear system.

Set kernel_1st_deriv!=nullptr if using robust kernel.

Definition at line 238 of file ba_common.cpp.

References ASSERT_BELOW_, ASSERTDEB_, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::frame_id, mrpt::vision::TFeatureObservation::id_feature, mrpt::vision::TFeatureObservation::id_frame, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_frame, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_frame_valid, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_point, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_point_valid, MRPT_END, MRPT_START, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::point_id, and mrpt::math::UNINITIALIZED_MATRIX.

Referenced by bundle_adj_full().

Here is the caller graph for this function:

◆ ba_compute_Jacobians()

template<bool POSES_ARE_INVERSE>

void mrpt::vision::ba_compute_Jacobians	(	const TFramePosesVec &	frame_poses,
		const TLandmarkLocationsVec &	landmark_points,
		const mrpt::utils::TCamera &	camera_params,
		mrpt::aligned_containers< JacData< 6, 3, 2 >>::vector_t &	jac_data_vec,
		const size_t	num_fix_frames,
		const size_t	num_fix_points
	)

Definition at line 198 of file ba_internals.h.

References ASSERT_, ASSERTDEB_, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::frame_id, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_frame, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_frame_valid, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_point, mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::J_point_valid, MRPT_END, MRPT_START, and mrpt::vision::JacData< FrameDof, PointDof, ObsDim >::point_id.

◆ build_linear_system()

void mrpt::vision::build_linear_system	(	const TResidualJacobianList &	res_jac,
		const vector_size_t &	var_indxs,
		Eigen::VectorXd &	minus_g,
		Eigen::MatrixXd &	H
	)

Definition at line 754 of file chessboard_stereo_camera_calib.cpp.

References mrpt::vision::TResidJacobElement::J, mrpt::system::pause(), and mrpt::vision::TResidJacobElement::residual.

Referenced by checkerBoardStereoCalibration().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ frameJac()

template<bool POSES_ARE_INVERSE>

void mrpt::vision::frameJac	(	const mrpt::utils::TCamera &	camera_params,
		const mrpt::poses::CPose3D &	cam_pose,
		const mrpt::math::TPoint3D &	landmark_global,
		mrpt::math::CMatrixFixedNumeric< double, 2, 6 > &	out_J
	)

The projective camera 2x6 Jacobian $\frac{\partial h(f,p)}{\partial p}$ (wrt the 6D camera pose)

Note: Jacobians as described in http://www.mrpt.org/6D_poses:equivalences_compositions_and_uncertainty (Appendix A)

Definition at line 68 of file ba_internals.h.

References ASSERT_, mrpt::poses::CPose3D::composePoint(), mrpt::utils::TCamera::fx(), mrpt::utils::TCamera::fy(), mrpt::poses::CPose3D::getRotationMatrix(), mrpt::poses::CPose3D::inverseComposePoint(), mrpt::math::CMatrixFixedNumeric< T, NROWS, NCOLS >::loadFromArray(), R, mrpt::mrpt::math::square(), mrpt::math::square(), mrpt::poses::CPoseOrPoint< DERIVEDCLASS >::x(), mrpt::math::TPoint3D::x, mrpt::poses::CPoseOrPoint< DERIVEDCLASS >::y(), mrpt::math::TPoint3D::y, and mrpt::math::TPoint3D::z.

Here is the call graph for this function:

◆ pointJac()

template<bool POSES_ARE_INVERSE>

void mrpt::vision::pointJac	(	const mrpt::utils::TCamera &	camera_params,
		const mrpt::poses::CPose3D &	cam_pose,
		const mrpt::math::TPoint3D &	landmark_global,
		mrpt::math::CMatrixFixedNumeric< double, 2, 3 > &	out_J
	)

Jacobians wrt the point.

Note: Jacobians as described in http://www.mrpt.org/6D_poses:equivalences_compositions_and_uncertainty (Appendix A)

Definition at line 157 of file ba_internals.h.

References ASSERT_, mrpt::poses::CPose3D::composePoint(), mrpt::utils::TCamera::fx(), mrpt::utils::TCamera::fy(), mrpt::poses::CPose3D::inverseComposePoint(), mrpt::math::square(), mrpt::math::UNINITIALIZED_MATRIX, mrpt::math::TPoint3D::x, mrpt::math::TPoint3D::y, and mrpt::math::TPoint3D::z.

Here is the call graph for this function:

◆ recompute_errors_and_Jacobians()

double mrpt::vision::recompute_errors_and_Jacobians	(	const lm_stat_t &	lm_stat,
		TResidualJacobianList &	res_jac,
		bool	use_robust_kernel,
		double	kernel_param
	)