-
Notifications
You must be signed in to change notification settings - Fork 1
Getting Started
There are two types of TTool usage: TTool as an API and as a standalone program.
In order to use this project as an API or start developing from this project, please start from ttool.h. This is the header of the class ttool from which you can make a call to all the APIs.
In order to see how ttool.h is being used, we created a standalone program for this project. This executable file is main.cc. It is a good demonstration of how ttool is being used a called.
Essentially, this is how ttool is used in the main. Other parts of the main.cc are just the object for ttool standalone program.
#include "ttool.hh"
...
// ttool setup with `std::string` pathToTToolRootDir, `std::string` configFilePath and `std::string` calibFilePath
std::shared_ptr<ttool::TTool> ttool = std::make_shared<ttool::TTool>(pathToTToolRootDir, configFilePath, calibFilePath);
// detect tool head model
std::string label = ttool->Classify(image);
// change the object according to the label
int modelID = ttool->GetModelManager()->GetObjectID(label);
if (modelID != -1)
ttool->GetModelManager()->SetObjectID(modelID);
// changing the model to be tracked, adjusting the 6DoF pose of the model via `char` key.
ttool->ManipulateModel(key);
// Estimate the pose from `cv::Mat` image
ttool->RunOnAFrame(image);
// Getting the estimated pose
cv::Matx44f pose = ttool->GetPose();Now that the ttool runs alone, there might be no GLFW initialization/OpenGL context that the SLET tracking can run. Thus, in the main.cc, it calls intailization and termination before and after using ttool or using any OpenGL.
🚧 THIS APPLIES TO EVERTHING RELATED TO OPENGL, INCLUDING the ttool::standaloneUtils::Visualizer 🚧
#include "util.hh"
...
// Before intializing and using `ttool`
// Intiailizing GLFW
auto GLFWWindow = ttool::standaloneUtils::InitializeStandalone();
... // Using `ttool`, or anything related to OpenGL
// After finished using `ttool`
// Terminate GLFW
ttool::standaloneUtils::TerminateStandalone(GLFWWindow);Consequently, we can see the main.cc of the standalone program be the following
#include "ttool.hh"
#include "util.hh"
// Before intializing and using `ttool`
// Intiailizing GLFW
auto GLFWWindow = ttool::standaloneUtils::InitializeStandalone();
//======== BEGIN Using `ttool`, or anything related to OpenGL ========//
// ttool setup with `std::string` pathToTToolRootDir, `std::string` configFilePath and `std::string` calibFilePath
std::shared_ptr<ttool::TTool> ttool = std::make_shared<ttool::TTool>(pathToTToolRootDir, configFilePath, calibFilePath);
// detect tool head model
std::string label = ttool->Classify(image);
// change the object according to the label
int modelID = ttool->GetModelManager()->GetObjectID(label);
if (modelID != -1)
ttool->GetModelManager()->SetObjectID(modelID);
// changing the model to be tracked, adjusting the 6DoF pose of the model via `char` key.
ttool->ManipulateModel(key);
// Estimate the pose from `cv::Mat` image
ttool->RunOnAFrame(image);
// Getting the estimated pose
cv::Matx44f pose = ttool->GetPose();
//======== END Using `ttool`, or anything related to OpenGL ========//
// After finished using `ttool`
// Terminate GLFW
ttool::standaloneUtils::TerminateStandalone(GLFWWindow);There are other ttool::standaloneUtils that might be useful, but not vital for TTool API. This includes ttool::standaloneUtils::Camera and ttool::standaloneUtils::Visualizer.
ttool::standaloneUtils::Camera manages the camera and un-distort the frame with the calibration file.
ttool::standaloneUtils::Visualizer shows the user interface of the standalone program that shows all the keymap help, useful information and most importantly, visualize the 3D model to see the tracker.
How to initialize and use them can also be seen in the main.cc.
A calibration file contains the information of the camera device to be used with the TTool. This includes the camera image size, camera matrix and distortion coefficients of the camera. To obtain this file, please refer to this link. This is an example of how a calibration file looks like.
%YAML:1.0
---
image_width: 1280
image_height: 720
camera_matrix: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ 5.6160752462847097e+02, 0.,
6.4138172226013501e+02, 0.,
5.6291538638344332e+02, 3.4156306804983251e+02,
0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
rows: 1
cols: 5
dt: d
data: [ -2.5303888172190003e-01, 8.3509753739103201e-02,
3.7774747611573724e-05, 2.2045706561633034e-04,
-1.3650724983076233e-02 ]A configuration file contains the information about the View, SLET, Model and Augmented Carpentry configuration.
The comments indicate where a group of parameters belongs. All the path should be relative to the TTool root folder. (Behind the scene the class Config can receive the absolute path to the TTool directory and appends this path to all the path to make them absolute)
%YAML:1.0
---
# View configuration
zn: 9.9999997473787516e-05
zf: 4000.
# SLET configuration
histOffset: 100
histRad: 40
searchRad: 25
# Model configuration
groundTruthPoses:
- [ 9.84866619e-01, -4.03894819e-02, 1.68541461e-01, -1.62771076e-01,
1.18437737e-01, 9.79524612e-01, -5.95245212e-02, -9.92136419e-01,
1.10068895e-01, 0., 0., 1.39999986e-01 ]
- [ -5.42542815e-01, 8.39947045e-01, -1.11253709e-02, 5.06054997e-01,
3.16238910e-01, -8.02426398e-01, -6.70481861e-01, -4.40981954e-01,
-5.96643448e-01, -1.26680557e-03, 4.37926613e-02, 2.52720535e-01 ]
- [ 8.19440663e-01, 5.45331001e-01, -1.76395491e-01, 1.42224208e-01,
-4.91604596e-01, -8.59123707e-01, -5.55223823e-01, 6.78913355e-01,
-4.80406970e-01, 1.67723373e-02, 2.61279512e-02, 1.92897707e-01 ]
- [ 9.50834990e-01, 2.37247661e-01, -1.99040413e-01, 2.44926587e-02,
-6.98316872e-01, -7.15365469e-01, -3.08712870e-01, 6.75321281e-01,
-6.69801235e-01, -2.26506889e-02, 2.32661795e-02, 1.76990986e-01 ]
- [ -8.28246951e-01, 2.30366185e-01, 5.10807216e-01, 2.39123389e-01,
-6.79095149e-01, 6.94000125e-01, 5.06760836e-01, 6.96950197e-01,
5.07382751e-01, -2.56683957e-02, 2.72371043e-02, 1.72090665e-01 ]
- [ -8.07440519e-01, 4.48605478e-01, -3.83105516e-01, -5.85106611e-01,
-6.91857159e-01, 4.23044086e-01, -7.52737448e-02, 5.65754294e-01,
8.21124077e-01, -5.82594611e-03, 2.99568456e-02, 1.46538332e-01 ]
- [ -7.73640037e-01, 4.15423065e-01, -4.78431463e-01, -5.23223341e-01,
-8.44716668e-01, 1.12603128e-01, -3.57363552e-01, 3.37440640e-01,
8.70876431e-01, 0., 0., 2.20000029e-01 ]
- [ 1.33818597e-01, 8.97907972e-01, 4.19309229e-01, 9.80942905e-01,
-5.99305406e-02, -1.84715152e-01, -1.40733391e-01, 4.36035246e-01,
-8.88844311e-01, 2.64838710e-02, 1.45047512e-02, 1.02048650e-01 ]
- [ 1.85747266e-01, -4.76315729e-02, -9.81413007e-01, -4.00147825e-01,
9.08572972e-01, -1.19829834e-01, 8.97417665e-01, 4.14981604e-01,
1.49699569e-01, 6.66959509e-02, -1.05816480e-02, 7.78000727e-02 ]
- [ -5.42542815e-01, 8.39947045e-01, -1.11253709e-02, 5.06054997e-01,
3.16238910e-01, -8.02426398e-01, -6.70481861e-01, -4.40981954e-01,
-5.96643448e-01, -1.26680557e-03, 4.37926613e-02, 2.52720535e-01 ]
- [ 9.84866619e-01, -4.03894819e-02, 1.68541461e-01, -1.62771076e-01,
1.18437737e-01, 9.79524612e-01, -5.95245212e-02, -9.92136419e-01,
1.10068895e-01, 0., 0., 1.39999986e-01 ]
modelFiles:
- "/assets/toolheads/circular_saw_blade_makita_190/model.obj"
- "/assets/toolheads/chain_saw_blade_f_250/model.obj"
- "/assets/toolheads/auger_drill_bit_20_235/model.obj"
- "/assets/toolheads/twist_drill_bit_32_165/model.obj"
- "/assets/toolheads/self_feeding_bit_40_90/model.obj"
- "/assets/toolheads/self_feeding_bit_50_90/model.obj"
- "/assets/toolheads/saber_saw_blade_makita_t/model.obj"
- "/assets/toolheads/brad_point_drill_bit_20_150/model.obj"
- "/assets/toolheads/spade_drill_bit_25_150/model.obj"
# Augmented Carpentry
acitFiles:
- "/assets/toolheads/twist_drill_bit_32_165/metadata.acit"
- "/assets/toolheads/brad_point_drill_bit_20_150/metadata.acit"
- "/assets/toolheads/self_feeding_bit_40_90/metadata.acit"
- "/assets/toolheads/spade_drill_bit_25_150/metadata.acit"
- "/assets/toolheads/saber_saw_blade_makita_t/metadata.acit"
- "/assets/toolheads/auger_drill_bit_20_235/metadata.acit"
- "/assets/toolheads/self_feeding_bit_50_90/metadata.acit"
- "/assets/toolheads/chain_saw_blade_f_250/metadata.acit"
- "/assets/toolheads/circular_saw_blade_makita_190/metadata.acit"
The groundTruthPoses is in the format of 3x3 rotation matrix followed by a vector of a translation matrix. Notice that it is unlike normal pose convention where 6D pose matrix is [R|t], but instead...
{
R00, R01, R02,
R10, R11, R12,
R20, R21, R22,
T_x, T_y, T_z,
}