MILESTONE: public ready

ibois-epfl · Jan 10, 2024 · b222614 · b222614
1 parent b357383
commit b222614
Show file tree

Hide file tree

Showing 19 changed files with 68 additions and 340 deletions.
diff --git a/.gitignore b/.gitignore
@@ -67,6 +67,10 @@ include/divisible/*
 # Local 
 [Bb][Uu][Ii][Ll][Dd]/**/*
 
+#pose evaluation logging
+trackingPose.log
+trackingPose.txt
+
 
 ## >>>>>>>>> vscode >>>>>>>>>
 # gen

diff --git a/CITATION.cff b/CITATION.cff
@@ -0,0 +1,16 @@
+cff-version: 2.1.0
+message: "If you use this software, please cite it as below."
+authors:
+- family-names: "Settimi"
+  given-names: "Andrea"
+  orcid: "https://orcid.org/0000-0001-5020-7331"
+- family-names: "Naravich"
+  given-names: "Chutisilp"
+  orcid: "https://orcid.org/0009-0005-8782-732X"
+- family-names: "Nazgul"
+  given-names: "Zholmagambetova"
+title: "Software of TTool: a supervised AI-assisted Visual Pose Detector for AR Wood-working"
+version: 2.1.0
+doi: 10.5281/zenodo.10093230  # to replace
+date-released: 2024-01-10
+url: "https://github.com/ibois-epfl/TTool"
diff --git a/README.md b/README.md
@@ -7,16 +7,44 @@
     <img src="https://img.shields.io/badge/license-GPL--v3-brightgreen">
 </p>
 
-# TTool
-This is the repository hosting the API for TTool. TTool detects the 6dof of a fix toolhead from the feed of a fix camera view.
-The contributions of this project are
-- Making this program be both API and a standalone program.
-- Model manager that manages different 3D models for 6DoF tracking. This allows user to change the model to be tracked at ease.
-- Input pose to manipulate the 3D models for the initial pose. The initial pose is required by the SLET to begin the tracking. With input pose, the user can give a rough pose of the 3D, and have the SLET refine the pose afterward.
-- Config manager to read the config YAML and managing I/O for saving the poses of the models for later.
-- GLEW's OpenGL rendering. SLET is using QT5's OpenGL rendering. However, our [augmented carpentry](https://github.com/ibois-epfl/augmented-carpentry) is using GLFW. Hence, we changed the rendering back-end to suit our project.
-- RESET, FREEZE, TRACK implementation. We found the correlation between the average score of the search points and the quality of the tracking. We added a thresholding to the SLET to exploit this score. As a consequece, the tracking works better on our setting of a fixed camera mounting on the toolhead.
-- ML Tool head classification to detect the type of the tool from the camera.
-- 
-# Wiki
-The wiki of this project is hosted on [github](https://github.com/ibois-epfl/TTool/wiki)
+<p align="center">
+    <img src="vid/mosaic.gif">
+</p>
+
+🌲 `TTool` is developed at the [**Laboratory for Timber Construction**](https://www.epfl.ch/labs/ibois/) (director: Prof.Yves Weinand), at [**EPFL**](https://www.epfl.ch/en/), Lausanne, Switzerland. The project is part of the [**Augmented Carpentry Research**](https://www.epfl.ch/labs/ibois/augmented-carpentry/).
+
+🪚 `TTool` is an open-source **AI-powered and supervised 6DoF detector** for monocular camera. It is developed in C++ and for UNIX systems to allow **accurate end-effectors detection** during **wood-working operations** such as cutting, drilling, sawing and screwing with multiple tools. This is a fundamental component of any subtractive AR fabrication system since you can for instance, calculate and give users feedback on the correct orientation and depth to start and finish a hole or a cut.
+
+🖧 `TTool` is a AI-6DoF pose detector that recognizes automatically tools and allows the user to input an initial pose via an AR manipulator. The pose is then refined by a modified version of [SLET](://github.com/huanghone/SLET) (checkout our [changelog](docs/CHANGELOG.md)) and visualized as a projection onto the camera feed.
+
+
+↳ `TTool` can be imported as a C++ API in a third project or used as an executable. It is tailored to our specific use case in timber carpentry but see the [Caveats](#caveats) section below to adapt it to your use case.
+
+🚀 For a quick hands-on start or more details, check out our [Wiki](https://github.com/ibois-epfl/TSlam/wiki).
+
+<!-- ![Alt](https://repobeats.axiom.co/api/embed/1d0a652170a09d7569e8675428a319c6feb06866.svg "Repobeats analytics image") -->
+
+# How it works
+
+![Alt](./img/XX_fig_cutseuqence.png "Cut sequence for TTool")
+
+- `a`: the ML classifier detects the tool type from the camera feed and loads the corresponding 3D model.
+- `b`: the user inputs an initial pose of the tool via an AR manipulator.
+- `c`: the pose is refined with an edge-based algorithm.
+- `d`: the pose is projected onto the camera buffer and displayed to the user.
+- `e`: the user can now start the operation guided by computed feedback.
+
+![Alt](./img/XX_fig_UXaction_H.png "Show interface in action")
+
+On the left, the user can select the tool type and input an initial pose. On the right, the pose is refined and projected onto the camera feed. *The digital twin between the aligned model and the chainsaw plate (or any other tool) is preserved even when occuluded and inside the wood*.
+
+# Caveats
+TTool was tailored to our specific use case. If you want to adapt it to your use case, you will need to change the following files:
+- `CMakeLists.txt`: comment the following line, it won't use zenodo for the models, but you will have to provide the models yourself, se the wiki on how to do it.
+  https://github.com/ibois-epfl/TTool/blob/b357383e85708844efd9854291f0f85e521d227b/CMakeLists.txt#L12C1-L12C29
+- `assets/config.yaml`: list the models you want to use, and their path by replacing these lines:
+  https://github.com/ibois-epfl/TTool/blob/b357383e85708844efd9854291f0f85e521d227b/assets/config.yml#L57-L66
+  Be sure to erase these lines specific to our use case:
+  https://github.com/ibois-epfl/TTool/blob/b357383e85708844efd9854291f0f85e521d227b/assets/config.yml#L67-L76
+- `ML classifier`: to adapt the ML classifier to your use case, you will need to train your own model. We have a template [in this repo](https://github.com/ibois-epfl/TTool-ai).
+
diff --git a/TODO.md b/TODO.md
diff --git a/dev_log/2023.03.20/baby_step_viewer.gif b/dev_log/2023.03.20/baby_step_viewer.gif
diff --git a/dev_log/2023.03.27/Input.jpg b/dev_log/2023.03.27/Input.jpg
diff --git a/dev_log/2023.04.17/TSLET-integration.gif b/dev_log/2023.04.17/TSLET-integration.gif
diff --git a/dev_log/README.md b/dev_log/README.md
diff --git a/dev_log/main_readme/detect_track.gif b/dev_log/main_readme/detect_track.gif
diff --git a/dev_log/main_readme/input_track.gif b/dev_log/main_readme/input_track.gif
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
@@ -0,0 +1,7 @@
+Change Log to SLET in v.2.1.0:
+- Model manager that manages different 3D models for 6DoF tracking. This allows user to change the model to be tracked at ease.
+- Input pose to manipulate the 3D models for the initial pose. The initial pose is required by the SLET to begin the tracking. With input pose, the user can give a rough pose of the 3D, and have the SLET refine the pose afterward.
+- Config manager to read the config YAML and managing I/O for saving the poses of the models for later.
+- GLEW's OpenGL for rendering. SLET is using QT5's OpenGL rendering. However, our [augmented carpentry](https://github.com/ibois-epfl/augmented-carpentry) is using GLFW. Hence, we changed the rendering back-end to suit our project.
+- RESET, FREEZE, TRACK implementation. We found the correlation between the average score of the search points and the quality of the tracking. We added a thresholding to the SLET to exploit this score. As a consequece, the tracking works better on our setting of a fixed camera mounting on the toolhead.
+- ML Tool head classification to detect the type of the tool from the camera.
diff --git a/img/XX_fig_UXaction_H.png b/img/XX_fig_UXaction_H.png
diff --git a/img/XX_fig_cutseuqence.png b/img/XX_fig_cutseuqence.png
diff --git a/img/logo_linux_green.png b/img/logo_linux_green.png
diff --git a/img/logo_linux_green_original.png b/img/logo_linux_green_original.png
diff --git a/trackingPose.log b/trackingPose.log