Added feature to export the evaluation summary table in csv format #229

Drita-ai · 2024-10-06T06:38:50Z

This PR implements the feature to export the evaluation summary table in csv format to the outputs folder.

Usage : Run the code with python3 main.py -i ./samples/sample4 and you'll see the csv files outputted inside outputs/Evaluation

Fixes #221

Udayraj123

Thanks for the PR @Drita-ai! Please check my comments.

Udayraj123 · 2024-10-07T01:39:10Z

src/evaluation.py

+        evaluation_json = open_evaluation_with_validation(self.path)
+
+        if evaluation_json["options"].get("enable_evaluation_table_to_csv", False):


We should already have access to options in constructors. No need to load the json file again.

self.should_explain_scoring = options.get("should_explain_scoring", False) self.enable_evaluation_table_to_csv = options.get("enable_evaluation_table_to_csv", False)

Udayraj123 · 2024-10-07T01:40:23Z

src/evaluation.py

+    def conditionally_print_explanation(self, file_id):
        if self.should_explain_scoring:
            console.print(self.explanation_table, justify="center")

+        self.explanation_to_csv(file_id)
+        self.explanation_table_data_for_csv = []


Instead can we directory call a function conditionally_save_explanation_csv() from the parent? Wherever conditionally_print_explanation() is called currently?

Udayraj123 · 2024-10-07T01:42:07Z

src/evaluation.py

+
+            output_dir = os.path.join(
+                os.path.dirname(os.getcwd()),
+                f"OMRChecker/outputs/Evaluation/{processed_img_name}.csv",


OMRChecker works on multiple directories actually, so can't use hardcoded paths. You need to access outputs_namespace.paths (create evaluation path there) and pass it in this function from the parent (evaluate_concatenated_response)

Udayraj123 · 2024-10-07T01:43:45Z

src/evaluation.py

@@ -505,9 +538,10 @@ def conditionally_add_explanation(
                if item is not None
            ]
            self.explanation_table.add_row(*row)
+            self.explanation_table_data_for_csv.append(row)


Instead of appending the row here, can you check if `self.explanation_table.rows can be looped upon later? It will reduce the coupling.

Udayraj123 · 2024-10-07T01:44:23Z

src/evaluation.py

@@ -517,6 +551,6 @@ def evaluate_concatenated_response(concatenated_response, evaluation_config):
        )
        current_score += delta

-    evaluation_config.conditionally_print_explanation()
+    evaluation_config.conditionally_print_explanation(file_id)


let's call a separate function instead of coupling the logic with the dedicated small functions

Udayraj123 · 2024-10-07T01:45:28Z

src/utils/file.py

@@ -44,6 +46,11 @@ def setup_dirs_for_paths(paths):
            logger.info(f"Created : {save_output_dir}")
            os.makedirs(save_output_dir)

+    for save_output_dir in [paths.evaluation_dir]:
+        if os.path.exists(save_output_dir):
+            shutil.rmtree(save_output_dir)


let's not remove any of the user's output. Conditionally create the directory instead similar to other lines here

Udayraj123 · 2024-10-07T01:47:08Z

src/evaluation.py

        if self.should_explain_scoring:
            console.print(self.explanation_table, justify="center")

+        self.explanation_to_csv(file_id)
+        self.explanation_table_data_for_csv = []


We can also populate explanation_table_data_for_csv directly by looping on self.explanation_table.rows if that's available. That can make the code even cleaner

Drita-ai · 2024-10-07T16:53:31Z

Thank @Udayraj123 for the review and for pointing out the areas for improvement. I’ll make the necessary changes as you've suggested.

Drita-ai · 2024-10-09T10:09:28Z

Hey @Udayraj123, I've made the changes you've asked for. Please consider reviewing it.

Udayraj123

Good improvement on previous comments. Please check these comments as well.

Udayraj123 · 2024-10-13T05:22:22Z

src/utils/file.py

+    for save_output_dir in [paths.evaluation_dir]:
+        if not os.path.exists(save_output_dir):
+            logger.info(f"Created : {save_output_dir}")
+            os.makedirs(save_output_dir)
+
    for save_output_dir in [paths.multi_marked_dir, paths.errors_dir]:
        if not os.path.exists(save_output_dir):


Suggested change

for save_output_dir in [paths.evaluation_dir]:

if not os.path.exists(save_output_dir):

logger.info(f"Created : {save_output_dir}")

os.makedirs(save_output_dir)

for save_output_dir in [paths.multi_marked_dir, paths.errors_dir]:

if not os.path.exists(save_output_dir):

for save_output_dir in [paths.multi_marked_dir, paths.errors_dir, paths.evaluation_dir]:

if not os.path.exists(save_output_dir):

Udayraj123 · 2024-10-13T05:42:34Z

src/evaluation.py

+    # Explanation Table to CSV
+    def conditionally_save_explanation_csv(self, evaluation_path):
+        if self.enable_evaluation_table_to_csv:
+            data = {col.header: col._cells for col in self.explanation_table.columns}


Nice. Can you add a screenshot of what the output csv looks like now?

Udayraj123 · 2024-10-13T05:45:18Z

src/evaluation.py

+    def conditionally_save_explanation_csv(self, evaluation_path):
+        if self.enable_evaluation_table_to_csv:
+            data = {col.header: col._cells for col in self.explanation_table.columns}
+
+            output_dir = os.path.join(
+                os.getcwd(),
+                f"{evaluation_path}.csv",
+            )


let's use evaluation_output_dir here

Suggested change

def conditionally_save_explanation_csv(self, evaluation_path):

if self.enable_evaluation_table_to_csv:

data = {col.header: col._cells for col in self.explanation_table.columns}

output_dir = os.path.join(

os.getcwd(),

f"{evaluation_path}.csv",

)

def conditionally_save_explanation_csv(self, file_path, evaluation_output_dir):

if self.enable_evaluation_table_to_csv:

data = {col.header: col._cells for col in self.explanation_table.columns}

output_path = os.path.join(

evaluation_output_dir,

f"{file_path.stem}_evaluation.csv",

)

Udayraj123 · 2024-10-13T05:46:42Z

src/entry.py

+            score = evaluate_concatenated_response(
+                omr_response, evaluation_config, evaluation_path
+            )


Suggested change

score = evaluate_concatenated_response(

omr_response, evaluation_config, evaluation_path

)

score = evaluate_concatenated_response(

omr_response, evaluation_config, file_path, evaluation_output_dir

)

Udayraj123 · 2024-10-13T05:47:03Z

src/entry.py

@@ -209,6 +209,9 @@ def process_files(
    for file_path in omr_files:
        files_counter += 1
        file_name = file_path.name
+        evaluation_path = os.path.join(
+            outputs_namespace.paths.evaluation_dir, file_path.stem


let's directly pass evaluation_output_dir (= outputs_namespace.paths.evaluation_dir) into the function below

Drita-ai added 2 commits October 3, 2024 19:02

feat: Add feature to export explanation table to CSV

b1dcf3d

Remove print args statement

5bcaac1

Drita-ai changed the title ~~Feat/evaluation table to csv~~ [Feature] Export the evaluation summary table in csv format Oct 6, 2024

Drita-ai changed the title ~~[Feature] Export the evaluation summary table in csv format~~ Added feature to export the evaluation summary table in csv format Oct 6, 2024

Udayraj123 requested changes Oct 7, 2024

View reviewed changes

Udayraj123 added the hacktoberfest label Oct 7, 2024

Refactored code to improve reusability

32d9079

Udayraj123 requested changes Oct 13, 2024

View reviewed changes

Refactor code in file.py, evaluation.py and entry.py

fb0dd75

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added feature to export the evaluation summary table in csv format #229

Added feature to export the evaluation summary table in csv format #229

Drita-ai commented Oct 6, 2024

Udayraj123 left a comment

Udayraj123 Oct 7, 2024

Udayraj123 Oct 7, 2024

Udayraj123 Oct 7, 2024

Udayraj123 Oct 7, 2024

Udayraj123 Oct 7, 2024

Udayraj123 Oct 7, 2024

Udayraj123 Oct 7, 2024

Drita-ai commented Oct 7, 2024

Drita-ai commented Oct 9, 2024

Udayraj123 left a comment

Udayraj123 Oct 13, 2024

Udayraj123 Oct 13, 2024

Drita-ai Oct 13, 2024

Udayraj123 Oct 13, 2024

Udayraj123 Oct 13, 2024

Udayraj123 Oct 13, 2024

		evaluation_json = open_evaluation_with_validation(self.path)

		if evaluation_json["options"].get("enable_evaluation_table_to_csv", False):

Added feature to export the evaluation summary table in csv format #229

Are you sure you want to change the base?

Added feature to export the evaluation summary table in csv format #229

Conversation

Drita-ai commented Oct 6, 2024

Udayraj123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Drita-ai commented Oct 7, 2024

Drita-ai commented Oct 9, 2024

Udayraj123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment