perf: optimization for logging system with profiling and efficient log file management

[beats-dh]

PR Description

This PR improves the performance of the logging system and adds profiling functionality with automatic log generation. Additionally, optimizations have been performed to ensure that logs are written efficiently to text files.

Main changes:

Log Processing Optimization:

• Added check to avoid unnecessary log processing at trace and debug levels. Now, the system only formats and processes log messages at these levels if they are enabled, saving resources and improving overall performance.

Automatic Generation of Log Files for Profiling:

• The profile function now automatically generates a dedicated log file, containing the execution time of profiled functions. This allows for easier analysis of the performance of critical functions by keeping logs organized and separated by execution.

Optimization in Writing Logs to Text Files:

• Implemented a strategy to reuse loggers associated with existing log files. This avoids creating new files unnecessarily and improves performance by ensuring that subsequent log messages are written to the same file when appropriate.

Debug and Trace Logs Availability:

• Debug and trace level logs are now only available in the debug compilation level, ensuring that these detailed logs are not generated in production builds, thus further optimizing performance.

Usage Example:

OutputMessage_ptr OutputMessagePool::getOutputMessage() {
	return g_logger().profile(__FUNCTION__, [&]() {
			return  std::make_shared<OutputMessage>();
		});
}

class ExampleClass {
public:
    void performTask() {
        // Use the 'profile' function to measure the execution time of 'computeHeavyTask'
        g_logger().profile("computeHeavyTask", [this]() {
            computeHeavyTask();
        });
    }

private:
    void computeHeavyTask() {
        // Simulation of a time-consuming task
        std::this_thread::sleep_for(std::chrono::seconds(2));
        // Heavy task done
    }
};

• In this example, performTask calls the computeHeavyTask function and uses the logger to measure the execution time of this function, recording the result in a specific log file.

Justification:

• These improvements aim to optimize server performance, especially in production environments, where trace and debug level logs are not necessary. The new profiling log functionality provides a structured way to monitor and improve the performance of critical functions. Optimization of writing to log files reduces I/O overhead, improving the overall efficiency of the logging system.

Tests:

• Manual checks have been performed to ensure that trace and debug level log messages are not processed when disabled.
  The profiling functionality has been tested and it has been verified that the log files are generated correctly with the recorded execution times.
  Additional testing was performed to ensure that writing to log files is efficient and that files are reused correctly without unnecessary creation of new files.

[github-actions]

This PR is stale because it has been open 45 days with no activity.

[sonarcloud]

Quality Gate passed

Issues
11 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud