It takes unusually long to write secrets using polykey secrets write command

[aryanjassal]

Specification

While testing and development, I noticed that the polykey secrets write command was taking unusually long (over 4 seconds!) to write a small secret to a file.

Then, I built the program and got the average time for an invocation of the command. The result was that, on average, it took 2789ms to write a small secret to a file in the vault. In comparison, other commands like polykey secrets ls and polykey secrets get took approximately 500ms on average.

Then, after timing the process to update files in the vault, I discovered that every time the files are modified in the vault, it takes 500ms. The commands for polykey secrets ls and polykey secrets get don't rely on writing to the vault file system, so those commands are much faster. However, even after including the time for RPC connection to be established and the time it would take to write a single file, the command was still taking more than 1000ms longer than it should be.

The RPC handler and the tests written for it were run. Each test confirmed our previous results of a write operation taking approximately 500ms. The line in the RPC handler for writing into the file also took 500ms. The RPC call also took approximately 500ms to complete in the Polykey tests.

Next on the suspect list was binUtils.retryAuthentication. It was suspected to add significant time to the command itself, but the callback only took approximately 100ms to begin running, and the RPC command itself took 2 seconds. Note that this time was inconsistent, ranging from 1700ms to 3700ms.

Then, the middleware was suspected, as the tests don't replicate the middleware. I wrote a dummy middleware factory to then insert into the RpcClient and the ClientService's middleware factories. That returned a result of the command taking around 1000ms, which is acceptable for writing into the vault file system.

Finally, the password hashing was suspected. Tests were run with changing the password hash options from min to moderate and max for time and memory limit. However, somewhat conterintuitively, it took even less time the higher we went on the time and memory limits.

As of now, the RPC command itself takes over 1700ms, when it should take much less than that, around 500-700ms. The issue couldn't be pinpointed, and further research is necessary.

Additional context

• First encountered in Polykey-CLI#292
• Another issue tracking potential performance regressions: Polykey-CLI#283

Tasks

1. Discover the reason for the delay and which parts of the program are impacted by the delay
2. Fix the issue and resolve the additional delay
3. Look into adding benchmarks to ensure performance regression does not happen (also relevant: Polykey-CLI#283

[linear]

ENG-415 It takes unusually long to write secrets using `polykey secrets write` command