Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Mesh's Pocket-Core to RC-0.11.1 #13

Closed
wants to merge 95 commits into from

Commits on Jan 22, 2024

  1. Configuration menu
    Copy the full SHA
    16f6f31 View commit details
    Browse the repository at this point in the history
  2. Fix pokt-network#1457; Fix pokt-network#1484;

    Fix a high memory consumption that also is part of the issue pokt-network#1457.
    Under high load of requests (1000/rps or more) the RAM got crazy and scale up to 40GB or close to that.
    Now after the fix of pokt-network#1457 with the worker pool, the node remains under 14gb of ram in my local tests.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    5cabe5c View commit details
    Browse the repository at this point in the history
  3. remove idle timeout in favor to use default (5 seg) because otherwise…

    … keep cpu in high load.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    67eace1 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    db02135 View commit details
    Browse the repository at this point in the history
  5. Initial Geo-Mesh work

    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    8b7b3e4 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    3aa26c0 View commit details
    Browse the repository at this point in the history
  7. * Added background cache compaction

    * Fixed RPC timeout handled as Seconds instead of Milliseconds
    * Updated mesh.md to handle new cache configurations
    * Updated mesh.md to list /v1/private/mesh/session as required on the whitelist endpoints/paths
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    346f8d4 View commit details
    Browse the repository at this point in the history
  8. * Fixed chains hot reload with editor like vim

    * Fixed /v1/private/mesh/updatechains to properly update them on memory and disk
    * Added hot reload for servicer private key files (add & remove)
      * on add turn on the checks and start allowing it
      * on remove stop receiving and consume all the pending relays in queue
    * Version bump
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    d7695a3 View commit details
    Browse the repository at this point in the history
  9. * Removed unnecessary initCron call

    * Enhanced log about missing sessions
    * Version Bump
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    9ba6e99 View commit details
    Browse the repository at this point in the history
  10. * Fixed issue that allow session to remains on cache forever if the p…

    …rivate key is removed after it been supported by the mesh node.
    
    * Version Bump
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    c48811d View commit details
    Browse the repository at this point in the history
  11. * Removed disk session storage due to issues with size growing (tempo…

    …ral solution)
    
    * Fixed error that panic process when load servicer_url without http/https schema. Now it will properly report the error.
    * Added manual cron to compact relays database every hour.
    * Removed a log2.Fatal that was crashing the process.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    28d59a8 View commit details
    Browse the repository at this point in the history
  12. Fixed:

    * relay_cache_background_sync_interval was not used
    * relay_cache_background_compaction_interval was not used
    
    Added:
    * hot_reload_interval allow to turn off using 0 the hot reload of chains/servicers - otherwise the amount of MS it will check the files again
    
    Updated:
    * Now health check of servicers is done every 60s - was 30s - future: will be configurable through config.json
    * Now old sessions are evaluated to be removed every 30m - was 30s - future: will be configurable through config.json
    * config.json example of docs.
    
    Removed:
    * Manual relays db compaction job removed; We receive reports that it was corrupting relays database if you run at same time of background configured by relay_cache_background_compaction_interval
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    6e5eb46 View commit details
    Browse the repository at this point in the history
  13. Refactor handle relay to ensure relays are stored in time and deleted…

    … from storage in any case after they are success/failed.
    
    Fixed log that was printing node instead of app public key.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    0610cd0 View commit details
    Browse the repository at this point in the history
  14. Fixed issue preventing nodes to track remaining relays properly for e…

    …very servicer on same session.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    97fc658 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    90e3fbb View commit details
    Browse the repository at this point in the history
  16. Remove random selection of servicer because produce error for those r…

    …unning mixed servicers.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    546f31f View commit details
    Browse the repository at this point in the history
  17. Update go.mod. Update Servicer version to latest official release RC-…

    …0.9.2. Bump Mesh client to RC-0.2.6
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    38997c9 View commit details
    Browse the repository at this point in the history
  18. Update mesh.md

    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    0c17fee View commit details
    Browse the repository at this point in the history
  19. Initial rework to speed up bootstrap times of mesh client.

    Added different key format.
    Refactor connectivity checks.
    Refactor node/servicer internal structure of mesh to reduce amount of worker/cron instances.
    Refactor chains/keys reload.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    541bd36 View commit details
    Browse the repository at this point in the history
  20. Fixed minor issues.

    Added FullNode worker dynamic resize on servicers change.
    Updated servicers reload to only run the modification on maps when there is something new/removed.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    cc0efa6 View commit details
    Browse the repository at this point in the history
  21. Replace sync.Map with xsync.Map to use Generic in favor of performanc…

    …e and better readability of the code without so many casts.
    
    Refactor fullNode.Servicer to be a map instead of a slice.
    Enhance a bit more the logs and bootstrap time information.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    bd9c5b9 View commit details
    Browse the repository at this point in the history
  22. Added worker metrics.

    Added metrics config support.
    Refactor code to split in files.
    Bump pond version to 1.8.3 (patch).
    Clean up the code.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    29e1b9a View commit details
    Browse the repository at this point in the history
  23. Update docs.

    Update config to handle rpc timeout for different things like chains, client and pocket node calls with a different value.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    396647f View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    73b14f7 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    ed3449b View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    0b4aec2 View commit details
    Browse the repository at this point in the history
  27. Updated rpc-spec.yaml.

    Ensure that http response body is read even on errored request to reuse connections.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    1ab8574 View commit details
    Browse the repository at this point in the history
  28. Fixed issue with /v1/private/mesh/updatechains endpoint.

    Enhanced chains reload logs.
    Enhanced startup logs.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    45b2a69 View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    54cc0e7 View commit details
    Browse the repository at this point in the history
  30. Removed dynamic capacity on servicer worker due to the need of handle…

    … so many edge cases and possible infinite goroutine spams.
    
    Added name property to nodes as optional key, if not set use the hostname of the node url.
    Added minWorker, maxWorker, maxCapacity to prometheus metrics collectors.
    Refactor minWorker, maxWorker and maxCapacity option in config.
    Bump default to a more real world value.
    Updated docs.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    7989d31 View commit details
    Browse the repository at this point in the history
  31. Updated docs.

    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    d4e9380 View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    30de368 View commit details
    Browse the repository at this point in the history
  33. Added json schema support to additional properties. Those will not im…

    …pact on mesh code, but help node runners to keep internal track.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    0e4d60e View commit details
    Browse the repository at this point in the history
  34. Rework metrics to handle less metrics with more labels. Enhance group…

    …/queries on prometheus.
    
    Added chains name map so those metrics could contain the chain name you wish.
    ChainsNameMap could work with a local file or remote endpoint (GET)
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    1da8ed6 View commit details
    Browse the repository at this point in the history
  35. Set chains_name_map default as empty so avoid to read it and write an…

    … error log. Does not affect the code but is unnecessary.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    ce66a6f View commit details
    Browse the repository at this point in the history
  36. Added metrics_moniker.

    Added status_type and status_code labels to error metrics.
    Added internal, notify and chain error metrics.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    13d1cb0 View commit details
    Browse the repository at this point in the history
  37. Configuration menu
    Copy the full SHA
    7484dfe View commit details
    Browse the repository at this point in the history
  38. Rename status_label to status_code.

    Added metrics docs and basic geo-mesh grafana dashboard.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    ee3b529 View commit details
    Browse the repository at this point in the history
  39. Moved authtoken to Authorization header to avoid the security token c…

    …ould be leaked on logs.
    
    Moved Grafana Dashboard to a file to easily compare/copy from raw github files.
    Updated mesh.md
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    629a2f8 View commit details
    Browse the repository at this point in the history
  40. Enhance code based on blade suggestions.

    Enhance docs based on blade suggestions.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    7c7b4e7 View commit details
    Browse the repository at this point in the history
  41. Configuration menu
    Copy the full SHA
    17931ec View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    a7272f4 View commit details
    Browse the repository at this point in the history
  43. Added support to configure more http client that handle mesh request …

    …and the one dispatching to chains.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    22d4032 View commit details
    Browse the repository at this point in the history
  44. Configuration menu
    Copy the full SHA
    a19a3d3 View commit details
    Browse the repository at this point in the history
  45. Configuration menu
    Copy the full SHA
    c53a5cc View commit details
    Browse the repository at this point in the history
  46. Updated default values.

    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    bde223a View commit details
    Browse the repository at this point in the history
  47. Updated mesh.md

    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    c26b742 View commit details
    Browse the repository at this point in the history
  48. Configuration menu
    Copy the full SHA
    dd87d83 View commit details
    Browse the repository at this point in the history
  49. Renamed invalidate session codes.

    Added non retryable relays codes.
    Change retryable handler to use a dedicated function for it.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    dfff2d8 View commit details
    Browse the repository at this point in the history
  50. Bump version

    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    a6d193b View commit details
    Browse the repository at this point in the history
  51. Fixed issue on report metrics without proper return http code from bl…

    …ockchain call.
    
    Reduce default values for the retries.
    Removed CodeRequestHash & CodeInvalidBlockHeightError from invalidate session codes.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    35ce5da View commit details
    Browse the repository at this point in the history
  52. Fixed issue on report metrics without proper return http code from bl…

    …ockchain call.
    
    Reduce default values for the retries.
    Moved CodeRequestHash & CodeInvalidBlockHeightError from invalidate session codes to non retryable ones.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    9074e75 View commit details
    Browse the repository at this point in the history
  53. Added support to lean node for evidence worker. Now is one worker per…

    … address. Prevent the race condition on the evidence store and do not hold relays evidence due to other address is using the queue.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    52bb32c View commit details
    Browse the repository at this point in the history
  54. Remove chain url from the error message returned as part of the respo…

    …nse. Instead, return a more generic information but log the real error for internal node runner tracking.
    
    Enhance error status types for servicer and chain not found.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    e39d129 View commit details
    Browse the repository at this point in the history
  55. Fixed issue that was forcing retry relays notifications when they sho…

    …uld be discarded.
    
    Added session validation before notify to avoid call a node if the session was already invalidated.
    Bump version to 0.3.2
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    ad5b804 View commit details
    Browse the repository at this point in the history
  56. Configuration menu
    Copy the full SHA
    140d730 View commit details
    Browse the repository at this point in the history
  57. Configuration menu
    Copy the full SHA
    deeb135 View commit details
    Browse the repository at this point in the history
  58. Configuration menu
    Copy the full SHA
    d08aa41 View commit details
    Browse the repository at this point in the history
  59. Rebase on latest geo-mesh commit.

    Enhance code base on @PoktBlade comments.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    6635f3a View commit details
    Browse the repository at this point in the history
  60. Rework ShouldAssumeOptimisticSession function.

    Added unit-test for ShouldAssumeOptimisticSession function.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    4f7126f View commit details
    Browse the repository at this point in the history
  61. Configuration menu
    Copy the full SHA
    49de50e View commit details
    Browse the repository at this point in the history
  62. Added missing blockchain validation on relays.

    Added missing session height validation on relays.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    998ce04 View commit details
    Browse the repository at this point in the history
  63. Configuration menu
    Copy the full SHA
    000603c View commit details
    Browse the repository at this point in the history
  64. Added "current" session to the same logic of the "optimistic" one to …

    …reduce the latency on session changes.
    
    Fixed issue on /v1/private/mesh/check that throw error asking latest height params.
    Fixed relay_time metric that was not wrapping around "validate" method, and it is the one that could take more time after call blockchain.
    Added missing initialization of the metrics for session storage.
    Added metric for call blockchain as a separated one.
    Enhance few error logs.
    Added additional metrics.
    Fixed multiple issues detected around offloaded session validate tasks.
    Fixed an issue that prevent session behind validated after the session already exists for other servicer handled by the mesh.
    Added more defensive code question to avoid panic due to nil pointers.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    97d752c View commit details
    Browse the repository at this point in the history
  65. Refactored session.go to handle session in a single map to avoid race…

    … conditions and reduce memory consumption.
    
    Enhance logs for relay and session.
    Enhance metrics for notify process.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    ae6857f View commit details
    Browse the repository at this point in the history
  66. Configuration menu
    Copy the full SHA
    a1266d5 View commit details
    Browse the repository at this point in the history
  67. add lookback session rollover support

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    83d8ef6 View commit details
    Browse the repository at this point in the history
  68. cleanup relay.go and fix potential dereference runtime error

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    b657c97 View commit details
    Browse the repository at this point in the history
  69. refractor session cleanup math

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    4287bb5 View commit details
    Browse the repository at this point in the history
  70. add node session comments

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    9113732 View commit details
    Browse the repository at this point in the history
  71. add comments about node session

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    e68dc22 View commit details
    Browse the repository at this point in the history
  72. add comments about node session

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    4eb1ccb View commit details
    Browse the repository at this point in the history
  73. fix relay vaidation and small cleanup on if else

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    1e49704 View commit details
    Browse the repository at this point in the history
  74. Bump Version to RC-0.4.0

    Update docs.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    3bb85dc View commit details
    Browse the repository at this point in the history
  75. Added log_relay_request to help PNI debug ongoing issue with the stuc…

    …k session height 100589.
    
    Removed jump lines (\n) on the errors provided by the pocketcore code. This difficult the usage of tooling like Loki that will collect a line of text before the jump line as an entry.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    0e51a5a View commit details
    Browse the repository at this point in the history
  76. Avoid evaluation of the tolerance for the future session because that…

    … will be done by GetSession a few lines below.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    aee1aca View commit details
    Browse the repository at this point in the history
  77. Moved where the non processed relay error is printed to be able to ad…

    …d the request information like the headers.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    55f650f View commit details
    Browse the repository at this point in the history
  78. Configuration menu
    Copy the full SHA
    5f5c2cb View commit details
    Browse the repository at this point in the history
  79. add better proof validation

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    ef18426 View commit details
    Browse the repository at this point in the history
  80. add duplicate proof detection

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    3706fec View commit details
    Browse the repository at this point in the history
  81. Configuration menu
    Copy the full SHA
    b983243 View commit details
    Browse the repository at this point in the history
  82. remove unnecessary comment

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    56f4d0e View commit details
    Browse the repository at this point in the history
  83. add relay to bloom filter immediately validating the relay

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    5050494 View commit details
    Browse the repository at this point in the history
  84. seperate to its own functions

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    d327697 View commit details
    Browse the repository at this point in the history
  85. add optimistic duplicate relay map

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    dc78a20 View commit details
    Browse the repository at this point in the history
  86. remove unnecessary return of ns

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    e627d7d View commit details
    Browse the repository at this point in the history
  87. add a more compute/memory efficient key

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    4b9500f View commit details
    Browse the repository at this point in the history
  88. add byte<> string conversion and replace storing relay with a empty m…

    …emory
    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    0cd0477 View commit details
    Browse the repository at this point in the history
  89. use the relay proof instead of relay obj

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    894e51e View commit details
    Browse the repository at this point in the history
  90. init map

    poktblade authored and jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    4850a1f View commit details
    Browse the repository at this point in the history
  91. Fixed memory issue due to old sessions not been deleted from local st…

    …orage. Fixed a typo. Bump version to RC-0.4.2
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    37c4d62 View commit details
    Browse the repository at this point in the history
  92. Updated workflow to achieve multi-arch build on an automated way.

    Updated entrypoint.sh with a more complete one.
    Updated go.mod/sum with geo-mesh needs.
    jorgecuesta committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    ac22752 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2024

  1. Configuration menu
    Copy the full SHA
    3aadc69 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2024

  1. Update AppVersion to RC-0.11.1 (pokt-network#1595)

    This patch promotes BETA to RC to create a release tag (instead of pre-release).  There is no functional change between two tags.
    msmania authored Jan 24, 2024
    Configuration menu
    Copy the full SHA
    e47bc70 View commit details
    Browse the repository at this point in the history

Commits on Jan 28, 2024

  1. Merge branch 'staging' of github.com:pokt-network/pocket-core into ME…

    …SH-RC-1.4.2-RC-0.11.1
    blade committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    7044e46 View commit details
    Browse the repository at this point in the history