Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken metrics part 1 #31

Merged
merged 81 commits into from
Oct 25, 2023
Merged

Fix broken metrics part 1 #31

merged 81 commits into from
Oct 25, 2023

Commits on Sep 13, 2023

  1. Document the bgp-tcp-in unit settings, and cleanup dangling footnotes…

    …, fix typos and make various improvements to the config file text and structure.
    ximon18 committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    b832532 View commit details
    Browse the repository at this point in the history
  2. BGP, not BMP.

    ximon18 committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    88ead40 View commit details
    Browse the repository at this point in the history
  3. Typo.

    ximon18 committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    4753271 View commit details
    Browse the repository at this point in the history
  4. Initial MVP default config support.

    When run without --config the embedded copy of etc/rotonda.conf is used instead, and targets.proxy is removed if --proxy-destination is not supplied on the command line.
    
    If --bgp-listen or --bmp-listen are supplied on the command line they override the BMP and BGP listen settings in the embedded config.
    
    - Allow BGP peers to be missing.
    ximon18 committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    d682a54 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. bmp-asn-filter.roto fixes:

      - Add missing semi-colon.
      - Filter out not in.
    ximon18 committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    505486e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    eabc6b7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    43e7142 View commit details
    Browse the repository at this point in the history
  4. Log at info rather than debug level that we are listening for HTTP co…

    …nnections on a particular socket.
    ximon18 committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    94a03f9 View commit details
    Browse the repository at this point in the history
  5. Clippy.

    ximon18 committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    20d2879 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    64f7d7c View commit details
    Browse the repository at this point in the history
  7. FIX: Store HTTP provider relative base URLs so that the router info p…

    …age can link to the actual RIB query HTTP API rather than incorrectly assume a fixed path of /prefixes/.
    ximon18 committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    b4c84c4 View commit details
    Browse the repository at this point in the history
  8. Add missing changes.

    ximon18 committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    dd1a128 View commit details
    Browse the repository at this point in the history
  9. Add missing import.

    ximon18 committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    67b76d6 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. Configuration menu
    Copy the full SHA
    b59fe39 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    321d2f0 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'add-default-mvp-config' into fix-broken-router-info-pag…

    …e-prefixes-query-link
    ximon18 committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    5096f6c View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. Configuration menu
    Copy the full SHA
    8dabed6 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Configuration menu
    Copy the full SHA
    3b5c78b View commit details
    Browse the repository at this point in the history
  2. Adjust the config syntax to use filter name references to filters def…

    …ined in roto scripts loaded from a roto script directory.
    
    - Refactors roto script file reading from the units/targets to the Manager.
    
    - Checks script loading, compilation and valid filter names at config load time instead of at filter execution time.
    
    - Add MVP behaviour tests (WIP).
    
    - Introduces a FilterName type.
    ximon18 committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    1270935 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Configuration menu
    Copy the full SHA
    c4af03a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6f396ec View commit details
    Browse the repository at this point in the history
  3. And mark the process_metrics as unused, but leave them there for now …

    …in case I return to this later.
    ximon18 committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    a57ec2a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    77884bb View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    f66a9f5 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e4d5e55 View commit details
    Browse the repository at this point in the history
  7. Add TODO comment.

    ximon18 committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    d1f6f98 View commit details
    Browse the repository at this point in the history
  8. Less childish name.

    ximon18 committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    312480a View commit details
    Browse the repository at this point in the history
  9. Permit filters to be missing in MVP mode, as the user may not have th…

    …e required .roto files, e.g. if running from `cargo install`. Cleanup config initialisation and standard variable naming.
    ximon18 committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    4b6cf0c View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    9b543e7 View commit details
    Browse the repository at this point in the history
  11. Don't abort when using the embedded config if the defined `roto_scrip…

    …ts_dir` dir `etc` doesn't exist.
    
    This change had several knock on effects:
    - Introduced RotoError::LoadError.
    - Upgrade clap crate to fix bug with clap switcing help on own line or not even when disabled.
    - Upgrade toml crate to fix inconsistent line breaks in diagnostic dump of post-processed config TOML.
    - Adjust position handling in `Marked` due to changes in serde_spanned pulled in by other crate upgrades.
    - Added new cmd line arg --print-config-and-exit and introduced `Terminated` for use in cases where `ExitError` wrongly limits us to just an error exit rather than a normal exit.
    - Various minor tweaks to the clap config for improved read and usability.
    - Log when exiting.
    ximon18 committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    d10642d View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    7ef5528 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Configuration menu
    Copy the full SHA
    ab077b7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b68ceb9 View commit details
    Browse the repository at this point in the history
  3. Add the MQTT target to match the MVP diagram, and a command line argu…

    …ment to set the destination in MVP mode. Also make the MQTT unit consistent with the BMP proxy unit by having a destination setting rather than individual server_host and server_port settings.
    ximon18 committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    f874c09 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e6fb6c1 View commit details
    Browse the repository at this point in the history
  5. Remove errant whitespace.

    ximon18 committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    c316a25 View commit details
    Browse the repository at this point in the history
  6. Apply the Cargo mqtt feature guard consistently also in new code by s…

    …imply removing the Cargo feature entirely.
    ximon18 committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    f8bdf10 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    bdba816 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    efc4568 View commit details
    Browse the repository at this point in the history
  9. Minor doc correction.

    ximon18 committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    940b645 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    d325dc4 View commit details
    Browse the repository at this point in the history
  11. Remove outdated comment.

    ximon18 committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    17ad67c View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    fb74130 View commit details
    Browse the repository at this point in the history
  13. WIP

    ximon18 committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    5a422c4 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. Remove the post-filters, they were a misunderstanding in the absence …

    …of the no longer absent filtering within the BGP unit itself.
    ximon18 committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    1557d97 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a9d38c1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6af63da View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    eb7095c View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Add a special metric output format for use by unit tests, and a callb…

    …ack fired on each VM exec that rejects the input (to enable easy counting of filtered out input messages), and remove support for metric types we don't need. Use the new callback to invoke the till now unused message_filtered() status reporter fn.
    ximon18 committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    4511262 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    79e01cd View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Remove the bmp_in_connection_count metric as connections are handled …

    …by bmp_tcp_in while this metric is based on BMP initiation messages received, so this is (a) misleading and (b) doesn't work anyway as when the sys name changes so can the router id and thus the changed metric would be in a different metric set, there'd be no effect on the previous metric.
    ximon18 committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    7852503 View commit details
    Browse the repository at this point in the history
  2. Follow changes in router id caused by a new sysName received by a sub…

    …sequent BMP Initiation Message.
    ximon18 committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    7d1cba7 View commit details
    Browse the repository at this point in the history
  3. Make test metrics easier to query, and add initial tests for the one …

    …and only remaining bmp-in metric. TODO: add a test showing bmp_in_num_invalid_bmp_messages increasing.
    ximon18 committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    1ee9a34 View commit details
    Browse the repository at this point in the history
  4. Remove the footgun of having to initialiize metrics per router, just …

    …ensure they are initialized on first use, otherwise metrics get lost.
    ximon18 committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    980891a View commit details
    Browse the repository at this point in the history
  5. Add a test showing that invalid BMP messages cause the bmp_in_num_inv…

    …alid_bmp_messages counter to increase.
    ximon18 committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    d860db2 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. Mark metric as tested.

    ximon18 committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    bba52fd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ed9aad7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d8a409d View commit details
    Browse the repository at this point in the history
  4. Sort test metric output for easier reading when inspecting the conten…

    …ts manually during development.
    ximon18 committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    96dd214 View commit details
    Browse the repository at this point in the history
  5. Metric name typo correction.

    ximon18 committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    eb7b53b View commit details
    Browse the repository at this point in the history
  6. Mock the TcpStream as well so that we can accept a simulated connecti…

    …on to test the `bgp_tcp_in_connection_accepted_count` metric.
    ximon18 committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    09a8713 View commit details
    Browse the repository at this point in the history
  7. s/think/thin/

    ximon18 authored Oct 5, 2023
    Configuration menu
    Copy the full SHA
    a7e355d View commit details
    Browse the repository at this point in the history
  8. Also test the connection lost and disconnected bgp-tcp-in metrics by …

    …using a mock BGP `Session`.
    ximon18 committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    f918b9c View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2023

  1. Configuration menu
    Copy the full SHA
    8aa601b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    48fd954 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Configuration menu
    Copy the full SHA
    d881c89 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2ea5c25 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. Fix broken links in bmp-in HTML interface, and name the RIB instances…

    … whose APIs are linked to.
    ximon18 committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    2de35f7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1b30295 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e6ae4e5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    3db6f53 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c30420a View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    75345a5 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b0abd6e View commit details
    Browse the repository at this point in the history
  8. Rename fn to make it clearer that its results are only for now, and l…

    …ater results may be different if called again.
    ximon18 committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    0cda43b View commit details
    Browse the repository at this point in the history
  9. Introduce the concept of fatal vs non-fatal I/O errors and don't drop…

    … the BMP TCP input receiver on non-fatal errors (otherwise loss of the receiver causes the router read loop to abort anyway).
    ximon18 committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    f423b75 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    90f4844 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    e636a71 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    f42e5a8 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. Configuration menu
    Copy the full SHA
    8aa7755 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    01ffb8f View commit details
    Browse the repository at this point in the history