elastalert-test-rule is locked to file loader only?

[akusei]

We're creating a custom rule loader and everything works great except for when I test the rules. The rules need to pass through this loader in order to be valid but it seems the code in test_rules.py is locking the rules loader to "file" as seen below:

test_rules.py lines 443 - 449

overwrites = {
    'rules_loader': 'file',
}

conf = load_conf(self.args, defaults, overwrites)
rule_yaml = conf['rules_loader'].load_yaml(self.args.file)
conf['rules_loader'].load_options(rule_yaml, conf, self.args.file)

This is hardcoding the loader as the file loader so it completely ignores when I have rules_loader configured in the main config file. I also don't see an option in the command line arguments to specify a loader. Was it intentional to hardcode the rule loader? We can't properly test rules (specifically the alerting aspect) without being able to specify a different loader

[jertel]

The overwrites was added 4-5 years ago:

515b15a

So I can't answer why this was done. The commit message doesn't fully align with the committed code. If removing those 3 lines doesn't cause a problem for users not customizing the rule loader, and solves the issue you're experiencing, then perhaps they need to be removed permanently.

[akusei]

I think you're right. I too was a little confused on that commit message. I tested removing the overwrite completely and it works perfectly fine when I define rules_loader in the config and it properly defaults to file when rules_loader is absent from the config. I'll do a PR to have that removed if you're okay with that

[jertel]

Sounds good!

[akusei]

I don't think this was ever really working. I'm starting the PR and I noticed there was no test for the overwrites and defaults so I wrote them. I ran the tests and the overwrites failed because the for loop in config.py for the overwrites is using iteritems() instead of iter(). I'll add a fix for that as well