Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IS-12] Add auto tests for validating MS-05 standard class and datatype definitions #819

Merged
merged 63 commits into from
Jul 18, 2023
Merged

[IS-12] Add auto tests for validating MS-05 standard class and datatype definitions #819

merged 63 commits into from
Jul 18, 2023

Conversation

jonathan-r-thorpe
Copy link
Contributor

  • Overrides "auto" tests to dynamically check all standard classes and datatypes
  • Creates JSON schemas for each datatype on the fly from MS-05 descriptor models
  • Queries ClassManager for all classes and datatypes
  • Validates against JSON schema and MS-05 descriptor models

jonathan-r-thorpe and others added 30 commits April 25, 2023 14:11
@jonathan-r-thorpe
Initial commit
7338dc0
@jonathan-r-thorpe
Added IS-12 spec to Config
b515423
@jonathan-r-thorpe
Updated specs for IS-12-01 test suite
011c5c4
@jonathan-r-thorpe
Implement NCP endpoint and create WebSocket tests
1497e79
@jonathan-r-thorpe
Merge branch 'AMWA-TV:master' into is-12-jrt
11b7711
@jonathan-r-thorpe
Updated API name
f00f937
@jonathan-r-thorpe
Updated test definition.
abeef7c 
Handle WebSocket urls in api definition
@jonathan-r-thorpe
Improve control endpoint checking
16b9392
@lo-simon
Use of compare_urls to test Control Protocol href
ee9b314
@lo-simon @jonathan-r-thorpe
Update nmostesting/suites/IS1201Test.py
3ad74e6 
Co-authored-by: jonathan-r-thorpe <[email protected]>
@lo-simon @jonathan-r-thorpe
Update nmostesting/suites/IS1201Test.py
043c6a0 
Co-authored-by: jonathan-r-thorpe <[email protected]>
@jonathan-r-thorpe
Merge pull request #3 from jonathan-r-thorpe/is-12-sl
5bb73fd 
Use of compare_urls to test Control Protocol `href`
@jonathan-r-thorpe
Corrected formatting. Improved WebSocket handling
42666cc
@jonathan-r-thorpe
Refactored control API checking
ea41b46
@jonathan-r-thorpe
Corrected failure case in test_02
16a9b83
@garethsb
Merge remote-tracking branch 'origin/master' into is-12-jrt
6c0ec6f
@garethsb
Use NMOSUtils.do_test_device_control
68eac96
@garethsb
Test WebSocket services and controls are as secure as HTTP endpoints
47763ce
@jonathan-r-thorpe
Merge pull request #4 from garethsb/is-12-jrt-patch-1
4308aa1 
Use NMOSUtils.do_test_device_control
@jonathan-r-thorpe
Fix protocol bug, remove stripping of trailing slash
2c38aba
@jonathan-r-thorpe
Add is_message_received to WebsocketWorker
36062b6
@jonathan-r-thorpe
Added IS12Utils
9475ff9
@jonathan-r-thorpe
Add Root Block tests
c069487
@jonathan-r-thorpe
Check Root Block and check Class Manager OID and Role
8fafa88
@jonathan-r-thorpe
Merge branch 'is-12' into is-12-jrt
c182024
@jonathan-r-thorpe
Add MS-05-02 specification to Config.py
d2984ba
@jonathan-r-thorpe
Add MS-05 descriptor to MS-05 schema conversion
bf02734
@jonathan-r-thorpe
Add MS-05-02 spec to IS-12 test suite
215185a
@jonathan-r-thorpe
flake8 FTW!
6cd6c45
@jonathan-r-thorpe
Add auto testing validation of Datatypes
2110bf6
@jonathan-r-thorpe jonathan-r-thorpe marked this pull request as ready for review June 23, 2023 15:15
@jonathan-r-thorpe
Copy link
Contributor Author

Added tests for checking Node handles following errors:

  • Incorrect protocol version
  • Invalid command handle
  • Invalid command type
  • Invalid JSON
  • Invalid Oid
  • Invalid property identifier
  • Invalid method identifier
  • Read only error

@jonathan-r-thorpe
only fail error test if error not handled. Warn if status code is inc…
318cff4 
…orrect
@jonathan-r-thorpe
Refactoring and tidy up
e24f6c9
@jonathan-r-thorpe
tweak names
5fc36c1
@jonathan-r-thorpe
Replace match with dict
177b2e2
@jonathan-r-thorpe
Improve error reporting.
a4494d5
@jonathan-r-thorpe
"loose" checking of error status codes
72bc634
@jonathan-r-thorpe
Class id helper functions.
6503fa1 
Primitive to python type
@jonathan-r-thorpe
Validate device model
dcdf301
@jonathan-r-thorpe
Formatting and comments
dd4bc89
@jonathan-r-thorpe
is_manager helper function
6559e37
@jonathan-r-thorpe
Managers are members of root block and singletons
57a69f5
@jonathan-r-thorpe
Added Device Manager property definition
be01fb6
@jonathan-r-thorpe
Generalized get_manager function
b3430fd 
Device Manager test
@jonathan-r-thorpe
is non standard class helper
f3b5eb1
@jonathan-r-thorpe
Check authority key in non-standard classes
94668bc
@jonathan-r-thorpe
Validation of feature set classes and datatypes
291e37d
@jonathan-r-thorpe
remove redundant class manager helper function
64e1d21
@jonathan-r-thorpe
Formatting
eff801f
@jonathan-r-thorpe
Refactored descriptor and schema handling.
eae8c33 
Device model now validated by *all* classes and types discovered from class manager
@jonathan-r-thorpe
Push configuration of feature sets used in test into the specificatio…
cb0cb83 
…n configuration
@jonathan-r-thorpe
Replace error tuples with NMOSTestExceptions
290b284
@jonathan-r-thorpe
Improved error reporting
6969ab2
@jonathan-r-thorpe jonathan-r-thorpe merged commit c033475 into AMWA-TV:is-12 Jul 18, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jonathan-r-thorpe @lo-simon @garethsb