Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add image diff tests #43

Open
jan-xyz opened this issue Dec 5, 2017 · 2 comments
Open

Add image diff tests #43

jan-xyz opened this issue Dec 5, 2017 · 2 comments

Comments

@jan-xyz
Copy link
Contributor

jan-xyz commented Dec 5, 2017

Test the output from the visualizers against expected images.

When we have that we can also submit these pictures for calculation and make sure that the probabilities are as expected.

That means:

  • collect some sample sumbers for the MNIST model testing
  • compare output images against expected images
  • compare output probabilieties against expected images
@jan-xyz
Copy link
Contributor Author

jan-xyz commented Dec 31, 2017

rhsimplex pushed a commit that referenced this issue Jan 5, 2018
* initial commit of Javascript front end

* implement getting list of visualisers from REST API

* clean up index.html for better readability

 I copied the layout from the existing files and begin to clean that stuff up so that it is easier to read and work with.

* dynamically add and remove vis settings boxes

 This isn't fully working yet. I still have to add the actual settings to it and make it work on the initial loading of the page. Currently you have to at least select something in the box before getting boxes for the options.

* follow a few best practices

* only update the list once all elements are collected

* cache the visulalizer selector in the beginning and reference

* trigger change event on visualizer selector once all vis are appended

* refactor AJAX calls into functions for better readability

* move string containing new li elements into the appropriate scope

* remove bullet points in front of settings

* update jquery to the latest version

* implement loading the setting options into the respective fields

 The UI loads now all necessary elements to properly send a visualize request.

* make sure elements for appending are properly defined.

 Fixes the problem that the first element has 'undefinded' prepended to the string.

* reference new layout that doesn't have backend connection

 This new layout has all references to app_state removed in order to pull this information dynamically via REST from the backend.

* further simplify the html document

* remove dependency on utility module from web frontend

* implement file uploading

 for now it only  uploads a single file and no information about the visualizer to use. Next step is to implement the backend for a post to visualize with a picture attached.

* request visualize via REST

 It is still without the visualization settings

* add settings to query string

* add block for results

* display input image when results come in

 The partial occlusion visualizer uses a different property than the others when delivering the image. This should be fixed in the backend.

* display probabilities

* empty the probability list after every request

* append the output probabilities to the correct section

* fix reference to outputs according to latest API contract

* display results based on the old layout

* add selenium to test front-end

* reduce the web app to only one document / remove layout.html

* add the REST call to the javascript frontend

* add license headers

* correct the settings handover from API to backend

* fix class probabilities test

* move rest API tests to separate test_* file

* refactor current website implementation to work with factory pattern

 By switching to blueprints for this as well I am able to use the factory pattern which is essential for Selenium testing.

* rename app inside create_app to avoid shadowing the actual app

* add first selenium integration test for loading the new front page

 Currently the integration tests have to run first. It's weird but the tests will hang if this is executed last. That is the reason why it has the *_a_* in it. I assume there is some collision in the fixtures that causes this behavior, probably related to app shutdown.

* Skip Selenium test

 This is still broken in the travis file. We need to download a webdriver first.

* use firefox 56 instead of chrome for selenium tests

* fix requirements aka moving selenium to test_requirements

* move integration test into its own step

 This way it doesn't interfere with the unit tests and can also be triggered separately.

* fix typos and style warnings

* Add basic image verification

this still doesn't check parameters.

relates to: #43

* verify output images

no checks of parameters

relates to: #43

* Do not ask for confirmation when creating conda env

* remove params from conda create

* verify output images based on settings parameters

This only checks one set of settings as a reference check.

relates to: #43

* add integration tests to travis

* put unit tests and integration tests into same stage

* fix build stages according to tutorial

https://docs.travis-ci.com/user/build-stages/#Naming-your-Build-Stages
@jan-xyz
Copy link
Contributor Author

jan-xyz commented Apr 9, 2018

I think this can be closed as it was part of PR #41

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant