Testing Guide: Difference between revisions
From ParaQ Wiki
Jump to navigationJump to search
No edit summary |
No edit summary |
||
Line 1: | Line 1: | ||
== Overview == | == Overview == | ||
The current ParaQ testing framework makes it possible to "record" and "play back" user interaction with the ParaQ UI, and record render-window images for later comparison against reference images. By combining these two capabilities it is possible to create regression tests. | |||
=== Running Tests === | == Recording Simple Tests == | ||
* Start the ParaQ client. | |||
* From the menu, choose ''Tools > Record Test''. | |||
* In the file dialog that opens, choose a path where the test output will be written. | |||
* The "Recording User Input" dialog will appear. Move it somewhere out-of-the-way. | |||
* Use the ParaQ client UI normally, exercising the functionality you wish to test. | |||
* Once the client is in the desired end-state, go to the "Recording User Input" dialog and hit the "Stop Recording" button. | |||
== Running Tests == | |||
* Start the ParaQ client. | |||
* From the menu, choose ''Tools > Play Test''. | |||
* In the file dialog that appears, choose a previously-recorded test. | |||
* The test will play-back. | |||
* Once the test completes, the client should be in the same end-state as when the test was recorded. | |||
== Setting-up Regression Tests == | |||
* Start the ParaQ client and record a test. | |||
* Once test recording is complete, choose ''File > Save Screenshot''. | |||
* Use the file dialog to save the contents of the render window as a reference image. | |||
* Exit the ParaQ client. | |||
* To run the test, use the appropriate command-line options to play the test, and compare its output with a reference image: | |||
$ pqClient --run-test /path/to/test.xml --compare-view /path/to/reference/image.png | |||
* The --run-test option plays-back a test scenario as if you had used the ''Tools > Play Test'' menu item. | |||
* The --compare-view option compares the contents of the render window with a reference image, exiting the client with a non-zero exit code if the images don't match. | |||
== Future Work == | |||
The current test framework has a number of limitations that will likely be addressed as development progresses: | |||
* The image-capture and comparison functionality should be expanded to include chart windows in-addition-to normal render windows. | |||
* Image-capture and comparison must be expanded to handle multi-view setups involving more-than-one render window. | |||
* Image comparison must be robust in the face of small differences in image size - on different platforms (and using different styles on a single platform), the sizes of windows will vary even with a fixed main window size, due to differences in e.g: the width of a splitter bar by a few pixels. |
Revision as of 13:30, 20 March 2006
Overview
The current ParaQ testing framework makes it possible to "record" and "play back" user interaction with the ParaQ UI, and record render-window images for later comparison against reference images. By combining these two capabilities it is possible to create regression tests.
Recording Simple Tests
- Start the ParaQ client.
- From the menu, choose Tools > Record Test.
- In the file dialog that opens, choose a path where the test output will be written.
- The "Recording User Input" dialog will appear. Move it somewhere out-of-the-way.
- Use the ParaQ client UI normally, exercising the functionality you wish to test.
- Once the client is in the desired end-state, go to the "Recording User Input" dialog and hit the "Stop Recording" button.
Running Tests
- Start the ParaQ client.
- From the menu, choose Tools > Play Test.
- In the file dialog that appears, choose a previously-recorded test.
- The test will play-back.
- Once the test completes, the client should be in the same end-state as when the test was recorded.
Setting-up Regression Tests
- Start the ParaQ client and record a test.
- Once test recording is complete, choose File > Save Screenshot.
- Use the file dialog to save the contents of the render window as a reference image.
- Exit the ParaQ client.
- To run the test, use the appropriate command-line options to play the test, and compare its output with a reference image:
$ pqClient --run-test /path/to/test.xml --compare-view /path/to/reference/image.png
- The --run-test option plays-back a test scenario as if you had used the Tools > Play Test menu item.
- The --compare-view option compares the contents of the render window with a reference image, exiting the client with a non-zero exit code if the images don't match.
Future Work
The current test framework has a number of limitations that will likely be addressed as development progresses:
- The image-capture and comparison functionality should be expanded to include chart windows in-addition-to normal render windows.
- Image-capture and comparison must be expanded to handle multi-view setups involving more-than-one render window.
- Image comparison must be robust in the face of small differences in image size - on different platforms (and using different styles on a single platform), the sizes of windows will vary even with a fixed main window size, due to differences in e.g: the width of a splitter bar by a few pixels.