- marked as critical
- removed comment
Implement a testing mechanism for simfactory
Issue #124
closed
Currently there is no easy way to check that a change to simfactory hasn't broken critical functionality. A system could be introduced to run through each command and check that it does the right thing. It won't be possible to check every single combination of possible options, but the most commonly used options should be tested.
A first attempt could involve testing that the commands work locally on the current machine. This would involve submitting jobs and testing that they have the expected behaviour. On busy machines, this means that the tests could potentially take a long time to complete.
Remote submission could be tested by running these tests from a central location (e.g. the developer's workstation or laptop).
We could also include a python correctness checking step in these tests (e.g. using pychecker or pylint).
Keyword:
Comments (4)
-
reporter
-
reporter
Erik sent the following email to the SimFactory mailing list:
"SimFactory 2 (the Python version) comes with a script "simfactory/bin/distribute" that tests syncing, configuring, building, submitting, and running a test job to remote systems. Instructions for using this script are in the script. If you are developing SimFactory 2, I recommend using this script to test your changes."
I think that the availability of this script means that this ticket has been partially addressed and is no longer "critical". If a few people report success with using the test script, we can close the ticket.
-
reporter
- changed status to resolved
- removed comment
-
- edited description
- changed status to closed
- Log in to comment
- Assignee
-
anonymous
- Type
- Priority
- Status
- closed
- Component
- SimFactory
- Milestone
- –
- Version
- –
- Votes
- 0
- Watchers
- 0
I am raising the priority of this ticket to "critical" as the rate of regressions in SimFactory 2 is fairly high, causing problems when trying to use it in a production environment.