When our jobs run they check previous test results and by default skip tests which passed in the previous run. There's a need when we run custom test runs to skip this check and run all the tests regardless if they passed or failed in the previous run.
Second thing is when we push empty file to S3 bucket with the test name which passed we also add some other parameters like kubernetes version, database version etc. - but we don't include all which is not optimal - you can have a run with some different parameter and it could think that some test passed. The idea is to add a hash of all parameters used in the test to the filename in S3.
When our jobs run they check previous test results and by default skip tests which passed in the previous run.
There's a need when we run custom test runs to skip this check and run all the tests regardless if they passed or failed in the previous run.
Second thing is when we push empty file to S3 bucket with the test name which passed we also add some other parameters like kubernetes version, database version etc. - but we don't include all which is not optimal - you can have a run with some different parameter and it could think that some test passed.
The idea is to add a hash of all parameters used in the test to the filename in S3.