git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[jira] [Created] (DAFFODIL-1961) TDML Runner - enhance round-trip to distinguish simple parse-unparse from multi-trip cases


Michael Beckerle created DAFFODIL-1961:
------------------------------------------

             Summary: TDML Runner - enhance round-trip to distinguish simple parse-unparse from multi-trip cases
                 Key: DAFFODIL-1961
                 URL: https://issues.apache.org/jira/browse/DAFFODIL-1961
             Project: Daffodil
          Issue Type: New Feature
          Components: TDML Runner
    Affects Versions: 2.1.0
            Reporter: Michael Beckerle
             Fix For: 2.2.0


TDML Runner does parse-unparse round-trip tests.

Some tests the input data is not cannonical, but after parse-unparse, the data *IS* cannonical. Hence, a Parse-Unparse-Parse is required before the Infoset matches. 

However, very few tests require this. Those should be explicitly identified by changing the TDML roundTrip='true/false' attribute into an enumeration:

roundTrip="false/true/simple/multiPass"

The values true and simple mean the same thing. The value multi-pass means that a parse-unparse-parse cycle is needed for a parse test, and an unparse-parse-unparse cycle is needed for an unparse test.

TBD: are longer cycles actually needed? If so then tests that require P-U-P-U should be distinghished from those that require only P-U-P, perhaps by changing the enums for round trip to "twoPass" and "threePass". 

By specifying this more specific need for passes, the intent of the test writer is clearer.

This also avoids cascading errors where a test that should not require multiple passes, is failing, multiple-passes are attempted, and the failure one observes is the multi-pass failure. This can just be some artifact, and not have much useful value when debugging. You want the first failure, in the case of a simple round-trip test. You don't want it to try multi-pass looping.

Right now I have 140-or-so failures in our TDML-based test suite (daffodil-test). I'd very much like to know which of these are simple failures, and debug those, and ignore the multi-pass failures for now, but I have no way of distinguishing them currently. 

This change could also improve test time - because re-running tests in the multi-test loops wouldn't even be attempted for tests that don't require it. 




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)