[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Does NeedsRunner work on parameterized tests?

Upon digging in, it appears to be a JUnit issue. Did it ever work? The includes/excludes are passed along from Gradle to JUnit. Fixed in the as-yet-still-beta 4.13 (notably in RC stages since Feb, and declared to be the last release of JUnit 4).


On Tue, Dec 18, 2018 at 12:45 PM Scott Wegner <scott@xxxxxxxxxx> wrote:
We hit a similar issue when trying to refactor some tests to use the 'Enclosed' runner. It seems the Gradle test filtering is not compatible with the Enclosed runner See BEAM-4375 [1] and pr/5440 [2]. 

I'm not familiar with the Parametrized runner, but perhaps it has similar issues.


On Mon, Dec 17, 2018 at 6:15 PM Kenneth Knowles <kenn@xxxxxxxxxx> wrote:
Also FWIW when I tried to repro this, I consistently got many failures and a hang when I tried to run the whole suite via `./gradlew needsRunnerTests`. Filed with some details.


On Mon, Dec 17, 2018 at 8:54 PM Reuven Lax <relax@xxxxxxxxxx> wrote:
It looks like maybe this is a JUnit bug:

The fix is targeted for JUnit 4.13, which is still not fully released (and is intended to be the final JUnit 4 release). 4.13.-beta1 is out though, and it might be worth trying that out to see if it fixes this. Does anyone know how we control the version of JUnit used?

On Mon, Dec 17, 2018 at 5:39 PM Reuven Lax <relax@xxxxxxxxxx> wrote:
Ok, this is worrying - we're silently skipping some important test suites. Does anyone know what might be causing us to skip them?

On Mon, Dec 17, 2018 at 5:03 PM Kenneth Knowles <kenn@xxxxxxxxxx> wrote:
Flipping through the latest Java PreCommit Cron, it does look like NeedsRunner tests within the Parameterized sub-suites of TextIOReadTest are not executed.


On Sat, Dec 15, 2018 at 12:16 PM Reuven Lax <relax@xxxxxxxxxx> wrote:

I tried to switch one of our tests (AvroIOTest) over to be a parameterized test. However after doing so, I cannot get the NeedsRunner tests to execute! All attempts fail with the following:

No tests found for given includes: [](filter.includeTestsMatching)

I also can't find evidence that Jenkins will execute this either.

Am I missing something, or do our NeedsRunner tests no longer work for parameterized tests? If the latter this is concerning, as there are a number of NeedsRunner paramaterized tests in our codebase (e.g. the TextIO tests).