Home » Java » java – run sequential tests from parallel sets-Exceptionshub

java – run sequential tests from parallel sets-Exceptionshub

Posted by: admin February 25, 2020 Leave a comment

Questions:

From one list of input, I make unique sets of inputs. All sets should run in parallel, but each set itself needs to run the test sequentially, with permutations of the set as input for the test.

How’s it’s set up:

DataProvider takes in a list of devices (the number of devices varies). It makes n-size sets of devices, say 3 devices per set.
Each set of devices is input for the same Factory method for the same test case. (This part should be parallel)
The Factory method then makes permutations of this set, and executes the test case for each permutation. (This part should be sequential)

For example:

List of devices as input to DP: {A, B, C, D, E, F}

DP creates sets of 3 devices: {A, B, C} and {D, E, F}

{A, B, C} is input for Factory. Factory makes: {A, B, C}, {B, C, A}, and {C, A, B}.

Test case should run first for: A, B, C

Then test case should run again for B, C, A

And run a last time for C, A, B

In parallel,

{D, E, F} is input for the same Factory as above. Factory makes: {D, E, F}, {E, F, D}, and {F, D, E}

As above, the test case cycles through the permutations sequentially.

What I expect is that each Factory runs in a thread, and that each factory makes the permutations and runs the permutations sequentially. Factories should be running parallel.

First question: Is this possible? I’m still fairly new to TestNG, so maybe it just isn’t.

Second question:(dependsOnQuestion=”First question”) – How? I might not have this structured in the best way as-is. Using threads is a possibility, bit it also seems like something TestNG should already be handling.

In the XML, I’ve tried changing what is run in parallel. For suite, it’s all 1 thread (no surprise there). For Test, it only makes 1 separate thread. For Classes, it makes separate threads seemingly for each @Test-annotated method, but the final thread runs all remaining methods. Methods runs each method in one of the 3 threads allocated in the XML, but each Factory is still in the single “main” thread. Of course, if I just remove the parallel param, everything runs in a single thread.
group-by-instances needs to be true so each TC’s step execute before the next instance of the TC runs.
I’m also playing with the structure – if I find a solution, I’ll post an update.

//with parallel "tests"
DP thread: main
Creating a permutation from: A, B, C
Factory thread: main
Set: A, B, C
Set: B, C, A
Set: C, A, B
Creating a permutation from: D, E, F
Factory thread: main
Set: D, E, F
Set: E, F, D
Set: F, D, E
TestNG-tests-1: step 1
TestNG-tests-1: step 2
TestNG-tests-1: step 3
TestNG-tests-1: step 4
TestNG-tests-1: step 1
TestNG-tests-1: step 2
TestNG-tests-1: step 3
TestNG-tests-1: step 4
(repeats 4x)
//with parallel "classes"
DP thread: main
Creating a permutation from: A, B, C
Factory thread: main
Set: A, B, C
Set: B, C, A
Set: C, A, B
Creating a permutation from: D, E, F
Factory thread: main
Set: D, E, F
Set: E, F, D
Set: F, D, E
TestNG-test=Dev test-1: step 1
TestNG-test=Dev test-2: step 2
TestNG-test=Dev test-3: step 3
TestNG-test=Dev test-3: step 4
TestNG-test=Dev test-3: step 1
TestNG-test=Dev test-3: step 2
TestNG-test=Dev test-3: step 3
TestNG-test=Dev test-3: step 4
(repeats 4x with thread 3 running remaining iterations)

Expected/Desired output:

DP thread: main
Creating a permutation from: A, B, C
Factory thread: 1
Set: A, B, C
Set: B, C, A
Set: C, A, B
Creating a permutation from: D, E, F
Factory thread: 2
Set: D, E, F
Set: E, F, D
Set: F, D, E
TestNG-test=Dev test-1: step 1
TestNG-test=Dev test-1: step 2
TestNG-test=Dev test-1: step 3
TestNG-test=Dev test-1: step 4 //completed TC for A, B, C
TestNG-test=Dev test-1: step 1
TestNG-test=Dev test-1: step 2
TestNG-test=Dev test-1: step 3
TestNG-test=Dev test-1: step 4 //completed TC for B, C, A
TestNG-test=Dev test-1: step 1
TestNG-test=Dev test-1: step 2
TestNG-test=Dev test-1: step 3
TestNG-test=Dev test-1: step 4 //completed TC for C, A, B
TestNG-test=Dev test-2: step 1
TestNG-test=Dev test-2: step 2
TestNG-test=Dev test-2: step 3
TestNG-test=Dev test-2: step 4 //completed TC for D, E, F
TestNG-test=Dev test-2: step 1
TestNG-test=Dev test-2: step 2
TestNG-test=Dev test-2: step 3
TestNG-test=Dev test-2: step 4 //completed TC for E, F, D
TestNG-test=Dev test-2: step 1
TestNG-test=Dev test-2: step 2
TestNG-test=Dev test-2: step 3
TestNG-test=Dev test-2: step 4 //completed TC for F, D, E
(parallel output, so not necessary in order as above)

XML:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xml>
<suite name="Suite" parallel="suites" thread-count="3" data-provider-thread-count="3" verbose="1">
     <test name="Dev test" group-by-instances="true"> 
        <classes> 
            <class name="package.TestCase1Factory" />
<!--            <class name="package.TestCase2Factory" /> -->
<!--            <class name="package.TestCase3Factory" /> -->
<!--  will eventually have other test cases in this suite -->
        </classes> 
    </test> 
</suite>

DP:

public class DP_forFactory {

    @DataProvider (name = "setsAndNum", parallel = true)
    public static Object[][] testInput3() throws IOException {

        System.out.println("DP thread: "  + Thread.currentThread().getName());

        //just some sample data. Each row could also just be {A, B, C}, etc
        Object[][] output = new Object[][] {
            {new String[] {"A", "B", "C"}, "Parallel set 1"},
            {new String[] {"D", "E", "F"}, "Parallel set 2"},   
        };      
        return output;
    }
}

Factory:

public class TestCase1Factory {
    private static final int DEVICES_PER_TEST = 3;

    @Factory (dataProviderClass = DP_forFactory.class, dataProvider = "setsAndNum")
    public Object[] createTCInstances(String[] deviceSet, String setNum) {

        System.out.println("Creating a permutation from: " + deviceSet[0] + ", " + deviceSet[1] + ", " + deviceSet[2]);
        System.out.println("Factory thread: " + Thread.currentThread().getName());

        Object[] sequentialSet = new Object[deviceSet.length];

        //create permutations
        for (int i = 0; i < DEVICES_PER_TEST; i++) {
            sequentialSet[i] = new TestCase1(deviceSet[i], deviceSet[(i+1) % DEVICES_PER_TEST], deviceSet[(i+2) % DEVICES_PER_TEST]);
        }       
        return sequentialSet;
    }   
}

Test Case (Class):

//this is the part that receives the @Factory input
public class TestCase1 {

    //these are specific to the TC, so don't need to have a flexible setup
    private String role1;
    private String role2;
    private String role3;

    public TestCase1(String dev1, String dev2, String dev3) {
        this.role1= dev1;
        this.role2= dev2;
        this.role3= dev3;
        System.out.println("Set: " + role1 + ", " + role2 + ", " + role3);
    }

    @Test
    public void step1() {
        System.out.println(Thread.currentThread().getName() + ": step 1");
    }

    @Test (dependsOnMethods = "step1")
    public void step2() {
        System.out.println(Thread.currentThread().getName() + ": step 2");
    }

    @Test (dependsOnMethods = "step2")
    public void step3() {
        System.out.println(Thread.currentThread().getName() + ": step 3");
    }

    @Test (dependsOnMethods = "step3")
    public void step4() {
        System.out.println(Thread.currentThread().getName() + ": step 4");
    }
}
How to&Answers: