Tips on Using JUnit for Testing in Java

Recently took a short test task on a blog prototype, here are notes about tips on applying JUnit into Testing, especially Web Testing. As a new comer to advanced JUnit features, this post is also a track of learning. The current topics include tips about using @Rule, customize test strategy, to parameterize test, hints on Java Lambda, JUnit 5 new features, et.

Structure and Test Pyramid

Chart drafted with sketchboard.io

Above is the concept mapping from Martin Fowler Test Pyramid. There are still other verticle dimension test as performance test with JMeter (or curl, JPerf, et.), as OWASP test with ZAP. And in the time dimension, which is managed by Maven from drafting, coding, building, testing to deployment.

Therefore the three dimenssions could be measured:

Chart drafted with draw.io

Space dimession: Tests happen in different levels with different parameter scopes, exploring depth (test strength):
- JUnit cases;
- HTTP Client URL/Params;
- Selenium PageObjects and activities;
Logic dimession: Controlled by JUnit Runner and Cucumber;
- States could be: Igored, Pass, Failure, or, Pending (during an execution);
- Relationship between cases: Execution dependency, extra resources, parameterized suites, et;
- Weights on suites or cases;
- Criticality on discovered issues (e.g. processing ExceptionCollector in JUnit)
Time dimenssion: Maven Lifecycle Management and the history quality track (for data analysis and cost/operation prediction).
- Maven Configuration, agile deployment, supports to DevOps;
- History track and data analysis;
- Iteration Planning supports.

Typically JUnit is not recognized as Integration or Performance testing tool. However, JUnit could be utilized as a test suite management/report tool in scenario or integration test as well.

Following sections track the tips and hints for applying JUnit and following sections are updated lively with corrections, new findings, or, thoughts.

1. Applying @Rule

Annotation Rule is introduced with several builtin samples with JUnit4.

TestName
An example is to ignore test method in @Before. The simple way is to check test method name with @Rule annotation.

//...In JUnit test class
public static List<String> testListByPassBeforeHook = Arrays.asList(
        "$YourTestMethodName"
);
    
@Rule
public TestName name = new TestName();

//...In test setup
@Before
pubic void casualNameTest(){
	String testName = name.getMethodName();
	//Checksteps against name.
	if (testListByPassBeforeHook.contains(testName)){
		//special steps
		return; //avoid further init steps.
	}
	// further init steps as PageObject building.
}

2. Execution Sequence

This section is the relatively easy part of the article. JUnit test execution sequence could be explained as below. The order among T1, T2, .., Tx, .., Tn is by default up to JVM reflection API (Ref to Ref-1). Annotation @FixMethodOrder isis offered with v4.11+ to customize test case execution order. For example, name ascending order could be specified by @FixMethodOrder(MethodSorters.NAME_ASCENDING)

@BeforeClass –> 
	// See MethodSorters for more order preference.
	@Before –> @Test T1 –> @After 

	@Before –> @Test T2 –> @After 

–> @AfterClass

3.Using Test Runner

Parameterized Test with Runner

Customized Test Suite with Runner

Execute Cucumber Specs with Runner

4.Three Ways of Negative Test with Exception

Using @Test annotation parameter from JUnit4

@Test(expected = YourException.class)

Applying a @Rule of ExpectException

@Rule could hanlde RuntTimeException, however, above @Test annotation parameter could only capture Exceptions.

Another benefit is that @Rule could check the exception details.

Traditinal Try-Catch block

(Pay attention not to forget the fail() in final{} block)

5.Test the output stream

(TBC)

6.Test Http Level and Web API with OkHttp

Selenium is designed as a web test tool. It does not intend to support Http level verification, neither an API test tool. This is well discussed on selenium github about whether to add HTTP response header extraction, https://github.com/seleniumhq/selenium-google-code-issue-archive/issues/141.

Therefore, to complete the pyramid shap, we shall find a robust but efficient Http client test utility to build the low level coverage. OkHttp is one of the choice.

The major benefits of applying OkHttp include:

HTTP/2 support allows all requests to the same host to share a socket.
Connection pooling reduces request latency (if HTTP/2 isn’t available).
Transparent GZIP shrinks download sizes.
Response caching avoids the network completely for repeat requests.

An example to test against JSON WebAPI:

@Test
public void httpFreegeoipJsonTest() throws Exception {
    Request request = new Request.Builder()
            .url(Constants.Freegeoio_Url_Json)
            .build();
    Response response = client.newCall(request).execute();

    if (!response.isSuccessful()) {
        throw new IOException("Service Error" + response);
    }

    Assert.assertEquals(200, response.code());
    Assert.assertEquals("application/json", response.body().contentType().toString());

    JSONObject jsonResp = new JSONObject(response.body().string());
    String ip = jsonResp.getString("ip");
    Assert.assertTrue(checkIpv4(ip));

    //Debug dump without verification.
    Headers respHeaders = response.headers();
    respHeaders.names().forEach(n -> logger.debug(ColorPrint.green("header name:" + n + ", value:" + respHeaders.get(n))));
    logger.debug(ColorPrint.blue(strResp));
}

The sample code works with log4j configuration record log4j.logger.org.maxwu.jrefresh.HttpApi=DEBUG.

7.Other Tests

Web Performance Test

"ab" is Apache Benchmark Testing Tool.

A kick-start sample as Link

With gnuplot, the test result data could generate a chart.

JMeter

JMeter is widely used as performance test tool to execute preconfigured test sceanrio files.

Locust

Locust is a Python based web performance test toolbox with an embedded Web GUI to execute the specified Python script. It is progammer friendly and requests Python Web Programming fundamentals.

OWASP Web Security Test

RESTful CRUD Model Test

[Updated by Feb15, several sections TBD]

[Updated by Jan23, items TBD]

Backup

JUnit	Selenium	Maven

[EOF]