programming assignment parameterized unit tests with junit quickcheck

• Custom Social Profile Link

An introduction to property-based testing with JUnit-Quickcheck

Even though I consider having an extensive set of unit and unit integration tests a good thing in almost any case, there is still a fundamental shortcoming with automated checks that I feel should be addressed more often. Almost all checks I come across are very much example-based, meaning that only a single combination of input values is being checked. This goes for unit-, API-level as well as UI-driven tests, by the way. Of course, this is already far better than not checking anything at all, but how do you make sure that your check passes (and thus your application works for all, or at least for a considerable subset of all possible input parameter combinations?

In most cases, it is computationally impossible to perform a check for every combination of all possible input values (have you ever tried to work out what all possible values for a single string input parameter are?), so we need an approach that is able to generate a lot of (preferably random) input values that satisfy a set of predicates, and subsequently verify whether the check holds for all these input values and value combinations. Enter property-based testing (see this post on Jessica Kerr’s blog for an introduction to the concept).

What is property based testing? In short, property-based testing pretty much exactly addresses the problem described above: based on an existing check, randomly generate a lot of input parameter combinations that satisfy predefined properties and see if the check passes for each of these combinations. This includes using negative and empty parameter values and other edge cases, all to try and break the system (or prove its robustness, if you want to look at it from the positive side…). You can do this either manually – although this does constitute a lot of effort – or, and given the nature of this blog this is of course the preferred method, use a tool to do the menial work for you.

Why should you use property based testing? Again, the answer is in the first paragraph: because property-based testing will give you far more information about the functional correctness and the robustness of your software than executing mere example-based checks. Especially when combined with mutation testing , property-based testing you will likely have you end up with a far more powerful and effective unit- and unit integration test suite.

Tool: JUnit-Quickcheck So, what tools are available for property-based testing? The archetypical tool is QuickCheck for the Haskell language. Most other property-based testing tools are somehow derived from QuickCheck. This is also the case for JUnit-Quickcheck , the tool that will be used in the rest of this blog post. As the name suggests, it is a tool for the Java language, based on JUnit.

Our system under test: the Calculator class For the examples in this blog post, we will use the same Calculator class that was used when we talked about mutation testing . In particular, we are going to perform property-based testing on the add() method of this simple calculator:

An example-based unit test for the add() method of this admittedly very basic calculator could look like this:

Using JUnit-Quickcheck, we can replace this example-based unit test to a property-based test as follows:

The @Property annotation defines this test as a property-based test, while the @RunWith class-level annotation defines that these tests are to be run using JUnit-Quickcheck. Note that you can mix example-based and property-based tests in the same test class. As long as you run your test class using JUnit, all public void no-parameter methods annotated with @Test will be run just as plain JUnit would do, while all tests annotated with @Property will be run as property-based tests.

The trials = 5 attribute tells JUnit-Quickcheck to generate 5 random parameter values (also known as samples). Default is 100. The System.out.println call is used to write the generated parameter value to the console, purely for demonstrational purposes:

As you can see, JUnit-Quickcheck randomly generated integer values and performed the check using each of these values. The property passes, telling us that our add() method seems to work quite well, even with quite large or negative integers. This is information you wouldn’t get when using purely example-based tests. Sweet!

Constraining property values Purely random integers seem to work well for our simple add() method, but obviously there are cases where you want to define some sort of constraints on the values generated by your property-based testing tool. JUnit-Quickcheck provides a number of options to do so:

1. Using the JUnit Assume class Using Assume, you can define assumptions on the values generated by JUnit-Quickcheck. For example, if you only want to test your add() method using positive integers, you could do this:

When you run these tests, you can see that values generated by JUnit-Quickcheck that do not satisfy the assumption (in this case, three of them) are simply discarded:

2. Using the @InRange annotation With this annotation, you can actually constrain the values that are generated by JUnit-Quickcheck:

Contrary to the approach that uses the Assume class, where values generated by JUnit-Quickcheck are filtered post-generation, when you’re using the @InRange approach you will always end up with the required number of samples:

3. Using constraint expressions Here, just like when you’re using the Assume approach, values generated by JUnit-Quickcheck are filtered after generation by using a satisfies predicate:

The difference is that when the discard ratio (the percentage of generated values that do not satisfy the constraints defined) exceeds a certain threshold (0.5 by default), the property fails:

Depending on your preferences and project requirements, you can select any of these strategies for constraining your property values.

Additional JUnit-Quickcheck features JUnit-Quickcheck comes with some other useful features as well:

• Fixing the seed used to generate the random property values. You can use this to have JUnit-Quickcheck generate the same values for each and every test run. You may want to use this feature when a property fails, so that you can test the property over and over again with the same set of generated values that caused the failure in the first place.
• When a property fails for a given set of values, JUnit-Quickcheck will attempt to find smaller sets of values that also fail the property, a technique called shrinking. See the JUnit-Quickcheck documentation for an example.

Download an example project You can download a Maven project containing the Calculator class and all JUnit-Quickcheck tests that have been demonstrated in this post here . Happy property-based testing!

Guide to JUnit 5 Parameterized Tests

Last updated: June 27, 2024

Java applications have a notoriously slow startup and a long warmup time. The CRaC (Coordinated Restore at Checkpoint) project from OpenJDK can help improve these issues by creating a checkpoint with an application's peak performance and restoring an instance of the JVM to that point.

To take full advantage of this feature, BellSoft provides containers that are highly optimized for Java applications. These package Alpaquita Linux (a full-featured OS optimized for Java and cloud environment) and Liberica JDK (an open-source Java runtime based on OpenJDK).

These ready-to-use images allow us to easily integrate CRaC in a Spring Boot application:

Improve Java application performance with CRaC support

Modern software architecture is often broken. Slow delivery leads to missed opportunities, innovation is stalled due to architectural complexities, and engineering resources are exceedingly expensive.

Orkes is the leading workflow orchestration platform built to enable teams to transform the way they develop, connect, and deploy applications, microservices, AI agents, and more.

With Orkes Conductor managed through Orkes Cloud, developers can focus on building mission critical applications without worrying about infrastructure maintenance to meet goals and, simply put, taking new products live faster and reducing total cost of ownership.

Try a 14-Day Free Trial of Orkes Conductor today.

Azure Container Apps is a fully managed serverless container service that enables you to build and deploy modern, cloud-native Java applications and microservices at scale. It offers a simplified developer experience while providing the flexibility and portability of containers.

Of course, Azure Container Apps has really solid support for our ecosystem, from a number of build options, managed Java components, native metrics, dynamic logger, and quite a bit more.

To learn more about Java features on Azure Container Apps, you can get started over on the documentation page .

And, you can also ask questions and leave feedback on the Azure Container Apps GitHub page .

Whether you're just starting out or have years of experience, Spring Boot is obviously a great choice for building a web application.

Jmix builds on this highly powerful and mature Boot stack, allowing devs to build and deliver full-stack web applications without having to code the frontend. Quite flexibly as well, from simple web GUI CRUD applications to complex enterprise solutions.

Concretely, The Jmix Platform includes a framework built on top of Spring Boot, JPA, and Vaadin , and comes with Jmix Studio, an IntelliJ IDEA plugin equipped with a suite of developer productivity tools.

The platform comes with interconnected out-of-the-box add-ons for report generation, BPM, maps, instant web app generation from a DB, and quite a bit more:

>> Become an efficient full-stack developer with Jmix

DbSchema is a super-flexible database designer, which can take you from designing the DB with your team all the way to safely deploying the schema .

The way it does all of that is by using a design model , a database-independent image of the schema, which can be shared in a team using GIT and compared or deployed on to any database.

And, of course, it can be heavily visual, allowing you to interact with the database using diagrams, visually compose queries, explore the data, generate random data, import data or build HTML5 database reports.

>> Take a look at DBSchema

Get non-trivial analysis (and trivial, too!) suggested right inside your IDE or Git platform so you can code smart, create more value, and stay confident when you push.

Get CodiumAI for free and become part of a community of over 280,000 developers who are already experiencing improved and quicker coding.

Write code that works the way you meant it to:

>> CodiumAI. Meaningful Code Tests for Busy Devs

The AI Assistant to boost Boost your productivity writing unit tests - Machinet AI .

AI is all the rage these days, but for very good reason. The highly practical coding companion, you'll get the power of AI-assisted coding and automated unit test generation . Machinet's Unit Test AI Agent utilizes your own project context to create meaningful unit tests that intelligently aligns with the behavior of the code. And, the AI Chat crafts code and fixes errors with ease, like a helpful sidekick.

Simplify Your Coding Journey with Machinet AI :

>> Install Machinet AI in your IntelliJ

Do JSON right with Jackson

Download the E-book

Get the most out of the Apache HTTP Client

Get Started with Apache Maven:

Working on getting your persistence layer right with Spring?

Explore the eBook

Building a REST API with Spring?

Get started with Spring and Spring Boot, through the Learn Spring course:

Explore Spring Boot 3 and Spring 6 in-depth through building a full REST API with the framework:

>> The New “REST With Spring Boot”

Get started with Spring and Spring Boot, through the reference Learn Spring course:

>> LEARN SPRING

Yes, Spring Security can be complex, from the more advanced functionality within the Core to the deep OAuth support in the framework.

I built the security material as two full courses - Core and OAuth , to get practical with these more complex scenarios. We explore when and how to use each feature and code through it on the backing project .

You can explore the course here:

>> Learn Spring Security

Spring Data JPA is a great way to handle the complexity of JPA with the powerful simplicity of Spring Boot .

Get started with Spring Data JPA through the guided reference course:

>> CHECK OUT THE COURSE

1. Overview

JUnit 5 , the next generation of JUnit, facilitates writing developer tests with shiny new features.

One such feature is p arameterized tests . This feature enables us to execute a single test method multiple times with different parameters.

In this tutorial, we’re going to explore parameterized tests in depth, so let’s get started.

Further reading:

A guide to junit 5, using springjunit4classrunner with parameterized, introduction to junitparams, 2.  dependencies.

In order to use JUnit 5 parameterized tests, we need to import the  junit-jupiter-params artifact from JUnit Platform. That means, when using Maven, we’ll add the following to our pom.xml :

Also, when using Gradle, we’ll specify it a little differently:

3.  First Impression

Let’s say we have an existing utility function, and we’d like to be confident about its behavior:

Parameterized tests are like other tests except that we add the  @ParameterizedTest  annotation:

JUnit 5 test runner executes this above test — and consequently, the isOdd  method — six times. And each time, it assigns a different value from the @ValueSource array to the  number method parameter.

So, this example shows us two things we need for a parameterized test:

• a source of arguments , in this case, an int array
• a way to access them , in this case, the  number parameter

There is still another aspect not evident with this example, so we’ll keep looking.

4.  Argument Sources

As we should know by now, a parameterized test executes the same test multiple times with different arguments.

And we can hopefully do more than just numbers, so let’s explore.

4.1. Simple Values

With the  @ValueSource  annotation, we can pass an array of literal values to the test method.

Suppose we’re going to test our simple isBlank  method:

We expect from this method to return true for null for blank strings. So, we can write a parameterized test to assert this behavior:

As we can see, JUnit will run this test two times and each time assigns one argument from the array to the method parameter.

One of the limitations of value sources is that they only support these types:

• short (with the  shorts attribute)
• byte ( bytes attribute)
• int ( ints attribute)
• long ( longs attribute)
• float ( floats attribute)
• double ( doubles attribute)
• char ( chars attribute)
• java.lang.String ( strings attribute)
• java.lang.Class ( classes attribute)

Also, we can only pass one argument to the test method each time.

Before going any further, note that we didn’t pass null as an argument. That’s another limitation — we can’t pass null  through a  @ValueSource , even for String and Class .

4.2. Null and Empty Values

As of JUnit 5.4, we can pass a single  null  value to a parameterized test method using  @NullSource :

Since primitive data types can’t accept  null  values, we can’t use the  @NullSource  for primitive arguments.

Quite similarly, we can pass empty values using the  @EmptySource  annotation:

@EmptySource  passes a single empty argument to the annotated method.

For String arguments, the passed value would be as simple as an empty String . Moreover, this parameter source can provide empty values for Collection types and arrays.

In order to pass both  null  and empty values, we can use the composed  @NullAndEmptySource  annotation:

As with the @EmptySource , the composed annotation works for  String s,  Collection s, and arrays .

To pass a few more empty string variations to the parameterized test, we can combine  @ValueSource , @NullSource , and @EmptySource  together :

In order to run a test with different values from an enumeration, we can use the @EnumSource annotation.

For example, we can assert that all month numbers are between 1 and 12:

Or, we can filter out a few months by using the  names  attribute.

We could also assert the fact that April, September, June and November are 30 days long:

By default, the  names will only keep the matched enum values.

We can turn this around by setting the  mode attribute to EXCLUDE :

In addition to literal strings, we can pass a regular expression to the names attribute:

Quite similar to  @ValueSource , @EnumSource  is only applicable when we’re going to pass just one argument per test execution.

4.4. CSV Literals

Suppose we’re going to make sure that the toUpperCase() method from  String generates the expected uppercase value. @ValueSource  won’t be enough.

To write a parameterized test for such scenarios, we have to

• Pass an  input value  and  an  expected value to the test method
• Compute the actual result with those input values
• Assert  the actual value with the expected value

So, we need argument sources capable of passing multiple arguments.

The @CsvSource is one of those sources:

The @CsvSource accepts an array of comma-separated values, and each array entry corresponds to a line in a CSV file.

This source takes one array entry each time, splits it by comma and passes each array to the annotated test method as separate parameters.

By default, the comma is the column separator, but we can customize it using the delimiter attribute:

Now it’s a colon-separated value, so still a CSV.

4.5. CSV Files

Instead of passing the CSV values inside the code, we can refer to an actual CSV file.

For example, we could use a CSV file like this:

We can load the CSV file and ignore the header column with  @CsvFileSource :

The  resources  attribute represents the CSV file resources on the classpath to read. And, we can pass multiple files to it.

The  numLinesToSkip  attribute represents the number of lines to skip when reading the CSV files.  By default,  @CsvFileSource  does not skip any lines, but this feature is usually useful for skipping the header lines  like we did here.

Just like the simple  @CsvSource , the delimiter is customizable with the  delimiter  attribute.

In addition to the column separator, we have these capabilities:

• The line separator can be customized using the  lineSeparator attribute — a newline is the default value.
• The file encoding is customizable using the encoding attribute — UTF-8 is the default value.

4.6. Method

The argument sources we’ve covered so far are somewhat simple and share one limitation. It’s hard or impossible to pass complex objects using them.

One approach to providing more complex arguments is to use a method as an argument source.

Let’s test the  isBlank  method with a  @MethodSource :

The name we supply to  @MethodSource needs to match an existing method.

So, let’s next write provideStringsForIsBlank , a  static  method that returns a Stream  of Argument s :

Here we’re literally returning a stream of arguments, but it’s not a strict requirement. For example, we can return any other collection-like interfaces like  List.  

If we’re going to provide just one argument per test invocation, then it’s not necessary to use the  Arguments  abstraction:

When we don’t provide a name for the  @MethodSource , JUnit will search for a source method with the same name as the test method.

Sometimes, it’s useful to share arguments between different test classes. In these cases, we can refer to a source method outside of the current class by its fully qualified name:

Using the  FQN#methodName format, we can refer to an external static method.

Using a method as the argument source proved to be a useful way to supply the test data. Consequently, starting with JUnit 5.11, we can now use a similar feature with static fields, through the experimental annotation @FieldSource :

As we can see, the annotation points to a static field referencing the test data, which can be represented as a Collection , an Iterable , an object array, or a Supplier< Stream>. After that, the parameterized test will be executed for each test input. Similar to @MethodSource , if the name of the static field matches the name of the test, the value of the annotation can be omitted :

4.8. Custom Argument Provider

Another advanced approach to pass test arguments is to use a custom implementation of an interface called  ArgumentsProvider :

Then we can annotate our test with the  @ArgumentsSource  annotation to use this custom provider:

Let’s make the custom provider a more pleasant API to use with a custom annotation.

4.9. Custom Annotation

Suppose we want to load the test arguments from a static variable:

Actually, JUnit 5 does not provide this.  However, we can roll our own solution.

First, we can create an annotation:

Then we need to somehow consume the annotation details and provide test arguments. JUnit 5 provides two abstractions to achieve those:

• AnnotationConsumer  to consume the annotation details
• ArgumentsProvider  to provide test arguments

So, we next need to make the  VariableArgumentsProvider  class read from the specified static variable and return its value as test arguments:

And it works like a charm.

5. Repeatable Argument Source Annotations

In the previous section, we used various annotations to supply arguments for our parameterized tests. Starting with JUnit version 5.11, most of these annotations were improved and became repeatable. As a result, we can annotate a parameterized test multiple times with the same argument source annotation .

For example, we can use @MethodSource twice to execute the test with all the elements supplied by two different methods:

As we can see, this can be a convenient way of running the test with data coming from different sources. Here are all the argument source annotations supporting this feature:

• @ValueSource
• @EnumSource
• @MethodSource
• @FieldSource
• @CsvFileSource
• @ArgumentsSource

6. Argument Conversion

Now we know how to use various argument source annotations to supply primitive test data to our tests. Argument converters are a convenient way of mapping the primitive arguments of a parameterized test to more complex data structures .

6.1. Implicit Conversion

Let’s re-write one of those  @EnumTest s with a @ CsvSource :

This seems like it shouldn’t work, but it somehow does.

JUnit 5 converts the String  arguments to the specified enum type. To support use cases like this, JUnit Jupiter provides a number of built-in implicit type converters.

The conversion process depends on the declared type of each method parameter. The implicit conversion can convert the  String instances to types such as the following:

• LocalDate , LocalTime , LocalDateTime , Year , Month , etc.
• File  and  Path
• URL  and  URI
• Enum  subclasses

6.2. Explicit Conversion

We sometimes need to provide a custom and explicit converter for arguments.

Suppose we want to convert strings with the  yyyy/mm/dd   format to  LocalDate instances.

First, we need to implement the ArgumentConverter interface:

Then we should refer to the converter via the  @ConvertWith  annotation:

7. Argument Accessor

By default, each argument provided to a parameterized test corresponds to a single method parameter. Consequently, when passing a handful of arguments via an argument source, the test method signature gets very large and messy.

One approach to address this issue is to encapsulate all passed arguments into an instance of  ArgumentsAccessor  and retrieve arguments by index and type.

Let’s consider our Person class:

To test the fullName() method, we’ll pass four arguments:  firstName , middleName , lastName , and the  expected fullName . We can use the ArgumentsAccessor  to retrieve the test arguments instead of declaring them as method parameters:

Here, we’re encapsulating all passed arguments into an  ArgumentsAccessor  instance and then, in the test method body, retrieving each passed argument with its index. In addition to just being an accessor, type conversion is supported through  get* methods:

• getString(index)  retrieves an element at a specific index and converts it to  String — the same is true for primitive types.
• get(index)  simply retrieves an element at a specific index as an  Object .
• get(index, type)  retrieves an element at a specific index and converts it to the given  type .

8. Argument Aggregator

Using the  ArgumentsAccessor  abstraction directly may make the test code less readable or reusable. In order to address these issues, we can write a custom and reusable aggregator.

To do that, we implement the  ArgumentsAggregator  interface:

And then we reference it via the  @AggregateWith  annotation:

The  PersonAggregator  takes the last three arguments and instantiates a  Person  class out of them.

9. Customizing Display Names

By default, the display name for a parameterized test contains an invocation index along with a  String  representation of all passed arguments:

However, we can customize this display via the name attribute of the  @ParameterizedTest  annotation:

April is 30 days long surely is a more readable display name:

The following placeholders are available when customizing the display name:

• {index}  will be replaced with the invocation index. Simply put, the invocation index for the first execution is 1, for the second is 2, and so on.
• {arguments}  is a placeholder for the complete, comma-separated list of arguments.
• {0}, {1}, .. . are placeholders for individual arguments.

10. Conclusion

In this article, we explored the nuts and bolts of parameterized tests in JUnit 5.

We learned that parameterized tests are different from normal tests in two aspects: they’re annotated with the  @ParameterizedTest , and they need a source for their declared arguments.

Also, by now, we should know that JUnit provides some facilities to convert the arguments to custom target types or to customize the test names.

As usual, the sample code is available over on GitHub .

Looking for the ideal Linux distro for running modern Spring apps in the cloud?

Meet Alpaquita Linux : lightweight, secure, and powerful enough to handle heavy workloads.

This distro is specifically designed for running Java apps . It builds upon Alpine and features significant enhancements to excel in high-density container environments while meeting enterprise-grade security standards.

Specifically, the container image size is ~30% smaller than standard options, and it consumes up to 30% less RAM:

>> Try Alpaquita Containers now.

Explore the secure, reliable, and high-performance Test Execution Cloud built for scale. Right in your IDE:

Basically, write code that works the way you meant it to.

AI is all the rage these days, but for very good reason. The highly practical coding companion, you'll get the power of AI-assisted coding and automated unit test generation . Machinet's Unit Test AI Agent utilizes your own project context to create meaningful unit tests that intelligently aligns with the behavior of the code.

Assignment 4: Property-Based Testing

Due Wednesday, November 21

Goals for This Assignment

By the time you have completed this work, you should be able to:

• Use basic generation and testing functionality of an existing property-based testing library
• Write properties, and test code with these properties

Step-by-Step Instructions

Step 1: choose a language and property-based testing library.

For this assignment, template code in Java has been provided, which uses the junit-quickcheck library. However, you do not have to use this code; you may use any language and any existing property-based testing library. If you choose a different language or library, you'll need to port the template code over, though most of this should be straightforward (there isn't much implementation code). If you choose to port the code to another language or testing library, you will get a 30% bonus on this assignment. This bonus is intended as compensation for the extra work involved in porting the code, and a reward for exploring something we didn't cover in class.

If you're interested in working with another language/library, a good starting point is the Wikipedia page on QuickCheck . All the listed languages are annotated with citations corresponding to property-based testing libraries written in those languages. QuickCheck itself is a property-based testing library for the Haskell programming language , and introduced the idea of property-based testing in 2000 . While the Wikipedia page equates property-based testing and QuickCheck, QuickCheck is an instance of a property-based testing library , not property-based testing in and of itself.

If you choose to work with a different language / library, be sure to have a README.txt file which explains how to run your properties, ideally with a single command . I should not have to understand how to use your specific language and property-based testing library in order to run your code, and a single command for me to run should give me everything I need.

The rest of these steps assume you're using the provided template code, as opposed to porting it. You may need to tweak these steps for your own port. If you're not sure what you need to change, don't hesitate to ask.

Step 2: Download, Compile, and Test Needed Code

Download the template code here . This template code encapsulates a project that encodes natural numbers in Java, without ever using any of the built-in primitive types (i.e., no int , long , double , etc.). If you're interested, the particular encoding is based on the Peano arithmetic , though you shouldn't need this information.

The provided template code comes with the following:

• A complete, (hopefully!) bug-free implementation, underneath src/main/java/arithmetic/*
• A series of unit tests, in src/test/java/arithmetic/NatUnitTest.java . These tests make use of JUnit .
• A generator of natural numbers, in src/test/java/arithmetic/NatGenerator.java . This generator is written for the junit-quickcheck library.
• A single property-based test in src/test/java/arithmetic/NatPropertiesTest.java . This again makes use of junit-quickcheck , along with the aforementioned natural number generator.

The provided code should already compile with Maven ( mvn compile ). Similarly, the code should already be testable ( mvn test ), which includes the aforementioned property-based test. Make sure the above commands work before moving on. If the commands don't work, you may need to first edit pom.xml , as described in Assignment 2 .

Step 3: Write Properties, With Limitations

Write additional properties in src/test/java/arithmetic/NatPropertiesTest.java , so that the code is much better tested than it currently is. The following restrictions apply:

• You may not simply convert Nat into a more Java-friendly type like int . Your properties need to work directly with values of type Nat .
• Each property must take at least one Nat as an input, and this input must somehow be used in an assertion (e.g., assertEquals , assertTrue , etc.). You cannot simply write unit tests under the guise of properties.
• Collectively, your properties must get you 100% code coverage . If you're writing good properties, this should not be difficult to acheive. From my own code, 1-3 properties involving a given method were sufficient for this, and the same property usually used multiple methods. If you're not using Java, you can skip this requirement (I don't want to put you through the extra trouble of finding an appropriate code coverage library).

Step 4: Submit Everything

Zip up everything in your arithmetic directory, including the target directory. Be sure your target directory holds up-to-date coverage results. Name your zipfile arithmetic.zip , and submit it on Canvas . In the comments for the submission, list everyone you worked with, if applicable.

If you did not use Java or junit-quickcheck, be sure to submit a README.txt , as specified in Step 1 .

• Books Get Your Hands Dirty on Clean Architecture Stratospheric
• Contribute Become an Author Writing Guide Author Workflow Author Payment
• Services Book me Advertise
• Categories Spring Boot Java Node Kotlin AWS Software Craft Simplify! Meta Book Reviews

JUnit 5 Parameterized Tests

• January 29, 2023

If you’re reading this article, it means you’re already well-versed with JUnit .

Let me give you a summary of JUnit - In software development, we developers write code which does something simple as designing a person’s profile or as complex as making a payment (in a banking system). When we develop these features, we tend to write unit tests. As the name suggests, the main purpose of unit tests is to ensure that small, individual parts of code are functioning as expected. If the execution of the unit test fails for any reason, it means the functionality is not working as intended. One such tool available for writing unit tests is JUnit. These unit tests are tiny programs, yet so powerful and execute in a (Thanos) snap. If you like to learn more about JUnit 5 (also known as JUnit Jupiter), please check out - JUnit5 article here

Now that we know about JUnit. Let’s now focus on the topic of parameterized tests in JUnit 5. The parameterized tests solve the most common problems while developing a test framework for any old/new functionalities.

• Writing a test case for every possible input becomes easy.
• A single test case can accept multiple inputs to test the source code, helping to reduce code duplication.
• By running a single test case with multiple inputs, we can be confident that all possible scenarios have been covered and maintain better code coverage.

Development teams aim to create source code that is both reusable and loosely coupled by utilizing methods and classes. The way the code functions is affected by the parameters passed to it. For example, the sum method in a Calculator class is able to process both integer and float values. JUnit 5 has introduced the ability to perform parameterized tests, which enables testing the source code using a single test case that can accept different inputs. This allows for more efficient testing, as previously in older versions of JUnit, separate test cases had to be created for each input type, leading to a lot of code repetition.

Example Code

Just like the mad titan Thanos who is fond of accessing powers, you can access the power of parameterized tests in JUnit5 using the below maven dependency

Let’s do some coding, shall we?

Our First Parameterized Test

Now, I would like to introduce you to a new annotation @ParameterizedTest . As the name suggests, it tells the JUnit engine to run this test with different input values.

In the above example, the annotation @ValueSource provides multiple inputs to the checkEvenNumber() method. Let’s say we are writing the same using JUnit4, we had to write 2 test cases to cover the inputs 2 and 4 even though their result (assertion) is exactly the same.

When we execute the ValueSourceTest, what we see:

It means that the checkEvenNumber() method is executed with 2 input values.

In the next section, let’s learn about the various arguments sources provided by the JUnit5 framework.

Sources Of Arguments

JUnit5 offers a number of source annotations. The following sections provide a brief overview and an example for some of these annotations.

@ValueSource

It is one of the simple sources. It accepts a single array of literal values. The literal values supported by @ValueSource are: short, byte, int, long, float, double, char, boolean, String and Class .

@NullSource & @EmptySource

Let’s say when verifying if the user has supplied all the required fields (username and password in a login function). We check the provided fields are not null, not empty or not blank using the annotations

• @NullSource & @EmptySource in unit tests will help us supply the source code with null, empty and blank values and verify the behaviour of your source code.
• We can also combine the passing of null and empty inputs using @NullAndEmptySource .
• Another trick to pass null , empty and blank input values is to combine the @NullAndEmptySource and @ValueSource(strings = { " ", " " }) to cover all possible negative scenarios.

@MethodSource

This annotation allows us to load the inputs from one or more factory methods of the test class or external classes. Each factory method must generate a stream of arguments.

• Explicit method source - The test will try to load the supplied method.
• Implicit method source - The test will search for the source method that matches the test-case method name.
• Multi-argument method source - We must pass the inputs as a Stream of Arguments. The test will automatically map arguments based on the index.
• External method source - The test will try to load the external method.

This annotation will allow us to pass argument lists as comma-separated values (i.e. CSV String literals). Each CSV record results in one execution of the parameterized test. There is also a possibility of skipping the CSV header using the attribute useHeadersInDisplayName .

@CsvFileSource

This annotation lets us use comma-separated value (CSV) files from the classpath or the local file system. Similar to @CsvSource, here also, each CSV record results in one execution of the parameterized test. It also supports various other attributes - numLinesToSkip , useHeadersInDisplayName , lineSeparator , delimiterString etc.

Example 1: Basic implementation

src/test/resources/csv-file-source.csv

Example 2: Using attributes

src/test/resources/csv-file-source_attributes.csv

@EnumSource

This annotation provides a convenient way to use Enum constants as test-case arguments. Attributes supported -

• value - The enum class type, example - ChronoUnit.class

The ChronoUnit is an enum type that contains standard date period units.

@EnumSource will pass all 16 ChronoUnit enums as an argument in this example.

• names - The names of enum constants to provide, or regular expression to select the names, example - DAYS or ^.*DAYS$

@ArgumentsSource

This annotation provides a custom, reusable ArgumentsProvider . The implementation of ArgumentsProvider must be an external or a static nested class.

• External arguments provider
• Static nested arguments provider

Argument Conversion

First of all, imagine without Argument Conversion , we would have to deal with the argument data type ourselves.

Source method: Calculator class

If we have String arguments and the source method we are testing accepts Integers , it becomes our responsibility to make this conversion before calling the source method.

Different argument conversions made available by the JUnit5 are

• Widening Primitive Conversion

The parameterized test annotated with @ValueSource(ints = { 1, 2, 3 }) can be declared to accept an argument of type int, long, float, or double.

• Implicit Conversion

JUnit5 provides several built-in implicit type converters. The conversion depends on the declared method argument type. Example - The parameterized test annotated with @ValueSource(strings = "DAYS") converted implicitly to an argument type ChronoUnit .

• Fallback String-to-Object Conversion

JUnit5 provides a fallback mechanism for automatic conversion from a String to a given target type if the target type declares exactly one suitable factory method or a factory constructor. Example - The parameterized test annotated with @ValueSource(strings = { "Name1", "Name2" }) can be declared to accept an argument of type Person that contains a single field name of type string.

• Explicit Conversion

For a reason, if you don’t want to use the implicit argument conversion, then you can use @ConvertWith annotation to define your argument converter. Example - The parameterized test annotated with @ValueSource(ints = { 100 }) can be declared to accept an argument of type String using StringSimpleArgumentConverter.class which converts an integer to string type.

Argument Aggregation

@argumentsaccessor.

By default, each argument provided to a @ParameterizedTest method corresponds to a single method parameter. Due to this, when argument sources that supply a large number of arguments can lead to large method signatures. To solve this problem, we can use ArgumentsAccessor instead of declaring multiple parameters. The type conversion is supported as discussed in Implicit conversion above.

Custom Aggregators

We saw using an ArgumentsAccessor can access the @ParameterizedTest method’s arguments directly. What if we want to declare the same ArgumentsAccessor in multiple tests? JUnit5 solves this by providing custom, reusable aggregators.

• @AggregateWith

Implement the ArgumentsAggregator interface and register it via the @AggregateWith annotation in the @ParameterizedTest method. When we execute the test, it provides the aggregation result as an argument for the corresponding test. The implementation of ArgumentsAggregator can be an external class or a static nested class.

Since you have read the article to the end, I would like to give you a bonus - If you’re using assertion frameworks like - Fluent assertions for java you can pass the java.util.function.Consumer as an argument that holds the assertion itself.

JUnit5’s parameterized tests feature allows for efficient testing by eliminating the need for duplicate test cases and providing the capability to run the same test multiple times with varying inputs. This not only saves time and effort for the development team, but also increases the coverage and effectiveness of the testing process. Additionally, this feature allows for more comprehensive testing of the source code, as it can be tested with a wider range of inputs, increasing the chances of identifying any potential bugs or issues. Overall, JUnit5’s parameterized tests are a valuable tool for improving the quality and reliability of the code.

Pralhad Hadimani

As a professional software engineer, I have always loved to code. Here is my attempt to share knowledge, contribute to the greater community and engage with extraordinary people around the world. I strongly believe in one of the famous quotes by Stan Lee - "With great power comes great responsibility" .

Recent Posts

Guide to JUnit 5 Functional Interfaces

• July 12, 2024

In this article, we will get familiar with JUnit 5 functional interfaces. JUnit 5 significantly advanced from its predecessors. Features like functional interfaces can greatly simplify our work once we grasp their functionality.

Getting Started with Spring Security and JWT

• June 19, 2024

Spring Security provides a comprehensive set of security features for Java applications, covering authentication, authorization, session management, and protection against common security threats such as CSRF (Cross-Site Request Forgery).

Creating and Publishing an NPM Package with Automated Versioning and Deployment

• June 16, 2024

In this step-by-step guide, we’ll create, publish, and manage an NPM package using TypeScript for better code readability and scalability. We’ll write test cases with Jest and automate our NPM package versioning and publishing process using Changesets and GitHub Actions.

Make it correct, make it clear, make it concise, make it fast. In that order.

• Design Patterns
• Lambda Expressions

Testing Junit 4

Parameterized unit tests with junit 4.

If you are a governed by conscience developer, you write tests, period.

As any piece of code, well written tests should be clear, concise, easy to maintain and actually test something. As everything in life, those skills come with experience.

Today we will talk about parameterized unit tests , as they provide ways of making your unit test development more complete, less tedious, more readable and much easier to maintain.

It is worth noticing that JUnit 4 has not chosen a very flexible approach to parametrized tests. They improved significantly in JUnit 5 (which I will visit in a future post).

If you aim for the maximum power and flexibility, then you should take a look to JUnitParams (I will also visit this JUnit addon in a future post).

What kind of problems does a Parameterized Unit Test solves ?

You are aware of the DRY principle, however if you are not yet using parameterized tests, you may have found yourself in situations where you have to repeat the same code across several tests methods just to assert some outcome, based on a situation where only one, or a small number of field from the same object change.

That is precisely the problem solved by parameterized tests.

Let’s create a small, but illustrative problem and see how we can benefit from a parameterized unit test.

This is not a post about TDD, so if you are a unit test purist, please accept my apologies for the “code first” approach and for the lack of null checks.

We have a Purchase

The purchase can have 4 different statuses that are enumerated

We need to process the purchases. For the purpose we have a purchaseJob that will do the work. As long as the purchase has a COMPLETED status it will be processed and for all the other statuses it will not. In our example, we just return true or false, depending on the Purchase status.

So, what can we see from the code ?

A purchase can arrive to the method process() with any of the 4 possible status.

Depending on that status things will happen (although here as example, the only thing happening is returning a boolean).

We easily realize that if we want to test all the 4 possibilities we will have to write 4 test methods, however the only thing that changes is the status field of the purchase.

There is a better way, and lets go directly to it.

We start by creating a pretty standard unit test to deal with the positive outcome

If we where to create methods to test all the 4 possibilities, then we would need to write 3 additional @test methods, where the only difference amongst them would be the status of the Purchase Object. Imagine if we had 10 different statuses, or even if the process() method received 2 parameters, each with a couple of different possible status.

When we encounter this kind of situations, we can take advantage of parameterized tests.

We will do so to handle the negative outcomes.

Let’s dissect what we have.

We start by annotating the class with @RunWith(Parameterized.class) to ensure that JUnit will invoke the referenced class to run the tests instead of the default runner built into JUnit (see JUnit-runWith-JavaDoc )

The second step is to create a public static method annotated with @Parameterized.Parameters that returns a Collection of object arrays. I’ve used the Iterable interface which is the root interface in the collection hierarchy, but we could have chosen to return a List <Purchase> and, in our case, it would work without any problems.

The objects that are part of the collection returned from this dataProvider() method are the ones that will be supplied to all the test methods in the class.

Although not practical, in JUnit4 that’s how it works. The test methods will run as many times as the number of objects present in the collection. In other words, the runner will iterate trough each @test method as many times as the number of objects in the collection, and in every iteration one object from that collection will be supplied.

Bear in mind that, had we included the positive test method in this class, the test would continue to pass, as we are forcing the purchase to be in a COMPLETE status manually, however this positive test method would run 3 times, which is a completely waste of time and resources.

The final step is the not mandatory @Parameterized.Parameter(0) . The advantage of this annotation is that it will handle the injection of the parameter for us, therefore we do not need a constructor for the test class. If we choose not to use this annotation, then we will have to provide a constructor for the test class.

Note the number zero in the annotation. It is also not mandatory as it always defaults to zero and if you just type @Parameterized.Parameter , in this case it will work without any problem. I prefer to use it (although some IDE’s might complaint) , to be explicit that is the object present in the first array column. This means, obviously that we can have more than one parameter being supplied, and we will annotate their respective fields with @Parameterized.Parameter(1) , @Parameterized.Parameter(2) , and so forth, according to the column they live in the array.

• We only have one test method that covers 3 possible status.
• If one day we need to add a new status, we just need to add the purchase to the data() method.
• It’s small, clean and readable.
• Executes faster than 3 methods, due to the way JUnit handles the work internally.

and now for the cons :

• We should have one parameterized test class per method of the UUT (unit under test) . (JUnit 5 solves this).
• We can achieve the same results with less code by adding JUnitParams library to our projects.

I hope this little example is useful, and drives your curiosity to explore more about JUnit.

An example for the code in this post can be found in git-hub

Happy coding :)

JUnit Quickcheck Example

In this example we shall show users the usage of property based testing. JUnit quickcheck example will demonstrates the way to test the methods with the help of property based testing. There are cases when you want to test your methods with some constraints and with random parameter values.

We can achieve the same with manual process also, but in that case every test scenario may not be covered. You may want to test your cases to be passed with random values of parameters. This can be achieved by the property based testing.

This concept was evolved from the Haskell language and the part that covers it is known as QuickCheck . This is also available for Java as an extra library. We will be using the same in our example.

1. Introduction

First of all, lets start with the question. What is property based testing? The way which provides the solution for the above scenarios i.e. to test with random parameter values is known as property based testing. In Java, we can achieve this by using the junit-quickcheck library. This library will provides the way to test our cases with the property based testing.

2. Tools Used

We will use the following tools and technologies for this example and show how this works.

• junit-quickcheck – Library that will be used for property based testing
• Eclipse – IDE for code
• Maven – build and dependency tool

3. Project Setup

Create a new Maven project for JUnit quickcheck example.

Click File -> New -> Maven Project . Fill in the details as shown and click on Next button.

Fill in the details and click on Finish button.

With the click on Finish button, setup is done. Now we will start with the JUnit quickcheck example.

4. JUnit Quickcheck Example

Open pom.xml and add the following lines to the file.

In our example we are using some matchers to work with. For that purpose, we are using Java Hamcrest (highlighted above) library. To run the class with the quickcheck, we need to annotate the class with @RunWith(JUnitQuickcheck.class) annotation

QuickCheck.java

Line 14: Run with @RunWith(JUnitQuickcheck.class) annotation usage Line 17: @Property annotation usage Line 26: assumeThat() method usage Line 32: @InRange annotation usage Line 38: @When annotation usage Line 44: seed usage

In the following sections we will be explaining each and every case defined in this class. For the sake of reading and knowledge, we have used the println statements in the class.

4.1 Simple Unit Test

We will start by testing with very simple test case where we put @Property annotation on a method.

In this method, we have used @Property annotation with trials as attribute to it. By default junit-quickcheck library uses 100 random generated values. But we can increase or decrease accordingly as to suit the test cases. This test will run with 5 random numbers. The test may or may not pass, due the random numbers generated. In our case it was failed as some values are negative and 0 as well. See the output of above case.

4.2 Using Assume Class

Now we want to assume something before running our test cases. Like in above example, we want to generate only positive values. In that case we will use the Assume class from JUnit. The assumeThat() method will assume the values to be passed before any other values to be tested.

We will contact you soon.

It is cleared from output, cases that pass the assumeThat will only go through the case for testing. Others will be ignored safely.

4.3 @InRange Annotation

We can also test our cases by allowing the parameters to be in some range. For this scenario, we will be using @InRange annotation.

In this case, we are passing the @InRange annotation to set the minimum and maximum value to be generated.

As seen in this output, only numbers between 0 and 100 are generated. The primary difference between assumeThat() and @InRange is that, in assumeThat() case all numbers are generated and then values are tested. Whereas in case of @InRange , values are generated accordingly and then passed to test.

4.4 @When Annotation

@When is used in case where we wan to have a constraint. We will be using @When annotation with satisfies attribute which is an OGNL expression for this example to work.

This test case will pass, only if the parameter satisfies the condition. junit-quickcheck generates values for a property parameter with a constraint expression until the ratio of constraint failures constraint passes is greater than the discardRatio specified by @When . Exceeding the discard ratio raises an exception and thus fails the property. In most cases this test fails, so you need to increase your trials to more, may be around 2000.

4.5 Using Seed

For each property, junit-quickcheck uses a unique value as a seed for the source of randomness for generating values. We can fix the seed to some specific value with @When  annotation . You may want to fix the seed when a property fails, so that you can test the property over and over again with the same set of generated values that caused the failure.

Here every time test case runs, it will start generating number from seed value only.

4.6 Shrinking

When a property is disproved for a given set of values, junit-quickcheck will attempt to find “smaller” sets of values that also disprove the property, and will report the smallest such set.

4.6.1 Various parameters used with shrinking process

• By default, shrinking is enabled. To disable it, set the shrink attribute of a @Property annotation to false. Example: @Property(shrink = false)
• To reduce or increase the maximum number of shrink attempts made for a given property, set the maxShrinks attribute of that @Property . Example: @Property(maxShrinks = 5)
• To reduce or increase the maximum “depth” of the shrink search “tree” for a given property, set the maxShrinkDepth attribute of that @Property . Example: @Property(maxShrinkDepth = 3)

For the details of how to use it, users are advised to visit junit-quickcheck library.

5. Conclusion

In conclusion, JUnit Quickcheck example demonstrates the way to test our code by passing the random values to parameters during running. This method of testing is known as property based testing. Users have learnt the use of junit-quickcheck library for implementing the property based testing in Java.

6. Download

This is a JUnit Quickcheck Example

Vinod Kumar Kashyap

Related articles.

JUnit AssertThat Example

Junit mockito when thenreturn example, junit test void method example, junit setup / teardown example, junit code coverage, junit test constructor example, junit report generation example, junit no runnable methods.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

In Section “4.4 @When Annotation” the discardRatio can/should be ajusted as well. Otherwise this test will almost always fail.

Property-based testing in Java with JUnit-Quickcheck – Part 2: Generators

In Part 1 of this tutorial we created a Property-based test (PBT) from a normal JUnit test with basic types . Now let us extend the domain object PostalParcel with a list of Products.

All examples are written with Java 8 and can be downloaded from my gitlab repository.

Write an unit test for the function deliveryCosts with a fixture for the entities. An implementation could look something like this:

As explained in the previous part, these tests will not cover the entire behaviour we are testing here. We will refactor this to a Property-based test with the help of generators.

In this certain case we use entities as our input for our unit tests. A possible way of using generated properties for entities is to generate a basic type for each of the needed arguments for our entities. It is easier to let JUnit-Quickcheck generate the entities by creating your own generators; this way your test does not get bloated with properties and is therefore better readable. To create a generator you need to do three things. First create a class that extends from generator<T> :

Secondly, implement the constructor with a super to the type you are generating. Lastly, override the function T generate:

We can use the generator in our Property-based test in two ways, we can use the @From(T.class) annotation in front of your entity. This annotation tells Junit-Quickcheck which generator to use, for example:

It is much easier to let the serviceloader of JUnit-Quickcheck discover the generators by creating a file called com.pholser.junit.quickcheck.generator.Generator in META-INF/services. In this file you add the package name plus class name of the generator. This way, there is even less plumbing going on in your unit tests.

Implement generate

The function generate looks pretty straightforward. It requires you to return the type  you are generating.

There are multiple JUnit-Quickcheck functions going on here. First is the argument SourceOfRandomness. You need this to create random primitive types, like you see in generating the randomTotalWeight. Secondly, you have the argument GenerationStatus, which we do not use in this example, but we can use this to influence the results of a generation for a particular property parameter.

When you want to generate other types than primitives, use the  gen()  interface. The most common usages are:

• The gen().make() method makes the generators available to current instance, and configures it with whatever configuration annotations live on the generator class.
• The gen().type() method asks for an arbitrary generator that can produce instances of the given type. Use this for a string, it will create a total random string with all characters available (even Chinese and Korean characters).

Take notice, as was said in the previous part, if you take a string as argument then the works of Shakespeare in Japanese & Korean are ONE valid input. If you can, wrap the string into its own type, like explained in the  Object Calisthenics  practise.

Configuring generators

In the implementation of our PostalParcelGenerator you might have noticed the arguments minWeight and maxWeight in generating the randomTotalWeight. These are two variables which we need to configure the generator with. We can do this by creating a public function void called configure. JUnit-Quickcheck requires us to name it like this in order to match this with the annotation we are going to use. In this case we will use the standard JUnit-Quickcheck annotation @InRange , but you can also use your annotation.

You can easily configure your generators with the behaviour you need for your test cases. The only problem that remains is that where code is written errors can be made; we all make mistakes. This is also why in my previous post I advised to both use configuration by annotation and to use assumeThat; with this approach mistakes are easily spotted during your tests. After we implemented the rest of the generators, (which you can see in my gitlab repository) we can write our PBT.

Creating the generators will cost some startup investment, but afterwards you really benefit from it. For each PBT, you can just add the type of the generator, and maybe some added behaviour you want with it. The minor investment will result in time saving in the long run, and better designed and written software. Additionally the tests will find edge cases that could cause bugs in your software, and resolve them before they occur.

Performance

After my last post I got a comment with a question about performance. I do not know how the exact implementation works, but I do know that running it does not take 100x slower. I did a run in my Intellij for both the tests and the PBT is about 10x slower. In my opinion 10x is not unreasonable, considering the benefit it gives you.

This concludes part 2 of this tutorial, in the next part I will show you how to repeat failed tests.

Also, this post is published on the blog of  Xebia

Kenny Baas-Schwegler

As a socio-technical systems thinker, agile architect, and Domain-Driven Design expert, I work with CTOs, managers, architects, and teams to change how we design software. Through facilitating and doing collaborative modeling, I catalyze organizations, teams, and groups of people to an agile architecture approach to building sustainable quality software products.

Recent news

• How the bias towards additive can lead us to a suboptimal and costly software design
• Resilient Bounded Contexts: A Pragmatic Approach with Residuality Theory
• Coach your Architects in Agile Architecture!
• Remote collaborative modelling part 1: Check-in
• EventStorming; Core concepts, glossary and legend

Beyond Unit Tests: Property-based Testing

Expressing program behaviours.

In a previous article , we looked at the use of EvoSuite framework for automated test case generation in Java. As mentioned in that article, EvoSuite uses search-based and constraint-based methods for generation of test cases. These methods are guided by coverage criteria (e.g. branch coverage) and ability to explore different paths in the program. The test cases generated by these methods may not capture the intended behavior of the program. In today's article we will see how we can generate test cases that capture certain behaviors about programs. This can be done using property-based testing.

JUnit-QuickCheck is a library that provides property-based testing for Java programs. It is inspired by the QuickCheck library for Haskell that first pioneered this approach for automated testing. The library makes use of JUnit's Theory feature to support parameterized test cases. These test cases allow the developer to specify the property that the method under test should satisfy. JUnit-QuickCheck then uses randomly generated values to test the property. The following example shows how to use the @Theory annotation to specify a test method:

This unit test is calling the encode and decode functions of the EncodeBase64 class from the wox cross platform serialization library . The property of interest here is that encode is invertible , with decode as its inverse, i.e. x = decode(encode(x)) . In other words, we want to check that encoding a byte array and then decoding it back leads to the same byte array. The assertArrayEquals at the last line ensures that this property is satisfied. This property is tested by randomly generating a large number (100 by default) of byte arrays and calling the testEncodeBase64 with those values as input. The @ForAll annotation is provided by the JUnit-QuickCheck library and takes care of generating the appropriate random inputs.

If there are two inputs to the method, then all possible combinations of the randomly generated inputs are tested. In order to avoid running so many tests we can specify constraints on the input as shown below:

The assumeThat ensures that only byte arrays with length greater than 32 are generated. The library already comes with generators for all primitive Java types and there is also a separate module junit-quickcheck-guava containing generators for Guava types. However, if we need to generate inputs of custom type we need to provide a generator. It can be done by extending the Generator class and overriding the generate method. The following example shows one possible way to generate random inputs of the org.jdom2.Element type.

Every time the generate method is called, it creates a random alphabetic string that is used as the name of the element and adds up to 8 random attribute values in it. To use this generator for the Element type we need to specify the class with the @From annotation after the @Forall in the test method as shown below:

The use of custom generators allows us to use property-based testing for arbitrary classes and methods with little effort. The source code for all the tests is available under the wox repository on GitHub. In addition, some other frameworks (under active development) providing similar functionality for Java are Quickcheck and ScalaCheck . However, JUnit-QuickCheck is the only one to use the Theory support in JUnit which makes it a lot easier to integrate in the development workflow.

Checking Informal Specifications

Property-based testing shifts our focus from simple assertions about values to logical properties that code should satisfy. Another use is checking informal, natural-language specifications; translating these statements into logical terms allows us to leverage a property-based test to check them.

One of the building blocks we implemented building SourceClear was a library for interpreting CVSS , a set of metrics used to assess the severity of software vulnerabilities.

The textual encoding of a set of CVSS metrics is called a vector :

Vectors are structured, consisting of several metric groups .

Furthermore, there are many rules governing the relationships between metric groups and values, specified informally . Both of these make it a prime candidate for property-based testing.

We start by defining data structures and generators for vectors:

As this library parses and pretty-prints vectors, we test a simple property of the implementation : that parsing is invertible, with pretty-printing as its inverse:

So far so good.

CVSS also defines a means of turning a vector into a numeric score, allowing vectors to be ordered (and vulnerabilities prioritized). We transcribe the equations faithfully, then test a simple property from the spec relating the scores of the base and temporal metric groups:

Surprisingly, the third assertion fails and a counter-example is printed:

Our first thought is that the implementation is wrong in some way. Perhaps floating-point error? We check two other implementations of CVSS2 scoring and see the same result: a base score of 4.8, a temporal score of 4.0, and the impossible assertion 4.0 < 4.8 * (1-0.33) .

Looking at the temporal equation analytically, we see that the minimum temporal score is:

In other words, "33% lower than the base score" is the lower bound . Clearly the spec should have said no smaller than , i.e. base * (1-0.33) <= temporal <= base . Note also the inconsistent uses of higher and greater , and the difficulty of parsing that sentence in general; issues like these illustrate why natural language should be kept to a minimum in specifications.

We continue transcribing and testing, and find a similar error in the computation for the environmental score.

Here the equations are given in terms of both the base and temporal scores, and consequently the cause of the error is much less obvious, demonstrating the value of the property-based approach.

The interesting thing about this example is that it shows that once the implementation is correct (e.g. comparable to another implementation), the property-based test extends to a test of the specification .

This idea is very similar to grammar-based fuzzing, which we'll cover next.

results matching " "

No results matching " ".

Johannes Link

Software Therapist

• Heidelberg, Germany
• Custom Social Profile Link

Property-based Testing in Java: Jqwik - a JUnit 5 Test Engine

8 minute read

In the previous episode you’ve already seen jqwik in action. One of the interesting aspects of this PBT library is the fact that it’s not a standalone framework but that it hooks into JUnit 5 in order to “inherit” IDE and built-tool support.

Jqwik and the JUnit Platform

The fifth generation of JUnit does not only come with a modernized approach to write and execute tests, but it is based on the idea of providing a platform for a large spectrum of different test engines . An engine provides two entry points: One entry point is for discovering tests and test suites - e.g. through scanning parts of the classpath for methods with a certain annotation. The other entry point is used to run tests and test suites, usually the ones you have discovered during the discovery step. The rest, like filtering or selecting subsets of tests, is done by the platform itself.

The big advantage of such an approach is that any IDE and any build tool only has to integrate the platform and not the individual engines. It’s also a big plus for engine developers who don’t have to bother with aspects like public APIs for discovering and running their test specifications.

Moreover, the platform allows to have any number of engines in parallel. That’s how JUnit 5 provides full backwards-compatibility to JUnit 4 and how a smooth migration path can be realized. Using JUnit 4 (called Vintage), JUnit 5 (called Jupiter) and jqwik in a single project is not only possible, it’s really simple.

IntelliJ has been an early platform adopter for over a year now. As of March 2018, we also see native support from Eclipse, Gradle and Maven-Surefire. If you’re already using JUnit 5, using jqwik as additional engine requires a single additional dependency. If jqwik is your first contact with the platform you should check out this part in jqwik’s user guide .

Wildcards and Type Parameters

Let’s get back to the concrete property test of the previous episode:

You might miss the tiny change : Instead of a concretely typed List<Integer> I used the wildcard variant: List<?> . Actually, this reflects the precondition better, since the method under test - Collections.reverse() - should work with any element type. Under the hood jqwik will create instances of a special subtype of Object . Just run the property with reporting switched on.

This would, by the way, also work with a type variable instead of a wildcard. Upper or lower bounds, however, are not fully supported yet.

Many Parameters

You might have guessed that parameter generation is not restricted to a single one but works for as many as you need:

If you look at the generated lists you will notice that the variance in list size and string length is quite high. You can also see that now and then an empty list and an empty string is generated. This is due to the fact that value generation is not purely random and not equally distributed across the allowed domain. Instead, jqwik tries to be “smarter”:

Smaller values (for numbers, sizes and lengths) are generated more frequently than higher values. The exact distribution depends on an internal genSize parameter which by default is set to the number of tries. The more often a property is tried, the larger the generated values are - on average.

jqwik will also routinely inject typical border cases like empty lists, empty strings, maximum and minimum values and others depending on the type of the values to generate and additional constraints.

This smart generation approach aims at raising the probability to detect not so obvious specification gaps and implementation bugs.

Automatic Parameter Generation

Out of the box jqwik is able to generate objects of the most common JDK types:

• All primitive numerical types, their boxed counter parts, as well as BigInteger and BigDecimal
• String , Character and char
• Boolean and boolean
• All enum types
• List<T> , Set<T> , Stream<T> and Optional<T> as long as T can be generated
• Arrays of types that can be generated
• Map<K, V> and Map.Entry<K, V>
• java.util.Random and Object

You might notice that Map and all calendar related classes are not covered (yet). It’s quite easy, though, to provide and register generators yourself.

What’s also not covered are functional types. They are on the backlog but it’s not easy to come up with a good variance of functions to try. Constant functions are obvious, everything else not so much.

Influencing Automatic Parameter Generation

The easiest way to influence and constrain the domain of values considered for generation is to use additional annotations provided for many of the default types. Here are a few examples:

• All number types come with their respective range annotation. E.g. use @DoubleRange(min=5.0, max=10.0) to only generate doubles between 5 and 10.
• Strings can be constrained in both their length and the pool of character to be used. E.g. use @StringLength(min = 1, max = 5) @AlphaChars to generate strings of 1 to 5 characters with only upper and lower case letters.

Here is the full list of built-in constraining annotations .

Programmatic Generation

Sometimes we’re dealing with classes that cannot be generated by default. On other occasions the domain-specific constraints of a primitive type is so specific that the existing annotations are not powerful enough. In these cases you can delegate provision of parameter generators to another method in your test container class. The following example shows how to generate German zip codes:

The String value of the @ForAll annotation serves as a reference to a method within the same class (or one of its superclasses or owning classes). This reference refers to either the method’s name or the String value of the method’s @Provide annotation. The providing method has to return an object of type @Arbitrary<T> where T is the static type of the parameter to be provided. Arbitrary is a bit more than a value generator; think of it as a “configurator for value generators”.

Parameter provision methods usually start with a static method call to Arbitraries , maybe followed by one or more filtering, mapping or combining actions as described in the next section.

Filter and Map

As the core type of all value generation Arbitrary has quite a few default methods that can be used to modify generating behaviour. You usually start with one of the static basic generator functions in class Arbitraries . Most base generators return a specific subtype of Arbitrary that gives you additional constraining possibilities.

Let’s say we want to generate integers between 1 and 300 that are multiples of 6 . Here are two alternatives to do that:

• Arbitraries.integers().between(1, 300).filter(anInt -> anInt % 6 == 0)
• Arbitraries.integers().between(1, 50).map(anInt -> anInt * 6)

Which way is better? Sometimes it’s only a matter of style or readability. Sometimes, however, the way you choose can influence performance. When comparing the two options above, the former is close to the given spec but it will - through filtering - throw away five sixths of all generated values. The latter is therefore more efficient but also less comprehensible when coming from the spec. Usually generating primitive values is so fast that readability trumps efficiency.

Combining Arbitraries

Real domain objects often have several distinct and mostly unrelated parts. That’s why - when you generate them - you want to start from unrelated base generators.

Given our domain class Person :

We want to write a property which checks that any person has a full name. The person generator method combines three arbitraries into the needed one:

Fighting Indeterminism

The values generated during a test run are random - at least to a large degree. That way you enhance the chance of hitting upon bugs and specification gaps that no one considered during conception and implementation. The downside of introducing chance into testing is inherent indeterminism: A falsified property in this run could succeed in the next since different test data might lead to different test results.

There are two things jqwik does to keep this problem in check:

• To fix a certain suite of generated values, you can set the random seed from a falsified test run directly in the property annotation:
• If you run your tests from an IDE or through a build tool from the command line, jqwik remembers the seeds from falsified properties. That means that until you get rid of the problem, e.g. by fixing the bug, you will always see the same suite of generated values.

Other PBT Frameworks and Libs for Java

jqwik being a JUnit 5 test engine requires you to use the JUnit platform. If you cannot or do not want to use JUnit 5 yet, there are a few alternatives for doing PBT on the JVM:

JUnit-Quickcheck : This is where jqwik has stolen the annotation approach. So the way you write your properties is similar. JUnit-Quickcheck is tightly integrated with JUnit 4 and also supports shrinking.

QuickTheories : QuickTheories is completely test framework independent, which has pros and cons. Besides shrinking it also has an experimental feature in which value generation is being influenced by coverage data.

Vavr : The functional library for Java also comes with a property-based testing module . Currently no shrinking support, though.

ScalaCheck : Definitely a mature property based testing system with shrinking and all, iff you prefer Scala over Java. There’s also a book out there about ScalaCheck.

test.check for Clojure : Strongly inspired and influenced by QuickCheck. Since Clojure does not have static types generators must always be declared explicitly.

KotlinTest also has basic support for PBT. Currently no shrinking yet.

Frege, a Haskell for the JVM, comes with a classical QuickCheck implementation. This article from Dierk König’s Frege book provides a short introduction.

Next Episode

The next article will focus on a crucial feature of mature Property-based Testing libraries: Shrinking .

Leave a comment

• Java Course
• Java Arrays
• Java Strings
• Java Collection
• Java 8 Tutorial
• Java Multithreading
• Java Exception Handling
• Java Programs
• Java Project
• Java Collections Interview
• Java Interview Questions
• Spring Boot

JUnit 5 – How to Write Parameterized Tests

By using parameterized tests, we can reuse the single test configuration between multiple test cases. It will allow us to reduce the code base and easily verify multiple test cases without the need to create a separate test method for each one. This article will describe how we can develop parameterized tests using the JUnit 5 framework. We will go through the following steps:

• Setup the required dependencies with Maven and Gradle build tools
• Create the first parameterized test
• Discover arguments sources for simple data types

Before we start, please make sure that you’re familiar with the following topics:

• Creating software projects using either Maven or Gradle build tools
• Writing unit tests using Java and the JUnit 5 framework 

Dependency setup

JUnit 5 doesn’t provide support for running parameterized tests out of the box. To enable this feature, we have to include the junit-jupiter-params dependency into our project. In case you’re using Maven, you need to include such a code snippet in your pom.xml file:

The full example of the pom.xml file:

Gradle users need to add this dependency to the test implementation dependency section. It can be done by adding the following code block to the build.gradle file:

Creating a parameterized test

Let’s create a simple test service for use in our first parameterized test. We’ll need something primitive, and I will suggest going with the phone validation service, which accepts a single phone number as a String and validates it against some regex. Please refer to the following example below:

Our method for validating phones should return true in the case the provided phone number is not null and matches the regex, and false otherwise. To create simple unit tests with JUnit we use the @Test annotation. In the case of parameterized tests, we need to use the @ParameterizedTest annotation instead. Besides that, we need to provide an argument source — this is the structure that will hold our test arguments (phone numbers in our case). Please refer to the whole test class below, and then we will go through it in detail.

As was mentioned above the @ParameterizedTest simply replaces the @Test annotation. But since it’s a parameterized test, we have also added another annotation @ValueSource. It has an array of strings with valid phone numbers in the first method and nonvalid ones in the second one. Each string is a separate test case with which our test will run. Also, you may have noticed one last obvious difference with non-parameterized tests — our test method has a parameter in its signature, the phone number of type String in our case. Running this test class will give us the following execution results:

Execution results of the ValueSourceExampleParameterizedTest class

We may specify arrays of strings, classes, shorts, bytes, ints, longs, floats, doubles, chars, and booleans in the @ValueSource annotation. However, it supports only one type of input. Because of this, we’re not able to specify several test arguments using this annotation, so it makes sense to use it only for simple test cases. This article focuses only on simple data types and argument sources for parameterized tests. All available argument sources may be found in the org.junit.jupiter.params.provider package. Let’s review a few more simple argument source annotations, which may come in handy.

Using @NullSource and @EmptySource

We often need to test our code with null and empty values to be assured that we can process them correctly and avoid the most loved `NullPointerException`. Parameterized test argument sources provide us the built-in support for such test data with  @NullSource and @EmptySource annotations. Let’s modify our parameterized tests for the phone validation services and see how it works:

As you can see, we have simply added these additional annotations besides @ValueSource. This change will add two additional test cases (null and empty values) to our method for testing invalid phone arguments. This how the execution results will look like for our updated test class:

Execution results of the ValueNullAndEmptySourceExampleParameterizedTest class

Also, we can replace  @NullSource and @EmptySource annotations with a single one @NullAndEmptySource , which combines these two for our convenience. Next, we will review one more useful argument source, which may be used for simple test cases.

Using @EnumSource

Let’s assume that we have a method that receives some enum value as a parameter and performs some operations based on the value of this parameter. For example, the method for sending messages through different channels:

The enum class with possible channel values and the Message class will look like this:

It will make sense to test this method’s behavior with all possible channel types. And with parameterized tests, we can do this with a single test method, instead of creating a separate one for each channel type. This will be the case for using our next possible arguments source — @EnumSource . Let’s firstly look at the code example and then analyze it:

To make it more like a real-world test class, we have initiated the ‘MessageService’ class and created a separate method called ‘createMessage’ which will create a test message for us based on the channel type. We’re using the builder pattern implementation provided by Lombok to create a message, please check it here if you’re not familiar with it. The @ValueSource annotation from the previous example was replaced with @EnumSource. Everything else should look familiar. As a value for @EnumSource, we have added our Channel enum class. It will make the parameterized test run with every value from the enum class. Running this test class will give us the following execution results:

Execution results of the EnumSourceExampleParameterizedTest class

As we can see, our test method was executed four times with every value from the Channel enum class. But let’s imagine the case when a couple of our channels have different behavior, and we need to implement a separate test method for it. Would it be possible to use a parameterized test from this case? Definitely, please refer to the example below to see how it works:

We have decomposed the test method into two: the first will test sending messages through WhatsApp and Slack, and the second will test SMS and Email channels. To achieve this, we have modified our @EnumSource annotation. The value property stayed the same — it’s the enum class from which we want to load channel parameters. However, now we have specified the exact list of enum properties for each test method in the ‘names’ property of the enum argument source. Execution results for the updated test class:

Execution results of the EnumSourceExampleParameterizedTest class with decomposed test methods

Customizing names of test cases

Before finishing our experiments with parameterized tests, let’s review one more nice feature, which may make our tests more detailed and understandable for others. We will continue working on our methods for testing the message service. As we have seen on 5.2 execution results, each test case is displayed as a simple enum String representation. Let’s try to improve it by adding a custom detailed message, which may help better understand our test results. To do this, we will modify the @ParameterizedTest annotation by specifying the optional name property with a pattern for the displayed test case title. Please review the code snippet below, and then we will review it in detail:

Let’s review the text pattern from the name property of the @ParameterizedTest annotation step by step:

• All dynamic variables should be placed in curly brackets. Like {index}, for example.
• The {index} variable is the index of the test method run. Like ‘[1] SMS’ and ‘[2] EMAIL’. 
• The {0} variable is the reference to the first method argument from our test method. It’s the ‘Channel’ in our case. In case we would have more than one argument, we can reference the second one as {1}, the third one as {2}, and so on.

This how the execution results will look like after this change:

Execution results of the EnumSourceExampleParameterizedTest with customized names

Such customizations may be helpful when we have complex test cases which aren’t intuitively understandable.

In this article, we have reviewed the process of creating parameterized tests with the JUnit 5 framework. We have learned how to configure necessary dependencies with Maven and Gradle build tools. We have discovered how to create parameterized tests using value, null, empty and enum argument sources, and learned the differences in configuration between parameterized and regular tests. Besides this, we have discovered how to configure and customize the displayed names of individual test cases. Complete source code and examples used in this article can be found here .

Please Login to comment...

Similar reads.

• Software Testing

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

Property-based testing in Java with JUnit-Quickcheck – Part 1: The basics

To be able to show you what Property-based testing (PBT) is, let’s start by grasping the concept of a property in programming languages. Since this is a Java tutorial, I will start with Oracle and their definition of a property in their  glossary :

Characteristics of an object that users can set, such as the color of a window.

Property is neither a variable/field or a method; it is something in between which is always true in your context. An example is weight in a postal parcel: this always is greater than zero.  In Java the following example implementation would follow:

[code language="java"] public class PostalParcel { private int weight; private String uuid; public PostalParcel(String uuid, int weight) { this.uuid = uuid; if(weight > 0) { this.weight = weight; } else { throw new IllegalArgumentException(""); } } }[/code] In this case, we made weight a property that always needs to be greater than 0. When designing your software, it is important to search for these properties, because they usually have some sort of business behaviour associated with them.

What is Property-based testing (PBT)

Back to PBT, let’s add a function to PostalParcel that decides the delivery costs of the package. When the weight is greater than 20 the delivery costs will be 4.99 euro, otherwise the delivery cost will be 1.99 euro. We will first write our tests for this, which without PBT, will usually end up something like this: [code language="java"] public class PostalParcelTest { @Test public void deliveryCostsShouldBeMaxWhenWeightIsLargerThan20(){ PostalParcel postalParcel = new PostalParcel("uuid", 23); assertThat(postalParcel.deliveryCosts(), equalTo(PostalParcel.MAX_DELIVERY_COSTS)); } @Test public void deliveryCostsShouldBeMinWhenWeightIsLessThanOrEqualTo20(){ PostalParcel postalParcel = new PostalParcel("uuid", 19); assertThat(postalParcel.deliveryCosts(), equalTo(PostalParcel.MIN_DELIVERY_COSTS)); } @Test(expected = IllegalArgumentException) public void shouldThrowIllegalArgumentExceptionWhenWeightIsBelowOne() { PostalParcel postalParcel = new PostalParcel("uuid", -100); } } [/code] Secondly, we have to make these test succeed by writing the implementation of deliveryCosts in a Postal Parcel: [code language="java"] public static final double MAX_DELIVERY_COSTS = 4.99; public static final double MIN_DELIVERY_COSTS = 1.99; public double deliveryCosts() { if(weight > 20) { return MAX_DELIVERY_COSTS; } return MIN_DELIVERY_COSTS; } [/code] When you run the tests, they will succeed, but if we think about it, 2 problems will remain. First of all we nicely described our behaviour in the test function name, but the implementation does not match the behaviour. The second test, deliveryCostsShouldBeMinWhenWeightIsLessThanOrEqualTo20, only tests with a weight of 19, but the behaviour clearly states Less Than Or Equal To 20. We can add another test which tests with a weight of 20, but what about the other cases? It will be a lot of work, and a waste of time, to create all the tests our self. Luckily, PBT comes to the rescue! Furthermore, and this is what PBT is all about, we need a UUID as String for the test.  Since we do not care about the value of that property, we usually use a fixture for a case like this. It might happen that a UUID can give strange errors in executing the behaviour under test now, or in the future. Because we can use anything as a String input, or like Romeu Moura once tweeted:

. @kenny_baas like I said in my talk: if you take a String as arg then the works of Shakespeare in Japanese & Korean are ONE valid input — Romeu Moura (@malk_zameth) 23 februari 2017

I advice everyone to watch his talk about Property-based testing. Now what you usually want to do, is make a Value Object of a UUID, but for now, because i want to show you a PBT example, we will leave it as a String. Even though, you can already see the benefit PBT will give: it makes you think about your input values. For a more in depth explanation of what Property-based testing is, you can also check the  blog post of Hypothesis. Hypothesis also has a Java version, but at the moment this is still a prototype.

JUnit-Quickcheck implementation

Start of with adding the  JUnit-Quickcheck  dependency to our project. In order to use JUnit-Quickcheck we need to change 2 things. First add the @RunWith annotation above our test class. [code language="java"] @RunWith(JUnitQuickcheck.class) [/code] Second, instead of using @Test we use @Property . [code language="java"] @Property public void deliveryCostsShouldBeMaxWhenWeightIsLargerThan20(){ ….. [/code] JUnitQuickcheck will now run each property test 100 times. We will get the added value of this when we let the JUnitQuickcheck generate the property parameters: for each run it will generate a random parameter. Let’s add them: [code language="java"] @Property public void deliveryCostsShouldBeMaxWhenWeightIsGreaterThan20(String uuid, int weight) PostalParcel postalParcel = new PostalParcel(uuid, weight); assertThat(postalParcel.deliveryCosts(), equalTo(PostalParcel.MAX_DELIVERY_COSTS)); } …. [/code] Running this will most likely fail the tests, because the number can be any int now, but what we want is, for example, Larger Than 20. This can be accomplished in 2 ways, either use assumeThat of JUnit, or use the @InRange of JUnitQuickcheck  in front of the variable. [code language="java"] @RunWith(JUnitQuickcheck.class) public class PostalParcelTest { @Property public void deliveryCostsShouldBeMaxWhenWeightIsGreaterThan20(String uuid, @InRange(minInt = 21) int weight){ assumeThat(weight, greaterThan(20)); PostalParcel postalParcel = new PostalParcel(uuid, weight); assertThat(postalParcel.deliveryCosts(), equalTo(PostalParcel.MAX_DELIVERY_COSTS)); } @Property public void deliveryCostsShouldBeMinWhenWeightIsLessThanOrEqualTo20(String uuid, @InRange(minInt = 1, maxInt = 20) int weight){ assumeThat(weight, is(both(greaterThan(0)).and(lessThanOrEqualTo(20)))); PostalParcel postalParcel = new PostalParcel(uuid, weight); assertThat(postalParcel.deliveryCosts(), equalTo(PostalParcel.MIN_DELIVERY_COSTS)); } @Property public void shouldThrowIllegalArgumentExceptionWhenWeightIsBelowOne(String uuid, @InRange(maxInt = 0) int weight) { assumeThat(weight, lessThanOrEqualTo(0)); IllegalArgumentException illegalArgumentException = null; try { PostalParcel postalParcel = new PostalParcel(uuid, weight); } catch(IllegalArgumentException e) { illegalArgumentException = e; } assumeThat(illegalArgumentException, notNullValue()); } } [/code] I would advice to combine both the options, for 2 reasons. First, assumeThat won’t work sufficient enough in small ranges, like in the second example. You will get errors of violating the assumptions. Adding the assumeThat will make your tests more readable and explicit though.

Trials and Success

JUnit-Quickcheck by default will run the test 100 times, each with a random value. In some cases, like the LessThanOrEqual20, running 100 can be a waste (can, because it still is random). For this we can set the trials modifier on the @Property(trials = 25) annotation. This tells JUnit-Quickcheck it should only run the amount of trials set here, in this case 25 should be enough. It should, but it is still random, so it might happen that the checks will succeed the first time, but will fail after a while. This i.m.o. is the real benefit of PBT, it will eventually test all the corner cases for you, and find possible bugs in your system. Success just means it did not find errors this time, but maybe next time it will.

Now we have written our very first, basic, Property-based test for Java. This is already really helpful. In the next part of this tutorial we will go deeper into the framework and we will let JUnit-Quickcheck generate our entities for us as input values. You can find the sourcecode of this example on my gitlab. Also, this post is published on my private  blog .

Get in touch with us to learn more about the subject and related solutions

Explore related posts.

Software Development Property-based testing in Java with JUnit-Quickcheck - Part 2: Generators Kenny Baas-Schwegler 12 Jun, 2017

Java Eclipse Tip: Keep static imports for JUnit 4 Vincent Partington 12 Oct, 2008

Java | Software Development | Testing Mutation testing with Pitest Jan Vermeir 12 Feb, 2020

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

Course-3: Introduction to Automated Analysis - Coursera -Offered By University of Minnesota

MubashirSiddiqui/Introduction_to_Automated_Analysis

Folders and files.

Repository files navigation

[software-testing-and-automation-specialization].

Introduction to Automated Analysis - Coursera -Offered By University of Minnesota

[About this Course]

• Understand the foundations of automated verification: randomization and symbolic representations
• Distinguish the strengths and weaknesses of random testing, symbolic analysis, static analysis, and model checking
• Use a variety of state-of-the-art static analysis and automated testing tools for automated verification
• Create executable requirements as an oracle suitable for automated testing and symbolic analysis
• Understand how the choice of oracle affects fault-finding for automated analysis strategies.
• Use automated testing to achieve full mutation coverage
• Create a test plan that utilizes both manually-written tests and automated tests towards maximizing rigor, minimizing effort and time, and minimizing test costs.

This course is intended for learners interested in understanding the principles of automation and the application of tools for analysis and testing of software.

Introduction In this module we will learn about a range of techniques for analysis of programs and methods to automate testing. Along the way we will learn how to specify properties of interest to check about a program and capture assumptions about the environment for effective testing. To reinforce some of the important concepts learned we will practice automated testing using effective tools on a concrete example.

Automated Test Generation The focus of this module is to understand how various techniques can help us automate the generation of useful and numerous tests. We will examine ways to specify and use properties of the system and the environment to guide the generation of test data.

Static Analysis The goal of this module is to introduce the learner to the principles of statically analyzing programs, understand how analysis techniques work by looking at some example analyses, and some good practices to follow when designing programs to enable the tools to help us detect and avoid defects. The learner will gain an understanding of using static analysis tools by looking at one concrete tool.

Effective Automated Verification This module will examine how to use effective automation techniques for a variety of purposes such as performing effective regression testing, discovering security vulnerabilities and monitoring software at run-time for critical properties.

• Gherkin 1.8%
• JavaScript 1.7%

• Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers
• Advertising & Talent Reach devs & technologists worldwide about your product, service or employer brand
• OverflowAI GenAI features for Teams
• OverflowAPI Train & fine-tune LLMs
• Labs The future of collective knowledge sharing
• About the company Visit the blog

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Get early access and see previews of new features.

Best Practice for multiple Parameterized JUnit Tests

Often, a Java class has more than one public method that we want to test with JUnit. What happens when both public methods are something we can use Parameterized techniques?

Should we keep a JUnit Test class for each public method we want to test with Parameterized parameters, or how do we keep both in one JUnit Test class?

Sample Parameterized JUnit Test class for testing the public method RegionUtils.translateGeographicalCity(...)

• parameterized

layout: remember you don't do 'class testing', you do 'unit testing'. organize and name your tests in a meaningful way that won't surprise any future readers of your code. usually it means testing all simple methods in one test class, using one or two parameterized tests for each. but sometimes one method is complicated and requires much more detailed testing and then it may be a good idea to create a dedicated test file for it

tools: you need some kind of data providers. then usually each test has its own data providers. sometimes multiple methods can share the same data provider. Sadly, JUnit sucks on that field. recently appeared some libraries extending the JUnit. also JUnit on its own is adding @Parameterized, @Theories etc., but still - it sucks. so if you can't switch to TestNG or Spock try these:

• if you know all your parameters upfront, check https://github.com/piotrturski/zohhak
• if you need to read from file and or build data dynamically, check https://github.com/TNG/junit-dataprovider or https://github.com/Pragmatists/JUnitParams

Your Answer

Reminder: Answers generated by artificial intelligence tools are not allowed on Stack Overflow. Learn more

Sign up or log in

Post as a guest.

Required, but never shown

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy .

Not the answer you're looking for? Browse other questions tagged java junit parameterized or ask your own question .

• Featured on Meta
• We've made changes to our Terms of Service & Privacy Policy - July 2024
• Bringing clarity to status tag usage on meta sites
• Feedback requested: How do you use tag hover descriptions for curating and do...

Hot Network Questions

• Can science inform philosophy?
• UART pin acting as power pin
• What is the best way to calculate the partial charge?
• Uneven Spacing with Consecutive Math Environments
• Do temperature variations make trains on Mars impractical?
• A large maze with locked doors and keys
• How should Form 990: Part IV Question 3 be answered?
• For applying to a STEM research position at a U.S. research university, should a resume include a photo?
• What's the sales pitch for waxing chains?
• Use all eight of the given polygons to tile a parallelogram
• Who became an oligarch after the collapse of the USSR
• Ubuntu/Linux: Use WiFi to access a specific LPD/LPR network print server and use Ethernet to access everything else
• Creating a deadly "minimum altitude limit" in an airship setting
• How to install a second ground bar on a Square D Homeline subpanel
• LED lights tripping the breaker when installed on the wrong direction?
• Are the peer reviewers of a journal article allowed to voice surprise to the editor at a "minor revision" decision?
• What if something goes wrong during the seven minutes of terror?
• How Subjective is Entropy Really?
• MOSFETs keep shorting way below rated current
• Unexpected behaviour during implicit conversion in C
• Has technology regressed in the Alien universe?
• Is there a way to say "wink wink" or "nudge nudge" in German?
• What does the Serre functor of equivariant category of fractional CY category look like?
• Why would an incumbent politician or party need to be re-elected to fulfill a campaign promise?

Get the Reddit app

Computer Programming

Parameterized unit tests in JUnit

By continuing, you agree to our User Agreement and acknowledge that you understand the Privacy Policy .

Enter the 6-digit code from your authenticator app

You’ve set up two-factor authentication for this account.

Enter a 6-digit backup code

Create your username and password.

Reddit is anonymous, so your username is what you’ll go by here. Choose wisely—because once you get a name, you can’t change it.

Reset your password

Enter your email address or username and we’ll send you a link to reset your password

Check your inbox

An email with a link to reset your password was sent to the email address associated with your account

Choose a Reddit account to continue