ABAP Test Double Framework versus mockA

I recently had the chance to have a look into SAP’s ABAP Test Double Framework. Previously, the main tool for mock object creation was mockA, which did a great job in my previous projects.

Code readability

The Test Double Framework’s fake object creation is quite straight forward. Creating such an instance is handy and even allows you to use the native method call to specify the importing-parameters. Please see the blog post from SAP:

"create test double object
lo_currency_converter_double ?= cl_abap_testdouble=>create( 'if_td_currency_converter' ).

“configuration for stubbing method ‘convert’:
“step 1: set the desired returning value for the method call
cl_abap_testdouble=>configure_call( lo_currency_converter_double )->returning( 80 ).
“step 2: specifying which method should get stubbed
lo_currency_converter_double->convert(
EXPORTING
amount = 100
source_currency = ‘USD’
target_currency = ‘EUR’
).

The same test double configuration in mockA:

"create test double object
lo_currency_converter_double ?= zcl_mocka_mocker=>zif_mocka_mocker~mock( 'if_td_currency_converter' )->method( ‘convert’ )->with( i_p1 = 100 i_p2 = ‘USD’ i_p3 = ‘EUR’ )->returns( 80 )->generate_mockup( ).

As you can see, mockA doesn’t allow to verify whether you correctly supplied the method’s importing-parameters and method name at design time. This restriction applies for the Test Double Framework only for exporting and changing-parameters.
The only two disadvantages that I’ve recognized are

  • checked exceptions need to be catched in a dummy TRY…CATCH…RETURN…ENDTRY-statement to avoid ATC messages
  • the exporting- and returning parameters specification comes before the actual method call specification and configuration of the importing parameter. You need to get used to it, but it’s fine

Let the framework ignore importing parameters

The Test Double Framework allows you to separately ignore importing parameters by attaching “->ignore_parameter(…)” to your “configure_call(…)” method call. This feature is not yet existent in mockA. However, if you avoid the “->with(…)” method call in mockA completely, it will return the specified output in any case . This is equivalent to “->ignore_all_parameters(…)”.

Verify interactions

Quite handy in the Test Double Framework. Just call

cl_abap_testdouble=>verify_expectations( lo_currency_converter_double ).

..and let the framework assert that your specified methods have been called as intended.

mockA requires you to store the mock creation object of type ZIF_MOCKA_MOCKER.
It allows you to verify method calls like this:

cl_abap_unit_assert=>assert_true( mo_mocker->method( 'convert' )->has_been_called_with( i_p1 = 100 i_p2 = 'USD' i_p3 = 'EUR' ) ).

Custom matchers

The Test Double Framework allows you to pass custom matchers which will implement interaction verifications on the fake method input parameters. The interface is called if_abap_testdouble_matcher and it is very easy to implement. mockA does not offer such a concept.

System requirements

mockA requires NW 7.01. The Test Double Framework requires NW 7.40.

Object support

mockA supports classes and interfaces, while the Test Double Framework supports interfaces, but no classes at the moment.

Conclusion

The Test Double Framework is a really nice mocking framework. It works very well and offers functionality, which mockA doesn’t offer. However, some drawbacks still exist, e.g. when you want to mock classes or work with NW releases lower than 7.40

News about mockA

The current release of mockA is available at Github. It contains an error fix that I would like to outline in today´s blog post.

 

The bug

MockA allows you to mock classes as described in one of my previous blog posts. Technically, mockA tries to create a subclass of the class which is subject to the mock creation. This means, it will only work, if the class is not marked as final and has a constructor which is at least protected or public.

MockA overrides methods that should be mocked, with a local implementation that returns the values expected to be returned. It follows the specifications set up by the unit test, according to the method( ), with( ) and exports( ) or returns( ) -calls (and so on) during mock creation.

 

There is another feature that reuses generated subroutine pools that have been created by mockA, because the Web Application Server ABAP allows only about 36 subroutine pools for each program, or, in our case, per unit test. The generated code does not contain any hard coded method output parameters as there would be no benefit in buffering the generated coding then. Instead, the instance of type ZIF_MOCKA_MOCKER is passed to the mock object. In the mock object´s method implementations, the fake values are read from that instance. If a new mock object should be created, a new instance of ZIF_MOCKA_MOCKER will be passed to the mock object. Hence, the method output may change.

 

If mockA generates the local implementation of an interface, each method is implemented during the initial mock class generation, so this feature poses no issues here.

However, in case a class needs to be mocked, mockA also tried to reuse these generated subroutine pools in the past. Do you see the error?

 

What could possibly happen

Imagine, mockA should create a mock implementation of the following class: ZCL_I_CAUSE_TROUBLE which has two methods:

  • METHOD_A
  • METHOD_B

  In our first unit test, we will tell mockA to simulate the output of METHOD_A, without defining any output for METHOD_B. 

When the mock object is created, mockA will generate a local subclass of ZCL_I_WILL_CAUSE_TROUBLE, with a local implementation of method METHOD_A that overrides the parent´s class method. The parent´s class method cannot be called any longer via the mock object. METHOD_B_ remains untouched.

After generation of the subroutine pool, the class implementation is buffered for later usage.

 

If another unit test, that is executed after the first one, wants to control the output of METHOD_B, mockA won´t return that output, as the method has not been overridden in the first unit test and therefore no output control takes place in the locally created class implementation: The logic that is responsible for returning the specified fake values is simply not called. Instead, the super implementation of ZCL_I_WILL_CAUSE_TROUBLE is called.

 

The solution

Subroutine pool buffering is now generally switched off if a class needs to be simulated. For interfaces, the current logic remains unchanged.

Unfortunately, this change is a breaking change, which can lead to failing unit tests, which have passed in the past.

This change could cause some new issues that I would like to outline briefly:

  1. Subroutine pool limits might be violated for existing unit tests. As there might be multiple implementations generated per class within the same unit test report, the subroutine pool limit might be violated once you updated mockA. In this case, please split up your test methods into various reports, if possible.
  2. Please see the example above: If your second unit test tells mockA to simulate the output of METHOD_B, but actually expects a result that is returned by the super implementation, your unit test might fail now, as the super implementation is not called any longer due to the correction and instead, the specified fake values will be returned.
    I know that this is just a theoretical consideration but important to be mentioned. Nevertheless, these test cases can be considered incorrectly implemented, as the unit test possibly expects other values than the values that have been defined as output for METHOD_B. Hence, these test cases should be reviewed anyway!

 

Feedback welcomed

There is no possibility to switch off the currently implemented behaviour of mockA as I think it is more important to fix the error than to allow old and incorrect unit test implementations not to fail.

 

Please let me know, if you run into trouble with the new update, and if issue 1) or 2) applies, or maybe both. Please also tell me, if you figured out another issue that I didn´t think of now.

Isolate components for better testing with mockA

Introduction

This blog post is strongly connected to the presentation that has been given by Damir Majer and Martin Steinberg during SAP Inside Track 2014 in Munich.

The presentation focuses on solving a code kata using Test-Driven Development (TDD).

The purpose is to show how TDD can lead to better tests coverage, thus more robust and maintainable software components. This blog post focuses not about the solution of the code kata itself, but rather on how to test each component separately from each other using mockA.

The Kata

Summarized, the Kata says:

Implement a simple String calculator class that accepts a flexible amount of numbers in a string. The numbers should be summed up and the sum needs to be returned.

Examples:

  • An empty string returns “0”
  • For single numbers, the number itself will be returned, e.g. for “1”, the sum 1 will be returned
  • For “1,2”, the sum 3 will be returned
  • Also multiple delimiters will have to be accepted, e.g. “1;2\3;1” will lead to 7
  • This also applies to line breaks like “\n”. “1\n2,3” results in 6
  • Delimiters might have variable length. “//***\1***2***3\2***2” results in 10
  • Raise an exception in case negative numbers are passed to the method
  • Numbers bigger than 1000 should be ignored

The Kata requires you to implement the code step by step, without skipping steps. Every step should contain

  • A unit test that tests the requirement and will fail at the first run
  • An implementation that covers the requirement
  • A new unit test run that will succeed
  • Refactoring
  • Running the test again to ensure nothing broke

The Solution

The solution class can be found in the attachments (“zcl_string_calculator.txt”).

The class ZCL_STRING_CALCULATOR contains

  • One protected method that replaces all delimiters with a comma (“replace_delimiter_with_comma”)
  • One protected method that sums up the consolidated string (“compute”)
  • One public method to rule them all (“add”)
  • Several attributes

“add” basically delegates the task of replacing all delimiters with commas to a specific protected method. It uses its output to sum up the values.

The Unit Test report “Unit Test v1.txt” shows the corresponding unit tests.

Isolate helper methods from the add-method

While “replace_delimiter_with_comma” and “compute” are helper methods, the public “add”-method delegates its own calls to these methods. Thus, it is dependent from the helper methods.

In some point of time, it might be helpful to check, if the “add”-method works as expected, which means, that it delegates its calls correctly to the helper method.

Think of the following unit test, which does not directly link to the code kata, but may ensure code quality:

  • Test the “add” method with the string “<1sd2,3rtrt,4”
  • Ensure, that “add” calls “replace_delimiter_with_comma” with “<1sd2,3rtrt,4”
  • The call will return “1,2,3,4”
  • Ensure, that “compute” will be called with “1,2,3,4”
  • Ensure, that the output of compute is returned without modification (result will be 10)

Such a test will need you to subclass ZCL_STRING_CALCULATOR and redefine the helper methods with hard coded return values based on specific inputs. Furthermore, some logic behind “compute” should allow you verify if the method has been called with the correct parameters.

Subject to the test will be the subclass of ZCL_STRING_CALCULATOR, which will partly contain so called faked functionality regarding “replace_delimiter_with_comma”. But it will also contain some mock features, as “compute” should not only conditionally return some values based on its input, but it should also allow you to determine, if it has been called with the expected input.

mockA allows you to skip this subclassing and lets you focus on the test. It will create a subclass at runtime for you, which follows constraints like conditional method output. These constraints can be hard coded by you. It will also allow you to verify method calls of mocked methods.

“Unit Test v2.txt” shows you how to do it. Keep a look a test method “test_add”.

The first call

lo_mocker = zcl_mocka_mocker=>zif_mocka_mocker~mock( ‘ZCL_STRING_CALCULATOR’ ).

lo_mocker->method( ‘replace_delimiter_with_comma’

)->with_changing( ‘<1sd2,3rtrt,4’

)->changes( ‘1,2,3,4’

).

tells mockA to fake the method “replace_delimiter_with_comma”, while

lo_mocker->method( ‘compute’

)->with( ‘1,2,3,4’

)->returns( 10

).

tells mockA to fake the output of “compute”.

Subject to the test will be the object generated by mockA (which is a subclass of ZCL_STRING_CALCULATOR in reality)

go_string_calculator ?= lo_mocker->generate_mockup( ).

After the call of “add”, the result is verified in the unit test. But besides this verification, you may also ensure, that “compute” has been called correctly with the input value “1,2,3,4”:

DATA lv_has_been_called_correctly TYPE abap_bool.

lv_has_been_called_correctly = lo_mocker->method( ‘compute’ )->has_been_called_with( ‘1,2,3,4’ ).

assert_not_initial( lv_has_been_called_correctly ).

Further information

You may find further information of the presenters at

damir-majer.com / @majcon

http://about.me/martin.steinberg / @SbgMartin

attachments: https://code.google.com/p/uwekunath-wordpress-com/source/browse/#git%2FCodeKATA