ABAP Test Double Framework versus mockA

I recently had the chance to have a look into SAP’s ABAP Test Double Framework. Previously, the main tool for mock object creation was mockA, which did a great job in my previous projects.

Code readability

The Test Double Framework’s fake object creation is quite straight forward. Creating such an instance is handy and even allows you to use the native method call to specify the importing-parameters. Please see the blog post from SAP:

"create test double object
lo_currency_converter_double ?= cl_abap_testdouble=>create( 'if_td_currency_converter' ).

“configuration for stubbing method ‘convert’:
“step 1: set the desired returning value for the method call
cl_abap_testdouble=>configure_call( lo_currency_converter_double )->returning( 80 ).
“step 2: specifying which method should get stubbed
lo_currency_converter_double->convert(
EXPORTING
amount = 100
source_currency = ‘USD’
target_currency = ‘EUR’
).

The same test double configuration in mockA:

"create test double object
lo_currency_converter_double ?= zcl_mocka_mocker=>zif_mocka_mocker~mock( 'if_td_currency_converter' )->method( ‘convert’ )->with( i_p1 = 100 i_p2 = ‘USD’ i_p3 = ‘EUR’ )->returns( 80 )->generate_mockup( ).

As you can see, mockA doesn’t allow to verify whether you correctly supplied the method’s importing-parameters and method name at design time. This restriction applies for the Test Double Framework only for exporting and changing-parameters.
The only two disadvantages that I’ve recognized are

  • checked exceptions need to be catched in a dummy TRY…CATCH…RETURN…ENDTRY-statement to avoid ATC messages
  • the exporting- and returning parameters specification comes before the actual method call specification and configuration of the importing parameter. You need to get used to it, but it’s fine

Let the framework ignore importing parameters

The Test Double Framework allows you to separately ignore importing parameters by attaching “->ignore_parameter(…)” to your “configure_call(…)” method call. This feature is not yet existent in mockA. However, if you avoid the “->with(…)” method call in mockA completely, it will return the specified output in any case . This is equivalent to “->ignore_all_parameters(…)”.

Verify interactions

Quite handy in the Test Double Framework. Just call

cl_abap_testdouble=>verify_expectations( lo_currency_converter_double ).

..and let the framework assert that your specified methods have been called as intended.

mockA requires you to store the mock creation object of type ZIF_MOCKA_MOCKER.
It allows you to verify method calls like this:

cl_abap_unit_assert=>assert_true( mo_mocker->method( 'convert' )->has_been_called_with( i_p1 = 100 i_p2 = 'USD' i_p3 = 'EUR' ) ).

Custom matchers

The Test Double Framework allows you to pass custom matchers which will implement interaction verifications on the fake method input parameters. The interface is called if_abap_testdouble_matcher and it is very easy to implement. mockA does not offer such a concept.

System requirements

mockA requires NW 7.01. The Test Double Framework requires NW 7.40.

Object support

mockA supports classes and interfaces, while the Test Double Framework supports interfaces, but no classes at the moment.

Conclusion

The Test Double Framework is a really nice mocking framework. It works very well and offers functionality, which mockA doesn’t offer. However, some drawbacks still exist, e.g. when you want to mock classes or work with NW releases lower than 7.40

Advertisements

Developer Open Space 2015

Letztes Jahr habe ich über den Developer Open Space 2014 gebloggt. Auch dieses Jahr werde ich wieder teilnehmen.

Das Besondere am Konzept Open Space ist der freie Tagungscharakter. Bis kurz vor Beginn existiert keine festgelegte Agenda. Die Teilnehmer bestimmen die Themen des jeweiligen Tages gemeinsam. Die konkrete Programmiersprache ist dabei weniger entscheidend. Der Grund für alles ist einfach erklärt: Gute Gespräche hat man häufig ohne Agenda, beim Kaffee und beim „du“.

Vor acht Jahren gab es die erste Ausgabe der (Un-)Konferenz mit dem Namen Developer Open Space, die genau nach diesem Prinzip eines freien Tagungscharakters aufgebaut ist und sie findet dieses Jahr vom 16.–18. Oktober 2015 in Leipzig statt. Natürlich gibt es auch „Klassiker“, die jedes Jahr von den Teilnehmern als Themen festgelegt werden, nämlich vor allem der Erfahrungsaustausch wie „Git richtig einsetzen“, „Scrum ruinieren“, „PowerPoint zerstört Wirkung“, „NuGet ist kaputt“, „Telemetrie in Web-Apps“, „Gescheiterte Projekte – Was gelernt?“ oder „Warum ist die WCF nur so langsam?“. Daneben gibt es auch rein technische Themen wie etwa AngularJS, Docker, HTML5/CSS3 oder NoSQL.

Ergänzt wird der Open Space um einen Workshop-Tag, bei dem die Teilnehmer aus rund 20 Workshops auswählen können, beispielsweise zum Thema Agilität. Die Fotos von der Konferenz sprechen für sich. Alles ist soweit wie möglich selbst organisiert, das heißt, die Räume, die Infrastruktur, das Catering, aber eben auch nicht mehr.

Anmeldung

Die Anmeldung ist seit einiger Zeit möglich. Die Plätze sind begrenzt. Nimm teil! Informiere gerne auch Kollegen, befreundete Unternehmen oder Bekannte über diese gute Sache.

News about mockA

The current release of mockA is available at Github. It contains an error fix that I would like to outline in today´s blog post.

 

The bug

MockA allows you to mock classes as described in one of my previous blog posts. Technically, mockA tries to create a subclass of the class which is subject to the mock creation. This means, it will only work, if the class is not marked as final and has a constructor which is at least protected or public.

MockA overrides methods that should be mocked, with a local implementation that returns the values expected to be returned. It follows the specifications set up by the unit test, according to the method( ), with( ) and exports( ) or returns( ) -calls (and so on) during mock creation.

 

There is another feature that reuses generated subroutine pools that have been created by mockA, because the Web Application Server ABAP allows only about 36 subroutine pools for each program, or, in our case, per unit test. The generated code does not contain any hard coded method output parameters as there would be no benefit in buffering the generated coding then. Instead, the instance of type ZIF_MOCKA_MOCKER is passed to the mock object. In the mock object´s method implementations, the fake values are read from that instance. If a new mock object should be created, a new instance of ZIF_MOCKA_MOCKER will be passed to the mock object. Hence, the method output may change.

 

If mockA generates the local implementation of an interface, each method is implemented during the initial mock class generation, so this feature poses no issues here.

However, in case a class needs to be mocked, mockA also tried to reuse these generated subroutine pools in the past. Do you see the error?

 

What could possibly happen

Imagine, mockA should create a mock implementation of the following class: ZCL_I_CAUSE_TROUBLE which has two methods:

  • METHOD_A
  • METHOD_B

  In our first unit test, we will tell mockA to simulate the output of METHOD_A, without defining any output for METHOD_B. 

When the mock object is created, mockA will generate a local subclass of ZCL_I_WILL_CAUSE_TROUBLE, with a local implementation of method METHOD_A that overrides the parent´s class method. The parent´s class method cannot be called any longer via the mock object. METHOD_B_ remains untouched.

After generation of the subroutine pool, the class implementation is buffered for later usage.

 

If another unit test, that is executed after the first one, wants to control the output of METHOD_B, mockA won´t return that output, as the method has not been overridden in the first unit test and therefore no output control takes place in the locally created class implementation: The logic that is responsible for returning the specified fake values is simply not called. Instead, the super implementation of ZCL_I_WILL_CAUSE_TROUBLE is called.

 

The solution

Subroutine pool buffering is now generally switched off if a class needs to be simulated. For interfaces, the current logic remains unchanged.

Unfortunately, this change is a breaking change, which can lead to failing unit tests, which have passed in the past.

This change could cause some new issues that I would like to outline briefly:

  1. Subroutine pool limits might be violated for existing unit tests. As there might be multiple implementations generated per class within the same unit test report, the subroutine pool limit might be violated once you updated mockA. In this case, please split up your test methods into various reports, if possible.
  2. Please see the example above: If your second unit test tells mockA to simulate the output of METHOD_B, but actually expects a result that is returned by the super implementation, your unit test might fail now, as the super implementation is not called any longer due to the correction and instead, the specified fake values will be returned.
    I know that this is just a theoretical consideration but important to be mentioned. Nevertheless, these test cases can be considered incorrectly implemented, as the unit test possibly expects other values than the values that have been defined as output for METHOD_B. Hence, these test cases should be reviewed anyway!

 

Feedback welcomed

There is no possibility to switch off the currently implemented behaviour of mockA as I think it is more important to fix the error than to allow old and incorrect unit test implementations not to fail.

 

Please let me know, if you run into trouble with the new update, and if issue 1) or 2) applies, or maybe both. Please also tell me, if you figured out another issue that I didn´t think of now.

Developer Open Space Leipzig

Vom 17. bis 19. Oktober fand in Leipzig der alljährliche Developer Open Space statt, ein Art Anti-Konferenz in der es keine festen Strukturen, dafür aber genug Freiraum für die Teilnehmer gibt, sich zu Themen auszutauschen, die sie wirklich interessieren. Neben allerlei Häppchen und Getränken war auch genug Zeit für genügend leckere Workshops und von den Teilnehmern selbst organisierten Sessions.

Los ging es am 17.10. mit dem Thema IT-Sicherheit mit Lars Kumbier. Bereits bekannte Angriffe wurden ebenso systematisch beleuchtet, wie ein komplettes Praxisbeispiel zum Thema SQL-Injection, ein Angriff, bei welchem des Testsystem binnen Minuten in der Hand des Angreifers war. Am Schluss durfte jeder seine eigenen kleinen Sicherheitslücken in den Code der Plattform seiner Wahl klopfen, angreifen und anschließend korrigieren.

Am 18. Oktober ging es mit den Sessions weiter. Nach der üblichen einstündigen Vorstellung der Sessions schwärmten die Entwickler und solche, die es noch werden wollen, in die jeweiligen Räume aus, um sich in die einzelnen Runden zu setzen und ihre Erfahrungen in die fachlichen Diskussion einwerfen zu können oder einfach schweigend zuzuhören beziehungsweise die Live-Demo zu betrachten.

Es ist erstaunlich, wie viel der Developer Open Space, der noch vor einigen Jahren “.NET Open Space” hieß, auch für nicht-.NETler zu bieten hat. Nennenswert ist insbesondere der hohe Anteil an Soft-Skill und Projektmanagement-Sessions. Naturgemäß geht man als SAPler dort eher hin als, zur Session “Microservices”, aber auch einige Microsoft-spezifische Diskussionen haben durchaus Charme, auch wenn nicht alle technischen Details auf den ersten Blick klar sind. Zumindest erkennt man, wohin in anderen Bereichen der Enterprise-IT die Reise hin geht. Oder man bietet kurzerhand einfach seine eigene Session an.

Session-Plan

Bemerkenswert ist auch die hohe Professionalisierung und Agilität der .NET Community in Bezug auf Entwicklungsprozesse, die bei Projektmanagement-Methoden wie Scrum und Entwicklungsansätzen wie Test-Driven-Development deutlich wird und bei einer kritischen Betrachtung der für den jeweiligen Einsatzzweck geeigneten Frameworks noch nicht des Ende der Fahnenstange erreicht hat.

Fazit: Wer aus den DACH-Ländern kommt, sollte hier unbedingt mal vorbei schauen, ungeachtet der Profession. Es ist für jeden etwas dabei, nicht nur Kaffee, Club Mate und Kuchen.

Isolate components for better testing with mockA

Introduction

This blog post is strongly connected to the presentation that has been given by Damir Majer and Martin Steinberg during SAP Inside Track 2014 in Munich.

The presentation focuses on solving a code kata using Test-Driven Development (TDD).

The purpose is to show how TDD can lead to better tests coverage, thus more robust and maintainable software components. This blog post focuses not about the solution of the code kata itself, but rather on how to test each component separately from each other using mockA.

The Kata

Summarized, the Kata says:

Implement a simple String calculator class that accepts a flexible amount of numbers in a string. The numbers should be summed up and the sum needs to be returned.

Examples:

  • An empty string returns “0”
  • For single numbers, the number itself will be returned, e.g. for “1”, the sum 1 will be returned
  • For “1,2”, the sum 3 will be returned
  • Also multiple delimiters will have to be accepted, e.g. “1;2\3;1” will lead to 7
  • This also applies to line breaks like “\n”. “1\n2,3” results in 6
  • Delimiters might have variable length. “//***\1***2***3\2***2” results in 10
  • Raise an exception in case negative numbers are passed to the method
  • Numbers bigger than 1000 should be ignored

The Kata requires you to implement the code step by step, without skipping steps. Every step should contain

  • A unit test that tests the requirement and will fail at the first run
  • An implementation that covers the requirement
  • A new unit test run that will succeed
  • Refactoring
  • Running the test again to ensure nothing broke

The Solution

The solution class can be found in the attachments (“zcl_string_calculator.txt”).

The class ZCL_STRING_CALCULATOR contains

  • One protected method that replaces all delimiters with a comma (“replace_delimiter_with_comma”)
  • One protected method that sums up the consolidated string (“compute”)
  • One public method to rule them all (“add”)
  • Several attributes

“add” basically delegates the task of replacing all delimiters with commas to a specific protected method. It uses its output to sum up the values.

The Unit Test report “Unit Test v1.txt” shows the corresponding unit tests.

Isolate helper methods from the add-method

While “replace_delimiter_with_comma” and “compute” are helper methods, the public “add”-method delegates its own calls to these methods. Thus, it is dependent from the helper methods.

In some point of time, it might be helpful to check, if the “add”-method works as expected, which means, that it delegates its calls correctly to the helper method.

Think of the following unit test, which does not directly link to the code kata, but may ensure code quality:

  • Test the “add” method with the string “<1sd2,3rtrt,4”
  • Ensure, that “add” calls “replace_delimiter_with_comma” with “<1sd2,3rtrt,4”
  • The call will return “1,2,3,4”
  • Ensure, that “compute” will be called with “1,2,3,4”
  • Ensure, that the output of compute is returned without modification (result will be 10)

Such a test will need you to subclass ZCL_STRING_CALCULATOR and redefine the helper methods with hard coded return values based on specific inputs. Furthermore, some logic behind “compute” should allow you verify if the method has been called with the correct parameters.

Subject to the test will be the subclass of ZCL_STRING_CALCULATOR, which will partly contain so called faked functionality regarding “replace_delimiter_with_comma”. But it will also contain some mock features, as “compute” should not only conditionally return some values based on its input, but it should also allow you to determine, if it has been called with the expected input.

mockA allows you to skip this subclassing and lets you focus on the test. It will create a subclass at runtime for you, which follows constraints like conditional method output. These constraints can be hard coded by you. It will also allow you to verify method calls of mocked methods.

“Unit Test v2.txt” shows you how to do it. Keep a look a test method “test_add”.

The first call

lo_mocker = zcl_mocka_mocker=>zif_mocka_mocker~mock( ‘ZCL_STRING_CALCULATOR’ ).

lo_mocker->method( ‘replace_delimiter_with_comma’

)->with_changing( ‘<1sd2,3rtrt,4’

)->changes( ‘1,2,3,4’

).

tells mockA to fake the method “replace_delimiter_with_comma”, while

lo_mocker->method( ‘compute’

)->with( ‘1,2,3,4’

)->returns( 10

).

tells mockA to fake the output of “compute”.

Subject to the test will be the object generated by mockA (which is a subclass of ZCL_STRING_CALCULATOR in reality)

go_string_calculator ?= lo_mocker->generate_mockup( ).

After the call of “add”, the result is verified in the unit test. But besides this verification, you may also ensure, that “compute” has been called correctly with the input value “1,2,3,4”:

DATA lv_has_been_called_correctly TYPE abap_bool.

lv_has_been_called_correctly = lo_mocker->method( ‘compute’ )->has_been_called_with( ‘1,2,3,4’ ).

assert_not_initial( lv_has_been_called_correctly ).

Further information

You may find further information of the presenters at

damir-majer.com / @majcon

http://about.me/martin.steinberg / @SbgMartin

attachments: https://code.google.com/p/uwekunath-wordpress-com/source/browse/#git%2FCodeKATA

mockA – Carry out checks against parameters passed to the mock object’s methods

What has been missing so far

In one of my last blog posts, I showed you how mock objects can be easily created. I also showed you how mock object’s method calls can be counted and how verifications can be implemented, to ensure that a mocked object’s method has been called.
Up to now, it was not possible to check, which parameters have been passed.

Ensure, that a mocked object has been called with certain parameters

In the current build, I also included the possibility to verify the parameters that have been passed to a mock object’s method.
This is how it works:
First of all, you need to keep track of the mocker object of type ZIF_MOCKA_MOCKER (which means, please do not throw it away 😉 )
By utilizing this mocker object, you can get data about your mocked methods, such as if it has been called with certain parameters or not:

DATA lv_has_been_called TYPE abap_bool.
lv_has_been_called = lo_mocker->method( 'get_delay' )->has_been_called_with(
i_p1 = 'LH'
i_p2 = '300'
i_p3 = sy-datlo
).

For further details, please see report ZMOCKA_DEMO that is shipped with the current release.

Limitations

Only IMPORTING parameters are supported so far, this means no CHANGING parameters can be verified yet.

Mock specific classes with mockA

With mockA it is quite easy to mock interfaces for unit tests. But it is also capable of creating mock objects that are not based only on interfaces, but specific classes. This blog post shows how it works and what needs to be considered.

Basics

Creating such an instance is quite the same as the creation of mock objects based on interfaces. The only restriction that applies is the fact that the class which is to be mocked may not be final class. This is necessary because mockA couldn’t create a subclass to override method outputs.

The classes in the following examples can be found in the mockA package provided at Github.

DATA lo_mocker TYPE REF TO zif_mocka_mocker.
DATA lo_mocker_method TYPE REF TO zif_mocka_mocker_method.
DATA lo_flight_observer TYPE REF TO zcl_mocka_flight_observer.
lo_mocker = zcl_mocka_mocker=>zif_mocka_mocker~mock( zcl_mocka_flight_observer=>gc_name ).
lo_mocker_method = lo_mocker->method( 'observe_flight' ).
lo_mocker_method->with( i_p1 = 'NA' i_p2 = 007 i_p3 = sy-datlo ).
lo_mocker_method->returns( abap_true ).
lo_flight_observer ?= lo_mocker->generate_mockup( ).

Constructor parameters

However, this little example will still fail, as ZCL_MOCKA_FLIGHT_OBSERVER’s constructor expects non-optional IMPORTING parameters. This is an issue which is not existent for interfaces. As you mock already existing implementations, you also need to take care of that.

But that is also no problem at all. Consider the following example that passes some earlier created instances to the constructor. It can be achieved by calling the method PASS_TO_SUPER_CONSTRUCTOR of the mocker instance.
DATA lo_is_in_time_info TYPE REF TO zif_mocka_is_in_time_info.
DATA lo_flight_alert_process TYPE REF TO zif_mocka_flight_alert_process.
*create lo_is_in_time_info and lo_flight_alert_process... (not shown here)
DATA lo_mocker TYPE REF TO zif_mocka_mocker.
DATA lo_mocker_method TYPE REF TO zif_mocka_mocker_method.
DATA lo_flight_observer TYPE REF TO zcl_mocka_flight_observer.
lo_mocker = zcl_mocka_mocker=>zif_mocka_mocker~mock( zcl_mocka_flight_observer=>gc_name ).

lo_mocker->pass_to_super_constructor(
i_p1 = lo_flight_alert_process i_p2 = lo_is_in_time_info ).
*mock some method output (not shown here)
lo_flight_observer ?= lo_mocker->generate_mockup( ).

The example s are also in the mockA package. Take a look at the unit test report ZTEST_CL_MOCKA_MOCKER and the test methods mock_class_with_construc_param, mock_class_with_method_output and mock_intf_with_construc_param.