Property Definition Tutorial

In this tutorial, you will learn how to write application properties and test them using Kea.

In mobile applications, properties define the expected behavior of the application. If the application violates this property, it indicates that a bug has been discovered.

The user-defined application functionality properties consist of three key components. <P, I, Q>, (1) P is a precondition, (2) I is an interaction scenario defining how the application function is executed, and (3) Q is a postcondition defining the expected behavior.

Kea provides users with @initializer() to help define an initialization function that allows the application to skip the welcome or login page.

In Kea, properties are defined using decorators on property functions such as @rule().

To define the preconditions of a property, users can use the @precondition() decorator on the function decorated with @rule().

Postconditions are defined using assert within the function decorated with @rule().

For mobile applications, users can derive application properties from multiple sources, such as application specifications, documentation, test cases, and bug reports.

Let’s start with a few simple examples to introduce how to derive a property, how to write it in Kea, and how to test it using Kea.

Extracting Application Properties from Bug Reports

The following example will show how to extract a property from the application OmniNotes.

OmniNotes is an application for recording and managing notes.

This example is based on the bug report #634 where a user reported that when they deleted a tag, other tags sharing the same prefix were also deleted.

From this bug report, the following application property can be derived:

After deleting a tag, the tag should be successfully removed, and the note content should remain unchanged.

Based on the bug report, you can derive the following application property:

  • P (Precondition): A tag should exist.

  • I (Interaction Scenario): Remove a tag from the tag list.

  • Q (Postcondition): The specified tag is deleted, and the rest of the text content remains unchanged.

Next, let’s define this property in Kea using property description language.

@precondition(lambda self: d(resourceId="it.feio.android.omninotes:id/menu_tag").exists() and
               "#" in d(resourceId="it.feio.android.omninotes:id/detail_content").info["text"]
               )
@rule()
def rule_remove_tag_from_note_shouldnot_affect_content(self):
    # get the text from the note's content
    origin_content = d(resourceId="it.feio.android.omninotes:id/detail_content").info["text"]
    # click to open the tag list
    d(resourceId="it.feio.android.omninotes:id/menu_tag").click()
    # select a tag to remove
    selected_tag = random.choice(d(className="android.widget.CheckBox",checked=True))
    select_tag_name = "#"+ selected_tag.right(resourceId="it.feio.android.omninotes:id/md_title").info["text"].split(" ")[0]
    selected_tag.click()
    # click to uncheck the selected tag
    d(text="OK").click()
    # get the updated content after removing the tag
    new_content = d(resourceId="it.feio.android.omninotes:id/detail_content").info["text"].strip().replace("Content", "")
    # get the expected content after removing the tag
    origin_content_exlude_tag = origin_content.replace(select_tag_name, "").strip()
    # the tag should be removed in the content and the updated content should be the same as the expected content
    assert not d(textContains=select_tag_name).exists() and new_content == origin_content_exlude_tag

The @precondition decorator defines the state node where this property should begin to be tested. In the code, d(resourceId="it.feio.android.omninotes:id/menu_tag").exists() checks if the tag button exists on the screen, and "#" in d(resourceId="it.feio.android.omninotes:id/detail_content").info["text"] checks if the note content contains the ‘#’ character.

The @rule() decorator defines the application property function. In this segment of code, the interaction scenario is to perform the operation of removing a tag.

Postconditions are defined using the assert statement. Here, Kea checks whether the specified tag is deleted and the remaining text remains unchanged.

A property like this should be followed by the OmniNotes application.

Additionally, users can define an initialization function to set the initial state of the application before testing the property.

To achieve this functionality, users can use an @initializer() decorator to define an initialization function and write UI operation instructions to guide the application to complete the initialization action:

@initializer()
def set_up(self):
    for _ in range(5):
        d(resourceId="it.feio.android.omninotes:id/next").click()
    d(resourceId="it.feio.android.omninotes:id/done").click()
    if d(text="OK").exists():
        d(text="OK").click()

Here, the above code automatically skips the OmniNotes welcome page through UI operations. You can use the @initializer() decorator to define an initialization function for any application. This way, Kea will execute this initialization function before testing the application property, ensuring that the application is in the expected initial state at the start of each test.

Tip

This function can be used to set the initial state of the application before testing properties. For example, it can be used for logging in, adding data to the application, etc. If setting the initial state of the application is not necessary, this step can be skipped.

Moreover, if users want to use a main path guiding exploration strategy, they need to define a function using the @mainPath() decorator to set a main path function.

To complete this step for the application, you can use the following code to define the main path.

@mainPath()
def test_main(self):
    d(resourceId="it.feio.android.omninotes.alpha:id/fab_expand_menu_button").long_click()
    d(resourceId="it.feio.android.omninotes.alpha:id/detail_content").click()
    d(resourceId="it.feio.android.omninotes.alpha:id/detail_content").set_text("read a book #Tag1")
    d(description="drawer open").click()
    d(resourceId="it.feio.android.omninotes.alpha:id/note_content").click()

The above code guides Kea to create a note with the content ‘read a book #Tag1’ in Omninotes.

Tip

In the main path definition section, only UI operation commands can be used for the definition; this function currently does not support other Python statements, such as for loops. However, we believe this method is sufficient to implement the functionality of the main path.

Awesome! By now, you have learned how to extract and define an application property using property description language from a bug report.

To test this property, users need to place it in a defined class that inherits from the KeaTest class.

from kea import *

class Test(KeaTest):

    @initializer()
    def set_up(self):
        for _ in range(5):
            d(resourceId="it.feio.android.omninotes:id/next").click()
        d(resourceId="it.feio.android.omninotes:id/done").click()
        if d(text="OK").exists():
            d(text="OK").click()

    @mainPath()
    def test_main(self):
        d(resourceId="it.feio.android.omninotes.alpha:id/fab_expand_menu_button").long_click()
        d(resourceId="it.feio.android.omninotes.alpha:id/detail_content").click()
        d(resourceId="it.feio.android.omninotes.alpha:id/detail_content").set_text("read a book #Tag1")
        d(description="drawer open").click()
        d(resourceId="it.feio.android.omninotes.alpha:id/note_content").click()

    @precondition(lambda self: d(resourceId="it.feio.android.omninotes:id/menu_tag").exists() and
                "#" in d(resourceId="it.feio.android.omninotes:id/detail_content").info["text"]
                )
    @rule()
    def rule_remove_tag_from_note_shouldnot_affect_content(self):
        # get the text from the note's content
        origin_content = d(resourceId="it.feio.android.omninotes:id/detail_content").info["text"]
        # click to open the tag list
        d(resourceId="it.feio.android.omninotes:id/menu_tag").click()
        # select a tag to remove
        selected_tag = random.choice(d(className="android.widget.CheckBox",checked=True))
        select_tag_name = "#"+ selected_tag.right(resourceId="it.feio.android.omninotes:id/md_title").info["text"].split(" ")[0]
        selected_tag.click()
        # click to uncheck the selected tag
        d(text="OK").click()
        # get the updated content after removing the tag
        new_content = d(resourceId="it.feio.android.omninotes:id/detail_content").info["text"].strip().replace("Content", "")
        # get the expected content after removing the tag
        origin_content_exlude_tag = origin_content.replace(select_tag_name, "").strip()
        # the tag should be removed in the content and the updated content should be the same as the expected content
        assert not d(textContains=select_tag_name).exists() and new_content == origin_content_exlude_tag

Here, the property needs to be defined in a class Test that inherits from the KeaTest class.

We place this property script file example_mainpath_property.py in the example directory. Users can test the application property by running the following command.

kea -f example/example_mainpath_property.py -a example/omninotes.apk

When you attempt to test this property, you may quickly discover two new bugs that violate this property. You can then write the corresponding bug reports and submit them to the application’s developers. Both of these bugs have now been fixed by the developers.

You can check the reports of these two bugs:

  1. Bug Report: Note tag cannot be removed.

  2. Bug Report: Deleting One Tag in a Note Affects Another Tag in the Same Note.

Extracting Properties from Specific Application Functions

Next is a complete example showcasing how to extract properties from the functionality of the application Amaze.

Amaze is a file management application that offers a simple and intuitive user interface, allowing users to easily browse, manage, and manipulate files.

In Amaze, you can create a folder, and the new folder should exist after creation. Therefore, you can define a property create_folder_should_exist. This means that when you want to create a folder, it should be successfully created.

You still need to use @rule() and @precondition() to complete the definition of the application property. In this example, the precondition P is that the button to create a new folder should exist and be on the interface capable of creating a folder. The interaction scenario I is a sequence of events for creating a folder. Finally, the postcondition Q is to check whether the newly created folder exists.

@precondition(lambda self: d(resourceId="com.amaze.filemanager:id/sd_main_fab").exists() and
                           not d(textContains = "SDCARD").exists())
@rule()
def create_folder_should_exist(self):
    d(resourceId="com.amaze.filemanager:id/sd_main_fab").click()
    d(resourceId="com.amaze.filemanager:id/sd_label", text="Folder").click()
    file_name = self._files.get_random_value()
    d.send_keys(file_name, clear=True)
    d(resourceId="com.amaze.filemanager:id/md_buttonDefaultPositive").click()
    d(scrollable=True).scroll.to(resourceId="com.amaze.filemanager:id/firstline", text=file_name)
    assert d(text=file_name).exists()

Great! You have learned how to write application properties from application functionality.

Note

Users can write a single property or multiple properties for an application in one .py file. They can also write multiple properties in multiple .py files. If opting for the first method, users need to ensure there is at most one @initializer() and one @mainPath() in a .py file, with multiple @rule() and @precondition() corresponding to different properties. The structure of test cases is illustrated in the diagram below (please add images or sample code as necessary).