Testing (Chapter 6)

Testing

6.1. Structural Testing (White Box)

6.1.1. Instrumentation Testing

6.1.2. Stress Test

6.1.3. Database Test

6.1.4. Conditional Tests: Individual values

6.1.5. Conditional Tests: Value ranges

6.2. Functional Testing (Black Box)

6.2.1. User Acceptance Testing

6.2.1.1. Sponsor Acceptance Testing

6.2.1.2. Sample User Acceptance Testing

6.2.2. Alpha-Testing

6.2.3. Beta-Testing

6.3. Compatibility Tests

 

There are a lot of software testing methods. No perfect methods do exist; therefore, it is reasonable to use several types of procedures combined. Almost every method might be appropriate for testing because the project was huge, but there was no possibility for that. One recommended summary of the already known methods is the onlive available Software Testing Help (2015).

 

This section summarizes the methods applied from the listed ones. Since the project was huge, it was practical to test its usability in real environment as well, according to the End-to-end testing’ method. It has happened, and the details can be found in the Testing section (6.2.1.1.-6.2.1.2)

6.1. Structural Testing (White Box)

The first main category of testing methods is structural testing. In this category, elements and integration of development were tested. There are several possible testing methods, but it was reasonable to embed automatic tests into the application as well, which can be repeated stably and run in unlimited number. Therefore, testing also began at this point.

6.1.1. Instrumentation Testing

Many activities can be found in the program; therefore, there was no possibility to test each of them. Automatic tests were designed for the four most used activities. The tests verified the operation of buttons and text fields on the page. Different texts have been placed in the text boxes; then, after closing and restarting the program, the texts were read back and identity was checked. Buttons were checked by an automatic click event.

  • EditUserInstrumentationTest
  • FirstActivityInstrumentationTest
  • LoginActivityInstrumentationTest
  • RegisterInstrumentationTest

All four automatic tests were run at least 10 times. During the tests, operational problems did not occur. The results of tests contained in Table 1 and proving screen shots are found in Appendix G.

6.1.2. Stress Test

Monkey method was applied for testing stress. The test in the command line was performed with the following instructions.

 

cd C:\Users\Fpisti35\AppData\Local\Android\android-sdk\platform-tools

adb shell monkey -p com.example.fpisti35.msmas -v 2000

 

The test was run more than 10 times on an emulated virtual Nexus 7 API21 mobile device. A test simulated 2000 touches in every case, and no errors were noticed. It was impossible to make a table about the test because of the large number of touches. The screen shot about the test can be seen in Appendix G.

 

6.1.3. Database Test

An automatic test was created for the connection and operation of the application's internal MySQLite database. The basis of the test was provided by an aid to the program. Hafiz (2014).

 

The program created a database with a user table. After that, the program inserted user data, and then, controlled their existence. This test was also run without errors on each occasion.

 

The operation of database test was ensured by the db library DBCustomer, files of DBHelper, and Utilities class found in the application. After testing, test.db was downloaded with the help of Android Device Monitor. Data found in test.db were displayed in SQLite Browser.

Table 2 about the test and the screen shot can be found in Appendix G.

6.1.4. Conditional Tests: Individual values

Custom values testing were done with EditUserActivity Spinners. Two spinners prevent to define wrong values on this side. The first one defines the sex of the user (1: male, 2: female), and the second one defines the sport skill level. (1: Professional, 2: Amateur, 3: Recreation)

 

Only these values were modified during the test. Like in the first test, the msmas database file was downloaded, and database values were checked in SQLite Browser.

 

Table number 3 and screenshot about this test can be viewed in G Appendix.

 

6.1.5. Conditional Tests: Value ranges

Ranges are checked in two places in the application. These are the user’s birth year and month. The connecting input fields can be found on RegisterActivity and EditUserActivity pages. The test was performed on RegisterActivity page. Validation class checks the proper characters and format here. The user can only enter numbers. If not four characters were written for the year, and not two characters for the month, the application gives an error message about the proper format. Only years in the range between 1900 and 2015, and months between 0 and 12 are accepted.

 

Table number 4 with screenshot about this test can be viewed in G Appendix

6.2. Functional Testing (Black Box)

6.2.1. User Acceptance Testing

The developer is not the best for every task of software testing process because the developer sees the software working from different viewpoint than end users. This is why it was important to test the application prototype on other persons. First person was the project sponsor, and the second one was an active-sporting person. This test was important because it was recommended to check the usability of documents for this purpose with a Pilot Test.

6.2.1.1. Sponsor Acceptance Testing

Application test was done by the sponsor, Dr. Istvan Soos, in Darwin building of University of Sunderland Sport Campus at 10/03/2016. Testing was done with a previously, more than a week before given Amazon Fire 7 tablet and Mi Band fitness bracelet.

 

In the beginning of the test, the sponsor received the Participant Consent Form, Study Information Sheet, task list and questionnaire. Then, the sponsor approved of the participation by signing the consent form. His participant number became 002.

 

The result of the test was the following. The application performed all tests and other tasks flawlessly. The following few small problems occurred. During the second physical test, the balance task, the bracelet’s sensor terminated the task abortively. Relatively bad result came from the memory game. Fourth physical test, sustaining the squatting posture, was rated hard caused by the bad knee joints. There was grammar inexactness in text at question number 19. Test results were not saved to the database.

 

The following solutions came for the problems. Mi Band is a very cheap category product, which means its sensors are not really perfect. The integrated SDK that was used is not an official publication, so it may contain deficiencies and inaccuracies. It came to light during the developing process that the sensors showed not causeless significant excursion in motionless state, but it worked in most cases but solution was not found. Optimization of the application and minimization of flaw was successful because the program originally examined the values of accelerometer’s three axes, but it was not necessary. This measures the revolutions of 3rd most stable vale around the horizontal axis, which is enough for the completion of the task, so the values of two other axes are not checked. The bad result in memory game was caused by the sponsor’s bad skills, so it did not need modification. Flaw in the fourth task is caused by the user too, so no modification was needed here. The inexactness of 19th question was corrected. The fail of data saving was caused by the lack of task completion; however, the description was in the list. After rethinking the function, it was not necessary to use an independent SAVE button because the ‘Continue’ button at the last test means that the user wants to continue the process. The manual saving was removed from the application because saving happens automatically before the results are displayed.

 

The rating pointed on some minor flaws, which have been fixed, and made the application better. The sponsor’s consent form and questionnaire can be seen in G Appendix. The person’s data table number 5/1 can be seen there as well.

6.2.1.2. Sample User Acceptance Testing

As sample user a person was chosen, who frequently does sports activities. The chosen user is a player in 3rd division of local table tennis league and member of university table tennis team. Like the sponsor’s test, it was performed on 10/03/2016 at Darwin building, where later group tests were performed.

 

Participant number 001 received the documents that were made for this purpose, and verified with signature that he is participating in the test, but he did not allow audio and picture recordings. Test was performed with Acer Iconia One 7 and connected Mi Band devices. The application worked flawless during the test. The testing user made some comments about exercises on the questionnaire. The user rated hard the holding of the circle above the target area during the first physical test. Completion of tasks was not full because the usage of SAVE button was missed here too.

 

The following statements were made after rating the test. The program is well functioning and usable. The designed tasks are performable, and the questionnaire was capable for feedback. The differences between each device’s sensors were experienced during the early stages of developing. This means that the devices have different sensitivities, so performing the task on each device can be harder or easier. Possibly same devices will be used for testing, so their results are comparable to previous ones. The hardness of task surprised the user during the test, and the user started it without getting ready, so the user needed to correct the initial large displacement, and it was noticeable that the user tries to control the circle with disproportionately large movements. Probably in the next tests the user will know what to do exactly, and pays more attention to concentration and fine hand movement for task completing, so it will give better results. If controlling with sensors will causes problems in later tests, re-calibration of sensors will be necessary for control improving. The saving was simplified to eliminate this error due to previous test performed by the sponsor.

 

Scanned versions of Consent Form and Questionnaire of the participant can be found in Appendix G along with the data table number 5/2. During recording the table, it was noticed that the participant’s first name was not entered carefully. One e-mail address was entered here because the program only checks the existence of data which is not necessary for later usage, so the system was not required to change this data. In these cases, there is a possibility for changing the user’s data.

6.2.2. Alpha-Testing

Alpha Testing is always done in the development environment. This is why the test was performed on Virtual Devices that was made and managed by Android Studio. The created virtual device was Nexus 7 type with Google API21 operating system, 1 GB RAM, 800x1280 resolution and x86_64 processor support.

 

Virtual devices have limited abilities. Bluetooth connection was not available in the defined environment, so these functions could not be tested on installed Facebook Messenger and e-mail clients. Movement with built-in sensors were not available during the first physical survey, so only the graphics and counter routines test were checked.

 

Table number 6 was made about the test, which can be found in G Appendix with other test results.

6.2.3. Beta-Testing

Beta Testing is mostly done on real devices. Test was performed on Acer Iconia One 7 B1-760HD, Android version 5.0.1, which was bought for this purpose with connected Mi Band fitness bracelet. Additional applications were installed on tablet to fulfil the tests’ needs. These applications were Facebook Messenger and Gmail e-mail client.

 

Every function of the application was tested, and the wished results were checked during the test. No errors were experienced during the test. Only one design flaw was detected. Only one small part of the background of Progress Bar could be seen during the fourth physical test. This was caused by the wrong values in Layout File, which later were corrected.

 

Table number 7 was made about this, which can be found in G Appendix.

 

 

6.3. Compatibility Tests

Wide support was one of the project’s main principles, but the experienced compatibility problems during developing progress narrowed down the number of usable devices. It was important to change from initial API 19 version to API 21 version because the support of Low Energy Bluetooth 4.0.

 

The development of Prototyping on a One Plus One device was implemented without any flaws. Three pieces of Elephone P6000 Pro mobile phones with Android 5.1 operating system were available. These phones were not capable to be paired with Mi Band bracelet. The factory that made the Mi Fit application sometimes worked, sometimes not. At least five mobile devices with same features were needed for Usability Test, which can connect to Mi Band. First of all, Nexus 9 tablets were tested at University of Sunderland Computer Science campus. The Android 6 operating system on these devices could not connect to Mi Band accessories. Then, sport department’s Samsung GT-P5210 type tablets were tested. However, these devices’ Android 4.4 operating system could support the factory made application Mi Fit for Mi Band bracelet, but Bluetooth connection failed on these devices too.

 

Two low price category tablets with 5.1 version operating system and connectivity to BLE devices were found during researching. This is why 3 pieces of Amazon Fire 7 type tablets and 2 pieces of Acer Iconia One 7 B1-760HD type tablets were bought. Three versions of Mi Band fitness bracelets does exist, so this is why it was important obtaining one piece of each type. One piece of MI old type, 5 pieces of MIA type and 2 pieces of MI1S with heart rhythm sensors bracelets were bought.

All of the listed devices above were tested, and table number 8 was made about the tests which can be found in G Appendix.

 

Few different personal profiles were needed to be created for each different device. Factory made Mi Fit application was needed for pairing. Preinstalled Facebook Messenger and Gmail clients were needed for sharing. Amazon Fire 7 tablets run a special operating system, which allows the installation of applications via Amazon user profiles, so those needed to be created and configured on the tablets too.

 

Tables below contain the properties of these devices.

Gmail (http://gmail.com),

Xiaomi Mi (https://account.xiaomi.com/pass/serviceLogin),

Amazon (only for fires) (https://www.amazon.co.uk/ ),

and Facebook (https://www.facebook.com/)

accounts.

Device Number

Email address

Password

Type

MAC Address

1

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MIA

C8:0F:10:0B:D1:BF

2

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MIA

C8:0F:10:0A:64:A8

3

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MIA

C8:0F:10:15:0E:DD

4

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MIA

C8:0F:10:0B:45:64

5

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MIA

C8:0F:10:0A:61:34

6

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MI1S

C8:0F:10:32:BB:7F

7

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MI1S

C8:0F:10:32:62:C8

8

This email address is being protected from spambots. You need JavaScript enabled to view it.

********

MI

88:0F:10:89:57:43

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Device Number

Firmware ver.

Paired Device

Colour code

Facebook, Amazon first name

Last Name

1

5.15.12.10

AMAZON Fire 7

Purple

Msmasone

Uniofsunderland

2

5.15.12.10

AMAZON Fire 7

Blue

Msmastwo

Uniofsunderland

3

5.15.12.10

Acer B1-760HD

Orange

Msmasthree

Uniofsunderland

4

5.15.12.10

Acer B1-760HD

L. Green

Msmasfour

Uniofsunderland

5

5.15.12.10

AMAZON Fire 7

Red

Msmasfive

Uniofsunderland

6

4.15.12.10

One+ One

Yellow

********

********

7

4.15.12.10

Lenovo A806

Green

********

********

8

 

Lenovo A806

Grey

********

********

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Demo user:

Email: This email address is being protected from spambots. You need JavaScript enabled to view it.

Password: demo