Milestone 12: Unit Tests, Behavioral Tests, Automated Testing

Initial due date: February 4 @11:59pm
Final due date: Same as 1.0 Release
Grading: Team

For this milestone you will implement unit tests, using a third-party unit-testing framework, as well as some form of behavioral tests. You will also devise a method for automating some of this testing and making it part of your workflow.

Background

Yes. You do not need testing. It is a waste of time. If your goal is to write a program that is just good enough for me to approve it and which will be deleted the moment you get your grade then, yes, testing is just a waste of your time.

Imagine, however, that this program you write is actually used by people: 1, 10, 100, 1000, then 10,000 by the end of the year. These are your paying customers. Your salary depends on their happiness with your app. When they ask for a new feature, or a bugfix, you want to make sure that the new code you just added does not break something else: erase their data, reveal their private information to others, crash your website, etc. You do not want to get 100 emails every day about some bug you have no idea how to fix.

This is where testing comes in. The goal of testing is to make sure, super sure, that your code works the way it is supposed to before you hand it over to your customers.

Unit Testing

In unit testing you write tests which verify that your methods/functions do what you think they should do. If a function is meant to sort an array of integers, then you write a unit test that calls that functions with a few example arrays and then check the result to make sure they are sorted. You are a programmer, so you know that it is most likely that errors will appear in boundary conditions: when the array is empty, when it has 1 element, when all the elements are the same, when they are in inverse order, etc. So, your examples are informed, not just random.

Unit testing has been around for decades and there are now many Unit testing frameworks for every programming language imaginable. The most popular of these tie in with your IDE to give you pretty output. For example, in eclipse you can run junit tests and get a special window with red marks for the tests that failed and green marks for those that passed. You will use a unit testing framework in your code.

The unit tests are also part of your repo. Make sure to push them to github.

You will also set it up so that running these tests is a simple matter of just a click, or one command. Everyone should run all tests, and make sure they pass, before they push their changes to github.

Behavioral/Integration Testing

In behavioral testing you make sure your app does what the user expects it to do, that is, what your Specifications say. This is functionality at a much higher level than the unit-testing functionality, so you are testing many parts of your code.

Some of this testing is done by just having people use the app. Unfortunately, that does not scale well. You want to be able to perform some of these tests quickly and automatically for every major code change. Luckily there are tools that help us automate this type of testing.

For example, if you are building a webapp then you might want to use Selenium which lets you automate web browsing, that is, write a program that emulates a user using a web browser to access your site, do some stuff, get results, and check that those results are what he should be getting. There are many other website testing libraries out there, such as casper.js, Nightwatch, Phantom, and others. Android has built-in libraries for both unit and behaviors (which they call "instrumented") testing.

Deliverables

For the first deadline:

  1. Have at least one unit and one behavioral test in your repo.
  2. In your README file add a "Testing" section which lists
    1. the commands/instructions one must type to run your tests
    2. the directory (or filename regexep) where all your tests are located

For the final deadline:

  1. Multiple unit tests (even if they are silly)
  2. Multiple behavioral tests. Test the core functionality of the app.
  3. In your README file add a "Testing" section which lists
    1. the commands/instructions one must type to run your tests
    2. the directory (or filename regexep) where all your tests are located