Week 9 [Mon, Oct 16th] - Topics

A software requirement specifies a need to be fulfilled by the software product.

A software project may be,

• a brownfield project i.e., develop a product to replace/update an existing software product
• a greenfield project i.e., develop a totally new system from scratch

In either case, requirements need to be gathered, analyzed, specified, and managed.

Requirements come from stakeholders.

Identifying requirements is often not easy. For example, stakeholders may not be aware of their precise needs, may not know how to communicate their requirements correctly, may not be willing to spend effort in identifying requirements, etc.

Requirements can be divided into two in the following way:

1. Functional requirements specify what the system should do.
2. Non-functional requirements specify the constraints under which the system is developed and operated.

You may have to spend an extra effort in digging NFRs out as early as possible because,

1. NFRs are easier to miss e.g., stakeholders tend to think of functional requirements first
2. sometimes NFRs are critical to the success of the software. E.g. A web application that is too slow or that has low security is unlikely to succeed even if it has all the right functionality.

In a brainstorming session there are no "bad" ideas. The aim is to generate ideas; not to validate them. Brainstorming encourages you to "think outside the box" and put "crazy" ideas on the table without fear of rejection.

Studying existing products can unearth shortcomings of existing solutions that can be addressed by a new product. Product manuals and other forms of documentation of an existing system can tell us how the existing solutions work.

Observing users in their natural work environment can uncover product requirements. Usage data of an existing system can also be used to gather information about how an existing system is being used, which can help in building a better replacement e.g. to find the situations where the user makes mistakes when using the current system.

Surveys can be used to solicit responses and opinions from a large number of stakeholders regarding a current product or a new product.

Interviewing stakeholders and domain experts can produce useful information about project requirements.

Focus groups are a kind of informal interview within an interactive group setting. A group of people (e.g. potential users, beta testers) are asked about their understanding of a specific issue, process, product, advertisement, etc.

Prototyping can uncover requirements, in particular, those related to how users interact with the system. UI prototypes or mock ups are often used in brainstorming sessions, or in meetings with the users to get quick feedback from them.

Prototyping can be used for discovering as well as specifying requirements e.g. a UI prototype can serve as a specification of what to build.

A textual description (i.e. prose) can be used to describe requirements. Prose is especially useful when describing abstract ideas such as the vision of a product.

Avoid using lengthy prose to describe requirements; they can be hard to follow.

A common format for writing user stories is:

You can write user stories using a physical medium or a digital tool. For example, you can use index cards or sticky notes, and arrange them on walls or tables. Alternatively, you can use a software (e.g., GitHub Project Boards, Trello, Google Docs, ...) to manage user stories digitally.

User stories in use

With sticky notes

^{[credit: https://www.flickr.com/photos/jakuza/2682466984/]}

---

With paper

^{[credit: https://www.flickr.com/photos/jakuza/with/2726048607/]}

---

With software

^{[credit: https://commons.wikimedia.org/wiki/File:User_Story_Map_in_Action.png]}

---

---

The {benefit} can be omitted if it is obvious.

It is recommended to confirm there is a concrete benefit even if you omit it from the user story. If not, you could end up adding features that have no real benefit.

You can add more characteristics to the {user role} to provide more context to the user story.

You can write user stories at various levels. High-level user stories, called epics (or themes) cover bigger functionality. You can then break down these epics to multiple user stories of normal size.

You can add conditions of satisfaction to a user story to specify things that need to be true for the user story implementation to be accepted as ‘done’.

Other useful info that can be added to a user story includes (but not limited to)

• Priority: how important the user story is
• Size: the estimated effort to implement the user story
• Urgency: how soon the feature is needed

User stories capture user requirements in a way that is convenient for , , and .

[User stories] strongly shift the focus from writing about features to discussing them. In fact, these discussions are more important than whatever text is written. _{[Mike Cohn, MountainGoat Software 🔗]}

User stories differ from mainly in the level of detail. User stories should only provide enough details to make a reasonably low risk estimate of how long the user story will take to implement. When the time comes to implement the user story, the developers will meet with the customer face-to-face to work out a more detailed description of the requirements. [more...]

User stories can capture non-functional requirements too because even NFRs must benefit some stakeholder.

While use cases can be recorded on in the initial stages, an online tool is more suitable for longer-term management of user stories, especially if the team is not .

A use case describes an interaction between the user and the system for a specific functionality of the system.

System: Online Banking System (OBS)
Use case: UC23 - Transfer Money
Actor: User
MSS:
  1. User chooses to transfer money.
  2. OBS requests for details of the transfer.
  3. User enters the requested details.
  4. OBS requests for confirmation.
  5. User confirms.
  6. OBS transfers the money and displays the new account balance.
  Use case ends.

    

    


    
    

Extensions:
  3a. OBS detects an error in the entered data.
      3a1. OBS requests for the correct data.
      3a2. User enters new data.
      Steps 3a1-3a2 are repeated until the data entered are correct.
      Use case resumes from step 4.

  3b. User requests to effect the transfer in a future date.
      3b1. OBS requests for confirmation.
      3b2. User confirms future transfer.
      Use case ends.

  *a. At any time, User chooses to cancel the transfer.
      *a1. OBS requests to confirm the cancellation.
      *a2. User confirms the cancellation.
      Use case ends.

    

    


    
    

UML includes a diagram type called use case diagrams that can illustrate use cases of a system visually, providing a visual ‘table of contents’ of the use cases of a system.

In the example on the right, note how use cases are shown as ovals and user roles relevant to each use case are shown as stick figures connected to the corresponding ovals.

Use cases capture the functional requirements of a system.

A supplementary requirements section can be used to capture requirements that do not fit elsewhere. Typically, this is where most Non-Functional Requirements will be listed.

Combining parts of a software product to form a whole is called integration. It is also one of the most troublesome tasks and it rarely goes smoothly.

Build automation tools automate the steps of the build process, usually by means of build scripts.

In a non-trivial project, building a product from its source code can be a complex multi-step process. For example, it can include steps such as: pull code from the revision control system, compile, link, run automated tests, automatically update release documents (e.g. build number), package into a distributable, push to repo, deploy to a server, delete temporary files created during building/testing, email developers of the new build, and so on. Furthermore, this build process can be done ‘on demand’, it can be scheduled (e.g. every day at midnight) or it can be triggered by various events (e.g. triggered by a code push to the revision control system).

Some of these build steps such as compiling, linking and packaging, are already automated in most modern IDEs. For example, several steps happen automatically when the ‘build’ button of the IDE is clicked. Some IDEs even allow customization of this build process to some extent.

However, most big projects use specialized build tools to automate complex build processes.

Some build tools also serve as dependency management tools. Modern software projects often depend on third party libraries that evolve constantly. That means developers need to download the correct version of the required libraries and update them regularly. Therefore, dependency management is an important part of build automation. Dependency management tools can automate that aspect of a project.

Developer testing is the testing done by the developers themselves as opposed to dedicated testers or end-users.

Delaying testing until the full product is complete has a number of disadvantages:

• Locating the cause of a test case failure is difficult due to the larger search space; in a large system, the search space could be millions of lines of code, written by hundreds of developers! The failure may also be due to multiple inter-related bugs.
• Fixing a bug found during such testing could result in major rework, especially if the bug originated from the design or during requirements specification i.e. a faulty design or faulty requirements.
• One bug might 'hide' other bugs, which could emerge only after the first bug is fixed.
• The delivery may have to be delayed if too many bugs are found during testing.

Therefore, it is better to do early testing, as hinted by the popular rule of thumb given below, also illustrated by the graph below it.

The earlier a bug is found, the easier and cheaper to have it fixed.

Such early testing software is usually, and often by necessity, done by the developers themselves i.e., developer testing.

A test driver is the code that ‘drives’ the for the purpose of testing i.e. invoking the SUT with test inputs and verifying if the behavior is as expected.

public class PayrollTest {
    public static void main(String[] args) throws Exception {

        // test setup
        Payroll p = new Payroll();

        // test case 1
        p.setEmployees(new String[]{"E001", "E002"});
        // automatically verify the response
        if (p.totalSalary() != 6400) {
            throw new Error("case 1 failed ");
        }

        // test case 2
        p.setEmployees(new String[]{"E001"});
        if (p.totalSalary() != 2300) {
            throw new Error("case 2 failed ");
        }

        // more tests...

        System.out.println("All tests passed");
    }
}

    

    


    
    

JUnit is a tool for automated testing of Java programs. Similar tools are available for other languages and for automating different types of testing.

    // other test methods

    @Test
    public void testTotalSalary() {
        Payroll p = new Payroll();

        // test case 1
        p.setEmployees(new String[]{"E001", "E002"});
        assertEquals(6400, p.totalSalary());

        // test case 2
        p.setEmployees(new String[]{"E001"});
        assertEquals(2300, p.totalSalary());

        // more tests...
    }

    

    


    
    

Most modern IDEs have integrated support for testing tools. The figure below shows the JUnit output when running some JUnit tests using the Eclipse IDE.

In OOP code, it is common to write one or more unit tests for each public method of a class.

class Foo {
    String read() {
        // ...
    }
    
    void write(String input) {
        // ...
    }
    
}

    

    


    
    

class FooTest {
    
    @Test
    void read() {
        // a unit test for Foo#read() method
    }
    
    @Test
    void write_emptyInput_exceptionThrown() {
        // a unit tests for Foo#write(String) method
    }  
    
    @Test
    void write_normalInput_writtenCorrectly() {
        // another unit tests for Foo#write(String) method
    }
}

    

    


    
    

import unittest

class Foo:
  def read(self):
      # ...
  
  def write(self, input):
      # ...


class FooTest(unittest.TestCase):
  
  def test_read(self):
      # a unit test for read() method
  
  def test_write_emptyIntput_ignored(self):
      # a unit test for write(string) method
  
  def test_write_normalInput_writtenCorrectly(self):
      # another unit test for write(string) method

    

    


    
    

When writing JUnit tests for a class Foo, the common practice is to create a FooTest class, which will contain various test methods for testing methods of the Foo class.

public class IntPair {
    int first;
    int second;

    public IntPair(int first, int second) {
        this.first = first;
        this.second = second;
    }

    /**
     * Returns the result of applying integer division first/second.
     * @throws Exception if second is 0
     */
    public int intDivision() throws Exception {
        if (second == 0){
            throw new Exception("Divisor is zero");
        }
        return first/second;
    }

    @Override
    public String toString() {
        return first + "," + second;
    }
}

    

    


    
    

import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;

public class IntPairTest {

    @Test
    public void intDivision_nonZeroDivisor_success() throws Exception {
        // normal division results in an integer answer 2
        assertEquals(2, new IntPair(4, 2).intDivision());

        // normal division results in a decimal answer 1.9
        assertEquals(1, new IntPair(19, 10).intDivision());

        // dividend is zero but devisor is not
        assertEquals(0, new IntPair(0, 5).intDivision());
    }

    @Test
    public void intDivision_zeroDivisor_exceptionThrown() {
        try {
            assertEquals(0, new IntPair(1, 0).intDivision());
            fail(); // the test should not reach this line
        } catch (Exception e) {
            assertEquals("Divisor is zero", e.getMessage());
        }
    }

    @Test
    public void testStringConversion() {
        assertEquals("4,7", new IntPair(4, 7).toString());
    }
}

    

    


    
    

• Note how each test method is marked with a @Test annotation.
• Tests use assertEquals(expected, actual) methods (provided by JUnit) to compare the expected output with the actual output. If they do not match, the test will fail.
  JUnit comes with other similar methods such as assertNull, assertNotNull, assertTrue, assertFalse etc. [more ...]
• Java code normally use camelCase for method names e.g., testStringConversion but when writing test methods, sometimes another convention is used:
  unitBeingTested_descriptionOfTestInputs_expectedOutcome
  e.g., intDivision_zeroDivisor_exceptionThrown
• There are several ways to verify the code throws the correct exception. The second test method in the example above shows one of the simpler methods. If the exception is thrown, it will be caught and further verified inside the catch block. But if it is not thrown as expected, the test will reach fail() line and will fail as a result.

What to test for when writing tests? While test case design techniques is a separate topic altogether, it should be noted that the goal of these tests is to catch bugs in the code. Therefore, test using inputs that can trigger a potentially buggy path in the code. Another way to approach this is, to write tests such that if a future developer modified the method to unintentionally introduce a bug into it, at least one of the test should fail (thus alerting that developer to the mistake immediately).

Given below are some noteworthy JUnit concepts, as per the JUnit 5 User Guide.

• Annotations: In addition to the @Test annotation you've seen already, there are many other annotations in JUnit. For example, the @Disabled annotation can be used to disable a test temporarily. [more ...]

• Pre/post-test tasks: In order to allow individual test methods to be executed in isolation and to avoid unexpected side effects due to mutable test instance state, JUnit creates a new instance of each test class before executing each test method. It is possible to supply code that should be run before/after every test method/class (e.g., for setting up the environment required by the tests, or cleaning up things after a test is completed) by using test instance lifecycle annotations such as @BeforeEach @AfterAll. [more ...]

• Conditional test execution: It is possible to configure tests to run only under certain conditions. For example, @TestOnMac annotation can be used to specify tests that should run on Mac OS only. [more ...]

• Assumptions: It is possible to specify assumptions that must hold for a test to be executed (i.e., the test will be skipped if the assumption does not hold). [more ...]

• Tagging tests: It is possible to tag tests (e.g., @Tag("slow") so that tests can be selected based on tags. [more ...]

• Test execution order: By default, JUnit executes test classes and methods in a deterministic but intentionally nonobvious order. This ensures that subsequent runs of a test suite execute tests in the same order, thereby allowing for repeatable builds. But it is possible to specify a specific testing order. [more ...]

• Test hierarchies: Normally, we organize tests into separate test classes. If a more hierarchical structure is needed, the @Nested annotation can be used to express the relationship among groups of tests. [more ...]

• Repeated tests: JUnit provides the ability to repeat a test a specified number of times by annotating a method with @RepeatedTest and specifying the total number of repetitions desired. [more ...]

• Parameterized tests make it possible to run a test multiple times with different arguments. The parameter values can be supplied using a variety of ways e.g., an array of values, enums, a csv file, etc. [more ...]

• Dynamic tests: The @TestFactory annotation can be used to specify factory methods that generate tests dynamically. [more ...]

• Timeouts: The @Timeout annotation allows one to declare that a test should fail if its execution time exceeds a given duration. [more ...]

• Parallel execution: By default, JUnit tests are run sequentially in a single thread. Running tests in parallel — for example, to speed up execution — is available as an opt-in feature. [more ...]

• Extensions: JUnit supports third-party extensions. The built-in TempDirectory extension is used to create and clean up a temporary directory for an individual test or all tests in a test class. [more ...]