The continuous evolution of technology allows us to establish new business objectives and strategies to create added value in organizations.
As main actors of this evolution, we find people. In order to satisfy the growing and constant market needs, this evolution requires from us both more speed and quality.
Apart from IT professionals, it is increasingly important that business users get involved. Their participation as well as their commitment partly define a project‘s success from its early stages.
The way in which business users get involved means, among other activities, performing the Acceptance Testing of the solution built. This participation is independent of the development methodology used.
What is User Acceptance Testing about?
According to IEEE Std. 610:
«Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system».
What would an ideal context for User Acceptance Testing be like?
An ideal context is that in which those who must accept the system are involved in the development process from the beginning. Also, those who sufficiently know the business and, finally, the allocation of time to perform the required tasks. We wonder what would happen if we had invested our human, technological and economic resources and the business needs were not clearly defined or transmitted...
resources and the business needs were not clearly defined or transmitted... In this case, users would receive a system that works correctly “on the desktop”, but which does not allow them to run their business in the “real world”.
Overall context
The expectation for a new technical solution is very high.
“Clients” generally expect the software to speed up their day-to-day tasks.
One of the most serious risks in software development is to fail due to not having developed the right product.
The right product is developed by departing from the requirements that result from the information business users provide when they “tell us what they need and/or want.”
Our IT task is making their requirements visible.
We must adapt the IT support to their business. It is in the Acceptance Testing where the user validates if the technical solution developed allows them to run their business (under the previously defined conditions).
Once at the Acceptance Testing stage, we assume that the software was revised by a testing technical team which did the following:
- all the interface testing
- went through all the logical paths of the code
- validated that it is possible to navigate the entire product
- that all the inputs and outputs of the screens work correctly and in order
- that the product faithfully corresponds to its definition
Requirements definition
Some industry data indicate that a significant proportion of the problem at this stage is related to the lack of business participation, ambiguous definitions, and conflicts between requirements, among others.
We draw from the premise that ‘requirements definition’ could mean different concepts for different actors. The parties involved must previously agree on the level of definition.
At this stage, business user commitment and intervention are vital. It has been demonstrated by some experiences in the industry that, given the limited time allocated for this task, part of the necessary definitions has been left in charge of development, and that we “have relied on telepathy.”
This can result in expensive reworks and delays in connection to previously agreed deadlines.
We wonder what would happen if the software is fit for purpose, but:
- It doesn‘t perfectly fit the business processes.
- Makes processes more complex, difficult, or extensive.
- Makes it necessary to define additional processes, rendering some old processes obsolete.
This information should be documented and agreed upon from the beginning, in such a way that the required attributes are “born” together with the product instead of being inserted afterwards.
All that is not somehow made explicit is likely not to be present in the final product. When it comes to defining the acceptance criteria of a new product, it is interesting to examine similar applications operating.
This means making use of a product that can provide us with parameters to define acceptance criteria.
It is also about not creating false expectations. Not all the functionalities of an application are critical. What is important is to establish some scale of values to prioritize.
When defining acceptance criteria, it is convenient to evaluate:
- How critical will this system be for the organization?
- What functionalities are the most critical ones for the business purpose?
- What functionalities are the most visible ones for the User?
- What functionalities have the greatest impact on security?
- What functionalities have the most significant financial impact?
- What aspects of the application are the most important ones for the business?
- What aspects of previous or similar projects caused problems?
- What aspects of previous or similar projects caused more maintenance expenses?
- What kind of problems would cause the most unfavorable publicity?
- What kind of problems would cause most of Customer complaints?
- What is the cost of a system failure?
- What critical decisions are made based on the information the system provides?
Testing Management
It is necessary to identify all the tasks to be carried out at this stage and then define:
- Who will perform each task?
- When it will be performed?
- For what purpose?
- How much time each of the assigned tasks will take?
A document with a brief description of how to perform the tasks assigned during the Acceptance Testing would be useful in terms of the definition of the time allocated to carry them out.
In other words, they will not need to investigate how to do their tasks.
It is not all the same
Each part of the system may have different acceptance criteria. For example:
- Online transactions used in the presence of a Client generally require a high level of performance
- a batch process that is executed at night
- or an online process for internal use that is only occasionally executed.
Another case would be that our application has a security module which -by organization policy- is only tested by the security area team, or by audit.
When should software acceptance be a contractual issue?
If the product is developed by an organization with a (traditional) software production methodology that includes the stages of Unit Testing, Integration Testing, System Testing, etc. we would then be facing a testing stage where we will only focus on ‘the business’ and how it operates.
We assume that the software must have been previously debugged and stabilized.
On the contrary, if this is the only testing that the application will have, the process will be more complex and difficult. The probability of being confronted with problems that we should not find at this stage is very high. If the software was developed under contract, it is the Client the one who must perform the Acceptance Testing, considering what was previously agreed for that product.
Results Documentation
Each development methodology has its own recipe for documenting or not the testing results. However, recording the failures found during the testing constitutes an important source of knowledge for an organization or development team to identify what is wrong and what must be improved. From the analysis of the registered failures, it is possible to obtain a common denominator that could result in the definition of new development standards.
Some aspects to consider about the business user community
- The users that have spent more time in the organization are not necessarily the ones who know the business process better, nor are they the most willing to collaborate in the Acceptance Testing process.
- Many users are not aware of how much they know about the entire business because only the input and output of the tasks they perform are involved.
- On many occasions, they do not know how to convey their knowledge. We should remember that, somehow, we speak different languages.
- A user that feels directly or indirectly harmed by: The appearance of a new application. The changes made to an already existing one. If the user‘s task becomes more complex or extensive, the user will not collaborate much.
- Regarding the training required, we must train users in the tools that will be used during the testing process.
- The time required for this training must be included in the project deadlines.
- It is important to consider if the automation of some processes is planned and the user is responsible for its execution. It must also be part of the training.
Brief final conclusion
My intention in this article is to convey how essential the commitment of some of the actors involved in the development process is. I also want to highlight the role of the business user as an essential and indispensable actor for the fulfillment of the objectives to achieve.
