Due to the lack of strategy, knowledge and resources related to testing, countless developer hours are spent fixing bugs, all at the expense of profits. This time and money could be spent on delivering new business value.
Save 70% of the time you spend fixing bugs.
Increase customer satisfaction
Creating a culture of ever-increasing quality and speed
Increasing profit margin
Your experts will be proud of their work
What is it?
Its purpose is to examine the satisfaction of the user and all beneficial owners (stakeholders). This level is primarily affirmative, so a good test produces a successful operation. For example, the confirmatory purpose is to detect defects that can only be investigated in the final operating environment.
What does it lead to?
Its task is to determine whether the system can be put into production.
How is it typically measured?
The number of tickets from users is decreasing.
To what extent do automatic tests cover functions, generic processes.
When can it be called successful?
It’s a business decision how many of the errors that come back from UAT are actually addressed.
It prepares go and no-go decision-making situations.
It is absolutely successful if, with proper testing, no tickets are returned and all test cases are run flawlessly.
Implementation tools
Automated tests and test robots that model end-user behavior.
What kind of customer preparation does it require?
Identified processes.
Test scenarios derived from user stories, test cases.
Economic view
The occasional price of a given UAT test is always the same if all other conditions are considered unchanged.
There is a critical UAT test volume, above which the automatic test with a higher entry threshold is more worthwhile, which, if the test scenario (and/or user story) changes, will have to be updated.
With automated testing, with each test run, the specific cost of the test becomes less and less.
Prerequisite
Test scenario and/or User story
Feedback management
Operational connection and coordination of customer service and development.
What is it?
Testing of a smaller component, a code-level element of the software. For example, testing the methods and functions created.
What does it lead to?
Testing will verify that the unit is functioning as expected. For a method, the resulting value is the same as expected.
How is it typically measured?
With manual and/or automatic tests, where we do not get the expected values.
When can it be called successful?
The smaller components and elements of the software work as expected.
Implementation tools
For example, JUnit, PHPUnit
Economic view
This is the lowest level of the code testing hierarchy.
Through the Unit test, we can ensure that when packing up the various newer development and test elements during the development process, regression errors and minor coding problems can be detected easily.
Using the Unit test, instead of a bug, there will be a mistake, which is cheaper: a bug comes back from the market, a mistake is caught during development before it goes live.
Therefore profit will not burn due to bugfixing, there will be no custmer dissatisfaction, the brand will not be damaged.
What kind of customer preparation does it require?
Developers need to build a unit test writing competency.
Prerequisite
Identify backbone processes (Business analyst does it)
Technological preparation, eg. Jenkins, for monitoring
Developer readiness
Feedback management
At the end of an iteration (Sprite), based on measurements and team recommendations, a decision is made on how much resources to spend on bug fixing in the next iteration.
Knowledge matrix
Developer competence, system knowledge of test engineering, knowledge of testing software, comprehensive test scenario
What is it?
Search for errors that occur during the assembly of modules. Testing, verifying communication between two software or components of the same software. Interface testing is one of the most important tests to ensure smooth and secure communication between the components of the software.
What does it lead to?
Communication between software and/or software components will be as expected.
How is it typically measured?
With manual and/or automated testing
When can it be called successful?
Communication between software and/or software components as expected.
Implementation tools
For example, manually, JMeter, Selenium, Silk Performer, Other automated tool, depends on whether web or desktop applications need to be tested
Economic view
Integration test:
A stable service is created, which ensures that new solutions work well with existing elements, and this maintains customer satisfaction.
Integration tests make it safe to sharpen versions, which in turn increases the number and frequency of returning receivers and users. If you amplify the retention, that’s fine.
Interface test:
Helps to ensure that different systems are well connected to each other in a product that provides complex services. Stable operation between systems relieves users of problems caused even with liability for damages.
Data leaking and data loss do not occur.
What kind of customer preparation does it require?
Knowledge of interaces. Integration procedure, creation and supervision of rules and expectations.
Prerequisite
Comprehensive business AND technological documentation of interfaces.
Cross-system Business logic workflow.
Feedback management
Interface and integration managers and the team are closely connected to the product owner.
Knowledge matrix
developer competence, system knowledge of test engineering, knowledge of testing software, comprehensive test scenario
What is it?
It targets the complete system. Typically, Black-Box test. Its primary purpose is to increase the level of trust of the buyer that the system meets the overall professional and business requirements.
What does it lead to?
System complies with specification and system design.
How is it typically measured?
With automated tests.
Errors that occur during the tests:
When can it be called successful?
Termination of the defect or a significant decrease in its frequency and effect.
Implementation tools
For example, JMeter, Selenium, Silk Perfomer
Other automated tools, depending on whether Web or Desktop applications need to be tested.
Economic view
If we define requirements more and more expediently and follow product development according to the agile methodology, we minimize overdevelopment losses. We also minimize underdevelopment loss, which is the source of many errors at this level.
What kind of customer preparation does it require?
Comprehensive knowledge of the business logic of the product, its structure, and test tools.
Prerequisite
Clearly list and adapt requirements for the continuous development of the product.
Feedback management
It is a business decision to decide how many errors that come back from the system test are actually addressed.
Knowledge matrix
Developer competence, system knowledge of test engineering, knowledge of testing software, comprehensive test scenario.
What is it?
A type of software testing that checks if a recent program or code change has adversely affected existing services. Regression testing is nothing more than the full or partial selection of already performed test cases, which are re-executed to ensure the correct functioning of existing functions.
What does it lead to?
It provides a list that shows whether there have been any errors in previously operating service units.
How is it typically measured?
With automated tests. With the number of errors that occur during the tests.
When can it be called successful?
Detect the error earlier before users become aware of it.
Implementation tools
For example, JMeter, Selenium, Silk Perfomer
Other automated tools depend on whether web or desktop applications need to be tested.
Economic view
If we define requirements more and more expediently and follow product development according to the agile methodology, we minimize overdevelopment losses. We also minimize underdevelopment loss, which is the source of many errors at this level.
What kind of customer preparation does it require?
Designing in sprints in an agile board, evaluating tests, managing feedback
Prerequisite
Being part of the development processes, test scenario based on user story
Feedback management
Ticketing, go-no go decision making
Knowledge
Developer competence, system knowledge of test engineering, knowledge of testing software, comprehensive test scenario
What is it?
It targets critical parts of the software. It contains cases in which the failure would render the subsequent tests ineffective.
What does it lead to?
It will give you a quick answer as to whether you should pass the current version of the software for testing. If successful, we can pass the product on to the next step in the process (transfer to testers or move on to version-build)
How is it typically measured?
Test cases selected in the test suite run without errors.
When can it be called successful?
Test cases selected in the test suite run without errors.
Implementation tools
Soon
Economic view
It makes the use of specific tester resources more efficient.
No build in production with a high bug risk
The delivery speed will be accelerated because the error can be noticed earlier in the value chain, so it does not result in extra resource allocation and confusion.
Highly prioritized tasks can be made even more certain and safe.
What kind of customer preparation does it require?
For the corresponding milestones, the design of separate test cases.
Prerequisite
Tech lead determines the critical points during development.
Feedback management
Improve the development, possibly reposition it and then continue the development.
Knowledge matrix
Developer competence, system knowledge of test engineering, knowledge of testing software, comprehensive test scenario
What is it?
Testing a certain, targeted detail of the software.
What does it lead to?
In general, it is advisable to apply it after code changes. Tests are targeting the features affected by the change, saving time while verifying that the affected code is correct.
How is it typically measured?
Test cases selected in the test suite run without errors.
When can it be called successful?
Test cases selected in the test suite run without errors.
Implementation tools
Soon
Economic view
The random test increases the team’s confidence in their work by ensuring that even random testing of the product runs without errors.
What kind of customer preparation does it require?
Existence of test cases for core functions, knowledge of core functions, need for targeted testing
Prerequisite
Pre-testing of previously developed codes or external systems.
Feedback management
Ticketing, go no-go decision.
Knowledge matrix
Developer competence, system knowledge of test engineering, knowledge of testing software, comprehensive test scenario
What is it?
Checking the resilience of the software to known attacks.
What does it lead to?
Its task is to determine whether it is possible to gain unauthorized access to sensitive/critical parts of the software.
How is it typically measured?
By running test scenarios that expose security vulnerabilities.
When can it be called successful?
When running test scenarios, not once does it manage to gain unauthorized access, to find a vulnerability.
Implementation tools
Safety test requirement documentation
Economic view
The storage and use of the data used by SW becomes more reliable.
SW’s license and SW itself cannot be stolen/copied by others.
What kind of customer preparation does it require?
The part highlighted in project planning and scheduling
Prerequisite
Defining security-intensive points.
What is it?
Examines the load capacity/performance of the software under basic, raised, and critically elevated loads.
What does it lead to?
Soon
Achievable goals
We make sure that the system responds to the given load.
What does it lead to?
The limitations and resource requirements of software/system performance become known.
How is it typically measured?
Of the written scenarios, parallel instances are run in bulk on the system at the same time, and – in addition to the effectiveness of the test scenarios – we monitor the load indicators of the system.
When can it be called successful?
If the achieved limits meet the requirements of the software (the software can stably handle the required amount of parallel task load with sufficient responsiveness)
Implementation tools
For example, Jmeter, Silk Performer
Economic view
It returns a result that will allow the SW owner/operator to serve its customers under SLA.
What kind of customer preparation does it require?
While using the system in real time, it will surely work well with the increased number of users.
The part highlighted in the planning and scheduling of the project and the expected estimate of the load.
Prerequisite
Description of the nature of use, plan
Preliminary load estimation
Seasonal effect, if any,
Scalability criteria/plan
Knowledge matrix
Soon
What is it?
A set of actions that are performed to control a specific function or functionality of the software application. A test case contains test steps, test data, prerequisites,added conditions that have been developed for a specific test scenario to justify any requirement. The test case contains specific variables or conditions.
What does it lead to?
It allows a tester to compare expected and actual results to determine whether a software product is operating in accordance with the customer’s requirements.
How is it typically measured?
For each function of the software, there is a sufficient number of test cases, which records the expected response to the different input states of the functionality
When can it be called successful?
The creation of test cases helped to improve the efficiency and quality of testing. Or an increase in the number of errors detected by test cases.
Implementation tools
HP-QC
Economic view
Effectively caught cases, increase the error-free nature of the product, so customers will be more satisfied, they will receive software that can be used with confidence.
What kind of customer preparation does it require?
Testing should be present when designing the project.
Prerequisite
Test testers or team member who replaced them determines the testing
Knowledge matrix
developer competence, system knowledge of test engineering, knowledge of testing software, comprehensive test scenario
What is it?
Automated tests are performed without human intervention. Their technical background is diverse: scripts, programming languages, special utilities. Automated tests can handle large amounts of tests and test data quickly, so they are suitable for tasks that manual tests cannot.
What does it lead to?
Tests can also be performed that are not feasible with manual testing or just not economically.
How is it typically measured?
By increasing and accelerating testing efficiency
When can it be called successful?
Testing time/cost efficiency increased.
Implementation tools
Jenkins, Ranorex. JMeter, Silk Performer, Selenium
Economic view
Excluding development costs, it can be run “for free”
Capable of instant reports, e.g. warning, ticket, notification – > fast information
It can be timed to run during periods when it is most ideal, e.g. no one is working on SW – > load of testing does not take away users’ resources.
What kind of customer preparation does it require?
Automated testing competency
Prerequisite
Test scenario
Runtime environment
Decide where to run (Production/Dev)
Information is received by whom and in what form
Knowledge Matrix
2
What is Checkstyle?
A set of syntactic rules defined by the development team, along which the written code should be formatted, including Lint.
What is the expected result?
The existing development team can perform peer-review more efficiently, as the code is easy to read due to the format written in the same way by everyone.
It is also easier to onboard new developers because the entire code base is written in a unified syntactic format, so there is only one set of rules to learn to read it.
How is it typically measured?
It is measured by checking if there is a set of rules for code formatting in the development team
When is it considered successful?
The development team has a set of rules for code formatting.
Economic view
The more uniformly the developers develop the software, the less room there is for making mistakes. This standardization does not kill the developers’ creativity. For example: If we write a Microsoft Word document together, it is recommended to use the same font style and size. The employees become more replaceable while the code becomes more easily transferable. It also becomes easier to integrate a new team member. Also, the code is more transparent when debugging.
What is needed on the customer’s side?
Principles set by the Architect/Lead Developer (the experts team might as well get involved).
Prerequisites
Coding standard within the experts team
What is Lint?
Lint is a developmental tool for building good developmental culture to control the specificities of the used programming language and recommend productive solutions.
What is the expected result?
Developers can get recommendations on productive ways to use the programming language in addition to or instead of the usual ways.
How is it typically measured?
It is measured by checking if the development team has Lint software and it is used in daily development.
When is it considered successful?
The development team has Lint software, and its use is a part of the daily routine. Also the experts team periodically discusses recommendations as they arise.
Economic view
The coding standard reduces the time for reviews, so code coverage can increase. As a result, the freed-up time can be reallocated without further time need. The mental burden coming from code legacy can be reduced.
Replaceability and the integration of a new team member is easier.
Knowledge transfer within the experts team is accelerated through standard coding practices. The experts team can react to potential bugs faster, making the whole development process faster and cheaper.
What is needed on the customer’s side?
Standards defined by the Architect/Lead Developer
Prerequisites
Available (community level) coding standards
Examining available tools (programming language, project, team preferences, etc.)
What is Code Coverage Testing?
A measure of how much of the production code is covered by tests.
What is the expected result?
It provides the stability of the code covered by the tests by guaranteeing that the expected functionality is preserved when the production code is modified.
How is it typically measured?
It is measured by using CodeCoverage tools to run the tests (e.g. JaCoCo).
When is it considered successful?
If at least 60-70% of the business-critical functionality is covered. It is even better if 60-70% applies to the coverage of the entire code base (while the coverage of the business-critical functionality does not fall below 60-70%).
Implementation
JaCoCo
Economic view
The optimisation of costs coming from the wastes of overdevelopment and underdevelopment.
What is needed on the customer’s side?
Goal setting depending on resources, time, market, product strategy, expected product lifecycle, and contractual conditions
Prerequisites
Knowing the profitability of the product in light of the expected testing costs and the planned code coverage
What is Branch Testing?
Branch Testing is less related to testing technology,. Rather, it is a method to reduce the possibility of errors.
Creating and following a branching strategy is important, without it development may include risks, e.g. conflicts resolvable and unresolvable by git, e.g. in case of automerge.
What is the expected result?
Coming soon
How is it typically measured?
Coming soon
When is it considered successful?
Coming soon
Implementation
Coming soon
Economic view
The branching strategy and the used tests can pre-validate the integration test, as a result of which resources are not used for relatively expensive tests whose focus elements can be performed with cheaper tools. Resources not spent at the integration level can be used to create additional value.
What is needed on the customer’s side?
Coming soon
Prerequisites
Coming soon
What is GUI Testing?
GUI Testing is used to avoid errors resulting from inputs that are mixed up in a form. This can be an automated GUI test, e.g. Selenium, or a manual one.
What is the expected result?
Based on the aforementioned example, checking for incorrect form filling and limiting it from the developer side will help ensure that the correct data is entered into the software.
How is it typically measured?
Unit tests based on validation scenario
When is it considered successful?
If only the right content/input is integrated into the software.
Implementation
For example Regex or other frontend validation methods
Economic view
The more secure the data entry and the more comprehensive the education of the customer is, the less chance accidental data loss or software malfunction has.
What is needed on the customer’s side?
Resources for development
Prerequisites
Validation planning
What is Beta, Stable Version Testing?
Beta, Stable Version Testing can increase customer satisfaction, that is, without it customer satisfaction can potentially decrease. For example if a non-stable software version is released or they cannot return to a stable one before users become aware of it.
What is the expected result?
Customers are already using the stable version, which is an active free UAT, version update is also safer.
How is it typically measured?
The number of bugs reported on the stable version decreases while feedback due beta/unstable anomalies as a bug report becomes less frequent.
When is it considered successful?
If customers are satisfied while using the stable version, the users of the beta version provide feedback.
Implementation
Live server duplication, version tracking
Economic view
Customer satisfaction with the stable version has increased while new upgrades become available earlier for customers who could use some development earlier.
Those using the latest version (beta) indirectly increase testing resources.
What is needed on the customer’s side?
Planning versioning
Preparing go/no-go/go-if decisions
Prerequisites
Coming soon
What is a test scenario?
A test scenario is a summary of test cases, not a specific test method. It declares testing needs, expectations and test cases while it identifies core processes and non-trivial test cases.
What is the expected result?
A document making the extent, quality, methods and aspects of the testing clear for the QA experts.
How is it typically measured?
It is typically measured with satisfaction on the QA expert’s side (with spider diagram), for example to what extent it is understandable, can be processed and reproduced, etc.
When is it considered successful?
If QA experts can perform their testing tasks without having any further questions.
Implementation
A document containing testing needs, procedures, expectations, and test cases.
Economic view
The software will be tested with the quality, content and coverage as described in the test scenario.
Test scenarios make it possible to use the resources available for testing in a self-organized and optimal way.
The testing process and content will be transparent to all project participants, both customers and suppliers.
What is needed on the customer’s side?
Knowledge of the operation of the developed software
Knowledge of test feasibility tools
Business domain knowledge
Prerequisites
Coming soon
© 2023