Questions
In the projects we work on we want to answer the following concerns and questions:
- Will my application be stable in production?
- Will my system be able to respond quickly so that the performance does not hinder business?
- Will I need to make larger hardware purchases later this quarter?
- Does the system fail over correctly in case of a disaster?
- What is the breaking point of my system (in terms of users or transactions)?
- Will my system meet the SLA I have promised my business sponsor?
- Will the system be able to handle the heavy month end processing?
- Will my network be able to handle the heavy load (even from the smaller offices)?
- What should my users expect in regards to response times?
- Should I accept this system based upon the non-functional requirements?
Answers:
Thorough Performance Testing is the answer to all of the previous questions.
The goal of Performance Testing is to check, whether a system meets performance, Response times (end to end), Resource usage (CPU, memory, disk, network), Capacity handling (workload), Stability, Availability.
We provide a suite of performance tests, diagnoses and benchmarking to ensure optimal application and system performance. This helps prevent the following as it is always better to find the problems in testing then to have your users find them in production.
- Delay of project deployment due to bad performance (longer time to market)
- Image Disruption, if your site or application is not working your customers may lose faith in your company, or worse take their business elsewhere
- Loss of productivity due to poor or slow performance and time equals money
- Lowers your risk of downtime
- Helps prevent large performance defects from reaching production
- Best way to verify that failover and balancing is working
Tools
PVO performance is focused on delivering the right tool for your needs. We believe that LoadRunner is one of the most versatile and useable tools in the market today (it covers over 90% of the technologies in the market today) along with the monitoring and diagnostics tools is our initial recommendation.
Sometimes our clients need a quick and cheap solution, and in those situations, we recommend Jmeter, or another tool specific for that system or test. We have specialised consultants in the firm who are considered experts on the Performance Test Tool Set and can help you with all your needs.
Performance Types
In the performance testing, validation and optimisation area there are many types of performance tests available.
We believe in selecting those tests which best help to answer our client’s questions, and therefore will tailor the following test executions to best meet our client’s project needs.
Load Testing
- Typical usage under normal Load with differing load levels
- What performance should my users expect on a normal day under differing load levels?
Stress and Peak Performance Testing
- Testing of the system or component until something breaks
- May be an unrealistic load in order to determine a specific system condition (or break point)
- Uncover the high-water mark for a specific area or process
- At what level will something in my system break?
- Typically, a “knee” result where something breaks and an exponential increase in utilization or response occur
Endurance (also called Soak) Performance Testing
- A typical load level over a long duration (12, 24, 48 hours for example)
- Great for finding memory leaks or other issues that are not uncovered without a long duration of usage
Scalability
- Continue adding typical users to the system until the breaking point is uncovered
- Understand what the capacity is of the system, assuming regular usage increases
Elasticity Performance Testing
- Adding and removing users to simulated the peaks and ebbs of the users during a normal working day (for example at lunch time ½ the users log off the system)
- Great for understanding how the system allocated resources and if they are returned to the pool when they are no longer needed or Testing of AutoScaling
Technical Intake Report
Short advice recommended for the duration of the scripting and the usage of LoadRunner/Jmeter on the system under test.
Proposal
This will be a detailed document which will contain all of the timelines, deliverables, requirements, commercial costs, and expectations in order to have a successful project.
Test Plan
This will include all of the information related to the Performance Testing including but not limited to: Business goals, Traceability matrix (if relevant), Load Model, Business Processes, Test Activities, Monitoring Plan, Test Types, Detailed Execution Plan, Key Personnel and Contacts. Basically, everything you need to know about that Performance Testing Activities.
Ad-hoc Reporting
After each test run (or test day) a short one-page result will be provided to the key stakeholders so that they are fully included in the testing process.
End Report
This contains all of the key results findings uncovered during the testing and recommendations. Also contains anything not performed or executed and why.
Status Report
A weekly update on the project status.