Firewall Product Testing
Introduction
Network firewalls are systems that enforce access control policy between an organization's network and the Internet for security purposes. Functionally, firewalls work closely with a router program to examine each network packet and determine whether to forward it toward its destination. Firewalls also include or work with a proxy server that makes network requests on behalf of workstation users. Firewalls come in different forms as hardware and software based. Testing for a wide array of hardware/software levels of the firewall has not only became a recommended procedure but also an essential part because of the sensitivity of the domain it addresses. Firewall product testing may be primarily categorized in three areas: Functionality, Performance/Stress, and Security. Firewall Functionality testing simulates the "real-world" scenarios and compliance testing. Performance and Stress Testing is done to isolate any scalability issues or limitations of the firewall product and determines the performance o f the firewall at boundary or near-boundary levels defined. Security Testing covers the vulnerability of the firewall product against DoS, DDoS attacks, application/port vulnerabilities.
Firewall Compatibility / Functionality Testing
Firewall Compatibility / Functionality Testing is done to verify that the product under test functions correctly without anomalies, difficulties or discrepancies. This phase of testing is performed to test and verify the functionality of primarily the following features:
- Inbound and outbound traffic support for FTP, HTTP(s), SMTP, DNS, ICMP, NAT, PAT protocols at allow and denial of service levels supported.
- Access Control and QoS via filtering at user, group, address, application, Time, Access rules, etc.
- Traffic behaviors during reboots or restart of the firewall
- Test recovery plans for firewall system failures
- VPN tunnels management using different encryption and hashing methods and functionality of common protocols through VPN tunnel: (SMTP, HTTP, FTP, etc)
- Security Policy Configurations using the available administration interfaces like CLI, HTML, GUI or any proprietary or non-proprietary application.
- Policy/Rules management of the product via remote/local mode method.
- Verification of logging features: Activity, Intruder Access, Traffic Statistic Logs, external logging and type-based logging.
- Real Time monitoring or alert features supported
Performance and Stress Testing
Performance and Stress Testing isolates any scalability or limitation issues of the firewall product and determines the performance of the firewall. Following tests may be performed to have a check on this area:
- FTP/HTTP Throughput test: This test determines the maximum throughput the firewall can provide under controlled conditions of rules/policies configured, number of clients connected etc.
- FTP/HTTP Stress test: This test determines the maximum load the firewall can support under extreme conditions over HTTP/FTP connections in long haul real world scenarios.
- FTP/HTTP throughput over VPN Test: This test determines the maximum throughput achieved through the Firewall that can be achieved by a VPN client over a VPN tunnel.
- TCP Connection Setup: This test determines the maximum rate of TCP sessions that can be established through the Firewall.
- TCP Session Rate: This test determines the maximum rate of TCP establishment and teardown that can be established through the firewall.
- TCP Connection Data: This test determines the maximum amount of concurrent connections that can be sustained by the firewall.
Security Testing
Security testing determines the level of vulnerability in the firewall against different attacks like dos, ddos, application/port vulnerabilities. Following tests may be performed to have a check in this phase:
- Management Access (Local & Remote) Test: Verifies that unauthorized access cannot be established to the configuration and remote console
- Port Scan (TCP & UDP) Test: Performs an exhaustive TCP & UDP port scan of the trusted network and firewall to identify all hosts with TCP & UDP services running or in a listening state. Verify any information of hosts hidden by the Security Policy cannot be compromised
- Operating System Detection / Stack Fingerprinting Test: This test attempts to determine operating systems of all hosts within the trusted network. Verify any information of hosts hidden from the Security Policy cannot be compromised.
- Network Ping Sweep Test: Utilizes the network ping sweep of trusted network from the Public Network. Verify that the Network Ping Sweep cannot compromise IP address or other information of any host that is not permitted access from the Public Network.
- Denial of Service (DOS) vulnerability assessment test: This test determines if the Firewall Product can be made inoperable when attacked with common types of Denial of Service attacks
- ActiveX/Java applet filter Test: Verifies the ability of Firewall to block ActiveX and Java applets when accessing any Web Site.
Case Study
The Customer
A Large Networking and Firewall vendor
The challenge
This customer had recently launched a new firewall product line to enhance their portfolio. The product needed to be tested for its entire functionality
The solution
Equipped with varied experience in networking and security domain QA processes the CalSoft team worked closely with the customer to have new test plans and processes in place. CalSoft helped the customer to meet the stipulated deadlines for the launch of the product via in-place and in-time completion of QA cycle. QA Team helped in elimination of defects much before they reach the field. Verifying the product, right from the inception stage has helped deployment of a mature regression strategy and ensured that the products meet the stringent quality standards. This has allowed the customer to greatly reduce product defects before reaching the field.