User experience
To complement the ‘benchmarking’ of the content filters relative performance and accuracy, the TestLab also sought to determine the impact of content filters on an average user’s ‘Internet experience’. A one-on-one user based testing methodology was selected as the most appropriate for this project as it provides the richest information and a high level of user feedback.
This basic user experience testing was added in an attempt to gauge the reaction of a small number of users running these applications and to discover whether they noted any performance impact using any of the filters. The test was a simple ‘blind’ test with the users unaware of what was on test. The TestLab polled six Internet users and asked them to run through a series of scripted tests, the user’s reactions were then recorded for each filter with specific emphasis placed on noting whether any of them noticed a performance issue. The test was run at the TestLab on a 100Mbps network, with a 5Mbps-7Mbps Internet connection. An automatic test tool and script was run to simulate 20 users accessing the Internet hence placing a user load on the Internet connection. Only one out of the six users noted any performance difference.
From the user experience testing, extremely little, if any, end-user performance impact could be attributed to the filters. Testing was carried out in a different environment and at faster speeds than an average Internet user would be accustomed to, therefore any speed difference/decline during the testing would be even more noticeable. At the more normal and slower speeds (say 256kbps or 512kbps) the result would most likely be even less noticeable; unless the upstream service provider’s aggregate bandwidth was being restricted by the content filter. In an ISP scenario this is an additional issue to be dealt with along with other performance bottlenecks relating to Internet service provision such as contention ratios, bandwidth, etc.
In summary the results for the user experience with regard to connection speed were:
- The participants noticed only little difference in connection speed between the individual filters,
- Only one out of the six users noted any performance difference,
- In the few cases where participants expressed they felt a slow-down of connection speed, they mostly felt the difference was minor and the speed still acceptable.
In addition, the user experience results for filter blocking were:
- Users felt insecure and in some cases frustrated if filters blocked content without appropriate feedback. Some of the filters blocked images without informing the users that the content was being blocked by filters, which caused participants to either blame the Internet Service Provider or assume they had made a mistake,
- Some of the filters caused some web sites to be blocked unnecessarily or interrupted tasks at critical points,
- In some scenarios filters crashed entirely or caused error messages to appear in the browser window that could not be recovered.
Back to Accuracy | Table of contents