How do we test and verify VPNs?

VPN evaluation and scoring method

Our VPN ratings are bifurcated: holistic assessments and situational assessments. Holistic assessments focus on the universal characteristics of a VPN service, while situational assessments focus on specific scenarios or user needs, such as device compatibility or geographic access.

Holistic Score

The overall score for all our VPN ratings is based on nine fixed testing sites:

Privacy and data storage policy (20%)
Speed (20%)
Security and technological features (15%)
Streaming compatible (15%)
Ease of use (10%)
Geographic distribution of servers (5%)
Torrent support (5%)
Customer Support (5%)
Bypassing restrictions (5%)

1. Privacy and data storage policy (20% of total score)

VPN data retention policies are of utmost importance. If a VPN stores redundant user data, it could become a potential target for cyberattacks or injunctions. Testing approach: We analyze each data retention policy to understand what data is retained, how long it is retained, and where it is stored. Following the analysis, the VPN's jurisdiction, past discrepancies, and user privacy features such as diskless servers are assessed. Multiple experts cross-check for accuracy. Evaluation Criteria: We have a specialized calculator to obtain objective and accurate estimates. Every VPN starts with a great score, with royalties based on logged data, with harsher penalties for more serious violations.

2. Speed (20% of total score)

VPN speed indicates the speed of data transfer. Faster VPNs provide uninterrupted internet access and media streaming. Testing Approach: Our proprietary speed testing tool records data transfer speeds between VPN servers in eight locations around the world. Each month we compare twelve readings from each server to arrive at an average. Evaluation Criteria: Download speed matters the most, especially for local connections. Ratings are assigned based on VPN performance compared to baseline speeds.

3. Security and technological features (15% of total score)

The primary role of a VPN is to provide an encrypted connection that protects your online activity. A security breach may result in data disclosure. Testing Approach: Tools like Wireshark and our own leak testing tools evaluate the strength of VPN security. Evaluation Criteria: Core features greatly influence the evaluation of a VPN in this sector. Points are awarded for successful encryption, IP address leak prevention and other technical functions.

All categories provide a breakdown of what each score means, helping users accurately determine the meaning of their scores.

4. Streaming compatible (15% of total score)

This score evaluates VPN consistency when accessing region-restricted content on platforms such as Netflix, BBC iPlayer and Hulu.

Testing Methodology: To evaluate the streaming capabilities, we connect to a VPN server in the desired region and try to stream content. Successful streaming requires smooth video playback, synchronized audio-video, and uninterrupted viewing. We have subscriptions to 20 of the world's leading streaming services. Popular platforms such as US Netflix and BBC iPlayer are tested each week, while less visited platforms are assessed monthly. We use this information to determine the percentage of reliability of each VPN service at quarterly and annual intervals.

Services that are regularly tested include:

Channel 4
HBO Max
Prime Video
Trustworthiness: 5%
And others...

Rating approach: VPN streaming scores are determined as follows:

0.1 – 0.9: Streaming errors
And so on... Our criteria allows experts to evaluate aspects such as video clarity and streaming smoothness to provide a score based on a range.

5. Ease of use (10% of total score)

This reflects the VPN software's user-centric design and ease of navigation.

Testing Methodology: Usability tests are conducted during everyday use of the VPN to evaluate the overall feel, ease of navigation of the interface, and clarity of server settings. This metric focuses on cross-platform VPN usage.

Rating approach: Unlike other indicators, this one depends more on the practical experience of users. Multiple testers eliminate bias, and measurable factors such as setup time and task clicks also play a role.

6. Torrent support (5% of total score)

This allows you to evaluate the effectiveness of a VPN in masking P2P file sharing while maintaining speed.

Test Methodology: In a controlled environment using qBittorrent and fixed speed hosting, we measure the VPN's performance on a 1GB file download.

Rating Approach: P2P VPN ratings take into account speed, security, logging policies, and more.

7. Geographic distribution of servers (5% of total score)

This evaluates the global distribution and range of VPN IP addresses.

Testing Methodology: We review VPN server lists, noting the total number of servers, countries available, and cities covered.

Rating Approach: The rating system uses quantitative data such as the number of servers, with little flexibility for certain qualitative ratings.

8. Customer Support (5% of total score)

This indicates the VPN's commitment to helping users.

Testing Methodology: We interact with support teams across all channels, assessing responsiveness, clarity and accuracy.

Rating Approach: Support ratings combine quantitative and qualitative elements, such as response time and agent experience.

9. Bypassing restrictions (5% of total score)

This tests the VPN's ability to access restricted web platforms.

Test Methodology: First-hand tests in China evaluate the effectiveness of each VPN when accessing websites such as YouTube.

Rating Approach: The rating measures the success of a VPN in China, but also takes into account aspects such as speed due to bandwidth limitations in censored regions.

Ratings for specific goals

For specific VPN applications or devices, our team develops customized testing and rating methods. The variables for these special ratings can range from "Mac App Compatibility" to "Netflix Reliability in the US." Detailed methods are presented on the corresponding pages.