Testing the nation’s wireless networks is the largest project PCMag.com’s mobile desk has ever undertaken. It involved four staffers, 10 freelancers, two laptops, 10 modems, and 18 cities. We equipped two identical HP Elitebook 2540p laptops with modems from each of the six networks that provide relatively widespread 3G or 4G service in the U.S. We picked the Elitebook 2540p because we wanted a Windows 7 laptop with good power and battery life. For the modems, we took suggestions from each carrier and chose the AT&T USBConnect Lightning, the Cricket A600, the Sprint U301, the T-Mobile WebConnect Rocket, and the Novatel U727 and U760 for Verizon. (We have reviews of many of these modems in our Cellular Modem Reviews section.)
PCMag.com’s Testing Script
To test the networks, we used eight to ten locations in each city. At each location, we ran an automated test script with each modem. The script, written in Windows Powershell, performed a number of upload and download tests including:
FILE DOWNLOADS: The script downloaded 1 MB and 5 MB ZIP files via FTP from ftp.apple.com and Limelight Networks respectively, using curl 7.20.1 for Windows.
FILE AND PACkET UPLOADS: The script uploaded TCP data to an iperf server on a 50/20 Verizon Fios connection and uploaded a 1 MB ZIP file via FTP to Limelight Networks.
WEB ACTIvITY: Using curl, the script downloaded the entire Web pages PCMag.com and cnn.com and recorded both the “time to first byte” and the speed of the whole complex download.
SPEED TEST: The script ran the Ookla Speedtest available at speedtest.net, which is the official speed test chosen by the FCC to measure broadband speeds. The Ookla Speedtest measures both upload and download speeds.
CONSISTENCY: Any FTP test that took more than 60 seconds to transfer a megabyte was recorded as failed. That translates into a speed of 134 Kbps, which is below the ITU 3G standard of 144 Kbps. If the Ookla speed test did not return a valid download speed, that test was also recorded as failed.
How We Calculated the Speed Index
The PCMag.com Mobile Speed Index is a weighted average taking into account several factors. Of 100 points, 20 were for consistency; 40 were for FTP and speedtest downloads; 10 were for Web-page downloads; 10 for Web-page time to first byte; and 20 were for FTP, iperf, and speed test uploads. The HTTP-based speed test simulates the kind of traffic you’d see in either Web streaming or Web-page viewing, so it got a lot of weight. The index is normalized against the best result in each category for the geography being measured. (In other words, the best carrier in a given comparison for each test got the maximum score for each test.) Awards went to the carrier with the highest Speed Index in a given city or region. Sprint 4G and Cricket were not eligible for regional and national awards, and were not included in the regional and national Speed Index calculations, because they weren’t available in all the cities we tested.
Frequently Asked Questions
WHAT DOES ALL THIS TELL YOU? Our tests will tell you about Internet connections from smartphones, tablets, and laptops. The results tell you which networks are the fastest and which are the most consistent. The “time to first byte” is especially relevant with Web pages, as it tells you how long it takes before things start appearing on your screen.
HOW DID WE PICk THE CITIES? We wanted a mix of larger and smaller cities, distributed over regions of the country, including several cities with Cricket and Sprint 4G service. We tested in 20 cities, but had to throw out two (Philadelphia and Las Vegas) because of technical problems. Of the 18 cities we ended up with, ten have Cricket and nine have Sprint 4G.
WHAT ABOUT DROPPED CALLS? We didn’t test anything involving phone calls. Yes, it’s important, but there are too many variables involved. Whether a call drops is about the individual phone as well as the network.
Source of Information : PC Magazine July 2010
PCMag.com’s Testing Script
To test the networks, we used eight to ten locations in each city. At each location, we ran an automated test script with each modem. The script, written in Windows Powershell, performed a number of upload and download tests including:
FILE DOWNLOADS: The script downloaded 1 MB and 5 MB ZIP files via FTP from ftp.apple.com and Limelight Networks respectively, using curl 7.20.1 for Windows.
FILE AND PACkET UPLOADS: The script uploaded TCP data to an iperf server on a 50/20 Verizon Fios connection and uploaded a 1 MB ZIP file via FTP to Limelight Networks.
WEB ACTIvITY: Using curl, the script downloaded the entire Web pages PCMag.com and cnn.com and recorded both the “time to first byte” and the speed of the whole complex download.
SPEED TEST: The script ran the Ookla Speedtest available at speedtest.net, which is the official speed test chosen by the FCC to measure broadband speeds. The Ookla Speedtest measures both upload and download speeds.
CONSISTENCY: Any FTP test that took more than 60 seconds to transfer a megabyte was recorded as failed. That translates into a speed of 134 Kbps, which is below the ITU 3G standard of 144 Kbps. If the Ookla speed test did not return a valid download speed, that test was also recorded as failed.
How We Calculated the Speed Index
The PCMag.com Mobile Speed Index is a weighted average taking into account several factors. Of 100 points, 20 were for consistency; 40 were for FTP and speedtest downloads; 10 were for Web-page downloads; 10 for Web-page time to first byte; and 20 were for FTP, iperf, and speed test uploads. The HTTP-based speed test simulates the kind of traffic you’d see in either Web streaming or Web-page viewing, so it got a lot of weight. The index is normalized against the best result in each category for the geography being measured. (In other words, the best carrier in a given comparison for each test got the maximum score for each test.) Awards went to the carrier with the highest Speed Index in a given city or region. Sprint 4G and Cricket were not eligible for regional and national awards, and were not included in the regional and national Speed Index calculations, because they weren’t available in all the cities we tested.
Frequently Asked Questions
WHAT DOES ALL THIS TELL YOU? Our tests will tell you about Internet connections from smartphones, tablets, and laptops. The results tell you which networks are the fastest and which are the most consistent. The “time to first byte” is especially relevant with Web pages, as it tells you how long it takes before things start appearing on your screen.
HOW DID WE PICk THE CITIES? We wanted a mix of larger and smaller cities, distributed over regions of the country, including several cities with Cricket and Sprint 4G service. We tested in 20 cities, but had to throw out two (Philadelphia and Las Vegas) because of technical problems. Of the 18 cities we ended up with, ten have Cricket and nine have Sprint 4G.
WHAT ABOUT DROPPED CALLS? We didn’t test anything involving phone calls. Yes, it’s important, but there are too many variables involved. Whether a call drops is about the individual phone as well as the network.
Source of Information : PC Magazine July 2010
|
|
0 comments
Post a Comment