Our hosting partner in Sweden, Elastx, recently partnered with SolidFire as part of a study by Cloud Spectator. They performed various tests to measure cloud I/O performance with Amazon AWS, Rackspace and Elastx (powered by Jelastic) and the results are extraordinary!
- Elastx processed the I/O requests 69% faster than Rackspace
- Performance for Elastx was also more consistent than Rackspace
- Elastx's Average read performance was 22,590 operations per second – more than twice the performance of Amazon Web Services and nearly 37 times the performance of Rackspace Cloud!
Below is an expert from the final report:
Elastx significantly outperformed Amazon and Rackspace in both overall performance and consistency.
Measuring Cloud I/O Performance
The tests used to measure performance were run from June 25th through July 9th , 2013 for 15 consecutive days – and focused mainly on I/O performance.
Disk I/O using SysBench
SysBench measured disk I/O performance using many parallel requests. Elastx significantly outperformed Amazon and Rackspace in both overall performance and consistency. It took Elastx (powered by Jelastic) 56 seconds on average to perform the requested I/O, and 182 seconds for Rackspace. This means that Elastx processed the I/O requests 69% faster than Rackspace. Performance for Elastx was also more consistent than Rackspace, with Elastx producing a relative standard deviation of 5% from the average score and Rackspace producing 7%. AWS vs Rackspace - AWS scored slightly lower than Rackspace on these measures.
A High-Performance PaaS
Elastx built its cloud PaaS powered by Jelastic on a scale-out, all-SSD storage system from SolidFire, specifically designed to allow guaranteed service quality to a large number of applications at cloud scale. Instead of simply boosting average or maximum I/O performance like standard SSD solutions, SolidFire produced an end-to-end storage architecture specifically for multi-tenant public cloud environments. This allows Elastx to produce guaranteed performance and enables I/O-sensitive applications and databases to realize extreme levels of performance.
Sysbench is used to evaluate servers by simulating a database running under intensive load. It is primarily used in the industry for MySQL benchmarking. The test in this table is a Sysbench run to evaluate file I/O with 32 threads, all running in parallel. With the results, Sysbench times the seconds until completion of the benchmark, so a lower score in seconds represents a higher efficiency for a database server.
Mongoperf checks disk I/O performance independently of MongoDB, though the application comes with the installation of MongoDB. Mongoperf was run without* any caching that might increase the I/O results, which would give an unrepresentative sample of disk performance. The test above isolates read performance.
The comparison chart above shows Elastx read results with cache enabled because the underlying virtualization platform always uses a read cache. However, since it is not a fair comparison to both Amazon and Rackspace a separate benchmark test was set up that was able to avoid caching and deliver more accurate results. The secondary test result showed an average read performance of 22,590 operations per second – more than twice the performance of Amazon and nearly 37 times the performance of Rackspace.
Mongoperf checks disk I/O performance independently of MongoDB, though the application comes with the installation of MongoDB. Mongoperf was run without any caching that might increase the I/O results, which would give an unrepresentative sample of disk performance. The test above isolates write performance.
About the Test
For this testing effort, Cloud Spectator compared the base cloud offerings of Elastx (powered by Jelastic), Rackspace and Amazon.
Tests were automated to run 3 times a day over 15 days using a limited version of Cloud Spectator’s CloudSpecs application, which runs a series of CPU, disk, RAM, network, and application tests automated across providers. Only tests designed to measure disk I/O were included in the measurements.