No. 8: Random and dynamic I/O patterns
Though it might be possible to perform 2-4 reads at once to a flash die, they each have to go to a separate plane of the die.
(Source: Jan Willemsen on Flickr)
If the two locations you want to read are located in the same plane, then the second read has to wait for the first one to finish. If there are 256 planes in the dies of your SSD, and it takes 50 outstanding reads to reach a desired bandwidth, what are the chances that one of those reads has to wait behind another read? The answer is 99% of the time. Search online for "The Birthday Problem" to find out why.
No. 9: Transport matters (PCIe vs. FC vs. SAS)
To take advantage of the speed and parallelism offered by flash products, it is important to choose the right interconnection technology between processors and data.
(Source: Mike Foote, Bo Insogna, and Scott McLeod on Flickr)
Choosing the wrong technology can eliminate the many advantages of flash. This is frequently seen when multiple SSDs are put in a standard disk drive shelf, where the SAS link to the server is filled by the input/output per second (IOPs) of two or three SSDs, so 90% of the potential performance in the shelf is never available. Attaching flash with PCIe or SAN links allows the full performance of large collections of flash to be fully utilized.
No. 10: Cost-efficient architecture
(Source: Chris Kim and Ian Lamont)
Because of the high performance and low cost now possible with a single SSD, many people have suggested moving back to a storage-in-the-server model and away from the SAN model that has come to dominate enterprise disk storage systems.
What often gets forgotten is the performance lost in communication between the servers needed for access and redundancy, the two or three times additional capacity needed for redundancy in that model, and the amount of flash that goes to waste. This means that the cost analysis is a lot more complicated than just comparing the dollar per GB of each product.