Samsung Releases Firmware Update to Fix the SSD 840 EVO Read Performance Bug
by Kristian Vättö on October 16, 2014 2:40 PM ESTThe news of Samsung's SSD 840 EVO read performance degradation started circulating around the Internet about a month ago. Shortly after this, Samsung announced that they have found the fix and a firmware update is expected to be released on October 15th. Samsung kept its promise and delivered the update yesterday through its website (download here).
The fix is actually a bit more than just a firmware update. Because the bug specifically affects the read speed of old data, simply flashing the firmware isn't enough. The data in the drive has to be rewritten for the changes in the new firmware to take place. Thus the fix comes in the form of a separate tool, which Samsung calls Performance Restoration Software.
For now the tool is limited to the 840 EVO (both 2.5" and mSATA) and will only work under Windows. An OS-independent tool will be available later this month for Mac and Linux users, but currently there is no word on whether the 'vanilla' 840 and the OEM versions will get the update. Samsung told me that they've only seen the issue in the 840 EVO, although user reports suggested that the 'vanilla' 840 is affected as well. I'll provide an update as soon as I hear more from Samsung.
The performance restoration process itself is simple and doesn't require any input from the user once started. Basically, the tool will first update the firmware and ask for a shut down after the update has been completed. Upon the next startup the tool will run the actual three-step restoration process, although unfortunately I don't have any further information about what these steps actually do. What I do know is that all data in the drive will be rewritten and thus the process can take a while depending on how much data you have stored in your drive. Note that the process isn't destructive if completed successfully, but since there is always a risk of data loss when updating the firmware, I strongly recommend that you make sure that you have an up-to-date backup of your data before starting the process.
The restoration tool has a few limitations, though. First, it will require at least 10% of free space or the tool won't run at all, and there is no way around the 10% limitation other than deleting or moving files to another drive before running the tool. Secondly, only NTFS file system is supported at this stage, so Mac and Linux users will have to wait for the DOS version of the tool that is scheduled to be available by the end of this month. Thirdly, the tool doesn't support RAID arrays, meaning that if you are running two or more 840 EVOs in a RAID array, you'll need to delete the array and switch back to AHCI mode before the tool can be run. Any hardware encryption (TCG Opal 2.0 & eDrive) must be disabled too.
In regards to driver and platform support, the tool supports both Intel and AMD chipsets and storage drivers as well as the native Microsoft AHCI drivers. The only limitation is with AMD storage drivers where the driver must be the latest version, or alternatively you can temporarily switch to the Microsoft driver by uninstalling the AMD driver. Samsung has a detailed installation guide that goes through the driver switch process along with the rest of the performance restoration process.
Explaining the Bug
Given the wide spread of the issue, there has been quite a bit of speculation about what is causing the read performance to degrade over time. I didn't officially post my theory here, although I did Tweet about it and also mentioned it in the comments of the original news post. It turns out that my theory ended up being pretty much spot on as Samsung finally disclosed some details of the source of the bug.
As most of you likely know already, the way NAND works is by storing a charge in the floating gate. The amount of charge determines the voltage state of the cell, which in turn translates to the bit output. Reading a cell basically works by sensing the cell voltage, which works by increasing the threshold voltage until the cell responds.
However, the cell charge is subject to multiple variables over time. Electron leakage through the tunnel oxide reduces the cell charge over time and may result in a change in the voltage state. The neighboring cells also have an impact through cell-to-cell interference in the form of floating gate coupling, which is at its strongest when programming a neighbor (or just a nearby) cell. That will affect the charge in the cell and the effect becomes stronger over time if the cell isn't erased and reprogrammed for a long time (i.e. more neighbor cell programs = more interference = bigger shift in cell charge).
Because cell voltage change is a characteristic of NAND, all SSDs and other NAND-based devices utilize NAND management algorithm that takes the changes into account. The algorithm is designed to adjust the voltage states based on the variables (in reality there are far more than the two I mentioned above) so that the cell can be read and programmed efficiently.
In case of the 840 EVO, there was an error in the algorithm that resulted in an aggressive read-retry process when reading old data. With TLC NAND more sophisticated NAND management is needed due to the closer distribution of the voltage states. At the same time the wear-leveling algorithms need to be as efficient as possible (i.e. write as little as possible to save P/E cycles), so that's why the bug only exists on the 840 and 840 EVO. I suspect that the algorithm didn't take the change in cell voltage properly into account, which translated into corrupted read points and thus the read process had to be repeated multiple times before the cell would return the correct value. Obviously it takes more time if the read process has to be performed multiple times, so the user performance suffered as a result.
Unfortunately I don't have an 840 EVO that fits the criterion of the bug (i.e. a drive with several months old data), so I couldn't test more than the restoration process itself (which was smooth, by the way). However, PC Perspective's and The Tech Report's tests confirm that the tool restores the performance back to the original speeds. It's too early to say whether the update fixes long-term performance, but Samsung assured that the update does actually fix the NAND management algorithm and should thus be a permanent fix.
The EVO has been the most popular retail SSD so far, so it's great to see Samsung providing a fix in such a short time. None of the big SSD manufacturers have been able to avoid widespread bugs (remember the 8MB bug in the Intel SSD 320 and the 5,000-hour bug in the Crucial m4?) and I have to give Samsung credit for handling this well. In the end, this bug never resulted in data loss, so it was more of an annoyance than a real threat.
95 Comments
View All Comments
Ebonstar - Friday, October 17, 2014 - link
Here's my anecdotal results from before/after running the fix on my 1TB 840EVO:Before: http://i.imgur.com/FL09X8p.png
After: http://i.imgur.com/nrkdBLg.png
I should also note the time taken for the SSD read speed util to run after the fix was far quicker, suggesting there really was quite a chunk of data that just wasn't reading fast at all.
VoraciousGorak - Saturday, October 18, 2014 - link
HDTach looked like a series of canyons for mine, but after running the utility HDTach flatlines at 450MB/sec sequential reads. It's worth looking at it again a month or two from now to see if the promise of a permanent fix holds up.LeadStarDude - Saturday, October 18, 2014 - link
Can this be done if the EVO is the OS drive? I have two of the 1TB 840 EVOs. One is my OS drive & the other is dedicated to Steam & my game library. I am currently running the repair tool on my secondary Steam drive. Hasn't finished yet... Anyway I am wondering if it is safe to run this repair tool if for the OS drive. Is that a no no, or is it safe? Thanks.the.tx.freds - Friday, November 14, 2014 - link
Unfortunately the Performance Restoration Software v1.1 killed my performance. Before and After using the performance benchmark in the sansung software bundle:Before
Random Read/Write IOPS Seq Read/Write (MB/s)
87137/66572 542/511
After
Random Read/Write IOPS Seq Read/Write (MB/s)
51771/48737 284/270
zhenya00 - Friday, October 17, 2014 - link
I applied this fix to a couple of our drives as soon as it was released. Straightforward and no data loss, but at least using the ATTO DiskBench tool that I've used for years, the results are mixed. When I first heard of this bug, I benched the drive and found that performance was abysmal compared to new. I ran DiskFresh and that mostly restored the performance - but only temporarily. A month later and I benched the drive again and it was as bad as ever. Applying the fix has restored the performance - mostly. At small block sizes it's still only a fraction of what it was new.I'm no SSD expert so anyone want to explain?
Anyhow, Samsung has already lost our business. Our newest round of SSD's purchased went to a different brand.
ThatMouse - Friday, October 17, 2014 - link
Unfortunately Samsung offers no help on how to roll back to the Microsoft AHCI drivers, so if you installed a 3rd party SATA controller that came with your motherboard, you will have a much harder time. You may as well pull your other drives and do a clean install.skeezix99 - Friday, October 17, 2014 - link
On my gaming system, the 840 EVO is connected via the Marvel 9xxxxx chip (4 1TB WDs and an 840 Pro on the Intel chip). When I ran the Samsung software, it flashed the new firmware on the 840 EVO, restarted, and could no longer see the SSD. I ended up connecting the EVO via the Intel chipset to run the "restoration", then moving back to the Marvel.Jzpunltd - Friday, October 17, 2014 - link
I was thinking of buying this for my macbook pro mid 2012 for photo and video editing. I also have a Windows desktop and I was wondering if I could connect the drive to one of the ports as a slave drive and update or does it have to be the OS drive?gpisic - Friday, October 17, 2014 - link
There is no such thing as a slave drive with SATA. However you will have to enable AHCI in the BIOS in order to be able to update the firmware. And it needs to be connected to a SATA port on the motherboard. Connecting it over an USB to SATA adapter cable won't work with the update.gpisic - Friday, October 17, 2014 - link
The performance restoration tool is very annoying, i had to update 5 unused SSD's today. The tool require the SSD to be initialized and to have a partition on it before fw update. Also you can't update only the firmware on a drive without data. You are forced to go through the whole refresh procedure even without data. Also the automatic shutdowns, very annoying :(