The 2021 AnandTech DAS Testbed and Test Suite

The evaluation routine for direct-attached storage devices – portable SSDs, storage bridges (including RAID enclosures), and memory cards – all utilize the same testbed and have similar workloads with slight tweaks based on the end market for the product. Our testbeds have kept pace with the introduction of new external interfaces - Thunderbolt 2, Thunderbolt 3, and USB 3.2 Gen 2 via Type-C. In mid-2014, we prepared a custom desktop based on Haswell, which was then upgraded to Skylake in early 2016. A botched Thunderbolt 3 firmware upgrade on the Skylake machine meant that we had to shift to the Hades Canyon NUC starting in early 2019. With USB 3.2 Gen 2x2 gaining traction, the inability to use an add-in card in the Hades Canyon NUC meant that we had to go hunting for a new DAS testbed platform.

I/O ports on PCs are enabled by one or more of the following approaches:

  • Direct interface from the SoC / CPU
  • Direct interface from the Platform Controller Hub (PCH) - either part of the package, or, discrete
  • External bridge chip interfacing between one of the direct interfaces above or other available high-speed I/O lanes and the relevant I/O port

The first option is the preferred one, followed by an external bridge chip that connects to the high-speed I/O lanes directly from the SoC / processor. Connecting via the PCH is the least preferred, as it can introduce DMI bottlenecks as the data shuttles between the processor and the PCH. In almost all Thunderbolt 3 platforms prior to Ice Lake, the OEMs ended up connecting the Thunderbolt controller to the PCH's PCIe lanes. One of the requirements for the new testbed, therefore, was to ensure that we had PCIe lanes off the processor exposed via multiple slots in a compact package (as similar to the previous testbed - the Hades Canyon NUC - as possible).

The 2021 AnandTech DAS Testbed

After considering various options in the market, we figured out that the Quartz Canyon NUC (essentially, the Xeon / ECC version of the Ghost Canyon NUC) was a good fit for our requirements. Intel provided us with a sample of the Quartz Canyon NUC, and ADATA helpfully sponsored 2x 16GB DDR4-3200 ECC SODIMMs and a PCIe 3.0 x4 NVMe SSD - the IM2P33E8 1TB.

The most attractive aspect of the Quartz Canyon NUC is the presence of two PCIe slots (electrically, x16 and x4) for add-in cards. In the absence of a discrete GPU - for which there is no need in a DAS testbed - both slots are available. In fact, we also added a spare SanDisk Extreme PRO M.2 NVMe SSD to the CPU direct-attached M.2 22110 slot in the baseboard in order to avoid DMI bottlenecks when evaluating Thunderbolt 3 devices. This still allows for two add-in cards operating at x8 (x16 electrical) and x4 (x4 electrical).

One of the issues with the Yottamaster PCIe AIC used in our previous USB 3.2 Gen 2x2 review was the need for a SATA power connector. SilverStone provided their SST-ECU06 add-in card based on the ASMedia ASM3242 host controller for use in the testbed.

The specifications of the testbed are summarized in the table below:

AnandTech DAS Testbed Configuration
System Intel Quartz Canyon NUC9vXQNX
CPU Intel Xeon E-2286M
Memory ADATA Industrial AD4B3200716G22
32 GB (2x 16GB)
DDR4-3200 ECC @ 22-22-22-52
OS Drive ADATA Industrial IM2P33E8 NVMe 1TB
Secondary Drive SanDisk Extreme PRO M.2 NVMe 3D SSD 1TB
Add-on Card SilverStone SST-ECU06 USB 3.2 Gen 2x2 Type-C Host
OS Windows 10 Enterprise x64 (21H1)
Thanks to ADATA, Intel, and SilverStone for the build components

The bus configuration of the testbed is brought out in the annotated picture below.

External I/O devices are capped at 40Gbps for the near future, with USB4 products just starting to appear in the market. The USB 3.x products top out at 20Gbps (for the Gen 2x2 variant), and all incoming products belonging to the USB 3.x category are evaluated using the Type-C host port enabled by the SilverStone SST-ECU06 add-in card. For Thunderbolt 3 devices, the Type-C port enabled by the Titan Ridge controller is used.

The plans for the new testbed were set in motion late last year, and our hope was that an add-in card with Thunderbolt 4 / USB4 support would become ready by the time Thunderbolt 4 / USB4 peripherals started rolling in. A system based on Tiger Lake such as the Beast Canyon NUC (with processor-attached Thunderbolt 4 / USB4 ports and spare PCIe slots for a USB 3.2 Gen 2x2 AiC) would have been an ideal single system capable of evaluating all types of direct-attached storage devices. Unfortunately, the Beast Canyon NUC was a long way off when we started planning for this testbed. When the USB4 / Thunderbolt 4 peripherals start to come in, our plan is to evaluate them using the Titan Ridge host as well as the Thunderbolt 4 ports off one of the Tiger Lake mini-PCs to figure out the performance differences. In the worst case, we might end up using a different testbed for Thunderbolt 4 / USB4 devices.

The 2021 AnandTech DAS Suite

The testbed hardware is only one segment of the evaluation. Over the last few years, the typical direct-attached storage workloads have also evolved. High bit-rate 4K videos at 60fps have become quite common, and 8K videos are starting to make an appearance. Game install sizes have also grown steadily, thanks to high resolution textures and artwork. Backups tend to involve larger number of files, many of which are small in size. The vendors have also appropriately responded, with 4TB bus-powered units already available in the market. Keeping these in mind, we have adopted some tweaks to our evaluation methodology.

The evaluation scheme for DAS units involves multiple workloads which are described in detail in the corresponding sections.

  • Synthetic workloads using CrystalDiskMark and ATTO
  • Real-world access traces using PCMark 10's storage benchmark
  • Custom robocopy workloads reflective of typical DAS usage
  • Sequential write stress test

The new test suite makes the following updates:

  • Updated CrystalDiskMark software version to 8.0.2 from 7.0.0
  • Increased ATTO file size from 8 GB to 32 GB
  • Increased CrystalDiskMark workload span from 8GiB to 32GiB
  • Restructured customm robocopy workloads, with multi-threaded copying enabled
  • Temperature tracking enabled during all idling intervals

The major update is in our custom robocopy workloads - the earlier version transferred around 42GB of data back and forth from the DAS over each of three iteration sets for a total of around 250GB of traffic. The new version ups this to around 95GB per iteration direction for a total of around 570GB of traffic.

The robocopy workloads have typically transferred data between the DAS and a RAM drive to remove any bottlenecks from the testbed's storage subsystem. In the new test suite version, we also include a disk-to-disk iteration set where the whole suite (around 318GB) is transferred from the CPU-attached NVMe SSD to the DAS and back. One of the interesting aspects discovered while developing the new workloads was that the high-speed DAS units were woefully under-utilized when subject to transfers using default robocopy parameters. Using the /MT:32 parameter helps queue up more traffic to the DAS and make optimal use of the capabilities of NVMe SSDs. A downside is the increased CPU usage compared to the non-multi-threaded copies, but that is rarely a concern even for processors operating in the 15 - 28W TDP envelop. Practically speaking, the ability to use the MT option effectively depends on how busy the host system is with other tasks taxing the CPU.

Introduction and Product Impressions Synthetic Benchmarks - ATTO and CrystalDiskMark
POST A COMMENT

16 Comments

View All Comments

  • meacupla - Thursday, August 12, 2021 - link

    Oh, look at that! a newish NVMe to USB bridge chip from ASMedia

    Even though I know this chip blows the competition out of the water, I would have liked to see how this chip compares to its predecessors, ASM2362, RTL9210 and JMS583.

    Was there any stability or drop out issues with these units? I didn't see any mention in the article, so I assume it was smooth sailing, which is nice to see.
    Reply
  • Drkrieger01 - Thursday, August 12, 2021 - link

    I'd also be curious to know if they experienced any drop outs during testing. I saw this a fair bit with some JMicron controller based units, but I found flashing to another firmware helped tremendously.
    Nice to see some new hardware available for those still using Sneaker-Mail.
    Reply
  • ganeshts - Friday, August 13, 2021 - link

    No issues with these two storage bridges attached to the ECU06. I did experience dropouts last year with the Yottamaster C5 [ as documented here : https://www.anandtech.com/show/16133/usb-32-gen-2x... ]. Reply
  • name99 - Thursday, August 12, 2021 - link

    "USB 2.0 ports are guaranteed to deliver only 4.5W (900mA @ 5V). "

    No. USB2 ports guarantee 2.5W. It's USB3 ports (blue connector) that guarantee 4.5W.
    And lots of USB3 stuff will just fail (possibly randomly) when connected to USB2 because
    - 2.5W is enough to get the drive awake and to read, maybe even an occasional write, but not enough for a run of sustained writes.
    - I don't think power negotiation is part of baseline USB3 spec. (Or if it is, most of that hardware doesn't seem to implement it properly.)
    Reply
  • repoman27 - Sunday, August 15, 2021 - link

    Yep. USB 2.0 is up to 2.5 W (500 mA @ 5 V), while USB 3.2 specifies up to 4.5 W (900 mA @ 5 V) for single lane operation and up to 7.5 W (1500 mA @ 5 V) for dual-lane operation. Reply
  • DigitalFreak - Thursday, August 12, 2021 - link

    USB 3.2 Gen 2x2, a standard that had little reason to exist. Reply
  • DigitalFreak - Thursday, August 12, 2021 - link

    I guess at least it wasn't called USB 3.3 Gen 3. Reply
  • back2future - Monday, August 16, 2021 - link

    within time, seen from now, yes, but what to recommend?
    Waiting for Thunderbolt 4 or building on USB 4.x?
    Reply
  • dwillmore - Thursday, August 12, 2021 - link

    I read the testbed page--which I usually skip--because I wanted to know where the heck you found a USB3.2 gen 2x2 port. AiC of course.... On the way I learned about the new Type-E connector for internal connection to case USB-C ports. So, that was nice. I even found a USB 3.2 Gen1x2 host adapter--which is a cursed item if ever there was one. Reply
  • watersb - Friday, August 13, 2021 - link

    One of my favorite segments to review right now. My laptop purchased in 2021 is by far the most capable device I own. USB4 or whatever they are calling it this week for fast mobile backup storage is a key part of the workflow.

    Utterly stupefying how clumsy the market messaging has been. The only way to know what I'm are buying is to test it myself. Or get a review from a source that I can trust to go beyond "it looks pretty and we have a relationship with this brand". YouTube reviews have become more sophisticated, but the professionals here help a lot. Thanks!
    Reply

Log in

Don't have an account? Sign up now