A dual processor system sounds awesome to the home user but in reality it is almost entirely a professional market. The prosumer has to use Xeons at JEDEC memory speeds and then ensure that the software is NUMA aware, especially if it decides searching for data in the other processor's L3 cache. However now GIGABYTE Server is selling to the prosumer via Newegg, and they sent us the $640 GA-7PESH3 for review.

For most users, a dual processor system affords several issues, aside from the cost. Performance with a 2P system is very dependent on the software in use. With a single processor system, each core can ‘snoop’ into the other core cache in order to see how data is updated. In a 2P system, the latency of talking between the two CPUs is at least an order of magnitude higher. This means that memory accesses can be delayed causing branched and locked code to be slow. In order to get around this, the software has to be NUMA aware, and the majority of regular applications are not.

There is no overclocking with a 2P system, and single thread speeds can be lower. As a result, gaming often sees a hit in performance, as well as basic tasks. The optimal use case scenario, for most software that is not aware of dual processor architecture, is any workload that needs as few memory accesses as possible and is ‘embarrassingly parallel’ such as ray tracing, video editing, virtualization or particular types of scientific compute. 2P motherboards, particularly those built by server teams, also often come with system management tools not seen in the consumer space, allowing users to access their system as it processes data and monitor progress as well.

GIGABYTE GA-7PESH3 Overview

This motherboard is built with storage and GPUs/coprocessors in mind. Although each CPU socket has four DRAM slots (rather than a more usual complement of eight each), the PCIe is a full array of PCIe 3.0 slots offering x16/-/x16/-/x16/-/x16 or x16/x8/x8/x8/x8/x8/x8 whether the system is using four PCIe devices or seven. This means that the PCIe lane allocation comes from both CPUs, which in itself can produce issues for CrossfireX and SLI setups, although compute does not rely on those technologies and should suffer no issues. Interestingly GIGABYTE does not supply any extra power connectors for these PCIe slots, indicating that not each port might not be able to provide 75W when all are populated (e.g. seven graphics cards).

In terms of storage, the system has eight SAS 6 Gbps ports in white on the right hand side, rather than the mini-SAS connectors we saw on other C60x motherboards. For SATA storage there are two SATA 6 Gbps ports in red and four SATA 3 Gbps in black, each at the bottom of the board. The motherboard also has two USB 3.0 type A ports sticking out of the motherboard – this feature is useful for users with software that requires USB dongle verification.

We tested the motherboard in both dual E5-2697 v2 (2x 12C/24T, 2x 130W) and dual E5-2687W v2 (2x 8C/16T, 2x 150W), showing where more cores of the former and the higher base frequency of the latter pay off in our CPU benchmarking suite. This especially shows through in our rendering benchmarks, although not so much in the x264 video conversion. In these configurations, idle power consumption with a single GTX 770 installed gave 115-130W at idle and 300W-320W with 100% CPU load. The Realtek ALC892 codec is more a token gesture for basic audio; other server motherboards tend to eschew audio, but as this is more a workstation outfit then basic audio should be sufficient with dedicated audio professionals using their own PCIe sound cards. On the audio note, the DPC Latency was clocked in as 300+ microseconds, similar to other server products, but ultimately quite high compared to mainstream motherboards.

Server motherboards are designed to be left on for extended periods of time, and thus not a lot of effort is made in POST times – this is indicated by the SAS and two Intel NICs having their own OPROM POST sequences, making the total power on to Windows loading screen over 60 seconds. This reduces to 38 seconds when the SAS and NICs are disabled.

Our gaming benchmarks were limited by the motherboard not displaying an image when more than one GPU was used, but the single GPU benchmarks all showed a regression in performance against an i7-4770K setup. One could suggest that this is not a platform for gaming, although users who want to have this system in a dual work/home role should have no problems matching mainstream system performance.

Motherboards in this bracket are all about the feature set, and the GA-7PESH3 oozes capability, especially when pairing two 12 or 15 core CPUs with a four-way co-processor setup matched with a SAS RAID. The only downside is the cost to get there. Server motherboards are often limited in BIOS and Software, and the 7PESH3 does not move things into a more useful role there, indicating a potential gap in the server board market.

Visual Inspection

For those that are more used to the consumer sized motherboards, the SSI CEB standard is both longer and uses different fixing points to the case, so users who invest should ensure that their case supports the standard as most do not. One of the poignant features the GIGABYTE Server team pointed out to me was the socket implementation which on this board has the sockets side by side and level, compared to others (such as the 7PESH1 we reviewed) which are often offset due to PCB layout.

Because of this orientation, equipping each CPU with eight DRAM slots is a little unwieldy unless concessions are made, so GIGABYTE went with one DIMM per channel for each CPU. While this limits the system to eight slots, this does afford 64GB of UDIMMs or 256GB of quad-rank RDIMMs both at 1866 MHz rather than the 1600 MHz standard. Given how Xeon server and workstations are often memory speed limited, this might be a good positive step.

Each CPU needs its own 8-pin CPU power, and because there is no overclocking on this platform each of the power delivery segments is built to server specifications: several high-rated power phases built for 24/7 operation.

While there are six fan headers on board, the placement does seem to detract from any liquid cooler setups by providing each of the sockets only one header to play with. The other four headers are located in pairs – one pair above the SAS ports and the other pair below the PCIe sockets.

For storage the motherboard uses SAS ports directly on the PCB rather than through a mini-SAS interface. These eight ports are in white, with the SATA ports in red and black at the bottom of the board. The red ports are SATA 6 Gbps while the black ports are SATA 3 Gbps. The heatsinks above these ports are for the chipset and a controller. Next to the two black SATA ports at the bottom are two USB 3.0 type A ports (powered by an Etron controller) directly in the motherboard. These ports allow for permanent USB storage to be attached while in a case, or as is often the case with workstation builds, suitable for USB verification dongles for professional software that is sold on a per-system license. My experience is that these dongles are being phased out due to the prevalence of a central license server or the web, but this is a feature that makes it on to some consumer motherboards.

The GA-7PESH3 has a maximum complement of full-length PCIe slots, although the naming configuration is odd. The bottom one is labeled PCIE_1 and the top is PCIE_7, this is because PCIE_1/3/5/7 are all x16 (two from each CPU) with PCIE_2/4/6 being x8. When something is installed into an x8 slot, the slot underneath reduces down to x8 mode, ending up with two possible extremes:

Four PCIe devices: x16/-/x16/-/x16/-/x16
Seven PCIe devices: x16/x8/x8/x8/x8/x8/x8

For users with single slot RAID cards or single slot compute GPUs, such as AMD’s FirePro V7900 or S7000, each element of bandwidth is more than welcome. However for users interested in SLI or Crossfire, the card that gets the default video output is often the one at the bottom. It should be noted that there is no additional power delivery on the motherboard for the PCIe slots. This is a little worrying - should a user put in several devices that consume ~70W each and not require extra power plugs, the motherboard might have a hard time delivering 7x70W and cause the 24-pin ATX connector to burn out. This comes across as a design flaw, perhaps suggesting that while the system should be geared towards 4-way GPUs or 7-way Compute, only devices that have external power should be used. Even then, this might cause issues for any combination of cards requiring >200-250W. from the 24-pin power connector.

Like other motherboards in this segment, the 7PESH3 uses an Aspeed IC to provide system management tools. This allows the user to log in to the system via a dedicated network port, turn the machine on, and access the graphical output. The Aspeed also provides the VGA output on the back if the system is under a desk but filled with compute performance. The base design of an AST2300 is a 400 MHz ARM9 core, which GIGABYTE has paired with a small amount of onboard memory.

The rear panel on the motherboard contains a PS/2 combination port, four USB 2.0 ports, four USB 3.0 ports (from an Etron controller), a VGA port, a COM port, two Intel 82574L NICs, a management NIC from the Aspeed IC and audio jacks from the Realtek ALC892 audio codec.

Board Features

GIGABYTE GA-7PESH3
Price US (Newegg)
Size SSI EEB
CPU Interface 2x LGA2011-0
Chipset Intel C602
Memory Slots Eight DDR3 DIMM slots
Supporting up to 64 GB UDIMM / 256 GB RDIMM
Up to Quad Channel, 1066-1866 MHz, ECC and non-ECC
Video Outputs VGA via Aspeed 2300
Onboard LAN 2x Intel 82574L
1 x Aspeed Management NIC
Onboard Audio Realtek ALC892
Expansion Slots 7 x PCIe 3.0 x16
- x16/-/x16/-/x16/-/x16
- x16/x8/x8/x8/x8/x8/x8
Onboard SATA/RAID 2 x SATA 6 Gbps
4 x SATA 3 Gbps
8 x SAS 6 Gbps
SATA RAID 0/1/10/5
SAS RAID via LSI SAS2008, RAID 0/1/1E/10
USB 3.0 4 x USB 3.0 on Rear Panel
2 x USB 3.0 Type-A on Motherboard
1 x USB 3.0 Header
Onboard 2 x SATA 6 Gbps
4 x SATA 3 Gbps
8 x SAS 6 Gbps
6 x Fan Headers
1 x SATA SGPIO Header
1 x LSI RAID Key Header
2 x USB 3.0 Type-A
1 x USB 3.0 Header
1 x USB 2.0 Header
1 x HDD Back Plane Board Header
1 x BIOS Recovert Jumper
Front Audio Header
Front Panel Server Header
Power Connectors 1 x 24-pin ATX
2 x 8-pin CPU
Fan Headers 2 x CPU (4-pin)
4 x SYS (4-pin)
IO Panel 4 x USB 2.0
4 x USB 3.0
2 x Intel NIC
1 x PS/2 Combination Port
1 x VGA
1 x Serial
1 x Management NIC
Audio Jacks
ID LED
Warranty Period 3 Years
Product Page Link

In The Box

Workstation and server motherboards are in that weird nexus where system admins might not want a lot of extras. Their systems are planned out and all the separate items are ordered separately. Now that the GIGABYTE Server division is selling to end-users, the packages have to represent what a home/prosumer might need in with the box. As a result, we get the following:

Driver DVD (Manual on Disk)
Quick Reference Guide
Rear IO Shield
3.5-inch USB 3.0 Front Panel with 2.5” bracket
Six SAS/SATA Cables
USB 2.0 Rear Bracket
2/3/4 way SLI Connectors
Three Crossfire Connectors

This is a nice amount of kit to get with a motherboard, especially with a server/workstation based product. One could levy an argument that for non-multi GPU implementations, the price could be reduced by $25 by removing all the additional SLI/Crossfire Connectors, but items like the USB 3.0 front panel are well received.

BIOS and Software
POST A COMMENT

35 Comments

View All Comments

  • ShieTar - Friday, September 5, 2014 - link

    Well, there is a tiny group of people who will be interested: They need a Workstation because they independently work with their PC to create content, but if possible they would also like to use the existing setup to game occasionally, instead of spend another 1k$ and put another Case next to their desk.

    Also, 10 years ago a lot of Trolls were posting "Yeah but can it run Quake" in the comment section of server hardware reviews. This kind of testing shuts them up.
    Reply
  • xxsk8er101xx - Saturday, September 6, 2014 - link

    Because the motherboard is designed for the prosumer market. Not the professional/server market. The point of the article is to basically say that this motherboard servers no purpose. It can't be used for the professional/server market because there are too few RAM slots and no mini-sas slots which the majority of the professional market has a need for. Reply
  • Kevin G - Sunday, September 7, 2014 - link

    I think this motherboard has priced itself out of the prosumer market a bit at $650 USD and then you have Xeon prices on top of that. I do see this system being a decent pick for some professional tasks though. RAM capacity is still pretty decent as this board accepts 32 GB DIMMs for 256 GB capacity. That's a bit expensive due to 32 GB DIMM pricing but I see 128 GB via 16 GB DIMMs as reasonably priced for what you get. Needing more than 128 GB is a very small niche and one that overlaps widely with the idea of just using servers to batch process the large data sets anyway.

    The PCIe setup does scream professional though. Components like capture cards and PCIe SSDs are not going to be restricted in performance. (Though it is worrisome that this board doesn't have a dedicated aux power for just the PCIe slots.) The presence of onboard audio also indicates that this certainly isn't a server board.

    I have yet to encounter a good reason for SAS in the professional space. 6 Gbit SATA provides enough bandwidth and SSD's can saturate that. SATA hard drives are adequate for bulk storage locally. (There is definitely a niche for SAS in server space with multi path IO for redundancy.) I can see a need for RAID5/6 in the prosumer and professional space but SAS isn't a requirement for it (though incidentally most RAID5/6 cards are SAS based). Even then, in most cases software RAID1 is 'good enough' for redundancy to prevent downtime in the professional space.
    Reply
  • AnnihilatorX - Sunday, September 7, 2014 - link

    Kindly read the article!: "While gaming is not a focus of motherboards like the GIGABYTE GA-7PESH3, the system may be in use by content developers relying on an accurate representation with what they are making. " Reply
  • gchernis - Thursday, September 4, 2014 - link

    Nice write-up! Do you have any data on total CPU utilization with multithreaded tasks? What about kernel-time vs. user mode time split? Reply
  • JellyRoll - Thursday, September 4, 2014 - link

    "Server motherboards historically have done rather poor in this test,"....perhaps because it is a server motherboard? are you testing a server workload on it? Reply
  • Samus - Thursday, September 4, 2014 - link

    Anand used to do SQL benchmarks. This is what we need for this review to be complete. Reply
  • mavere - Thursday, September 4, 2014 - link

    The Handbrake results are a bit disappointing because data-dependency breakpoints occur both naturally (at scene boundaries) and manually (at regular ~10s key-frame intervals). Conceptually, the transcoder can easily reuse the source's chapter-timing metadata and keyframes to outsource chunks of work to each socket with minimal communication overhead.

    I'm guessing that NUMA performance was never a pressing concern for any of Handbrake's and x264's contributors.
    Reply
  • JDG1980 - Thursday, September 4, 2014 - link

    I just can't see a good reason to buy into Ivy Bridge at this time, when Haswell-EP is just around the corner. Personally, I'm seriously considering the E5-1650 v3, assuming it comes in at a similar price point to its predecessor (and assuming that there are some decent workstation boards with ECC support available).

    And it's especially hard for AnandTech to justify the time reviewing this soon-to-be-obsolescent board when they still haven't gotten around to Tonga. I know it's not the most groundbreaking new GPU design (and AMD picked about the worst place in their lineup they possibly could to slot it), but still, it's a new GPU and we haven't seen a review despite every other major site doing so 2 days ago. What's going on with this?
    Reply
  • ruthan - Thursday, September 4, 2014 - link

    I would buy such machine for gaming if there would be performance benefit, so its good to know and mainstream everytime rules.. Its like SPARC or Power architecture, they cant compete because of mainstream x86 market is huge so, research revenue there are also huge, users requirements are bigger, more universal. This Sparc, Ithanium or Power market exist only because of stupid company policies, like everything from one company, support only for this HW, lies about super duper optimalisation for such HW, and especially license per CPU socket (here realy doesnt matter about price of CPU because, SW licence per socket is astronomical, so you are forced to buy Sparc or Power machine..). Reply

Log in

Don't have an account? Sign up now