A dual processor system sounds awesome to the home user but in reality it is almost entirely a professional market. The prosumer has to use Xeons at JEDEC memory speeds and then ensure that the software is NUMA aware, especially if it decides searching for data in the other processor's L3 cache. However now GIGABYTE Server is selling to the prosumer via Newegg, and they sent us the $640 GA-7PESH3 for review.

For most users, a dual processor system affords several issues, aside from the cost. Performance with a 2P system is very dependent on the software in use. With a single processor system, each core can ‘snoop’ into the other core cache in order to see how data is updated. In a 2P system, the latency of talking between the two CPUs is at least an order of magnitude higher. This means that memory accesses can be delayed causing branched and locked code to be slow. In order to get around this, the software has to be NUMA aware, and the majority of regular applications are not.

There is no overclocking with a 2P system, and single thread speeds can be lower. As a result, gaming often sees a hit in performance, as well as basic tasks. The optimal use case scenario, for most software that is not aware of dual processor architecture, is any workload that needs as few memory accesses as possible and is ‘embarrassingly parallel’ such as ray tracing, video editing, virtualization or particular types of scientific compute. 2P motherboards, particularly those built by server teams, also often come with system management tools not seen in the consumer space, allowing users to access their system as it processes data and monitor progress as well.

GIGABYTE GA-7PESH3 Overview

This motherboard is built with storage and GPUs/coprocessors in mind. Although each CPU socket has four DRAM slots (rather than a more usual complement of eight each), the PCIe is a full array of PCIe 3.0 slots offering x16/-/x16/-/x16/-/x16 or x16/x8/x8/x8/x8/x8/x8 whether the system is using four PCIe devices or seven. This means that the PCIe lane allocation comes from both CPUs, which in itself can produce issues for CrossfireX and SLI setups, although compute does not rely on those technologies and should suffer no issues. Interestingly GIGABYTE does not supply any extra power connectors for these PCIe slots, indicating that not each port might not be able to provide 75W when all are populated (e.g. seven graphics cards).

In terms of storage, the system has eight SAS 6 Gbps ports in white on the right hand side, rather than the mini-SAS connectors we saw on other C60x motherboards. For SATA storage there are two SATA 6 Gbps ports in red and four SATA 3 Gbps in black, each at the bottom of the board. The motherboard also has two USB 3.0 type A ports sticking out of the motherboard – this feature is useful for users with software that requires USB dongle verification.

We tested the motherboard in both dual E5-2697 v2 (2x 12C/24T, 2x 130W) and dual E5-2687W v2 (2x 8C/16T, 2x 150W), showing where more cores of the former and the higher base frequency of the latter pay off in our CPU benchmarking suite. This especially shows through in our rendering benchmarks, although not so much in the x264 video conversion. In these configurations, idle power consumption with a single GTX 770 installed gave 115-130W at idle and 300W-320W with 100% CPU load. The Realtek ALC892 codec is more a token gesture for basic audio; other server motherboards tend to eschew audio, but as this is more a workstation outfit then basic audio should be sufficient with dedicated audio professionals using their own PCIe sound cards. On the audio note, the DPC Latency was clocked in as 300+ microseconds, similar to other server products, but ultimately quite high compared to mainstream motherboards.

Server motherboards are designed to be left on for extended periods of time, and thus not a lot of effort is made in POST times – this is indicated by the SAS and two Intel NICs having their own OPROM POST sequences, making the total power on to Windows loading screen over 60 seconds. This reduces to 38 seconds when the SAS and NICs are disabled.

Our gaming benchmarks were limited by the motherboard not displaying an image when more than one GPU was used, but the single GPU benchmarks all showed a regression in performance against an i7-4770K setup. One could suggest that this is not a platform for gaming, although users who want to have this system in a dual work/home role should have no problems matching mainstream system performance.

Motherboards in this bracket are all about the feature set, and the GA-7PESH3 oozes capability, especially when pairing two 12 or 15 core CPUs with a four-way co-processor setup matched with a SAS RAID. The only downside is the cost to get there. Server motherboards are often limited in BIOS and Software, and the 7PESH3 does not move things into a more useful role there, indicating a potential gap in the server board market.

Visual Inspection

For those that are more used to the consumer sized motherboards, the SSI CEB standard is both longer and uses different fixing points to the case, so users who invest should ensure that their case supports the standard as most do not. One of the poignant features the GIGABYTE Server team pointed out to me was the socket implementation which on this board has the sockets side by side and level, compared to others (such as the 7PESH1 we reviewed) which are often offset due to PCB layout.

Because of this orientation, equipping each CPU with eight DRAM slots is a little unwieldy unless concessions are made, so GIGABYTE went with one DIMM per channel for each CPU. While this limits the system to eight slots, this does afford 64GB of UDIMMs or 256GB of quad-rank RDIMMs both at 1866 MHz rather than the 1600 MHz standard. Given how Xeon server and workstations are often memory speed limited, this might be a good positive step.

Each CPU needs its own 8-pin CPU power, and because there is no overclocking on this platform each of the power delivery segments is built to server specifications: several high-rated power phases built for 24/7 operation.

While there are six fan headers on board, the placement does seem to detract from any liquid cooler setups by providing each of the sockets only one header to play with. The other four headers are located in pairs – one pair above the SAS ports and the other pair below the PCIe sockets.

For storage the motherboard uses SAS ports directly on the PCB rather than through a mini-SAS interface. These eight ports are in white, with the SATA ports in red and black at the bottom of the board. The red ports are SATA 6 Gbps while the black ports are SATA 3 Gbps. The heatsinks above these ports are for the chipset and a controller. Next to the two black SATA ports at the bottom are two USB 3.0 type A ports (powered by an Etron controller) directly in the motherboard. These ports allow for permanent USB storage to be attached while in a case, or as is often the case with workstation builds, suitable for USB verification dongles for professional software that is sold on a per-system license. My experience is that these dongles are being phased out due to the prevalence of a central license server or the web, but this is a feature that makes it on to some consumer motherboards.

The GA-7PESH3 has a maximum complement of full-length PCIe slots, although the naming configuration is odd. The bottom one is labeled PCIE_1 and the top is PCIE_7, this is because PCIE_1/3/5/7 are all x16 (two from each CPU) with PCIE_2/4/6 being x8. When something is installed into an x8 slot, the slot underneath reduces down to x8 mode, ending up with two possible extremes:

Four PCIe devices: x16/-/x16/-/x16/-/x16
Seven PCIe devices: x16/x8/x8/x8/x8/x8/x8

For users with single slot RAID cards or single slot compute GPUs, such as AMD’s FirePro V7900 or S7000, each element of bandwidth is more than welcome. However for users interested in SLI or Crossfire, the card that gets the default video output is often the one at the bottom. It should be noted that there is no additional power delivery on the motherboard for the PCIe slots. This is a little worrying - should a user put in several devices that consume ~70W each and not require extra power plugs, the motherboard might have a hard time delivering 7x70W and cause the 24-pin ATX connector to burn out. This comes across as a design flaw, perhaps suggesting that while the system should be geared towards 4-way GPUs or 7-way Compute, only devices that have external power should be used. Even then, this might cause issues for any combination of cards requiring >200-250W. from the 24-pin power connector.

Like other motherboards in this segment, the 7PESH3 uses an Aspeed IC to provide system management tools. This allows the user to log in to the system via a dedicated network port, turn the machine on, and access the graphical output. The Aspeed also provides the VGA output on the back if the system is under a desk but filled with compute performance. The base design of an AST2300 is a 400 MHz ARM9 core, which GIGABYTE has paired with a small amount of onboard memory.

The rear panel on the motherboard contains a PS/2 combination port, four USB 2.0 ports, four USB 3.0 ports (from an Etron controller), a VGA port, a COM port, two Intel 82574L NICs, a management NIC from the Aspeed IC and audio jacks from the Realtek ALC892 audio codec.

Board Features

GIGABYTE GA-7PESH3
Price US (Newegg)
Size SSI EEB
CPU Interface 2x LGA2011-0
Chipset Intel C602
Memory Slots Eight DDR3 DIMM slots
Supporting up to 64 GB UDIMM / 256 GB RDIMM
Up to Quad Channel, 1066-1866 MHz, ECC and non-ECC
Video Outputs VGA via Aspeed 2300
Onboard LAN 2x Intel 82574L
1 x Aspeed Management NIC
Onboard Audio Realtek ALC892
Expansion Slots 7 x PCIe 3.0 x16
- x16/-/x16/-/x16/-/x16
- x16/x8/x8/x8/x8/x8/x8
Onboard SATA/RAID 2 x SATA 6 Gbps
4 x SATA 3 Gbps
8 x SAS 6 Gbps
SATA RAID 0/1/10/5
SAS RAID via LSI SAS2008, RAID 0/1/1E/10
USB 3.0 4 x USB 3.0 on Rear Panel
2 x USB 3.0 Type-A on Motherboard
1 x USB 3.0 Header
Onboard 2 x SATA 6 Gbps
4 x SATA 3 Gbps
8 x SAS 6 Gbps
6 x Fan Headers
1 x SATA SGPIO Header
1 x LSI RAID Key Header
2 x USB 3.0 Type-A
1 x USB 3.0 Header
1 x USB 2.0 Header
1 x HDD Back Plane Board Header
1 x BIOS Recovert Jumper
Front Audio Header
Front Panel Server Header
Power Connectors 1 x 24-pin ATX
2 x 8-pin CPU
Fan Headers 2 x CPU (4-pin)
4 x SYS (4-pin)
IO Panel 4 x USB 2.0
4 x USB 3.0
2 x Intel NIC
1 x PS/2 Combination Port
1 x VGA
1 x Serial
1 x Management NIC
Audio Jacks
ID LED
Warranty Period 3 Years
Product Page Link

In The Box

Workstation and server motherboards are in that weird nexus where system admins might not want a lot of extras. Their systems are planned out and all the separate items are ordered separately. Now that the GIGABYTE Server division is selling to end-users, the packages have to represent what a home/prosumer might need in with the box. As a result, we get the following:

Driver DVD (Manual on Disk)
Quick Reference Guide
Rear IO Shield
3.5-inch USB 3.0 Front Panel with 2.5” bracket
Six SAS/SATA Cables
USB 2.0 Rear Bracket
2/3/4 way SLI Connectors
Three Crossfire Connectors

This is a nice amount of kit to get with a motherboard, especially with a server/workstation based product. One could levy an argument that for non-multi GPU implementations, the price could be reduced by $25 by removing all the additional SLI/Crossfire Connectors, but items like the USB 3.0 front panel are well received.

BIOS and Software
POST A COMMENT

35 Comments

View All Comments

  • tuxRoller - Thursday, September 4, 2014 - link

    So, you've got a workstation class board but are running Windows 8...
    Is it so difficult to throw centos on there for the benefit of those working at Pixar?
    Reply
  • MrSpadge - Friday, September 5, 2014 - link

    Those working at Pixar likely won't need an Ivy-EP review. Apart from that.. I'm sure Ian is glad not to have to deal with CentOS :p Reply
  • tuxRoller - Friday, September 5, 2014 - link

    Centos is easy to setup, and the new release is quite nice.
    My point with Pixar was that there are many serious creators who use Linux workstations, and not just for simulations. Unless you work in the area it's possible you may not know the prevalence of Linux among workstations, especially relative to windows 8, in my experience.
    That aside, seeing the comparative performance numbers of the two OS's is useful to get an idea of what the hardware itself can do.
    Reply
  • mpbrede - Friday, September 5, 2014 - link

    I really hope that this crappy grammar is just an indication of haste and not of lower editorial standards now that Anand has left the room.

    "As a result, gaming often sees a hit in performance, as well as basic tasks." -> ...result, gaming, as well as basic tasks, sees a hit...

    "Interestingly GIGABYTE does not supply any extra power connectors for these PCIe slots, indicating that not each port might not be able to provide 75W when all are populated (e.g. seven graphics cards)." -> read it slowly, how many "nots" should there be in a single sentence?
    Reply
  • aryonoco - Friday, September 5, 2014 - link

    A server/workstation motherboard requires its own benchmarks.

    For anyone who is in the target market of this product, this review is absolutely worthless. Gaming benchmarks? Really?!
    Reply
  • colonelclaw - Friday, September 5, 2014 - link

    Thanks for the review. In the future please consider including the following benchmarks:
    3DSMax + Vray, Maya + VRay/Arnold/Renderman, Adobe Media Encoder, Adobe After Effects, Adobe Premiere Pro. Some VRay RT and Octane multi-GPU benchmarks would be a nice addition, too.
    The one thing I find slightly strange about this motherboard is the low number of memory slots. All our DP motherboards have twice as many per processor (and many are full). I wonder why Gigabyte decided on just eight?
    Reply
  • eanazag - Friday, September 5, 2014 - link

    I think the point of making the board where the CPUs line up was to allow better airflow to the add-on cards (GPUs). Lining up the CPUs didn't allow for the normal # of mem. slots.

    In actuality I would think most buyers of this board would just go with one CPU. It allows for the video compute cards to have more juice from the ATX 24 pin. I have bought 2P boards for one proc because of the peripheral items like slots, networking, RAID, and built-in network management. When you look at non-server boards and add the pricing of add-on management it starts to make you consider the server/workstation board.
    Reply
  • The Von Matrices - Friday, September 5, 2014 - link

    The reason for this is because Gigabyte chose to use square ILM sockets.

    For those who do not know, LGA2011 has two sockets - square ILM and narrow ILM. Most consumer motherboards use the square ILM sockets; you only see the narrow ILM sockets in servers. The narrow ILM socket is obviously narrower, which allows more memory slots on the same size motherboard. The disadvantage of the narrow ILM socket is that the memory slots are so close to the processor that large and quiet tower heatsinks can't be used since they cover the closest memory slots.

    Basically, when designing a dual LGA2011 SSI CEB motherboard, you have to choose two of the three following features:

    16 DIMM slots
    7 PCIe slots
    Square ILM sockets

    If you use 16 DIMMs and 7 PCIe slots, you have to use narrow ILM sockets, which eliminates the use of tower heatsinks.

    If you use 7 PCIe slots and square ILM sockets (like Gigabyte did) you now can use tower heatsinks. However, then there is only enough space on the motherboard for 8 or 12 DIMM slots.

    If you use 16 DIMMs and square ILM sockets, there's only enough space on the motherboard for 6 PCIe slots.
    Reply
  • joannecdinkins - Friday, September 5, 2014 - link

    just as Larry answered I didnt even know that people able to get paid $6104 in a few weeks on the internet .
    go to this site>>>>> paygazette.ℭOM
    Reply
  • KAlmquist - Friday, September 5, 2014 - link

    One thing that's not addressed in this review is whether it makes sense to run Windows 7 on a multi-socket system. The benchmarks showing dual E5-2687W processors running significantly slower than a single E5-2687W suggest that Windows 7 is making bad scheduling decisions. Would Windows Server or Linux do better?

    Let me explain my point about scheduling decisions in a bit more detail. If you run a piece of software on a system with a single processor, the operating system has to make decisions about when to run each thread, and which core to run it on. Now switch to a system which is similar to the first system except that it has two processors. If the operating system simply ignores the second processor, and makes the same scheduling decisions as it made on the first system, then the performance of the two systems should be quite close. I'm not suggesting that this approach to scheduling on a dual processors system makes sense, but it is a possible approach. So if the approach to scheduling used by Windows 7 results in even worse performance, we know that Windows 7 could do better. In fairness to Microsoft, Windows 7 is not marketed as a server OS, so maybe it shouldn't be expected to perform well on server hardware.
    Reply

Log in

Don't have an account? Sign up now