FreeNAS Storage Server 2017 Part 2- The Build

Posted on January 17, 2017August 27, 2020 by Terry Sheridan

Happy New Year to all of you! And with a new year comes a new server. On New Year’s Eve I was doing some Hard drive shopping and found a seller on Amazon clearance out 2TB Seagate SAS2 Drives for $47 each… So I bought 8 for this build.

Because I bought 8 hard drives it started the hunt for a Server with at least 8 hotswap bays with 12 being preferable. Right off the bat HP was out because they are not my favorite servers. HP has some cool technology but, as they enforce more control over their products it’s difficult to find product upgrades or replacement parts. I will admit the HP iLO .net KVM feature is the most reliable option but the other head ache over parts, compatibility, and service contracts is not worth the headache for the homelab.

I was first looking at Dell R510 servers because of the options that came with that line supported 8 or 12 drive bays. Then I started looking into the storage controller, and all units I found didn’t include a storage controller that supported pass-through or JBOD. Then I started looking into compatible or flash able models and the costs were starting to add up for 6-8 year old hardware.

Then I went back to my system building days and found the perfect server on ebay. Intel Server System R2308GZ4GC.

This server is a 2U rackmount with the following specs:

Dual E5-2630Lv2 2.0 Ghz CPUs
32 GBs of ECC Registered RAM
Intel Remote Management Card
RMS25JB080 raid card

Plus, the Server has a “connector for I/O expansion” card slot for add-on cards to do infiniband or 10GbE. A “connector for I/O Expansion” is really just a mezzanine card slot. In the server itself, the mezzanine card slot is located under neither the PCI Express slots so it does not take away from traditional expansion needs.

When this particular generation of Intel Server Systems was released Intel provided a few options for storage controllers. On most server systems you can, add a dedicated RAID card (Intel or other vendor), use the built in Intel Embedded storage controller and corresponding plug in key for RAID types (SAS/SATA/RAID 5/RAID 6) or use an embedded add-on card. The Intel RAID cards and embedded add-on cards are Intel Rebranded LSI/Avago RAID Cards. So all the LSI drivers that are bundled into all Linux/VMWare/Windows/FreeBSD work with Intel RAID cards.

This Server came with the RMS25JB080 embedded RAID card and the most important part of this model is that the card supports pass-through. All the drives will be able to be individually controlled by FreeNAS but will have all the blinking lights, manageability, and hot swap capabilities from the RAID Card. Since this card is really just an LSI Storage Controller, LSI/Avago’s MegaCli utility works with it to retrieve storage information and MegaCli is also supported in FreeNAS.

Also, after the server came in and was tested successful I ordered the Intel 10GbE Dual Port I/O Module AXX10GBNIAIOM to put 10Gbe into the server via mezzanine slot. The one I purchased was pulled from a working server immediately from when the seller purchased it because the sellers in environment is 10 GbE copper and this card is SFP+.

1/11/2017 Update:

The first item to arrive were the hard drives, and Fedex decided that they wanted to delay this project. My wife works from home and sent me a picture of how Fedex handled the hard drives.

Needless to say, I didn’t have high hopes on all 8 drives working and after some testing on a spare server I had 3 out of 8 were DOA. I contacted the seller and they shipped out replacement drives.

1/13/2017 Update:

A few days later the Intel Server System came in and I was a little nervous about shipping because I saw that Fedex was handling that shipment as well. Luckily, that box was delivered without a scratch.

Upon inspection, I found a Q-logic QLE7370 Infiniband card installed in the server. I double checked the listing and saw that the card was pictured on the eBay listing, but was not listed in the component list. This model is a 40Gbps card. This unexpected find sparked a bunch of ideas and possibilities for creating a 40 Gbps storage network…. Then I double checked the hardware compatibility and saw that infiniband is only community supported and this model does not support IP over Infiniband (IPoIB). Also it looks like the main support for infiniband is community support in FreeNAS. Within the community support the main focus is for Mellanox inifiniband cards and there is next to no support for Q-logic. So I went back to my original 10 GbE idea, but if your interested here is the general path to run infiniband with FreeNAS. You have to manually install FreeBSD on the system and then build your own FreeNAS installation from source code with the required Infiniband and IPoIB drivers and software. Eitherway back to the 10 GbE iSCSI build.

After the rise and fall of the possibility of Inifiniband, I continued to inspect the server. The server itself was clean and everything powered on correctly. The only issue I found was the internal SAS cables that connect the storage controller to the Hot Swap bay were installed backwards. When looking at the hard drives in the RAID card versus the physical layout on the front of the chassis the order of the drives went 4, 5, 6, 7, 0, 1, 2, 3. After swapping the cables on the storage controller, the storage controller and the physical layout line up correctly.

After that, I installed the 2 SSDs I bought for Read Caching and the USB Drive I bought for the FreeNAS installation.

At this point, I was waiting on the replacement drives and the Dual Port 10 GbE card add-on card arrive. But with the current state of the system I started running firmware updates on the system, raid card and hot swap bay, and SSDs. The step by step process of updating Intel Server Hardware will be covered in Part 3 of the this series.

1/16/2016 Update:

The replacement drives came in and powered on successfully. I started building a RAID array on all 8 drives to put a strain on the drives to see if any more will fail before we deploy FreeNAS on the unit.

Update 1/17/2017

The Dual Port 10GbE mezzanine card arrived today! I installed the card into the server.

After the card was installed I turned the server back on. The Server still has Server 2012 R2 installed on it for Hard drive testing. I opened up Network Connection and boom! there is 2 10GbE adapters listed now.

Now that the 10GbE card is installed and there hasn’t been anymore hard drive failures. Let’s wipe and reload this beast with FreeNAS.

Vendors

I want to take a quick moment to thank the vendors where I got the major components of this build from. First, the part that kicked off the whole project 2 TB SAS hard drives. I purchased the hard drives through Amazon, from a seller named TechnoDealsUSA.com. I have never ordered from this vendor before this project. I am happy to say after seeing the damage that Fedex caused to the box full of hard drives, their customer service was very easy to work with to get replacement parts.

Next is the Server system. The Intel Server System came from an ebay Store called The Server Store. Some times when you purchase enterprise equipment from eBay, you can tell that whoever listed just unracked the equipment and then put it up on eBay. This is not the case with these guys. This server in question was very clean right off the bat. It was almost as if I had bought all brand new equipment an assembled it myself. The box the server came was not the original Intel Server box, but it was made to fit the system and had custom shaped foam molding to secure the server.

The last main vendor is FS.com. In a business environment if you want to have valid service contracts on all your equipment you have to buy all genuine accessories for your products. Well in a homelab environment that is a little difficult to do when most 10 GbE SFP+ transceivers are $300+. FS.com makes Fiber accessories for a ton of current and old products. I’ve ran them in my home lab before so there was no question that I’d go back to them for more fiber equipment.

Most of the little/more common accessories came from amazon.com. A Amazon Prime membership is worth it’s weight in gold and makes getting new items like SSDs and flash drives really easy.

Build Cost:

8 2TB Seagate SAS2 Drives $376

Intel Server System R2308GZ4GC Server $657

Intel 10GbE Dual Port I/O Module AXX10GBNIAIOM (eBay) $90.00

2 Samsung 850 EVO SSDs $196

Scandisk Cruzier Blade 16 GB flash drive $5.49

2 Intel E10GSFPSR Compatible 1000BASE-SX and 10GBASE-SR SFP+ Transceiver $32

Total Cost of the Storage Server: $1356.49

Other Costs:

2 Cisco SFP-10G-SR Compatible 10GBASE-SR SFP+ $32

6 Fiber Patch Cables $17.40

Total Cost of the Server and parts to connect it: $1405.89

The next parts the Series will cover everything from updating firmware to install/configure FreeNAS to bringing the new SAN into a VMWare environemt. Stay tuned!

-TDSheridan

Part 1- The Plan
Part 2- The Build
Part 3- Firmware Updates
Part 4- Install FreeNAS
Part 5- Configure FreeNAS
Part 6- VMWare