Project spinnamathing

[looz]

Sadly got no photos to share for this starter post, but making one so I won't skip documenting my project.

I've had a bit of a storage problem lately and finally pulled the trigger on shared storage.

I've got some hardware laying around from older PCs, so going to take advantage of them and ordered some filler parts to get a whole NAS.

Currently I've got the following:

• Random laptop HDDs (probably not using)
• 4Tb hard drive (probably using, but has some data that needs backing up first)
• Samsung EVO 970 500GB
• Z370-F Gaming
• Antec 300 "flavour town edition" (more on that later, lol)

Ordered, hopefully arriving for next weekend:

• 3* 4Tb Toshiba P300 drives (Not optimal because SMR, but I'll make do - were cheap)
• 3* 500GB MX500 (50€ a pop on Amazon, maximum order)
• Pentium G5420 (Cheapest non-Celeron compatible CPU - also ECC compatible if I come to my senses later)
• 2*16Gb Non-ECC 2666MHz memory
• Eaton 5S 550i UPS
• Cooler Master MWE White V2 400W

Plan:
FreeNAS as OS, with two zpools. 3*500GB SSD in RAID-Z1 for flash storage. HDDs in "RAID10" arrangement for mass storage, aka. two VDEVs, each with two drives which are mirrored forming the second zpool.

OS drive will be the Samsung nVME, which will also host VMs and jails, whichever is feasible for the software intended. Lancache requires linux so that will run inside a VM and also use the SSD pool.

Since the setup will be used over 1Gb NIC at least initially, no need for fast cache drive. 32Gb ram should make it snappy regardless for everyday use. And yeah, silent corruption is a possibility with non-corrected memory, but whatever. At least it will be on top of UPS which can initiate a controlled shutdown in case of power outage - so complete data loss should be relatively unlikely.

This is a subject to change, I might go LVM on top of linux software raid, but let's see how it pans out.

 

[hmmblah]

I thought ZFS didn't play well with SMR?

 

[looz]

I thought ZFS didn't play well with SMR?

Any RAID mechanism is slow when the cache is exhausted, which happens during scrubbing and resilvering. However, the worst offender is WD Red, which sometimes just shuts down and exits cluster during resilvering due to firmware bug.

 

[Avet]

I haven't read recently about zfs and smr, but from what i know it is a straight up no no.

 

[looz]

Well, here's the word from freenas devs themselves: https://www.ixsystems.com/blog/library/wd-red-smr-drive-compatibility-with-zfs/

They'll likely have pretty disappointing performance, especially with random writes and resilvering, but my data should be safe. WD implementation had an additional incompatibility.

In the case of a drive failure, I'll replace them with SMR, but for now I won't cancel the order.

 

[looz]

Weighed my options again and decided that going for four new CMR disks is better for peace of mind, so upgraded to 4x Toshiba X300. Going to consume more power but should be much more performant.

 

[looz]

Right, got some goodies by mail today, so took the first steps.


This isn't going to be a treat on your eyes like most build logs here, you've been warned. All pictures can be enlarged by clicking.


Motherboard with temporary single 8GB stick of RAM installed, along with nVME which is under the heatsink.


Set up in Antec 300, note the elegantly covered PCI-E-slots.


As a complete surprise, I still had the manual (and IO shield!). Never really enjoyed setting up front panel connectors.


Case supports 120mm but I had these Arctic 140mm fans which I quite fancy. Some static pressure for the upcoming 7200RPM drives.


Took the dremel out to fit them. Lower lip was in the way.


As a surprise to no-one, it looks like complete wank.


My cover job doesn't improve it by much. :lol:


Stuck the utter beast of a CPU in.


I noticed the mounting pressure was a bit... disappointing. I guess the history of hefty air coolers has done its number on the motherboard.


Replaced the old antec fan with one of those horrendous EVGA fans.


All wired up, just missing the drives. Note the way CPU cable is routed. :tongue2:

I also never read up on the cabling the PSU comes with, but in a stroke of good luck, it had six SATA power connectors - exactly what I need!


It posts! Note to self, integrated DVI doesn't like being adapted to DisplayPort - was a bit worried since I got no signal.


In my infinite wisdom I never reset this UEFI. So it booted with my 8700k / b-die config. :lol:

Anyhow, freeNAS installation went swimmingly, continuing when more goodies arrive. Currently it's a NAS with no shares whatsoever. And despite the modern curved motherboard, CPU sits at 30c. Good enough.

 

[looz]

More goodies are in, so building the rag-tag NAS continues.

SSDs and Memory - note that there's only two SSDs, need to save four remaining SATAs for hard drives. Maths is difficult!



Close-up of memory


The low standards set earlier must be strictly abided to.


Created a striped pool, essentially RAID0. These won't be used for long term storage for anything important, so losing all data in case of a failure won't be end of the world. Encryption enabled since the CPU supports it natively.


UPS works like a charm too, here's log output of me pulling the power cord, simulating an outage.


And here's monitoring data for the UPS for that period. 30 seconds of UPS = 5% power. Though with four 7200RPM disks the power consumption should go up noticeably.


CrystalDiskMark from my Windows PC. As expected, 1Gbps NIC is rather limiting. Random performance isn't too good either, I suppose there's a fair bit of overhead due to a network stack in between. Still, gaming over it worked without hitches and lancache should work pretty well, too.


Copying a game over pegged the connection without issues. With SSDs I wouldn't take anything less.


Top output when copying the game over - it's apparent that SMB has a fair bit of CPU overhead. Lancache will be mounted through NFS, so there should be less of a penalty.

 

[looz]

Last piece of the puzzle arrived!


Close-up of the drive.


Finished system from inside - with the limited cable management options it's... okay I guess.


And this is why I mentioned the case being "flavour town edition".


Story behind it is that my mate needed a temporary rig for a LAN party - so I build him this.


So now they're configured in "RAID 10" and after all overhead and whatnot I'm looking at the following network stores:

It saturates my 1GBps link without issues as well - as expected. Now I can just hide that monstrosity to my broom closet and let it run there forever.


And the temperatures turned out to be a complete non-issue - thanks to the static pressure fans, all the drives are sitting at 34 degrees Celsius.

 

[tinytomlogan]

Home server is always a cracking idea for a project with spare parts - you can do so much with them once you have them too!

Get a plex server running on it

 

[looz]

I still haven't decided on whether or not do I want to run the apps on that server or get a separate application server. 2C/4T isn't the most blazing configuration, though for Plex the on-board GPU does support encoding.