Go Back   OC3D Forums > [OC3D] Project Logs & Gallery > Project Logs
Reply
 
Thread Tools Display Modes
 
  #1  
Old 26-07-20, 08:14 PM
looz's Avatar
looz looz is offline
OC3D Elite
 
Join Date: Feb 2013
Location: Finland
Posts: 1,820
Project spinnamathing

Sadly got no photos to share for this starter post, but making one so I won't skip documenting my project.

I've had a bit of a storage problem lately and finally pulled the trigger on shared storage.

I've got some hardware laying around from older PCs, so going to take advantage of them and ordered some filler parts to get a whole NAS.

Currently I've got the following:
  • Random laptop HDDs (probably not using)
  • 4Tb hard drive (probably using, but has some data that needs backing up first)
  • Samsung EVO 970 500GB
  • Z370-F Gaming
  • Antec 300 "flavour town edition" (more on that later, lol)

Ordered, hopefully arriving for next weekend:
  • 3* 4Tb Toshiba P300 drives (Not optimal because SMR, but I'll make do - were cheap)
  • 3* 500GB MX500 (50€ a pop on Amazon, maximum order)
  • Pentium G5420 (Cheapest non-Celeron compatible CPU - also ECC compatible if I come to my senses later)
  • 2*16Gb Non-ECC 2666MHz memory
  • Eaton 5S 550i UPS
  • Cooler Master MWE White V2 400W

Plan:
FreeNAS as OS, with two zpools. 3*500GB SSD in RAID-Z1 for flash storage. HDDs in "RAID10" arrangement for mass storage, aka. two VDEVs, each with two drives which are mirrored forming the second zpool.

OS drive will be the Samsung nVME, which will also host VMs and jails, whichever is feasible for the software intended. Lancache requires linux so that will run inside a VM and also use the SSD pool.

Since the setup will be used over 1Gb NIC at least initially, no need for fast cache drive. 32Gb ram should make it snappy regardless for everyday use. And yeah, silent corruption is a possibility with non-corrected memory, but whatever. At least it will be on top of UPS which can initiate a controlled shutdown in case of power outage - so complete data loss should be relatively unlikely.

This is a subject to change, I might go LVM on top of linux software raid, but let's see how it pans out. :P

__________________
i7 8700k - 16GB - 2060 FE - 660p 1TB + MX500 2TB - HE-4XX w/ Topping D30+A30
Reply With Quote
  #2  
Old 27-07-20, 04:26 PM
hmmblah's Avatar
hmmblah hmmblah is offline
Moderator
 
Join Date: Jul 2010
Posts: 4,651
I thought ZFS didn't play well with SMR?
__________________
CaseLabs Mercury S3 | AMD 3900X | ROG CH8 Impact | RTX 2080 Super | 32GB DDR4 | 1TB NVMe
Acer Predator Z35P


Reply With Quote
  #3  
Old 27-07-20, 06:16 PM
looz's Avatar
looz looz is offline
OC3D Elite
 
Join Date: Feb 2013
Location: Finland
Posts: 1,820
Quote:
Originally Posted by hmmblah View Post
I thought ZFS didn't play well with SMR?
Any RAID mechanism is slow when the cache is exhausted, which happens during scrubbing and resilvering. However, the worst offender is WD Red, which sometimes just shuts down and exits cluster during resilvering due to firmware bug.
__________________
i7 8700k - 16GB - 2060 FE - 660p 1TB + MX500 2TB - HE-4XX w/ Topping D30+A30
Reply With Quote
  #4  
Old 28-07-20, 07:32 AM
Avet's Avatar
Avet Avet is offline
OC3D Elite
 
Join Date: Dec 2016
Posts: 1,636
I haven't read recently about zfs and smr, but from what i know it is a straight up no no.
__________________
Turbos on an engine is God's way of telling you it ain't got enough cylinders.
Reply With Quote
  #5  
Old 28-07-20, 10:04 AM
looz's Avatar
looz looz is offline
OC3D Elite
 
Join Date: Feb 2013
Location: Finland
Posts: 1,820
Well, here's the word from freenas devs themselves: https://www.ixsystems.com/blog/libra...lity-with-zfs/

They'll likely have pretty disappointing performance, especially with random writes and resilvering, but my data should be safe. WD implementation had an additional incompatibility.

In the case of a drive failure, I'll replace them with SMR, but for now I won't cancel the order.
__________________
i7 8700k - 16GB - 2060 FE - 660p 1TB + MX500 2TB - HE-4XX w/ Topping D30+A30
Reply With Quote
  #6  
Old 28-07-20, 02:01 PM
looz's Avatar
looz looz is offline
OC3D Elite
 
Join Date: Feb 2013
Location: Finland
Posts: 1,820
Weighed my options again and decided that going for four new CMR disks is better for peace of mind, so upgraded to 4x Toshiba X300. Going to consume more power but should be much more performant.
__________________
i7 8700k - 16GB - 2060 FE - 660p 1TB + MX500 2TB - HE-4XX w/ Topping D30+A30
Reply With Quote
  #7  
Old 01-08-20, 01:05 PM
looz's Avatar
looz looz is offline
OC3D Elite
 
Join Date: Feb 2013
Location: Finland
Posts: 1,820
Right, got some goodies by mail today, so took the first steps.


This isn't going to be a treat on your eyes like most build logs here, you've been warned. All pictures can be enlarged by clicking.


Motherboard with temporary single 8GB stick of RAM installed, along with nVME which is under the heatsink.


Set up in Antec 300, note the elegantly covered PCI-E-slots.


As a complete surprise, I still had the manual (and IO shield!). Never really enjoyed setting up front panel connectors.


Case supports 120mm but I had these Arctic 140mm fans which I quite fancy. Some static pressure for the upcoming 7200RPM drives.


Took the dremel out to fit them. Lower lip was in the way.


As a surprise to no-one, it looks like complete wank.


My cover job doesn't improve it by much.


Stuck the utter beast of a CPU in.


I noticed the mounting pressure was a bit... disappointing. I guess the history of hefty air coolers has done its number on the motherboard.


Replaced the old antec fan with one of those horrendous EVGA fans.


All wired up, just missing the drives. Note the way CPU cable is routed.

I also never read up on the cabling the PSU comes with, but in a stroke of good luck, it had six SATA power connectors - exactly what I need!


It posts! Note to self, integrated DVI doesn't like being adapted to DisplayPort - was a bit worried since I got no signal.


In my infinite wisdom I never reset this UEFI. So it booted with my 8700k / b-die config.

Anyhow, freeNAS installation went swimmingly, continuing when more goodies arrive. Currently it's a NAS with no shares whatsoever. And despite the modern curved motherboard, CPU sits at 30c. Good enough.
__________________
i7 8700k - 16GB - 2060 FE - 660p 1TB + MX500 2TB - HE-4XX w/ Topping D30+A30
Reply With Quote
  #8  
Old 05-08-20, 12:40 AM
looz's Avatar
looz looz is offline
OC3D Elite
 
Join Date: Feb 2013
Location: Finland
Posts: 1,820
More goodies are in, so building the rag-tag NAS continues.

SSDs and Memory - note that there's only two SSDs, need to save four remaining SATAs for hard drives. Maths is difficult!



Close-up of memory


The low standards set earlier must be strictly abided to.


Created a striped pool, essentially RAID0. These won't be used for long term storage for anything important, so losing all data in case of a failure won't be end of the world. Encryption enabled since the CPU supports it natively.


UPS works like a charm too, here's log output of me pulling the power cord, simulating an outage.


And here's monitoring data for the UPS for that period. 30 seconds of UPS = 5% power. Though with four 7200RPM disks the power consumption should go up noticeably.


CrystalDiskMark from my Windows PC. As expected, 1Gbps NIC is rather limiting. Random performance isn't too good either, I suppose there's a fair bit of overhead due to a network stack in between. Still, gaming over it worked without hitches and lancache should work pretty well, too.


Copying a game over pegged the connection without issues. With SSDs I wouldn't take anything less.


Top output when copying the game over - it's apparent that SMB has a fair bit of CPU overhead. Lancache will be mounted through NFS, so there should be less of a penalty.
__________________
i7 8700k - 16GB - 2060 FE - 660p 1TB + MX500 2TB - HE-4XX w/ Topping D30+A30
Reply With Quote
  #9  
Old 06-08-20, 02:56 PM
looz's Avatar
looz looz is offline
OC3D Elite
 
Join Date: Feb 2013
Location: Finland
Posts: 1,820

Last piece of the puzzle arrived!


Close-up of the drive.


Finished system from inside - with the limited cable management options it's... okay I guess.


And this is why I mentioned the case being "flavour town edition".


Story behind it is that my mate needed a temporary rig for a LAN party - so I build him this.


So now they're configured in "RAID 10" and after all overhead and whatnot I'm looking at the following network stores:

It saturates my 1GBps link without issues as well - as expected. Now I can just hide that monstrosity to my broom closet and let it run there forever.


And the temperatures turned out to be a complete non-issue - thanks to the static pressure fans, all the drives are sitting at 34 degrees Celsius.
__________________
i7 8700k - 16GB - 2060 FE - 660p 1TB + MX500 2TB - HE-4XX w/ Topping D30+A30
Reply With Quote
  #10  
Old 06-08-20, 03:10 PM
tinytomlogan's Avatar
tinytomlogan tinytomlogan is offline
The Guvnor
 
Join Date: Dec 2007
Location: TTL Towers
Posts: 21,079
Home server is always a cracking idea for a project with spare parts - you can do so much with them once you have them too!

Get a plex server running on it
__________________
If you recommend a product LINK THE REVIEW


Get along or run along. Your choice
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump










All times are GMT. The time now is 10:19 AM.
Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2020, vBulletin Solutions, Inc.