Once I decided on visualization and settled on Proxmox as the host platform for the new NAS, I stranded in decisions to make with regard to the installation.
The first problem was the installation medium itself: I wrote the installer to a uSD card, put it in an adapter to regular SD, that into SD2USB and finally in the USB3 port of the NAS.
Booting OK, installer menu OK, but starting the installer: not OK.
I tried a few times, but the image seemed broken. The kernel would halt somewhere between loading the initial ramdisk and actually starting the installer.
I checked the SHA256-checksum, reflashed on another memory card, used another SD-card-reader: all to no avail. In the end I started trying other USB ports; not just USB3 in front, but first USB3 on the back, and in the end USB2 on the backside.
That worked. The card readers are USB3, but maybe the uSD cards get upset at some point. Anyway, on to the installation.
Target harddisk: SSD or HDD? Which file system? Am I happy with automatic partitioning?
System disk: SSD. I want to be able to spin down as many HDD’s as soon as possible, as long as possible. If I start building the system on one of the spinning disks, it is more difficult to replace it in the future and am I stuck with it. Besides that, the base system should take only tens of GB’s, so there is enough space left for caching, swap and other things that also benefit from increased I/O.
Which file system? As long as I can remember, I have used LVM, the last ten years in combination with EXT4 on a md-stacks. It is very flexible and offers live resizing of nearly anything.
Proxmox does not do LVM out of the box, but it does offer ZFS (ZoL, ZFS on Linux). I have been postponing ZFS usage for a couple of years now, and am still hesitant to take the plunge. How many disks? How many pools? What size? Shit, non-ECC-RAM; and how about mirrors with differently sized disks? What about TRIM on the SSD?
I started reading about ZFS on a single device pool, retraced my steps and considered a pool with SSD and spinning disk mirrored. I found one viable option is to combine the 1TB spinning disk with the SSD for some reasons:
- It is the only non-SMR-disk I have available at the moment;
- It is similar in size to the 1TB SSD;
- I am not committed to keeping either the 4TB disk or the 6TB disk in the machine:
- I might take one or the other out and use it in a USB case for external back up
- I might swap either for a disk the size of the other, to create another mirrored set.
I hope that the pool will be smart enough not to spin up the HDD for read, that would nullify the benefit of the non-spinning SSD. If that is the case, I’ll investigate how to make a single device pool out of it.
The other option is running the pool on a single device with copies = 2; I’d use double the space, but there is no spinning disk involved.
Then there’s the question of partitioning. Just the root partition could do with some 10GB. I’d put at least /var on a separate partition, so that log files and temporary files don’t cause / to fill to 100%, some 3GB. ZFS copies = 2 means some 30GB. With swap equal to RAM is another 32GB, and then a couple of GB for ZFS ZIL. That leaves roughly 900GB of unpartitoned space.
It is all a bit theoretical up front; I hesitate to just take the defaults and see down the road. Those times I did that, I regretted it later on. Now in particular, with the quad-level flash disk and less than 400 rewrites per cell (360, according to Samsung), I prefer not to start on the wrong footing. Anyway, it can’t be helped so I just have to bite the bullet. All things considered, I take the SSD-only way for the initial installation.
Phew, that took just four nights to get through the first screen of the installer!
That was the single choice to make then; from there to installation complete and reboot took about five minutes.