[Leica] about NAS devices

[spencer at aotera.org (Spencer Cheng)]

Very good suggestions, Brian. 

I?ll add a few more. This works for me. YMMV.

1/ Don?t buy cheap hard drives. I?ve had too many Western Digital and 
Seagate consumer HDDs fail on me. Seagate, in particular, went through a 
period of very high failure rates a few years ago. I no longer trust them. 
For NAS type of use, I now by WD Ultrastar HDDs (which use to be Hitachi 
Ultrastar and before that IBM Fireball) in 14TB or bigger in size when the 
price ($/TB) is reasonable. 

2/ I no longer use RAID of any sort, not even RAID 1 (1:1 replication) 
because RAID disk rebuild when one disk is replaced has too high probability 
of failure for disks bigger than about 750GB. My version of RAID 1 is 
basically a 14TB Ultrastar HDD in a tiny backup ?PC? connected to my NAS 
which has various HDDs and SSDs in it. Like Brian, important data is 
replicated to the backup ?PC?. 

3/ I use TrueNAS to run my NAS but I am a geek and I like to tinker a bit. I 
like ZFS rather than whatever file system commercial NAS uses, to achieve 
better data integrity. One thing TrueNAS/ZFS supports is is both internal 
and remote replication. I use both. I need to take my backup ?PC? to my 
friend?s place and do all the firewall and remote configuration.

4/ ZFS allows me to encrypt the data on the NAS and the remote ?backup? PC 
and still do remote replication. Not perfect since ZFS needs a bit of 
metadata at the remote end to enable remote replication but no decryption or 
encryption keys are stored on the remote backup.

5/ Never *EVER* buy 3.5? hard drives over the net and have it delivered if 
you care about reliability. This is the one thing I will always buy locally. 
The drive mechanisms in modern HDDs are delicate things. You don?t know how 
many times it got chucked around by the courier company(ies). I always go to 
a local shop and pick up in stock HDDs. Not a guarantee but much more likely 
they would drop a sled of HDDs delivering it to the store. 

6/ Corollary to #5 is if you receive a HDD by mail from warranty claim, 
don?t put anything you care on it. Firstly, the warranty drives are 
typically returned drives which were re-tested and deemed acceptable by the 
manufacturer. Secondly, it?s shipped by courier to you which means it?s 
suspect even if it was perfectly fine leaving the manufacturer.

Regards,
Spencer

> On Oct 25, 2022, at 15:46, Brian Reid <reid at mejac.carlsbad.ca.us> wrote:
> 
> The term "NAS" stands for "Network Attached Storage". But NAS boxes are 
> being used in other ways. Here's some background and explanation.
> 
> NAS boxes provide 3 core functions:
> 
> 1. Aggregation. You can have a bunch of drives but talk to them as if they 
> were just one drive. This way you don't have to remember which drive you 
> put something on. A NAS is seen by your computer as just one drive, 
> regardless of how many drives are actually in it.
> 
> 2. Access. This is the "NA" part of NAS, and is why they were invented. If 
> you have a NAS box and it is connected to an ethernet, then more than one 
> computer can access it without you having to move any wires.
> 
> 3. Safety. NAS boxes can (if you want) combine multiple drives in such a 
> way that if one fails, you haven't lost any data, and theoretically you 
> can replace that one failed drive and keep going. Doing this gives you 
> less overall storage (you have to store multiple copies of data in order 
> to make this work) but more reliability.
> 
> There are dozens of other non-core functions that add stress, complexity, 
> and marketing advantage to a NAS box. You don't have to use any of them, 
> but some are awfully convenient.
> 
> If two computers are trying to access the same storage, they need to take 
> special care to make sure they don't trip over each other. If Computer A 
> and Computer B are trying to update the same storage at the same time, it 
> will make a mess. To prevent this from happening, software programmers use 
> a technique called "file locking". Unfortunately, standard file locking 
> techniques don't work with ethernet, so many applications (such as 
> Lightroom) just say "sorry, you can't do that". Lightroom won't let you 
> access a catalog across an ethernet because it doesn't want to be 
> responsible for getting the locking done right.
> 
> If you use a NAS box but access it by a USB cable, then you are not using 
> it via a network and therefore are not subject to network file locking 
> problems. You get functions 1 and 3.
> 
> I use NAS boxes extensively, but I am a sufficiently jaded computer 
> systems person that I will never trust them to do simultaneous shared 
> access because of the inherent complexity of those file locking protocols.
> 
> But this is just me and my family. When I managed the IT department for a 
> big company, I couldn't make "no sharing" rules, so instead I had to make 
> and enforce rules for sharing.  I am long gone from the big companies 
> whose IT departments I managed, but my rules remain pretty much as I wrote 
> them, which is:
> 
> * Never edit on a NAS. Make a local copy and edit that, and put the edited 
> copy back when you are done. Even though a NAS can be used as a virtual 
> drive to edit in place, don't do it.
> 
> * Devise, use, and enforce a system to coordinate your updates with other 
> people. I always use a "check-out/check-in" model. To edit a file, check 
> it out of the library. When you are done editing and have uploaded the 
> revised version, check it back in. The check-out process must prohibit the 
> checking out of a file that someone else has checked out.
> 
> My personal process to manage my files (including photographs) is to keep 
> the master copy of everything on a NAS at my house, to have that NAS 
> automatically copied every night to storage that is not in my house, and 
> to do check-out/check-in of files from the home NAS to my computers. The 
> reason I use check-out/check-in even though it's just me doing it is that 
> I have more than one computer. If I start an edit on my laptop, and then 
> try to edit from my iMac, it will not let me do it.
> 
> Because I am paranoid and suspicious, I actually have 2 NAS devices from 
> different manufacturers at my house, and replicate the master copy to the 
> mirror copy by an automatic means. (Master is ReadyNAS, mirror is TrueNAS).
> 
> I also run Time Machine service on my master NAS device. Time Machine data 
> is part of what is copied every night to offsite storage.
> 
> 
> 
> _______________________________________________
> Leica Users Group.
> See http://leica-users.org/mailman/listinfo/lug for more information