My backup / NAS evolution

First I started syncing up some stuff to the cloud and other stuff to USB drives. But that did not protect from much other than drive failure.

So I added Carbonite which kept getting messed up causing me to start the upload to the cloud over from scratch. It was also uploading so slow that it never seemed to finish. At the time it did not support USB drives or network mounted drives. The only real plus was it linked straight into Windows Explorer making it easy to select and unselect what got uploaded to the cloud.

So I did a bit more research added Crashplan Home which let me back up stuff from several Windows and Linux servers to a 16 TB Linux box in the shop and the cloud. Technically I could back up everything to the cloud but only did a subset given the time involved. It did not care if the data was local or not and was a lot faster uploading to the cloud than Carbonite was.

At about this point I had 2 of the 4 4 TB drives on my main workstation with 8 TB drives and was having to move stuff around to make space so a RAID seemed to make sense. After a bit of research I decided on the Drobo 5n which supported putting all sizes of drives (which I had loads of laying about). Another selling point was it said it ran Crashplan native although this turned out to no longer be true when I bought it. I went with the n (NAS) version instead of the USB version thinking this would give me more options. Given that Crashplan supported backing up a network mapped drive this seemed to work fine for awhile.

Then the year of adventure started. 2017 saw a lot of services start cutting features, support and or jacking prices. While already dealing with site hosting changes Google Drive where I had about 1.5 TB of stuff uploaded (technically it was more like 3 to 4 TB but Google does not count most of it) decided when my free 1 TB offer ran out (which gave me 2 TB not the 1 TB I already had free for a year) they would not let me pay for it. My options were remove stuff down to, a counted, 1 TB of data or pay for 10 TB (at 10 times the price). So I started moving stuff to Amazon Drive. which was half the price of Google Drive for unlimited space. Downside was the Amazon Drive interface is REALLY bad. Plus sharing is limited and it does not seem to support versioning like Google Drive does. Odrive helps make it usable but has its own costs.

Then Amazon decided they were they dropping from unlimited to 1 TB for the $60/yr price. Then Crashplan announced it was dropping their Home plan meaning you had to start paying per computer and only supporting backing up network mapped drives if you paid by the TB. (This is where getting the Drobo 5n over the 5c, USB version came back to haunt me.) At a minimum this was going to increase my cost 5 fold. While updating my backup research I started having connection issues with the Linux server I was backing up to and the Drobo was running out of space.

Drobo fun starts

So I babied the Linux server Crashplan connection to the PC doing the Drobo backup for several days. (Remember it turned out Crashplan did not actually run on the 5n model.) I finally got the drive backed up to the Linux box so I had a full backup. Then I swapped out the smallest drive, a 2 TB, for an 8 TB and waited for the rebuild to finish. But it seems I should have checked the logs first because one of the 4 TB drives had dropped offline a couple times and if did it again during the rebuild which corrupted the whole Drobo. I should note here though not actually an issue the Drobo had been telling my I needed to update the software even though the software was already at the version it was telling me to upgrade to. Anyway now I'm panicking since a good bit of the data is is raw video from concerts that is only on the Drobo and the Linux backup and the Drobo is now toast AND the Linux box is getting iffy. So I ordered a Synology 12 bay NAS DiskStation DS2415+ (Diskless)  plus 4 8 TB drives.

Adding Synology (SHR vs SHR-2)

The main difference between SHR and SHR-2 is the SHR-2 allows 2 drives to die at the same time and still be able to recover versus only one with SHR. The plan is to run it in SHR-2 mode so I can use unmatched size drives and handle 2 drives failing at the same time. This way I can start with the 4 8TBs and then move over the 4 4 TB drives from the Linux server. From what I had read this seemed doable.

Turns out it is not.  While SHR lets you add anything,  SHR-2 only lets you add drives to a volume (RAID) larger than the smallest already in the volume. Plus adding a drive larger than the smallest is treated like the smallest drive in the volume. (Note according to what I'm reading if you add 2 of the larger drives it will start to use all of the drives again. The calculator seems to say you need 4.) So for instance if you have a 4 TB and 3 8 TBs you get the same space as 4 4 TBs. With SHR that gets you 20 TB of space after overhead but only 8 TB with SHR-2.

Swapping that 4 out with an 8 TB gets you 24 TB with SHR and 16 TB with SHR-2. The confusing bit is the Synology online calculator shows if you start with those 4 8s and add a 4 TB you get 28 and 20 TB but you can't actually do that with SHR-2.

What you can do is start with a 4 TB and 4 8s and you still get 28 TB in SHR but only 16 TB with SHR-2.

Oddly the Drobo calculator appears to split the difference. Presumably they are miss reporting "reserved" space as protection space since Protections space is larger than available.

A quick note on price

For a 5 bay NAS the Drobos will run you a bit less $400-$500 than the one model 5 bay NAS Synology at $790 though there are 2 and 4 bay Synology version that as a bit cheaper. However for around $800 you can get an 8 bay NAS from either and for $1333 you can get a 12 bay Synology. What you get on top of the extra bays is extra Ethernet ports (2 with the 8 bay and 4 with the 12 bay) allowing the NAS to talk to multiple networks and or aggregate ports to get higher throughput so it can handle more things talking to it at a once.

The interfaces

Drobo makes you install an app on a PC to configure and otherwise control your Drobos. And it is pretty simple. As noted about above it was having issues with the 3.15 version not seeing it had been updated.

Synology on the other hand you access via a built in web server that looks like GUI desktop.

It should be noted too from Windows explorer Drobo shows you the theoretical max size volume NOT the actual size and available space of the volume. Synology shows you correct info.

Of course shares are basically folders in a volume so each share shows the space reported for the volume.

The new backup design

After checking out a lot of options I was about to buy a Drobo 5c so I could go with Backblaze without having to pay per TB when someone told me how they had unlimited space on Google Drive. Seems if you have a business account with Google (aka G Suite), which costs the same per month per user as 1 TB of Google Drive space, you get 1 TB of Google Drive space per user (shared between users till you have 5 users, at which point it goes unlimited). So now I can buy as many TBs as I want, no 1 to 10 TB jump AND if even if I get as high as 20 TB it will cost half the 10 TB Google Drive cost. Plus you get more features like shared private drive separate from users private drive space where they can share with outside users. Plus Synology as an app called Cloud Sync which lets you sync your cloud services with the NAS. So now it makes sense to move everything to Google. Note for some odd reason when linking Cloud Sync to a Google Drive account you need to use a browser OTHER THAN Chrome. This is a known bug.
You can also link multiple Google accounts allowing you sync to the the various drive spaces of your G Suite users. You can also encrypt the data uploaded to Google if you are worried about someone getting a hold of it. Coupling this with Odrive lets me do something as wild as syncing triggered snaps from the security camera servers to Google Drive via Odrive while automatically removing them locally after a day to make viewing the triggered snaps easier and reduce used space. On the Synology side you can tell it to sync all but the camera snaps.

Update: some Google Drive gotchas

First to move files between accounts is going to take some work. Some oddness I found was:

• If you have a folder owned by account A and shared with B, B can "make a copy" of that folder but B does not own any files in the resulting folder. However if B uploads to that folder they do own that new file and that new file is shared with the everyone the original folder was shared with. 
• You can not change the owner of the files or folders to your new account unless they both use an email address in the same domain which would seem impossible if you are moving from end user account to business account since the business account requires a domain owned by you while the end user accounts are gmail based.
• Moving the files from the folder owned by A to one owned by B does NOT change their owner either even though it warns you that the files will no longer be available to the people they were shared with and it actually seems to be copying the files, not moving them, as in it takes a while to move them. You will also need to reshare all the folders and files if you want the same people to see them again like they were in user A's folders.
• After doing the above I found video files in User A's Google Photos folder though somewhat hidden since clicking on that folder in the web interface (without Photo app open) takes you to the Photos app where videos do not show. Using left nav you can get to the Photos sub folders and see the files. However these seem to be raw video files that should not even be there.
• If you remove the above video files from the Photo folders you get pop ups telling you people are loosing access and they are in the trash but when you look at the Trash folder online they are not there.
• Even after all the above the files still count against user A's storage total even though they are only visible from user B's account. You will need to remove them from users B's folders too before user A gets anymore space made available.
• So the only way that works to move files between end user and business accounts seems to be to download them local and then upload them again as user B. This is in fact what Google support tells you despite most of the other methods above were posted by people as workarounds. Kind of insane way to move TBs of files between folders on Google servers. And can take forever depending on your upload speed. Over 3 months to upload 1 TB over a 1Mb/s upload connection. Fortunately I have 20 Mbp/s so it should only take about a month.

Note if you are thinking, like I was, you could avoid having to reshare everything by just having B upload the downloaded files to user B's "copy" of the original folder that does not seem to work either because when you upload the file it appears to act as a restore reassigning the owner to user A. You need to start with a new folder owned by B and have B upload to it then reshare all the folders A originally had them shared to. A major pain.