read more
Notes on new backup system
Plan:
• Four backups
⁃ homeWirelessCCCBackup (1TB Partition), disk image backup.
⁃ apartmentWirelessCCCBackup (500GB SSD), disk image backup.
⁃ Time Machine backup to drive C in SF (3TB HDD)
⁃ apartmentWiredCCCBackup (2TB), direct volume backup.
Buying notes:
• https://thewirecutter.com/electronics/storage-devices/
• Don't really care about the speed of the time machine backup, but we do care about the speed of the carbon copy clone.
• Probably should get: a 500GB SSD for Carbon Copy and a 3+ TB Hard Drive for Time Machine
• "You should consider replacing your backup drives between the third and sixth year of use" -Wirecutter
• Probably a good idea to get an SSD and a Hard Drive, because an SSD might last longer.
• Nothing faster than USB 3.0 is necessary for a hard drive, because they're limited by the storage mechanism. So 3.1 is useless on a hard drive.
• Since we may want to replace kind of soon, may make more sense to get a smaller drive? Get a 3TB instead of a 6TB?
⁃ 4 TB Western Digital My Book (determine if we want 3TB or 6TB)
⁃ Okay, probably want a 3TB My Book.
• Should probably get the 500 GB Samsung T5 Portable SSD
⁃ This is for Carbon Copy Clone.
• Should get a desktop drive when possible, because they're less expensive per gigabyte and a bit faster. Only downside is they require an extra power adapter.
• From Wirecutter, the write speed difference between their top pick portable and top pick desktop drive is pretty significant. Write of 138.4 MB/s vs 86.4 MB/s, so nearly twice as fast. Their Blu-ray rip speeds also show that. Read and write of 4.36 minutes vs read and write of 6.55 minutes.
Learnings:
• Slow performance of network transfer is mitigated by backing up to disk image: https://bombich.com/print/174
⁃ When you're backing up tens of thousands of small files, doing so directly to the drive over the network can be slow. The overhead is bad.
⁃ But it can be mitigated by backing up a disk image instead.
⁃ CCC says:
⁃ If the data that you're backing up consists primarily of large files, e.g. music, photos, video — backing up directly to a network appliance will be fine.
⁃ If you're backing up system files or applications, or many files that are smaller than a few MB, we strongly recommend that you back up to a disk image on your network appliance to improve performance and to maintain important filesystem metadata.
• Backing up to a disk image:
⁃ set the image format to one of the read/write formats
• Sparse disk image vs Sparse bundle disk image
⁃ Basically the same thing, but it seems sparse bundle is new and more flexible, and less monolith (things are stored as smaller files), which allows certain formats (like FAT32) which have restrictions on file sizes.
⁃ Both only consume as much files as are there, so that makes it pretty easy to share the same partition with multiple people.
⁃ It seems like because of (https://bombich.com/kb/ccc4/backing-up-disk-image) flush related things, it may be better not to do a sparse bundle? Not sure.
⁃ Never mind, huge difference. Sparse Bundle is necessary, otherwise entire drive would have to be backed up each time: https://apple.stackexchange.com/a/5753/218091
• 128-bit AES vs 256-bit AES
⁃ While 256 does seem unnecessary, and we could do 128, let's start with 256 and see how it goes. It appears as though Filevault 2 is 256, so we should try that. If it's too slow, we can drop down to 128.
• Encrypted at rest?
⁃ Yes, so long as you set an encryption.
• Encrypted in transit?
⁃ Yes, appears so, as long as you have set an encryption.
• Bootable?
⁃ No, disk images aren't bootable. However, they are of course restorable in a number of ways.
⁃ Just boot from recovery HD, select the disk image as the source, and your HD as the destination.
⁃ More info on restoring is here: https://bombich.com/kb/ccc4/restoring-from-disk-image
⁃ Basically just boot from recovery. That's it.
⁃ You can also use migration assistant instead, as detailed in the link above.
⁃ "The destination volume format must match the format of the disk image that you're restoring from. This limitation is specific to Disk Utility"
⁃ So basically since every new Mac is going to have APFS, the drives should be APFS so that we can use Disk Utility easily to restore from.
⁃ It looks like you can't share APFS volumes over AFP. Basically APFS is just a mess. So even though we can't easily restore from an APFS volume, we're just going to make the volumes HFS+ for now so that we can get this done.
• Mounting?
⁃ "Without any additional configuration, CCC will attempt to mount your source and destination volumes before a backup task begins. This applies to many different volume types — ordinary volumes on locally-attached hard drives, disk images, network volumes, and encrypted volumes. If your source or destination volume is on a disk that is physically attached to your Mac (e.g. via Firewire, Thunderbolt, or USB), but it is not mounted, CCC can "see" that device and will attempt to mount it. If your source or destination is a network volume, CCC will obtain the credentials that you use to mount that device when you create the backup task, and will use those credentials to mount the volume before the task begins."
⁃ So it seems like CCC might do this automatically. "therefore it is not necessary to implement a shell script to pre-mount the source or destination."
Wireless Carbon Copy Clone to drive A in NC (1TB Partition)
• Crontab will run mountNetworkDrives.sh every minute, which will ensure that it's available to be backed up to.
⁃ Turns out this is unnecessary. Carbon Copy Cloner mounts it for you.
• Use finder to connect to network, afp://10.0.0.1, rather than going through the network pane in Finder.
• Source is Macintosh HD, all files.
• Destination is
⁃ JOE_NETWORK_CC
⁃ Filename is homeWirelessCCCBackup
⁃ Read/write sparse bundle disk image
⁃ 256-bit AES
• Store the time capsule password in CCC so it can automatically connect to the drive
• Store the backup password and kept the password in keychain so it saves
• Run daily
• Run when the system next wakes
• Run task as soon as missing volume reappears
• Safety net off
Wireless Carbon Copy Clone to Drive B in SF (500GB SSD)
• Format the drive as Mac OS Extended (Journaled) or APFS
⁃ The only limitation is that you may not be able to see APFS over the network. If you can, then APFS is preferable. If you can't, just use HFS+.
• Follow the directions we did for drive A.
• NOTE: it seems smb with the Netgear router fails every time, and it seems like we can't have the read/write password set and connect via AFP, so going to disable the read/write password and use AFP (as per CCC support's recommendation).
⁃ This actually fixed things. So now we're able to backup, thankfully, so long as it goes over AFP. Could probably find a way to get AFP working with a read/write password eventually, that way we could be more secure, but frankly the worst that could happen is someone could delete the backup, they can't read the contents unless they're on the network and even then they need the encryption key, so the data is safe.
Wireless Time Machine Backup to Drive C in SF (3TB)
• Format the drive as Mac OS Extended (Journaled), with GUID Scheme, which is HFS+, because APFS currently isn't supported with Time Machine. It requires HFS+. Update: looks like APFS is now supported by time machine, but see disclaimer below:
⁃ The only limitation is that you may not be able to see APFS over the network. If you can, then APFS is preferable. If you can't, just use HFS+.
• Use finder to connect to network, smb://10.0.0.1, rather than going through the network pane in Finder. Apple recommends SMB.
• Be sure to select "encrypt backups"
• I believe you have to do the initial backup wirelessly too, because it chooses the format of sparse bundle when you do wireless, and won't do that when it's backed up over the wire.
• Just set it up via Time Machine. Easy.
• Not sure if should be doing this over AFP or SMB, it seems like SMB is the future and Apple has started doing all backups via SMB.
⁃ 1/26/19: going to try doing AFP.