OpenZFS on OS X - User contributions [en]

Time Machine Backups

2023-10-13T12:06:27Z

188.136.90.201: Tested Time machine on ZFS raidz1/cache successfully.

== Backup to Local Disk ==
There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.

=== Sparsebundles ===
Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

<code># tmutil setdestination -a /Volumes/[sparse bundle volume name]</code>

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

Does not work under Ventura.

=== zvols ===

<ol>

Tested successfully under Ventura using raidz1 with cache disk (all disks on USB).

<li>Create a zvol to use for Time Machine backups:

<blockquote>

<code>sudo zfs create -V [size] [pool]/[volume]</code>

</blockquote>

For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:

<blockquote>

<code>sudo zfs create -V 1T Backup/tmdisk1</code>

</blockquote>

</li>

<li>Get OSX disk number corresponding to the zvol:

<blockquote>

<code>ioreg -trn "ZVOL [pool]/[volume] Media" | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:

<blockquote>

<code>ioreg -trn 'ZVOL Backup/tmdisk1 Media' | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

</li>

<li>Format the zvol as a HFS+, using its disk number from the previous step:

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]</code>

</blockquote>

For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5</code>

</blockquote>

For newer systems (Big Sur), because Time Machine now prefers apfs (see
https://support.apple.com/en-au/guide/mac-help/mh15139/mac), the following
commands should create a apfs zvol that Time Machine can use:

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] GPT [disk number]
sudo diskutil apfs create [disk number]s[slice number] [OSX Disk Name]</pre>

</blockquote>

For example, if the zvol was disk5 in OSX, and the HFS+ volume on that zvol
was on slice 1 (which can be confirmed using '<code>diskutil list disk5</code>')
the following should format that disk as a apfs disk named "TimeMachine":

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ "TimeMachine" GPT disk5
sudo diskutil apfs create disk5s1 "TimeMachine"</pre>

</blockquote>

</li>

<li>Set the zvol as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/[OSX Disk Name]</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in
the previous steps, the following will set it as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/TimeMachine</code>

</blockquote>

</li>

<li>(Optional) Enable encryption for Time Machine backups on the zvol:

<ul>

<li>For HFS+ formatted zvols, the following procedure should work:

<ol>

<li>Get the disk identifier for the HFS+ disk on the zvol:

<blockquote>

<code>diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:

<blockquote>

<code>diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'</code>

</blockquote>

</li>

<li>Convert the HFS+ disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert [Disk ID] -passphrase</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert disk5s2 -passphrase</code>

</blockquote>

Please note, this process can take a considerable amount of time (several hours for a 1TB volume). You can check on the progress of the encryption using the following command:

<blockquote>

<code>diskutil cs list | grep -e "Conversion" -e "Volume Name"</code>

</blockquote>

</li>
</ol>

</li>

<li>For apfs formatted zvols, the following procedure should work (based on
comments in the forums, see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612):

<blockquote>

<code>sudo diskutil apfs encryptvolume [Disk ID] -user disk</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the zvol was formatted as apfs named "TimeMachine" in a prior
step, whose disk identifier was returned as disk5s1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil apfs encryptvolume disk5s1 -user disk</code>

</blockquote>

Based on the forums (see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612), it is possible to "bless" the Time Machine
zvol by "open /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericTimeMachineDiskIcon.icns", select the first icon in the resulting window, copying (⌘-C) the icon, then Get Info (⌘-I) on the Time Machine disk icon, and pasting (⌘-V) the copied image to the Time Machine disk to "bless" it as a Time Machine disk. For some reason, Time Machine won't do that by itself, as it does with normal external volumes when it first starts using them for Time Machine.
</li>
</ul>
</li>
</ol>

== Backup to Network Share ==

Time Machine can backup over the network to either HFS+ or APFS target volumes, shared via SMB. If you try to do this with a ZFS target volume, the client gets an error saying "The selected network backup disk does not support the required capabilities."

You have two options from this point on:
# You can create an HFS+ or APFS target volume on your zpool. There are two ways to do that, explained in the section above about local backups: create the volume as a sparse bundle, or as a zvol.
# You can set the properties <code>com.apple.devdisk</code> and <code>com.apple.mimic_hfs</code> to 'on' on the dataset you wish to put your backups onto, through the <code>zfs set</code> command. (note that if you do this through an already mounted dataset, you may have to unmount it and re-mount it again after issuing the <code>zfs set</code> commands)

You don't need the steps that set the volume as a time machine destination on the server; it's just going to be shared over the network. When you share the volume on the network, follow the instructions at https://support.apple.com/guide/mac-help/a-shared-folder-time-machine-mac-mchl31533145/mac to make it available as a TimeMachine volume.

Time Machine Backups

2023-10-13T12:05:22Z

188.136.90.201: /* zvols */

Time Machine Backups

2023-10-08T16:53:25Z

188.136.90.201: Does not work under Ventura

== Backup to Local Disk ==
There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.

=== Sparsebundles ===
Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

<code># tmutil setdestination -a /Volumes/[sparse bundle volume name]</code>

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

Does not work under Ventura.

=== zvols ===

<ol>

Does not work under Ventura.

<li>Create a zvol to use for Time Machine backups:

<blockquote>

<code>sudo zfs create -V [size] [pool]/[volume]</code>

</blockquote>

For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:

<blockquote>

<code>sudo zfs create -V 1T Backup/tmdisk1</code>

</blockquote>

</li>

<li>Get OSX disk number corresponding to the zvol:

<blockquote>

<code>ioreg -trn "ZVOL [pool]/[volume] Media" | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:

<blockquote>

<code>ioreg -trn 'ZVOL Backup/tmdisk1 Media' | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

</li>

<li>Format the zvol as a HFS+, using its disk number from the previous step:

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]</code>

</blockquote>

For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5</code>

</blockquote>

For newer systems (Big Sur), because Time Machine now prefers apfs (see
https://support.apple.com/en-au/guide/mac-help/mh15139/mac), the following
commands should create a apfs zvol that Time Machine can use:

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] GPT [disk number]
sudo diskutil apfs create [disk number]s[slice number] [OSX Disk Name]</pre>

</blockquote>

For example, if the zvol was disk5 in OSX, and the HFS+ volume on that zvol
was on slice 1 (which can be confirmed using '<code>diskutil list disk5</code>')
the following should format that disk as a apfs disk named "TimeMachine":

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ "TimeMachine" GPT disk5
sudo diskutil apfs create disk5s1 "TimeMachine"</pre>

</blockquote>

</li>

<li>Set the zvol as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/[OSX Disk Name]</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in
the previous steps, the following will set it as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/TimeMachine</code>

</blockquote>

</li>

<li>(Optional) Enable encryption for Time Machine backups on the zvol:

<ul>

<li>For HFS+ formatted zvols, the following procedure should work:

<ol>

<li>Get the disk identifier for the HFS+ disk on the zvol:

<blockquote>

<code>diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:

<blockquote>

<code>diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'</code>

</blockquote>

</li>

<li>Convert the HFS+ disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert [Disk ID] -passphrase</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert disk5s2 -passphrase</code>

</blockquote>

Please note, this process can take a considerable amount of time (several hours for a 1TB volume). You can check on the progress of the encryption using the following command:

<blockquote>

<code>diskutil cs list | grep -e "Conversion" -e "Volume Name"</code>

</blockquote>

</li>
</ol>

</li>

<li>For apfs formatted zvols, the following procedure should work (based on
comments in the forums, see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612):

<blockquote>

<code>sudo diskutil apfs encryptvolume [Disk ID] -user disk</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the zvol was formatted as apfs named "TimeMachine" in a prior
step, whose disk identifier was returned as disk5s1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil apfs encryptvolume disk5s1 -user disk</code>

</blockquote>

Based on the forums (see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612), it is possible to "bless" the Time Machine
zvol by "open /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericTimeMachineDiskIcon.icns", select the first icon in the resulting window, copying (⌘-C) the icon, then Get Info (⌘-I) on the Time Machine disk icon, and pasting (⌘-V) the copied image to the Time Machine disk to "bless" it as a Time Machine disk. For some reason, Time Machine won't do that by itself, as it does with normal external volumes when it first starts using them for Time Machine.
</li>
</ul>
</li>
</ol>

== Backup to Network Share ==

Time Machine can backup over the network to either HFS+ or APFS target volumes, shared via SMB. If you try to do this with a ZFS target volume, the client gets an error saying "The selected network backup disk does not support the required capabilities."

You have two options from this point on:
# You can create an HFS+ or APFS target volume on your zpool. There are two ways to do that, explained in the section above about local backups: create the volume as a sparse bundle, or as a zvol.
# You can set the properties <code>com.apple.devdisk</code> and <code>com.apple.mimic_hfs</code> to 'on' on the dataset you wish to put your backups onto, through the <code>zfs set</code> command. (note that if you do this through an already mounted dataset, you may have to unmount it and re-mount it again after issuing the <code>zfs set</code> commands)

You don't need the steps that set the volume as a time machine destination on the server; it's just going to be shared over the network. When you share the volume on the network, follow the instructions at https://support.apple.com/guide/mac-help/a-shared-folder-time-machine-mac-mchl31533145/mac to make it available as a TimeMachine volume.

Time Machine Backups

2023-10-08T16:52:41Z

188.136.90.201: /* Sparsebundles */

== Backup to Local Disk ==
There are two approaches to using ZFS for Time Machine Backups - Sparsebundles on a ZFS data set and zvols formatted as HFS+.

=== Sparsebundles ===
Here's one approach to using ZFS for your Time Machine Backups.

Because Time Machine doesn't recognize ZFS datasets as a compatible disk for Time Machine backups, as a work around, we create an HFS+ sparsebundle disk image, store it on a ZFS dataset, and set the mounted image as a backup destination (no "TMShowUnsupportedNetworkVolumes" needed).

1. Create, and mount, a sparsebundle from a ZFS dataset (e.g., with makeImage.sh or Disk Utility.app).

2. Set the sparsebundle as the (active) backup destination:

<code># tmutil setdestination -a /Volumes/[sparse bundle volume name]</code>

While it has been discussed in heated arguments (e.g., https://github.com/openzfsonosx/zfs/issues/66) I still believe there's at least one ZFS feature I'd like to test with Time Machine: compression.

The hypothesis being that an HFS+ sparsebundle stored on a compressed (gzip, lz4), deduped dataset should
yield a compression ratio > 1.0 (previously observed 1.4 with compression=on, dedup=off, FreeBSD network Time Machine drives).

Does not work under Ventura.

=== zvols ===

<ol>

<li>Create a zvol to use for Time Machine backups:

<blockquote>

<code>sudo zfs create -V [size] [pool]/[volume]</code>

</blockquote>

For example, the following will create a 1TB volume named tmdisk1 on a zpool named Backup:

<blockquote>

<code>sudo zfs create -V 1T Backup/tmdisk1</code>

</blockquote>

</li>

<li>Get OSX disk number corresponding to the zvol:

<blockquote>

<code>ioreg -trn "ZVOL [pool]/[volume] Media" | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

For example, if a zvol named tmdisk1 was created as shown above, the following will get its disk number and store it in the shell variable DISKID:

<blockquote>

<code>ioreg -trn 'ZVOL Backup/tmdisk1 Media' | awk -F'"' '/BSD Name/ {print $4;}'</code>

</blockquote>

</li>

<li>Format the zvol as a HFS+, using its disk number from the previous step:

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] [disk number]</code>

</blockquote>

For example, if the zvol was disk5 in OSX, the following will format that disk as a HFS+ disk named "TimeMachine":

<blockquote>

<code>sudo diskutil eraseDisk JHFS+ "TimeMachine" disk5</code>

</blockquote>

For newer systems (Big Sur), because Time Machine now prefers apfs (see
https://support.apple.com/en-au/guide/mac-help/mh15139/mac), the following
commands should create a apfs zvol that Time Machine can use:

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ [OSX Disk Name] GPT [disk number]
sudo diskutil apfs create [disk number]s[slice number] [OSX Disk Name]</pre>

</blockquote>

For example, if the zvol was disk5 in OSX, and the HFS+ volume on that zvol
was on slice 1 (which can be confirmed using '<code>diskutil list disk5</code>')
the following should format that disk as a apfs disk named "TimeMachine":

<blockquote>

<pre>sudo diskutil eraseDisk JHFS+ "TimeMachine" GPT disk5
sudo diskutil apfs create disk5s1 "TimeMachine"</pre>

</blockquote>

</li>

<li>Set the zvol as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/[OSX Disk Name]</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in
the previous steps, the following will set it as a Time Machine destination:

<blockquote>

<code>sudo tmutil setdestination -a /Volumes/TimeMachine</code>

</blockquote>

</li>

<li>(Optional) Enable encryption for Time Machine backups on the zvol:

<ul>

<li>For HFS+ formatted zvols, the following procedure should work:

<ol>

<li>Get the disk identifier for the HFS+ disk on the zvol:

<blockquote>

<code>diskutil list [OSX Disk Name] | awk '/[OSX Disk Name]/ { print $NF;}'</code>

</blockquote>

For example, if the zvol was formatted as HFS+ named "TimeMachine" in step 3, the following will get its disk identifier:

<blockquote>

<code>diskutil list TimeMachine | awk '/TimeMachine/ { print $NF; }'</code>

</blockquote>

</li>

<li>Convert the HFS+ disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert [Disk ID] -passphrase</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the if the zvol was formatted as HFS+ named "TimeMachine" in step 3, whose disk identifier was returned as disk5s2 in step 5.1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil coreStorage convert disk5s2 -passphrase</code>

</blockquote>

Please note, this process can take a considerable amount of time (several hours for a 1TB volume). You can check on the progress of the encryption using the following command:

<blockquote>

<code>diskutil cs list | grep -e "Conversion" -e "Volume Name"</code>

</blockquote>

</li>
</ol>

</li>

<li>For apfs formatted zvols, the following procedure should work (based on
comments in the forums, see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612):

<blockquote>

<code>sudo diskutil apfs encryptvolume [Disk ID] -user disk</code>

</blockquote>

This will ask for the encryption password (which may be stored in the Keychain). For example, if the zvol was formatted as apfs named "TimeMachine" in a prior
step, whose disk identifier was returned as disk5s1, the following will convert the disk to use encryption:

<blockquote>

<code>sudo diskutil apfs encryptvolume disk5s1 -user disk</code>

</blockquote>

Based on the forums (see https://openzfsonosx.org/forum/viewtopic.php?f=26&t=3612), it is possible to "bless" the Time Machine
zvol by "open /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericTimeMachineDiskIcon.icns", select the first icon in the resulting window, copying (⌘-C) the icon, then Get Info (⌘-I) on the Time Machine disk icon, and pasting (⌘-V) the copied image to the Time Machine disk to "bless" it as a Time Machine disk. For some reason, Time Machine won't do that by itself, as it does with normal external volumes when it first starts using them for Time Machine.
</li>
</ul>
</li>
</ol>

== Backup to Network Share ==

Time Machine can backup over the network to either HFS+ or APFS target volumes, shared via SMB. If you try to do this with a ZFS target volume, the client gets an error saying "The selected network backup disk does not support the required capabilities."

You have two options from this point on:
# You can create an HFS+ or APFS target volume on your zpool. There are two ways to do that, explained in the section above about local backups: create the volume as a sparse bundle, or as a zvol.
# You can set the properties <code>com.apple.devdisk</code> and <code>com.apple.mimic_hfs</code> to 'on' on the dataset you wish to put your backups onto, through the <code>zfs set</code> command. (note that if you do this through an already mounted dataset, you may have to unmount it and re-mount it again after issuing the <code>zfs set</code> commands)

You don't need the steps that set the volume as a time machine destination on the server; it's just going to be shared over the network. When you share the volume on the network, follow the instructions at https://support.apple.com/guide/mac-help/a-shared-folder-time-machine-mac-mchl31533145/mac to make it available as a TimeMachine volume.