How to format a USB disk device on MacOS (Catalina) for use as a Linux boot disk with an MBR partition table. Maybe you have a legacy device that you need to flash the firmware on which requires booting Linuxand you’ve only got a Mac to work on.
First, this is how MacOS automatically formatted my USB drive when I first inserted it:
$ diskutil list disk2
/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_partition_scheme *15.6 GB disk2
1: Apple_partition_map 4.1 KB disk2s1
2: Apple_HFS 2.5 MB disk2s2
WARNING ensure you are working with the correct device before continuing.
You can get a full list of available filesystem types with
$ diskutil listFilesystems
I chose to repartition the entire disk with a Master Boot Record partition table and a 2GB ExFAT partition, but MacOS decided to use the entire disk:
$ diskutil partitionDisk disk2 MBR ExFAT BROCADE 2g
Started partitioning on disk2
Unmounting disk
Creating the partition map
Waiting for partitions to activate
Formatting disk2s1 as ExFAT with name BROCADE
Volume name : BROCADE
Partition offset : 2048 sectors (1048576 bytes)
Volume size : 30545920 sectors (15639511040 bytes)
Bytes per sector : 512
Bytes per cluster: 32768
FAT offset : 2048 sectors (1048576 bytes)
# FAT sectors : 4096
Number of FATs : 1
Cluster offset : 6144 sectors (3145728 bytes)
# Clusters : 477184
Volume Serial # : 5e1e6c4d
Bitmap start : 2
Bitmap file size : 59648
Upcase start : 4
Upcase file size : 5836
Root start : 5
Mounting disk
Finished partitioning on disk2
/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *15.6 GB disk2
1: Windows_NTFS BROCADE 15.6 GB disk2s1
Note that ExFAT uses the same partition type as NTFS, which is 7 in hexadecimal.
$ sudo fdisk -d /dev/disk2
Password:
2048,30545920,0x07,-,1023,254,63,1023,254,63
0,0,0x00,-,0,0,0,0,0,0
0,0,0x00,-,0,0,0,0,0,0
0,0,0x00,-,0,0,0,0,0,0
We need to change that to a Linux filesystem, 0x83 which we can do by using MacOS’ fdisk utility. I’m going to use the raw disk device rdisk2 to avoid the buffer layer.
$ sudo fdisk -e /dev/rdisk2
Password:
fdisk: could not open MBR file /usr/standalone/i386/boot0: No such file or directory
Enter 'help' for information
fdisk: 1> help
help Command help list
manual Show entire man page for fdisk
reinit Re-initialize loaded MBR (to defaults)
auto Auto-partition the disk with a partition style
setpid Set the identifier of a given table entry
disk Edit current drive stats
edit Edit given table entry
erase Erase current MBR
flag Flag given table entry as bootable
update Update machine code in loaded MBR
select Select extended partition table entry MBR
print Print loaded MBR partition table
write Write loaded MBR to disk
exit Exit edit of current MBR, without saving changes
quit Quit edit of current MBR, saving current changes
abort Abort program without saving current changes
fdisk: 1> print
Disk: /dev/rdisk2 geometry: 1901/255/63 [30548096 sectors]
Offset: 0 Signature: 0xAA55
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
------------------------------------------------------------------------
1: 07 1023 254 63 - 1023 254 63 [ 2048 - 30545920] HPFS/QNX/AUX
2: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
3: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
4: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
Modify the partition’s type to 0x83:
fdisk: 1> edit 1
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
------------------------------------------------------------------------
1: 07 1023 254 63 - 1023 254 63 [ 2048 - 30545920] HPFS/QNX/AUX
Partition id ('0' to disable) [0 - FF]: [7] (? for help) 83
Do you wish to edit in CHS mode? [n]
Partition offset [0 - 30548096]: [63]
Partition size [1 - 30548033]: [30548033]
fdisk:*1> print
Disk: /dev/rdisk2 geometry: 1901/255/63 [30548096 sectors]
Offset: 0 Signature: 0xAA55
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
------------------------------------------------------------------------
1: 83 0 1 1 - 1023 254 63 [ 63 - 30548033] Linux files*
2: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
3: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
4: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
fdisk:*1>
Set the bootable flag on partition 1:
fdisk:*1> flag 1
Partition 1 marked active.
fdisk:*1> print
Disk: /dev/rdisk2 geometry: 1901/255/63 [30548096 sectors]
Offset: 0 Signature: 0xAA55
Starting Ending
#: id cyl hd sec - cyl hd sec [ start - size]
------------------------------------------------------------------------
*1: 83 0 1 1 - 1023 254 63 [ 63 - 30548033] Linux files*
2: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
3: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
4: 00 0 0 0 - 0 0 0 [ 0 - 0] unused
Write the modified partition table to disk:
fdisk:*1> write
Device could not be accessed exclusively.
A reboot will be needed for changes to take effect. OK? [n] y
Writing MBR at offset 0.
fdisk: 1> quit
To be certain the kernel had re-read the partition table, I ejected the disk from a Finder window, removed the device and re-inserted it.
$ diskutil list disk2
/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *15.6 GB disk2
1: Linux 15.6 GB disk2s1
Now I can make my Ext2/3/4 filesystem
$ brew install e2fsprogs
$ sudo $(brew --prefix e2fsprogs)/sbin/mkfs.ext2 /dev/rdisk2s1
Password:
mke2fs 1.44.5 (15-Dec-2018)
Creating filesystem with 3818504 4k blocks and 954720 inodes
Filesystem UUID: 0e8be438-fd4b-4b34-9a10-ac440aa3eee1
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208
Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done
To write files into the filesystem, I need to be able to mount it but OSX does not come with drivers for the Ext2/3/4 filesystems so that is where FUSE comes in. It’s a Filesystem in user space so if it crashes, it will only crash a process and not the entire kernel.
All of the below methods depend on the osxfuse kernel extension to pass system calls through to the user space process.
$ brew cask install osxfuse
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/cask).
No changes to formulae.
==> Caveats
To install and/or use osxfuse you may need to enable its kernel extension in:
System Preferences → Security & Privacy → General
For more information refer to vendor documentation or this Apple Technical Note:
https://developer.apple.com/library/content/technotes/tn2459/_index.html
You must reboot for the installation of osxfuse to take effect.
==> Downloading https://github.com/osxfuse/osxfuse/releases/download/osxfuse-3.10.4/osxfuse-3.10.4.dmg
==> Downloading from https://github-production-release-asset-2e65be.s3.amazonaws.com/1867347/58615480-1769-11ea-8f1f-f6cc029e4f08?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200
######################################################################## 100.0%
==> Verifying SHA-256 checksum for Cask 'osxfuse'.
==> Installing Cask osxfuse
==> Running installer for osxfuse; your password may be necessary.
==> Package installers may write to any location; options such as --appdir are ignored.
Password:
installer: Package name is FUSE for macOS
installer: Installing at base path /
installer: The install was successful.
==> Changing ownership of paths required by osxfuse; your password may be necessary
🍺 osxfuse was successfully installed!
We can check that osxfuse has been loaded into the kernel with:
$ kextstat -k | grep osxfuse
212 0 0xffffff7f849da000 0x19000 0x19000 com.github.osxfuse.filesystems.osxfuse (3.10.4) 184072A6-C133-38A8-84A5-E8A3BC937ADD <8 6 5 3 1>
Method 1: ext2fuse
ext2fuse is installable with Homebrew but is based on a fairly outdated package from Sourceforge.
$ brew install ext2fuse
I was not able to get it working at all though. It just crashed every time I tried to mount the filesystem, so…
$ brew uninstall ext2fuse
Method 2: fuse-ext2
The Fuse Ext2 project’s maintainer seems to be busy on other things so some of the code that worked before now needs patching. I found this patch for a homebrew formula in a forked project to get it building properly with the new Xcode version. You can learn more about patching formulae in the Homebrew Cookbook.
$ mkdir -p tmp/fuse-ext2 && cd tmp/fuse-ext2 && wget https://raw.githubusercontent.com/yalp/homebrew-core/fuse-ext2/Formula/fuse-ext2.rb
$ cat <<EOF | patch
--- fuse-ext2.rb.orig 2020-01-17 10:54:44.000000000 +1000
+++ fuse-ext2.rb 2020-01-17 10:55:40.000000000 +1000
@@ -58,6 +58,11 @@
s
end
+ patch do
+ url "https://github.com/alperakcan/fuse-ext2/files/2576060/0001-Fix-new-Xcode-compilation.patch.txt"
+ sha256 "a2a8ff14f36754aead1745b4b5f53b0333376d1bf6abe659ec4eacdbb232aceb"
+ end
+
test do
# Can't test more here as an ext2 image mounting test
# would require fuse-ext2.fs to be installed (see caveats)
EOF
patching file fuse-ext2.rb
After patching the homebrew formula, I can build and install fuse-ext2 with a simple command:
$ brew install --head ./fuse-ext2.rb
==> Downloading https://github.com/alperakcan/fuse-ext2/archive/v0.0.10.tar.gz
==> Downloading from https://codeload.github.com/alperakcan/fuse-ext2/tar.gz/v0.0.10
##O#- #
==> Downloading https://github.com/alperakcan/fuse-ext2/files/2576060/0001-Fix-new-Xcode-compilation.patch.txt
==> Downloading from https://github-production-repository-file-5c1aeb.s3.amazonaws.com/32933629/2576060?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200117%2Fus-east-1%2Fs3%2Faws
######################################################################## 100.0%
==> Patching
==> Applying 0001-Fix-new-Xcode-compilation.patch.txt
patching file tools/macosx/prefpane/English.lproj/fuse_ext2Pref.xib
patching file tools/macosx/prefpane/fuse-ext2.xcodeproj/project.pbxproj
==> ./autogen.sh
==> ./configure --prefix=/usr/local/Cellar/fuse-ext2/0.0.10
==> make
==> cd tools/macosx && DESTDIR=/usr/local/Cellar/fuse-ext2/0.0.10/System make prefpane install
==> cd fuse-ext2 && make install
==> Caveats
For fuse-ext2 to be able to work properly, the filesystem extension and
preference pane must be installed by the root user:
sudo cp -pR /usr/local/opt/fuse-ext2/System/Library/Filesystems/fuse-ext2.fs /Library/Filesystems/
sudo chown -R root:wheel /Library/Filesystems/fuse-ext2.fs
sudo cp -pR /usr/local/opt/fuse-ext2/System/Library/PreferencePanes/fuse-ext2.prefPane /Library/PreferencePanes/
sudo chown -R root:wheel /Library/PreferencePanes/fuse-ext2.prefPane
Removing properly the filesystem extension and the preference pane
must be done by the root user:
sudo rm -rf /Library/Filesystems/fuse-ext2.fs
sudo rm -rf /Library/PreferencePanes/fuse-ext2.prefPane
==> Summary
🍺 /usr/local/Cellar/fuse-ext2/0.0.10: 24 files, 548.4KB, built in 37 seconds
As the build notes advise, the filesystem extension and preference pane need to be installed as root:
$ sudo cp -pR /usr/local/opt/fuse-ext2/System/Library/Filesystems/fuse-ext2.fs /Library/Filesystems/
$ sudo chown -R root:wheel /Library/Filesystems/fuse-ext2.fs
$ sudo cp -pR /usr/local/opt/fuse-ext2/System/Library/PreferencePanes/fuse-ext2.prefPane /Library/PreferencePanes/
$ sudo chown -R root:wheel /Library/PreferencePanes/fuse-ext2.prefPane
$ hash -r
$ fuse-ext2 --help
fuse-ext2 0.0.9 29 - FUSE EXT2FS Driver
Copyright (C) 2008-2015 Alper Akcan <[email protected]>
Copyright (C) 2009 Renzo Davoli <[email protected]>
Usage: fuse-ext2 <device|image_file> <mount_point> [-o option[,...]]
Options: ro, force, allow_other
Please see details in the manual.
Example: fuse-ext2 /dev/sda1 /mnt/sda1
http://github.com/alperakcan/fuse-ext2/
$ mkdir mnt
$ sudo fuse-ext2 /dev/disk2s1 ~/tmp/fuse-ext2/mnt -o allow_other,ro
$ mount | grep disk2
/dev/disk2s1 on /Users/myuser/tmp/fuse-ext2/mnt (osxfuse_ext2, local, read-only, synchronous)
$ ls -l ~/tmp/fuse-ext2/mnt/
total 32
drwx------ 2 root wheel 16384 15 Jan 13:52 lost+found
$ sudo umount /dev/disk2s1
Now I remount it as my user and check that it is writable:
NOTE: write support is still experimental.
$ sudo fuse-ext2 /dev/disk2s1 ~/tmp/fuse-ext2/mnt -o allow_other,rw+,uid=$(id -u),gid=$(id -g)
$ mount | grep disk2
/dev/disk2s1 on /Users/myuser/tmp/fuse-ext2/mnt (osxfuse_ext2, local, synchronous)
$ ls -la mnt
ls: mnt: Socket is not connected
The fuse-ext2 process seems to run and behave normally for a minute or 2 then die silently. I/O operations hang for quite a while before returning a Socket is not connected error.
Method 3: e2fsprogs
The e2fsprogs project actually has a FUSE driver for ext2 but it does not get built by default. All I had to do was patch the formula by telling it where to search for the fuse.h headers.
$ cp $(brew formula e2fsprogs) e2fsprogs.rb
$ cat <<EOF | patch
diff --git a/e2fsprogs.rb b/e2fsprogs.rb
index c1b09cd976..b38756042d 100644
--- a/e2fsprogs.rb
+++ b/e2fsprogs.rb
@@ -23,7 +23,7 @@ class E2fsprogs < Formula
# see https://github.com/Homebrew/homebrew-core/pull/35339
# and https://sourceforge.net/p/e2fsprogs/discussion/7053/thread/edec6de279/
system "./configure", "--prefix=#{prefix}", "--disable-e2initrd-helper",
- "MKDIR_P=mkdir -p"
+ "MKDIR_P=mkdir -p", "--enable-fuse2fs", "CPPFLAGS=-I/usr/local/include/osxfuse"
system "make"
system "make", "install"
EOF
patching file e2fsprogs.rb
$ brew install --head ./e2fsprogs.rb
With this driver, I was able to mount the filesystem and make a directory:
$ sudo fuse2fs /dev/disk2s1 /tmp/mnt -o rw,uid=$(id -u),gid=$(id -g),allow_other
$ ps -u 0 | grep fuse
0 26879 ?? 0:00.00 fuse2fs /dev/disk2s1 /tmp/mnt -o rw,uid=504,gid=20,allow_other
$ sudo mkdir /tmp/mnt/foo
$ sudo bash
# cd /tmp/mnt
# ls -la /tmp/mnt
total 48
drwxr-xr-x@ 4 myuser staff 4096 20 Jan 13:12 .
drwxrwxrwt 10 root wheel 320 20 Jan 13:10 ..
drwxr-xr-x 2 myuser staff 4096 20 Jan 13:12 foo
drwx------ 2 myuser staff 16384 15 Jan 13:52 lost+found
# ls -la /tmp/mnt/foo
total 16
drwxr-xr-x 2 myuser staff 4096 20 Jan 13:12 .
drwxr-xr-x@ 4 myuser staff 4096 20 Jan 13:12 ..
However, permissions didn’t seem to work correctly as I was not able to write to the file as my user:
$ echo bar > /tmp/mnt/foo/bar.txt
zsh: permission denied: /tmp/mnt/foo/bar.txt
And when I tried to write a file as root, the process again hung before returning an error - and the fuse2fs process had crashed, even though the OS still reported it was mounted:
# echo bar > foo/bar.txt
# cat foo/bar.txt
cat: foo/bar.txt: Device not configured
# ps -u 0 | grep fuse
# mount | grep disk2
/dev/disk2s1 on /private/tmp/mnt (osxfuse, synchronous)
Strangely though, after remounting the filesystem, we can see that the file did actually get written:
# ls -l
total 8
-rw-r--r-- 1 myuser staff 4 20 Jan 13:14 bar.txt
# cat bar.txt
bar
A note about leaving the disk mounted
Several times when testing this method, I would unplug all peripherals from my MacBook Pro to go home and when I plugged them all in again the next day, none of the USB devices (keyboard, mouse etc) would work. However, as soon as I unmounted the filesystem on /dev/disk2s1, everything started working again.