Wed 15 September 2021
DriverKit, /Applications, and Quarantine
Earlier this year, I released a DriverKit-based port of GCAdapterDriver. For the most part it's been pretty smooth sailing, once you get accustomed to how things work in a DriverKit-based world rather than Kext-based world. There's one odd bug that's sprouted up recently that only seems to be related to distribution, though - and I've not seen it noted anywhere else, so I figured I'd throw up a quick entry about it in case any other developers start trying to debug this.
The Confusing Bug
The bug is simple (ish) in nature, but only shows up when you're shipping your application. Some quick context: DriverKit-based applications must be in /Applications in order to load an extension (at least, when a user has System Integrity Protection enabled). If your user doesn't place the app in there and attempts to install the extension, the resulting error will be OSSystemExtensionErrorDomain Code 3, which roughly corresponds to: Application is not in /Applications.
Now, users of GCAdapterDriver started getting this message even though the app was in the /Applications folder. Odd, right? I started going through the usual debugging experience, making sure it was installed correctly, and that they had restarted the app after moving it into the folder (if they had started previously). All this turned up fine, though.
So what's going on?
Enter: Quarantine
A fitting title for the past year and some change, I guess.
I recalled from other projects that if the com.apple.quarantine attribute was on an Application bundle, it could cause some odd errors to occur. This generally happens when an app is downloaded from the internet and flagged; for whatever reason, it sometimes doesn't get cleared when the user trusts the app. I've mostly seen this with Chrome, but I see no reason for it to not happen with any other browser.
In this case, I asked the user to check their status with the following:
xattr /Applications/GCAdapterDriver.app
Which, sure enough, produced the following:
com.apple.quarantine
On a hunch, I asked them to clear the attribute - which you can do with the following:
sudo xattr -r -d com.apple.quarantine /Applications/GCAdapterDriver.app
Lo and behold, after starting the app in a non-quarantined state, the driver activation worked just fine.
Is this really a bug?
It's hard to say. I can see some logic for not activating a quarantined app's driver, but at the very least I think the error should be updated to spot the case where the application is in /Applications and the quarantine attribute is what's blocking driver activation. I've filed feedback (FB9628611) for this, so here's hoping it gets updated in Monterey at least.