Getting Ready for Apple's OS Updates: How Mac Admins Are Testing

Back in June, Apple announced new versions of its operating systems: macOS Monterey and iOS/iPadOS 15. Sometime this fall, those new OSes will be officially released. In the meantime, Apple has made prerelease versions of the new operating systems available to you for testing. 

As we've said before, we think it’s a really good idea to test those OSes before they’re released, so you can (a) get an idea of how well they’ll work with your existing tech infrastructure and (b) give Apple and other vendors feedback on the new software and its compatibility with the apps and services your organization relies on. 

But that’s just our opinion. We were curious to find out what real-world Apple admins think about the whole testing question. So we asked a select group of our blog readers what their testing regimes were like. Their answers are anecdotal, of course. But we think they shine some light on what’s really happening out there as we all wait for macOS Monterey and iOS/iPadOS 15 to officially ship.

How You’re Testing

Of the 25 IT admins we spoke to, 15 said they’re testing beta versions of the new OSes; the rest said they have no plans to do so. (More on the latter in a bit.) Of those 15 who are testing, nine said that (as of the beginning of September) they’re still in the middle of their testing programs; five said they plan to start testing soon, and one said he’s done. 

Several respondents argued for waiting before testing:

Don't start testing until the later betas are announced. The first few betas—more than a month ahead of release—still have bugs that are being worked out. 

Most of the admins in our survey are focusing on macOS Monterey; only eight said they’re testing iOS 15 or iPadOS 15. Most say they get those prerelease versions from Apple’s developer site or the public beta program—not from AppleSeed for IT. We think that’s too bad, given that AppleSeed has so many great tools for testers, including sample test plans and easy feedback tools.

Many of the people we checked with say they’re testing both by upgrading existing devices and by installing the beta software on freshly cleaned and wiped ones. Only a few said they test on virtual machines, which don’t always deliver the most reliable results. 

As for the testing process itself, respondents expected the job to take anywhere from four person-hours to 90. Roughly half expected their testing programs to require more than 20 person-hours, the other half expected it to take less.

One bit of advice cropped up repeatedly among our survey respondents: Don’t restrict testers to the IT department. A good number recommend “enlist[ing] employees to do the testing and validation for their own apps and systems.” As one admin put it:

Since we're a software company, the biggest test of an OS prerelease is if it can hold up in our developer environment. So I will usually find a senior or principal engineer to test against our CLI tools, VPN configurations, IDEs, and so on.

Why Some Admins Wait

As previously noted, we think pre-release testing is a good idea. But a significant number of the admins we spoke to said they didn’t plan to do it. They cited two principal reasons.

First: resource constraints. Even a minimal testing program requires an investment of time that isn’t always practical for some organizations. As one admin wrote, “We are busy with some major changes at our office, so just do not have time to test.” 

But the bigger reason is caution. Many organizations say they plan to wait three months or more before they start installing new operating systems on their users’ devices. They essentially crowdsource the first round of testing. As one admin put it, “We wait and see how others in the Mac community deal with any surprises.” 

However, these waiting admins will still test. 

“We are testing, just not the beta,” says Zack Czengoldi, director of information technology at the Tennessee Credit Union League. 

His plan: Keep everyone locked on the current version of the OS for three to six months after the new release. During that time, he’ll install the new macOS Monterey on an older notebook and a newer Mac with Apple silicon and test those. 

“We always wait a minimum of three months before we embark on our testing,” says Majdi Al-Ramahi, IT support technician at Axonify. “It’s best for us to hold off for a little while before we make the jump.”

After those three months, his IT staffers will spend roughly two weeks testing on their own company-owned computers. If the new OS passes those tests without any issues, they will then grab one Mac user from each department and have them upgrade and test for about a week. As long as that testing doesn’t uncover any issues, they will then roll out the new OS to the rest of their users.

One downside to the wait-and-see approach: You won’t be able to buy a new model Mac, iPhone, or iPad—which will ship with the new OSes on them—until your due diligence is done. If users are allowed to bring their own devices to work, you may find the new OSes on your network anyway. 

About Kandji

Whether you upgrade everyone on day one or take a more leisurely upgrade route, you can make sure the transition is smooth by leveraging your MDM solution. Here at Kandji, we’re making sure our device-management tools will be ready on release day, so you can get the most out of your fleet of Apple devices. With powerful features like zero-touch deployment, one-click compliance, and offline remediation, Kandji has everything you need to enroll, configure, and secure your devices.