How to Report Bugs to Apple So They Get Fixed

posted in: July 2020, uncategorized | 0

How to Report Bugs to Apple So They Get Fixed

WWDC has come and gone and left in its wake a slew of new operating system betas—macOS 11 Big Sur, iOS 14, iPadOS 14, watchOS 7, and tvOS 14.

If you’re feeling brave and want to donate some of your time to helping Apple improve these operating systems for everyone, you can test these betas, either immediately if you’re in the Apple Developer program, or shortly afterward, once Apple releases the public betas.  Once you install the beta (not on your primary device, of course, because that would be foolhardy!), you’ll promptly run into bugs. That’s why it’s a beta release, after all. If it was ready, it would be shipping.

It’s easy to feel like Apple releases these betas mostly for PR reasons, but that’s far from the truth. Apple does care about receiving bug reports from developers and users alike. But, as you can imagine, hundreds of thousands of developers reporting bugs can overwhelm even Apple’s resources.

How do you report bugs so Apple actually fixes them?

Use Feedback Assistant

Apple solicits bug reports from users and third-party developers via Feedback Assistant, which is both a pre-installed app and a Web site. I recommend using the app to report new bugs. It guides you through specific questions based on the type of bug you’re reporting, automatically generates a sysdiagnose file, and prompts you for logs and reports relevant to that type of bug. The iOS version of Feedback Assistant can also report bugs on connected Apple TV, Apple Watch, and HomePod devices.

The Feedback Assistant app is installed in macOS 10.15 Catalina and later, and in iOS 12.4 and later. On beta operating systems, the Feedback Assistant app appears on the Dock (macOS) or the Home screen (iOS). If you don’t want it in the Dock on the Mac, you can find Feedback Assistant in /System/Library/CoreServices/Applications/Feedback Assistant. It’s often easiest to launch it with Spotlight. You can also launch Feedback Assistant from Safari on any device by navigating to applefeedback://

Feedback Assistant

Every Feedback Assistant bug report is reviewed by someone from Apple Developer Support. That person may answer easy problems themselves or kick the report back if it’s obviously missing key information. If your bug report looks good, they copy it from Feedback Assistant into Radar, Apple’s internal bug tracking system. Radar then sends the report to the appropriate engineering team, where it may or may not be fixed, or even read.

Apple has published an introduction to reporting bugs that’s worth a look, but read on and I’ll tell you how to maximize your chances of getting your bug resolved.

Write Good Bug Reports

Between WWDC in June and the iPhone launch in September, Apple operating system engineers work 12-hour days, six or even seven days a week. They’re finishing up features that their boss told the next boss up were already done (oops!). They’re fixing bugs reported by Apple Quality Assurance (QA) engineers. They drop everything to fix the occasional bug reported by an Apple vice president. (Yes, Apple management uses pre-release products, and they report bugs too.) After all that is done, they’re also supposed to fix bugs reported by third-party developers and users of the public betas.

When I was an Apple software engineer, this was my job. I’d get dozens of new external bug reports daily, and I had time to address only some of them. Many developers wrote great bug reports, but some looked like they were written by a third-grader who skipped their ADHD medication. You can guess which ones got more attention. Here’s what you need to do to write a good bug report. Some of these are specific to developers, but most apply to everyone:

  • Craft a descriptive title. A good bug report starts with a good title. Make the title clear and concise. Bug meetings are long, grueling affairs in which engineers and product managers prioritize hundreds of new bugs. Bugs are listed by title, so your bug’s title is just one in a very long list. A descriptive title makes it easier for engineers to know what it covers without having to open the bug report and read the description. It’s all too likely that bug reports with generic or too-short titles will get passed over with a vague promise to come back later.
  • Describe the problem. Explain what actually happens, and what you think should happen. Don’t assume that it’s obvious what’s wrong. If Apple developer documentation describes what should happen, link to it. Don’t assume I know all the Apple docs by heart.
  • Include clear steps for reproducing the bug. Don’t forget to list any initial conditions, and don’t skip any steps. If the bug doesn’t happen every time, note how frequently it occurs.
  • Be concise. I have a stack of bugs to fix after yours. Please use my time efficiently. The more easily I can reproduce your bug, the more likely I am to fix it.
  • Include screenshots or video. If possible, include a screenshot or screen recording showing how to reproduce the bug, including the bug actually happening. A picture makes the problem obvious. It also makes it harder for an engineer to ignore the problem. Out of sight, out of mind.
  • Clarify the context. Does the bug occur on the device, on the simulator, or on both? Does it occur in both the debug build and release build? Have you tried different device models? Include all this information.
  • Include code. If I’ll need code to see the bug, include code. At a minimum, include a few lines of code that demonstrate the bug. If at all possible, attach a sample Xcode project that demonstrates the bug. Don’t attach your entire app; I don’t want to wade through it. Create a new, empty app in Xcode and add just enough code to show the bug. Add a big, obvious comment saying, “The bug happens here!” so I don’t have to look through all the source files. Attaching an Xcode project has one more advantage, which is that it includes all your build settings. Some bugs happen only with certain build settings.
  • Specify build numbers. If you discuss operating system versions, include the build number (such as 17E262) in addition to the operating system version. (To find the build number in macOS, choose About This Mac from the Apple menu and then click the macOS version number. In iOS, go to Settings > General > About and tap the version number.) Inside Apple, operating system builds are identified by build number. Apple engineers don’t always know offhand which build number corresponds to a particular external beta or release. They can look it up, of course, but anything that saves them time makes it more likely your bug gets fixed.Build numbers in macOS and iOS
  • Include sysdiagnose and logs if relevant. If you use the Feedback Assistant app, it will automatically include a sysdiagnose and prompt you for logs relevant to that type of bug. A sysdiagnose is a report that includes a metric shipload of information about what’s running in your system. It’s extremely useful for those who know how to read it. If you use the Feedback Assistant Web site, you’ll have to generate a sysdiagnose by hand, which isn’t hard. Be sure to attach any relevant console logs or other logs that might contain relevant information. Apple provides instructions for generating a sysdiagnose, along with all kinds of specialized logs for specific types of bugs.
  • Specify the Xcode version. Each operating system beta release typically includes a corresponding Xcode beta release, so be sure to use the latest Xcode. The bug could be in one of the frameworks included in Xcode, not in the operating system itself. You don’t want to report a bug that has already been fixed because you tested it with an old version of Xcode.

Report Bugs Promptly

Apple creates a new internal operating system build each night, called the daily build. Apple engineers install the daily build each morning. Some daily builds are fairly stable; others are not. Every few weeks between WWDC and shipping, a stable build becomes a beta candidate. It gets additional testing to ensure all aspects seem stable. If approved, it gets a beta number and is released to developers and users participating in the public beta.

By the time the beta is posted, Apple engineers already consider it old since they’ve installed weeks of daily builds in the meantime. By the time you download it, test it, file a bug report, your bug report is copied from Feedback Assistant to Radar, and an Apple engineer looks at it, they think that beta is positively antediluvian. The longer it takes you to report a bug, the harder it is for engineers to recreate the situation you’re seeing.

So when a new beta comes out, test it and file bug reports immediately. Make your bug report as current as possible when an Apple engineer sees it.

Check Each Subsequent Operating System Release

It would be nice if Apple informed you if your bug was fixed. It would be nice if the tooth fairy was real, too. Apple receives dozens, if not hundreds, of bug reports about each issue from outside the company. Internal Apple engineers may have already written up the bug as well. One bug report, usually the one with the clearest explanation of the problem, is designated the original. The others are marked as duplicates, or dups, and at least in theory, all bug reporters will be notified when the bug is fixed. If your description doesn’t closely match the description in the original, it may not get detected as a dup, and you may not be notified when the bug is fixed.

As a result, for each newly released beta, you’ll want to test to see if your bug is fixed. If not, consider writing a new bug report against the new release. That brings the bug back to Apple’s attention and lets the engineers know it wasn’t fixed in the latest update.

Even if you think your bug report will be marked duplicate, report it anyway. Think of your report as a vote for that bug to be fixed. When engineers are sitting in bug meetings and deciding which bugs to prioritize, a bug with 100 dups gets more consideration than a bug with 3 dups because it’s obviously affecting a lot more people.

If your bug is fixed, it won’t be marked fixed in Feedback Assistant until the operating system version with the fix ships, whether that’s a beta or the final release. Apple doesn’t like to announce fixes until they ship.

Developers, if a bug is blocking a key feature of your product and doesn’t seem to be getting fixed, email Developer Support and ask if there’s a status update, or if they have any suggestions for a workaround. Wheels sometimes need to squeak to get the grease.

If you have a friend who works in Apple engineering, you might ask them to look up the bug in Radar, to see if it looks like it’s going to be fixed. There are several things that could prevent this. Your bug may not have been copied from Feedback Assistant to Radar yet. Not all engineers can view all Radar bugs for security reasons. And your friend may not feel comfortable doing this for you. If they do, they’ll probably give you general information, like “it’s on track to be fixed in a coming release.” Even knowing this can be useful. Don’t ask your friend to pressure Apple to fix the bug. Chances are that your friend doesn’t work in the department that owns the bug and doesn’t know the engineer it’s assigned to. Apple employs thousands of engineers. They’re not supposed to discuss bug status with non-employees, so don’t put your friend in an awkward spot.

One final note. Feedback Assistant and Radar use separate bug numbering systems. Apple engineers can translate one to the other. Radar is not an acronym. It has been Apple’s bug reporting system for 30 years. Developers used to have a Web portal directly into the Radar database called RadarWeb, though they were restricted to the bugs they reported. Feedback Assistant replaced RadarWeb, and it provides a separate database for tracking bugs reported by third-party developers and users.

Respond to Apple Quickly

Sometimes an Apple engineer will ask for more details or logs. Try to short-circuit this by including any relevant logs with the initial bug report. Occasionally Apple asks for specific additional data or sends a profile that will generate a special log file. Respond quickly. The longer the bug sits, the more likely the engineer will move on to another task. Check your open bugs regularly.

Sometimes it seems like an Apple engineer asks for more info just to stall. When I worked there, I saw that tactic deployed against bugs written by other Apple engineers as well as third-party developers. An engineer asks for some plausible additional information, which takes time to collect. Half the developers never respond, and the engineer can then ignore those bugs, as they’re “waiting for additional information.”

Other Bug Report Sources

Apple’s Crash Reporter automatically detects and reports crashing bugs. Teams are rated on how many active crashes are in their code. Apple has a good system, both technically and procedurally, for fixing crashing bugs. That’s not to say you shouldn’t report crashing bugs—bug reports can still increase the urgency of the fix.

When reporting crashing bugs or kernel panics, hardware bugs, or printing problems on the Mac, you’ll need to include a System Information report. Use the System Information app in the Utilities folder to generate this report (just choose File > Save).

Apple engineers do pay attention to some developer forums, blogs, and podcasts. Complaints that pop up repeatedly may be written up as bugs in Radar. I’ve seen text from a developer post copied into a Radar bug report. But Apple engineers rarely post in developer forums because the company discourages it. Or at least that used to be true: Apple’s redesigned developer forums encourage developer participation (see “Apple Announces WWDC 2020 Schedule,” 11 June 2020). Although it’s rarely acknowledged, Apple engineering does track developer opinions.

Understand Bug Priority

Apple assigns all bugs a priority, from P1 to P5. P1 is a crashing or data loss bug, and such bugs generally must be fixed. P2 is a feature that doesn’t work, and Apple almost always fixes such problems. P3 is a feature that doesn’t work correctly; Apple likes to fix P3 bugs, but some get postponed. P4 is a user interface issue or other minor bug, but such bugs often get postponed. P5 is an enhancement request; such requests tend to get ignored.

Some Apple engineers just fix the bugs they’re assigned, working from high priority to low priority. Often they never make it to the low priority bugs. But more experienced Apple engineers also cherry-pick some lower priority bugs that they know will annoy users and make sure they get fixed. Apple engineers use Apple products too, and want them to work well. Some departments even give out prizes to the engineer who fixes the most bugs.

When the ship date is far away, Apple engineers are allowed to check in pretty much whatever code they like. As we get closer to alpha, beta, and eventually golden master, there are more constraints. First, only P1 or P2 bugs can be fixed. Then only P1 bugs from an approved list. Each bug and fix is rated, for the seriousness of the bug, and the risk of the fix. No one wants a high-risk fix for a minor bug. As release nears, even serious bugs can be postponed. Having a bad, but rare, bug with well-understood characteristics is better than a poorly understood fix with insufficient testing time. The fix will get added to the first patch release, after more rigorous testing.

Apple’s Development Cycle

To maximize the chance of getting your bugs fixed, it helps to understand Apple’s development cycle. After the major X.0 releases of iOS and macOS ship, usually in September, development work starts on the next year’s X+1 releases for each. Development continues through the end of the year, and by February or March, QA starts seriously testing builds and reporting bugs. Over time, the development version becomes more stable, going from barely usable to not that bad. By May, the daily build is starting to look decent. Then comes the rush to put together the WWDC beta. All those unfinished features and ignored bugs suddenly get more attention.

Beta 1 at WWDC may look pretty rough when you first install it (on a test device!), but it still represents a ton of work. That’s when you want to start reporting bugs! Apple engineers have been assigned to read bug reports and fix problems. Take advantage of the attention and report any bugs you find.

More beta releases will come in July and August. Test each one, to see if your bugs are fixed, and look for regressions, which are previously working features that broke. Remember, if a bug you reported in a previous beta still isn’t fixed, don’t be afraid to report it again.

As September approaches, the betas become more stable, but Apple also makes fewer changes. If you found a minor bug that isn’t fixed by August, it will probably still be there in the golden master. Report it anyway, and Apple will hopefully fix it in one of the early patch releases.

The new operating system usually ships in September as version X.0.0. The patch releases follow immediately. X.0.1 and sometimes X.0.2 are planned even before X.0.0 ships. X.0.1 has a few important bug fixes that didn’t make it into the X.0.0 release. X.0.2 has less vital bug fixes that were too late for X.0.0. In a bad bug year, there may even be an X.0.3 release.

Once those first patch releases are out of the way, Apple engineers start working on the next major release X+1. They don’t have much bandwidth to fix bugs in the currently shipping operating system anymore. Continue to report bugs, but the likelihood they’ll get fixed drops considerably.

(As an aside, Apple teams are usually divided by area or feature, not by main release versus patch release. If you’re a power management engineer, for instance, you’ll work on that feature in both the initial X.0 release and all patch releases. It works poorly to have one engineer write the initial code and another engineer come along later and fix bugs because the second engineer isn’t intimately familiar with the code, why it was designed that way, what ideas were tried and discarded, and so on. It’s better to have an engineer own a particular piece of code, including all releases.)

There will probably be an X.1 and X.2 release, but those typically introduce new features, like iCloud folder sharing and battery health management in Catalina. These releases will have some bug fixes, but that’s not their primary mission. It never hurts to report bugs during this time, but unless they’re particularly bad, don’t expect them to be addressed until the next major release at the earliest.

In the end, if you want Apple to fix the bugs you find (and who doesn’t?), follow these simple rules. Write clear, complete bug reports. Include screenshots, video, and sample code, if possible. Report bugs as early as possible. Work within Apple’s development cycle.

And most important, keep sending in your reports. Bugs that aren’t reported don’t get fixed.