Closed Bug 1735345 Opened 3 years ago Closed 2 years ago

Huge memory usage on macos Monterey in [NSCursor set]; macOS popup warning saying machine is maxed out. 70+ GBs of memory shown in warning

Categories

(Core :: Widget: Cocoa, defect, P2)

Firefox 95
defect

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: williamyoung, Unassigned)

References

(Blocks 1 open bug)

Details

User Story

On macOS 12 Monterey, using a non-standard mouse pointer color (set in the system Accessibility Display settings) causes a large memory leak in Firefox. Firefox version 94 includes a fix that reduces the memory leak, but the problem can still occur. The problem has been reported to Apple and a fix is expected in a future update to macOS 12.

Workaround: until a newer version of macOS 12 is available with the fix, use the standard cursor size and colors. In macOS 12 System Preferences, select Accessibility, then Display, then Pointer. Set the pointer size to normal and reset the pointer colors by clicking the Reset button.

The problem is known to occur on macOS 12.0.1 (21A559) and earlier.

Attachments

(6 files)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:95.0) Gecko/20100101 Firefox/95.0

Steps to reproduce:

Standard browsing, nothing heavier than normal

Actual results:

Mac popped up warning of max memory usage and prompted me to quit firefox ((95.0a1 (2021-10-12) (64-bit))

Expected results:

never had this happen before about 2 weeks ago

The Bugbug bot thinks this bug should belong to the 'Core::Widget: Cocoa' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → Widget: Cocoa
Product: Firefox → Core
Component: Widget: Cocoa → Performance

Could you save a copy of about:memory (anonymized) and upload here when you are experiencing the high memory? (Before it gets to the point of force quitting)

Flags: needinfo?(williamyoung)
Attached file memory-report.json.gz

Memory report attached. MacOS popup showed 78.96GB memory usage, soon as it popped up I grabbed this report.

Flags: needinfo?(williamyoung)

Thanks. 70 GB of heap unclassified in the parent process.

You seem to have a decent number of addons, is it possible to disable some of those to see if they are causing the problem?

Disabled all extensions but for 8 shown in pic linked below

Enabled Extensions

Disabled all but those mentioned in previous post, restarted browser, then generated that report. I'll monitor this and update you if it happens again with extensions disabled or if i do not encounter the problem over next 1-2 days I will also let you know.

https://www.reddit.com/r/firefox/comments/q7yeyv/insane_memory_leak/ is another similar report.

williamyoung, do you use fullscreen very much?

Flags: needinfo?(williamyoung)

I can confirm I am seeing this same behaviour - uncapped memory consumption using FF in the latest macOS Monterey beta - often being used full-screen.

Summary: Huge memory usage; macOS popup warning saying machine is maxed out. 70+ GBs of memory shown in warning. Have had this happen multiple times over past 2 weeks. Running macos monterey dev beta 9 on mac mini m1. Ffox nightly most up to date version. → Huge memory usage on M1 macos Monterey; macOS popup warning saying machine is maxed out. 70+ GBs of memory shown in warning
Blocks: monterey
Status: UNCONFIRMED → NEW
Component: Performance → Widget: Cocoa
Ever confirmed: true
Severity: -- → S2
Priority: -- → P2

mstange is going to try updating to Monterey to see if he can reproduce.

Vic, would you be able to try running Nightly Firefox and getting a DMD report? You should be able to do this by starting Firefox from the terminal:

DMD=1 /Applications/Firefox\ Nightly.app/Contents/MacOS/firefox-bin

Then go to about:memory and click "Save" under "Save DMD output". It should output a bunch of extra stuff in the terminal like:

DMD[836] $DMD = '1'
DMD[837] $DMD = '1'
DMD[839] $DMD = '1'
DMD[840] $DMD = '1'
DMD[841] $DMD = '1'
DMD[842] $DMD = '1'
DMD[844] $DMD = '1'
DMD[845] $DMD = '1'
DMD[846] $DMD = '1'
DMD[847] $DMD = '1'
DMD[849] $DMD = '1'
DMD[850] $DMD = '1'
DMD[851] $DMD = '1'
Unable to read VR Path Registry from /Users/jrmuizel/Library/Application Support/OpenVR/.openvr/openvrpaths.vrpath
DMD[855] $DMD = '1'
DMD[856] $DMD = '1'
DMD[857] $DMD = '1'
DMD[836] opened /var/folders/z2/52nrrnjx24z2fd60wd6glw1w0000gn/T/dmd-1634226999-836.json.gz for writing

Then you can run dmd on the first file that it writes (e.g. /var/folders/z2/52nrrnjx24z2fd60wd6glw1w0000gn/T/dmd-1634226999-836.json.gz)

$ /Applications/Firefox\ Nightly.app/Contents/Resources/dmd.py /var/folders/z2/52nrrnjx24z2fd60wd6glw1w0000gn/T/dmd-1634226999-836.json.gz  > /tmp/report.txt

You can then attach /tmp/report.txt to the bug.

More details here: https://firefox-source-docs.mozilla.org/xpcom/dmd.html and I'm happy to help if any steps are unclear.

Flags: needinfo?(mveinot)

I am unable to run the nightly build - I get a dialog pop up that says "Your Firefox profile cannot be loaded. It may be missing or inaccessible".

In the console. I see:

DMD=1 /Applications/Firefox\ Nightly.app/Contents/MacOS/firefox-bin 
DMD[23944] $DMD = '1'
DMD[23951] $DMD = '1'
2021-10-14 13:09:55.912 plugin-container[23951:635616] nil host used in call to allowsSpecificHTTPSCertificateForHost
2021-10-14 13:09:55.912 plugin-container[23951:635616] nil host used in call to allowsAnyHTTPSCertificateForHost:
2021-10-14 13:09:55.922 plugin-container[23951:635616] nil host used in call to allowsSpecificHTTPSCertificateForHost
2021-10-14 13:09:55.922 plugin-container[23951:635616] nil host used in call to allowsAnyHTTPSCertificateForHost:
2021-10-14 13:09:55.922 plugin-container[23951:635623] nil host used in call to allowsSpecificHTTPSCertificateForHost
2021-10-14 13:09:55.922 plugin-container[23951:635623] nil host used in call to allowsAnyHTTPSCertificateForHost:
UNSUPPORTED (log once): POSSIBLE ISSUE: unit 1 GLD_TEXTURE_INDEX_2D is unloadable and bound to sampler type (Float) - using zero texture because texture unloadable
Flags: needinfo?(mveinot)

I was able to run it by removing my Firefox/ folder and letting it make a new one

I was able to dump DMD files, however I get this when trying to analyze it:

Traceback (most recent call last):
  File "/Applications/Firefox Nightly.app/Contents/Resources/./dmd.py", line 1010, in <module>
    main()
  File "/Applications/Firefox Nightly.app/Contents/Resources/./dmd.py", line 1002, in main
    digest = getDigestFromFile(args, args.input_file)
  File "/Applications/Firefox Nightly.app/Contents/Resources/./dmd.py", line 307, in getDigestFromFile
    fixStackTraces(inputFile, isZipped, opener)
  File "/Applications/Firefox Nightly.app/Contents/Resources/./dmd.py", line 293, in fixStackTraces
    tmpFile.write(fix(line))
  File "/Applications/Firefox Nightly.app/Contents/Resources/./dmd.py", line 268, in fix
    return fixModule.fixSymbols(line, jsonMode=True)
  File "/Applications/Firefox Nightly.app/Contents/Resources/fix_stacks.py", line 82, in fixSymbols
    initFixStacks(jsonMode, slowWarning, breakpadSymsDir, hide_errors)
  File "/Applications/Firefox Nightly.app/Contents/Resources/fix_stacks.py", line 38, in initFixStacks
    raise Exception("cannot find `fix-stacks`; please run `./mach bootstrap`")
Exception: cannot find `fix-stacks`; please run `./mach bootstrap`

You can try again with the --no-fix-stacks option.

Got it - download the report at https://mvgrafx.net/~vmark/ff_n_DMD_report.txt

Attached file report.txt

DMD report

FYI - still confirm seeing unbound memory use with the nightly build with a completely fresh profile and no extensions.

This is super useful, thank you Vic! It seems to point at mouse cursor images being leaked.

Could you run dmd.py again, and also specify --max-frames 16? At the moment we only get 8 stack frames (the default), which might hide some interesting details.

Attached file report-16.txt

DMD report with --max-frames = 16

Summary: Huge memory usage on M1 macos Monterey; macOS popup warning saying machine is maxed out. 70+ GBs of memory shown in warning → Huge memory usage on M1 macos Monterey in [NSCursor set]; macOS popup warning saying machine is maxed out. 70+ GBs of memory shown in warning

I've filed FB9704004 about this.

-[NSCursor set] appears to be leaking images, causes 70GB leak in Firefox

This appears to be a regression in one of the recent macOS Monterey Betas.

Two Firefox users reported a very large memory usage. We’re tracking this in https://bugzilla.mozilla.org/show_bug.cgi?id=1735345 .
We have a memory report which blames the allocations on this stack:

#02: _malloc_zone_malloc[/usr/lib/system/libsystem_malloc.dylib +0x1e770]
#03: createImageWithSizeRenderInstructions[/System/Library/PrivateFrameworks/AccessibilitySupport.framework/Versions/A/Frameworks/AccessibilityFoundation.framework/Versions/A/AccessibilityFoundation +0x7430]
#04: -[_AXFMouseCursorGeneratorLayer _createImageForScale:color:][/System/Library/PrivateFrameworks/AccessibilitySupport.framework/Versions/A/Frameworks/AccessibilityFoundation.framework/Versions/A/AccessibilityFoundation +0x66e0]
#05: -[_AXFMouseCursorGenerator createImageForScale:][/System/Library/PrivateFrameworks/AccessibilitySupport.framework/Versions/A/Frameworks/AccessibilityFoundation.framework/Versions/A/AccessibilityFoundation +0x5064]
#06: -[_AXFMouseCursorGenerator _registerWithName:connection:isGlobal:setActive:seed:][/System/Library/PrivateFrameworks/AccessibilitySupport.framework/Versions/A/Frameworks/AccessibilityFoundation.framework/Versions/A/AccessibilityFoundation +0x5418]
#07: _AXFCursorSetAndReturnSeed[/System/Library/PrivateFrameworks/AccessibilitySupport.framework/Versions/A/Frameworks/AccessibilityFoundation.framework/Versions/A/AccessibilityFoundation +0x70b8]
#08: _AXInterfaceCursorSetAndReturnSeed[/System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices +0x2d1d8]
#09: -[NSCursor _reallySet][/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit +0x1e01d0]
#10: -[NSCursor set][/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit +0x1e0050]

Firefox calls -[NSCursor set] very frequently during mouse motions, so many leaked images might accumulate.

Wanted to add to this if it helps: I realized that this issue probably corresponded to when I used the new accessibility features to make the mouse cursor larger and inverted the colours. To test the theory I went back to accessibility and set the cursor size back to normal and the colours to default.

The issue is either gone or significantly mitigated. More time is needed to see if it grows out of control - but as I write this I've been to a few different sites with multiple tabs and memory usage sits at a comfortable 608MB.

So this is definitely related to the new cursor accessibility features.

That makes a lot of sense.

Thanks for filing this issue with Apple; one of my coworkers is looking into it now.

Thanks Eric!

I'm also debugging why Firefox sets the cursor so frequently. This is some ancient code.

(In reply to Markus Stange [:mstange] from comment #23)

I'm also debugging why Firefox sets the cursor so frequently. This is some ancient code.

We do this unconditionally, but it seems we could avoid it?

I guess this is the problematic situation, and we can probably avoid it nowadays that binary plugins are not a thing?

(mid-aired with emilio here)

It's from this code, which I added 12 years ago to work around a problem with plug-ins in bug 496601, and which gets called on every mouse move:

  [[NSCursor currentCursor] set];

I've filed bug 1736049 to try to remove it.

Depends on: 1736049

This is reproducible on x64 as well.

Summary: Huge memory usage on M1 macos Monterey in [NSCursor set]; macOS popup warning saying machine is maxed out. 70+ GBs of memory shown in warning → Huge memory usage on macos Monterey in [NSCursor set]; macOS popup warning saying machine is maxed out. 70+ GBs of memory shown in warning

It seems like the underlying issue in macOS has been fixed as of Monterey RC2 (21A559)

Vic, are you sure? Are you testing Firefox 93? Haik can still reproduce it on 21A559

Flags: needinfo?(mveinot)

I re-enabled a large cursor and didn’t observe any rampant memory growth. I didn’t test extensively though. I will give it a more thorough test this evening

Flags: needinfo?(mveinot)
Attached image Repro Settings

I'm using the attached settings to reproduce the problem. Moving the mouse around a lot seems to accelerate the memory usage and it is attributed to the Firefox process in Activity Monitor. Closing all tabs doesn't reduce the Firefox memory use. In my case, it is still reproducible on macOS 21A559 with Firefox 93.0 on x64, MacBook Pro 2018.

Yes, sorry - I retract my claim. Though I will say it appears setting the colours has more of an impact than setting the size.

See Also: → 1737735

I've been encountering this bug as well. I'm on the official MacOS Monterey release and had my cursor set to a larger-than-normal size as well as custom colors. Having returned those settings to their default, the memory leak seems to have stopped. Happy to share a memory dump or anything if y'all need it, but it looks like Markus has fixed the issue in #1736049.

We received similar report on Twitter recently: https://twitter.com/y80481629/status/1453305283317354496

macOS 12.0.1 (21A559)
Firefox 93.0 (64-bit)
MacBook Pro (16-inch, 2021)

Firefox 94 just came out, and it contains a change which greatly reduces the severity of the leak. But some leak still remains, and it needs to be fixed by Apple in a macOS update.

Thanks for the info, Markus! I'll ask folks to update in the meantime.

User Story: (updated)
User Story: (updated)
User Story: (updated)
User Story: (updated)

Seems this may have fixed it for some users:
https://news.ycombinator.com/item?id=29143270

Apple engineering expects that the underlying issue in macOS is fixed in Monterey 12.1 Beta 3 (21C5039b), released today.

I just tested 12.1 Beta (21C5039b) and it does not appear to be fixed. In Firefox 93 I can still get memory usage to climb up to 5GB within two minutes of moving the mouse.

User Story: (updated)

Redirect a needinfo that is pending on an inactive user to the triage owner.
:spohl, since the bug has high priority, high severity and recent activity, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(williamyoung) → needinfo?(spohl.mozilla.bugs)

I'm not able to reproduce this in macOS Monterey 12.5 Beta (21G5027d), but I didn't originally experience this issue. Haik or Markus, could one of you confirm that this is indeed fixed now?

Flags: needinfo?(spohl.mozilla.bugs)
Flags: needinfo?(mstange.moz)
Flags: needinfo?(haftandilian)

Looking at my comment history on the Apple bug report FB9704004, it looks like this was fixed a while ago in macOS 12.1 beta 4 (21C5045a). I confirmed the fix in macOS at the time.

Status: NEW → RESOLVED
Closed: 2 years ago
Flags: needinfo?(mstange.moz)
Flags: needinfo?(haftandilian)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: