NetWorx floods .xsession-errors with GLib-GObject-CRITICAL GdkColormap errors on Linux Mint
8 posts
I am running NetWorx 25.11 (apparently the latest version) on Linux Mint 21.3. Has been fine until sometime today.
I suddenly had the drive with my home directory (1TB SSD) start running out of free space (!) this afternoon or evening. Shut down everything and re-booted, and found the now-previous .xsession-errors file (before the reboot, so .xsession-errors.old after the reboot) had suddenly ballooned to over half a terabyte (!). On checking through its contents, I found a HUGE portion of its contents were error messages apparently stemming from NetWorx:
... (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: g_object_ref: assertion '!object_already_finalized' failed (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: g_object_ref: assertion '!object_already_finalized' failed (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: g_object_ref: assertion '!object_already_finalized' failed (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: g_object_ref: assertion '!object_already_finalized' failed ... (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: Free queue for GdkColormap (0x10b88040) is larger than 65535, called g_object_freeze_notify() too often. Forgot to call g_object_thaw_notify() or infinite loop (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: g_object_unref: assertion 'old_ref > 0' failed (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: Free queue for GdkColormap (0x10b88040) is larger than 65535, called g_object_freeze_notify() too often. Forgot to call g_object_thaw_notify() or infinite loop (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: g_object_unref: assertion 'old_ref > 0' failed (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: Free queue for GdkColormap (0x10b88040) is larger than 65535, called g_object_freeze_notify() too often. Forgot to call g_object_thaw_notify() or infinite loop (networx:2221): GLib-GObject-CRITICAL **: 18:44:57.816: g_object_unref: assertion 'old_ref > 0' failed ...
...right up to where I started the shutdown. No idea exactly how much of the file is made up of the "assertion failed" errors, but it is a big majority of the file. The number of "too often" errors is also huge, but smaller, likely cut off by my shutting down for a reboot.
My system has been up and running after the reboot for the past 50 minutes or so, and I don't see any errors like these showing in the new .xsession-errors file. So the reboot apparently solved something.
Are you aware of anything about this? The error entries suggest they are from something happening in NetWorx. Thanks.
In case it makes any difference: File properties say that the session for .xsession-errors.old started on 8 November, and the system stayed up continuously until it was on the verge of crashing this evening.
The file is big enough that I don't have anywhere to archive it, and I need the drive space for my work, so am deleting the file but am keeping a small sample of these error messages in a document file for my own record. I don't see any errors with "NetWorx" on them outside of the ones sampled above.
Cheers
Your log was very helpful here. It points at the Linux tray icon code, where GTK signal handlers weren't being disconnected before their objects were freed. Callbacks could then fire on objects that were already gone, which explains the reference counting errors, and over weeks of uptime they build up into the flood you saw.
I've made a test build with a fix. Could you try it and let me know if it happens again? If you can leave it running for about as long as before, that would confirm it.
Thanks Andrew. Have downloaded and installed it. I'll let you know by sometime around the New Year how it goes (since it had lasted nearly 4 weeks before deciding to blow up). If you don't hear anything back by then, feel free to ask for an update.
I obviously have to reboot the system after updates to the kernel, which had been coming about every week or two for a long time, but there haven't been any for the past while. That's likely why the system had stayed running long enough to start seeing the issue now.
Update: Continuing to happen. I had made the mistake of updating again since installing your test version (! - apologies), so ended up re-installing that as part of this. Hence this taking longer than I had intended. The flip side is I also have results for the release versions since then.
Also set up a script in cron to watch for the errors when they happen, so I can catch them before they blow the file up.
Results:
- version 25.12 has the same error.
- version 26.1 has the same error. (Looks like this is still the latest version as I write this.)
- The patched version 25.11 you provided for testing on 2 Dec (I finally removed the other versions and re-installed this one fresh on Jan 5th) has the same error.
From .xsession-errors, currently for the patched version you provided on 2 Dec ("About" shows it as 25.11, although the errors showing for later versions were the same):
(The "unhandled exception" seems to relate to my exiting Networx. I have shut it down to keep the errors file from growing incessantly again.)
$ ls -l .xsession-* -rw------- 1 douglas douglas 2415695425 Feb 5 04:27 .xsession-errors -rw------- 1 douglas douglas 35515 Jan 30 00:08 .xsession-errors.old
Errors found in .xsession-errors: (last bit of "tail" results, just after my script for watching just got triggered):
(networx:2968): GLib-GObject-CRITICAL **: 04:16:25.664: g_object_unref: assertion 'old_ref > 0' failed (networx:2968): GLib-GObject-CRITICAL **: 04:16:25.664: Free queue for GdkColormap (0x3ee06840) is larger than 65535, called g_object_freeze_notify() too often. Forgot to call g_object_thaw_notify() or infinite loop (networx:2968): GLib-GObject-CRITICAL **: 04:16:25.664: g_object_unref: assertion 'old_ref > 0' failed (networx:2968): GLib-GObject-CRITICAL **: 04:16:25.664: Free queue for GdkColormap (0x3ee06840) is larger than 65535, called g_object_freeze_notify() too often. Forgot to call g_object_thaw_notify() or infinite loop (networx:2968): GLib-GObject-CRITICAL **: 04:16:25.664: g_object_unref: assertion 'old_ref > 0' failed (networx:2968): GLib-GObject-CRITICAL **: 04:16:26.416: Free queue for GdkColormap (0x3ee06840) is larger than 65535, called g_object_freeze_notify() too often. Forgot to call g_object_thaw_notify() or infinite loop (networx:2968): GLib-GObject-CRITICAL **: 04:16:26.416: g_object_unref: assertion 'old_ref > 0' failed (networx:2968): GLib-GObject-CRITICAL **: 04:16:26.416: Free queue for GdkColormap (0x3ee06840) is larger than 65535, called g_object_freeze_notify() too often. Forgot to call g_object_thaw_notify() or infinite loop (networx:2968): GLib-GObject-CRITICAL **: 04:16:26.416: g_object_unref: assertion 'old_ref > 0' failed An unhandled exception occurred at $00007FC58CA84225: EAccessViolation: Access violation $00007FC58CA84225 An unhandled exception occurred at $00000000004292F7: EAccessViolation: $00000000004292F7 $0000000000426E9C $0000000000425D51 $0000000000425D51 Gjs-Message: 04:16:26.561: JS LOG: [LookingGlass/info] Removing XAppStatusIcon: networx (:1.45/org/x/StatusIcon/Icon_1)
I missed including: The first set of errors showing in the file:
$ grep networx .xsession-errors |head Gjs-Message: 00:09:36.470: JS LOG: [LookingGlass/info] Adding systray: networx (1x1px) Gjs-Message: 01:48:24.217: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.45/org/x/StatusIcon/Icon_1) (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.660: g_object_ref: assertion '!object_already_finalized' failed (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.660: g_object_ref: assertion '!object_already_finalized' failed (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.660: g_object_ref: assertion '!object_already_finalized' failed (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.661: g_object_ref: assertion '!object_already_finalized' failed (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.661: g_object_ref: assertion '!object_already_finalized' failed (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.661: g_object_ref: assertion '!object_already_finalized' failed (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.661: g_object_ref: assertion '!object_already_finalized' failed (networx:2968): GLib-GObject-CRITICAL **: 02:52:13.661: g_object_ref: assertion '!object_already_finalized' failed
...It then carries on until what I had mentioned in the previous note.
This one turned out to be a bug in a dependency, the GTK2 interface of the Lazarus LCL, rather than in NetWorx itself. Mismatched ref and unref calls in its theme stock image code were corrupting the reference count of the system GdkColormap, and over time that caused the GLib critical errors you were seeing.
I've made another test build with the GTK2 interface fixed. Since 26.10 the Linux version is built on GTK 3 and doesn't go through that GTK2 code at all any more, so the current version on the NetWorx page is clear of it either way.
If it still happens, let me know and we'll look into it again.
Hi Andrew - update: I have been holding off on some system updates to give your patch (dated 6 Feb) as much time to run as I could.
So far it has been running (28 Feb - 9 Feb) = 19 days, and is showing NO SIGNS of any trouble. Log file size is not even close yet to what it had been getting up to when the failures got going. It does show some log entries for NetWorx, which seem to be fairly normal spaced-out occurrences, so I'm guessing they are probably nothing to worry about:
$ ls -l ~/.xsession-errors* -rw------- 1 douglas douglas 967667 Feb 28 07:38 /home/douglas/.xsession-errors -rw------- 1 douglas douglas 192184 Feb 9 12:19 /home/douglas/.xsession-errors.old $ grep networx ~/.xsession-errors Gjs-Message: 19:48:52.779: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.46/org/x/StatusIcon/Icon_1) Gjs-Message: 14:45:14.082: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.46/org/x/StatusIcon/Icon_1) Gjs-Message: 00:57:41.411: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.46/org/x/StatusIcon/Icon_1) Gjs-Message: 01:36:39.771: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.46/org/x/StatusIcon/Icon_1) Gjs-Message: 02:37:12.244: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.46/org/x/StatusIcon/Icon_1) Gjs-Message: 09:30:23.363: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.46/org/x/StatusIcon/Icon_1) Gjs-Message: 03:01:31.759: JS LOG: [LookingGlass/info] Adding XAppStatusIcon: networx (:1.46/org/x/StatusIcon/Icon_1)
For comparison:
- your first patch dated 2 Dec blew up after (5 Feb - 29 Jan) = 8 days.
- version 26.1 blew up after (29 Jan - 17 Jan) = 12 days.
- version 25.12 blew up after (17 Jan - 11 Jan) = 6 days.
This one has lasted 19 days, with no complaints yet.
(All of these are based on my little cron script noting when the blow-up was just getting under way.)
I have to do my system updates and reboot. But it is looking like this patch has fixed the problem. Cheers
Thanks for testing it so thoroughly.
As I said, the cause was a bug in the GUI library NetWorx is built on: a mismatch between ref and unref calls on a system GTK object. The log only starts filling up once the mismatch passes 0xFFFF, which is why it took days to build up rather than appearing at once.
The networx lines you're still seeing in .xsession-errors are normal. That's just the tray icon being registered.
It should be fixed now, but if it ever comes back, let me know and we'll look again in case any mismatches remain.