How to debug broken compass?

I've got LineageOS 20 on my Fairphone 4 and my compass is constantly broken.

After a calibration by doing the figure 8 motion, N/S works pretty okay but it's very jittery; a few degrees which is quite a lot. This might still be acceptable since it'd be smoothed out.

When I point the needle north and rotate the device 90 degrees however, I'm almost 30 degrees off which is borderline unusable.

Google maps claims the compass has low accuracy, no matter how long I do the figure 8 motion.

What gives? How could I debug this?