@chergert I've noticed in general that libadwaita does slow down things a lot, at least that's just my experience.
Using nautilus on an old armv7 board for example, I get much better performance when I break/disable libadwaita (GTK_THEME=asdf). It makes nautilus look strange, but performance is a lot better. Maybe it's that?