[ubuntu] Convert headed install to unheaded

I have a small server that I once installed as a regular Ubuntu, but I have not connected a monitor to it for many years now.

I want to convert it to a headless install, that is, remove the x-server and graphical login and stuff to release back some CPU or other resources...

Is it at all possible? If so; how?

Would I still be able to run a firefox from that machine when I log in with X forwarding?