If there insulation behind the fiberboard, I'd be inclined to just leave it. If it has asbestos in it that's the cheapest and easiest. It's a fair insulator too. I hate fiberboard. No real reason, I just do. lol If you take it off you have to do something with it. It's made not to burn worth a darn. Dumpsters are expensive.

The electrical presents some interesting challenges. You can move the boxes out by cutting around the fiberboard and moving them or extend the boxes. Depending on who wired it you may have enough wire to extend them. Murphy's law says some of the wires will be way short, always!

If you take the fiberboard off you could move the wires in the exposed conduit into the walls and clean that up. That's a factor to consider. I only saw the 1 section of conduit.

There's not a wrong way to go. It's really your choice. Just because I have an opinion, doesn't make me right. Hopefully, I'm given you somethings to consider.

What you have on you wall is concrete board many brands make it hardi-board or dura rock ect which you can skim coat with concrete (mix to the consistency of drywall mud) to make it smooth. Or you can go right overtop with drywall and construction adhesive.