Can't quite tell but I do not think that is drywall, looks like it may be fibreboard? If so you can't really mud that, may want to consider installing some type of durable panelling (there are products made for this type of thing, or even mdf or plywood) and use that as the finished wall. Or could put new drywall over top and mud that. But Mudding and taping is a lot of work if you're new to it, and drywall is easily damaged in a garage, although it does look great when new.

As for insulation, guess it may depend where you live. If you have cold winters and ever want to hear the garage it's a must.