Yes, but nominally he's directly accountable to the President as he's a presidential appointee to a legislatively created position. Congress could impeach him, and the courts could enjoin him, but only the president can fire him.

Edit: clarified it's a legislatively created position. Constitutional positions, such as Federal Judges, are presidential appointments, but are not directly accountable to the President and are only answerable to Congress.