I'm assuming you mean MDF or something similar? That seems like a weird choice for a window frame, where there's a risk of condensation but I'm not familiar with (I'm assuming) America construction rules. Yes, wood filler is definitely a better choice than plaster and will be fine if it's something like MDF.