Do the positive classes make up 0.17% of your data? A softmax of a single neuron should be 1 always right?

I'd make sure you know what outputs go with what prediction. Metrics can come after that.

The model is doing what you told it to. In that scenario it said keep the lane and it was right. Congrats! You should not think of the outputs as probabilities. They add to 1 but if the model has a score of 0.3 for keep lane that doesn't mean there is a 30% chance it should keep the lane. It's just a score (unless you've built some more sophisticated probabilistic modeling into it)

As mentioned above cross entropy is a good metric. Another metric you may consider is a ROC curve. It will show performance across thresholds. Maybe 0.5 as a cut off isn't best?

And for what it's worth I wouldn't want to be in a vehicle that incorrectly switched lanes 7% of the time ;)

That's about right. Convolution priors in particular lend themselves to edge detection. CV xai is weird in general though so I've stepped back a bit. Is a good explanation one that looks good or one that is faithful to the model or what? Everyone disagrees. So Ive moved to inputs with interpretable features (text, tables, science, etc).

Great list. To add, in the CV space you should be very careful with many "xai" methods. Usually they're just fancy edge detectors. Been Kim is pretty good on this stuff.

https://arxiv.org/abs/1810.03292

Regularization and drop out helps with overfitting. It will almost always reduce your training accuracy. What you need is a testing dataset and compare there.