Viewing a single comment thread. View all comments

Beneficial_Law_5613 OP t1_ixlx0hc wrote

Reply to comment by PredictorX1 in [D] inference on GNN by Beneficial_Law_5613

After training the model I saved the weights. Then I loaded the weight to the model and gave the model a sequence of the data to see/visualize the output of the model on images(basically if the ego vehicle should make a lane change or nor) and even when it should keep the lane change I get a 49% it should change the lane. I am trying to predict if the ego vehicle should make a lane change or not in some data(even if it hasn't been trained on all that data)

−1

jellyfishwhisperer t1_ixlzco4 wrote

The model is doing what you told it to. In that scenario it said keep the lane and it was right. Congrats! You should not think of the outputs as probabilities. They add to 1 but if the model has a score of 0.3 for keep lane that doesn't mean there is a 30% chance it should keep the lane. It's just a score (unless you've built some more sophisticated probabilistic modeling into it)

As mentioned above cross entropy is a good metric. Another metric you may consider is a ROC curve. It will show performance across thresholds. Maybe 0.5 as a cut off isn't best?

And for what it's worth I wouldn't want to be in a vehicle that incorrectly switched lanes 7% of the time ;)

2

PredictorX1 t1_ixlzohu wrote

Good points! I'd also mention that, if probability estimates are desired, the numeric model outputs could be calibrated as a separate step at the end.

2

Beneficial_Law_5613 OP t1_ixm3fpz wrote

Yes but in some cases when the car should keep the lane(or is keeping the lane/its not making any lane change) I get a 76% that it should make a lane change. Thats why I am confused, and for more information: Pred=model(data) PredS = nn.Sigmoid(Pred)*100 Print(PredS) # and here when I give the data of a car that is not making a lane change i get 76% for all its data points/frames. But being honest I don't know if this 76% is for lane changing or lane keeping.

−1

jellyfishwhisperer t1_ixm5zz6 wrote

I'd make sure you know what outputs go with what prediction. Metrics can come after that.

3