Aren’t they the same thing?

Yes. Adding negative bias is same as subtraction, so to say, similarly subtracting negative bias is like addition. Ultimately it doesn't make a difference.

Doesn't matter. The bias can be negative if that is what the model learns

Yeah I get that but if you add a positive bias?

While training the model, bias will be trained accordingly. Suppose ideal bias value is +x, if you are subtracting the bias instead of +x bias value will become -x, so overall w -(-x) will become w+x and vice versa.

They are the same thing.

> 1 + (-1) = 0

> 1 - 1 = 0

Aside from what others have mentioned, let's assume that we don't have a symmetrical situation, i.e. that the range of the function we're learning, as well as the domain of weights and biases, is [0, inf>. Then it makes more sense to add bias than to subtract it, as it will lead to smaller weights and less chance to overflow or for the gradients to explode.

It makes more sense to subtract the biases if in the scenario described above, you want a more expressive layer, but with less numerical stability. This is because a subtractive bias allows the weights to be of greater magnitude, which in terms gives you more effective range for the weights.

But note that neural networks are not done with integer weights, and in some libraries there is no autograd for integers even.

If you add a negative number then you are adding a subtraction if you get my point.

Afaik, So if you say adding or substracting bias doesn't matter.

All subtraction is just addition of a negative number. The model will learn a value for the bias parameter, which can be between negative infinity and infinity

Convention coming from linear algebra: Ax+B where B is a vector of real numbers, positive or negative.

What makes you feel that subtracting is in any way more meaningful?