Way back when they would be built one then the other often followed by a connection between the two (or possibly additions to a house followed eventually by the barn). It breaks up the monotony of the main siding, sometimes saves money, and looks more quaint / historic / organically developed. It was also a trend in the 60s / 70s when a lot of housing stock was built to use different siding on the garage.

https://people.uwec.edu/ivogeler/w188/articles/NewEngland-connecting-barn.htm