>I'm a bit confused how my brokerage (Merrill) can offer an interest rate on an upcoming treasury bill when it hasn't been auctioned yet and the interest is not known until auction date?

That's just an estimate of what the return will be. You're correct - you won't know the real return until after the auction.

>And how does a brokerage offer so many treasury bills outside of an upcoming auction (https://www.treasurydirect.gov/auctions/upcoming/)? Is that through the secondary market, as in someone is selling a bill before maturity and someone else is buying that bill?

Probably. I use Fidelity and you can buy new issue T-bills at auction, or existing bills on the secondary market. Their user interface has settings where you can view just new issues, or those being sold on the secondary market.