I let my girlfriend play games on my phone, use it to take pictures and a few other things in situations where her phone isn't handy. She does the same for me. We trust each other, and neither of us uses it as an opportunity to go digging through texts, browser history or anything else.

At some point saying "no" to "can I use your phone to do <insert innocuous task>" get's weird.