I mean.. there is always a small chance of something happening but I wouldn't let it stop you. I love the woods and I have done the same thing. Parked along the road where it was safe and just walked into the woods. Made a nice fire, a hot drink, and some lunch.
Could something happen? Sure. I could also trip and hit my head. Do I let it stop me? Not at all.
Hell, I almost had a vehicle run me over in a parking lot because this woman was looking down at her phone. Since I was paying attention I managed to avoid her. She never looked up either.
My point is.. there are dangers EVERYWHERE. Not just in the woods.
Keep doing what you're doing.