What should I do with the mouse I just caught?

mice just look for the closest food source to make themselves at home. so yes go ahead and release him at that park you mentioned. he will likely make a nest in some bushy area where he can come out and eat scraps that people leave on the streets or in trash cans. i cant kill a mouse either. yes they are dirty and carry disease and damage homes, but so do children...lol.