You drive to green canyon hot spring. It’s a 30 minute drive from Rexburg and you can spend the day there. They also have a small cage too. 

Soda Vine and Porter Park are within walking distance. Bring a hammock, grab some sodas and pretzels from Soda Vine and hang out in the park!