That's kinda how it is early game.
But once you do your morning chores, you should leave your farm and do some foraging.
Just keep those foraged items on you, and eat them when you run low on energy.
Eventually you will level up and have more energy.
But even later in the game you'll need to keep food in your inventory for an energy boost.
Also exploring the map and meeting the villagers is important.
Don't just spend your time tending farm and doing chores. Get out there and explore.
Fishing was really hard at first too but worth it in the end.
also this website is helpful
https://stardewvalleywiki.com/Getting_Started