Browsing on a phone or with Debian works ok for me with Firefox, though I don’t like Firefox that much.
I found Organic Maps preferable to OsmAnd but neither are that great. It should be possible to do something reasonable without a lot of CPU demands, given how dedicated GPS map navigation devices existed ih the early 2000s.
Yes if you ditch Youtube and anything else that requires modern codecs, that solves another issue. I’ve found Newpipe has broken a few times but it usually works, so that is what I use.
Modern apps and games (requiring GPU even) are another story, but let’s assume you don’t want to run them.
This leaves the question: if you want a BIFL smart phone but you don’t want to make phone calls with it, don’t want to run a web browser, and don’t want to watch videos on it, what DO you want it to do?
GPS decoding is less computationally difficult than you seem to think, and in any case, in phones it’s done by a hardware module. The Garmin Geko handheld GPS was made in 2003 and ran on two AAA cells for 12 hours or something like that. Today’s GPS’s fit inside wristwatches and use even less power. It’s just not that big a deal. The cpu load of mapping applications on phones is dealing with the maps, computing driving directions, etc.
I wouldn’t worry about map updates by internet. The roads don’t change that often. You can update from a USB-connected computer once a year or so and be fine.
The other stuff doesn’t sound too bad, though idk why you want a phone for the purpose. If the GPS is for road navigation you can get an old dedicated unit that runs on 12 volts do you don’t have to mess with batteries. Those were nicer than phones in some ways. I still have a couple of them kicking around.