I've done many specialist runs that fully invest into a single attribute. You do not need strength at all. POIs have chairs, stone/cobblestone/cement pallets, and tons of stuff you can scrap/smelt to metal or sell to traders. You get clay from buried treasure quests. I fully reinforce concreted my converted POI base out by day 30. Between the POI resources and the trader you an absolutely get the resources you need without being strength and it's not a problem at all. I've never had to dedicated mine for anything. I only recently upgraded to iron tools so I did 90% of that with stone tools.
I feel like you're still stuck in a mentality born in old alphas. But mining is no longer the only way to effectively get resources and hasn't been for multiple updates now. Only serious mining you really have to do as a non-strength build is one overnight trip for shale. That's about it. Get the POI resources and use the trader effectively and you'll be fine

.