"Fruiting" occurs at growth stage 6, so it does seem like you're moving a little slowly. Assuming you fertilize your plants with compost upon planting, you should be able to move them to "Young" within 1.5 days. Just 3 more growth stages will get you to "Fruiting", which can be done as quickly as 4-5 days.
Are you actively managing your crops? If you don't do anything to them they will grow very slowly. If you water them when they're dry, and apply compost and worms and fungicide when they need it, they should grow much more quickly for you.
Growth and events are very random, though. I have a garden of just over 100 crops. I often have crops hitting growth tier 20 while others are barely "Young"; and the majority are in the tier 10-12 range. This is normal, and meant to approximate reality. You may just have gotten unlucky with a lot of weeds. You could change the likelihood of weeds in blockPlaceholders.xml to remediate this.
Also keep in mind, though, that when you harvest you get more crops than with the standard system. The first 3 tiers give nothing, but after that you receive an additional crop per tier. So although you harvest a lot less often, you still get a similar yield per time harvesting. This just helps keep you from constantly needing to replace over and over.
But if you really want to speed things up, open blocks.xml and find every instance of PlantGrowing.GrowthRate; cut the value in half and see how that works for you. I don't know if this will work in an existing game, though. You'll need to test.
