Nothing is purely simple (for example even if there were such a thing as a 1e0 first baseman, he would still not be 100% out for Gbx to him, because a few times he would be holding a runner and become a 2e0)
That said, there are MORE rolls on the pitching cards with a * starter than you estimate.
example my last completed team (a very mediocre one at 82-80) had 4 SP*
Camnitz faced 1429 batters
Grove 1313
Veale 1321
and Higbe 1055
so my average SP* saw 1280 batters over the season and roughly 36.1% were not x-rolls and off the pitchers card (13.9% are x-rolls)
so----462 rolls per season on the SP* card that fielding does not come into play
My best starting batter had 726 plate appearances (0 injury Frank Thomas) - roughly half of those rolls were off his own card (363 rolls over the season)
So - my
average SP* got to play more rolls off his card that did not involve fielding than my BEST PA batter (with 0 injury possibility) got off his hitter card and impacted 27% more rolls than the hitter.
my
best SP* contributed even more -
42% more rolls than off my best starting batterFWIW