There are hidden damage multipliers in the game. Xwings are generalist. They do more damage to most buildings (except turrets), but in a smaller area. So for most buildings, Xwings are your star fighter of choice. Bwings do more damage to turrets, but less to all other buildings. So for turrets - especially clustered turrets - Bwings are your star fighter of choice. For walls? Z95s. Ywings? never use them personally.
I usually carry a few Xwings for sniping SCs or dealing with SC troops and a few Bwings incase I need to destroy some turrets. Rarely use Z95s any more (used to at lower levels) and never use Ywings.
The only buildings I use star fighters on are SC or resources (Xwings) and turrets (Bwings). That said I generally try to avoid using star fighters unless I have to.