It would be nice when playing private matches (and possibly public matches) to be able to broadcast audio to the entire server instead of just your team. When playing private matches with friends it's nice to be able to congratulate (or curse) them for a good shot. I think I've seen it done in another game where holding a particular direction on the d-pad while you talk allows you to chat to the server instead of just your team.
Parties are made for this, you should probably speak to Microsoft if you'd like the party chat to hold more people and be more stable. Regardless of that, I think an all talk for private matches only could be good. Perhaps have it in private game settings so that you can set whether it's team talk or all talk. That way no button is required.