Mishnah/Seder Zeraim/Tractate Berakhot/Chapter 7/1
Three who ate together are obliged to say zimmun [formally say the grace together.]
One who ate produce which may not have been properly tithed;
or tithed produce from which [only] teruma was separated;
or second tithe or consecrated produce that was [not fully] redeemed;
or the attendant who ate [an amount] like an olive;
or a Cuthean;
--Zimmun is said for them.
But one who ate untithed produce;
or tithed produce from which teruma was not separated;
or second tithe or consecrated produce that was not redeemed;
or the attendant who ate [an amount] less than an olive;
or a non-Jew;
--Zimmun is not said for them.
In general, the mishnah is contrasting two sets of situations:
In the first group, there are problems or potential problems with the nature of the food or the status of the person eating it; in those cases, the eater of the food and the person of that status is nonetheless included in Zimmun.
In contrast, the second group describes foods that were clearly forbidden, and persons who cannot qualify for a Zimmun; eaters of these foods and persons of this status are not included in Zimmun.