One thing that bothered me about some versions of D&D is just that distinction. Druids, for instance, could speak with animals. Could they speak with Owlbears? Displacer Beasts? Stirges? My understanding was that they generally could not... because these were monsters rather than animals.
There are (at least) two things that I can think of that might validly differentiate between monster and animal:
- Intelligence and instinct: Animals are ruled by instinct, are possessed of sub-human intelligence, and are generally incapable of using language. Monsters violate at least one of these. They might have cruel natures that are malevolent rather than instinctual.
- Origin: Animals evolved or were created in a process (possibly divine) that ensured they would be in harmony with the natural world. Monsters violate the natural order. They might be from another plane of existence where the laws of nature are different. They might be the result of magical experimentation. They might be the creation of a malevolent god.