You are not doing it correctly, you need to set the Key and Value property while using ToDictionary, just change the ToDictionary call to be like:
var groupedCollection = listOfAnimals.GroupBy(animal => animal.ID)
.ToDictionary( group => group.Key,
group => group.ToList()
);