How to fix max() error "sequence contains no elements"

See more:

C#

dictionary

I am building an Evil Hangman game (like Guess the Word, but very hard to win by the user, because the computer cheats by keeping its secret word options open)

I have a Dictionary of strings and string Lists. The strings are the keys and they represent patterns of word families (i.e ----, e---, -e--). The lists of strings are the values, and each list contains words that correspond to the pattern (---- includes all words that don't have the user input letter, e--- includes all words that start with e and only have 1 'e' throughout the word and so on)

The Dictionary gets updated with each letter guess from the user. Basically, new families get created based on each new guess, and the computer selects the largest family as its current list of "secret words". I am using Max() to get the int value of the biggest family, and then compare that value to the Count of each family in order to select just largest.

int max_value = families.Max(family => family.Value.Count());

While testing and guessing more and more, at some point the code breaks and I get the error

"Sequence contains no elements"

What I have tried:

I looked up the error and found it was caused by the Max() function, and found solutions like DefaultIfEmpty(), but I'm not sure how I could apply it in my situation.
I am also confused, as the code breaks when, for example, there are still 6 lists of families in the dictionary, and hence should not be empty.

Any suggestions are welcome, both including Max() or something I could replace Max with.