I think one easy way is to do this in two phases. The first query will convert the data from XML to variable data types and the second does the grouping based on the first query.
So consider something like the following
var intermediate = from item in docm.Descendants("Positions")
select new {
Mod = (string)item.Element("Mod1"),
Quantity = (decimal?)item.Element("Quantity"),
AmountClosingPrice = (decimal?)item.Element("AmountClosingPrice")
};
var result = from item in intermediate
group item by item.Mod into groupedItem
select new {
Mod = groupedItem.Key,
mktval = groupedItem.Sum(item => item.Quantity * item.AmountClosingPrice)
};