I guess the problem is in the following line
ddl.SelectedValue = dtTo.Rows[i]["Sort_By"].ToString();
Because the
Sort_By Row gets updated each time you delete a row and for a deleted row it will not exist.
So, if you will assign that value, it will obviously give blank value to the dropdown selection.
You also need to update the value in the viewstate in order to reflect the changes.
When you delete the 4th row, at that time, if you can update the value of the dropdowns in viewstate according to the row number, then it will give you what you want. Try it out...
All the best...
Thanks...