For anyone who is intersted I found out the the problem seems to be with having the toolbar and the the Provider, remove one and the problem is fine, add one and the issue is there. I don't know why.
However I found a workaround for the issue. You can get around it by doing the following.
var portableSettingsProvider = new PortableSettingsProvider();
Properties.Settings.Default.Providers.Add(portableSettingsProvider);
foreach (SettingsProperty property in Properties.Settings.Default.Properties)
{
property.Provider = portableSettingsProvider;
}
In your Main() before you application is ran.
If anyone know WHY those two have a problem with each other I would love to know.