How do I store a single filename between application runs

First of all, please see my comment to Solution 2.

Main idea is: you should keep your intermediate and configuration data in a "special folder", one of those set up per user of for "all users". Please see my past answer:
How to find my programs directory[^].

In principle, the settings file is one of the way to use this approach, so you can use the advice of Solution 2.

However, I seriously consider setting as morally obsolete and never use them. I use a bit different approach which appeared after the setting was introduced (with WCF) and provides much better maintainability. This is just serialization based on Data Contract: https://msdn.microsoft.com/en-us/library/ms733127%28v=vs.110%29.aspx[^].

This approach pe se has nothing to do with WCF, is only used by WCF and is the most convenient for all kinds of settings, because everything is automated much better as with settings. Essentially, you create any object graph (making it a contract only takes adding some .NET attributes) of any arbitrary data, without any limitations (no need to use any certain base class or implement any interfaces, access control keywords also don't matter) and just tell the system to store it all in any file or a stream. Later on, this data will be restored to memory the way it was before. Your data does not even be a tree, it could be a graph of objects with any arbitrary relationships. Of course, it could be just one data type (class, structure), and your file name can be a property or a field. That's all.

—SA

>> could I do this in the user settings part of the appication properties (sic)

I don't see why not - I'm sure people will have all sorts of 'religious' "you must/cant/shouldn't rules", but for a low volume (1 update per app run) sort of thing I cant see why you'd make it harder than it has to be

(if you said you needed to update a 'stored value' every second, then the app.config isn't a good choice)

'g'