Memory allocated in your app comes from a pool called the Heap: this does not go down in size when you release memory in your app, and it not automatically released back to the operating system.
Calling Dispose does not necessarily release memory back to the heap: that is only done by the GarbageCollector and that only kicks in when the heap is getting full.
You can't "see" your apps real usage of memory in Task manager: that shows the framework use of memory. To see what your heaps (plural, because there are more than one heap) is doing, you need to use a profiling tool, such as WinDbg, or the VS profiller:
Diagnosing memory issues with the new Memory Usage Tool in Visual Studio – Microsoft DevOps Blog[
^]