non-static const inside functions in C++

First of all, your code will not compile: you should change first constant declaration to initialize it. Consider this:

class MyClass {
    int Expression(int value) { return value * 2; }
    void ConstantUser(int parameter) {
        const int immediate = 3; //as your j
        const int calculated = Expression(2); //as your k
        //...
    }
};

What you show is the local non-static constant. It is either stored on you local stack frame or as immediate constant (that is, in code), it depends on both context and implementation. For example, in the code above, the constant immediate can be done immediate but can be placed on stack if it used use elsewhere below and the optimizer "decides" to do so. As to the constant calculated, it can only be on stack if its value cannot be statically determined (during compile-time, of course). In this case the modifier const only serves a role of syntax safeguard: the object cannot be assigned to something else after initialization. In C++, initialization is conceptually different from assignment.

Note: your idea of code, data or stack segments are obsolete. Even though the segments are technically present in the CPUs of Intel architecture, in modern systems with virtual memory the user mode work with "flat" memory model based on paging. Nevertheless, the code is protected from mixing up the memory addresses allocated for different roles, but the mechanism of it is quite different; it's called Data Execution Prevention, see: http://en.wikipedia.org/wiki/Data_Execution_Prevention[^].

—SA