C / C++ / MFC

26-Aug-06 10:24

Why does it have to be so complex? I'm still trying to convert my code but I have ran into a brick wall. I'm reading UTF16 encoded strings from a byte array, most of these strings are Chinese simplified. So for a MBCS I use this

WideCharToMultiByte(936/* GB_2312 */,0,(unsigned short *)data,length/2,text,length+1,NULL,NULL);

The data member there is a pointer within this array. This works, it gives me a string of Multibyte characters. Here is the first character: 0xB0 0xB6. Now, for a unicode build, I should be able to read directly from the array: 0x73ED. According to my charmap utility, this is the correct value for this character. But all I see is "?". Ok, try converting the first multibyte result back to wide:

MultiByteToWideChar(CP_ACP,0,text,length,lpW,length+1);

Exactly the same result: 0x73ED. But I am still not seeing it. Lets try the other, simpler, multi to wide function:

mbstowcs(lpW,text,length);

This time the output is a little different: 0x00B0 0x00E0, and I can see it.
Can anybody explain what is going on here?

Re: I hate Unicode!! [modified]

Michael Dunn26-Aug-06 21:27

Michael Dunn

26-Aug-06 21:27

Read the WideCharToMultiByte() docs closer, you'll see that there's a fall back character, which defaults to '?'. This character is inserted in the MBCS string when there is no character in the given codepage that matches the Unicode code point.
I can't tell from your post if WideCharToMultiByte() is doing the conversion right. What could be happening is the program you're using to inspect the strings is doing its own WideCharToMultiByte() and the fallback behavior is happening there, which leads to '?' in the display.

--Mike--
Visual C++ MVP Cool | :cool:

LINKS~! Ericahist | PimpFish | CP SearchBar v3.0 | C++ Forum FAQ

Waldermort26-Aug-06 22:08

Re: I hate Unicode!! [modified]

26-Aug-06 22:08

I figured out that MultiByteToWideChar() was infact doing the correct convertion. I was seeing the '?' in the debugger aswell as the console. The strange thing is that when I added <tchar.h> to the file, I could see the correct character in the debugger (tchar was already included in the stdafx). The thing that was throwing me off was the inability to see the unicode in the console. I tried variations of wcout and wprintf yet they showed nothing.

I have just stripped out all the tchar routines I added to the code, I am instead going to create overide functions similar to the A's and W's in the stl.

-- modified at 4:09 Sunday 27th August, 2006

fefe.wyx26-Aug-06 22:24

fefe.wyx

26-Aug-06 22:24

If you want use wcout or wprintf in your program to print non-ASCII characters, you must correctly set you program's locale.
Call setlocale(LC_CTYPE, ".936"); at the beginning of your program and try wprintf again.
If you want to use wcout, locale::global must be called instead of setlocale.
Have a look at setlocale[^] and locale::global[^]

-- modified at 4:26 Sunday 27th August, 2006

Waldermort26-Aug-06 22:48

26-Aug-06 22:48

When usiong unicode the locale goes out of the window, 936 (for Chinese simplified) does the job when displaying multibyte characters but I am trying to display Unicode. For example:

wchar_t *tmp = L"你好";
wcout << tmp << endl;

This displays correctly, but it is not unicode (sorry if you can't see the characters). tmp maps to 0x00C4 0x00E3 0x00Ba 0x00E3 (which is the Chinese NiHao. But this is not unicode, it's multibyte. The correct unicode value should be:

wchar_t *tmp = L"\x4f60\x597d";
wcout << tmp << endl;

This does not display anything to the console, and in the debugger all I see is "??", even though the characters have been assigned the correct value.

fefe.wyx26-Aug-06 23:33

fefe.wyx

26-Aug-06 23:33

That is a strange behavior.
I've heard that in some English version of Windows and with some English version of VC, the compiler would convert L"你好"(The Chinese Nihao) to the character squence of 0x00C4 0x00E3 0x00Ba 0x00E3, but I've NEVER met that situation. Chinese characters can be treated very well in wide string constants in all my programs.
In unicode programs, locale MUST be set to display non-ASCII characters in console, while MBCS programs do not need locale.
I just checked one of my programs, and found that locale::global has no effect on wcout, while it works well on all file streams. One must call wcout.imbue explicitly. Try this. By the way, I'm using MS VC2005.

Waldermort27-Aug-06 0:27

Re: Registry Unicode Problem

27-Aug-06 0:27

Thanks for that.

wchar_t *tmp = L"\x4f60\x597d";
wcout.imbue( locale("Chinese_china") );
wcout << tmp << endl << endl;

This now works displaying the correct output.

Registry Unicode Problem

lucki_luke26-Aug-06 9:55

lucki_luke

26-Aug-06 9:55

Hey,

I wrote a program that takes input from a user from a text input and then saves it to the registry.
Now I tried converting it to Unicode but it just doesnt save the values as unicode, e.g. doesnt convert the german öüä to its unicode equivalent. Im pretty new to unicode and so on so i would greatly appreciate any help.

In visual studio i selected as character set unicode.

Thanks,

lucki_luke

Mike Dimmick26-Aug-06 11:09

Mike Dimmick

26-Aug-06 11:09

If you have 'Use Unicode Character Set' selected in Project Properties, the registry APIs will be in Unicode mode (technically, RegSetValueEx is a macro which maps to RegSetValueExW) and expect to be passed a UTF-16 string (an array of WCHAR, an LPWSTR).

If this is defined, however, all GUI controls should be giving you text in Unicode format anyway (again, strictly it's that the GetWindowText macro maps to GetWindowsTextW), so no conversions should be required. A window can be in 'ANSI' (byte-oriented character set) mode or in 'Unicode' mode; which it is depends on the variant of RegisterClassEx called. If this mismatches the rest of your program, some unexpected conversions can occur. I'm not quite sure how the standard predefined controls get set up.

Advanced: this setting in VS determines whether UNICODE (and _UNICODE), MBCS (and _MBCS), or neither, are defined on the compiler's command line. The TCHAR.H header handles the underscore-prefixed variants for the C runtime string functions which include a 't' in their names (which are Microsoft extensions), while the Windows headers handle the non-prefixed variants for Windows APIs.

Stability. What an interesting concept. -- Chris Maunder

reading values within std::list types

jon-8026-Aug-06 8:34

jon-80

26-Aug-06 8:34

I am trying to retrieve values from within a list of strings

SentenceList.h
----------------------------------

#pragma once
..
#include <list>

...
using std::list;
...

class CSentenceList
{
....
string display(unsigned int iLineNumber = 1);
list<string> Sentences; //list of sentences
// Note: MOVE TO PRIVATE AFTER IMPLEMENTING display

....
};

The list is being created and I would like to create a method 'display' that returns a string according to the line number. The method should loop through the list and return a string according to the value of iLineNumber.

I have tried using Sentences.front, but I could not picture the correct return value to be read to return the string value.

How do I read values from Sentences?

Jon

Re: reading values within std::list types

Waldermort26-Aug-06 9:23

Re: reading values within std::list types

26-Aug-06 9:23

there you have create an instance of string with size 1. That is the equivlent of char str[1]; So where are the strings that you want to return? For the list object, just treat it like an array. If you want the 10th string just call Sentences[10].

jon-8026-Aug-06 10:40

jon-80

26-Aug-06 10:40

When I tried the following, the compiler didn't like it...

string CSentenceList::display(unsigned int iLineNumber)
{
return Sentences[iLineNumber];
}

Error:
(51): error C2676: binary '[' : 'std::list<_Ty>' does not define this operator or a conversion to a type acceptable to the predefined operator
with
[
_Ty=std::string
]

Jon

Re: reading values within std::list types

Waldermort26-Aug-06 10:53

Re: reading values within std::list types [modified]

26-Aug-06 10:53

Oh, sorry my fault, I was confusing list with vector. You are using list, but want to provide a line number. You do realise a list is sorted. Everything you add to it will be rearranged. If you want to add sentences one ata time and keep them in order you should really use the vector class. If you really want to use the list then you will have to go through a loop testing the value of each sentence, but there is no way for you to know which line goes with which sentence.

Mike Danberg27-Aug-06 17:53

Mike Danberg

27-Aug-06 17:53

You need to use an iterator to traverse the elements in the list.

Try something like:

list<string>::iterator start = Sentences.begin()

while ( start != Sentences.end() )
{
if(sentence matches) // probably want to try something like string == (*start)
{
//do something
break; // if you hit a match break
}

start++;
}

Hope that helps. There are plenty of articles on using iterators as well.

-- modified at 23:54 Sunday 27th August, 2006

Operating system info

Kiran Pinjala26-Aug-06 8:29

Kiran Pinjala

26-Aug-06 8:29

Can we get the info about which operating system is currently present in the system for example is it windows98 or xp or win2000.

Thank you

KIRAN PINJARLA

Re: Operating system info

Mike Dimmick26-Aug-06 9:47

Mike Dimmick

26-Aug-06 9:47

To find out the operating system version, use GetVersionEx. Windows 98 is major version 4, minor version 10. Windows 2000 is major 5, minor 0 and Windows XP major 5, minor 1. Windows Server 2003 is major 5, minor 2 and Windows Vista will be major 6, minor 0. Windows XP x64 may return major 5, minor 2 since it is based on the Windows Server 2003 SP1 kernel.

Be careful when testing version numbers. In the past people have frequently got it wrong, causing compatibility problems when new operating systems come out. If targetting Windows 2000 and higher only, it is recommended to use the VerifyVersionInfo API to check the OS version is what you require.

Generally it's recommended not to check the OS version at all, but for any advanced, optional features using functions added in later operating systems, to check whether the function is available using LoadLibrary and GetProcAddress. This covers any cases where the API may be back-ported to an older OS.

If you're trying to test the OS version to determine whether to install, and you're using Windows Installer, you should use a Launch Condition to test the Version9X and/or VersionNT properties. Version9X is only defined on Windows 95, 98 or ME: its value is 400 on Windows 95, 410 on Windows 98 and 490 on Windows ME. VersionNT is defined on NT 4.0, Windows 2000, Windows XP, Windows Server 2003 and later, and has the values, respectively: 400, 500, 501, 502, and 600 for Windows Vista.

At this stage I would consider dropping support for Windows 98. Recent statistics gathered from processing web server logs (e.g. OneStat[^]) suggest that Windows 98 is now used by less than 3% of users of those web sites, with Windows ME used by only a little over 1%. I would expect these numbers to continue to fall.

Stability. What an interesting concept. -- Chris Maunder

converting str::string to const char*

jon-8026-Aug-06 8:03

jon-80

26-Aug-06 8:03

The following code should read a text file and update Sentences (defined as std::list<string> Sentences).

However, fstream does not seem to like 'string' types. Is it possible to retain string types and convert it to char?

bool CSentenceList::readFile(const string strFileToRead)
{
string strLine;

fstream fileToRead((char*) strFileToRead,ios::in);

if (ios::good != false)
{while (!fileToRead.eof() && Size < MAX_LINES)
// Read line of text from file and update Sentences.strSentence.
{ fileToRead.getline(strLine, MAX_SENTENCE_LENGTH);
Sentences.push_back(strLine);
Size++;
}
return true;
}
else
{ throw "File cannot be read!";
return false;
}

fileToRead.close();
}

Errors:

SentenceList.cpp(23): error C2440: 'type cast' : cannot convert from 'const std::string' to 'char *'

SentenceList.cpp(28): error C2664: 'std::basic_istream<_Elem,_Traits>::_Myt &std::basic_istream<_Elem,_Traits>::getline(_Elem *,std::streamsize)' : cannot convert parameter 1 from 'std::string' to 'char *'
with
[
_Elem=char,
_Traits=std::char_traits<char>
]

Are there other objects I can use so I can read text files from 'string' types, since they are less prehistoric?

Jon

Pierre Leclercq26-Aug-06 8:34

Pierre Leclercq

26-Aug-06 8:34

First, about std::string, if you want a char*, you should use the c_str() function,
other methods are not safe.
But this is not relevant to your problem. You should take a look at streams.
You'd create an istream, from your file, and then do things like

istream >> strline;

Pierre Leclercq26-Aug-06 9:29

Pierre Leclercq

26-Aug-06 9:29

A precision to my former post.

the istream I was refering to was a stringstream, as you mentionned you were looking
for less prehistoric methods, I think streams provide nice high level objects, but you'd
need to do some readings on the topic.

For a quick fix to your code, you could do something like:

string strLine(MAX_SENTENCE_LENGTH, ' ');

This creates a string with the right size. To read directly into the string buffer,
you can acces it with &(*(strLine.begin())), this gives you a pointer to the internal
buffer.

Waldermort26-Aug-06 8:59

26-Aug-06 8:59

OK, where do we start? The constructor for the fstream class does not take a string class, it needs a const char*, easy to fix, call the c_str() member of strFileToRead. That should get your file open. Next you are tring to read the line directly into a string class, this is not possible with the istream class. You must allocate a buffer of type char and read into that, then assign that buffer to the string class.