Writing UTF8 from a windows C++ stream

Question

0.00/5 (No votes)

See more:

In Visual Studio 2022 the following C++ code gets a C++ stream to write UTF8 so that Asian characters written to the file will properly interpreted on all types of computers:

...

wofstream outFile;

outFile.open(logPath, wfstream::out | wfstream::app);

const std::locale utf8_locale = std::locale(std::locale(), new std::codecvt_utf8<wchar_t>());

outFile.imbue(utf8_locale);

...

LPCTSTR headings;

...

outFile << headings << endl;

Unfortunately

std::codecvt_utf8<wchar_t>()

has been deprecated. What can be used instead?

What I have tried:

Some commentators suggest using the "utf8" include to solve these problems, but I can't find it.
Changing to use the "C":

FILE *fp = fopen("newfile.txt", "rt+, ccs=UTF-8");

is not practical because the output writing is using "C++" streams.

Posted 13-Mar-23 0:05am

eddie.breeveld

Updated 21-Mar-23 8:28am

Add a Solution

2 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Rick York · Answer 1 · 2023-03-13T05:17:00

Solution 1

It appears that the current way to do it is to use std::codecvt with additional arguments. See this : std::codecvt - cppreference.com[^]

Posted 13-Mar-23 5:17am

Rick York

Comments

eddie.breeveld 15-Mar-23 11:12am

Thank you Rick. Just tried

const std::locale utf8_locale = std::locale(std::locale(), new std::codecvt<char16_t, char8_t, mbstate_t>());

but that doesn't work. It compiles OK, but Asian characters and all characters following them in the string being written are ignored, as if a NULL character has been found

longjmp · Answer 2 · 2023-03-21T08:28:00

I don't think you have to "convert a wide string (in this case a utf16 string) to a utf8 string", instead you can save the utf16 string directly to a file - whether using C++ streams or the C file IO API, you just need Just save in binary mode instead of text mode.

If you want to convert without using std, do it with CW2A, which is short compared to std's long and ugly function names:

#include <iostream>
#include <fstream>
using namespace std;
void CTestDlg::OnBnClickedOk()
{
    LPCTSTR lpsz = L"한국어 some Unicode string にちほん";
    CStringA s(CW2A(lpsz, CP_UTF8)); // s is a utf8 string

    ofstream fu8(".\\bin_u8.txt", ios_base::binary);
    fu8.write(s, s.GetLength() /* * sizeof(CHAR) */);
}