C++: New Text Streams

• Download source code - 13 KB

Table of Contents

• Introduction
• Escape Sequences
• Overloaded Operators
• Parsing INI file
• Points of Interest
• Related Articles
• History

Introduction

The in-house utility library used in my workplace includes an elaborate and flexible set of classes for parsing CSV. But it came up short when I need to write to CSV: to do that, I have to use STL file output stream! In fact, STL stream and .NET stream classes do not have read/write symmetry as well: They are easy to write formatted output to file but leave the file parsing up to the user. In retrospect, C printf and scanf does have read/write symmetry.

// Writing
std::ostringstream ostm
...
std::string name = "Steven";
int age = 35;
ostm << "Name:" << name << ", Age:" << age;

// Reading
std::istringstream istm;
istm.str(ostm.str());
...
std::string name = "";
int age = -1;
// No, you cannot do this!
istm >> "Name:" >> name >> ", Age:" >> age;

Sad to admit my Unicode file library article also does not have this symmetry. Let me first show you what I propose to do in greatly simplified pseudo-code. See the symmetry? Bulk of the verifying and parsing work are done in match function.

// Writing
ostm << name << age;
ostm.match("Name:{0}, Age:{1}");
ostm.write_line();

// Reading
istm.read_line();
istm.match("Name:{0}, Age:{1}");
istm >> name >> age;

Let us now see the real code in action!

struct Product
{
    Product() : name(""), qty(0), price(0.0f) {}
    Product(std::string name_, int qty_, float price_) 
        : name(name_), qty(qty_), price(price_) {}
    std::string name;
    int qty;
    float price;
};

void TestNormal()
{
    try
    {
        new_text::ofstream os("products.txt", std::ios_base::out);
        if(os.is_open())
        {
            Product product("Shampoo", 200, 15.0f);
            os << product.name << product.qty << product.price;
            os.match("{0},{1},{2}");
            os.write_line();
            Product product1("Soap", 300, 25.0f);
            os << product1.name << product1.qty << product1.price;
            os.match("{0},{1},{2}");
            os.write_line();
        }
        os.flush();
        os.close();

        new_text::ifstream is("products.txt", std::ios_base::in);
        if(is.is_open())
        {
            Product temp;
            while(is.read_line())
            {
                if(is.match("{0},{1},{2}"))
                {
                    is >> temp.name >> temp.qty >> temp.price;
                    // display the read items
                    std::cout << temp.name << "," 
                        << temp.qty << "," << temp.price 
                        << std::endl;
                }
            }
        }
    }
    catch(std::exception& e)
    {
        std::cout << "Exception thrown:" << e.what() << std::endl;
    }
}

Escape Sequences

There are some escape sequence you can use. The first one is hexadecimal conversion:x

os.match("{0},{1:x},{2}");     // converts to hexadecimal
is.match("{0},{1:x},{2}");     // converts from hexadecimal

If you like leading zero for the month and day of your date, you can use :0

os.match("{0},{1:02},{2:02}"); // put leading zero if less than 10
is.match("{0},{1:02},{2:02}"); // trim leading zero before convert to number

If you like specific width for your text, you can use :<width> and :t if you like to trim prior to reading.

os.match("{0:20}{1}");         // pad whitespace if less than 20 chars
is.match("{0:t}{1}");          // trim whitespace

Use \\ to escape { or }.

os.match("\\{{0}\\}");         // eg, "{Shampoo}" is saved
is.match("\\{{0}\\}");         // "Shampoo" is extracted from "{Shampoo}"

Call set_precision to set precision for floating point numbers.

Overloaded Operators

You can overload the << and >> for your defined types. First, I show you how to overload my library's << >> operators, then STL stream << >> ones. Both codes are similar.

Overloaded Stream Operators

Say we have a Date structure, this is how to overload the operators. Yes, we make use of temporary local new_text::ofstream and new_text::ifstream to help us as they can be used like std::stringstream without opening a file!

struct Date
{
    Date() : year(0), month(0), day(0) {}
    Date(int year_, short month_, short day_) 
        : year(year_), month(month_), day(day_) {}
    int year;
    short month;
    short day;
};

new_text::ofstream& operator << (new_text::ofstream& ostm, 
    const Date& date)
{
    new_text::ofstream ofs_temp;
    ofs_temp << date.year << date.month << date.day;
    ofs_temp.match("{0}-{1:02}-{2:02}");

    ostm.push_back(ofs_temp.str());

    return ostm;
}

new_text::ifstream& operator >> (new_text::ifstream& istm, 
    Date& date)
{
    new_text::ifstream ifs_temp;
    ifs_temp.set_string(istm.pop());
    ifs_temp.match("{0}-{1:02}-{2:02}");
    ifs_temp >> date.year >> date.month >> date.day;

    return istm;
}

Overloaded STL Stream Operators

Previously, we see overloaded operators for the library stream. But we cannot use those for std::cout! We will see next how to overload STL stream operators. Note: We can do that because my library uses STL stream underneath to do the work.

std::ostream& operator << (std::ostream& ostm, const Date& date)
{
    new_text::ofstream ofs_temp;
    ofs_temp << date.year << date.month << date.day;
    ofs_temp.match("{0}-{1:02}-{2:02}");

    ostm << ofs_temp.str();

    return ostm;
}

std::istream& operator >> (std::istream& istm, Date& date)
{
    new_text::ifstream ifs_temp;
    std::string str;
    istm >> str;
    ifs_temp.str(str);
    ifs_temp.match("{0}-{1:02}-{2:02}");
    ifs_temp >> date.year >> date.month >> date.day;

    return istm;
}

This is the same code on how to use 2 types of overloaded operators.

void TestOverloaded()
{
    try
    {
        new_text::ofstream os("products.txt", std::ios_base::out);
        if(os.is_open())
        {
            os.set_precision(17);
            Product product("Shampoo", 200, 15.83f);
            Date date(2014,9,5);
            os << product.name << date << product.qty << product.price;
            os.match("{0},{1},{2},{3}");
            os.write_line();
        }
        os.flush();
        os.close();

        new_text::ifstream is("products.txt", std::ios_base::in);
        if(is.is_open())
        {
            Product prod;
            Date date1;
            while(is.read_line())
            {
                if(is.match("{0},{1},{2},{3}"))
                {
                    is >> prod.name >> date1 >> prod.qty >> prod.price;
                    // display the read items
                    std::cout << prod.name << "," << date1 << "," 
                              << prod.qty << "," << prod.price 
                        << std::endl;
                }
            }
        }
    }
    catch(std::exception& e)
    {
        std::cout << "Exception thrown:" << e.what() << std::endl;
    }
}

Parsing INI file

Next, we see how to parse a date and RGB color from an INI file. The second parameter of match should be set to false when we only wish to verify string conformed to the specified format and not waste computation to process it. The default value is true.

void TestReadIni()
{
    try
    {
        new_text::ifstream is("settings.ini", std::ios_base::in);
        if(is.is_open())
        {
            while(is.read_line())
            {
                if(is.match("#{0:t}", false))    // Parse comment
                {
                    is.match("#{0:t}");
                    std::string comment="";
                    is >> comment;
                    std::cout << "Comment:" << comment << std::endl;
                }
                else if(is.match("[{0}]", false)) // Parse section
                {
                    is.match("[{0}]");
                    std::string section="";
                    is >> section;
                    std::cout << "Section:" << section << std::endl;
                }
                else if(is.match("{0:t}={1:t}", false))                  // Parse name/value
                {
                    if(is.match("{0:t}={1:t},{2:t},{3:t}", false))       // Parse RGB
                    {
                        is.match("{0:t}={1:t},{2:t},{3:t}");
                        std::string name="";
                        int red=0, green=0, blue=0;
                        is >> name >> red >> green >> blue;
                        std::cout << name << ":" << "r:" << red << ", g:" 
                            << green << ", b:" << blue << std::endl;
                    }
                    else if(is.match("{0:t}={1:04}-{2:02}-{3:02}", false)) // Parse date 
                                                                           // in YYYY-MM-DD
                    {
                        is.match("{0:t}={1:04}-{2:02}-{3:02}");
                        std::string name="";
                        int year=0, month=0, day=0;
                        is >> name >> year >> month >> day;
                        std::cout << name << ":" << year << "-" << month << "-" 
                                          << day << std::endl;
                    }
                    else // Parse normal name/value
                    {
                        is.match("{0:t}={1:t}");
                        std::string name="", value="";
                        is >> name >> value;
                        std::cout << name << ":" << value << std::endl;
                    }
                }
            }
        }
    }
    catch(std::exception& e)
    {
        std::cout << "Exception thrown:" << e.what() << std::endl;
    }
}

The source code is not shown to the reader in the article because it is merely the parsing code which is of no interest. Those who are keen to examine the source code, are free to download from the above link.

Points of Interest

Reader may wonder what my reason is for writing these file stream articles because I did not hide my disdain for STL stream in my Unicode file library article. The reason for the change of mind is I have been pondering to submit my file library to Boost in the future. One of the Boost requirements is that the library has to support the last 3 version of every C++ compiler. The mere thought of having to back-port variadic templates to pre-C++11 compilers gives me shudders. Then the idea to write new file streams came to me as they are C++98/03, so as to make porting relatively easy. These stream classes will be ported to my Unicode library.

To use the library, just include the header: NewTextStream.h. For those users who prefer to use Boost lexical_cast and trim, just define or uncomment the below macros in the header. The code is tested in VS2008 and GCC 4.4. Thank you for reading!

//#define USE_BOOST_LEXICAL_CAST
//#define USE_BOOST_TRIM

Related Articles

• C++: Minimalistic CSV Streams
• C++14: CSV Stream based on C File API
• C++: Simplistic Binary Streams
• C++: New Text Streams

The article source code is hosted at Github.

History

• 12^th April, 2016: Initial version
• 3^rd January, 2018: Amended ltrim and rtrim to use lambda