Issue while working with SuperCSV parser

See more:

Java

CSV

I am getting some issue with SuperCSV parser. I have created a web application module to import CSV file to DB. After the import process the user will be displayed the summary like, the total number of records in input file, the total number of good records that got imported to db and the total number of rejected records. The user can download the success and error records to verify.

The issue is some duplicate records are added to success and error files. For example the user gets a message as follows:

Total number of records: 2099
Number of good records: 1694
Number of skipped records: 405
Click to download success file: Import_20121012184828_success.csv
Click to download error file: Import_20121012184828_errors.csv

When we check the db the number of records in the table is exactly '1694' (Same as number of good records as expected.). But in Import_20121012184828_success.csv file there are '1741' records. But in eclipse console it prints same as in db i.e after the last success record there is no duplication. So once exited from the loop and when I call successCsvMapWriter.close(); and errorCsvMapWriter.close(); some where from buffer the records are getting added up. I cant figure this out.

Note: For small number of inputs I din't notice any issues at all.

Following is the code to create the resultant files:

File successFile = new File(csvFileDetails.getSuccessFileUrl());
    File errorFile = new File(csvFileDetails.getErrorFileUrl());
    // Save Result files.
    successFile.createNewFile();
    errorFile.createNewFile();

    String[] header = csvFileDetails.getHeader();
    String[] errorFileHeader = ArrayUtils.add(header,
        CatalogImportConstanst.CSV_ERROR_HEADER);

    // Also tried CsvPreference.STANDARD_PREFERENCE 
    ICsvMapWriter successCsvMapWriter = new CsvMapWriter(new BufferedWriter(
        new FileWriter(successFile)), CsvPreference.EXCEL_PREFERENCE);
    ICsvMapWriter errorCsvMapWriter = new CsvMapWriter(new BufferedWriter(
        new FileWriter(errorFile)), CsvPreference.EXCEL_PREFERENCE);

    successCsvMapWriter.writeHeader(header);
    successCsvMapWriter.flush();
    errorCsvMapWriter.writeHeader(errorFileHeader);
    errorCsvMapWriter.flush();

    int errorCount = 0;
    int successCount = 0;
    for (Map<String, String> csvRecord : csvAsList) {
        if (csvRecord.get(CatalogImportConstanst.CSV_ERROR_HEADER) != null
            && csvRecord.get(CatalogImportConstanst.CSV_ERROR_HEADER).trim().length() > 0) {
            errorCsvMapWriter.write(csvRecord, errorFileHeader);
            errorCsvMapWriter.flush();
            errorCount++;
			System.out.println("Error record: "+ csvRecord);
        } else {
            successCsvMapWriter.write(csvRecord, header);
            successCsvMapWriter.flush();
            successCount++;
			System.out.println("Success record: "+ csvRecord);
        }
      }
    successCsvMapWriter.close();
    errorCsvMapWriter.close();

Thanks in advance.