FileHelpers Home Page
FileHelpers v 3.1
The FileHelpers are an easy to use library to import/export data from fixed length or delimited flat files (like the CSV). FileHelpers also has support to import/export data from different data storages (Excel, Acces, SqlServer).
The library has a set of converters for the basic types and can be easily extended to provide custom converters.
The main idea is pretty simple:
You can strong type your flat file (fixed or delimited) simply describing a class that maps to each record of the file and work with your file like a strong typed .NET array.
Directly between the files and the .NET source code:
It also can be used as an intermediate between files and MS Access or MS SqlServer tables:
The recommend method to install the library is use the NuGet Package
www.nuget.org/packages/FileHelpers/
Via de NuGet Console: Install-Package FileHelpers
Or you can use the Visual Studio NuGet Explorer to search for FileHelpers
To start using the FileHelpers Library you only need to add a reference in your project to the file: FileHelpers.dll. You can find it in the Release directory of the distribution. Tip: remember to leave the FileHelpers.xml file to get Intellisense support.
Next you need to define a class that maps to the record in the source/detination file. For this example we use a file with this format delimited by a |:
10248|VINET|04071996|32.38
10249|TOMSP|05071996|11.61
10250|HANAR|08071996|65.83
10251|VICTE|08071996|41.34
...............
The class that we refer can be like this:
[DelimitedRecord("|")]
public class Orders
{
public int OrderID;
public string CustomerID;
[FieldConverter(ConverterKind.Date, "ddMMyyyy")]
public DateTime OrderDate;
public decimal Freight;
}
Later you must instantiate a FileHelperEngine and Read/Write files:
var engine = new FileHelperEngine<Orders>();
var res = engine.ReadFile("input.txt");
engine.WriteFile("output.txt", res);
Finally you can use the res array to access each item in the file, for example:
foreach (Orders order in res)
{
Console.WriteLine("Order Info:");
Console.WriteLine(order.CustomerID + " - " +
order.OrderDate.ToString("dd/MM/yy"));
}
Working with FixedLength files is exactly the same but you need to use the [FixedLengthRecord] and [FieldFixedLength] attributes.
We have ton of examples in our web site to learn to use the library from every angle
In almost every project there is a need to read/write data from/to a flat file of a specified format. Writing the code to process these files is not hard, you can use String.Split
or String.SubString
, but if you do that the code sometimes becomes hard to read and change. If you want to add support later for quoted string, multiline, convert types, etc. these things tend to get complicated.
So with the FileHelpers you don't need to worry about changes in the file structure because you can change the Type, Converters, Triming, Length in seconds.
Another place where you can find the FileHelpers useful is for log parsing, data warehouse and OLAP applications, communication between systems, file format transformations (for example from a fixed length to an CSV file), load test data into your NUnit tests and a lot more!!!
This library aims to provide an easy and reliable way to accomplish all these tasks.
Our Roslyn Analyzer helps you to use the library in the right way.
The current Analyzer implements:
- Recomends to use the generic version when you use an engine with a typeof() in the constructor
- Suggest to use [FieldHidden] instead of [FieldIgnored] or [FieldNotInFile]
- Use FileHelpers.Dynamic instead of FileHelpers.RunTime namespace
- Use IComparable instead of the obsolete IComparableRecord
next version will implement
- Check that record class contains valid record attributes
- Check that record class contains any field
- Check other obsolete methods
- Convert engine.ReadFile in async version with foreach
You can join the development of the analyzer, join us in the Gitter chat:
https://gitter.im/MarcosMeli/FileHelpers
Since version 1.3.5 the library has had a Record Class Wizard to generate the record class. The Record Class Wizard allows you to generate your record mapping class, save the definition, generate code based on snippets (Read File, Read with Generics, ReadAsync and so on).
Click to see how it works
Easy to use
The FileHelpers library is straightforward to learn and use.
Check the Examples
Auto Converters
The library has a set of converters for common types and can be easy extended to provide custom converters.
Check the Example
Event Support
The engines of the library contain events to make it easier to extend the behavior of the library
Check the
INotifyRead and
INotifyWrite Examples
High Performance
The library doesn't use reflection to get or set the field values, it uses dynamic code generation
Wide framework support
You can use the library in
.NET 2.0, 3.0, 4.0, 4.5, 4.6 and Mono!!
Multiple Error Modes
You can configure the library to use different ErrorModes.
Check the example
BigFileSorter
You can use the library to sort files with millon of records and MBs
Check the Example
File Transform Engine
To convert files in one format to another (for example a file with CSV to a fixed length record format)
Check the Example
FileDiffEngine
Provides the ability to compare files with the same record layout
Check the Example
MasterDetailEngine
You can process hierarchical data with this engine
Check the Example
Record by Record Mode
You can use the library to read line by line and not the whole file.
Check the Example
Format Autodetection
The library has an smart feature that based in some sample files can deduce the Record Class
Check the Example
Progress Notification
Here is a screenshot of the progress notification of the demo app.
Since versión 1.4.0 you can get notified of the progress of each operation of the library to show feedback to the user or whatever you want.
Detailed Diagram
Testing
The library contains more than 650 NUnit tests to ensure correctness:
List of Converters Arguments
Here is a list of the parameters that you can give to the default converters:
ConverterKind.Date
Arg1: A string with the DateTime format that the engine passes to the DateTime.Parse function.
Arg2: A string with the encoding used for string convertions
Examples:
[FieldConverter(ConverterKind.Date)]
public DateTime ShippedDate;
[FieldConverter(ConverterKind.Date, "d-M-yyyy" )]
public DateTime ShippedDate;
[FieldConverter(ConverterKind.Date, "M-d-yyyy" )]
public DateTime ShippedDate;
[FieldConverter(ConverterKind.Date, "d/M/yyyy" )]
public DateTime ShippedDate;
[FieldConverter(ConverterKind.Date, "ddMMyyyy" )]
public DateTime ShippedDate;
[FieldConverter(ConverterKind.Date, "dd/MMMM/yyyy", "en" )]
public DateTime ShippedDate;
[FieldConverter(ConverterKind.Date, "dd/MMMM/yyyy", "es" )]
public DateTime ShippedDate;
You can check all the supported format strings check the MSDN docs for DateTime.ParseExact
and here all support cultures
ConverterKind.Double, ConverterKind.Single and ConverterKind.Decimal
Arg1: A string with the character to be used as DecimalSeparator. Valid Values: "." or ",". By default: "."
Examples:
[FieldConverter(ConverterKind.Double)]
public double Freight;
[FieldConverter(ConverterKind.Double, ".")]
public double Freight;
[FieldConverter(ConverterKind.Double, ",")]
public double Freight;
Integer Converters
ConverterKind.Int16, ConverterKind.Int32, ConverterKind.Int64, ConverterKind.Byte,
ConverterKind.UInt16, ConverterKind.UInt32, ConverterKind.UInt64, and ConverterKind.SByte
Arg1: A string with the character to be used as DecimalSeparator. Valid Values: "." or ",". By default: "."
WARNING: The library requires a decimal separator here and internally creates the group separator with the counterpart (for example if you provide "." it uses ",")
Examples:
[FieldConverter(ConverterKind.Int32)]
public Int32 Amount;
[FieldConverter(ConverterKind.Int32, ",")]
public Int32 Amount;
ConverterKind.Boolean
Arg1: A string that represents the True value
Arg2: A string that represents the False value
By default, this converter takes the strings "True" (case insensitive) and "1" as true. The values "True" and "False" are returned when converting the field to string.
Examples:
[FieldConverter(ConverterKind.Boolean)]
public bool Shipped;
[FieldConverter(ConverterKind.Boolean, "Verdad", "MenTiRa")]
public bool Shipped;
[FieldConverter(ConverterKind.Boolean, "1", "0")]
public bool Shipped;
[FieldConverter(ConverterKind.Boolean, "X", "-")]
public bool Shipped;
Version 3.1, July 2015
Hi everyone again !!
The FileHelpers has reached a big Milestone today we are releasing the 3.1 version!
Now with full support for .NET 4.0, 4.5 and Mono and with a ton of news and enhancements.
After some time off for different reasons we are now trying to update the library in a more regular fashion. We redesigned the HomeSite with a Material Design Template from GeeksLabs
Main changes
- Docs rewritten thanks to the work of Ken Foskey and Matthias Cavigelli
- NEW: Full .NET 2.0, 4.0 & 4.5 support.
- NEW: Added support for MONO !!
- NEW: FieldOrder: a new attribute to override the default order of fields.
- NEW: Partial support to use Autoproperties
- NEW: Async Methods returns IDisposable to allow using() statements that autoclose files on ends or exception.
- NEW: Events for the Async engines.
- NEW: The events args now have TotalBytes and CurrentBytes to allow you show progress while reading or writing.
- NEW: FieldIndexers you can now get the values in the AsyncEngine like in the DataReader of ADO.NET =) You can access them via index or fieldName (at the moment case sensitive). This also completes the RunTime records support
- NEW: SmartFormatDetector: A class designed to discover the format of delimited and flat files based on sample files. It is integrated with the wizzard
- NEW: Better Delimited validations to make sure that the file contains the right number of fields
- NEW: FieldValueDiscarted: a new attribute to allow the value of a particular field to be read but not stored in the record class (this was a very requested feature)
- NEW: BigFileSorter: if you need to sort big files, you can use this feature that implements External Sorting
- MOD: The error messages have been made more meaningful. In all exceptions you get: FieldName, ColumnNumber, LineNumber etc.
- NEW: FieldNotEmpty: Indicates whether the target field needs to be populated with a non-empty value.
Breaking Changes
- Removed support for .NET Compact Framework
- [FieldIgnored] => [FieldHidden]
- IComparableRecord<T> => IComparable<T>
Minor Changes
- Better caching of internal classes
- The core code of the library was refactored to make easier to extend it.
- Better performance.
- ADD: UpdateLinks properties, allow to specify how the library must handle Workbook links (thanks to Stefan Schälle)
- A ton of new examples, autogenerated from FileHelpers.Examples project
- FIX: A little problem with the quoted fields in the wizard
- More documentation has been added to the Library source code. It is now even easier to get involved in development!
-
Licence (MIT)
FileHelpers Library is @Copyright 2005-2015 to Marcos Meli but it's source code and the binaries are free for commercial and non commercial use.
Vote for this article if you like the library.