C# 以最快、高效、优雅的方式将字符串解析为动态类型？
我正在寻找最快的（通用方法）将字符串转换为各种数据类型 我正在解析由某物生成的大型文本数据文件（文件大小为几兆字节）。这个特殊函数读取文本文件中的行，根据定界符将每行解析为列，并将解析后的值放入.NET数据表中。这将稍后插入到数据库中。到目前为止，我的瓶颈是字符串转换（Convert和TypeConverter） 我必须采用动态方式（即远离“Convert.ToInt32”等），因为我永远不知道文件中将包含哪些类型。该类型由运行时的早期配置确定 到目前为止，我已经尝试了以下两种方法，它们都需要几分钟来解析一个文件。注意 如果我注释掉这一行，它只需要几百毫秒C# 以最快、高效、优雅的方式将字符串解析为动态类型？,c#,string,performance,typeconverter,C#,String,Performance,Typeconverter,我正在寻找最快的（通用方法）将字符串转换为各种数据类型 我正在解析由某物生成的大型文本数据文件（文件大小为几兆字节）。这个特殊函数读取文本文件中的行，根据定界符将每行解析为列，并将解析后的值放入.NET数据表中。这将稍后插入到数据库中。到目前为止，我的瓶颈是字符串转换（Convert和TypeConverter） 我必须采用动态方式（即远离“Convert.ToInt32”等），因为我永远不知道文件中将包含哪些类型。该类型由运行时的早期配置确定 到目前为止，我已经尝试了以下两种方法，它们都需要几
row[i] = Convert.ChangeType(columnString, dataType);
row[i] = Convert.ChangeType(columnString, dataType);
- 方法调用：17720
- 参数化线程开始：13836
- ThreadPool.QueueUserWorkItem:14075
- Task.Factory.StartNew:16798
- Func BeginInvoke EndInvoke:16733
- 参数化线程开始：12591
- ThreadPool.QueueUserWorkItem:13832
- Task.Factory.StartNew:15877
- Func BeginInvoke EndInvoke:16395
Dictionary<Type, TypeConverter>
int.TryParse
public static Object ConvertTo(this String arg, CultureInfo cultureInfo, Type type, Object defaultValue)
{
Object value;
TypeConverter typeConverter = Extensions.GetTypeConverter(type);
try
{
// Try converting the string.
value = typeConverter.ConvertFromString(null, cultureInfo, arg);
}
catch
{
// If the conversion fails then use the default value.
value = defaultValue;
}
return value;
}
哇！正如您所看到的，消除异常带来了巨大的不同。我从来没有意识到例外情况到底有多昂贵！因此，如果我将异常最小化到真正未知的情况，那么解析算法的运行速度将提高三个数量级。我认为这个问题已经彻底解决了。我相信我会用TypeConverter保持动态类型转换，它只会慢几毫秒。在转换之前检查已知的无效类型可以避免异常，这将大大加快速度！感谢ricovox指出了这一点，这让我进一步测试了这一点。这里有一段快速的代码可以尝试：
Dictionary<Type, TypeConverter> _ConverterCache = new Dictionary<Type, TypeConverter>();
TypeConverter GetCachedTypeConverter(Type type)
{
if (!_ConverterCache.ContainsKey(type))
_ConverterCache.Add(type, TypeDescriptor.GetConverter(type));
return _ConverterCache[type];
}
是快一点吗？如果主要是将字符串转换为本机数据类型（字符串、int、bool、DateTime等），可以使用下面的代码，它缓存类型代码和类型转换器（对于非本机类型），并使用fast switch语句快速跳转到相应的解析例程。这应该比Convert.ChangeType节省一些时间，因为源类型（字符串）已经已知，您可以直接调用正确的解析方法
/* Get an array of Types for each of your columns.
* Open the data file for reading.
* Create your DataTable and add the columns.
* (You have already done all of these in your earlier processing.)
*
* Note: For the sake of generality, I've used an IEnumerable<string>
* to represent the lines in the file, although for large files,
* you would use a FileStream or TextReader etc.
*/
IList<Type> columnTypes; //array or list of the Type to use for each column
IEnumerable<string> fileLines; //the lines to parse from the file.
DataTable table; //the table you'll add the rows to
int colCount = columnTypes.Count;
var typeCodes = new TypeCode[colCount];
var converters = new TypeConverter[colCount];
//Fill up the typeCodes array with the Type.GetTypeCode() of each column type.
//If the TypeCode is Object, then get a custom converter for that column.
for(int i = 0; i < colCount; i++) {
typeCodes[i] = Type.GetTypeCode(columnTypes[i]);
if (typeCodes[i] == TypeCode.Object)
converters[i] = TypeDescriptor.GetConverter(columnTypes[i]);
}
//Probably faster to build up an array of objects and insert them into the row all at once.
object[] vals = new object[colCount];
object val;
foreach(string line in fileLines) {
//delineate the line into columns, however you see fit. I'll assume a tab character.
var columns = line.Split('\t');
for(int i = 0; i < colCount) {
switch(typeCodes[i]) {
case TypeCode.String:
val = columns[i]; break;
case TypeCode.Int32:
val = int.Parse(columns[i]); break;
case TypeCode.DateTime:
val = DateTime.Parse(columns[i]); break;
//...list types that you expect to encounter often.
//finally, deal with other objects
case TypeCode.Object:
default:
val = converters[i].ConvertFromString(columns[i]);
break;
}
vals[i] = val;
}
//Add all values to the row at one time.
//This might be faster than adding each column one at a time.
//There are two ways to do this:
var row = table.Rows.Add(vals); //create new row on the fly.
// OR
row.ItemArray = vals; //(e.g. allows setting existing row, created previously)
}
TypeConverter typeConverter = GetCachedTypeConverter(type);
/* Get an array of Types for each of your columns.
* Open the data file for reading.
* Create your DataTable and add the columns.
* (You have already done all of these in your earlier processing.)
*
* Note: For the sake of generality, I've used an IEnumerable<string>
* to represent the lines in the file, although for large files,
* you would use a FileStream or TextReader etc.
*/
IList<Type> columnTypes; //array or list of the Type to use for each column
IEnumerable<string> fileLines; //the lines to parse from the file.
DataTable table; //the table you'll add the rows to
int colCount = columnTypes.Count;
var typeCodes = new TypeCode[colCount];
var converters = new TypeConverter[colCount];
//Fill up the typeCodes array with the Type.GetTypeCode() of each column type.
//If the TypeCode is Object, then get a custom converter for that column.
for(int i = 0; i < colCount; i++) {
typeCodes[i] = Type.GetTypeCode(columnTypes[i]);
if (typeCodes[i] == TypeCode.Object)
converters[i] = TypeDescriptor.GetConverter(columnTypes[i]);
}
//Probably faster to build up an array of objects and insert them into the row all at once.
object[] vals = new object[colCount];
object val;
foreach(string line in fileLines) {
//delineate the line into columns, however you see fit. I'll assume a tab character.
var columns = line.Split('\t');
for(int i = 0; i < colCount) {
switch(typeCodes[i]) {
case TypeCode.String:
val = columns[i]; break;
case TypeCode.Int32:
val = int.Parse(columns[i]); break;
case TypeCode.DateTime:
val = DateTime.Parse(columns[i]); break;
//...list types that you expect to encounter often.
//finally, deal with other objects
case TypeCode.Object:
default:
val = converters[i].ConvertFromString(columns[i]);
break;
}
vals[i] = val;
}
//Add all values to the row at one time.
//This might be faster than adding each column one at a time.
//There are two ways to do this:
var row = table.Rows.Add(vals); //create new row on the fly.
// OR
row.ItemArray = vals; //(e.g. allows setting existing row, created previously)
}
//NOTE: using System.Globalization;
var styles = NumberStyles.AllowDecimalPoint | NumberStyles.AllowLeadingSign;
var d = double.Parse(text, styles);
var parserLookup = new Dictionary<Type, Func<string, dynamic>>();
parserLookup.Add(typeof(Int32), s => Int32.Parse(s));
parserLookup.Add(typeof(Int64), s => Int64.Parse(s));
parserLookup.Add(typeof(Decimal), s => Decimal.Parse(s, NumberStyles.Number | NumberStyles.Currency, CultureInfo.CurrentCulture));
parserLookup.Add(typeof(DateTime), s => DateTime.Parse(s, CultureInfo.CurrentCulture, DateTimeStyles.AssumeLocal));
// and so on for any other type you want to handle.