I am trying to build an application that will read through an excel files (.xlsx, .xls). Unfortunately, the OleDbDataAdapter.Fill() performance is surprisingly bad. I takes up to 2 minutes to read one record from the file.
More info about the file:
- Size – 257MB
- Columns – 104
- Rows – 1 000 000
Code I am currently using to read the file:
string conStr = string.Empty;
string strQuery = string.Empty;
switch (extension)
{
case ".xls": //Excel 97-03
conStr = @"Provider=Microsoft.Jet.OleDb.12.0;Data Source=" + file_source + ";Extended Properties='Excel 8.0;HDR=YES;IMEX=1;';";
strQuery = "SELECT " + col_no + " * FROM [" + workbook + "] ";
break;
case ".xlsx": //Excel 07
//connection string to connect to the xlsx file
conStr = @"Provider=Microsoft.Ace.OleDb.12.0;Data Source=" + file_source + ";Extended Properties='Excel 12.0 Xml;HDR=YES;IMEX=1;';";
strQuery = "SELECT " + col_no + " * FROM [" + workbook + "] ";
break;
}
DataTable tbl = new DataTable();
OleDbDataAdapter ada = new OleDbDataAdapter(strQuery, conStr);
ada.Fill(tbl);
ada.Dispose();
return tbl;
Your help would be greatly appreciated!
Thanks!
It is a bad idea to store large data sets in Excel files. In my opinion reading 250MB Excel file in 2 minutes is to be expected.
I would suggest switching to any database solution. But if you can’t change you data store, you can try using Excel COM objects (it will require Excel to be installed on server machine though). Here is a walktrough by Teddy Garland.
Tags: c#c#