Home » excel » sql – SSIS: Excel data source – if column not exists use other column

sql – SSIS: Excel data source – if column not exists use other column

Posted by: admin May 14, 2020 Leave a comment

Questions:

I am using select statement in excel source to select just specific columns data from excel for import.

But I am wondering, is it possible to select data such way when I select for example column with name: Column_1, but if this column is not exists in excel then it will try to select column with name Column_2? Currently if Column_1 is missing, then data flow task fails.

How to&Answers:

Use a Script task and write .net code to read the excel file and then perform the check for the Column_1 availability in the file. If the column does not present then use Column_2 as input. Script Task in SSIS can act as a source.

Answer:

The easiest way is to add two data flow tasks, one data flow for each Excel source select statement and use precedence constraints to execute the second data flow when the first one fails.

The disadvantage of this approach is that if the first data flow task fails for another reason, it will also try to execute the second one. You will need some advanced error handling to check if the error is thrown due to missing columns or not.

But if have a similar situation, I will use a Script Task to check if the column exists and build the SQL command dynamically. Note that this SQL command must always return the same metadata (you must use aliases).

Helpful links

Answer:

SSIS is metadata based and will not support dynamic metadata, however you can use Script Component as @nitin-raj suggested to handle all known source columns. There is a good post below on how it can be done.

Dynamic File Connections

If you have many such files that can have varying columns then it is better to create a custom component.However, you cannot have dynamic metadata even with custom component, the set of columns should be known upfront to SSIS.

If the list of columns keep changing and you cannot know in advance what are expected columns then you are better off handling the entire thing in C#/VB.Net using Script Task of control flow

Answer:

As a best practice, because SSIS meta data is static, any data quality and formatting issues in source files should be corrected before ssis data flow task runs.

I have seen this situation before and there is a very simple fix. In the beginning of your ssis package, using a file task to create copy of the source excel file and then run a c# script or execute a powershell to rename the columns so that if column 1 does not exist, it is either added at the appropriate spot in excel file or in case the column name is wrong is it corrected.

As a result of this, you will not need to refresh your ssis meta data every time it fails. This is a standard data standardization practice.