How to use selected SQL statement in SSIS package as source variable?

Question

I created SSIS package where I need to use FELC. The first step before the loop is to run sql task to obtain all SQL statements designed to generate different XML files and stored in a source table. Inside the FELC I would like to process the statements to generate XML files and send them to various folder locations with names and target folder coming from the source table. There is hundreds of files that needs to be refreshed on regular basis. Instead of running single jobs for each XML file generation I would like to amalgamate it into one process. Is it possible?

Answer 1

This is the basic Shred Recordset pattern.

I have 3 variables declared: SourceQuery, CurrentQuery and rsQueryData. The first 2 are Strings, the last is an Object type

SQL - Get source data

This is my query. It simulates your table and induces a failing SQL Statement if I take out the filter.

SELECT
    ProcessID
,   Stmt_details
FROM
    ( 
        VALUES 
            ( 1, 'SELECT 1;', 1)
        ,   ( 20, 'SELECT 20;', 1)
        ,   ( 30, 'SELECT 1/0;', 0) 
    ) Stmt_collection (ProcessID, Stmt_details, xmlFlag)
WHERE
    xmlFlag = 1

The Execute SQL Task is set with Recordset = Full and I assign it to variable User::rsQueryData which has a name of 0 in the mapping tab.

FELC

This is a standard Foreach ADO Recordset Loop container. I use my User::rsQueryData as the source and since I only care about the second element, ordinal position 1, that's the only thing I map. I assign the current value to User::CurrentStatement

SQL - Execute CurrentStatement

This is an Execute SQL Task that has as its source the Variable User::CurrentStatement. There's no scripting involved. The FELC handles the assignment of values to this Variable. This Task uses as its source that same Variable. This is very much how native SSIS developers will approach solving a problem. If you reach for a Script Task or Component as the first approach, you're likely doing it wrong.

Biml

If you're doing any level of SSIS/SSRS/SSAS development, you want Bids Helper It is a free add on to Visual Studio that makes your development life so much easier. The feature I'm going to leverage here is the ability to declaratively define an SSIS package. This language is called the Business Intelligence Markup Language, Biml, and I love it for many reasons but on StackOverflow, I love it because I can give you the code to reproduce exactly my solution. Otherwise, I have to build out a few hundred screenshots showing you everywhere I have to click and set values.

Or, you 1. Download and install BIDS Helper 2. Open up your existing SSIS project 3. Right click on the Project and select "Add new Biml file" 4. In the resulting BimlScript.biml file, open it up and paste all of the following code into it 5. Fix the value for your database connection string. This one assumes you have an instance on your local machine called Dev2014 6. Save the biml file 7. Right click that BimlScript.biml and select "Generate SSIS Packages" 8. Marvel at the resulting so_28867703.dtsx package that was added to your solution

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
    <Connections>
        <OleDbConnection Name="CM_OLE" ConnectionString="Data Source=localhost\dev2014;Initial Catalog=tempdb;Provider=SQLNCLI10.1;Integrated Security=SSPI;Auto Translate=False;" />
    </Connections>
    <Packages>
        <Package ConstraintMode="Linear" Name="so_28867703">
            <Variables>
                <Variable DataType="String" Name="QuerySource">SELECT ProcessID, Stmt_details FROM (VALUES (1, 'SELECT 1;', 1), (20, 'SELECT 20;', 1), (30, 'SELECT 1/0;', 0))Stmt_collection(ProcessID, Stmt_details, xmlFlag) WHERE xmlFlag = 1 </Variable>
                <Variable DataType="String" Name="CurrentStatement">This statement is invalid</Variable>
                <Variable DataType="Object" Name="rsQueryData"></Variable>
            </Variables>
            <Tasks>
                <ExecuteSQL 
                    ConnectionName="CM_OLE" 
                    Name="SQL - Get source data"
                    ResultSet="Full"
                    >
                    <VariableInput VariableName="User.QuerySource" />
                    <Results>
                        <Result VariableName="User.rsQueryData" Name="0" />
                    </Results>
                </ExecuteSQL>
                <ForEachAdoLoop 
                    SourceVariableName="User.rsQueryData" 
                    ConstraintMode="Linear" 
                    Name="FELC - Shred RS"
                    >
                    <VariableMappings>
                        <!--
                        0 based system
                        -->
                        <VariableMapping VariableName="User.CurrentStatement" Name="1" />
                    </VariableMappings>
                    <Tasks>
                        <ExecuteSQL ConnectionName="CM_OLE" Name="SQL - Execute CurrentStatement">
                            <VariableInput VariableName="User.CurrentStatement" />
                        </ExecuteSQL>
                    </Tasks>
                </ForEachAdoLoop>
            </Tasks>
        </Package>
    </Packages>
</Biml>

That package will run, assuming you fixed the connection string to a valid instance. You can see below that if you put break points on the Execute SQL Task, it will light up two times. If you have a watch window on CurrentStatement, you can see it change from the design time value to the values shredded from the result set.

While we await clarification on XML and files, if the goal is to take the query from the FELC and export to file, I answered that https://stackoverflow.com/a/9105756/181965 Although in this case, I'd restructure your package to just the Data Flow and eliminate the shredding as there's no need to complicate matters to export a single row N times.

Answer 2

If i understand you correctly; You can add a "Script Task" from Toolbox to first step of loop container and store the selected statement from the database in to the global variable and pass it for execution in the next step