One of the new features of SQL Server Integration Services 2008 is the parallelization of the pipeline. This is implemented by creating multiple execution threads that can be run on different processors, increasing the performance of the package. In SSIS 2005, this separation of threads could be emulated by using a Union All transformation to create a new buffer.
If you turn on logging in the data flow, you can see these separate execution trees in the events PipelineExecutionPlan and PipelineExecutionTrees. I created a simple package that creates a row set of integers, multicasts to a conditional split (as a terminator) and a flat file destination.
Here are the messages from those events:
User:PipelineExecutionTrees
Begin Path 0
output "Output 0" (32); component "Script Component" (29)
input "Multicast Input 1" (53); component "Multicast" (52)
Begin Subpath 0
output "Multicast Output 1" (54); component "Multicast" (52)
input "Flat File Destination Input" (64); component "Flat
File Destination" (63)
End Subpath 0
Begin Subpath 1
output "Multicast Output 2" (69); component "Multicast" (52)
input "Conditional Split Input" (57); component
"Conditional Split" (56)
End Subpath 1
End Path 0
User:PipelineExecutionPlan
Begin output plan
Begin transform plan
End transform plan
Begin source plan
Call PrimeOutput on component "Script Component" (29)
for output "Output 0" (32)
End source plan
End output plan
Begin path plan
Begin Path Plan 0
Call ProcessInput on component "Multicast" (52)
for input "Multicast Input 1" (53)
Create new execution item for subpath 0
Create new execution item for subpath 1
Begin Subpath Plan 0
Call ProcessInput on component "Flat File Destination" (63)
for input "Flat File Destination Input" (64)
End Subpath Plan 0
Begin Subpath Plan 1
Call ProcessInput on component "Conditional Split" (56)
for input "Conditional Split Input" (57)
End Subpath Plan 1
End Path Plan 0
End path plan
You can see that the new log creates a subpath for both the flat file destination path and the conditional split path. The best part is that each subpath can then be executed on different processors, increasing the speed of the package!
Version: SQL Server 2008 CTP6
If you turn on logging in the data flow, you can see these separate execution trees in the events PipelineExecutionPlan and PipelineExecutionTrees. I created a simple package that creates a row set of integers, multicasts to a conditional split (as a terminator) and a flat file destination.
Here are the messages from those events:
User:PipelineExecutionTrees
Begin Path 0
output "Output 0" (32); component "Script Component" (29)
input "Multicast Input 1" (53); component "Multicast" (52)
Begin Subpath 0
output "Multicast Output 1" (54); component "Multicast" (52)
input "Flat File Destination Input" (64); component "Flat
File Destination" (63)
End Subpath 0
Begin Subpath 1
output "Multicast Output 2" (69); component "Multicast" (52)
input "Conditional Split Input" (57); component
"Conditional Split" (56)
End Subpath 1
End Path 0
User:PipelineExecutionPlan
Begin output plan
Begin transform plan
End transform plan
Begin source plan
Call PrimeOutput on component "Script Component" (29)
for output "Output 0" (32)
End source plan
End output plan
Begin path plan
Begin Path Plan 0
Call ProcessInput on component "Multicast" (52)
for input "Multicast Input 1" (53)
Create new execution item for subpath 0
Create new execution item for subpath 1
Begin Subpath Plan 0
Call ProcessInput on component "Flat File Destination" (63)
for input "Flat File Destination Input" (64)
End Subpath Plan 0
Begin Subpath Plan 1
Call ProcessInput on component "Conditional Split" (56)
for input "Conditional Split Input" (57)
End Subpath Plan 1
End Path Plan 0
End path plan
You can see that the new log creates a subpath for both the flat file destination path and the conditional split path. The best part is that each subpath can then be executed on different processors, increasing the speed of the package!
Version: SQL Server 2008 CTP6
Comments