Running classic SSIS packages in cloud has been possible for some time already. It is done utilizing Azure Data Factory (ADF) which contains a specific runtime engine called a SSIS Integration Runtime (SSIS-IR). Basically it’s just a cluster of Windows servers which are managed automatically by Azure. Those virtual machines have SSIS engine installed which is used to run SSIS packages.
To reduce running costs, it’s a best practice to setup SSIS-IR to be on only when needed. In 2018 I was working in project in which we utilized SSIS-IR. I became curious about how long it takes to start up SSIS-IR so I setup a pipeline into ADF which just started and stopped SSIS-IR continuously. Based on these tests the average startup time was approx. 17,6 minutes.
Some time ago I bumped into a video in which a Microsoft representative announced that they had managed to speed up the starting process. Obviously it was time to rerun the tests. And indeed it is now much much faster than previously. Average time was now about 90 seconds. That means it’s now 10 times faster than previously. Here are the results visualized:
Both test runs contained about 100 full start-stop cycles. 2018 tests were run in Azure North Europe region, 2020 in West Europe. Node size D2_v3, number of nodes: 1. No custom configs.
But there’s a catch: Speedup only applies to those SSIS-IRs outside of a VNet! Inside the startup times are the same as previously, about 20 minutes on average. More details here.