#Azure data studio powershell 6 notebook update#

These line feeds, as we know, break our CSV structure meaning schema-on-read transformation tools struggle to read the file. My go-to example here is a CSV file, text qualified, but it contains an attribute of free text that some lovely frontend user has added carriage returns to when typing in notes or whatever. Typically in our data platform, data analytics or data science solution an element of data cleaning and preparation is required. If not, things can be switched around as you prefer and I’ll highlight alternatives as we go through. and executed on Windows virtual machines.

If you want to draw a comparison to SSIS, you can think of an ADF Custom Activity as a Script Task within an SSIS Control Flow.įor the ADF example in this post I’m going to talk about creating a Custom Activity using: So very quickly, in case you don’t know, an Azure Data Factory Custom Activity is simply a bespoke command or application created by you, in your preferred language and wrapped up in an Azure platform compute service that ADF can call as part of an orchestration pipeline. Let’s move onto the none Microsoft version, my version and knowledge 🙂 What is an ADF Custom Activity? It has a great comparison table near the bottom showing ADFv1 vs ADFv2 changes for the Custom Activity. If you haven’t already been through the Microsoft documents page I would recommend you do so before or after reading the below.

#Azure data studio powershell 6 notebook update#

I therefore feel I need to do an update post with the same information for Azure Data Factory (ADF) v2, especially given how this extensibility feature has changed and is implemented in a slightly different way to v1.

According to Google Analytics this proved to be one of my most popular blog posts on that site.

Many moons ago and in a previous job role I wrote a post for creating an Azure Data Factory v1 Custom Activity here.