- Performance Analysis of V-Ordering in Fabric Warehouse: On or Off?
by Andy
Overview V-Order is Microsoft’s “special sauce” to further optimise data within Parquet files and is…
- DP-700 Microsoft Fabric Data Engineering Associate – Skills Measured Resources
by Andy
New certification time! At FabCon Europe, a new certification was announced…the Microsoft Fabric Data Engineering…
- Reallocate Workspaces To A Different Fabric Capacity: Low Code Solution
by Andy
Overview I’m not saying this blog coincides with the current active Fabric Trials coming to…
- Miniblog: Fabric Workspace Storage Analysis
by Andy
Challenge: How can I see the underlying storage and files being used in my Fabric…
- The Reality of ALTER TABLE… in Fabric Warehouses
by Andy
Overview With the recent announcement from Microsoft that ALTER TABLE…ADD COLUMN is now supported in…
- Microsoft Fabric Warehouse for the Database Administrator (DBA) Part 3: Fabric Compute
by Andy
Overview Welcome to part 3 of this Fabric Warehouse for the DBA series in which…
- Microsoft Fabric Pay-As-You-Go vs Reserved Pricing And How To Purchase
by Andy
Overview One question I see quite regularly on forums and social media is the pricing…
- Microsoft Fabric Warehouse for the Database Administrator (DBA): Fabric & Warehouse Service Overview
by Andy
Overview This is the second part of the Fabric Warehouses for DBAs blog series, if…
- Microsoft Fabric Warehouse for the Database Administrator (DBA): Series Overview
by Andy
Overview Let me start off this blog post by saying that no matter what data…
- Fabric Networking: Trusted Workspace Access for OneLake Shortcuts
by Andy
Overview Along with Managed Private Endpoints, another security feature was released at the same time…
- Fabric Networking: Managed Private Endpoints with Setup and Considerations
by Andy
Managed Private Endpoints in Public Preview Security fans rejoice (and companies that don’t appreciate leaving…
- Fabric Link for Dataverse: Initial Setup and Considerations
by Andy
Overview I’ve been looking at implementing a proof-of-concept for the Fabric Link for Dataverse functionality…
- Using Copilot in Fabric Notebooks for Data Engineering: Quick Look
by Andy
Overview Probably one of the most hotly anticipated services is available within Fabric now, as…
- DP-600 Fabric Analytics Engineer Skills Measured Breakdown and Resource Links
by Andy
Overview As of 12th January 2024, the beta exam for the new Microsoft Certified: Fabric…
- Data Loading Options With Fabric Workspaces Part 1 of 2 – Yet Another Blog About The Medallion Architecture
by Andy
This blog will be split over 2 parts, in this first part we’ll look at…
- Exploring Fabric OneLake vs Azure Data Lake Gen2 Storage Accounts
by Andy
Overview Recently I’ve been reading through Microsoft’s documentation on implementing medallion lakehouse architecture in Microsoft…
- Explore This Data in Power BI is Simple But Effective for Basic Data Exploration
by Andy
Well this new Explore This Data (link) feature in Power BI surprised me, both in…
- Delta Lake Change Data Feed in Fabric Lakehouses
by Andy
As I start to look at how to architect and develop data solutions using Microsoft…
- New Microsoft Certification Coming in January 2024: Fabric Analytics Engineer Associate
by Andy
Ignite is always exciting as there’s always a bunch of new announcements, and included in…
- #DataWeekender 6.5 Fabric SQL Warehouse Session
by Andy
Here are the slides for my Fabric SQL Warehouse session at #DataWeekender 6.5. Download PDF…
- Microsoft Applied Skills – Bite Sized Learning and Accreditation
by Andy
Microsoft announced a new accreditation path recently called Microsoft Applied Skills (link here), a set…
- Fabric Lakehouse Loading using Data Pipelines & Notebooks – Inspired by MS End-to-End Tutorials
by Andy
I joke about “I’m not much of a Spark Developer” all the time and I’m…
- Synapse Analytics Source Control with Serverless SQL Pools Databases
by Andy
Working with source control in a Synapse Analytics workspace is a fairly straight-forward process, however…
- What Happens When You Clone A Fabric Warehouse Table?
by Andy
Fabric Warehouses are a SQL focused service for building data warehouse structures within a Fabric…
- Azure Synapse Analytics Custom Roles
by Andy
Recently I had a challenge in terms of providing a least permissions possible security model…
- Future Data Driven 2023 – Fabric SQL
by Andy
Session Abstract It all started with Parallel Data Warehouse, then came Azure SQL Data Warehouse…then…
- Azure Synapse Analytics Managed Virtual Networks for the Developer
by Andy
It’s recommended in a Synapse environment, especially a production/live environment, to lock a Synapse Analytics…
- Synapse Link for Dataverse with Azure Synapse Analytics eBook
by Andy
If you’re interested in using Synapse Link for Dataverse with a Dynamics 365 environment to…
- Transactions and Isolation Levels in Fabric Warehouse
by Andy
I trust everyone has been having fun with Fabric over the last few weeks, diving…
- Temp Tables in Fabric Lakehouse SQL Endpoint and Warehouse
by Andy
This blog has been adapted from an earlier blog about using temp tables in Synapse…
- Play Around: Deploy Database Projects to Fabric Warehouse
by Andy
I always like to have a dig around, try a few things and see if…
- Fabric in Production
by Andy
This is a placeholder blog which will be revisted many, many times over the coming…
- Getting Started with Microsoft Fabric: Enabling 60-Day Trial and Analysing Data
by Andy
There’s been an exciting new announcement at Microsoft Build in the form of Microsoft Fabric,…
- Synapse Link for Dataverse: Delta Lake Export Configuration
by Andy
Welcome to part 2 in the Synapse Link for Dataverse Delta Lake series. In the…
- Synapse Link for Dataverse: Exporting to Delta Lake
by Andy
Microsoft recently released the ability to configure Synapse Link for Dataverse and use Delta Lake…
- Parallel Loading of Tables with Power BI & Azure Synapse Analytics
by Andy
In this blog I’m continuing my series about using Power BI and Azure Synapse Analytics…
- What Does A “Conformed Dimension” Actually Do?
by Andy
Every so often the “star schema is dead! dimensional modelling is dead!” discussion rolls around…
- Azure Synapse Serverless SQL Pools Cheat Sheet
by Andy
Here’s an Azure Synapse Serverless SQL Pools Cheat Sheet I created to help anyone reference…
- Why Cloud Data Platforms?
by Andy
Always ask “why.” I really couldn’t think of a better blog title, I usually do…
- Understanding ResultSet Caching in Dedicated SQL Pools with Power BI
by Andy
Welcome to this new blog series in which I’ll be diving into both Power BI…
- Mastering DP-500 Exam: Upcoming Changes on February 6th
by Andy
Some of the skills measured for the DP-500 Azure Enterprise Data Analyst Associate certification are…
- Partition Pruning with Delta Tables in Azure Synapse Analytics
by Andy
A question that is often asked about Azure Synapse Analytics is how it handles Delta…
- Synapse Link for Dataverse: Video Series
by Andy
I’ve been diving into Synapse Link for Dataverse recently for a project and created a…
- Azure Analysis Services Migration to Power BI Premium
by Andy
Around November 2022, Microsoft announced a new feature within Power BI which allows you to…
- Azure Synapse Link for Dataverse: Understanding Advanced Configuration Settings
by Andy
In the first part of this blog series we looked at how to configure Synapse…
- Azure Synapse Analytics in 2022: Retrospective
by Andy
Retrospective Another year has rolled by (where does the time go…) and another year in…
- Azure Synapse Analytics at SQLBit 2023
by Andy
The General Session agenda for SQLBits 2023 is now available here and covers the agenda…
- Mastering DP-500 Exam: Configuring Synapse Analytics Source Control with Azure DevOps
by Andy
This blog forms part of the DP-500 blog series on this site here and also…
- Databricks Data Engineer Associate Certification Overview
by Andy
I’ve been looking at taking the Databricks Data Engineer Associate Certification (the Databricks page for…
- Azure Synapse Link for Dataverse: Getting Started with Dynamics 365 Sales and Synapse Analytics
by Andy
Part of the Azure Synapse Analytics suite of services is Synapse Link. This enables synchronization…
- Connecting Power BI to Azure SQL Database using Private Endpoints
by Andy
This was originally posted on my Datahai company blog in November 2021. This blog has…
- Mastering DP-500 Exam: Querying JSON Data in Serverless SQL Pools
by Andy
This blog forms part of the DP-500 blog series on this site here and also…
- Dedicated SQL Pools DMVs: Show Name of Synapse Studio Script Being Executed
by Andy
I’m currently in the process of building a Query and Workload Analyser Power BI report…
- User Permissions in Serverless SQL Pools: External Tables vs Views
by Andy
In a recent Synapse Analytics Serverless SQL Pools QnA session, the Serverless SQL Pools team…
- Up and Running: How to make your Azure Synapse Analytics Proof of Concept a Success (Data Relay Session)
by Andy
Firstly, I’d like to say a huge thank you to the Data Relay team, the…
- Elastic Queries with Azure SQL Database and Synapse Analytics Serverless SQL Pools
by Andy
Recently I have been exploring an option to use an Azure SQL Database for a…
- Use Azure Log Analytics to analyse Azure Data Lake Gen2 activity from Serverless SQL Pools
by Andy
This blog is marked as intermediate knowledge as an understanding of Serverless SQL Pools is preferable, but…
- Use Azure Storage Logs to analyse Synapse Analytics Serverless SQL Pools Activity
by Andy
This blog is marked as intermediate knowledge as an understanding of Serverless SQL Pools is…
- PASS Summit Recording Available: Creating a Logical Data Warehouse using Azure Synapse Analytics Serverless SQL Pools
by Andy
My PASS Data Community Summit 2021 recording is now available, please click here for the…
- How does Serverless SQL Pools deal with different file schemas? Part 3 – Delta
by Andy
Welcome to the 3rd and final part in this blog series in which we’re looking…
- How does Serverless SQL Pools deal with different file schemas? Part 2 – Parquet
by Andy
Welcome to part 2 in this blog series in which we’re looking at what happens…
- Lake Databases And The Importance of Data Types
by Andy
I encountered an issue with Azure Synapse Analytics Lake Databases and I’d like to share…
- Mastering DP-500 Exam: Using the Data Map in Microsoft Purview
by Andy
Part of the upcoming Azure Enterprise Data Analyst Associate exam, the DP-500, is to understand…
- Mastering DP-500 Exam: Integrating Microsoft Purview with Power BI
by Andy
Part of the upcoming Azure Enterprise Data Analyst Associate exam, the DP-500, is to understand…
- Mastering DP-500 Exam: Querying Partitioned Sources in Azure Storage
by Andy
This blog forms part of the Mastering DP-500 exam series. A list of available blogs…
- Mastering DP-500 Exam: Using Synapse Analytics Serverless SQL Pools to query Azure Storage
by Andy
This blog forms part of the Mastering DP-500 exam series. A list of available blogs…
- Mastering DP-500 Exam: Blog Resources
by Andy
I am collaborating with Data Mozart on blogging around the upcoming Microsoft Certified: Azure Enterprise…
- New Certification: Microsoft Certified Azure Enterprise Data Analyst Associate
by Andy
Microsoft has created a new certification, the Microsoft Certified Azure Enterprise Data Analyst Associate (link…
- How does Serverless SQL Pools deal with different file schemas? Part 1 – CSV
by Andy
Serverless SQL Pools offers a straight-forward method of querying data including CSV, JSON, and Parquet…
- Integrating Power BI & Azure Synapse Analytics – Power BI UK User Group
by Andy
Here is the link to the slides for the Power BI UK User Group session…
- SQLBits 2022 – The Dream Team: Synapse Analytics Serverless SQL Pools and Pipelines
by Andy
I presented a 20 minute session at SQLBits 2022 in which I talked about how…
- Trigger The Process – Using CETAS in Serverless SQL Pools to Generate Pre-Calculated Datasets – Part 2 of 2
by Andy
In part one we looked at a process in which a pre-aggregated dataset can be…
- Using CETAS in Serverless SQL Pools to Generate Pre-Calculated Datasets – Part 1 of 2
by Andy
Dedicated SQL Pools have a feature called Materialized Views which can generate a pre-calculated/processed dataset…
- DataMinutes #2 Session Companion for Turbocharging Power BI using Synapse Analytics
by Andy
The slides for the DataMinutes #2 session Turbocharging Power BI using Synapse Analytics is available…
- SQLBits Session – The Dream Team: Synapse Analytics Serverless SQL Pools and Pipelines
by Andy
SQLBits is back this year and is running from Tuesday 8th to Sunday 12th March…
- DataMinutes #2 Conference – Azure Synapse Analytics Sessions
by Andy
The next DataMinutes conference (#2) is taking place on Friday 21st January. What’s different about…
- Power BI Hybrid Tables with Synapse Analytics Serverless SQL Pools
by Andy
In this video we’re talking a look at the new Power BI Hybrid Tables feature…
- View Generation in Serverless SQL Pools
by Andy
One of the objects that can be created within a Serverless SQL Pools database is…
- Testing Error Handling in Serverless SQL Pools
by Andy
A recent feature to be released for Serverless SQL Pools is error handling when processing…
- Data Toboggan Azure Synapse Analytics Event – Call For Speakers
by Andy
Data Toboggan is a community-driven conference focused on Azure Synapse Analytics, Microsoft’s flagship data warehousing,…
- PASS Data Community Summit Companion Resources
by Andy
PASS Data Community Summit 2021 Slides Please click here for a PDF copy of the…
- Creating a Lake Database using the Database Designer
by Andy
Synapse Analytics allows the creation of External Tables and Views within a Serverless SQL Pools…
- Azure Synapse Analytics in the Azure Architecture Centre
by Andy
Microsoft provide an Azure Architecture Centre which contains resources to understand which Azure services are…
- Using Access Control Lists To Manage Fine-Grain Data Lake Permissions
by Andy
When data is stored in Azure Data Lake Gen2, we can use Azure Active Directory…
- Integrating Azure Purview with Synapse Analytics Serverless SQL Pools
by Andy
Azure Purview is a Microsoft Data Governance service which enables an organisation to map their…
- Using Temporary Tables within Serverless SQL Pools
by Andy
Temporary (temp) tables have been a feature of Microsoft SQL Server (and other database systems)…
- External Tables vs Views…Which to Use and Why?
by Andy
Serverless SQL Pools do not contain data (unlike Dedicated SQL Pools) but rather allow you…
- Azure Storage Folder Filtering using the Filepath Function
by Andy
Serverless SQL Pools includes 2 SQL functions, filepath and filename, that can be used to…
- 10 Billion Rows: Parquet File Size and Distribution When using CETAS
by Andy
When using Serverless SQL Pools to write data to Azure Storage/Data Lake Gen2 using the…
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 4 of 4 – Connecting Power BI to the Dimensional Model
by Andy
Welcome to Part 4 in the Creating a Logical Data Warehouse with Synapse Serverless SQL…
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 3 of 4 – Incremental Fact Loading and Slowly Changing Dimensions
by Andy
Welcome to part 3 of the Logical Data Warehouse with Serverless SQL Pools series. In…
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 2 of 4 – Creating a Dimensional Model
by Andy
Welcome to part 2 of this 4 part blog series on creating a Logical Data…
- Creating a Logical Data Warehouse with Synapse Serverless SQL: Part 1 of 4 – Setting Up and Querying Source Data
by Andy
Welcome to part one of a four part blog series where we’ll be looking at…
- Azure Synapse Analytics Serverless SQL Pools Learning Resources
by Andy
Welcome to this collection of links to resources focussed on Azure Synapse Analytics Serverless SQL…
- Azure Synapse Analytics Serverless SQL Best Practices
by Andy
Microsoft have a best practices document available here which highlights certain actions that can be…
- Getting Started with Azure Synapse Analytics SQL Serverless
by Andy
The new Azure Synapse Analytics SQL Serverless service enables querying file-based data stored in an…
- Harnessing Azure Synapse Analytics SQL Serverless in Power BI Dataflows
by Andy
Power BI Dataflows are a Power BI Service component that allow the creation of centralised,…
- Serverless SQL Pools Managed Private Endpoint
by Andy
Steps Created new Synapse Workspace and selected Managed VNet option Opened Synapse Studio and clicked…
- Fabric Warehouse Source Control in Azure Data Studio
by Andy
Overview V-Order is Microsoft’s “special sauce” to further optimise data within Parquet files and is…
New certification time! At FabCon Europe, a new certification was announced…the Microsoft Fabric Data Engineering…
Overview I’m not saying this blog coincides with the current active Fabric Trials coming to…
Overview With the recent announcement from Microsoft that ALTER TABLE…ADD COLUMN is now supported in…
Overview Welcome to part 3 of this Fabric Warehouse for the DBA series in which…
Overview One question I see quite regularly on forums and social media is the pricing…
Overview This is the second part of the Fabric Warehouses for DBAs blog series, if…
Overview Let me start off this blog post by saying that no matter what data…
Overview Along with Managed Private Endpoints, another security feature was released at the same time…
Managed Private Endpoints in Public Preview Security fans rejoice (and companies that don’t appreciate leaving…
Overview I’ve been looking at implementing a proof-of-concept for the Fabric Link for Dataverse functionality…
Overview Probably one of the most hotly anticipated services is available within Fabric now, as…
Overview As of 12th January 2024, the beta exam for the new Microsoft Certified: Fabric…
This blog will be split over 2 parts, in this first part we’ll look at…
Overview Recently I’ve been reading through Microsoft’s documentation on implementing medallion lakehouse architecture in Microsoft…
Well this new Explore This Data (link) feature in Power BI surprised me, both in…
As I start to look at how to architect and develop data solutions using Microsoft…
Here are the slides for my Fabric SQL Warehouse session at #DataWeekender 6.5. Download PDF…
Microsoft announced a new accreditation path recently called Microsoft Applied Skills (link here), a set…
I joke about “I’m not much of a Spark Developer” all the time and I’m…
Working with source control in a Synapse Analytics workspace is a fairly straight-forward process, however…
Fabric Warehouses are a SQL focused service for building data warehouse structures within a Fabric…
Recently I had a challenge in terms of providing a least permissions possible security model…
Session Abstract It all started with Parallel Data Warehouse, then came Azure SQL Data Warehouse…then…
It’s recommended in a Synapse environment, especially a production/live environment, to lock a Synapse Analytics…
If you’re interested in using Synapse Link for Dataverse with a Dynamics 365 environment to…
I trust everyone has been having fun with Fabric over the last few weeks, diving…
This blog has been adapted from an earlier blog about using temp tables in Synapse…
I always like to have a dig around, try a few things and see if…
This is a placeholder blog which will be revisted many, many times over the coming…
There’s been an exciting new announcement at Microsoft Build in the form of Microsoft Fabric,…
Welcome to part 2 in the Synapse Link for Dataverse Delta Lake series. In the…
Microsoft recently released the ability to configure Synapse Link for Dataverse and use Delta Lake…
In this blog I’m continuing my series about using Power BI and Azure Synapse Analytics…
Every so often the “star schema is dead! dimensional modelling is dead!” discussion rolls around…
Here’s an Azure Synapse Serverless SQL Pools Cheat Sheet I created to help anyone reference…
Always ask “why.” I really couldn’t think of a better blog title, I usually do…
Welcome to this new blog series in which I’ll be diving into both Power BI…
Some of the skills measured for the DP-500 Azure Enterprise Data Analyst Associate certification are…
A question that is often asked about Azure Synapse Analytics is how it handles Delta…
I’ve been diving into Synapse Link for Dataverse recently for a project and created a…
Around November 2022, Microsoft announced a new feature within Power BI which allows you to…
In the first part of this blog series we looked at how to configure Synapse…
Retrospective Another year has rolled by (where does the time go…) and another year in…
The General Session agenda for SQLBits 2023 is now available here and covers the agenda…
This blog forms part of the DP-500 blog series on this site here and also…
I’ve been looking at taking the Databricks Data Engineer Associate Certification (the Databricks page for…
Part of the Azure Synapse Analytics suite of services is Synapse Link. This enables synchronization…
This was originally posted on my Datahai company blog in November 2021. This blog has…
This blog forms part of the DP-500 blog series on this site here and also…
I’m currently in the process of building a Query and Workload Analyser Power BI report…
In a recent Synapse Analytics Serverless SQL Pools QnA session, the Serverless SQL Pools team…
Firstly, I’d like to say a huge thank you to the Data Relay team, the…
Recently I have been exploring an option to use an Azure SQL Database for a…
This blog is marked as intermediate knowledge as an understanding of Serverless SQL Pools is preferable, but…
This blog is marked as intermediate knowledge as an understanding of Serverless SQL Pools is…
My PASS Data Community Summit 2021 recording is now available, please click here for the…
Welcome to the 3rd and final part in this blog series in which we’re looking…
Welcome to part 2 in this blog series in which we’re looking at what happens…
I encountered an issue with Azure Synapse Analytics Lake Databases and I’d like to share…
Part of the upcoming Azure Enterprise Data Analyst Associate exam, the DP-500, is to understand…
Part of the upcoming Azure Enterprise Data Analyst Associate exam, the DP-500, is to understand…
This blog forms part of the Mastering DP-500 exam series. A list of available blogs…
This blog forms part of the Mastering DP-500 exam series. A list of available blogs…
I am collaborating with Data Mozart on blogging around the upcoming Microsoft Certified: Azure Enterprise…
Microsoft has created a new certification, the Microsoft Certified Azure Enterprise Data Analyst Associate (link…
Serverless SQL Pools offers a straight-forward method of querying data including CSV, JSON, and Parquet…
Here is the link to the slides for the Power BI UK User Group session…
I presented a 20 minute session at SQLBits 2022 in which I talked about how…
In part one we looked at a process in which a pre-aggregated dataset can be…
Dedicated SQL Pools have a feature called Materialized Views which can generate a pre-calculated/processed dataset…
The slides for the DataMinutes #2 session Turbocharging Power BI using Synapse Analytics is available…
SQLBits is back this year and is running from Tuesday 8th to Sunday 12th March…
The next DataMinutes conference (#2) is taking place on Friday 21st January. What’s different about…
In this video we’re talking a look at the new Power BI Hybrid Tables feature…
One of the objects that can be created within a Serverless SQL Pools database is…
A recent feature to be released for Serverless SQL Pools is error handling when processing…
Data Toboggan is a community-driven conference focused on Azure Synapse Analytics, Microsoft’s flagship data warehousing,…
PASS Data Community Summit 2021 Slides Please click here for a PDF copy of the…
Synapse Analytics allows the creation of External Tables and Views within a Serverless SQL Pools…
Microsoft provide an Azure Architecture Centre which contains resources to understand which Azure services are…
When data is stored in Azure Data Lake Gen2, we can use Azure Active Directory…
Azure Purview is a Microsoft Data Governance service which enables an organisation to map their…
Temporary (temp) tables have been a feature of Microsoft SQL Server (and other database systems)…
Serverless SQL Pools do not contain data (unlike Dedicated SQL Pools) but rather allow you…
Serverless SQL Pools includes 2 SQL functions, filepath and filename, that can be used to…
When using Serverless SQL Pools to write data to Azure Storage/Data Lake Gen2 using the…
Welcome to Part 4 in the Creating a Logical Data Warehouse with Synapse Serverless SQL…
Welcome to part 3 of the Logical Data Warehouse with Serverless SQL Pools series. In…
Welcome to part 2 of this 4 part blog series on creating a Logical Data…
Welcome to part one of a four part blog series where we’ll be looking at…
Welcome to this collection of links to resources focussed on Azure Synapse Analytics Serverless SQL…
Microsoft have a best practices document available here which highlights certain actions that can be…
The new Azure Synapse Analytics SQL Serverless service enables querying file-based data stored in an…
Power BI Dataflows are a Power BI Service component that allow the creation of centralised,…