Harnessing the Power of Streaming Data
Streaming data has emerged as a critical component of modern data strategies. Streaming data refers to data that is continuously generated by various sources, these typically send data records in small sizes (such as bytes or kilobytes). This blog explores what streaming data is, why it’s important, its applications, and the tools used to manage […]
Azure Data Factory vs Logic Apps:
Key Differences and Cost Implications On a recent project I have been looking at whether to use Azure Logic Apps as an alternative to Azure Data Factory (ADF). This brought me to compare these 2 technologies. This blog aims to summarise key differences between Azure Data Factory vs Logic Apps. It also aims to discuss […]
A Deep Dive into Slowly Changing Dimensions (SCDs)
Source systems are a treasure trove of historical information. Importing data in a data warehouse is a a key part of the gathering data for your business. But what happens when the very information within changes over time? How do you maintain historic data integrity and keep accurate analysis over time? The answer is Slowly […]
Azure Data Factory Best Practices and Tips
Data integration to drive their operations and make informed decisions. Microsoft Azure Data Factory (ADF) is a cloud-based data integration service. It is available as part of Azure and also as part of the Microsoft Fabric Offering. It offers a robust and scalable solution for managing, orchestrating, and automating data pipelines. However, getting the most […]
Code vs No-Code Solutions
In the data management landscape, businesses are seeking efficient and reliable solutions to store, process, and analyse their data. Two prominent approaches have emerged as frontrunners: traditional code-based solutions and the rising trend of no-code alternatives. It is crucial to understand the each approach to make informed decisions that enable you to drive your business […]
How and Why to Modernise Your Data Reporting
This blog aims to discuss the how to modernise your data reporting from legacy reports. By legacy reporting we are referring businesses functioning with multiple disparate Excel reports. What are the benefits of moving? How do you go about moving? Why move from Legacy Reports? 1. Improved Efficiency: Over our years of experience, we have seen many […]
The Microsoft Fabric Offering
Introduction Microsoft Fabric, the unified environment launched in 2023 is now central to Microsoft’s data strategy. What is the Microsoft Fabric Offering? What is Microsoft’s competition to this software? What other unified environments are available. This blog aims to demystify some of these areas. What is the Microsoft Fabric Offering? Microsoft Fabric brings together multiple […]
Overview of Robotic Process Automation (RPA)
Introduction Robotic Process Automation (RPA) is a software technology that automates repetitive tasks using software robots. RPA is sometimes referred to as software robotics and is a form of business process automation. The purpose of RPA is to streamline workflows, reduce the time and effort required to complete manual tasks, and improve accuracy. In this […]
ETL vs. ELT: What’s the Difference?
Introduction As in any industry there is jargon but in the world of data searches for what is ETL and what is ELT leads to confusion. ETLs and ELTs are two data integration processes. Both are used to extract, transform, and load data from one or more sources into a target system. The target could […]
When to transform data
As a general principle we should always look to transform data close to its source to gain efficiencies and minimise potential for discrepancies Over the last few weeks we have shown how data can be combined from multiple sources in three different ways UNION in SQL APPEND data in Power Query UNION in DAX A […]
