Microsoft Fabric has rolled out some exciting new features recently. Some of the standout capabilities that data engineers/architects might find interesting:
- OneLake unified security: Define data access once, enforce everywhere. A new OneLake security model lets you set granular permissions (including row- and column-level security) in one place and have them consistently applied across all Fabric engines (SQL/Spark, etc). This simplifies governance by removing the need to duplicate security rules in each service.
- Fabric Data Agents + Azure AI integration: Your data estate meets generative AI. Fabric’s AI-powered data agents (formerly “AI skills”) can now integrate with Azure AI Foundry to create custom conversational assistants that truly understand your enterprise data. These agents retrieve and reason over data in OneLake and, using Azure AI Agent Service, deliver accurate, context aware responses grounded in your domain knowledge. In short, you can build ChatGPT like agents for your organization’s data to drive insights and automation.
- Embedded AI functions for data engineering: LLM power with one line of code. A new set of AI Functions (preview) lets you apply large language model capabilities directly in Fabric data pipelines. For example, you can call functions to summarize text, classify content, translate, or analyze sentiment as part of a SQL query or Spark job all with a simple one-liner. This dramatically improves productivity by bringing generative AI into your ETL/ELT processes without complex setup.
- Direct Lake mode in Power BI (preview): Near real-time analytics without data moves. Fabric introduced Direct Lake semantic models in Power BI, allowing datasets to query data directly from OneLake without needing import or refresh cycles. You can build lightning-fast reports on huge datasets without duplication and even combine tables from multiple Fabric sources in one model. This is a game-changer for BI pros enabling up-to-date insights with zero data prep lag.
- Native Apache Airflow integration: Orchestrate with familiar tools, no infrastructure needed. Fabric now offers a built-in Apache Airflow runtime (GA) so you can run your Airflow DAGs natively in Fabric with a serverless execution model. This means you can leverage Airflow’s powerful workflow orchestration for your Fabric pipelines and data engineering tasks, without managing separate Airflow servers streamlining complex data workflows.
- New DevOps and automation tools: Better CI/CD and code-first control. For engineering teams, Fabric launched a CLI in preview, enabling you to script and automate tasks across the platform (no more manual clicking). Additionally, a Terraform provider for Fabric is now generally available, so you can manage Fabric workspaces and items as code, ensuring consistent deployments. These tools significantly improve deployment automation and integrate Fabric into your existing DevOps pipelines.
Each of these features opens up new possibilities from tighter security and governance, to AI driven analytics, to seamless integration with the tools data teams love. it’s a great time to explore these updates and consider how they can elevate your data projects.