Incremental refresh and real-time data for Power BI Premium datasets, with Hybrid tables
Incremental refresh and real-time data for Power BI Premium datasets, with Hybrid tables
Among the recent announcements for the new Power BI December 2021 release, the best feature introduced is the Public Preview of Hybrid Tables in Power Bi Premium.
✔Hybrid tables are large tables that have one or more partitions in Import mode and another partition in DirectQuery mode. This will make the refresh of huge tables faster with incremental refresh. Plus get the latest data in real time with Direct Query.
Incremental refresh supplemented by real-time data will allow us to take advantage of ultra-fast performance from the Import mode and view the latest real-time data reflected in the reports through the DirectQuery.
👍Import mode delivers best performance, 👎but not ideal if new data arrives at a very high cadence
👍DirectQuery mode ensures data freshness, 👎but with the loss of report performance.
==>Hybrid tables can help to strike the right balance between query performance and data freshness. it is a good solution for large tables with incremental refresh set up
✔ Incremental refresh enhances scheduled refresh tasks for huge tables in datasets that frequently load new data and grow exponentially, by automatically creating partitions upon them. The Power BI service dynamically partitions data that needs to be refreshed frequently from data that can be refreshed less frequently, based on the date/time parameters created in Power Query ( RangeStart and RangeEnd).
An incremental refresh policy is configured to only refresh the most recent import partition(s).
✔With the new "Incremental refresh and real-time data" it makes use of an additional DirectQuery partition for real-time data, which can significantly reduce the amount of data that needs to be refreshed while ensuring the display of the latest changes of the data.
So, the Hybrid tables, are an extension to the Incremental Refresh feature set, which combines of both Import Mode and DirectQuery partitions, by caching some data is in memory and some data are read directly from the data source when users interact with visuals.
==> The incremental refresh and real-time data will enable faster refreshes, fewer refresh cycles, less ressources consumption of memory for data refresh operations
We need to configure the incremental refresh policy with real time in Power BI Desktop, then publish to a Premium workspace=> Power BI will automatically partition the table as a hybrid table during data refresh.
