What is Power BI Real Time Streaming?
Power BI Real Time Streaming refers to functionality in Power BI Service that enables users to stream data and provide data updates in real-time to your dashboards. Using this feature, you can display and update any visual, dashboards created in Power BI Service with real-time datasets. There can be different devices and data streaming sources you can connect to display information on social media trends, key service metrics, stock market updates, blood bank details, other supply chain statistics, etc. on a real-time basis using time-sensitive data collection devices.
Power BI Real Time Streaming can help organizations in multiple ways including:
- Display data insights on a real-time basis enabling business decisions in real-time
- Automate data visualization, detect changes to data patterns, and project meaningful data insights
Power BI Service supports the Real Time Streaming through three different types of real-time semantic models. These models are designed to display in tiles and dashboards and assist users in analyzing the Power BI real-time streaming data. Power BI also provides features to use streaming dataflows for connecting with Power BI real-time streaming data, performing data ingestion, mash-up, model and build visualizations based on streaming data in the Power BI Service portal. However, note that this feature has been retired in Power BI and merged with Azure Stream Analytics.
A high-level diagram displaying the Power BI Real Time Streaming is provided below
In the next section, we will go through the different data classes for real-time data streaming,
Table of contents
Key Takeaways
- Power BI Real Time Streaming enables the user to display and update any visual, dashboards created in Power BI Service with real-time datasets
- Power BI Real Time Streaming helps organizations to project critical indicators or metrics on a real-time basis to detect changes to data patterns and hence facilitates quick decision-making.
- Power BI Service supports the Real Time Streaming through three different types of real-time semantic models i.e. Push, Streaming, and PubNub streaming semantic models
- You can implement the Power BI Real Time Streaming either through Tiles with visuals from streaming data or by creating Semantic models from streaming data that persist in Power BI
- The real-time streaming involves creating a Power BI real-time streaming data, designing a report, and configuring real-time refresh settings.
- Carefully consider the nature of your data and the requirements of your analysis when opting for real-time streaming in Power BI.
Classes of datasets for real-time data streaming
For real-time data streaming, the following 3 types of semantic models or datasets are supported in Power BI Service. These include
- Push semantic model
- In this model, datasets are pushed into the Power BI service using Power BI REST APIs. This creates a semantic model resulting in the automatic creation of a new database for data storage in the Power BI Service.
- Users can create Power BI reports using the data stored in the database and apply features such as data visualization, pin dashboard tiles, set up alerts.
- Streaming semantic model
- In this model, datasets are pushed into the Power BI service through Azure Analytics Streaming. The data is stored in a temporary cache that can be used for displaying temporary visuals such as a line chart visual for 1 hour or so.
- This model does not create any database for data storage and hence is not ideal to build reports using the data stream. However, you can add a tile to the dashboard and leverage this model as a custom streaming data source
- This model is ideal for data visualization requiring latency optimization between data push and visualization.
- PubNub streaming semantic model
- In this model, Power BI Service uses the PubNub SDK for reading an existing PubNub data stream and hence no data is stored in Power BI Service.
- To read the data stream, you must ensure to list only approved outbound traffic from your network to PubNub
- This model does not create any database and is not ideal for creating reports using the data that flows into the Power BI Service. However, you can add a tile to the dashboard and apply data configuration to use the PubNub data stream as a data source
The below table highlights the various capabilities and limitations that each of these 3 semantic models have for real-time streaming.
| Capability | Push | Streaming | PubNub |
|---|---|---|---|
| Dashboard tiles update in real-time as data is pushed in | Yes. For visuals built via reports and then pinned to the dashboard. | Yes. For custom streaming tiles are added directly to the dashboard. | Yes. For custom streaming tiles are added directly to the dashboard. |
| Dashboard tiles update with smooth animations | No. | Yes. | Yes. |
| Data stored permanently in Power BI for historical analysis | Yes. | No. Data is temporarily stored for one hour to render visuals. | No. |
| Build Power BI reports on top of the data | Yes. | No. | No. |
| Max rate of data ingestion | 1 requests 16 MB/request | 5 requests 15 KB/request | N/A Data isn’t being pushed into Power BI. |
| Limits on data throughput | 1M rows/hour | None. | N/A Data isn’t being pushed into Power BI. |
In the next section, we will understand how you can implement the Power BI Real Time Streaming in Power BI Service.
How to implement Power BI Real Time Streaming?
To implement Power BI Real Time Streaming, follow the instructions outlined below:
Step 1: Choose a method to consume Power BI real-time streaming data. You can choose one of the following methods for streaming data. These methods include:
- Tiles with visuals from streaming data
- Semantic models created from streaming data that persist in Power BI
Step 2: Set up Power BI real-time streaming data regardless of what method you have selected in Step 1. To get your real-time streaming semantic model working in Power BI, select the Add a tile option In either an existing or new dashboard.
Step 3: On the Add a tile page, select Custom Streaming Data, and then select Next.
Step 4: On the Add a custom streaming data tile page, you can choose from an existing semantic model, or select Manage datasets to import your streaming semantic model if it is already created. If you want to create it from scratch, then select Add streaming semantic model option.
Step 5: On the New streaming dataset page, select one of the options from API, Azure Stream, or PubNub, and then select Next.
Step 6: If you choose the API option on the New streaming dataset screen, i.e., Power BI REST API and click on Next, you need to provide details such as Dataset name and values from stream.
If you want Power BI to store the data this data stream sends, so you can do reporting and analysis on the collected data, enable Historic data analysis.
For example, you can provide the details of the values as shown below
After you successfully create your data stream, you get a REST API URL endpoint. You can configure your application to call the endpoint by using POST requests. This will push your streaming data to the Power BI semantic model.
You must ensure that in your POST requests, the request body aligns with the sample JSON code provided by the Power BI user interface.
Step 7: If you choose PubNub on the New streaming semantic model screen and select Next, you see the following screen:
For example, you can fill in the below details such as Dataset name, Sub-key, Channel name, etc., and click on Next.
Then provide the details such as the Dataset name and Values from stream. Click on Create.
You can view the created dataset in Your datasets section.
Step 8: If you choose Azure stream, you have to set up Azure stream analytics.
Note: You can also perform Power BI real-time streaming from the SQL server by connecting your SQL Server to the DirectQuery method.
Important Things to Note
The important things to note on Power BI Real Time Streaming include:
- Validate that Power BI Real Time Streaming is supported by your data source
- If you are using the Push semantic model, consider that using the Pin live option to pin the entire report will not result in an automatic data update
- If you are using the Streaming semantic model or PubNub streaming semantic model, you may notice little latency between the data push in Power BI Service and the visual update.
- You can use card streaming visual type to view the latest numeric data in your streaming data. However, it doesn’t support any DateTime or Text data types.
- You can’t perform any data modeling on a streaming semantic model because data storage is not permanent. However, you can create measures and establish relationships.
- For the Push semantic model, you can’t clear any data from a streaming semantic model though the data itself will be cleared after 1 hour.
- Always focus on robust security and control mechanisms for real-time streaming dataset.
- If you are using automatic page refresh for your reports, consider setting up the refresh intervals for visuals that are active during the page consumption.
Frequently Asked Questions
The key Benefits of Power BI Real-Time Streaming include:
• Real-time data insights, changes in data trends, and quick identification of any hidden patterns using the streaming data
• Real-time data visualization using the datasets received in Power BI Service that helps you to monitor/track critical metrics
• You can pin tiles to dashboards that can provide live streams to end users with automatic refresh capabilities
• Reduced time cycle for business decisions using real-time data analysis and reviewing the critical KPIs or other metrics
• Ability to integrate with different data sources enabling you to perform data ingestion, mash-up, model, and build visualizations based on streaming data in the Power BI Service portal.
• You can enhance the process optimization and efficiency in allocating resources with a reduced lead time by leveraging real-time information
• Provision of setting up alerts and notifications on a real-time basis to detect any changes in underlying information or triggers.
Yes, you can customize the refresh frequency of real-time data in Power BI. You can set up the refresh frequency for visuals at a report page level that is active during the page consumption. The minimum refresh frequency is dependent on various factors such as the type of workspace used for publishing the reports, and capacity admin settings for Premium workspaces.
Power BI Real-Time Streaming supports different data sources including:
• API: Power BI REST APIs or any other sources that can send data in JSON format using API
• Azure-based services: This includes Event Hub, Stream Analytics, IoT Hub, etc.
• Third-party APIs: You can use third-party APIs such as PubNub, Twitter, Kafka, etc.
When using Power BI Real-Time Streaming, you must evaluate the following limitations or considerations:
• Consider the data volume limitations as high data volume may require infrastructure setup and result in network latency leading to degradation in streaming performance and poor user experience.
• Real-time streaming may have additional cost aspects that must be factored into your considerations.
• Setting up and managing real-time streaming can be complex and require subject matter expertise as compared to other datasets in the Power BI Service.
• Power BI Real-time streaming only supports 3 types of datasets/semantic models for real-time visualization and dashboards.
• Other than the Push semantic model, the rest of the dataset classes may experience data latency between the data push in Power BI Service and visualization.
Recommended Articles
This has been a guide to Power BI Real Time Streaming. Here we learn how to implement it, with classes datasets & examples, and points to remember. You can learn more from the following articles –
Leave a Reply