Do you keep track of data from a certain web page constantly? Do you need to extract data from the web? In this article, we walk you through how to configure web automation and perform web data extraction with Microsoft Power Automate Desktop.
Once you master automating web scraping with Microsoft Power Automate Desktop, you won’t have to manually navigate to the website you always keep track of and do all that manual, error-prone — and most of all unsatisfying, tiring — work: entering the search terms, navigate to the right site, copying and pasting data, and more.
Here’s what we’ll cover:
- Enabling your browser extension
- Web Automation Actions in Microsoft Power Automate Desktop
- How to Search and Extract Data from the Web
If you’re someone who prefers video instructions, the content of this article is covered in the first 7 minutes of this video on our YouTube channel.
Before we start extracting data from the web pages with Microsoft Power Automate Desktop, you’ll first need enable the extension that will allow you to do that.
Enable the Power Automate Desktop Extension for Web Automation
Before your perform web automation, you’ll need to enable the Power Automate Desktop Extension.
You can refer to this URL – https://docs.microsoft.com/en-us/power-automate/desktop-flows/using-browsers or go to Power Automate Desktop and navigate to Tools, select Browser Extension and click Google Chrome
Install the extension and click close.
That’s it! With this, you’re ready to start using the web automation actions in Microsoft Power Automate Desktop and be on your way to scraping data form the web with Microsoft Power Automate Desktop.
Web Automation Actions in Microsoft Power Automate Desktop
You can search for keywords in the action pane to find these actions that help you with web automation:
- To open a browser, use “Launch new Chrome”
- To close a browser, use “Close web browser”
- For other actions related to web automation, you can simply search “web”, and you’ll find a long list of actions you can use, such as “Press button on a web page”, “Extract data from web page”, “Get details from web pages”, and more.
How to Search and Extract Data from the Web
To illustrate how you can extract data from the web with Power Automate for desktop, we will go through a simple use case that will involve the main actions you will need to use when scraping data from a website.
Let’s create a robot that does the following:
- Asks the user to input a city name
- Opens a chrome browser and navigate to google.com
- Types in the search bar ‘weather in (city) ’, and executes a search.
- Extracts the temperature data
- Displays the temperature in a message box
Step 1: Ask the user to input a city name
Find the “Display Input Dialog” action and drag it onto the workspace.
Type “Please enter a city” on the input dialogue message and enable the option “Keep input dialog always open” to set this on top. Click Save.
Step 2: Open a Chrome Browser and Navigate to google.com
Drag the “Launch New Chrome” action to the workspace. Type www.google.com as the initial URL, Select “Maximized” as window state, and click Save.
Step 3: Type in the search bar ‘weather in (city)’, and executes a search
Before you proceed, you’ll need to navigate to www.google.com on your Chrome browser. Open a chrome browser and navigate to google.com.
Then, open Microsoft Power Automate Desktop and Click “Add UI Element”
Hover and look for , then hold down the Ctrl button and left-click to select it.
Then, search for “weather in London”.
Hover your mouse over the “Google Search” button, then hold down the Ctrl button and left-click the button to select the UI element.
After selecting the UI Element, perform a normal left-click to perform the Google search.
When the result loads, hold Ctrl button and left-click to select the UI element of the data that we want to extract, then click Done.
As a best practice, rename the UI elements selected in the previous steps with recognizable names, so you can refer back to them easily in later steps, for example we’ve used “Search Bar”, “Search Button”, and “Temperature”.
Step 4: Extract the temperature data
Before we can scrape the temperature data, the robot needs to key in the search terms — as requested by the user — into the Google search bar. Search for the “Populate text field in web page” action and drag it into the workspace.
Click UI Element and select “Search Bar”
Type “Weather in” on text field, then click {x} and choose “UserInput”, then click Save. The variable %UserInput% was the city name we asked the user for in the first step with the “Display Input Dialog” action.
Search for the “Press button on web page” action and drag it into the workspace.
Click UI element, choose “Search Button” and click Select and then Save.
When you search for web automation actions, you’ll notice that you’ll get a few actions for web extraction that sound similar.
What’s the difference among these three actions?
- Extract data from web page: to extract a single value, or it can be a table as well
- Get details of web page: to extract information such as we page description, web page title, web page meta keywords, web page descriptions, web page source, and web browser’s current URL address
- Get details of element on web page: to only extract the detail of a particular element – such as in this case, where we want to just extract the temperature number of an element.
Choose “Get details of element on web page” and drag it into the workspace. Click UI element, choose “Temperature” and click Select.
Rename the variable produced (Attribute) to “Temperature” for easy reference and click Save.
Step 5: Display the temperature in a message box
Drag the action “Display Message” into the workspace.
We want to display the sentence “Temperature in ____ (city) is ____ (temperature).”
So, type “Temperature in”, or any other message you’d like to display to the user. Click {x} to find available variables to get values from. In this case, we choose “UserInput”, which is the city name input by the user in the earlier step.
Click {x} to find the other variable – Temperature. Click Select, then if you’d like to, you can enable “close message automatically” before clicking Save.
Save the Flow, close your Chrome browser, and you’re ready to run your Flow!
What happens when you encounter an error with your UI element?
When we first ran this flow, we discovered that the Google search button was not working. To fix it, click the UI Element tab button on the extreme right side (shown below with a red box).
Click the three dots beside Search Button
Click Edit Selectors
Click on New and Select “Selector with recapture”
Hover over the Google Search button, hold Ctrl button and left click on Google Search button to re-select it.
To delete the first element that didn’t work, click the three dots beside the first element, then choose Delete.
Close the window and we’re all set! Run the Flow again, and you should be good to go!
Start automating web scraping with Microsoft Power Automate Desktop
This simple use case of scraping the temperature data from a Google search is just the tip of the iceberg — it’s just a warm-up. Keep a lookout for more videos and articles from us with more advanced web extraction tutorials using Microsoft Power Automate Desktop.
For now, get start automating web-scraping for websites that you often keep track of and save yourself all the time and effort taken to manually navigate to a website, key in search terms, copy and paste data. You’ll avoid careless mistakes too!