How to Extract Data from Web Pages in Microsoft Power Automate Desktop

Do you keep track of data from a certain web page constantly? Do you need to extract data from the web? In this article, we walk you through how to configure web automation and perform web data extraction with Microsoft Power Automate Desktop.

Once you master automating web scraping with Microsoft Power Automate Desktop, you won’t have to manually navigate to the website you always keep track of and do all that manual, error-prone — and most of all unsatisfying, tiring — work: entering the search terms, navigate to the right site, copying and pasting data, and more.

Here’s what we’ll cover:

If you’re someone who prefers video instructions, the content of this article is covered in the first 7 minutes of this video on our YouTube channel.

Before we start extracting data from the web pages with Microsoft Power Automate Desktop, you’ll first need enable the extension that will allow you to do that.

 

Enable the Power Automate Desktop Extension for Web Automation

Graphical user interface, application Description automatically generated

Before your perform web automation, you’ll need to enable the Power Automate Desktop Extension.

You can refer to this URL – https://docs.microsoft.com/en-us/power-automate/desktop-flows/using-browsers or go to Power Automate Desktop and navigate to Tools, select Browser Extension and click Google Chrome

Graphical user interface, application, Word Description automatically generated

Install the extension and click close.

Graphical user interface, application Description automatically generated

That’s it! With this, you’re ready to start using the web automation actions in Microsoft Power Automate Desktop and be on your way to scraping data form the web with Microsoft Power Automate Desktop.

 

Web Automation Actions in Microsoft Power Automate Desktop

You can search for keywords in the action pane to find these actions that help you with web automation:

  • To open a browser, use “Launch new Chrome”
  • To close a browser, use “Close web browser”
  • For other actions related to web automation, you can simply search “web”, and you’ll find a long list of actions you can use, such as “Press button on a web page”, “Extract data from web page”, “Get details from web pages”, and more.

Graphical user interface, application, Word Description automatically generated

 

How to Search and Extract Data from the Web

To illustrate how you can extract data from the web with Power Automate for desktop, we will go through a simple use case that will involve the main actions you will need to use when scraping data from a website.

Let’s create a robot that does the following:

  1. Asks the user to input a city name
  2. Opens a chrome browser and navigate to google.com
  3. Types in the search bar ‘weather in (city) ’, and executes a search.
  4. Extracts the temperature data
  5. Displays the temperature in a message box

 

Step 1: Ask the user to input a city name

Find the “Display Input Dialog” action and drag it onto the workspace.

Type “Please enter a city” on the input dialogue message and enable the option “Keep input dialog always open” to set this on top. Click Save.

Graphical user interface, application, Word Description automatically generated

 

Step 2: Open a Chrome Browser and Navigate to google.com

Drag the “Launch New Chrome” action to the workspace. Type www.google.com as the initial URL, Select “Maximized” as window state, and click Save.

Graphical user interface, application Description automatically generated

 

Step 3: Type in the search bar ‘weather in (city)’, and executes a search

Before you proceed, you’ll need to navigate to www.google.com on your Chrome browser. Open a chrome browser and navigate to google.com.

Graphical user interface, application, Teams Description automatically generated

Then, open Microsoft Power Automate Desktop and Click “Add UI Element”

Graphical user interface, application, Word Description automatically generated

Hover and look for , then hold down the Ctrl button and left-click to select it.

A screenshot of a computer Description automatically generated with medium confidence

Then, search for “weather in London”.

A screenshot of a computer Description automatically generated with medium confidence

Hover your mouse over the “Google Search” button, then hold down the Ctrl button and left-click the button to select the UI element.

After selecting the UI Element, perform a normal left-click to perform the Google search.

A screenshot of a computer Description automatically generated with medium confidence

When the result loads, hold Ctrl button and left-click to select the UI element of the data that we want to extract, then click Done.

Graphical user interface, text, application, Teams Description automatically generated

As a best practice, rename the UI elements selected in the previous steps with recognizable names, so you can refer back to them easily in later steps, for example we’ve used “Search Bar”, “Search Button”, and “Temperature”.

Graphical user interface, text, application Description automatically generated

 

Step 4: Extract the temperature data

Before we can scrape the temperature data, the robot needs to key in the search terms — as requested by the user — into the Google search bar. Search for the “Populate text field in web page” action and drag it into the workspace.

Click UI Element and select “Search Bar”

Graphical user interface, application Description automatically generated

Type “Weather in” on text field, then click {x} and choose “UserInput”, then click Save. The variable %UserInput% was the city name we asked the user for in the first step with the “Display Input Dialog” action.

Graphical user interface, application Description automatically generated

Search for the “Press button on web page” action and drag it into the workspace.

Click UI element, choose “Search Button” and click Select and then Save.

Graphical user interface, application Description automatically generated

When you search for web automation actions, you’ll notice that you’ll get a few actions for web extraction that sound similar.

What’s the difference among these three actions?

  • Extract data from web page: to extract a single value, or it can be a table as well
  • Get details of web page: to extract information such as we page description, web page title, web page meta keywords, web page descriptions, web page source, and web browser’s current URL address
  • Get details of element on web page: to only extract the detail of a particular element – such as in this case, where we want to just extract the temperature number of an element.

Choose “Get details of element on web page” and drag it into the workspace. Click UI element, choose “Temperature” and click Select.

Graphical user interface, application Description automatically generated

Rename the variable produced (Attribute) to “Temperature” for easy reference and click Save.

Graphical user interface, application Description automatically generated

 

Step 5: Display the temperature in a message box

Drag the action “Display Message” into the workspace.

We want to display the sentence “Temperature in ____ (city) is ____ (temperature).”

So, type “Temperature in”, or any other message you’d like to display to the user. Click {x} to find available variables to get values from. In this case, we choose “UserInput”, which is the city name input by the user in the earlier step.

Graphical user interface, application, Word Description automatically generated

Click {x} to find the other variable – Temperature. Click Select, then if you’d like to, you can enable “close message automatically” before clicking Save.

Graphical user interface, application Description automatically generated

Save the Flow, close your Chrome browser, and you’re ready to run your Flow!

 

What happens when you encounter an error with your UI element?

When we first ran this flow, we discovered that the Google search button was not working. To fix it, click the UI Element tab button on the extreme right side (shown below with a red box).

Graphical user interface, text, application, email Description automatically generated

Click the three dots beside Search Button

Graphical user interface, text, application, email Description automatically generated

Click Edit Selectors

Graphical user interface, text, application, email Description automatically generated

Click on New and Select “Selector with recapture”

Graphical user interface, application Description automatically generated

Hover over the Google Search button, hold Ctrl button and left click on Google Search button to re-select it.

Graphical user interface, application Description automatically generated

To delete the first element that didn’t work, click the three dots beside the first element, then choose Delete.

Graphical user interface, text, application Description automatically generated

Close the window and we’re all set! Run the Flow again, and you should be good to go!

 

Start automating web scraping with Microsoft Power Automate Desktop

This simple use case of scraping the temperature data from a Google search is just the tip of the iceberg — it’s just a warm-up. Keep a lookout for more videos and articles from us with more advanced web extraction tutorials using Microsoft Power Automate Desktop.

For now, get start automating web-scraping for websites that you often keep track of and save yourself all the time and effort taken to manually navigate to a website, key in search terms, copy and paste data. You’ll avoid careless mistakes too!