PowerShell can be used to scrape websites for parsable content. This can allow your script to make smart decisions based on http, https, ftp, or file requests available.
The cmdlet Invoke-WebRequest has been in PowerShell since version 3.0. It is able to take different parameters and return the webpage as an object. You can then use standard PowerShell parsing in order to extract the relevant data.
In the simple example embedded below, you can see that it is invoking a web request on 404TechSupport.com. It then filters the content for the tags where the class is “entry-title” and “entry-time”. It then filters it further to only the first response, so that our script returns the latest article title and published timestamp.
https://gist.github.com/jhamilton09/8332c4a3797360ae0de3d80ef6f2d2a7
Pulling a webpage into a PowerShell script might not be something you need every day but it’s a handy function for when you do need it.