Invoke-WebRequest and RTFM Moments

Background

Before the weekend, I struggled with a PowerShell utility I had written a couple of months back.

To help matters, I goggled for how to capture the returned HTML Output and write it out a file. Hoping that by doing so, I will more likely find out whether I am being redirected or experiencing security, proxy or other Network errors.

 

Code

Found a simple code on the Net for capturing the returned HTML response and writing it out in its entirety to a file.

 

The code simply states that I should use the -OutFile parameter.


$htmlWebResponseObject = Invoke-WebRequest -URI $webSiteURL `
                                  -OutFile $debugFilename 

I added the code to my original file and ended up with something that looks like the following:

 


#requires -version 3.0

#Set-StrictMode

$websiteURL = 'https://www.pinterest.com/daniel_adeniji/autumn-october/'
$HTTP_SUCCESS = "200"
$FILE_EXT = "html"

$htmlWebResponseObject = $null;

function getTempFilename( [string]$extension)
{

	#How can I create a temp file with a specific extension with Net
	#http://stackoverflow.com/questions/581570/how-can-i-create-a-temp-file-with-a-specific-extension-with-net

	[string] $fileName = $null;

	#get Temp Folder
	$folderPath = [System.IO.Path]::GetTempPath()

	#get random filename
	$fileName = [System.IO.Path]::GetRandomFileName();

	# change file extension to extension (HTML)
	$fileName = [System.IO.Path]::ChangeExtension($fileName, $extension);

	# add folderpath and filename
	$fileName = [System.IO.Path]::Combine($folderPath, $fileName);	

	return $fileName
}	

try
{
	$debugFilename = getTempFilename($FILE_EXT)

	$log = "Debug File :- {0}" -f $debugFilename
	Write-Host $log -ForegroundColor white    

	# https://technet.microsoft.com/en-us/library/hh849901.aspx
	$htmlWebResponseObject = Invoke-WebRequest -URI $webSiteURL `
								-UseDefaultCredentials `
								-DisableKeepAlive `
								-OutFile $debugFilename
								#-PassThru
								#-Proxy $INTERNET_PROXY_SERVER `
								#-ProxyUseDefaultCredentials

	# Get Web Request Status Code
	$WROStatusCode = $htmlWebResponseObject.StatusCode 

	# Get Web Request Status Description
	$WROStatusDescription = $htmlWebResponseObject.StatusDescription

	# If we succcesfully connected to Web Site
	if (
		   ( $WROStatusCode -eq $HTTP_SUCCESS)
		)
	{

		$log = "We are good!, let us parse our returned Response Object"
		Write-Host $log -ForegroundColor white     

	}
	else
	{

		$logFormat = "We are not good! Our URL {0} and HTTP Status Code is {1}"
		$log = $logFormat -f $webSiteURL, $WROStatusCode
		Write-Error $log 

	}

}
catch
{

	Write-Host "_____________________________________________________________________" -ForegroundColor red

	$strLog = "Failed to connect to URL :- {0}" -f $webSiteURL
	Write-Host $strLog -ForegroundColor red

	$errLine = "Exception Type :- {0}" -f $_.Exception.GetType().FullName
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception Message :- {0}" -f $_.Exception.Message
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception Source :- {0}" -f $_.Exception.Source
	Write-Host $errLine -ForegroundColor red

	$lineNumber = $_.InvocationInfo.ScriptLineNumber
	$errLine = "Script Line Number :- {0}" -f $lineNumber
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception HResult :- {0}" -f $_.Exception.HResult
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception Inner :- {0}" -f $_.Exception.InnerException
	Write-Host $errLine -ForegroundColor red

	$errLine = "Exception HResult :- {0}" -f $_.Exception.HResult
	Write-Host $errLine -ForegroundColor red  	

	$errLine = "Exception StackTrace :- {0}" -f $_.Exception.StackTrace
	Write-Host $errLine -ForegroundColor red            

	Write-Host "_______________________________________________" -ForegroundColor red

}

But getting an error stating:

We are not good!

 

Correction

Temp Workaround

Unfortunately, the error was silent. I did not drop down to the exception handler. I simply failed to get a good .STATUSCODE.

To keep things moving, I simply removed the -OutFile parameter, added some work around code, and went on about my way.

RTFM Workaround

As I drove in to work this weekend, I waded into the bad habit of driving and multi-tasking … cleaning out my heavy work-bag full of felled trees.

The documentation clearly states that if I add the Outfile parameter, I need the Passthru parameter to be able get to the pipeline, as well.

 

Textual:

-Outfile

Saves the response body in the specified output file. Enter a path and file name. If you omit the path, the default is the current location.

By default, Invoke-WebRequest returns the results to the pipeline. To send the results to a file and to the pipeline, use the Passthru parameter.

 

Image:

-Outfile

 

 


      $htmlWebResponseObject = Invoke-WebRequest -URI $webSiteURL `
						-UseDefaultCredentials `
						-DisableKeepAlive `
						-OutFile $debugFilename
						-PassThru 

Once we added the PassThru argument, we are good; as we get the message emptied below.

 

We are good!, let us parse our returned Response Object

 

Another Error

But, as always that is not the complete story.

When one is behind a Corporate Firewall and we have to go through a Proxy Server, we can get the error pasted below:


Failed to connect to URL :- https://www.pinterest.com/daniel_adeniji/autumber/
Exception Type :- System.Net.WebException
Exception Message :- The remote server returned an error: (407) Proxy Authentication Required.
Exception Source :- Microsoft.PowerShell.Commands.Utility
Script Line Number :- 56
Exception HResult :- -2146233079
Exception Inner :-
Exception HResult :- -2146233079
Exception StackTrace :-    at Microsoft.PowerShell.Commands.WebRequestPSCmetResponse(WebRequest request)
   at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.ProcessRecord()

Will deal with this new issue in a later post.

 

 

Listening

True to the Sabbath, I had a good and restful weekend.

A good family friend, Yemi and Aja, celebrated their 10 year anniversary this weekend.

She changed outfit so many times and had a diva entrances the whole evening.

No, she did not go back to Diana Ross –  I am coming out.

But, out of many, she came out to this one

Collie Buddz – I Feel So Good
https://www.youtube.com/watch?v=n0LrXBBopPQ

 

 

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s