PowerShell to seach Hyper-V VM Hard Disks for a specific VHD/VHDX file path

Sometimes people may not always place the VHD/VHDX files in the path your expecting them to be which if this is on a different partition it could cause some administrative overhead when managing the disk space on your host server. I was faced with this problem and came up with the following little PowerShell script to help search all VMs for an attached disk with the path of the VHD file I found.

$VHDToFind = Read-Host "Enter the Full Path to the VHD that you want to search for?"
Get-VM | ForEach-Object {$vmHDD = Get-VMHardDiskDrive -VMName $_.Name; if ($vmHDD.Path -EQ $VHDToFind) {Write-Host "VHD attached to $($vmHDD.VMName)" -ForegroundColor Green;}}

Infrastructure Saturday 2013 - PowerShell for Azure..... a perfect partnership

The last time I presented at Infrastructure Saturday was 2010 and it is certainly great to see how the event has continued and grown. A huge thanks to everyone from contributors, sponsors, speakers and organizers, it was clearly appreciated by everyone.

So this year my original session on Automating SQLwith PowerShell had to be squashed to allow Heidi (WardyIT) to present on SQL. After shedding a tear for only a short moment I regrouped and decided to talk about PowerShell and Windows Azure, another passion of mine. So a massive thanks goes to everyone that chose my PowerShell session over the SQL Session because naturally we had to be scheduled at the same time so that I was then competing against my own primary skill set... nah lets be honest PowerShell is overtaking SQL in that way for me ;)

Ok ok enough banter.

A number of people have expressed interest in the scripts demonstrated during the session so I thought I would provide a walk-through recap of the session. The links to download the "cleansed" versions of the scripts is at the end of this post.

Session Key Takeaways

In my experience, the Azure Portal is a great tool for exploring new features or once off tasks, but due to some annoyances, like the lag for the portal to refresh when back end tasks complete, it can be time consuming when doing very large tasks or more specifically regular repeated tasks.

PowerShell overcomes this :)

Oh and one more takeaway, be careful what is a news headline at the time of your presentation otherwise you may have a docked picture of our PM and another guy kissing show on your Windows 8 Start Screen (it was quite comical though).

Links

http://www.windowsazure.com

SDKs: http://www.windowsazure.com/en-us/downloads/
Azure PowerShell: http://go.microsoft.com/?linkid=9811175&clcid=0x409
Scott Guthrie (ScottGu) Blog: http://weblogs.asp.net/scottgu

Session Demos

This is a walk through the PowerShell scripts that were demonstrated during the session.

For the first timers, you need to configure your PowerShell console session to be authorised to access your Azure subscription.

1. Download and install the Azure PowerShell pack. Reboot your computer to make it easier from here on out to import the module.
2. Import the Azure module with
   

   Import-Module Azure;
   

3. Next call the cmdlet to get the publish settings file. When you call this cmdlet it will launch your browser and ask you to log into your Azure subscription with your Live ID. It will then generate a new subscription management certificate and then prompt you to download the publishsettings file. Take note of the path where this file will be saved
   

   Get-AzurePublishSettingsFile;

   
   
4. Open the Publish Settings file and check that the Subscription ID matches the details of your management certificate as shown in the portal.
5. Next import the publishing settings file which will import the required certificate into your users certificate store.
   

   Import-AzurePublishSettingsFile –PublishSettingsFile "C:\Subscription-credentials.publishsettings"

6. You can verify that the certificate was imported correctly by traversing the certificate store:
   

   Get-ChildItem -Path Cert:\CurrentUser\my

   Or by retrieving a list of Azure Services available:
   

   Get-AzureService | Select ServiceName

Now that the management certificate has been imported you now need to bind to the Azure Subscription so that any subsequent cmdlets can authenticate without having to stipulate which subscription they should be executed against each time. This is very useful when you have more than one subscription within your organization.

1. Set the values in the variable block as required and then execute the script/command
   

   <#######################################################
   Variable Block
   #######################################################>
   
   # Subscription Information...
   $subscriptionName = "<<<<< Insert your Subscription Name here >>>>>"
   $subscriptionID = "<<<<< Insert your Subscription ID here >>>>>"
   $thumbPrint = "<<<<< Insert the Certificate Thumb Print here >>>>>"
   
   # Storage Account Information...
   $storageAccountName = "azurestorage" # Must be globally unique and all lowercase...
   $storageAccountLabel = "azurestorage"
   
   <#######################################################
   Select the Subscription
   #######################################################>
   
   #get the certificate definition from the certificate store via the PowerShell drive
   $certificate = Get-Item cert:\currentuser\my\$thumbPrint
   
   #set the context to the required subscription
   Set-AzureSubscription `
       -SubscriptionName $subscriptionName `
       -SubscriptionId $subscriptionID `
       -Certificate $certificate `
       -CurrentStorageAccount $storageAccountName
   
   #next mark the required subscription as active
   Select-AzureSubscription `
       -SubscriptionName $subscriptionName
   

TIP: The above could be configured as part of your PowerShell profile script to ensure this is set each time you start the Console, or added to a script file to easily execute, if working regularly with Azure through PowerShell.

Next we can look at the cmdlets available to us and other useful information available from within Azure:

1. Get all the cmdlets available from the Azure module

   Get-Command -Module Azure
   

2. Lets look further into the various commands available with
   

   #get all the "Get" commands
   Get-Command -Module Azure -Verb Get
   
   #get all the "Set" commands
   Get-Command -Module Azure -Verb Set
   
   #get all the "Restart" commands
   Get-Command -Module Azure -Verb Restart
   

3. When it comes time to deploy a VM we have the option of using some of the ready-made images, the following is an example of how to get a list of each of these:
   

   #Get the list of all windows images
   Get-AzureVMImage | where -property "OS" -EQ -Value "Windows" | select label,imagename | Sort -Property "label" | ft -AutoSize
   
   #what about all the images
   Get-AzureVMImage | where -property "OS" -NE -Value "Windows" | select label,imagename | Sort -Property "label" | ft -AutoSize
   

Platform as a Service

The first deployment demo during the session was to deploy a WebSite with a SQL Database. This demo used the Platform as a Service features of Azure. The script used for this deployment came from

1. Deploy the Web Site and SQL Database with the script by specifying the required parameters:
   

   #create the web site with a SQL Database (we have to use the location East Asia as Southeast Asia is not valid for web sites)
   .\New-AzureWebsitewithDB.ps1 `
       -WebSiteName "InfraSatWebSite" `
       -Location "East Asia" `
       -StorageAccountName $storageAccountName `
       -ClientIPAddress "***.***.***.***"; #validate this Client IP with WhatIsMyIP.com or the Azure Portal.
   

2. Once the deployment has completed we can see what was deployed. Firstly the new SQL Azure Server and Database

   #get the database server
   Get-AzureSqlDatabaseServer;
   
   #get the database(s) for the servers
   Get-AzureSqlDatabaseServer `
       | Get-AzureSqlDatabase;
   
   #get the database with extra info
   Get-AzureSqlDatabaseServer `
       | Get-AzureSqlDatabase -DatabaseName infrasatwebsite_db `
       | Select Name,CollationName,Edition,MaxSizeGB,Status,CreationDate `
       | Format-List;
   

3. We can also retrieve the information about the web sites created

   #get the website
   Get-AzureWebsite 

4. Thanks to the Azure management pack we can perform all sorts of operations on the services we created.

   #stop the web site
   Stop-AzureWebsite -Name InfraSatWebSite
   
   #start the web site
   Start-AzureWebsite -Name InfraSatWebSite
   
   #Default documents
   Get-AzureWebsite -Name "InfraSatWebSite" | Select DefaultDocuments;
   
   #Update the default document list for what we will publish
   Set-AzureWebsite -Name "InfraSatWebSite" -DefaultDocuments "default.aspx";
   

5. In my demo the next step here was to show how to deploy the web site code from Visual Studio through to the newly provisioned Web Site.

If you are testing this and wish to do some clean up of your Azure account following a deployment, the following commands are a good starting point.

#Remove the database (WARNING: This removes ALL databases provisioned on your account.... don't say I didn't warn you as there is no UNDO!)
Get-AzureSqlDatabaseServer | Remove-AzureSqlDatabaseServer;

#Remove the database (with a filter to exclude servers with a particular admin account)
Get-AzureSqlDatabaseServer | Where-Object -NE -Property AdministratorLogin -Value <adminaccountofserveryouwishtokeep> | Remove-AzureSqlDatabaseServer;

#remove the web site
Get-AzureWebsite -Name InfraSatWebSite | Remove-AzureWebsite;

#or
Remove-AzureWebsite -Name InfraSatWebSite;

#Check that it was removed
Get-AzureWebsite;

Infrastructure as a Service

During my session I demonstrated the automation of Infrastructure as a Service, unfortunately I cannot release the exact script that was demonstrated at the request of some colleagues until it is out of BETA. However there are plenty of scripts which perform some of the components of our complete solution.

The general concept of IaaS Automation is:

1. Deploy infrastructure components (e.g. Network, Storage, Virtual Machines, etc)
2. Simplify configuration and common tasks
3. Repetitive with minimal differences
4. Minimal user input (I recommend XML input files to achieve this)

Some existing example scripts that perform "some" of these tasks.

Deploy a Windows Azure Virtual Machine with Two Data Disks
http://gallery.technet.microsoft.com/scriptcenter/Create-a-Windows-Azure-ec901cbe

Deploy Windows Azure VMs to an Availability Set and Load Balanced on an Endpoint
http://gallery.technet.microsoft.com/scriptcenter/Create-Windows-Azure-VMs-244bd3cb

Deploy Multiple Windows Azure VMs in the Same Windows Azure Virtual Network
http://gallery.technet.microsoft.com/scriptcenter/Create-Multiple-Windows-df9e95b7

Deploy a SQL Server VM with Striped Disks in Windows Azure
http://gallery.technet.microsoft.com/scriptcenter/Create-a-SQL-Server-VM-in-0b8c6eed


At a time when the script demonstrated can be released I will update this post with references to that script.

PPT and Scripts Download

Get the PPT and the "cleansed" Scripts used during the Demos here.




Legal Stuff: As always the contents of this blog is provided “as-is”. The information, opinions and views expressed are those of the author and do not necessarily state or reflect those of any other company with affiliation to the products discussed. This includes any URLs or Tools. The author does not accept any responsibility from the use of the information or tools mentioned within this blog, and recommends adequate evaluation against your own requirements to measure suitability.
 

PowerShell: Clearing the values within a HashTable without removing construct of Keys

I had an interesting questions posed to me today:

"Is it possible to clear the contents of all the values within a HashTable in PowerShell but retaining the Key names (i.e. the construct of the hashtable)."

This sounds perfectly acceptable but unfortunately there is not native way of doing this and only have the following options:
1) Set the HashTable variable to $null
2) Rebuild the HashTable from a definition with empty values

Neither of these 2 options are very dynamic or suited the application.

The solution I thought was to loop through the HashTable keys collection and modify the value of one of the entities, however that results in:

Collection was modified; enumeration operation may not execute.

Our solution is though to use the .Clone() method of the HashTable to build a copy of it in memory just for the sake of looping through it's entities. Here is my example code:

$hash = @{FirstName="Bob";LastName="Smith";Age="23"}
$hash

#create a clone of the hash table, this avoids the 
# "Collection was modified" enumeration error
$hash2 = $hash.Clone();

#loop through each key in the cloned hastable and update the 
# value in the original table
foreach($key in $hash2.keys){$hash[$key] = '';}

#clear the cloned hashtable to free memory
$hash2 = $null;

#just output the hashtable for verification in this demo
$hash

Migration Automation Toolkit (MAT) - Best video I have seen in a while

This would have to be the best video I have seen in a while. It speaks for it self.

View Video

Obviously you need a NetApp storage device for this toolkit to work. But it is built on PowerShell so what I see is an opportunity to take the scripts and modify them for your environment even if you don't have a NetApp device there would be some use in the scripts to help you get started in automating your migrations.

Read about it http://migrate.azurewebsites.net/

Office Online feature enhancements

Some new features in Office Online makes #getitdone much easier regardless of where you and your colleagues are, but most of all regardless of what device you are using

http://blogs.office.com/b/office365tech/archive/2013/11/06/collaboration-just-got-easier-real-time-co-authoring-now-available-in-microsoft-office-web-apps.aspx

RML now supports SQL 2012 (and SQL 2014)

If you use the Replay Markup Language (RML) tool set then like me you will have been frustrated by the fact that it hadn't been updated to work with trace files captured from SQL 2012. You could probably assume that this was because the SQL Profiler Trace feature for purpose of trace capture (data engine only) is flagged for depreciation in future released (see http://technet.microsoft.com/en-us/library/ms181091.aspx).

Thankfully last week a new version was released which fully support SQL 2012 and even SQL 2014 CTP2 trace file definitions. I have just gone through the process of uninstalling my RML tools, installing the new version, and straight away I can process the trace files captured on a SQL 2012 instance. Hurray!

Be warned though you cannot use the new version of the Reporter tool to access any databases for previously processed trace data. I typically use this tool for performance analysis so I might keep the analysis databases around for a few weeks, so this just means I have to reprocess the trace files to generate reports compatible with the new version. Still worth it to get the benefit of the new supportability.


Description of the Replay Markup Language (RML) Utilities for SQL Server
http://support.microsoft.com/kb/944837

If you find that you cannot click on the navigational hyperlinks on the Main Performance Overview report (e.g. "Unique Batches", "Unique Statements", "Interesting Events", etc) then you will need to install the hotfix or ReportViewer.

Update fixes several Report Viewer issues after you install Visual Studio 2010 Service Pack 1
http://support.microsoft.com/kb/2549864
.....Or go directly to the download location at https://connect.microsoft.com/VisualStudio/Downloads/DownloadDetails.aspx?DownloadID=37409

Parsing SQL ERRORLOG's for errors with PowerShell

I've been finding the need to parse / grep the SQL Server ERRORLOG of late to look for any Errors which have occurred and for that I have been turning to my trusted friend PowerShell.

The cmdlet that gives us all the power here is Select-String

I will admit the hardest part about the use of the Select-String cmdlet for me was learning RegEx, but once you have the basics it is very powerful and this approach could be used for any number of applications.

So at the very basic level the following command can be used to report the lines with errors from the latest SQL ERRORLOG

Select-String -Path .\ERRORLOG -Pattern "(error:)\s\d{1,6}"

You could even use a wildcard in the path parameter to look for historical files, but for my need I was only interested in the current file.

Now that is all good and well but SQL Server doesn't record the description of the error on the same line as the error number, so I needed a way to pull the next line from the file as well. For this I used the powershell pipeline to then loop through each of the matched objects returned by Select-String using the Foreach-Object cmdlet and within the scriptblock of that cmdlet I first output the line we retrieved with Select-String, and then use Get-Content to pull in the details from the file and retrieve only the line number I am interested in (remember the Get-Content is a zero based array so I don't have to do anything with the line number from the Select-String matched object to get the next line.... think about it and you will understand).

Select-String -Path .\ERRORLOG.1 -Pattern "(error:)\s\d{1,6}" | Foreach-object {Write-Host "$($_.filename):$($_.linenumber):$($_.line)"; Write-Host "$($_.filename):$($_.linenumber+1):$((Get-Content "$($_.filename)")[$_.linenumber])"}

The performance of this one-line approach is not the greatest because it has to run the Get-Content cmdlet for each matched line. So if you are looking for speed this two-line approach has you covered there. The only downside is that if you change the file name you are looking at then you have to update this twice, where as the one line approach passes that value along the pipeline.

$filedata = (Get-Content .\ERRORLOG)
Select-String -Path .\ERRORLOG -Pattern "(error:)\s\d{1,6}" | Foreach-object {Write-Host "$($_.filename):$($_.linenumber):$($_.line)"; Write-Host "$($_.filename):$($_.linenumber+1):$($filedata[$_.linenumber])"}

Now if you were processing multiple files then you could do something like this multiline or scripted version.

$filelist = Get-ChildItem .\ERRORLOG*;
Foreach ($file in $filelist)
{
$filedata = (Get-Content $file.Name);
Select-String -Path $file.Name -Pattern "(error:)\s\d{1,6}" | Foreach-object {Write-Host "$($_.filename):$($_.linenumber):$($_.line)"; Write-Host "$($_.filename):$($_.linenumber+1):$($filedata[$_.linenumber])"}
} 

Of if you want to execute that from the powershell console then the two-line approach would be:

$filelist = Get-ChildItem .\ERRORLOG*;
Foreach ($file in $filelist){$filedata = (Get-Content $file.Name);Select-String -Path $file.Name -Pattern "(error:)\s\d{1,6}" | Foreach-object {Write-Host "$($_.filename):$($_.linenumber):$($_.line)"; Write-Host "$($_.filename):$($_.linenumber+1):$($filedata[$_.linenumber])"}}

Of if you are looking to summarize the number of errors occuring from all of the ERRORLOG files then the following provides an insight into that

Select-String -Path ERRORLOG* -Pattern "(error:)\s\d{1,6}" | Group-Object -Property {$_.Line.Substring(0,10)},{$_.Matches.Value} | Sort-Object Name

Happy investigations :)

If you are looking for a RegEx reference here are the ones I use:
http://www.regular-expressions.info/


Legal Stuff: As always the contents of this blog is provided “as-is”. The information, opinions and views expressed are those of the author and do not necessarily state or reflect those of any other company with affiliation to the products discussed. This includes any URLs or Tools. The author does not accept any responsibility from the use of the information or tools mentioned within this blog, and recommends adequate evaluation against your own requirements to measure suitability.