Breaking Blob Leases via PowerShell

We are utilizing SQL Backup to Azure blob and had a meltdown today where the log backups were erroring out leaving us with 1TB files up in Azure that were locked.  Needless to say it happened late last night and so there were multiple hourly files in multiple folder structures all over our storage accounts.  It took a bit, but the following script clears out all the locks on blobs within a container in all directories.  Please use carefully and don’t run it against your “vhds” container!

Also, it requires the Microsoft.WindowsAzure.Storage.dll assembly from the Windows Azure Storage NuGet package.  You can grab this by downloading the commandline nuget file and running the below.  Note, it will dump the file you need into .WindowsAzure.Storage.<ver>libnet40

nuget.exe install WindowsAzure.Storage

Break lease Script Below – one line modification from https://msdn.microsoft.com/en-us/library/jj919145.aspx

param(
[Parameter(Mandatory=$true)]
[string]$storageAccount,
[Parameter(Mandatory=$true)]
[string]$storageKey,
[Parameter(Mandatory=$true)]
[string]$blobContainer
)

$storageAssemblyPath = $pwd.Path + "Microsoft.WindowsAzure.Storage.dll"

# Well known Restore Lease ID
$restoreLeaseId = "BAC2BAC2BAC2BAC2BAC2BAC2BAC2BAC2"

# Load the storage assembly without locking the file for the duration of the PowerShell session
$bytes = [System.IO.File]::ReadAllBytes($storageAssemblyPath)
[System.Reflection.Assembly]::Load($bytes)

$cred = New-Object 'Microsoft.WindowsAzure.Storage.Auth.StorageCredentials' $storageAccount, $storageKey

$client = New-Object 'Microsoft.WindowsAzure.Storage.Blob.CloudBlobClient' "https://$storageAccount.blob.core.windows.net", $cred

$container = $client.GetContainerReference($blobContainer)

#list all the blobs in the container including subdirectories
$allBlobs = $container.ListBlobs($null,1)

$lockedBlobs = @()
# filter blobs that are have Lease Status as "locked"
foreach($blob in $allBlobs)
{
$blobProperties = $blob.Properties
if($blobProperties.LeaseStatus -eq "Locked")
{
$lockedBlobs += $blob

}
}

if ($lockedBlobs.Count -eq 0)
{
Write-Host " There are no blobs with locked lease status"
}
if($lockedBlobs.Count -gt 0)
{
write-host "Breaking leases"
foreach($blob in $lockedBlobs )
{
try
{
$blob.AcquireLease($null, $restoreLeaseId, $null, $null, $null)
Write-Host "The lease on $($blob.Uri) is a restore lease"
}
catch [Microsoft.WindowsAzure.Storage.StorageException]
{
if($_.Exception.RequestInformation.HttpStatusCode -eq 409)
{
Write-Host "The lease on $($blob.Uri) is not a restore lease"
}
}

Write-Host "Breaking lease on $($blob.Uri)"
$blob.BreakLease($(New-TimeSpan), $null, $null, $null) | Out-Null
}
}

 

Adding Additional Azure Disks to an VM's StoragePool that is part of a SQL Server AlwaysOn Cluster

To get the optimal performance out of your Azure VMs running SQL servers, MS recommends to use Storage Spaces and stripe multiple Azure disks[1]. The nice thing about storage pools in Storage Spaces is that it allows you to add disks behind the scenes without impacting the actual volume.

Now lets say you have a SQL AlwaysOn cluster (2+ nodes), and for performance reasons (IOPS) you realize that you need to add more disks.  As Storage Spaces shows all disks (physical, virtual, and storagepools) across the whole cluster, it is possible you won’t be able to simply add them due to naming mismatch.  Fear not though, it is still possible if you follow the steps below:

  • Add the new disks to the VM
  • Log into the VM
  • Failover SQL to a secondary if the current VM is the primary
  • Stop clustering service on the VM
  • Run Get-PhysicalDisks to get the disknames
  • Run Add-PhysicalDisk -StoragePoolFriendlyName <storagepool> -PhysicalDisks (Get-PhysicalDisk -FriendlyName <disks>)
  • Run Update-HostStorageCache (if we don’t do this sometimes the volume resize doesn’t work)
  • Run Resize-VirtualDisk -FriendlyName <diskName> -Size <size>
  • Run Update-HostStorageCache (if we don’t do this sometimes the disk resize doesn’t work)
  • Run Resize-Partition -Size <size> -DriveLetter <letter>
  • Start the clustering service on the machine
  • Failback SQL to the VM if required

Hopefully this helps someone as we were beating our heads in for quite a few days (along with MS).

[1] https://msdn.microsoft.com/en-us/library/azure/dn133149.aspx

Surface Pro 3 Function Keys

One of the bigger things I see as a complaint about the Surface Pro 3 (SP3) is that by default the Function keys are not the primary button press.  Instead the defaults are the shortcut keys.  Additionally, it’s not really documented anywhere how to switch what is primary.

Well, you can.  Just use CAPS + Fn to switch between what you want to be primary.

Unfortunately, I use a lot of the Fn keys, but also Home and End.  Oh well, guess I can’t have my cake and eat it too.

Bliss & Music Management

Another one of the great things about using a Synology is having access to all the great packages that both Synology and the community create.  You find out about so many different apps you may not have been familiar with.  Turns out Bliss was one of those (grab the app from Patters).

I have a fairly large music library I’ve composed over the years, and while I was picky about folder structure, all the other stuff was lacking: tags, images, etc.  Back in July of 2012 I gave it a first run and liked what it did.  However, I didn’t keep it installed (didn’t think I would need it continually).

Fast forward to last week, and I reinstalled it to help clean things up again – I had to restore lots of my music which meant the tags, folder structure, etc. were all messed up again.  I’m loving it all over again for it’s ease of use and continual running in the background.

Turns out in the current release they’ve added some new case rules which I love, but was having a few issues with it.  Shot of an email with some of the details, and got an immediate response saying it was being fixed in the next release that was in beta.  I asked if they wanted any help with testing, and sure enough Dan (creator) gave me links to the beta which I promptly installed (replaced the bliss-bundle folder after you extract the Linux jar file).

Can’t say enough about the customer service, plus it’s fixing all my music issues in the background.  Yay!

Sender Policy Framework (SPF) Records

With the move of my domains to Azure instead of Dreamhost (oh, did I not mention that?), one of the things I needed to do was setup some SPF records to keep email from going into junk folders.

Now, there is a great article on creating SPF records, but those that know me know I’m not a big reader when there is a lot of text.  Pictures FTW!

Anyways, as MS is a big proponent of utilizing SPF, they have created an excellent tool to create the record’s text.  Yay team!

Live Domain Issues

I’ve finally migrated my email from google apps to outlook.com.  The key item that caused some issues was using domains.live.com.  I had added the MX record that proved I owned it (and email was flowing correctly).

However, it was always set to “pending” and never switched to “active”.  The problem is that the Hotmail.com MX record is the only MX record you can have.  I had some of the old gmail records listed still, but at a much lower priority (higher number).  Removing them immediately made it switch to active.

Grr.

Synology L2TP/IPSec VPN

In the last few releases, Synology has added L2TP/IPSec as an option for a VPN.  I’ve never been able to get it to work on a Windows client until today.

Ports Required:

  • IPSec – UPD 500
  • IPSec NAT Traversal – UDP 4500
  • L2TP – UDP 1701

Fix on Windows clients – http://support.microsoft.com/kb/926179

  • For Win7 and Win8 Machines
    • HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesPolicyAgent
    • New DWORD (32-bit) – AssumeUDPEncapsulationContextOnSendRule
    • Set the Value Data to 2

Update:

Using OpenVPN with port 443, for easier access: http://forum.synology.com/enu/viewtopic.php?t=45545#p215354

Windows 8.1 Preview Initial Thoughts

I finally got around to installing Windows 8.1 Public Preview.  I am definitely liking the new digs.  However, I already started to look into how to uninstall due to the following items that I’ve now gotten sorted out (hopefully helps someone else):

  1. I couldn’t connect to any of my local network resources: iSCSI targets, NAS shares, NAS website.  However, I could ping them and SSH just fine.  I thought it was something to do with the iSCSI targets because whenever I tried to enable/disable task manager showed 5MB+ sustained traffic flowing over the network.  Obviously not a good thing.  Well, it turns out it was the “Microsoft LLDP Protocol Driver” on the network connection.  The picture below is what I unchecked to make everything start working again including being able to restart the machine.  And yes, while this is a Hyper-V virtual adapter, similar problems were happening when I wasn’t using Hyper-V (trust me, I tried everything!).Microsoft LLDP Protocol Driver
  2. Windows + S no longer works for OneNote screen clipping!!!!11one  I get that it makes sense for it to go to Search now, but dammit Microsoft, stop messing with my shortcuts!  First it was Windows + Q moving from lync to search (which also brings up search in 8.1 Preview), and now Windows + S is taken from me too!  Grrr

Oh well, with the bad comes the good:

  1. Love the desktop wallpaper being in metro
  2. Love the auto-complete for when I launch an app
  3. Love the minimalistic Start button
  4. I am enjoying IE 11, but we’ll see how long that lasts
  5. One I get the RT tab updated, the labelling of groups will be nice

Overall, seems to be good now that I’ve surpassed the biggest hurdle.

 

Bash Pass Variables Between Scripts

I have an app that can call a single script, sending it some variables.  Unfortunately, I needed to run multiple scripts, but both utilizing the same variables.  With bash, there is an easy way to do that, just append “$@” to the script.

For example, the application calls ‘combined_scripts.sh’ with variables $1 through $7.  The contents of ‘combined_scripts.sh’ is below and the variables are passed through to script1 and script2.

#!/bin/sh
./script1.sh "$@"
python ./script2.py "$@"