In the past, I’ve written a few blog posts about setting up different types of VPNs with Azure.
- Azure Point-to-Site VPN with RADIUS Authentication « The Tech L33T
- Azure Web Apps with Cost Effective, Private and Hybrid Connectivity « The Tech L33T
- Azure Site-to-Site VPN with PFSense « The Tech L33T
Since the market is now full of customers who are running Palo Alto Firewalls, today I want to blog on how to setup a Site-to-Site (S2S) IPSec VPN to Azure from an on-premises Palo Alto Firewall. For the content in this post I’m running PAN-OS 10.0.0.1 on a VM-50 in Hyper-V, but the tunnel configuration will be more or less the same across deployment types (though if it changes in a newer version of PAN-OS let me know in the comments and I’ll update the post).
Alright, let’s jump into it! The first thing we need to do is setup the Azure side of things, which means starting with a virtual network (vnet). A virtual network is a regional networking concept in Azure, which means it cannot span multiple regions. I’m going to use “East US” below, but you can use whichever region makes the most sense to your business since the core networking capabilities shown below are available in all Azure regions.
With this configuration I’m going to use 10.0.0.0/16 as the overall address space in the Virtual Network, I’m also going to configure two subnets. The “hub” subnet is where I will host any resources. In my case, I’ll be hosting a server there to test connectivity across the tunnel. The “GatewaySubnet” is actually a required name for a subnet that will later house our Virtual Network Gateway (PaaS VPN Appliance). This subnet could be created later in the portal interface for the Virtual Network (I used this method in my PFSense VPN blog post), but I’m creating it ahead of time. Note that this subnet is name and case sensitive. The gateway subnet does not need a full /24, (requirements for the subnet here), it will do for my quick demo environment.
Now that we have the Virtual Network deployed, we need to create the Virtual Network Gateway. You’ll notice that once we choose to deploy it in the “vpn-vnet” network that we created, it will automatically recognize the “GatewaySubnet” and will deploy into that subnet. Here we will choose a VPN Gateway type, and since I’ll be using a route-based VPN, select that configuration option. I won’t be using BGP or an active-active configuration in this environment so I’ll leave those disabled. Validate, and create the VPN Gateway which will serve as the VPN appliance in Azure. This deployment typically takes 20-30 minutes so go crab a cup of coffee and check those dreaded emails.
Alright, now that the Virtual Network Gateway is created we want to create “connection” to configure the settings needed on the Azure side for the site-to-site VPN.
Here we’ll name the connection, set the connection type to “Site-to-Site (IPSec)”, set a PSK (please don’t use “SuperSecretPassword123″…) and set the IKE Protocol to IKEv2. You’ll notice that you need to set a Local Network Gateway, we’ll do that next.
Let’s go configure a new Local Network Gateway, the LNG is a resource object that represents the on-premises side of the tunnel. You’ll need the public IP of the Palo Alto firewall (or otherwise NAT device), as well as the local network that you want to advertise across the tunnel to Azure.
Once that’s complete we can finish creating the connection, and see that it now shows up as a site-to-site connection on the Virtual Network Gateway, but since the other side isn’t yet setup the status is unknown. If you go to the “Overview” tab, you’ll notice it has the IP of the LNG you created as well as the public IP of the Virtual Network Gateway – you will want to copy this down as you’ll need it when you setup the IPSec tunnel on the Palo Alto.
Alright, things are just about done now on the Azure side. The last thing I want to do is kick off the deployment of a VM in the “hub” subnet that we can use to test the functionality of the tunnel. I’m going to deploy a cheap B1s Ubuntu VM. It doesn’t need a public IP and a basic Network Security Group (NSG) will do since there is a default rule that allows all from inside the Virtual Network (traffic sourced from the Virtual Network Gateway included).
Now that the test VM is deploying, let’s go deploy the Palo Alto side of the tunnel. The first thing you’ll need to do is create a Tunnel Interface (Network –> Interfaces –> Tunnel –> New). In accordance with best practices, I created a new Security Zone specifically for Azure and assigned that tunnel interface. You’ll note that it will deploy a sub interface that we’ll be referencing later. I’m just using the default virtual router for this lab, but you should use whatever makes sense in your environment.
Next we need to create an IKE Gateway. Since we set the Azure VNG to use IKEv2, we can use that setting here also. You want to select the interface that is publicly-facing to attach the IKE Gateway, in my case it is ethernet 1/2 but your configuration may vary. Typically you’ll have the IP address of the interface as an object and you can select that in the box below, but in my case my WAN interface is using DHCP from my ISP so I leave it as “none”.
It is important to point out though, that if your Palo Alto doesn’t have a public IP and is behind some other sort of device providing NAT, you’ll want to use the uplink interface and select the “local IP address” private IP object of that interface. I suspect this is an unlikely scenario, but I’ll call it out just in case.
The peer address is the public IP address of the Virtual Network Gateway of which we took note a few steps prior, and the PSK is whatever we set on the connection in Azure. Lastly, make sure the Liveness Check is enabled on the Advanced Options Screen.
Next we need an IPSec Crypto Profile. AES-256-CBC is a supported algorithm for Azure Virtual Network Gateways, so we’ll use that along with sha1 auth and set the lifetime to 8400 seconds which is longer than lifetime of the Azure VNG so it will be the one renewing the keys.
Now we put it all together, create a new IPSec Tunnel and use the tunnel interface we created, along with the IKE Gateway and IPSec Crypto Profile.
Now that the tunnel is created, we need to make appropriate configurations to allow for routing across the tunnel. Since I’m not using dynamic routing in this environment, I’ll go in and add a static route to the virtual router I’m using to advertise the address space we created in Azure to send out the tunnel interface.
Great! Now at this point I went ahead and grabbed the IP of the Ubuntu VM I created earlier (which was 10.0.1.4) and did a ping test. Unfortunately they all failed, what’s missing?
Yes yes, I did commit the changes (which always seems to get me) but after looking at the traffic logs I can see the deny action taking place on the default interzone security policy. Yes I could have not mentioned this, but hey, now if it doesn’t work perfectly for the first time for you – you can be assured you’re in good company.
Alright, if you recall we created the tunnel interface in its own Security Zone so I’ll need to create a Security Policy from my Internal Zone to the Azure Zone. You can use whatever profiles you need here, I’m just going to completely open interzone communication between the two for my lab environment. If you want machines in Azure to be able to initiate connections as well remember you’ll need to modify the rule to allow traffic in that direction as well.
Here we go, now I should have everything in order. Let’s go kick off another ping test and check a few things to make sure that the tunnel came up and shows connected on both sides of things. It looks like the new Allow Azure Security Policy is working, and I see my ping application traffic passing!
Before I go pull up the Windows Terminal screen I want to quickly check the tunnel status on both sides.
Success!!! Before I call it, I want to try a two more things so I’ll SSH into the Ubuntu VM, install Apache, edit the default web page and open it in a local browser.
At this point I do want to call out the troubleshooting capabilities for Azure VPN Gateway. There is a “VPN Troubleshoot” functionality that’s a part of Azure Network Watcher that’s built into the view of the VPN Gateway. You can select the gateway on which you’d like to run diagnostics, select a storage account where it will store the sampled data, and let it run. If there are any issues with the connection this will list them out for you. It will also list some specifics of the connection itself so if you want to dig into those you can go look at the files written to the blob storage account after the troubleshooting action is complete to get information like packets, bytes, current bandwidth, peak bandwidth, last connected time, and CPU utilization of the gateway. For further troubleshooting tips you can also visit the documentation on troubleshooting site-to-site VPNs with Azure VPN Gateways.
That’s it, all done! The site-to-site VPN is all setup. The VPN Gateway in Azure makes the process very easy and the Palo Alto side isn’t too bad either once you know what’s needed for the configuration.
If you have any questions, comments, or suggestions for future blog posts please feel free to comment blow, or reach out on LinkedIn or Twitter. I hope I’ve made your day a little bit easier!
*Edit* 6/9/20: At the time of writing this blog custom roles were not available in the Azure Portal. Since that feature now exists, I’ve added how to create this role through the portal to the end of this blog.
We all know that cloud environments are different than on premises. Development environments can be even more difficult, the right cross between giving freedom to produce business value and enough security and controls to maintain a good security posture. Recently I came across a scenario where the design was that the networking components (Virtual Network, S2S VPN, Peerings, Service Endpoints, UDRs, Subnets, etc.) were all under the control of a Networking team so that the Azure environment would be an extension of their local network. From a permissions standpoint though this caused some problems. The goal would be to empower developers to build whatever they need and just attach it to the vNet when connectivity was needed, with the caveat that they shouldn’t be able to modify any of the network settings or configurations in the shared vNet. This however, turned out not to be possible with default IAM roles.
In my testing, even though you’re not “modifying” anything per se in the vNet – when attaching a network interface card to a Subnet it does require write access. Reader access will give the user this error.
After looking through the default IAM roles, there aren’t any that do what I need them to do. Alas, a custom role is needed. For reference, please checkout this docs page for custom roles – https://docs.microsoft.com/en-us/azure/role-based-access-control/custom-roles.
First, I need to see what actions are available to pack into the custom role so I run the following command in Azure Powershell and get these results.
Get-AzProviderOperation “Microsoft.Network/virtualNetworks/*” | FT OperationName, Operation, Description -AutoSize
Great, now I can see what is available. It looks like the actions that I need are the /subnet/join/action and /subnet/joinViaServiceEndpoint/Action. Based on the descriptions these two will essentially give “operator” role to the vnet. The assignee will be able to use the subnet(s) but not able to modify them.
Next, you can use one of the template references in the Microsoft Docs link (https://docs.microsoft.com/en-us/azure/role-based-access-control/custom-roles-powershell#create-a-custom-role-with-json-template) and modify the actions.
“Name”: “Custom – Network Operator Role”,
“Description”: “Allows for read access to Azure network and join actions for service endpoints and subnets.”,
After modifying the assignable subscription IDs, save that as a .json file and use it in the following command to import the custom role.
New-AzRoleDefinition -InputFile “C:\FileFolderLocationPath\CustomNetworkOperatorRole.json”
After a portal refresh you get the custom role available as an IAM role assignment.
There you go, after assigning this role the user was able to create VMs and attach them to the Virtual Network while still leaving control of the Network configuration to the Networking Team.
EDIT 6/9/20: Now that custom roles are available in the portal I’ve added how to create this role that way below.
Using the documentation on how to create custom IAM roles in the Azure portal you can clone any role, but I’m just going to use the “Network Contributor” role to clone.
I am going to just “start from scratch”, though you can easily use this page to modify the permissions set of a certain role that already exists. Additionally, if you have the JSON (like we generated before with powershell) you can just upload that here.
Now go ahead and add new permissions by going to the “Microsoft Network” Category and selecting the same permissions we wrote in the JSON and used with the powershell command above.
After adding those the next page will show you the permission that you’ve added, let’s double check those and make sure we have what we need.
Looks good, now we can set a scope where this role can be applied. I have this one scoped to the whole subscription, but you can use the interface to easily change that on this page.
Alright, that’s about it – let’s take a quick look at the summary pages. You can see that the portal actually generates the JSON, which is nice if you want to assign roles using Azure Policy or any other automation tooling.
Nice! Now we can see that users can use the four above actions which will allow them to interact with, and use a shared hub-style virtual network while now allowing full contributor access.
I hope I’ve made your day at least a little bit easier!
If you have any questions or suggestions for future blog posts feel free to comment below, or reach out to me via email, twitter, or LinkedIn.
Reading Time: 2 minutesIn a recent design session, I had someone mention how they wish that Azure had dedicated and/or isolated IaaS VMs like some of the other cloud providers. They were shocked with I said – they do! Azure has a number of isolated VM types that you can provision and leverage just the same as any other IaaS VM.
The link to Microsoft documentation below states that “Azure Compute offers virtual machine sizes that are Isolated to a specific hardware type and dedicated to a single customer. These virtual machine sizes are best suited for workloads that require a high degree of isolation from other customers for workloads involving elements like compliance and regulatory requirements.”
Yup, you read that right, dedicated hardware. This means that using one of the isolated VM types (listed below) will guarantee that your virtual machine will be the only one running on that server instance (as of Nov. 1, 2018).
If VM isolation is what you’re after, you can also consider Nested Virtualization in Azure (link below). A few VM types in Azure support Nested Hyper-V (not suitable for production, mind you) and Hyper-V Containers using Docker.
Lastly, there is a new feature that recently went GA that’s sort of in-line with VM isolation – Confidential Compute. While this isn’t VM isolation, it is an isolated enclave for executing code in TEEs (Trusted Execution Environments) using Intel’s SGX capabilities. This allows for code to be executed in an environment that, while it’s being executed, can not be accessed by any source outside of the enclave (see diagrams in the links below).
Using physical isolation techniques in the cloud can be very different than the way we’re used to doing it on-prem. Hopefully, though, this post and its associated links will get you thinking about different ways you can design for isolation in Azure. Feel free to comment, or reach out on Twitter @MattHansen0.
Thanks, and I hope I’ve made your day at least a little bit easier.
Reading Time: 2 minutesI did a few new SCOM 2012 installs recently and noticed that after pushing the agent to the DCs, they showed up grayed out in Ops Manager. Here’s a quick tip on how to fix that.
Logon to the DC(s), and with an administrative comand prompt run the HSLockdown tool, and add the local system account to the allowed group.
C:\Program Files\System Center Operations Manager\Agent:
*NOTE* In newer version, this is now stored in “C:\Program Files\Microsoft Monitoring Agent\Agent”
Run the command “HSLockdown /L” to show what accounts are being allowed or denied. In this case, my local system isn’t even populated.
Now run the HSLockdown tool again with the add switch to allow local system.
“HSLockdown /A “NT AUTHORITY\SYSTEM”
Restart the agent with “net stop heathservice && net start healthservice” and give it 5 minutes or so then it should be all green in your dashboard.
Hope this made your day at least a little easier!
Reading Time: 3 minutesA few days ago the phone rings, I get an ear-full about how some application isn’t working correctly and how it’s all the network’s fault and the repercussions of this outage will possibly cause so much damage that the world will start turning…the OTHER DIRECTION. Unfortunately for us IT Professionals, this is all too common of an occurrence. Nonetheless, I jumped in to see what I could do. I had never seen this application before so I had to start troubleshooting from the ground up. Very quickly I noticed it was running (or supposed to be running) over web protocols, so I whipped out the handy-dandy wireshark to get a look. Hm…it establishes a TLSv1 tunnel and shoots all the data at the server that way. Well, the Apps team was no where to be found so I had to find out what was moving across the wire here to figure out the issue. This is where fiddler comes in to play *Trumpets Fanfare*.
Fiddler is a fantastic little tool that does different things with packet captures and things of the sort. For this blog, I want to talk about its’ ability to man in the middle your own machine to provide visibility into an encrypted tunnel. Lets do a little demonstration here.
I’ve done a quick search in on bing, using HTTPS — thing fancy here at all.
I started fiddler prior to performing the search above, and this is what it shows up with, a whole bunch of nothing. Tunnel Tunnel Tunnel Tunnel…dang security.
Alas, fiddler has an option to man in the middle yourself and decrypt the tunnel! Just go to Tools > Fiddler Options > HTTPS > and check the box that says “Decrypt HTTPS traffic”. I chose browsers only for this demonstration, though you can do all traffic for other uses and applications.
It lets you know that you’re doing something that defies the laws of CAs.
Now here we go, re-launch the browser and go to https://bing.com, it throws a security error stating that the certificate is untrusted.
For this to work, you will need to add the exception, if you view the cert you can see that it was assigned to fiddler, when it’s clearly stating that it is for bing.com
Once that is all excepted, you can do the same search we did before — plain and simple.
Back to Fiddler, and ta-da! Congratulations, you’ve bypassed the security of your own data and now have visibility into the tunnel.
That’s it, very simple. You can view inside your SSL/TLS tunnel using fiddler in just a few simple steps. Side note, I was able to use that to determine what was happening on the wire for my application failure and was able to remedy the failure.
I hope I’ve made your day at least a little bit easier!
Reading Time: 3 minutesOne of the things you often find yourself thinking is, hmm…I should probably test my windows machines for security flaws, right? I’ve decided to share some very good tools for testing security from basic button clicking to advanced security testing.
As it relates to Windows-based computers, there are seven general types of security testing tools. These are:
- Port scanners
- Network/OS vulnerability scanners
- Application/database vulnerability scanners
- Password crackers
- File searching tools
- Network analyzers
- Exploit tools
All of these types of tools can and should be used when performing penetration tests, vulnerability assessments, and security audits on your Windows systems.
For the most part with security tools, you get what you pay for. There are, however, a handful of free tools that are a solid choice.
– Super Scan v3: Very fast and easy to use port scanner that can find live systems, look for open ports and running services, grab banner information including software versions.
– SoftPerfect Network Scanner: Maps MAC addresses to IP addresses which can help you locate rogue wired and wireless systems.
– WebFingerPrint: Windows enumeration tool that can ferret out patch levels, NetBIOS information, user information, and more.
– Microsoft Baseline Security Analyzer: Checks your local machine to identify missing security updates and common security misconfigurations.
– Metasploit: A great tool to exploit those Windows-based vulnerabilities that other tools find, for advanced users only.
As you build your compilation of security testing tools over time, you’ll find that there is no one best tool. Keep in mind that security tools are not the “easy button” for finding security vulnerabilities. That’s where Operating System, Application, Networking Knowledge, and most importantly, experience will come into play.
Where tools are required, you’ll see that the ones that are more specialized in finding specific types of vulnerabilities will provide you with the best results. It all comes down to personal preference and how comfortable you feel using each tool, but in the end your goal should be to find the greatest number of vulnerabilities, exerting the least amount of work, in the shortest amount of time. Get to know the tools on this list, use them consistently and you’ll be well on your way to a storm of work that you never thought you had before ;).