Security

Azure Web Apps with Cost Effective, Private and Hybrid Connectivity

Posted on Updated on

With the advent of the cloud came the ever so attractive PaaS service model. The first time I heard about this, I was sold. Host my application without having to manage the infrastructure, the OS, patching, scaling and all the other things that I really don’t want to do anyways – sign me up! The “catch” is though that (to misquote all those before me) “with less responsibility comes less power”. When relinquishing responsibility (the positive side of PaaS), you also relinquish some control and ability for customization. This is a conversation I have with customers at least once a week – how do I control networking for my PaaS application.

One my peers, Steve Loethen who focuses more on the application development aspects in Azure, and I were speaking and noted that it would be great if we could leverage Azure Private Link Endpoints and Azure Application Gateway to get both private connectivity from the internal network and leverage a cloud-native Web Application Firewall for internet traffic. Traditionally using the “isolated” tier of Azure App Services called an Azure Application Service Environment (which provides a completely isolated and dedicated app service environment) would be the way to go. Unfortunately the nature of a private stamp of a PaaS service inherently comes with a pretty heft price tag. There are still some great features to be had with the ASE, but using Private Link we can get to a point where all external access is blocked unless its using the WAF – but can be still be accessed over the internet network. I’ll document that configuration below.

To setup this environment we will need to:

  • Setup the Web App
  • Create a Virtual Network in Azure
  • Setup a Site-to-Site VPN
  • Setup a Private Link Endpoint for the Web App
  • Restrict Network Access to the Web App to only the Private Link Endpoint
  • Setup a Public Application Gateway

*Note* As of 3/12/20 Private Link Endpoints for App Service Web Apps is in Public Preview.

As of 10/6/20 Private Link Endpoints for App Service Web Apps is Generally Available

Let’s get started! First, we’ll create the App Service. I’m going to be using the resource group named “pri-webapp-rg” and the app service name “private-webapp.azurewebsites.net” in the West US region.

*Note* At the time of writing this post (5/18/20) Private Endpoints require the App Service to be a Pv2.

I am not going to use it in this lab, but I am going to leave App Insights enabled. If you’ve not used App Insights yet as an APM tool I highly suggest you look at instrumenting your app with this tooling.

Go ahead and add any tags if you need any for your environment, then create that App Service. Next we’ll go and create our Azure Virtual Network. This will allow us to facilitate the private network connectivity. I’m going to create this in West US, I’ll note that Private Link Endpoints do not need to be deployed in the same region as the resource, but I want to reduce latency here so I’ll deploy the vnet (and subsequent Private Link Endpoints in the following steps) as the same region as the App Service.

I’m going to use the 10.2.0.0/16 address space, and initially create two subnets. The “WebApp” subnet will be for the Private Link Endpoint and the “AppGateway” subnet will be designated for the Application Gateway.

I’m going to leave the rest of the deployment settings with their default options selected for now and create the vnet.

Once the vnet has finished creating, we need to go create a “Gateway Subnet” which will be used for the VPN Gateway to be used for the Site-to-Site VPN for hybrid connectivity. By default, it will pre-select a subnet that is available in your address space. Once that subnet is created we have our finished vnet setup.

Now that the network in Azure is setup, we need to get the VPN Gateway configured. When the “West US” region is selected on the deployment screen you’ll get a list of available virtual networks in that region. I selected the one we just created, and it will validate that there is a “GatewaySubnet” created in that vnet and will select that as the deployment subnet for the VPN Gateway. I’m going to be using a Route-based VPN so I’ll select that as my VPN type then create a new public IP address and leave the Active-Active and BGP options disabled for this lab.

The VPN gateway does take 20-30 minutes to deploy, so go grab a cup of coffee. Once that’s done we’ll need to add the “connection” for the site-to-site VPN.

For this setup we’re using a Site-to-Site IPSec VPN Connection, and this is where you’ll also set your PSK and IKE protocol. You’ll also need to create a local network gateway, which is an ARM resource used for representing the on-premises network.

The “IP Address” field is where you’ll enter the public IP address of the appliance that’s going to terminate the VPN on-premises. You’ll also need to add which networks are on-prem, this will add add the local network address space to the route table of the VPN Gateway so it knows to use that link for traffic bound for those addresses.

Once that’s created, we’ll need to go to the overview page for the VPN Gateway to get its public IP address. That will be needed to setup the on-premises side of the VPN. I’m going to use a PFSense appliance in home lab network to accomplish this setup. If you want to test this just in Azure you can also use just a vnet peered network and create an emulated “client” machine, alternatively you could also setup a point-to-site VPN for just your local machine. I won’t be showing that process here, but I have another post that discusses the setup of PFSense S2S VPN with an Azure VPN Gateway.

Once the S2S VPN is setup, we can now go and setup the Private Link Endpoint for the App Service.

When choosing a private link endpoint you need to choose the resource type, so here I’m using the “Microsoft.Web/sites” resource type.

As noted earlier on, we created the “WebApp” subnet to hold the private link endpoint and we’ll select that here. A very important component of private link is DNS. As is stated so eloquently in my favorite Haku:

“It’s not DNS

  There’s no way it’s DNS

   It was DNS”. 

Before deploying private link you have to consider the DNS scenario. Clients cannot call private link endpoints by their IP addresses, it has to be by their DNS names. I highly recommend some of the Microsoft Documentation on the topic, or Daniel Mauser has an amazing article on Private Link DNS Integration Scenarios. I’m going to go ahead and allow the private endpoint to create an Azure Private DNS Zone, but ultimately for this lab environment I’ll just be using a host file entry.

Alright, the private link endpoint is all setup. Now to make sure we restrict network traffic to that which only originates from the private link endpoint. App Services can use virtual network service endpoints to restrict traffic originating from vnets in Azure quite easily. Though in our case we want to make sure that on-premises traffic can also access that app service so rather than allowing access at the vnet level we’ll just be allowing the single IP of the private endpoint.

After adding the access restriction rule, you’ll see that the “Allow All” rule switches to a “Deny All” rule and is given the lowest priority on the ACL. Above that we have the single IP of the Private Link Endpoint that is allowed. As a result, the only IP that’s allowed to call this app service is that of the private link endpoint.

Great! Okay, now we’ve setup my app service, my site-to-site VPN, private link, and access restrictions. We should be all set to test connectivity from on-premises now, let’s go give that a shot. As noted above, since this is just a lab environment I’m just going to be using a host file entry on my test machine on my lab network, so lets set that up real quick.

Great, now let’s validate this in a browser and with powershell to both make sure the page loads and that it’s using the private endpoint. We can see that the page loads using the “public” DNS name, but the address is the 10.2.2.4 address that is assigned to the private link endpoint. We can also see that on a machine that’s just using the typical internet path is given a 403, since the source traffic is not coming from the private endpoint – perfect!

If we go back and look at the site-to-site VPN configuration now we’ll also see that traffic has started passing over that link which further validates the fact that the traffic is remaining private.

Now that we have confirmed the hybrid traffic is passing through the private endpoint and that all internet traffic is denied, let’s configure the Application Gateway with the Web Application Firewall SKU so that we can facilitate external traffic communicating with our private Web App (forgive my ever so descriptive naming convention) and create a new public IP address to be associated with the Application Gateway. Lastly, we’ll use the subnet that we designated earlier for the Application Gateway for the deployment.

When creating the backend pool we’ll enter the IP address of the private link endpoint. For the routing rule, since this is a lab and I don’t have a TLS certificate on-hand I’ll use an HTTP listener on the App Gateway and use the same HTTPS endpoint for the App Service that we’ve been using internally. In a production environment you’ll want to have HTTPS on the public listener. When creating the HTTP setting to use for this routing rule we want to override the host name to that of the App Service (in this case, “private-webapp.azurewebsites.net”) because remember that private link needs to be called using a DNS name and this way we can add that host name to the request header.

*Note* Since we deployed a Private DNS Zone, which is automatically attached to our virtual network, and the application gateway is deployed into the virtual network, we can use the DNS name of the app here because it will resolve correctly. Though, for clarity in this lab environment I’m just using the IP address.

After the Application Gateway is finished deploying I want to go add a DNS label to the public IP which is associated and confirm that it was applied so I can use that DNS name rather than the IP itself.

Okay, moment of truth! Remember the machine we used earlier that used the public path to get to the app service but got a 403 error because of the access restrictions? Let’s go ahead and try hitting the newly provisioned Application Gateway that does have a public listener and try the site now. We see that it’s using the DNS label and subsequent HTTP listener that we setup on the App Gateway, using the public interface and is routing us appropriately back to our private web app!

That’s all folks! As a retrospective, here’s what we’ve done:

  • Configured an App Service with Hybrid, Private Networking
  • Configured a scalable public endpoint that’s using a WAF in an IDS mode
  • Maintained the flexibility, scalability, and cost effectiveness of the non-ASE App Service

If you have any questions or suggestions for future blog posts feel free to comment below, or reach out to me via email, twitter, or LinkedIn.

Thanks!


Change Feed:

10/8/20: Private Endpoints for App Service Web Apps now GA

Custom “Virtual Network Operator” Role in Azure

Posted on Updated on

*Edit* 6/9/20: At the time of writing this blog custom roles were not available in the Azure Portal. Since that feature now exists, I’ve added how to create this role through the portal to the end of this blog.

We all know that cloud environments are different than on premises. Development environments can be even more difficult, the right cross between giving freedom to produce business value and enough security and controls to maintain a good security posture. Recently I came across a scenario where the design was that the networking components (Virtual Network, S2S VPN, Peerings, Service Endpoints, UDRs, Subnets, etc.) were all under the control of a Networking team so that the Azure environment would be an extension of their local network. From a permissions standpoint though this caused some problems. The goal would be to empower developers to build whatever they need and just attach it to the vNet when connectivity was needed, with the caveat that they shouldn’t be able to modify any of the network settings or configurations in the shared vNet. This however, turned out not to be possible with default IAM roles.

In my testing, even though you’re not “modifying” anything per se in the vNet – when attaching a network interface card to a Subnet it does require write access. Reader access will give the user this error.

After looking through the default IAM roles, there aren’t any that do what I need them to do. Alas, a custom role is needed. For reference, please checkout this docs page for custom roles – https://docs.microsoft.com/en-us/azure/role-based-access-control/custom-roles.

First, I need to see what actions are available to pack into the custom role so I run the following command in Azure Powershell and get these results.

Get-AzProviderOperation “Microsoft.Network/virtualNetworks/*” | FT OperationName, Operation, Description -AutoSize

Great, now I can see what is available. It looks like the actions that I need are the /subnet/join/action and /subnet/joinViaServiceEndpoint/Action. Based on the descriptions these two will essentially give “operator” role to the vnet. The assignee will be able to use the subnet(s) but not able to modify them.

Next, you can use one of the template references in the Microsoft Docs link (https://docs.microsoft.com/en-us/azure/role-based-access-control/custom-roles-powershell#create-a-custom-role-with-json-template) and modify the actions.

{
  “Name”: “Custom – Network Operator Role”,
  “Id”: null,
  “IsCustom”: true,
  “Description”: “Allows for read access to Azure network and join actions for service endpoints and subnets.”,
  “Actions”: [
  “Microsoft.Network/virtualNetworks/subnets/join/action”,
  “Microsoft.Network/virtualNetworks/subnets/joinViaServiceEndpoint/action”,
  “Microsoft.Network/virtualNetworks/read”
   ],
  “NotActions”: [],
  “AssignableScopes”: [
  “/subscriptions/00000000-0000-0000-0000-000000000000”,
  “/subscriptions/11111111-1111-1111-1111-111111111111”
  ]
}

After modifying the assignable subscription IDs, save that as a .json file and use it in the following command to import the custom role.

New-AzRoleDefinition -InputFile “C:\FileFolderLocationPath\CustomNetworkOperatorRole.json”

After a portal refresh you get the custom role available as an IAM role assignment.

There you go, after assigning this role the user was able to create VMs and attach them to the Virtual Network while still leaving control of the Network configuration to the Networking Team.

————————————————————————-

————————————————————————-

EDIT 6/9/20: Now that custom roles are available in the portal I’ve added how to create this role that way below. 

Using the documentation on how to create custom IAM roles in the Azure portal you can clone any role, but I’m just going to use the “Network Contributor” role to clone. 

I am going to just “start from scratch”, though you can easily use this page to modify the permissions set of a certain role that already exists. Additionally, if you have the JSON (like we generated before with powershell) you can just upload that here.

Now go ahead and add new permissions by going to the “Microsoft Network” Category and selecting the same permissions we wrote in the JSON and used with the powershell command above.

After adding those the next page will show you the permission that you’ve added, let’s double check those and make sure we have what we need.

Looks good, now we can set a scope where this role can be applied. I have this one scoped to the whole subscription, but you can use the interface to easily change that on this page.

Alright, that’s about it – let’s take a quick look at the summary pages. You can see that the portal actually generates the JSON, which is nice if you want to assign roles using Azure Policy or any other automation tooling.

Nice! Now we can see that users can use the four above actions which will allow them to interact with, and use a shared hub-style virtual network while now allowing full contributor access.

I hope I’ve made your day at least a little bit easier!

If you have any questions or suggestions for future blog posts feel free to comment below, or reach out to me via email, twitter, or LinkedIn.

Thanks!

Isolated Virtual Machines in Azure

Posted on

In a recent design session, I had someone mention how they wish that Azure had dedicated and/or isolated IaaS VMs like some of the other cloud providers. They were shocked with I said – they do! Azure has a number of isolated VM types that you can provision and leverage just the same as any other IaaS VM.

The link to Microsoft documentation below states that “Azure Compute offers virtual machine sizes that are Isolated to a specific hardware type and dedicated to a single customer. These virtual machine sizes are best suited for workloads that require a high degree of isolation from other customers for workloads involving elements like compliance and regulatory requirements.”

https://docs.microsoft.com/en-us/azure/security/azure-isolation#isolated-virtual-machine-sizes

Yup, you read that right, dedicated hardware. This means that using one of the isolated VM types (listed below) will guarantee that your virtual machine will be the only one running on that server instance (as of Nov. 1, 2018).

  • Standard_E64is_v3
  • Standard_E64i_v3
  • Standard_M128ms
  • Standard_GS5
  • Standard_G5
  • Standard_DS15_v2
  • Standard_D15_v2

 

If VM isolation is what you’re after, you can also consider Nested Virtualization in Azure (link below). A few VM types in Azure support Nested Hyper-V (not suitable for production, mind you) and Hyper-V Containers using Docker.

https://azure.microsoft.com/en-us/blog/nested-virtualization-in-azure/

https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container

 

Lastly, there is a new feature that recently went GA that’s sort of in-line with VM isolation – Confidential Compute. While this isn’t VM isolation, it is an isolated enclave for executing code in TEEs (Trusted Execution Environments) using Intel’s SGX capabilities. This allows for code to be executed in an environment that, while it’s being executed, can not be accessed by any source outside of the enclave (see diagrams in the links below).

https://azure.microsoft.com/en-us/blog/introducing-azure-confidential-computing/

https://azure.microsoft.com/en-us/blog/azure-confidential-computing/

 

Using physical isolation techniques in the cloud can be very different than the way we’re used to doing it on-prem. Hopefully, though, this post and its associated links will get you thinking about different ways you can design for isolation in Azure. Feel free to comment, or reach out on Twitter @MattHansen0.

Thanks, and I hope I’ve made your day at least a little bit easier.

Domain Controllers “Grayed out” in SCOM 2012

Posted on Updated on

I did a few new SCOM 2012 installs recently and noticed that after pushing the agent to the DCs, they showed up grayed out in Ops Manager. Here’s a quick tip on how to fix that.

Logon to the DC(s), and with an administrative comand prompt run the HSLockdown tool, and add the local system account to the allowed group.

C:\Program Files\System Center Operations Manager\Agent:

*NOTE* In newer version, this is now stored in “C:\Program Files\Microsoft Monitoring Agent\Agent”

agent

 

Run the command “HSLockdown /L” to show what accounts are being allowed or denied. In this case, my local system isn’t even populated.

 

L

 

 

Now run the HSLockdown tool again with the add switch to allow local system.

“HSLockdown /A “NT AUTHORITY\SYSTEM”

add

 

Restart the agent with “net stop heathservice && net start healthservice” and give it 5 minutes or so then it should be all green in your dashboard.

 

Hope this made your day at least a little easier!

 

Decrypting HTTPS (SSL/TLS) Tunnels Using Fiddler

Posted on

A few days ago the phone rings, I get an ear-full about how some application isn’t working correctly and how it’s all the network’s fault and the repercussions of this outage will possibly cause so much damage that the world will start turning…the OTHER DIRECTION. Unfortunately for us IT Professionals, this is all too common of an occurrence. Nonetheless, I jumped in to see what I could do. I had never seen this application before so I had to start troubleshooting from the ground up. Very quickly I noticed it was running (or supposed to be running) over web protocols, so I whipped out the handy-dandy wireshark to get a look. Hm…it establishes a TLSv1 tunnel and shoots all the data at the server that way. Well, the Apps team was no where to be found so I had to find out what was moving across the wire here to figure out the issue. This is where fiddler comes in to play *Trumpets Fanfare*.

Fiddler is a fantastic little tool that does different things with packet captures and things of the sort. For this blog, I want to talk about its’ ability to man in the middle your own machine to provide visibility into an encrypted tunnel. Lets do a little demonstration here.

I’ve done a quick search in on bing, using HTTPS — thing fancy here at all.

encrypted_search_browser

 

 

I started fiddler prior to performing the search above, and this is what it shows up with, a whole bunch of nothing. Tunnel Tunnel Tunnel Tunnel…dang security.

fiddler_encrypted

 

 

Alas, fiddler has an option to man in the middle yourself and decrypt the tunnel! Just go to Tools > Fiddler Options > HTTPS > and check the box that says “Decrypt HTTPS traffic”. I chose browsers only for this demonstration, though you can do all traffic for other uses and applications.

select_decrypt

 

 

It lets you know that you’re doing something that defies the laws of CAs.

scary_rootcert

 

Now here we go, re-launch the browser and go to https://bing.com, it throws a security error stating that the certificate is untrusted.

untrusted

 

For this to work, you will need to add the exception, if you view the cert you can see that it was assigned to fiddler, when it’s clearly stating that it is for bing.com

cert_fiddler

 

Once that is all excepted, you can do the same search we did before — plain and simple.

encrypted_search_browser

 

Back to Fiddler, and ta-da! Congratulations, you’ve bypassed the security of your own data and now have visibility into the tunnel.

viewing_encrypted

 

 

That’s it, very simple. You can view inside your SSL/TLS tunnel using fiddler in just a few simple steps. Side note, I was able to use that to determine what was happening on the wire for my application failure and was able to remedy the failure.

 

I hope I’ve made your day at least a little bit easier!

 

 

5 free security tools for testing Windows

Posted on

One of the things you often find yourself thinking is, hmm…I should probably test my windows machines for security flaws, right? I’ve decided to share some very good tools for testing security from basic button clicking to advanced security testing.

As it relates to Windows-based computers, there are seven general types of security testing tools. These are:

  1. Port scanners
  2. Network/OS vulnerability scanners
  3. Application/database vulnerability scanners
  4. Password crackers
  5. File searching tools
  6. Network analyzers
  7. Exploit tools

All of these types of tools can and should be used when performing penetration tests, vulnerability assessments, and security audits on your Windows systems.

For the most part with security tools, you get what  you pay for. There are, however, a handful of free tools that are a solid choice.

Super Scan v3:  Very fast and easy to use port scanner that can find live systems, look for open ports and running services, grab banner information including software versions.

superscan

http://www.mcafee.com/us/downloads/free-tools/superscan3.aspx

 

 

SoftPerfect Network Scanner:   Maps MAC addresses to IP addresses which can help you locate rogue wired and wireless systems.

softperfect_network_scanner_network___internet_trace___ping_tools-18961

http://www.softperfect.com/products/networkscanner/

 

– WebFingerPrint:   Windows enumeration tool that can ferret out patch levels, NetBIOS information, user information, and more.

winfingerprint_1

http://winfingerprint.sourceforge.net/

 

Microsoft Baseline Security Analyzer:  Checks your local machine to identify missing security updates and common security misconfigurations.

 

Microsoft-Baseline-Security-Analyzer

 

http://www.microsoft.com/en-us/download/details.aspx?id=7558

 

– Metasploit:  A great tool to exploit those Windows-based vulnerabilities that other tools find, for advanced users only.

metasploit2

 

 

http://www.metasploit.com/free-download/

 

 

 

As you build your compilation of security testing tools over time, you’ll find that there is no one best tool. Keep in mind that security tools are not the “easy button” for finding security vulnerabilities. That’s where Operating System, Application, Networking Knowledge, and most importantly, experience will come into play.

 

Where tools are required, you’ll see that the ones that are more specialized in finding specific types of vulnerabilities will provide you with the best results. It all comes down to personal preference and how comfortable you feel using each tool, but in the end your goal should be to find the greatest number of vulnerabilities, exerting the least amount of work, in the shortest amount of time. Get to know the tools on this list, use them consistently and you’ll be well on your way to a storm of work that you never thought you had before ;).