Doing some work these past few weeks and all I can say is Microsoft Cloud security is difficult. Don’t believe the hype, when you delve into security with Microsoft Cloud products, things are difficult and broken at the worst. Recently, and there’ll be more posted about it, but I’ve been working to build a data platform on Azure. I wanted to migrate an existing database and SSIS to Azur PAAS. I planned to use all the security features to make it as impenetrable as it might be.
There have been a lot of challenges along the way, with many things not easily found. I’m going to say it here. Much of the security the way it appears and the tools to manage it appear to be an afterthought. There are way too many tough parts, not well documented to help these solutions in secure environments evolve. Even the Well-Architected documents and the Learn tutorials and learning don’t provide a context for how and when to use products. Following is where I ran into issues.
Things to Consider
In Azure, the default for PAAS services is to be open to the internet. Why is this the way it is? Things like VNet integration and Private Endpoints come later. They still have the option within the Azure SQL database to allow all Azure to access the database. (yes you read that right, All Azure) Why is this needed or an option? Is that anotyher security failing in the underpinning architecture?
You jump through several hoops with the Azure SQL database to secure it and lock it down from external access. Then when you look to segregate things, the hoops you need to jump through by using separated VNets is crazy. Much of this is not clear in the documentation and requires support tickets or other help from those who’ve done it to get this working. Azure SQL database default is to allow direct connections via the firewall.
PowerShell
Next is tools and Powershell is my target here. Currently, we have multiple versions the older 5 and the newer version 7. Firstly the separate experience is good, however, the good finishes there. Azure tools which are possibly one of the most important parts of Azure these days is broken. Both versions have problems, we have deprecated AzureRM, which seems to get installed somehow, I think that’s Visual Studio in my case, AzureRM should not be getting installed with any current version of Visual Studio now it’s got a defined EOL. I hope that update happens to all Visual Studio ASAP. This is one of the best tools around to clean that up and rid yourself of AzureRM unless you need it for existing scripts. It should be on Github. You might want to start a project to get those AzureRM scripts written for the az module so February 29th 2024 doesn’t sneak up on you.
Azure DevOps
Managed Identities appear to be the future of interconnected security in Azure. They’ve been around for a time. Yet for some parts, like adopting a Managed Identity to attach to an Azure SQL database is a no go in the Deployment pipeline, I’d love to know why that is? As soon as you properly secure your Azure SQL database with no public endpoints, deployment to said database becomes somewhat more problematic.
Ideally, you don’t want and SQL server user accounts on your Azure SQL database, Though because of either an omission of documentation or perhaps issues with PowerShell to get a token for the Service principal the only genuinely secure option for connecting to the database is a connection string. Ok, the connection string can be in the key vault. It’s only a variable while the pipeline is running, but then you’ve had to resort to using a database account. Not Identities, that are more secure. One step forward and two steps back.
There’s another situation to add here, and that’s the confusing tooltips. When you investigate to add a certificate to the Azure DevOps Service Connection, it states to add the contents of the PEM file. Seek to find in the documentation how to create one. The answer will be below.
The Point of This
Microsoft, it might look straightforward to individuals in the organisation, but from these cases, I believe you can understand there’s a stack of enhancements needed. Better documentation, good tooltips and links to information help people quickly and easily. Clearly, the way things are happening, there are genuine issues in how security is being designed into products. It causes me concern that at some point the house of cards, smoke and mirrors will collapse somewhat too spectacularly. It’s still hard for people to build secure platforms.
I had a plan to establish a data platform and drive in security at every point. VNet, No public endpoints, solely accessible via Bastions and sort of like a data centre in a cloud. Set up Azure SQL, simple, add it to a VNet not too complicated, Publish SQL as a service on the subnet and it looks great. Ok now access it from another VNet. Simple peer the VNets and all will be fine. Not entirely, now need to add a Private endpoint to Azure SQL to make it accessible from anywhere than the VNet it’s on.
Why would I want to do all this? To use CI/CD in Azure DevOps to deploy the build to the protected database, it requires you to set up a VM and using suitable design and not allowing the VM on the same network creates hell to get it working? Azure DevOps cannot make connections to a properly secured database with no public endpoint.
From this diagram and this is a little simplified, you can see how complex it is to get Azure DevOps to be able to conduct CI/CD builds on a secured Azure SQL database. I’ve provided key elements here and I’ll write up more in the total build chain. Through it all, I’ve found it difficult and found many issues with no information readily available. There were two support tickets in this mess to find out information and could have been another. For something as simple conceptually it seems overly complex. That leaves me nervous as to how people could get something wrong and leak access and worse data.
Solutions
There has to be better and I’ve got a few suggestions
- Improve the documentation, it’s woefully inedaguate, relying on blog posts like this to help people is poor, even thogu many blogs ar greaat
- Provide more design examples
- Fix isssues in critcal tools – Powershell is critical, it can’t have bugs for Azure related things for 6 weeks
- Turn off public access by default on PAAS services. Provide more guidance on setting up well architected
- Well architected docs are not well architected. VNets design doesn’t even give a hint as to why use multple Vnets over simpler subnetting
That’s my summation and why I think Azure Cloud security is difficult. I do hope it’s not for you, however, much of it seems like an afterthought and is therefore difficult to piece together and understand. Being difficult to piece together and understand means it’s prone to error. That’s risky
Oh and that PEM file, well it turns out it’s a CER file, and they contain the same info, so when you needed to paste your certificate to the PEM file, there it was and no it’s not mentioned in the PowerShell documentation or the Azure DevOps documentation. Hopefully, I’ll get to edit this
See ya round
Peter
