Delete orphaned synced Users or Groups in Azure AD

After Azure AD Connect has been removed, you discover that your Azure AD has a lot of orphaned users and groups from the no longer existing on-premises Active Directory. You cannot delete them from the Azure Portal GUI, but you can delete them by using PowerShell.

Sign-in to your tenant using an account with «Global admin» role assigned
Connect-MsolService
or as described here, if this is your first time you log in to a Office 365 / Azure tenant using PowerShell

To query all users in Azure AD use this cmdlet:
Get-MsolUser

Remove the orphaned user one at a time. Just replace UserPrincipalName in the following command with the value displayed in the UserPrincipalName column in the PowerShell console
Remove-MsolUser -UserPrincipalName UserPrincipalName

To query all groups in Azure AD use this cmdlet:
Get-MsolGroup

Remove the orphaned group one at a time. Just replace DisplayName in the following command with the value in the DisplayName column in the PowerShell console
Get-MsolGroup -SearchString "DisplayName" | Remove-MsolGroup

It may take a minute or two until the user is removed in the GUI too.