This post introduces NSX-T L7 capabilities. You will create a MySQL Daemon on an VM and configure NSX-T Distributed Firewall to allow traffic on TCP 3306. When replacing this MySQL Daemon with sshd listening on 3306 NSX-T L4 firewall will not prevent access – by design. After enabling NSX-T L7 Firewall (Context Profile / Application ID) NSX-T will enforce useage of MySQL protocol and prevent ssh access on TCP 3306.
Environment:
Have NSX-T 2.5 up and running.
A Demo VM “secdemo” running Ubuntu 18.04.3 LTS, Docker Daemon up and running. Network connection to GitHub & DockerHub. Demo VM is connected to NSX-T Logical Segment (vlan or overlay based).
Management Station (Linux?) having ssh client, mysql client and Web Browser installed and network access to Demo VM.
1. Preparation
On Demo VM clone the GitHub repo for this demo.
vmware@secdemo:~$ git clone https://github.com/danpaul81/nsx-sec-l7.gitChange into nsx-sec-l7 directory and create an ssh keypair for this demo. This will later be used for the ssh daemon.
vmware@secdemo:~/nsx-sec-l7$ ./a01_create_ssh_keys.sh
Generating public/private rsa key pair.
Your identification has been saved in ./ssh/nsx-l7-demo_id_rsa.
Your public key has been saved in ./ssh/nsx-l7-demo_id_rsa.pub.
The key fingerprint is:
SHA256:xInTfGw8r7LMdbMuFM1jYFq7p7Z9VRVe9GweJ4ZH8BQ vmware@secdemo
The key's randomart image is:
+---[RSA 2048]----+
| ..E++|
| = ++ =..+|
| o *+*=. =o*|
| o.ooo=o ++|
| S +.. o|
| o.. .|
| ..ooo . |
| o ++o o. |
| +..++. |
+----[SHA256]-----+
Copy the newly created nsx-sec-l7/ssh/nsx-l7-demo_id_rsa to your management station.
vmware@secdemo:~/nsx-sec-l7$ scp ssh/nsx-l7-demo_id_rsa vmware@master:/home/vmware2. Firewall Preparation
2.1 Tag your Demo VM
On “NSX Manager / Inventory / Virtual Machines” apply Tag “nsx-secdemo” / Scope “app” on your Demo VM. We’ll use this Tag later for NSX DFW.
2.2 Create a new Firewall Policy
On “Security / Distributed Firewall” Add Policy “L7Demo” Open “Applied To” Field, select “Groups” and click “ADD GROUP”. Type Name “SG_L7Demo”. Click “Set Members” and “Add Criteria”. “Criteria 1” should be “VirtualMachine Tag Equals nsx-secdemo Scope app”. Apply membership selection and Save Group configuration. Clicking “View Members” on your newly created Security Group should list your “secdemo” VM.
Policy Creation 
Security Group Creation 
Membership Criteria 
Security Group 
View members
2.3 Insert two basic Firewall Rules into the “L7Demo” Policy.
Name: “AllowSSH22” Destinations “SG_L7Demo” Services: “SSH” Action “Allow”
This will allow SSH access on Port 22 for Management purposes.
Name: “DenyAll” Destinations “SG_L7Demo” Action “Drop”
If you now hit “Publish” only SSH Traffic on port 22 will be possible to your Demo VM. Try to ping it, this should fail.
3. MySQL / sshd attack Demo
3.1 Now can start a MySQL Docker container on your Demo VM listening on host port 3306.
vmware@secdemo:~/nsx-sec-l7$ ./a02_docker_mysql.sh
*******snipped********
2020-01-20T09:12:21.592547Z 0 [Note] mysqld: ready for connections.
Version: '5.7.28' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
Connecting to this MySQL Server should fail as we don’t have a firewall rule in place. Add a L4 Firewall Rule for MySQL in NSX Manager.
Name “MySQL” Destinations “SG_L7Demo” Services “MySQL”
Your Management system should now be able to connect to this MySQL Server.
vmware@master:~/test$ mysql -h secdemo -uroot --password="password" -e "select User from mysql.user;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------+
| User |
+---------------+
| root |
| mysql.session |
| mysql.sys |
| root |
+---------------+3.2 Now lets guess someone makes it into your MySQL Server and shuts it down.
vmware@secdemo:~/nsx-sec-l7$ docker stop mysql && docker rm mysql
*****snipped*****
2020-01-20T09:17:55.434269Z 0 [Note] mysqld: Shutdown complete
mysql
mysql…and starts a ssh-daemon listening on TCP 3306.
vmware@secdemo:~/nsx-sec-l7$ ./a03_docker_ssh.sh
> Starting SSHD
>> Generating new host keys
ssh-keygen: generating new host keys: RSA DSA ECDSA ED25519
****snipped****
>> Adding user vmware with uid: 1000, gid: 1000.
INFO: root account is now locked by default. Set SSH_ENABLE_ROOT to unlock the account.
INFO: password authentication is disabled by default. Set SSH_ENABLE_PASSWORD_AUTH=true to enable.
Running /usr/sbin/sshd -D -e -f /etc/ssh/sshd_config
Server listening on 0.0.0.0 port 22.
Server listening on :: port 22.NSX-T L4 firewall will not prevent this as it only protects IP/Port connections. So your management system can now connect to ssh on TCP 3306.
vmware@master:~$ ssh vmware@secdemo -p3306 -i nsx-l7-demo_id_rsa
The authenticity of host '[secdemo]:3306 ([10.88.110.21]:3306)' can't be established.
ECDSA key fingerprint is SHA256:cQQ0w47kTT9+mZfXzozsCOu5kau9PP9gA+gl0PRGNro.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[secdemo]:3306,[10.88.110.21]:3306' (ECDSA) to the list of known hosts.
Welcome to SSH on MySQL Port
d7c6488b524d:~$3.3 secure your MySQL Application
Add a Context Profile to your NSX-T DFW MySQL Firewall Rule. Click the “Profile” Field of your MySQL Rule and select the MySQL Context Profile. This will enable NSX L7 capabilities to enforce useage of MySQL Application protocol.
After publishing your changes connecting ssh on TCP 3306 from your Management station to Demo VM should no longer be possible.
Stop the ssh Container on your Demo VM and re-start the MySQL Container.
vmware@secdemo:~/nsx-sec-l7$ docker stop ssh && docker rm ssh
Received SIGINT or SIGTERM. Shutting down sshd
Done.
Received signal 15; terminating.
ssh
sshvmware@secdemo:~/nsx-sec-l7$ ./a02_docker_mysql.sh
****snipped****
Version: '5.7.28' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
You now should be able to open a MySQL connection from your Management Station again.
vmware@master:~/test$ mysql -h secdemo -uroot --password="password" -e "select User from mysql.user;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------+
| User |
+---------------+
| root |
| mysql.session |
| mysql.sys |
| root |
+---------------+