VisualVM Remote EC2 Over SSH (jstatd and JMX)

VisualVM is a great tool for monitoring JVM, but remote to Amazon EC2 have little trouble. Here are steps make life easier

Local Amazon EC2

1. Setup an SSH Tunnel with SOCKS (use a SOCKS proxy) to Amazon EC2 on port 10000

If you're using ssh on Linux, this would be done using dynamic port forwarding (port 10000 opened on local):

% ssh -D 10000 -i /path/to/file.pem \
    ec2-user@[EC2 Public DNS or EC2 Elastic IP]

Examples host name:

  • ec2-user@ec2-54-49-94-45.ap-northeast-1.compute.amazonaws.com
  • ec2-user@54-49-94-45

If you're using Putty on Windows, the tunnel would be setup:

  • In Session, set Host Name (or IP address) as ec2-user@[EC2 Public DNS or EC2 Elastic IP] ec2-user@54-49-94-45, Port as [EC2 SSH Port]22 Setup Host Name
  • (if any) In Connection > SSH > Auth, set path ppk file to Private key file for authentication Setup Private Key File
  • Add a Dynamic Forwarded Port: In Connection > SSH > Tunnels, set Source port as 10000 and select radio button Dynamic, then click Add button, you can see value D10000 added in Forwarded ports Dynamic Forwarded Port
  • Click Open button to open tunnel, leave the tunnel open while you monitor the JVM

2 - For jstatd. jstatd is a daemon that is distributed with JDK, so using Oracle JDK is required (See Working with Remote Applications)

  • Step 1. Create a permissions file named jstatd-all-permissions

    % cat jstatd-all-permissions
    grant codebase "file:${java.home}/../lib/tools.jar" {
      permission java.security.AllPermission;
    };
    

    Note: Replace "${java.home}/../lib/tools.jar" with the absolute "/path/to/jdk/lib/tools.jar" if you have only copied but not installed the JDK. Required semicolon (;) at end.

  • Step 2. Start jstatd daemon

    % jstatd \
       -J-Djava.security.policy=jstatd-all-permissions
    

    Note: you can get more info with -J-Djava.rmi.server.logCalls=true

    Note: if java processes started with root permission, you must start jstatd with root permission for the daemon can see them (e.g. sudo jstatd ...)

2 - For JMX. Start Java application with JMX arguments (See Connecting to JMX Agents Explicitly)

% java \
   -Dcom.sun.management.jmxremote.port=3333 \
   -Dcom.sun.management.jmxremote.ssl=false \
   -Dcom.sun.management.jmxremote.authenticate=false \
   YourJavaApp

3. Configure VisualVM to use SOCKS proxy

  • In VisualVM, open menu Tools > Options > Network
  • Select Manual Proxy Settings and uncheck Use the same proxy settings for all protocols
  • Set SOCKS Proxy as localhost, Port as 10000

4. Get Private IP Address of Amazon EC2 (not Elastic IP, it not working)

You can find Private IP in AWS Console Management > EC2

Or using ifconfig on Linux, the private ip address after inet addr (e.g inet addr:10.27.72.15) in network interface eth0

5. Add Remote Host to VisualVM

  • In VisualVM, open menu File > Add Remote Host...
  • Add Private IP Address found in step 4 into Host name, and click OK
  • If you monitor with JMX then:
    1. Right click the added host, select "Add JMX Connection..."
    2. Enter [Private IP Address]:[JMX Port] (e.g 10.27.72.15:3333), and click OK
  • Waiting for VisualVM connect and then profiling the Java application started

Advantages and disadvantages of jstatd and JMX

jstatd

  • Advantages: can connect to a running JVM, no need to start it with special parameters
  • Disadvantages: no CPU usage monitoring, not possible to run the Sampler and take thread dumps

JMX

  • Advantages: can use full features of VisualVM
  • Disadvantages: need to start the JVM with some system properties

Comments

Popular posts from this blog

Reduce TIME_WAIT Socket Connections