1 1.1 christos The following are examples of j_profile.d. 2 1.1 christos 3 1.1 christos This script samples on-CPU stack traces and prints them with Java 4 1.1 christos translations. With DTrace, as well as tracing events, you can also sample 5 1.1 christos them. Each approach has its own advantages and disadvantages and you are 6 1.1 christos encouraged to try both when investigating performance issues (especially 7 1.1 christos tracing). Sampling is inaccurate and can miss events, yet is a quick and 8 1.1 christos easy way to discover a certain set of performance issues involving on-CPU 9 1.1 christos load. 10 1.1 christos 11 1.1 christos This script samples at 101 Hertz, printing out the 25 most frequently seen 12 1.1 christos stack traces down to 10 lines deep. All of these values can be tweaked in 13 1.1 christos the script. 14 1.1 christos 15 1.1 christos Here we run the script on Code/Java/Func_loop. The argument fed to the script 16 1.1 christos is the PID of the Java program we wish to investigate. Here the raw output is 17 1.1 christos show, then again after filtering using c++filt. 18 1.1 christos 19 1.1 christos # j_profile.d -p 1312 20 1.1 christos Sampling 10-level stacks at 101 Hertz... Hit Ctrl-C to end. 21 1.1 christos ^C 22 1.1 christos 23 1.1 christos Top 25 most frequently sampled stacks, 24 1.1 christos 25 1.1 christos 26 1.1 christos Func_loop.func_c()V 27 1.1 christos Func_loop.func_b()V 28 1.1 christos Func_loop.func_a()V 29 1.1 christos Func_loop.main([Ljava/lang/String;)V 30 1.1 christos StubRoutines (1) 31 1.1 christos libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x1a3 32 1.1 christos libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27 33 1.1 christos libjvm.so`__1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x2f 34 1.1 christos libjvm.so`__1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nLJNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_+0x1df 35 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 36 1.1 christos 30 37 1.1 christos 38 1.1 christos Func_loop.func_c()V 39 1.1 christos Func_loop.func_b()V 40 1.1 christos Func_loop.func_a()V 41 1.1 christos Func_loop.main([Ljava/lang/String;)V 42 1.1 christos StubRoutines (1) 43 1.1 christos libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x1a3 44 1.1 christos libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27 45 1.1 christos libjvm.so`__1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x2f 46 1.1 christos libjvm.so`__1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nLJNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_+0x1df 47 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 48 1.1 christos 31 49 1.1 christos 50 1.1 christos Func_loop.func_c()V 51 1.1 christos Func_loop.func_b()V 52 1.1 christos Func_loop.func_a()V 53 1.1 christos Func_loop.main([Ljava/lang/String;)V 54 1.1 christos StubRoutines (1) 55 1.1 christos libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x1a3 56 1.1 christos libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27 57 1.1 christos libjvm.so`__1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x2f 58 1.1 christos libjvm.so`__1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nLJNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_+0x1df 59 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 60 1.1 christos 32 61 1.1 christos 62 1.1 christos Func_loop.func_c()V 63 1.1 christos Func_loop.func_b()V 64 1.1 christos Func_loop.func_a()V 65 1.1 christos Func_loop.main([Ljava/lang/String;)V 66 1.1 christos StubRoutines (1) 67 1.1 christos libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x1a3 68 1.1 christos libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27 69 1.1 christos libjvm.so`__1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x2f 70 1.1 christos libjvm.so`__1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nLJNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_+0x1df 71 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 72 1.1 christos 33 73 1.1 christos 74 1.1 christos Func_loop.func_c()V 75 1.1 christos Func_loop.func_b()V 76 1.1 christos Func_loop.func_a()V 77 1.1 christos Func_loop.main([Ljava/lang/String;)V 78 1.1 christos StubRoutines (1) 79 1.1 christos libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x1a3 80 1.1 christos libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27 81 1.1 christos libjvm.so`__1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x2f 82 1.1 christos libjvm.so`__1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nLJNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_+0x1df 83 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 84 1.1 christos 41 85 1.1 christos 86 1.1 christos Func_loop.func_c()V 87 1.1 christos Func_loop.func_b()V 88 1.1 christos Func_loop.func_a()V 89 1.1 christos Func_loop.main([Ljava/lang/String;)V 90 1.1 christos StubRoutines (1) 91 1.1 christos libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x1a3 92 1.1 christos libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27 93 1.1 christos libjvm.so`__1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x2f 94 1.1 christos libjvm.so`__1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nLJNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_+0x1df 95 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 96 1.1 christos 72 97 1.1 christos 98 1.1 christos Func_loop.func_c()V 99 1.1 christos Func_loop.func_b()V 100 1.1 christos Func_loop.func_a()V 101 1.1 christos Func_loop.main([Ljava/lang/String;)V 102 1.1 christos StubRoutines (1) 103 1.1 christos libjvm.so`__1cJJavaCallsLcall_helper6FpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x1a3 104 1.1 christos libjvm.so`__1cCosUos_exception_wrapper6FpFpnJJavaValue_pnMmethodHandle_pnRJavaCallArguments_pnGThread__v2468_v_+0x27 105 1.1 christos libjvm.so`__1cJJavaCallsEcall6FpnJJavaValue_nMmethodHandle_pnRJavaCallArguments_pnGThread__v_+0x2f 106 1.1 christos libjvm.so`__1cRjni_invoke_static6FpnHJNIEnv__pnJJavaValue_pnI_jobject_nLJNICallType_pnK_jmethodID_pnSJNI_ArgumentPusher_pnGThread__v_+0x1df 107 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 108 1.1 christos 116 109 1.1 christos 110 1.1 christos 111 1.1 christos The most frequent stacks had Func_loop.func_c() on CPU, with a stack trace 112 1.1 christos showing func_b() and func_a() - as expected from the source to Func_loop.java. 113 1.1 christos 114 1.1 christos 115 1.1 christos Now passing that output through c++filt to translate the compiled C++ symbols 116 1.1 christos of the JVM. 117 1.1 christos 118 1.1 christos # j_profile.d -p 1312 -o out.j_profile 119 1.1 christos # c++filt out.j_profile 120 1.1 christos Sampling 10-level stacks at 101 Hertz... Hit Ctrl-C to end. 121 1.1 christos 122 1.1 christos Top 25 most frequently sampled stacks, 123 1.1 christos 124 1.1 christos 125 1.1 christos Func_loop.func_c()V 126 1.1 christos Func_loop.func_b()V 127 1.1 christos Func_loop.func_a()V 128 1.1 christos Func_loop.main([Ljava/lang/String;)V 129 1.1 christos StubRoutines (1) 130 1.1 christos libjvm.so`void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x1a3 131 1.1 christos libjvm.so`void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27 132 1.1 christos libjvm.so`void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2f 133 1.1 christos libjvm.so`void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0x1df 134 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 135 1.1 christos 10 136 1.1 christos 137 1.1 christos Func_loop.func_c()V 138 1.1 christos Func_loop.func_b()V 139 1.1 christos Func_loop.func_a()V 140 1.1 christos Func_loop.main([Ljava/lang/String;)V 141 1.1 christos StubRoutines (1) 142 1.1 christos libjvm.so`void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x1a3 143 1.1 christos libjvm.so`void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27 144 1.1 christos libjvm.so`void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2f 145 1.1 christos libjvm.so`void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0x1df 146 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 147 1.1 christos 13 148 1.1 christos 149 1.1 christos Func_loop.func_c()V 150 1.1 christos Func_loop.func_b()V 151 1.1 christos Func_loop.func_a()V 152 1.1 christos Func_loop.main([Ljava/lang/String;)V 153 1.1 christos StubRoutines (1) 154 1.1 christos libjvm.so`void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x1a3 155 1.1 christos libjvm.so`void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27 156 1.1 christos libjvm.so`void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2f 157 1.1 christos libjvm.so`void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0x1df 158 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 159 1.1 christos 19 160 1.1 christos 161 1.1 christos Func_loop.func_c()V 162 1.1 christos Func_loop.func_b()V 163 1.1 christos Func_loop.func_a()V 164 1.1 christos Func_loop.main([Ljava/lang/String;)V 165 1.1 christos StubRoutines (1) 166 1.1 christos libjvm.so`void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x1a3 167 1.1 christos libjvm.so`void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27 168 1.1 christos libjvm.so`void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2f 169 1.1 christos libjvm.so`void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0x1df 170 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 171 1.1 christos 21 172 1.1 christos 173 1.1 christos Func_loop.func_c()V 174 1.1 christos Func_loop.func_b()V 175 1.1 christos Func_loop.func_a()V 176 1.1 christos Func_loop.main([Ljava/lang/String;)V 177 1.1 christos StubRoutines (1) 178 1.1 christos libjvm.so`void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x1a3 179 1.1 christos libjvm.so`void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27 180 1.1 christos libjvm.so`void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2f 181 1.1 christos libjvm.so`void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0x1df 182 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 183 1.1 christos 29 184 1.1 christos 185 1.1 christos Func_loop.func_c()V 186 1.1 christos Func_loop.func_b()V 187 1.1 christos Func_loop.func_a()V 188 1.1 christos Func_loop.main([Ljava/lang/String;)V 189 1.1 christos StubRoutines (1) 190 1.1 christos libjvm.so`void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x1a3 191 1.1 christos libjvm.so`void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27 192 1.1 christos libjvm.so`void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2f 193 1.1 christos libjvm.so`void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0x1df 194 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 195 1.1 christos 53 196 1.1 christos 197 1.1 christos Func_loop.func_c()V 198 1.1 christos Func_loop.func_b()V 199 1.1 christos Func_loop.func_a()V 200 1.1 christos Func_loop.main([Ljava/lang/String;)V 201 1.1 christos StubRoutines (1) 202 1.1 christos libjvm.so`void JavaCalls::call_helper(JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x1a3 203 1.1 christos libjvm.so`void os::os_exception_wrapper(void(*)(JavaValue*,methodHandle*,JavaCallArguments*,Thread*),JavaValue*,methodHandle*,JavaCallArguments*,Thread*)+0x27 204 1.1 christos libjvm.so`void JavaCalls::call(JavaValue*,methodHandle,JavaCallArguments*,Thread*)+0x2f 205 1.1 christos libjvm.so`void jni_invoke_static(JNIEnv_*,JavaValue*,_jobject*,JNICallType,_jmethodID*,JNI_ArgumentPusher*,Thread*)+0x1df 206 1.1 christos libjvm.so`jni_CallStaticVoidMethod+0x15d 207 1.1 christos 74 208 1.1 christos 209 1.1 christos 210