git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Performance of the compiler


Hi folks,

I just wanted to share the result of profiling the performance of the compiler on "real world" classes, namely compiling the tests of Gradle. We have a lot of tests, so compilation times becomes really a pain point, so I have checked where we spend time. I have attached the export of hotspots from a real compilation session.

It's no surprise to me, most of the time (70%) is spent in the resolve visitor, and most of this time itself is spent in loading classes. We made some improvements in the past, by not initializing those classes, but it's still a crazy amount of time.

Similarly, we spend around 10% of our time in filling stack traces which are used for flow control. Unfortunately we don't have the opportunity to change this because it's either ClassNotFoundException (during resolution) sent by the classloader, or ANTLR recognition exception, used for flow control (duh!).

I remember that for Gradle I had implemented a custom ResolveVisitor that adds some assumptions for Gradle scripts to avoid too many lookups for classes which would obvisouly not exist, and it significantly improved the performance of compiling scripts, but that was because there were lots of implicit imports. For regular classes I'm not sure it's as simple.

Resolution is also very easy to break... Anyway, any change in this area would probably make the lives of our users better!


CPU hot spots

+-----------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+
|                                                                          Method                                                                           |   Time (ms)    |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+
|  java.net.URLClassLoader.findClass(String) URLClassLoader.java                                                                                            |  6,531   54 %  |
|  java.security.AccessController.doPrivileged(PrivilegedExceptionAction, AccessControlContext) AccessController.java (native)                              |  6,090   50 %  |
|  java.net.URLClassLoader$1.run() URLClassLoader.java                                                                                                      |  5,987   50 %  |
|  java.net.URLClassLoader$1.run() URLClassLoader.java                                                                                                      |  5,987   50 %  |
|  org.codehaus.groovy.control.ResolveVisitor.startResolving(ClassNode, SourceUnit) ResolveVisitor.java                                                     |  4,711   39 %  |
|  org.codehaus.groovy.control.ResolveVisitor.visitClass(ClassNode) ResolveVisitor.java                                                                     |  4,711   39 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolve(ClassNode, boolean, boolean, boolean) ResolveVisitor.java                                             |  4,486   37 %  |
|  org.codehaus.groovy.control.ClassNodeResolver.findClassNode(String, CompilationUnit) ClassNodeResolver.java                                              |  4,363   36 %  |
|  org.codehaus.groovy.control.ClassNodeResolver.resolveName(String, CompilationUnit) ClassNodeResolver.java                                                |  4,363   36 %  |
|  org.codehaus.groovy.control.ClassNodeResolver.tryAsLoaderClassOrScript(String, CompilationUnit) ClassNodeResolver.java                                   |  4,363   36 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolveToOuter(ClassNode) ResolveVisitor.java                                                                 |  4,363   36 %  |
|  groovy.lang.GroovyClassLoader.loadClass(String, boolean, boolean, boolean) GroovyClassLoader.java                                                        |  4,183   35 %  |
|  groovy.lang.GroovyClassLoader.loadClass(String, boolean, boolean) GroovyClassLoader.java                                                                 |  3,984   33 %  |
|  org.codehaus.groovy.tools.javac.JavaAwareCompilationUnit$1.call(SourceUnit, GeneratorContext, ClassNode) JavaAwareCompilationUnit.java                   |  3,438   28 %  |
|  java.net.URLClassLoader.defineClass(String, Resource) URLClassLoader.java                                                                                |  3,070   25 %  |
|  sun.misc.URLClassPath.getResource(String, boolean) URLClassPath.java                                                                                     |  2,978   25 %  |
|  java.lang.ClassLoader.defineClass(String, byte[], int, int, ProtectionDomain) ClassLoader.java                                                           |  2,292   19 %  |
|  java.lang.ClassLoader.defineClass1(String, byte[], int, int, ProtectionDomain, String) ClassLoader.java (native)                                         |  2,292   19 %  |
|  java.security.SecureClassLoader.defineClass(String, byte[], int, int, CodeSource) SecureClassLoader.java                                                 |  2,292   19 %  |
|  sun.misc.URLClassPath$FileLoader.getResource(String, boolean) URLClassPath.java                                                                          |  2,044   17 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolve(ClassNode) ResolveVisitor.java                                                                        |  1,926   16 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolveFromDefaultImports(ClassNode, boolean) ResolveVisitor.java                                             |  1,743   14 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolveOrFail(ClassNode, String, ASTNode) ResolveVisitor.java                                                 |  1,693   14 %  |
|  java.io.File.exists() File.java                                                                                                                          |  1,549   13 %  |
|  java.io.UnixFileSystem.getBooleanAttributes(File) UnixFileSystem.java                                                                                    |  1,549   13 %  |
|  java.io.UnixFileSystem.getBooleanAttributes0(File) UnixFileSystem.java (native)                                                                          |  1,549   13 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolveOrFail(ClassNode, ASTNode) ResolveVisitor.java                                                         |  1,538   13 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolveFromStaticInnerClasses(ClassNode, boolean) ResolveVisitor.java                                         |  1,520   13 %  |
|  org.codehaus.groovy.control.ResolveVisitor.visitConstructorOrMethod(MethodNode, boolean) ResolveVisitor.java                                             |  1,467   12 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolveNestedClass(ClassNode) ResolveVisitor.java                                                             |  1,383   11 %  |
|  org.codehaus.groovy.control.CompilationUnit$12.call(SourceUnit) CompilationUnit.java                                                                     |  1,329   11 %  |
|  org.codehaus.groovy.ast.ClassNode.lazyClassInit() ClassNode.java                                                                                         |  1,292   11 %  |
|  org.codehaus.groovy.vmplugin.v5.Java5.configureClassNode(CompileUnit, ClassNode) Java5.java                                                              |  1,292   11 %  |
|  sun.misc.URLClassPath$JarLoader.getResource(String, boolean) URLClassPath.java                                                                           |  1,096    9 %  |
|  java.lang.Throwable.fillInStackTrace(int) Throwable.java (native)                                                                                        |  1,078    9 %  |
|  java.lang.Throwable.fillInStackTrace() Throwable.java                                                                                                    |  1,078    9 %  |
|  java.util.jar.JarFile.getEntry(String) JarFile.java                                                                                                      |  1,044    9 %  |
|  java.util.jar.JarFile.getJarEntry(String) JarFile.java                                                                                                   |  1,044    9 %  |
|  org.codehaus.groovy.ast.ClassCodeExpressionTransformer.visitConstructorOrMethod(MethodNode, boolean) ClassCodeExpressionTransformer.java                 |  1,036    9 %  |
|  java.util.zip.ZipFile.getEntry(String) ZipFile.java                                                                                                      |  1,032    9 %  |
|  org.codehaus.groovy.control.CompilationUnit$17.call(SourceUnit, GeneratorContext, ClassNode) CompilationUnit.java                                        |  1,030    9 %  |
|  org.codehaus.groovy.control.ResolveVisitor.transform(Expression) ResolveVisitor.java                                                                     |    896    7 %  |
|  org.codehaus.groovy.control.ResolveVisitor.visitBlockStatement(BlockStatement) ResolveVisitor.java                                                       |    859    7 %  |
|  org.codehaus.groovy.classgen.AsmClassGenerator.visitClass(ClassNode) AsmClassGenerator.java                                                              |    791    7 %  |
|  groovyjarjarantlr.TokenBuffer.LA(int) TokenBuffer.java                                                                                                   |    747    6 %  |
|  java.util.zip.ZipFile.getEntry(long, byte[], boolean) ZipFile.java (native)                                                                              |    743    6 %  |
|  org.codehaus.groovy.ast.ClassCodeExpressionTransformer.visitExpressionStatement(ExpressionStatement) ClassCodeExpressionTransformer.java                 |    726    6 %  |
|  org.codehaus.groovy.classgen.AsmClassGenerator.visitConstructorOrMethod(MethodNode, boolean) AsmClassGenerator.java                                      |    720    6 %  |
|  org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.findMethod(ClassNode, String, ClassNode[]) StaticTypeCheckingVisitor.java                    |    709    6 %  |
|  org.codehaus.groovy.control.ResolveVisitor.resolveFromModule(ClassNode, boolean) ResolveVisitor.java                                                     |    685    6 %  |
|  java.lang.Class.getDeclaredMethods0(boolean) Class.java (native)                                                                                         |    683    6 %  |
|  java.lang.Class.privateGetDeclaredMethods(boolean) Class.java                                                                                            |    683    6 %  |
|  java.security.AccessController.doPrivileged(PrivilegedAction) AccessController.java (native)                                                             |    674    6 %  |
|  groovyjarjarantlr.TokenBuffer.fill(int) TokenBuffer.java                                                                                                 |    669    6 %  |
|  org.codehaus.groovy.classgen.AsmClassGenerator.visitStdMethod(MethodNode, boolean, Parameter[], Statement) AsmClassGenerator.java                        |    661    5 %  |
|  org.codehaus.groovy.ast.expr.BinaryExpression.visit(GroovyCodeVisitor) BinaryExpression.java                                                             |    649    5 %  |
|  org.codehaus.groovy.ast.stmt.IfStatement.visit(GroovyCodeVisitor) IfStatement.java                                                                       |    644    5 %  |
|  org.codehaus.groovy.classgen.AsmClassGenerator.visitBlockStatement(BlockStatement) AsmClassGenerator.java                                                |    629    5 %  |
|  org.codehaus.groovy.classgen.asm.StatementWriter.writeBlockStatement(BlockStatement) StatementWriter.java                                                |    627    5 %  |
|  java.lang.Class.getDeclaredMethods() Class.java                                                                                                          |    612    5 %  |
|  java.lang.Exception.<init>(String) Exception.java                                                                                                        |    610    5 %  |
|  java.lang.Throwable.<init>(String) Throwable.java                                                                                                        |    610    5 %  |
|  org.codehaus.groovy.classgen.AsmClassGenerator.visitMethod(MethodNode) AsmClassGenerator.java                                                            |    604    5 %  |
|  java.io.FileInputStream.read(byte[], int, int) FileInputStream.java                                                                                      |    562    5 %  |
|  java.io.FileInputStream.readBytes(byte[], int, int) FileInputStream.java (native)                                                                        |    562    5 %  |
|  org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.registerMethods(Class, boolean, boolean, Map) MetaClassRegistryImpl.java                     |    549    5 %  |
|  java.lang.Class.getDeclaredFields() Class.java                                                                                                           |    527    4 %  |
|  java.lang.Class.getDeclaredFields0(boolean) Class.java (native)                                                                                          |    527    4 %  |
|  java.lang.Class.privateGetDeclaredFields(boolean) Class.java                                                                                             |    527    4 %  |
|  org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.visitBinaryExpression(BinaryExpression) StaticTypeCheckingVisitor.java                       |    523    4 %  |
|  java.net.URL.<init>(URL, String) URL.java                                                                                                                |    508    4 %  |
|  java.net.URL.<init>(URL, String, URLStreamHandler) URL.java                                                                                              |    508    4 %  |
|  sun.net.www.protocol.file.Handler.parseURL(URL, String, int, int) Handler.java                                                                           |    508    4 %  |
|  java.lang.ClassNotFoundException.<init>(String) ClassNotFoundException.java                                                                              |    468    4 %  |
|  java.lang.Exception.<init>(String, Throwable) Exception.java                                                                                             |    468    4 %  |
|  java.lang.ReflectiveOperationException.<init>(String, Throwable) ReflectiveOperationException.java                                                       |    468    4 %  |
|  java.lang.Throwable.<init>(String, Throwable) Throwable.java                                                                                             |    468    4 %  |
|  org.codehaus.groovy.classgen.asm.OptimizingStatementWriter.writeBlockStatement(BlockStatement) OptimizingStatementWriter.java                            |    465    4 %  |
|  sun.misc.Resource.getBytes() Resource.java                                                                                                               |    460    4 %  |
|  java.lang.String.indexOf(char[], int, int, char[], int, int, int) String.java                                                                            |    451    4 %  |
|  java.lang.String.indexOf(String, int) String.java                                                                                                        |    451    4 %  |
|  java.net.URLStreamHandler.parseURL(URL, String, int, int) URLStreamHandler.java                                                                          |    451    4 %  |
|  sun.misc.Resource.getByteBuffer() Resource.java                                                                                                          |    449    4 %  |
|  sun.misc.Resource.cachedInputStream() Resource.java                                                                                                      |    443    4 %  |
|  org.codehaus.groovy.ast.stmt.ReturnStatement.visit(GroovyCodeVisitor) ReturnStatement.java                                                               |    441    4 %  |
|  org.codehaus.groovy.reflection.ReflectionCache.getCachedClass(Class) ReflectionCache.java                                                                |    441    4 %  |
|  org.codehaus.groovy.ast.ClassNode.getUnresolvedSuperClass() ClassNode.java                                                                               |    441    4 %  |
|  org.codehaus.groovy.ast.ClassNode.getUnresolvedSuperClass(boolean) ClassNode.java                                                                        |    441    4 %  |
|  org.spockframework.compiler.SpockTransform.visit(ASTNode[], SourceUnit) SpockTransform.java                                                              |    438    4 %  |
|  org.codehaus.groovy.ast.ClassNode.getDeclaredMethods(String) ClassNode.java                                                                              |    423    3 %  |
|  org.codehaus.groovy.classgen.AsmClassGenerator.visitExpressionStatement(ExpressionStatement) AsmClassGenerator.java                                      |    421    3 %  |
|  org.codehaus.groovy.classgen.asm.StatementWriter.writeExpressionStatement(ExpressionStatement) StatementWriter.java                                      |    421    3 %  |
|  org.codehaus.groovy.ast.expr.DeclarationExpression.visit(GroovyCodeVisitor) DeclarationExpression.java                                                   |    417    3 %  |
|  java.security.AccessController.doPrivileged(PrivilegedAction, AccessControlContext) AccessController.java (native)                                       |    416    3 %  |
|  org.codehaus.groovy.ast.expr.ConstructorCallExpression.visit(GroovyCodeVisitor) ConstructorCallExpression.java                                           |    412    3 %  |
|  org.codehaus.groovy.ast.ClassNode.getMethods(String) ClassNode.java                                                                                      |    400    3 %  |
|  org.codehaus.groovy.control.io.FileReaderSource.getReader() FileReaderSource.java                                                                        |    394    3 %  |
|  org.codehaus.groovy.control.ClassNodeResolver.tryAsScript(String, CompilationUnit, Class) ClassNodeResolver.java                                         |    378    3 %  |
|  org.codehaus.groovy.control.ResolveVisitor.transformMethodCallExpression(MethodCallExpression) ResolveVisitor.java                                       |    375    3 %  |
|  groovy.lang.GroovyClassLoader.getSourceFile(String, String) GroovyClassLoader.java                                                                       |    374    3 %  |
|  groovy.lang.GroovyClassLoader$1.loadGroovySource(String) GroovyClassLoader.java                                                                          |    374    3 %  |
|  groovy.lang.GroovyClassLoader$1$1.run() GroovyClassLoader.java                                                                                           |    374    3 %  |
|  groovy.lang.GroovyClassLoader$1$1.run() GroovyClassLoader.java                                                                                           |    374    3 %  |
|  java.lang.ClassLoader.getResource(String) ClassLoader.java                                                                                               |    374    3 %  |
|  java.net.URLClassLoader.findResource(String) URLClassLoader.java                                                                                         |    374    3 %  |
|  java.net.URLClassLoader$2.run() URLClassLoader.java                                                                                                      |    365    3 %  |
|  java.net.URLClassLoader$2.run() URLClassLoader.java                                                                                                      |    365    3 %  |
|  sun.misc.URLClassPath.findResource(String, boolean) URLClassPath.java                                                                                    |    365    3 %  |
|  groovyjarjarantlr.InputBuffer.LA(int) InputBuffer.java                                                                                                   |    358    3 %  |
|  org.codehaus.groovy.ast.ClassNode.getSuperClass() ClassNode.java                                                                                         |    356    3 %  |
|  org.spockframework.compiler.SpockTransform$Impl.visit(ASTNode[], SourceUnit) SpockTransform.java                                                         |    354    3 %  |
|  java.io.BufferedInputStream.fill() BufferedInputStream.java                                                                                              |    353    3 %  |
|  java.io.BufferedInputStream.read() BufferedInputStream.java                                                                                              |    353    3 %  |
|  org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructor(ConstructorNode) ClassCodeVisitorSupport.java                                           |    348    3 %  |
|  org.spockframework.compiler.SpockTransform$Impl.processSpec(ClassNode, ErrorReporter, SourceLookup) SpockTransform.java                                  |    343    3 %  |
|  org.codehaus.groovy.classgen.asm.BinaryExpressionHelper.evaluateEqual(BinaryExpression, boolean) BinaryExpressionHelper.java                             |    339    3 %  |
|  org.codehaus.groovy.util.LazyReference.get() LazyReference.java                                                                                          |    334    3 %  |
|  org.codehaus.groovy.util.LazyReference.getLocked(boolean) LazyReference.java                                                                             |    334    3 %  |
|  org.codehaus.groovy.ast.expr.PropertyExpression.visit(GroovyCodeVisitor) PropertyExpression.java                                                         |    334    3 %  |
|  org.codehaus.groovy.control.ResolveVisitor.transformVariableExpression(VariableExpression) ResolveVisitor.java                                           |    333    3 %  |
|  org.codehaus.groovy.ast.expr.Expression.transformExpressions(List, ExpressionTransformer) Expression.java                                                |    332    3 %  |
|  org.codehaus.groovy.transform.sc.StaticCompilationVisitor.findMethodOrFail(Expression, ClassNode, String, ClassNode[]) StaticCompilationVisitor.java     |    329    3 %  |
|  org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.findMethodOrFail(Expression, ClassNode, String, ClassNode[]) StaticTypeCheckingVisitor.java  |    329    3 %  |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+

Generated by YourKit Java Profiler 2017.02-b63 May 24, 2018 08:22:36 AM