转载

谈谈 JSP 的预编译加速及如何参与开源

很早之前的文章，写过一篇关于加速Web应用的功能之一：「JSP预编译」 JSP预编译，加速你的应用 ,在其中我们提到通过JSP的预编译，我们可以提前生成JSP对应的Servlet文件，从而节省执行时再生成所带来的时间占用。

JSPC

而整个JSP预编译的核心入口是 JspC 这个工具。

下面两篇是预编译具体的工作原理：

你了解Tomcat是怎样处理Jsp文件的吗?
修改JSP文件实时生效的秘密

新的 Tomcat Release Note 里，有一个功能，是新增了多线程编译，来加速JSP预编译。是一位热心网友提供的 patch。

我们来看下具体内容：

Make the Jasper (JSP Engine) Java file generation process multi-threaded. By default, one thread will be used per core. Based on a patch by Dan Fabulich.

这个 patch主要做了些什么事情呢？我把其中的重点代码摘出来一起看下：

通过输入参数，设置线程数

public void setThreadCount(String threadCount) {
        if (threadCount == null) {
            return;
        }
        int newThreadCount;
        try {
            if (threadCount.endsWith("C")) {
                double factor = Double.parseDouble(threadCount.substring(0, threadCount.length() - 1));
                newThreadCount = (int) (factor * Runtime.getRuntime().availableProcessors());
            } else {
                newThreadCount = Integer.parseInt(threadCount);
            }
        } catch (NumberFormatException e) {
            throw new BuildException("Couldn't parse thread count: " + threadCount);
        }
        if (newThreadCount < 1) {
            throw new BuildException("There must be at least one thread: " + newThreadCount);
        }
        this.threadCount = newThreadCount;
    }

增加线程池的使用

上面根据输入配置好的threadCount，会在后面生成线程池的时候被使用到，同时生成的线程池，会做为提交JSP编译工作的入口。

ExecutorService threadPool = Executors.newFixedThreadPool(threadCount);
ExecutorCompletionService<Void> service = new ExecutorCompletionService<>(threadPool);
int pageCount = pages.size();

            for (String nextjsp : pages) {
                File fjsp = new File(nextjsp);
                if (!fjsp.isAbsolute()) {
                    fjsp = new File(uriRootF, nextjsp);
                }
                if (!fjsp.exists()) {
                    if (log.isWarnEnabled()) {
                        log.warn(Localizer.getMessage(
                                "jspc.error.fileDoesNotExist", fjsp.toString()));
                    }
                    continue;
                }
                String s = fjsp.getAbsolutePath();
                if (s.startsWith(uriRoot)) {
                    nextjsp = s.substring(uriRoot.length());
                }
                if (nextjsp.startsWith("." + File.separatorChar)) {
                    nextjsp = nextjsp.substring(2);
                }
                service.submit(new ProcessFile(nextjsp));
            }

patch的完整diff

Index: java/org/apache/jasper/JspC.java
===================================================================
--- java/org/apache/jasper/JspC.java    (revision 1355850)
+++ java/org/apache/jasper/JspC.java    (working copy)
@@ -43,6 +43,11 @@
 import java.util.Stack;
 import java.util.StringTokenizer;
 import java.util.Vector;
+import java.util.concurrent.Callable;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorCompletionService;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;

 import javax.servlet.jsp.tagext.TagLibraryInfo;

@@ -126,6 +131,7 @@
     protected static final String SWITCH_ENCODING = "-javaEncoding";
     protected static final String SWITCH_SMAP = "-smap";
     protected static final String SWITCH_DUMP_SMAP = "-dumpsmap";
+    protected static final String SWITCH_THREAD_COUNT = "-threadCount";
     protected static final String SHOW_SUCCESS ="-s";
     protected static final String LIST_ERRORS = "-l";
     protected static final int INC_WEBXML = 10;
@@ -209,6 +215,9 @@
      * is UTF-8.  Added per bugzilla 19622.
      */
     protected String javaEncoding = "UTF-8";
+    
+    /** The number of threads to use; default is two per core */
+    protected int threadCount = 0;

     // Generation of web.xml fragments
     protected String webxmlFile;
@@ -363,6 +372,8 @@
                 smapSuppressed = false;
             } else if (tok.equals(SWITCH_DUMP_SMAP)) {
                 smapDumped = true;
+            } else if (tok.equals(SWITCH_THREAD_COUNT)) {
+                setThreadCount(nextArg());
             } else {
                 if (tok.startsWith("-")) {
                     throw new JasperException("Unrecognized option: " + tok +
@@ -720,6 +731,31 @@
         return javaEncoding;
     }

+    public int getThreadCount() {
+        return threadCount;
+    }
+    
+    public void setThreadCount(String threadCount) {
+        if (threadCount == null) return;
+        int newThreadCount;
+        if (threadCount.endsWith("C")) {
+            try {
+                double factor = Double.parseDouble(threadCount.substring(0, threadCount.length() - 1));
+                newThreadCount = (int) (factor * (double) Runtime.getRuntime().availableProcessors());
+            } catch (NumberFormatException e) {
+                throw new BuildException("Couldn't parse thread count: " + threadCount);
+            }
+        } else {
+            try {
+                newThreadCount = Integer.parseInt(threadCount);
+            } catch (NumberFormatException e) {
+                throw new BuildException("Couldn't parse thread count: " + threadCount);
+            }
+        }
+        if (newThreadCount < 1) throw new BuildException("There must be at least one thread: " + newThreadCount);
+        this.threadCount = newThreadCount;
+    }
+    
     /**
      * Sets the encoding to use for
      * java files.
@@ -1305,28 +1341,51 @@
             initWebXml();

             Iterator<String> iter = pages.iterator();
-            while (iter.hasNext()) {
-                String nextjsp = iter.next().toString();
-                File fjsp = new File(nextjsp);
-                if (!fjsp.isAbsolute()) {
-                    fjsp = new File(uriRootF, nextjsp);
-                }
-                if (!fjsp.exists()) {
-                    if (log.isWarnEnabled()) {
-                        log.warn
-                            (Localizer.getMessage
-                             ("jspc.error.fileDoesNotExist", fjsp.toString()));
+            if (threadCount == 0) {
+                threadCount = 2 * Runtime.getRuntime().availableProcessors();
+            }
+            ExecutorService threadPool = null;
+            try {
+                threadPool = Executors.newFixedThreadPool(threadCount);
+                ExecutorCompletionService<Void> service = new ExecutorCompletionService<Void>(threadPool);
+                int fileCount = 0;
+                while (iter.hasNext()) {
+                    String nextjsp = iter.next().toString();
+                    File fjsp = new File(nextjsp);
+                    if (!fjsp.isAbsolute()) {
+                        fjsp = new File(uriRootF, nextjsp);
                     }
-                    continue;
+                    if (!fjsp.exists()) {
+                        if (log.isWarnEnabled()) {
+                            log.warn
+                                (Localizer.getMessage
+                                 ("jspc.error.fileDoesNotExist", fjsp.toString()));
+                        }
+                        continue;
+                    }
+                    String s = fjsp.getAbsolutePath();
+                    if (s.startsWith(uriRoot)) {
+                        nextjsp = s.substring(uriRoot.length());
+                    }
+                    if (nextjsp.startsWith("." + File.separatorChar)) {
+                        nextjsp = nextjsp.substring(2);
+                    }
+                    service.submit(new ProcessFile(nextjsp));
+                    fileCount++;
                 }
-                String s = fjsp.getAbsolutePath();
-                if (s.startsWith(uriRoot)) {
-                    nextjsp = s.substring(uriRoot.length());
+                for (int i = 0; i < fileCount; i++) {
+                    service.take().get();
                 }
-                if (nextjsp.startsWith("." + File.separatorChar)) {
-                    nextjsp = nextjsp.substring(2);
+            } catch (InterruptedException e) {
+                throw new JasperException(e);
+            } catch (ExecutionException e) {
+                if (e.getCause() instanceof JasperException) {
+                    throw (JasperException) e.getCause();
+                } else {
+                    throw new JasperException(e.getCause());
                 }
-                processFile(nextjsp);
+            } finally {
+                if (threadPool != null) threadPool.shutdownNow();
             }

             completeWebXml();
@@ -1354,6 +1413,18 @@
             }
         }
     }
+    
+    private class ProcessFile implements Callable<Void> {
+        private final String file;
+        private ProcessFile(String file) { this.file = file; }
+        
+        @Override
+        public Void call() throws Exception {
+            processFile(file);
+            return null;
+        }
+        
+    }

     // ==================== protected utility methods ====================

Index: java/org/apache/jasper/resources/LocalStrings.properties
===================================================================
--- java/org/apache/jasper/resources/LocalStrings.properties    (revision 1355850)
+++ java/org/apache/jasper/resources/LocalStrings.properties    (working copy)
@@ -184,6 +184,7 @@
 /    -javaEncoding <enc> Set the encoding charset for Java classes (default UTF-8)/n/
 /    -source <version>   Set the -source argument to the compiler (default 1.6)/n/
 /    -target <version>   Set the -target argument to the compiler (default 1.6)/n/
+/    -threadCount <count>   Number of threads. /"2.0C/" means two threads per core/n/

 jspc.webxml.header=<?xml version="1.0" encoding="ISO-8859-1"?>/n/
 /n/

怎样提 Patch

根据不同的开源项目，风格上不太一样。

像 Tomcat 这种较大型的项目，虽然也在 GitHub 上有代码，但主要风格还是前几年流行的 SVN + JIRA + Bugzilla。主要的Bug 和 Feature 的需求，都需要先在 JIRA 或者 Bugzilla上创建。对于提交的 path，做为附件内容，同步提供。之后大家围绕这个话题进行讨论，commiter 会决定是否采用合入。

其它在GitHub上托管的开源项目，直接提交Pull Request，然后邮件联系 commiter即可。对方会在 PR 后面进行讨论，决定是否采用，流程上也更清晰。

一些感想

看上面的 patch 内容，我们可能感觉没啥难度，「不就是用个线程池？」可能现象确实如此，这个思路也比较容易想到。但在这其中的一些思路可以借鉴的，比如输入参数中，会默认设置一个core对应两个线程，我们一般的思路是根据 Runtime 直接取Processor数设置上就完事了。

另外一点是，我上面的历史文章中，有一篇分析文章是2015年写的，网上之前也有不少类似分析源码的文章，好像也都是「仅」分析，「这么容易想到的思路」，为什么没人在分析的时候提个patch? 所以，有些看似简单的地方也都是有可优化的点，甚至方法也不难，难在发现并下手。

说到这里顺便吐槽下 Tomcat，这个 patch 是热心网友在 2012提的，结果直到今年(2018.07)才采用合入，也真是够拖沓的。

谈谈 JSP 的预编译加速及如何参与开源