@TOC
推荐阅读Tomcat原理系列之二:由点到线,请求主干对于理解本文有很多帮助。
Tomcat版本8.
1. 接收连接:
Accptor在接受到socket请求后,执行setSocketOptions方法对socket进行初步的封装。 封装: 首先创建一个SocketBufferHandler用于socket输入输出的缓冲(SocketBuffer)。将SocketBufferHandler与socket一同封装成NioChannel.
public SocketBufferHandler(int readBufferSize, int writeBufferSize, boolean direct) { this.direct = direct; if (direct) { readBuffer = ByteBuffer.allocateDirect(readBufferSize);//默认8k writeBuffer = ByteBuffer.allocateDirect(writeBufferSize); } else { readBuffer = ByteBuffer.allocate(readBufferSize); writeBuffer = ByteBuffer.allocate(writeBufferSize); } }
2. 注册:
调用Poller.register()将NioChannel(socket)先进一步封装成NioSocketWrapper类,再封装成PollerEvent然后注册到Poller的events队列中去。
3. 消费:
Poller.run()消费队列的PollerEvent事件。将PollerEvent中准备就绪的socketChannel注册到Selector。
4. 处理请求:
Poler.run() 从Selector选择处就绪的Channel。调用NioEndpoint.processKey(),processKey()方法中,根据读写事件调用processSocket()处理。
5. Worker线程:
processSocket()会根据(NioSocketWrapper)socket创建一个SocketProcessor处理器。SocketProcessor本身实现了Runnable接口。可以作为任务。被Endpoint的Executor线程池执行。
try { if (socketWrapper == null) { return false; } SocketProcessorBase<S> sc = processorCache.pop(); if (sc == null) { sc = createSocketProcessor(socketWrapper, event); } else { sc.reset(socketWrapper, event); } Executor executor = getExecutor();//线程池 if (dispatch && executor != null) { executor.execute(sc); } else { sc.run(); } } catch (RejectedExecutionException ree) {
SocketProcessor在连接握手成功的情况下,调用ConnectionHandler.process()方法开始socket内容的读取
6. HTTP1.1协议处理器初始化:
ConnectionHandler.process()方法会创建Http11Processor处理器用于http协议的处理. Http11Processor构造方法主要做了,
- 首先会创建一对org.apache.coyote.Request和org.apache.coyote.Response内部coyoteRequest与coyoteResponse对象.
- 并创建Http11InputBuffer与Http11OutputBuffer用于coyoteRequest与coyoteResponse。Http11InputBuffer提供HTTP请求头的解析与编码功能。Http11InputBuffer在创建的时候会指定headerBufferSize的大小.默认也是8k.
7. Http11Processor.service()[HTTP协议头部的解析]:
拿到Http11Processor后.执行核心方法service();第一步:初始化读写缓冲区
// Setting up the I/O setSocketWrapper(socketWrapper); inputBuffer.init(socketWrapper); outputBuffer.init(socketWrapper);
init()方法为Http11InputBuffer内部创建一个读缓冲区byteBuffer.大小为headerBufferSize+socketbuffer的大小.也就是默认是2*8k
void init(SocketWrapperBase<?> socketWrapper) { wrapper = socketWrapper; wrapper.setAppReadBufHandler(this); int bufLength = headerBufferSize + wrapper.getSocketBufferHandler().getReadBuffer().capacity(); if (byteBuffer == null || byteBuffer.capacity() < bufLength) { byteBuffer = ByteBuffer.allocate(bufLength); byteBuffer.position(0).limit(0); } }
第二步开始请求行的解析在解析之前我先来看看HTTP请求报文格式.
inputBuffer.parseRequestLine()方法用来读取请求行。inputBuffer中有个parsingRequestLinePhase属性值.parsingRequestLinePhase值不同代表读取请求行的不同位置.
- 0:表示解析开始前跳过空行
- 2: 开始解析请求方法: POST
- 3: 跳过请求方法和请求uri之间的空格或制表符
- 4: 开始解析请求URI: chapter17/user.html
- 5:跳过请求URI与版本之间的空格
- 6:解析协议版本: HTTP/1.1
parseRequestLine()方法, 每读一个位置时,都会判断inputBuffer.bytebuffer中是否读取完毕。position = limit 即已经读完了,需要执行fill重新填充,参数是false表示非阻塞读(那什么时候阻塞读呢,是我们在调用getInputStream()时,是阻塞的)
// Read new bytes if needed if (byteBuffer.position() >= byteBuffer.limit()) {//判断 if (keptAlive) { // Haven't read any request data yet so use the keep-alive // timeout. wrapper.setReadTimeout(wrapper.getEndpoint().getKeepAliveTimeout()); } if (!fill(false)) {//填充 // A read is pending, so no longer in initial state parsingRequestLinePhase = 1; return false; } // At least one byte of the request has been received. // Switch to the socket timeout. wrapper.setReadTimeout(wrapper.getEndpoint().getConnectionTimeout()); }
fill()填充方法:填充buffer fill()的填充功能是通过调用socket的包装类NioSocketWrapper.read()方法实现的. 在read()方法中 首先会尝试从socketBufferHandler.readbuffer读,
- 如果socketBufferHandler.readbuffer有数据,把数据填充到inputBuffer.bytebuffer中。不需要从socket通道读取。
- 如果socketBufferHandler.readbuffer没有数据可读,且inputBuffer.bytebuffer的可写空间大于socketBufferHandler.readbuffer的容量: 则直接从socket通道中读取。设置该次读取的最大值limit,为socket buffer的大小
- 如果socketBufferHandler.readbuffer没有数据可读,且inputBuffer.bytebuffer的可写空间小于socketBufferHandler.readbuffer的容量:则先从socket通道读入socketBuffer(因为此时socketBuffer的容量大于inputBuffer.bytebuffer的可写空间,可以一次从OS读取更多数据)。然后再从socketBuffer填充到inputBuffer.bytebuffer.(此时填充的是剩余可写空间,这样socketBuffer也会剩余一些,当inputBuffer.bytebuffer读取完毕时,再调用fill()方法时,将剩余socketBuffer的数据填充到inputBuffer.bytebuffer,不需要去socket通道内读,本质上时减少OSread.然后这样循环执行下去,直到所有的读操作完成)
@Override public int read(boolean block, ByteBuffer to) throws IOException { //先从tomcat 底层socket buffer 缓冲区读,如果buffer缓冲区还有未读的buffer,则不需要到OS底层读缓冲区读 int nRead = populateReadBuffer(to); if (nRead > 0) { return nRead; /* * Since more bytes may have arrived since the buffer was last * filled, it is an option at this point to perform a * non-blocking read. However correctly handling the case if * that read returns end of stream adds complexity. Therefore, * at the moment, the preference is for simplicity. */ } // The socket read buffer capacity is socket.appReadBufSize int limit = socketBufferHandler.getReadBuffer().capacity(); if (to.remaining() >= limit) { to.limit(to.position() + limit); nRead = fillReadBuffer(block, to); if (log.isDebugEnabled()) { log.debug("Socket: [" + this + "], Read direct from socket: [" + nRead + "]"); } updateLastRead(); } else { // Fill the read buffer as best we can. nRead = fillReadBuffer(block); if (log.isDebugEnabled()) { log.debug("Socket: [" + this + "], Read into buffer: [" + nRead + "]"); } updateLastRead(); // Fill as much of the remaining byte array as possible with the // data that was just read if (nRead > 0) { nRead = populateReadBuffer(to); } } return nRead; }
read()调用fillReadBuffer()方法来完成从socket通道内读数据。fillReadBuffer有两种读模式阻塞读和非阻塞读.非阻塞读会调用socket的初始包装类NioChannel.read()方法,NioChannel.read()调用SocketChannel.read()此处是真正从通道里读数据.
总结起来说。填充功能其实从socket通道把数据读到inputBuffer.byteBuffer中。
解析:inputBuffer从byteBuffer中解析报文内容.例如请求方法,请求URI。inputBuffer并没有把字节转义。而是使用byte[]数组的包装类MessageBytes来表示请求行的各部分,在需要的时候进行转移并缓冲。
我们以请求方法读取为例:
if (parsingRequestLinePhase == 2) { // // Reading the method name // Method name is a token // boolean space = false; while (!space) { // Read new bytes if needed if (byteBuffer.position() >= byteBuffer.limit()) { if (!fill(false)) // request line parsing return false; } // Spec says method name is a token followed by a single SP but // also be tolerant of multiple SP and/or HT. int pos = byteBuffer.position(); byte chr = byteBuffer.get(); if (chr == Constants.SP || chr == Constants.HT) { space = true; //请求的方法(get/post) request.method().setBytes(byteBuffer.array(), parsingRequestLineStart, pos - parsingRequestLineStart); } else if (!HttpParser.isToken(chr)) { byteBuffer.position(byteBuffer.position() - 1); throw new IllegalArgumentException(sm.getString("iib.invalidmethod")); } }
看代码段,request.method().setBytes并没有把请求报文的请求方法转义为GET/POST字符,而是使用MessageBytes存储了请求报文(即inputBuffer.byteBuffer)起始位到第一个空格之前的字节数组的下标。 在使用的时候将字节转为GET/POST
第三步就是读取请求头inputBuffer.parseHeaders():过程类似读取请求行
第四步读取请求头后会执行prepareRequest():此方法设置request的filters和一些信息的设置。
第五步调用Adapter.service(request, response):将tomcat的内部coyoteRequest和coyoteReponse转换为servlet规范request ,response对象。这里有个一转换的过程。 就是创建servlet规范request ,response对象。然后将coyoteRequest,coyoteReponse分别设置给request,response 接下来就是调用各级容器,走过filter到达servlet中
// Calling the container connector.getService().getContainer().getPipeline().getFirst().invoke( request, response);
8. HTTP协议body的解析
HTTP协议请求body的解析延迟到servlet中在获取参数的时候解析的。body的解析放到其他章节在讲。
总结下:
数据从连接通道copy到堆外内存,然后从堆外内存copy到 tomcat Http11InputBuffer的堆内byteBuffer。然后根据HTTP协议解析byteBuffer中的字节数组。变成HTTP协议的coyoteRequest,coyoteReponse。最后包装成我们常用的request,response对象。
重用:
Tomcat中有很多重用的组件.以减少频繁创建和销毁的开销
- NioChannel:NioChannel channel = nioChannels.pop();
- PollerEvent: PollerEvent r = eventCache.pop();
- SocketProcessor:SocketProcessorBase sc = processorCache.pop();
- Processor:Processor processor = connections.get(socket);
。