httpclient如何实现在一个连接中发送多次请求?-问答-阿里云开发者社区-阿里云

开发者社区> 问答> 正文

httpclient如何实现在一个连接中发送多次请求?

蛮大人123 2016-03-10 15:11:57 8074

如题,最好能给个简单的demo,几行也可以,就是核心的在一个连接中允许多次发送请求的代码!谢谢各位了!我用的连接池,如下,就是想实现多次调用post均使用同一个打开的连接,如何实现?测试发现老是会重新建立连接(即新开端口进行通信)!

public class HttpClientTestDemo {

    public static final int MAX_TOTAL_CONNECTIONS = 400;

    public static final int MAX_ROUTE_CONNECTIONS = 100;

    public static final int CONNECT_TIMEOUT = 10000;

    public static final int SOCKET_TIMEOUT = 20000;

    public static final long CONN_MANAGER_TIMEOUT = 10000;

    public static HttpParams parentParams;

    public static PoolingClientConnectionManager cm;

    private static final HttpHost DEFAULT_TARGETHOST = new HttpHost("123.456.789.101", 80);
    public static HttpRequestRetryHandler httpRequestRetryHandler;

    static {

        SchemeRegistry schemeRegistry = new SchemeRegistry();

        schemeRegistry.register(
                new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));

        schemeRegistry.register(
                new Scheme("https", 443, SSLSocketFactory.getSocketFactory()));

        cm = new PoolingClientConnectionManager(schemeRegistry);

        cm.setMaxTotal(MAX_TOTAL_CONNECTIONS);

        cm.setDefaultMaxPerRoute(MAX_ROUTE_CONNECTIONS);

        cm.setMaxPerRoute(new HttpRoute(DEFAULT_TARGETHOST), 20);        

        parentParams = new BasicHttpParams();

        parentParams.setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);

        parentParams.setParameter(ClientPNames.DEFAULT_HOST, DEFAULT_TARGETHOST);    //设置默认targetHost

        parentParams.setParameter(ClientPNames.COOKIE_POLICY, CookiePolicy.BROWSER_COMPATIBILITY);

        parentParams.setParameter(ClientPNames.CONN_MANAGER_TIMEOUT, CONN_MANAGER_TIMEOUT);

        parentParams.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECT_TIMEOUT);

        parentParams.setParameter(CoreConnectionPNames.SO_TIMEOUT, SOCKET_TIMEOUT);

        parentParams.setParameter(ClientPNames.ALLOW_CIRCULAR_REDIRECTS, true);

        parentParams.setParameter(ClientPNames.HANDLE_REDIRECTS, true);

        //设置头信息,模拟浏览器
        Collection collection = new ArrayList();

        collection.add(new BasicHeader("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)"));

        collection.add(new BasicHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"));

        collection.add(new BasicHeader("Accept-Language", "zh-cn,zh,en-US,en;q=0.5"));

        collection.add(new BasicHeader("Accept-Charset", "ISO-8859-1,utf-8,gbk,gb2312;q=0.7,*;q=0.7"));

        collection.add(new BasicHeader("Accept-Encoding", "gzip, deflate"));

        parentParams.setParameter(ClientPNames.DEFAULT_HEADERS, collection);

        //请求重试处理
        httpRequestRetryHandler = new HttpRequestRetryHandler() {

            @Override
            public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
                if (executionCount >= 3) {
                    // 如果超过最大重试次数,那么就不要继续了
                    return false;
                }
                if (exception instanceof NoHttpResponseException) {
                    // 如果服务器丢掉了连接,那么就重试
                    return true;
                }
                if (exception instanceof SSLHandshakeException) {
                    // 不要重试SSL握手异常
                    return false;
                }
                HttpRequest request = (HttpRequest) context.getAttribute(ExecutionContext.HTTP_REQUEST);
                boolean idempotent = !(request instanceof HttpEntityEnclosingRequest);
                if (idempotent) {
                    // 如果请求被认为是幂等的,那么就重试
                    return true;
                }
                return false;
            }

           
        };
    }
    private static String readHtmlContentFromEntity(HttpEntity httpEntity) throws ParseException, IOException {
        String html = "";
        Header header = httpEntity.getContentEncoding();
        {
            InputStream in = httpEntity.getContent();
            if (header != null && "gzip".equals(header.getValue())) {
                html = unZip(in);
            } else {
                html = readInStreamToString(in);
            }
            if (in != null) {
                in.close();
            }
        }
        return html;
    }
    private static String unZip(InputStream in) throws IOException {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        GZIPInputStream gis = null;
        try {
            gis = new GZIPInputStream(in);
            byte[] _byte = new byte[1024];
            int len = 0;
            while ((len = gis.read(_byte)) != -1) {
                baos.write(_byte, 0, len);
            }
            String unzipString = new String(baos.toByteArray(), "utf-8");
            return unzipString;
        } finally {
            if (gis != null) {
                gis.close();
            }
            if (baos != null) {
                baos.close();
            }
            if(in!=null)
            {
                in.close();
            }
        }
    }

    private static String readInStreamToString(InputStream in) throws IOException {
        StringBuilder str = new StringBuilder();
        String line;
        BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(in, "utf-8"));
        while ((line = bufferedReader.readLine()) != null) {
            str.append(line);
            str.append("\r\n");
        }
        if (bufferedReader != null) {
            bufferedReader.close();
        }
        if(in!=null)
            in.close();
        return str.toString();
    }

    public static String post(DefaultHttpClient httpClient, String url) throws UnsupportedEncodingException {
        HttpGet httpGet = new HttpGet(url);
        String result = "";
        httpGet.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, DEFAULT_TARGETHOST);
         HttpResponse httpResponse;
        try {
            httpResponse = httpClient.execute(httpGet);//建立端口映射
            if (httpResponse.getStatusLine().getStatusCode() != 200) {
                httpGet.abort();
                return result;
            }
            result = readHtmlContentFromEntity(httpResponse.getEntity());
        } catch (ClientProtocolException e) {
            httpGet.abort();
            return e.toString();
        } catch (IOException ex) {
            httpGet.abort();
            return ex.toString();
        } finally {
            httpGet.releaseConnection();
            httpClient.close();
        }
        return result;
    }

    public static void main(String[] args) throws InterruptedException {
        Date start = new Date();
        DefaultHttpClient httpClient = new DefaultHttpClient(cm, parentParams);
        httpClient.setHttpRequestRetryHandler(httpRequestRetryHandler);
        try {
            post(httpClient, "https://www.baidu.com/index1.php");//这里的几个网址是随便举例的,实际页面是纯txt,没有任何其他元素
            post(httpClient, "https://www.baidu.com/index2.php");
            post(httpClient, "https://www.baidu.com/index3.php");
        } catch (UnsupportedEncodingException ex) {
            Logger.getLogger(HttpClientTestDemo.class.getName()).log(Level.SEVERE, null, ex);
        }
        Date end = new Date();
        System.out.println((end.getTime() - start.getTime()) / 1000.0 + " 秒");

    }
}
XML 定位技术 网络安全 数据格式 Windows
分享到
取消 提交回答
全部回答(1)
  • 蛮大人123
    2019-07-17 18:57:44

    只有同一个网站,且支持 keepalive 的情况下,才可以用同一个连接.
    否则你这个连接连到了服务器A,怎么可能在不改变端口的情况下,再去连上服务器B进行通信?
    而且 HTTP 服务器也不在意,你客户端用哪个端口连接它的 WEB 端口呀.不知道楼主研究这个是为了什么?
    Sample.java:

    public class Sample {
    
        private static ResponseHandler<String> responseHandler = new ResponseHandler<String>() {
            @Override
            public String handleResponse(
                    final HttpResponse response) throws ClientProtocolException, IOException {
                    HttpEntity entity = response.getEntity();
                    return entity != null ? EntityUtils.toString(entity) : null;
    
            }
    
        };
    
        public static String Get(CloseableHttpClient http, String url){
            HttpGet httpget = new HttpGet(url);
    
    
            System.out.println("Executing request " + httpget.getRequestLine());
    
            String html = "";
    
            try{
                html = http.execute(httpget, responseHandler);
            }finally{
                return html;
            }
        }
    
        public static void main(String[] args) throws Exception {
            CloseableHttpClient http = HttpClients.createDefault();
            try {
                Get(http, "http://www.qq.com/111.txt");
                Get(http, "http://www.qq.com/222.txt");
                Get(http, "http://www.qq.com/333.txt");
            } finally {
                http.close();
            }
        }
    }

    使用的是 httpclient-4.5.1, 下载地址: http://mirrors.hust.edu.cn/apache//httpcomponents/httpclient/binary/httpcomponents-client-4.5.1-bin.tar.gz

    编译方法:
    javac -classpath ".;.\lib\httpclient-4.5.1.jar;.\lib\httpcore-4.4.3.jar" Sample.java

    执行方法:
    java -classpath ".;.\lib\httpclient-4.5.1.jar;.\lib\httpcore-4.4.3.jar;.\lib\commons-logging-1.2.jar" Sample
    目录结构(将压缩包中的 lib 目录保存至与 Sample.java 同一目录内):
    screenshot
    执行结果:
    screenshot
    抓包对连接的监控截图:
    screenshot
    screenshot
    screenshot
    下面截图为以下代码的执行结果:

            `Get(http, "http://www.qq.com/111.txt");
            Get(http, "http://www.cnbeta.com/222.txt");
            Get(http, "http://www.jd.com/333.txt");
            Get(http, "http://www.qq.com/444.txt");`

    screenshot
    screenshot
    screenshot
    screenshot
    简单来讲, 只要你使用的是同一个 httpclient 的实例, 那么相同域名下的URL就会共用同一个连接.

    0 0
开发与运维
使用钉钉扫一扫加入圈子
+ 订阅

集结各类场景实战经验,助你开发运维畅行无忧

推荐文章
相似问题
推荐课程