工作中用到了curl来获取http返回的状态码，以至于对curl产生了兴趣，想要详细的了解学习一下。其实curl是一个很强大的命令。
1.访问http页面内容，输出到标准输出
- root@Ttpod:~# curl http://wbo.a2sy.0755cdy.com/index.htm
- index
基输出的是文件的源码，像当于在WIN下使用右键，查看源文件看到的内容
- root@Ttpod:~# curl http://www.baidu.com
- <!doctype html><html><head><meta http-equiv="Content-Type" content="text/html;charset=gb2312"><title>百度一下，你就知道 </title><style>html{overflow-y:auto}body{font:12px arial;text-align:center;background:#fff}body,p,form,ul,li{margin:0;padding:0;list-style:none}body,form,#fm{position:relative}td{text-align:left}img{border:0}a{color:#00c}a:active{color:#f60}#u{color:#999;padding:4px 10px 5px 0;text-align:right}#u a{margin:0 5px}#u .reg{margin:0}#m{width:680px;margin:0 auto;}#nv a,#nv b,.btn,#lk{font-size:14px}#fm{padding-left:90px;text-align:left}input{border:0;padding:0}#nv{height:19px;font-size:16px;margin:0 0 4px;text-align:left;text-indent:117px}.s_ipt_wr{width:418px;height:30px;display:inline-block;margin-right:5px;background:url(http://s1.bdstatic.com/r/www/img/i-1.0.0.png) no-repeat -304px 0;border:1px solid #b6b6b6;border-color:#9a9a9a #cdcdcd #cdcdcd #9a9a9a;vertical-align:top}.s_ipt{width:405px;height:22px;font:16px/22px arial;margin:5px 0 0 7px;background:#fff;outline:none;-webkit-appearance:none}.s_btn{width:95px;height:32px;padding-top:2px\9;font-size:14px;background:#ddd url(http://s1.bdstatic.com/r/www/img/i-1.0.0.png);cursor:pointer}.s_btn_h{background-position:-100px 0}.s_btn_wr{width:97px;height:34px;display:inline-block;background:url(http://s1.bdstatic.com/r/www/img/i-1.0.0.png) no-repeat -202px 0;*position:relative;z-index:0;vertical-align:top}#lg img{vertical-align:top;margin-bottom:3px}#lk{margin:33px 0}#lk span{font:14px "宋体"}#lm{height:60px}#lh{margin:16px 0 5px;word-spacing:3px}.tools{position:absolute;top:-4px;*top:10px;right:-13px;}#mHolder{width:62px;position:relative;z-index:296;display:none}#mCon{height:18px;line-height:18px;position:absolute;cursor:pointer;padding:0 18px 0 0;background:url(http://s1.bdstatic.com/r/www/img/bg-1.0.0.gif) no-repeat right -134px;background-position:right -136px\9}#mCon span{color:#00c;cursor:default;display:block}#mCon .hw{text-decoration:underline;cursor:pointer}#mMenu a{width:100%;height:100%;display:block;line-height:22px;text-indent:6px;text-decoration:none;filter:none\9}#mMenu,#user ul{box-shadow:1px 1px 2px #ccc;-moz-box-shadow:1px 1px 2px #ccc;-webkit-box-shadow:1px 1px 2px #ccc;filter: progid:DXImageTransform.Microsoft.Shadow(Strength=2, Direction=135, Color="#cccccc")\9;}#mMenu{width:56px;border:1px solid #9b9b9b;list-style:none;position:absolute;right:7px;top:28px;display:none;background:#fff}#mMenu a:hover{background:#ebebeb}#mMenu .ln{height:1px;background:#ebebeb;overflow:hidden;font-size:1px;line-height:1px;margin-top:-1px}#cp,#cp a{color:#77c}#seth{display:none;behavior:url(#default#homepage)}#setf{display:none;}#sekj{margin-left:14px;}</style>
- <script type="text/javascript">function h(obj){obj.style.behavior='url(#default#homepage)';var a = obj.setHomePage('http://www.baidu.com/');}</script></head>
- <body><div id="u"><a href="http://www.baidu.com/gaoji/preferences.html" name="tj_setting">搜索设置</a>|<a href="https://passport.baidu.com/v2/?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2F" name="tj_login">登录</a><a href="https://passport.baidu.com/v2/?reg&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2F" target="_blank" name="tj_reg" class="reg">注册</a></div>
- <div id="m"><p id="lg"><img src="http://www.baidu.com/img/baidu_sylogo1.gif" width="270" height="129" usemap="#mp"><map name="mp"><area shape="rect" coords="40,25,230,95" href="http://hi.baidu.com/baidu/" target="_blank" title="点此进入 百度的空间" ></map></p><p id="nv"><a href="http://news.baidu.com">新 闻</a> <b>网 页</b> <a href="http://tieba.baidu.com">贴 吧</a> <a href="http://zhidao.baidu.com">知 道</a> <a href="http://mp3.baidu.com">MP3</a> <a href="http://image.baidu.com">图 片</a> <a href="http://video.baidu.com">视 频</a> <a href="http://map.baidu.com">地 图</a></p><div id="fm"><form name="f" action="/s"><span class="s_ipt_wr"><input type="text" name="wd" id="kw" maxlength="100" class="s_ipt"></span><input type="hidden" name="rsv_bp" value="0"><input type="hidden" name="rsv_spt" value="3"><span class="s_btn_wr"><input type="submit" value="百度一下" id="su" class="s_btn" onmousedown="this.className='s_btn s_btn_h'" onmouseout="this.className='s_btn'"></span></form><span class="tools"><span id="mHolder"><div id="mCon"><span>输入法</span></div></span></span><ul id="mMenu"><li><a href="#" name="ime_hw">手写</a></li><li><a href="#" name="ime_py">拼音</a></li><li class="ln"></li><li><a href="#" name="ime_cl">关闭</a></li></ul></div>
- <p id="lk"><a href="http://hi.baidu.com">空间</a> <a href="http://baike.baidu.com">百科</a> <a href="http://www.hao123.com">hao123</a><span> | <a href="/more/">更多>></a></span></p><p id="lm"></p><p><a id="seth" onClick="h(this)" href="http://utility.baidu.com/traf/click.php?id=215&url=http://www.baidu.com" onmousedown="return ns_c({'fm':'behs','tab':'homepage','pos':0})">把百度设为主页</a><a id="setf" href="http://www.baidu.com/cache/sethelp/index.html" onmousedown="return ns_c({'fm':'behs','tab':'favorites','pos':0})" target="_blank">把百度设为主页</a><span id="sekj"><a href="http://www.baidu.com/search/baidukuijie_mp.html" target="_blank" onmousedown="return ns_c({'fm':'behs','tab':'kuaijie','pos':1})">把百度添加到桌面</a></span></p>
- <p id="lh"><a href="http://e.baidu.com/?refer=888">加入百度推广</a> | <a href="http://top.baidu.com">搜索风云榜</a> | <a href="http://home.baidu.com">关于百度</a> | <a href="http://ir.baidu.com">About Baidu</a></p><p id="cp">©2012 Baidu <a href="/duty/">使用百度前必读</a> <a href="http://www.miibeian.gov.cn" target="_blank">京ICP证030173号</a> <img src="http://www.baidu.com/cache/global/img/gs.gif"></p></div></body>
- <script type="text/javascript" src="http://s1.bdstatic.com/r/www/cache/global/js/home-1.0.js"></script><script>var bdUser = null;var w=window,d=document,n=navigator,k=d.f.wd,a=d.getElementById("nv").getElementsByTagName("a"),isIE=n.userAgent.indexOf("MSIE")!=-1&&!window.opera;for(var i=0;i<a.length;i++){a[i].onclick=function(){if(k.value.length>0){var o=this,h=o.href,q=encodeURIComponent(k.value);if(h.indexOf("qq=")!=-1){o.href=h.replace(/q=[^&\x24]*/,"q="+q)}else{this.href+="?q="+q}}}};(function(){if(/q=([^&]+)/.test(location.search)){k.value=decodeURIComponent(RegExp["\x241"])}})();if(n.cookieEnabled&&!/sug?=0/.test(d.cookie)){bds.se.sug();};function addEV(o, e, f){if(w.attachEvent){o.attachEvent("on" + e, f);}else if(w.addEventListener){ o.addEventListener(e, f, false);}}function G(id){return d.getElementById(id);}function ns_c(q){var p = encodeURIComponent(window.document.location.href), sQ = '', sV = '', mu='', img = window["BD_PS_C" + (new Date()).getTime()] = new Image();for (v in q) {sV = q[v];sQ += v + "=" + sV + "&";} mu= "&mu=" + p ;img.src = "http://nsclick.baidu.com/v.gif?pid=201&pj=www&rsv_sid=&" + sQ + "path="+p+"&t="+new Date().getTime();return true;}if(/\bbdime=[12]/.test(d.cookie)){document.write('<script src=http://s1.bdstatic.com/r/www/cache/ime/js/openime-1.0.0.js><\/script>');}(function(){var user = G("user"), userMenu = G("userMenu");if(user && userMenu){user.onmouseover = function(){userMenu.style.display = "block"};user.onmouseout = function(){userMenu.style.display = "none"};}})();(function(){var u = G("u").getElementsByTagName("a"), nv = G("nv").getElementsByTagName("a"), lk = G("lk").getElementsByTagName("a"), un = "";var tj_nv = ["news","tieba","zhidao","mp3","img","video","map"];var tj_lk = ["hi","baike","hao123","more"];un = bdUser == null ? "" : bdUser;function _addTJ(obj){addEV(obj, "mousedown", function(e){var e = e || window.event;var target = e.target || e.srcElement;ns_c({'fm':'behs','tab':target.name||'tj_user','un':encodeURIComponent(un)});});}for(var i = 0; i < u.length; i++){_addTJ(u[i]);}for(var i = 0; i < nv.length; i++){nv[i].name = 'tj_' + tj_nv[i];_addTJ(nv[i]);}for(var i = 0; i < lk.length; i++){lk[i].name = 'tj_' + tj_lk[i];_addTJ(lk[i]);}})();addEV(w,"load",function(){k.focus()});w.onunload=function(){};</script>
- </html>
2.把curl的结果保存到指定文件
- root@Ttpod:~# curl -o /tmp/index.htm http://www.baidu.com
- % Total % Received % Xferd Average Speed Time Time Time Current
- Dload Upload Total Spent Left Speed
- 100 8023 100 8023 0 0 104k 0 --:--:-- --:--:-- --:--:-- 244k
-o选项用于指定保存文件的位置，然后我们把文件下载到WIN下，使用IE访问一下
3.指定浏览器信息
可以使用 -A “” 来指定服务器WEB日志里记录的浏览器类型信息
- root@Ttpod:~# curl -A "Hello,buddy! " http://wbo.a2sy.0755cdy.com/index.htm
- index
来看下服务器日志记录
- 180.173.110.152`wbo.a2sy.0755cdy.com`HTTP/1.1`GET`80`234`/index.htm`?-`200`0.000`[11/Jun/2012:14:18:33 +0800]`-`Hello,buddy!
- 180.173.110.152`wbo.a2sy.0755cdy.com`HTTP/1.1`GET`80`234`/index.htm`?-`200`0.000`[11/Jun/2012:14:18:38 +0800]`-`Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1145.0 Safari/537.1
- 180.173.110.152`wbo.a2sy.0755cdy.com`HTTP/1.1`GET`80`214`/favicon.ico`?-`200`0.000`[11/Jun/2012:14:18:38 +0800]`-`Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1145.0 Safari/537.1
4.批量下载文件
- root@Ttpod:~# curl -o /tmp/{1..666}.mp3 http://wbo.a2sy.0755cdy.com/[0-9]/[0-9][0-9]/[0-9][0-9]/[a-z][0-9].mp3
批量爬取网站上的mp3文件，支持globing通配，如果globing通配写的好，就能得到自己想要的结果。
5.分块下载
curl -r 0-1024 http://www.xxx.com/aa.zip
curl -r 1025- http://www.xxx.com/aa.zip
先下1M,然后再下剩下的
6. 访问FTP
curl -u username:password ftp://www.xxx.com
上传文件到ftp
curl -T /tmp/test.mp3 -u username:password ftp://www.xxx.com
7.测试参数
测试连接www.google.com花费了多少时间
- curl -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total} www.google.com
查看http_code,可以查看http的返回状态码
- root@Ttpod:~# curl -o /dev/null -s -w %{http_code} http://hcw3.pz8.itaround.com/index.htm
- 200
查看网页文件的大小
- root@Ttpod:~# curl -o /dev/null -s -w %{size_header} http://hcw3.pz8.itaround.com/index.htm
- 228
常用http变量
- http_code:http返回类似404,200,500等
- time_total:总相应时间
- time_namelookup:域名解析时间
- time_connect:连接到目标地址耗费的时间
- time_pretransfer:从执行到开始传输文件的时间间隔
- time_starttransfer:从执行到开始传输文件的时间间隔
- size_download:下载网页或文件大小
- size_upload:上传文件大小
- size_header:响应头
- size_request:发送请求参数大小
- speed_download：传输速度
- speed_upload:平均上传速度
- content_type:下载文件类型.
8.指定超时时间
用一个简单的脚本来说明
- #!/bin/bash
- while read url; do
- service_status=$(curl -o /dev/null -s -m 10 --connect-timeout 10 -w %{http_code} $url)
- echo $url $service_status
- done < domain.txt
从文件domain.txt里面读入URL，然后用curl命令来测试，把测试的结果给变量service_status，然后输出URL 输出http状态
参数解释
-o 上面说过，指定文件保存文件
-s 静默模式，不输出到标准输出
-m 指定文件传输的超时时间，单位为秒
本文转自 gm100861 51CTO博客，原文链接:http://blog.51cto.com/gm100861/895417
