andyro1984
andyro1984_个人页
andyro1984
文章
211
问答
0
视频
0
个人介绍
本团队有11年以上的解决方案端到端开发经验，涉及的行业有云计算、应用软件(包括WEB)、嵌入式、分布式、大型服务程序(Windows/Linux)、操作系统等。
擅长的技术
-
发表了文章 2017-06-11
Standards-zigbee-smart-energy-1-2-revision-4
Standards-zigbee-smart-energy-1-2-revision-4 http://www.zigbee.org/download/standards-zigbee-smart-energy-1-2-revision-4/
-
发表了文章 2017-05-29
C/S模式充电桩管理软件向微服务架构演进
微服务架构在最近两年炒比较火热，最近有个朋友在做充电桩管理软件，该软件是两年前采用C/S模式开发的 ，主要Client（UI）和 Server端两个层次，中间采用数据库共享方式进行通信，如下图所示为充电桩管理软件的客户端界面： 这类应用是传统的C/S模式，适合于30个场站以下的管理和应用，在当前充电桩整体规模不大的情况下，还是勉强可以支撑试用的，最近我这位朋友遇到一个新需求，要接入到第三方的管理平台（B/S模式）中，要求提供标准的REST接口。由于传统的应用开发者（尤其是以嵌入式为主的开发工程师），对REST 等类型的互联网接口存在一定的陌生，在向我咨询后，我们仔细分析了该项目的需求，我给他推荐了向微服务框架演进的方法，最后他们顺利的接入到第三方的管理平台中，或者集成资质。本文将主要阐述我们将传统的充电桩管理软件向微服务框架演进的经验，以下分几个层面进行拆分和演进： 1\业务闭环、颗粒度细分、RPC交互 原来的充电桩管理软件主要划分为两个模块（UI和后台服务器），UI主要用于客户信息维护、充值、充电桩状态监视等功能，后台服务器主要负责充电桩接入、鉴权、执行任务、数据采集等功能。通过对这两个模块进行分析，我们将 UI和后台服务器都进行了拆分。 UI拆分成三个模块（本地大屏监控、本地运营客户端 、中心监控客户端，通过权限进行区分），本地大屏监控直接从本地服务程序中获取数据，部署在场站，用于实时反馈充电桩状态（忙、闲状态），便于客户快速的定位到空闲的充电桩进行充电。本地运营客户端主要用于本地充值、场站经营状况管理等功能。中心监控客户端是一个功能全集，用于监控所有的充电桩状态、经营情况等。 后台服务器拆分成接入管理、鉴权、计划、事件四个模块。接入管理负责充电桩的接入和心跳维护，采用Boost ASIO实现，支持30000+充电桩接入，心跳周期为15秒一次。鉴权：对设备进行认证管理，剔除非法的设备、并对设备进行分域管理。计划：负责根据客户定制的计划，定期执行任务或者采集数据。事件: 负责收集和存储充电桩的各类事件，并上报。 拆分的各个模块都支持按域动态扩容，在传输层支持主备IP备份策略，各个模块之间采用Google Protobuf RPC进行交付通信。 2\业务服务下沉、本地自治、多级缓冲 为了防止因为WLAN网络异常影响，我们做了多级防护，多级缓冲，保障数据不丢失，业务不中断。 a,所有模块均支持下沉部署，模块间通过标识和类型进行识别。 b,本地采用SQLite数据库进行本地业务最小化自治，SQLITE保持设备和用户最小信息，支持后付费同步功能，当WLAN侧出现异常，本地可保障业务基本运行，不中断，WLAN恢复后可自动同步信息到数据中心。 c,对于关键数据采用确认机制和多重缓冲，例如WLAN网络异常，在本地保存60+天以上的业务数据缓冲LOGS, WLAN恢复后，自动同步到数据中心。 d,数据进行层层过滤，每层过滤后将最小量级数据上报给上一层，避免数据中心数据库爆裂式增长，满足核心基础数据要求为准要。 3\Go语言尝试、统一应用层接口 除了在自身架构上的调整，为了提供接口给第三方应用，按照要求，我们采用REST接口，并将服务层和应用层进行彻底隔离，统一接口，按域区分不同的应用和权限，支持多应用接入。因此我们新拆分的中心监控客户端进行调整，也通过统一的接口模块接入到后台服务中，不过我们对域进行保留，0x00~0x80的域认为是自己系统的设备。其他域开放给第三方应用。在这个过程中第一次推荐采用Go语言实现，引用了BeeGo开源代码作为基础框架，同时支持WebSocket接口方式发布即时状态、告警和事件。在后续朋友的开发过程中，足以证明，Go语言天生就是做互联网的开发语言，极高的开发速率。为了防止统一接口管理模块存在性能瓶颈，我们采用Zookeeper + Nginx 作为集群基础框架，提高稳定性，便于后续轻松扩容。 经过2个月的架构调整，整套系统按期交付，即兼容了老系统的基础模块，又完成了和第三方应用的业务对接，做到了业务上的平滑过渡，性能有原来的30个场站规模，演进成可以平滑扩容的架构，初步预估可以做到10万个充电桩的接入和业务运营。经过这个项目历练，朋友在微服务框架上有了全新的认识，也被Go语言的高校的开发效率和朴质的编程规范折服。
-
发表了文章 2017-05-28
Node Joining Process in 6LoWPAN - ND, RPL
https://ez.analog.com/docs/DOC-12488 In typical 6LoWPAN networks, the registration is normally referred to as the node joining process. The complete flow is as outlined below. The first step in the network registration is Neighbor Discovery (ND). This helps the node to determine the neighbors in the vicinity and to select the best parent available. The node will first transmit a RS (Router Solicitation) packet as a multicast to all the routers. On receiving the RS packet all the routers respond back with a RA (Router Advertisement) as a unicast to the node. The RA packet will contain the following information: Prefix Information (PIO) : The prefix of the IPv6 address Context Option (CO) : The compression technique to be used. Authoritative Border Router Option (ABRO): Border Router address refer to IETF RFC 6775: Neighbor Discovery Optimization for IPv6 over Low-Power Wireless Personal Area Networks (6LoWPANs) Upon receiving the RA, the node selects a router as its default router (based on first received RA) and derives the global IPv6 address based on the prefix option. The node then sends a Neighbor Solicitation (NS) as a unicast message to its default router. The NS will contain the Address Registration Option (ARO). This option will tell the router that the node is directly reachable and also the link layer address of the node. The router will make an entry of the node in its Neighbor Cache and respond with a Neighbor Advertisement (NA) with the status of address registration. The following are the status of Address Registration that a Router can respond with: 0 - Success 1 - Duplicate Address 2 - Neighbor Cache Full The Node will send the ARO with a lifetime and will repeat the NUD (Neighbor Unreachability Detection) by sending periodic NS messages to its default router at regular intervals. On receiving the Neighbor Advertisement (NA), the Neighbor Detection (ND) is complete and the node will have the address of the default router in its Neighbor Cache. Similarly the default router of the Node will have the Node’s address in its neighbor cache. Upon completion of Neighbor Discovery, RPL is initialized and the network registration process will begin. The Node will now send a DODAG Information Solicitation (DIS) in response to which the router transmits a DODAG Information Object (DIO). The DIO contains the rank, metrics and PIO. The routers will keep broadcasting their DIOs in regular interval following a trickle timer. If a DIO is received from a router with better rank than the default router, the node re-registers itself with the new router (by sending NS). Once the DIO is received the upward path (to reach the border router) is established. In case of the AD6LoWPAN, the rank depends on the orbit of the Router. The Node will now send a Destination Advertisement Object (DAO) to its default router to be forwarded to the Border Router (BR). The RPL uses Destination Advertisement Object (DAO) messages to establish Downward route to reach the Node. On receiving the DAO, the border router responds with a DAO ACK. This packet is forwarded to the node from its parent. Subsequent to the node receiving the DAO ACK, it is considered that the network registration is completed. This FAQ was generated from the following discussion: How does a 6LoWPAN device register to network?
-
发表了文章 2012-11-17
SGI STL空间适配器freelist设计
今天重新看了《STL源代码剖析》，不禁要赞叹STL设计的经典。STL 的空间适配代码设计的尤为精辟，不仅考虑到内存碎片的隐患，而且考虑到指针空间的节俭和复用，降低维护链表（lists）带来额外的负担。我们来看看如下代码； 先看看STL的结构体， union obj{ union obj *free_list_link; char clent_data[1]; /* the client sees this */ }; obj 之所以用union，由于union之故，从其第一个字段观之，obj可被视为一个指针，指向相同形式的另一个obj。从其第二个字段观之，obj可被视为一个指针，指向实际区域。一物二用的结果是，不会为了维护链表所必须的指针而造成内存的另一种让费。 看看如下示例代码，你就发现它的设计精妙之处！ #include<stdio.h> #include<stdlib.h> #include<string.h> #include<iostream> using namespace std; union obj{ union obj *free_list_link; char clent_data[1]; /* the client sees this */ }; int main() { obj *op1 = (obj *)malloc(32); strcpy(op1->clent_data,"hello free!"); obj *op2 = (obj *)malloc(32); op2->free_list_link = NULL; obj *op3 = (obj *)malloc(32); strcpy(op3->clent_data,"hello free l"); printf("----------------------------------------\r\n"); printf("address freelist:%ld\r\n",(long)(op3->free_list_link)); printf("address clentdata:%ld\r\n",(long)&(op3->clent_data)); obj *op4 = (obj *)malloc(32); op4->free_list_link = op2; obj *op5 = (obj *)malloc(32); op5->free_list_link = op4; printf("----------------------------------------\r\n"); obj *begin = op5; while(begin){ printf("%ld\r\n",(long)begin); begin = begin->free_list_link; } /* release op3 */ op2->free_list_link = op3; op3->free_list_link = NULL; printf("----------------------------------------\r\n"); printf("address freelist:%ld\r\n",(long)(op3->free_list_link)); printf("address clentdata:%ld\r\n",(long)&(op3->clent_data)); printf("----------------------------------------\r\n"); begin = op5; while(begin){ printf("begin:%ld\r\n",(long)begin); begin = begin->free_list_link; } return 0; } 输出如下： ----------------------------------------address freelist:1819043176address clentdata:143143000----------------------------------------143143080143143040143142960----------------------------------------address freelist:0address clentdata:143143000----------------------------------------begin:143143080begin:143143040begin:143142960begin:143143000
-
发表了文章 2012-07-14
Why Cloud?
In data centers today, many computers suffer the same underutilization in computingpower and networking bandwidth. For example, projects may need a large amount of computing capacity to complete a computation, but no longer need the computingpower after completing the computation. You want cloud computing when you want a service that's available on-demand with the flexibility to bring it up or down through automation or with little intervention. The phrase "cloud computing" is often represented with a diagram that contains a cloud-like shape indicating a layer where responsibility for service goes from user to provider. The cloud in these types of diagrams contains the services that afford computing power harnessed to get work done. Much like the electrical power we receive each day, cloud computing provides subscribers or users with access to a shared collection of computing resources: networks for transfer, servers for storage, and applications or services for completing tasks. These are the compelling features of a cloud:• On-demand self-service: Users can provision servers and networks with little humanintervention.• Network access: Any computing capabilities are available over the network. Manydifferent devices are allowed access through standardized mechanisms.• Resource pooling: Multiple users can access clouds that serve other consumers according to demand.• Elasticity: Provisioning is rapid and scales out or in based on need.• Metered or measured service: Just like utilities that are paid for by the hour, clouds should optimize resource use and control it for the level of service or type of servers such as storage or processing. Cloud computing offers different service models depending on the capabilities a consumer may require • SaaS: Software as a Service. Provides the consumer the ability to use the software in acloud environment, such as web-based email for example.• PaaS: Platform as a Service. Provides the consumer the ability to deploy applications through a programming language or tools supported by the cloud platform provider. An example of platform as a service is an Eclipse/Java programming platform provided with no downloads required.• IaaS: Infrastructure as a Service. Provides infrastructure such as computer instances,network connections, and storage so that people can run any software or operatingsystem When you hear terms such as public cloud or private cloud, these refer to the deployment model for the cloud. A private cloud operates for a single organization, but can be managed on-premise or off-premise. A public cloud has an infrastructure that is available to the general public or a large industry group and is likely owned by a cloud services company. The NIST also defines community cloud as shared by several organizations supporting a specific community with shared concerns. Clouds can also be described as hybrid. A hybrid cloud can be a deployment model, as a composition of both public and private clouds, or a hybrid model for cloud computing may involve both virtual and physical servers. What have people done with cloud computing? Cloud computing can help with large-scale computing needs or can lead consolidation efforts by virtualizing servers to make more use of existing hardware and potentially release old hardware from service. People also use cloud computing for collaboration because of its high availability through networked computers. Productivity suites for word processing, number crunching, and email communications, and more are also available through cloud computing. Cloud computing also avails additional storage to the cloud user, avoiding the need for additional hard drives on each user's desktop and enabling access to huge data storage capacity online in the cloud
-
发表了文章 2012-06-14
placement new 在一块已分配内存上创建对象
placement new 是重载operator new的一个标准、全局的版本，它不能被自定义的版本代替（不像普通的operator new和operator delete能够被替换成用户自定义的版本）。它的原型如下： void *operator new( size_t, void *p ) throw() { return p; } 首先我们区分下几个容易混淆的关键词：new、operator new、placement new new和delete操作符我们应该都用过，它们是对堆中的内存进行申请和释放，而这两个都是不能被重载的。要实现不同的内存分配行为，需要重载operator new，而不是new和delete。 看如下代码： class MyClass {…}; MyClass * p=new MyClass; 这里的new实际上是执行如下3个过程： 1. 调用operator new分配内存 ；2. 调用构造函数生成类对象；3. 返回相应指针。 operator new就像operator+一样，是可以重载的，但是不能在全局对原型为void operator new(size_t size)这个原型进行重载，一般只能在类中进行重载。如果类中没有重载operator new，那么调用的就是全局的::operator new来完成堆的分配。同理，operator new[]、operator delete、operator delete[]也是可以重载的，一般你重载的其中一个，那么最后把其余的三个都重载一遍。 至于placement new才是本文的重点。其实它也只是operator new的一个重载的版本，只是我们很少用到它。如果你想在已经分配的内存中创建一个对象，使用new时行不通的。也就是说placement new允许你在一个已经分配好的内存中（栈或者堆中）构造一个新的对象。原型中void*p实际上就是指向一个已经分配好的内存缓冲区的的首地址。 我们知道使用new操作符分配内存需要在堆中查找足够大的剩余空间，这个操作速度是很慢的，而且有可能出现无法分配内存的异常（空间不够）。placement new就可以解决这个问题。我们构造对象都是在一个预先准备好了的内存缓冲区中进行，不需要查找内存，内存分配的时间是常数；而且不会出现在程序运行中途出现内存不足的异常。所以，placement new非常适合那些对时间要求比较高，长时间运行不希望被打断的应用程序。 使用方法如下： 1. 缓冲区提前分配 可以使用堆的空间，也可以使用栈的空间，所以分配方式有如下两种： class MyClass {…}; char *buf=new char[N*sizeof(MyClass)+sizeof(int)];或者char buf[N*sizeof(MyClass)+sizeof(int)]; 2. 对象的构造 MyClass * pClass=new(buf) MyClass; 3. 对象的销毁 一旦这个对象使用完毕，你必须显式的调用类的析构函数进行销毁对象。但此时内存空间不会被释放，以便其他的对象的构造。 pClass->~MyClass(); 4. 内存的释放 如果缓冲区在堆中，那么调用delete[] buf;进行内存的释放；如果在栈中，那么在其作用域内有效，跳出作用域，内存自动释放。 注意： 在C++标准中，对于placement operator new []有如下的说明： placement operator new[] needs implementation-defined amount of additional storage to save a size of array. 所以我们必须申请比原始对象大小多出sizeof(int)个字节来存放对象的个数，或者说数组的大小。使用方法第二步中的new才是placement new，其实是没有申请内存的，只是调用了构造函数，返回一个指向已经分配好的内存的一个指针，所以对象销毁的时候不需要调用delete释放空间，但必须调用析构函数销毁对象。 [使用方法] 在很多情况下，placement new的使用方法和其他普通的new有所不同。这里提供了它的使用步骤。第一步 缓存提前分配 为了保证通过placement new使用的缓存区的memory alignmen(内存队列)正确准备，使用普通的new来分配它： class Task ; char * buff = new [sizeof(Task)]; //分配内存 (请注意auto或者static内存并非都正确地为每一个对象类型排列，所以，你将不能以placement new使用它们。)第二步：对象的分配 在刚才已分配的缓存区调用placement new来构造一个对象。 Task *ptask = new(buff) Task第三步：使用 按照普通方式使用分配的对象： ptask->suspend(); ptask->resume(); //... 第四步：对象的毁灭 一旦你使用完这个对象，你必须调用它的析构函数来毁灭它。按照下面的方式调用析构函数： ptask->~Task(); //调用外在的析构函数 第五步：释放 你可以反复利用缓存并给它分配一个新的对象（重复步骤2，3，4）如果你不打算再次使用这个缓存，你可以象这样释放它： delete [] buff; 跳过任何步骤就可能导致运行时间的崩溃，内存泄露，以及其它的意想不到的情况。如果你确实需要使用placement new，请认真遵循以上的步骤。 【Q & A】 1、placement new 为何物？ placement new 是重载operator new 的一个标准、全局的版本，它不能够被自定义的版本代替（不像普通版本的operator new 和 operator delete能够被替换）。 void *operator new( size_t, void *p ) throw() { return p; } placement new的执行忽略了size_t参数，只返还第二个参数。其结果是允许用户把一个对象放到一个特定的地方，达到调用构造函数的效果。 class SPort { ... }; // represents a serial port const int comLoc = 0x00400000; // location of a port //... void *comAddr = reinterpret_cast<void *>(comLoc); SPort *com1 = new (comAddr) SPort; // create object at comLoc com1->~SPort(); //释放 2、new 、operator new 和 placement new 一样吗？ new ：不能被重载，其行为总是一致的。它先调用operator new分配内存，然后调用构造函数初始化那段内存。 operator new：要实现不同的内存分配行为，应该重载operator new，而不是new。 delete和operator delete类似。 placement new：只是operator new重载的一个版本。它并不分配内存，只是返回指向已经分配好的某段内存的一个指针。因此不能删除它，但需要调用对象的析构函数。 3、在已有的内存上用placement new分配数组 const int numComs = 4; //... SPort *comPorts = new (comAddr) SPort[numComs]; // create array int i = numComs; while( i ) comPorts[--i].~SPort(); 4、用Placement new 解决buffer的问题 用new分配的数组缓冲时，由于调用了默认构造函数，因此执行效率上不佳。若没有默认构造函数则会发生编译时错误。用Placement new可以解决此类问题。 const size_t n = sizeof(string) * BUFSIZE; string *sbuf = static_cast<string *>(::operator new( n )); int size = 0; //此时，buffer还没有初始化，因此需要用 placement new 调用copy构造函数初始化。 void append( string buf[], int &size, const string &val ) { new (&buf[size++]) string( val ); } // placement new //最后的清理 void cleanupBuf( string buf[], int size ) { while( size ) buf[--size].~string(); // destroy initialized elements ::operator delete( buf ); // free storage }
-
发表了文章 2012-05-06
Servlet 工作原理解析
从 Servlet 容器说起 要介绍 Servlet 必须要先把 Servlet 容器说清楚，Servlet 与 Servlet 容器的关系有点像枪和子弹的关系，枪是为子弹而生，而子弹又让枪有了杀伤力。虽然它们是彼此依存的，但是又相互独立发展，这一切都是为了适应工业化生产的结果。从技术角度来说是为了解耦，通过标准化接口来相互协作。既然接口是连接 Servlet 与 Servlet 容器的关键，那我们就从它们的接口说起。 前面说了 Servlet 容器作为一个独立发展的标准化产品，目前它的种类很多，但是它们都有自己的市场定位，很难说谁优谁劣，各有特点。例如现在比较流行的 Jetty，在定制化和移动领域有不错的发展，我们这里还是以大家最为熟悉 Tomcat 为例来介绍 Servlet 容器如何管理 Servlet。Tomcat 本身也很复杂，我们只从 Servlet 与 Servlet 容器的接口部分开始介绍，关于 Tomcat 的详细介绍可以参考我的另外一篇文章《 Tomcat 系统架构与模式设计分析》。 Tomcat 的容器等级中，Context 容器是直接管理 Servlet 在容器中的包装类 Wrapper，所以 Context 容器如何运行将直接影响 Servlet 的工作方式。 图 1 . Tomcat 容器模型从上图可以看出 Tomcat 的容器分为四个等级，真正管理 Servlet 的容器是 Context 容器，一个 Context 对应一个 Web 工程，在 Tomcat 的配置文件中可以很容易发现这一点，如下： 清单 1 Context 配置参数 <Context path="/projectOne " docBase="D:\projects\projectOne" reloadable="true" /> 下面详细介绍一下 Tomcat 解析 Context 容器的过程，包括如何构建 Servlet 的过程。 Servlet 容器的启动过程 Tomcat7 也开始支持嵌入式功能，增加了一个启动类 org.apache.catalina.startup.Tomcat。创建一个实例对象并调用 start 方法就可以很容易启动 Tomcat，我们还可以通过这个对象来增加和修改 Tomcat 的配置参数，如可以动态增加 Context、Servlet 等。下面我们就利用这个 Tomcat 类来管理新增的一个 Context 容器，我们就选择 Tomcat7 自带的 examples Web 工程，并看看它是如何加到这个 Context 容器中的。 清单 2 . 给 Tomcat 增加一个 Web 工程 Tomcat tomcat = getTomcatInstance(); File appDir = new File(getBuildDirectory(), "webapps/examples"); tomcat.addWebapp(null, "/examples", appDir.getAbsolutePath()); tomcat.start(); ByteChunk res = getUrl("http://localhost:" + getPort() + "/examples/servlets/servlet/HelloWorldExample"); assertTrue(res.toString().indexOf("<h1>Hello World!</h1>") > 0); 清单 1 的代码是创建一个 Tomcat 实例并新增一个 Web 应用，然后启动 Tomcat 并调用其中的一个 HelloWorldExample Servlet，看有没有正确返回预期的数据。 Tomcat 的 addWebapp 方法的代码如下： 清单 3 .Tomcat.addWebapp public Context addWebapp(Host host, String url, String path) { silence(url); Context ctx = new StandardContext(); ctx.setPath( url ); ctx.setDocBase(path); if (defaultRealm == null) { initSimpleAuth(); } ctx.setRealm(defaultRealm); ctx.addLifecycleListener(new DefaultWebXmlListener()); ContextConfig ctxCfg = new ContextConfig(); ctx.addLifecycleListener(ctxCfg); ctxCfg.setDefaultWebXml("org/apache/catalin/startup/NO_DEFAULT_XML"); if (host == null) { getHost().addChild(ctx); } else { host.addChild(ctx); } return ctx; } 前面已经介绍了一个 Web 应用对应一个 Context 容器，也就是 Servlet 运行时的 Servlet 容器，添加一个 Web 应用时将会创建一个 StandardContext 容器，并且给这个 Context 容器设置必要的参数，url 和 path 分别代表这个应用在 Tomcat 中的访问路径和这个应用实际的物理路径，这个两个参数与清单 1 中的两个参数是一致的。其中最重要的一个配置是 ContextConfig，这个类将会负责整个 Web 应用配置的解析工作，后面将会详细介绍。最后将这个 Context 容器加到父容器 Host 中。 接下去将会调用 Tomcat 的 start 方法启动 Tomcat，如果你清楚 Tomcat 的系统架构，你会容易理解 Tomcat 的启动逻辑，Tomcat 的启动逻辑是基于观察者模式设计的，所有的容器都会继承 Lifecycle 接口，它管理者容器的整个生命周期，所有容器的的修改和状态的改变都会由它去通知已经注册的观察者（Listener），关于这个设计模式可以参考《 Tomcat 的系统架构与设计模式，第二部分：设计模式》。Tomcat 启动的时序图可以用图 2 表示。 图 2. Tomcat 主要类的启动时序图（）上图描述了 Tomcat 启动过程中，主要类之间的时序关系，下面我们将会重点关注添加 examples 应用所对应的 StandardContext 容器的启动过程。 当 Context 容器初始化状态设为 init 时，添加在 Contex 容器的 Listener 将会被调用。ContextConfig 继承了 LifecycleListener 接口，它是在调用清单 3 时被加入到 StandardContext 容器中。ContextConfig 类会负责整个 Web 应用的配置文件的解析工作。 ContextConfig 的 init 方法将会主要完成以下工作： 创建用于解析 xml 配置文件的 contextDigester 对象 读取默认 context.xml 配置文件，如果存在解析它 读取默认 Host 配置文件，如果存在解析它 读取默认 Context 自身的配置文件，如果存在解析它 设置 Context 的 DocBase ContextConfig 的 init 方法完成后，Context 容器的会执行 startInternal 方法，这个方法启动逻辑比较复杂，主要包括如下几个部分： 创建读取资源文件的对象 创建 ClassLoader 对象 设置应用的工作目录 启动相关的辅助类如：logger、realm、resources 等 修改启动状态，通知感兴趣的观察者（Web 应用的配置） 子容器的初始化 获取 ServletContext 并设置必要的参数 初始化“load on startup”的 Servlet Web 应用的初始化工作 Web 应用的初始化工作是在 ContextConfig 的 configureStart 方法中实现的，应用的初始化主要是要解析 web.xml 文件，这个文件描述了一个 Web 应用的关键信息，也是一个 Web 应用的入口。 Tomcat 首先会找 globalWebXml 这个文件的搜索路径是在 engine 的工作目录下寻找以下两个文件中的任一个 org/apache/catalin/startup/NO_DEFAULT_XML 或 conf/web.xml。接着会找 hostWebXml 这个文件可能会在 System.getProperty("catalina.base")/conf/${EngineName}/${HostName}/web.xml.default，接着寻找应用的配置文件 examples/WEB-INF/web.xml。web.xml 文件中的各个配置项将会被解析成相应的属性保存在 WebXml 对象中。如果当前应用支持 Servlet3.0，解析还将完成额外 9 项工作，这个额外的 9 项工作主要是为 Servlet3.0 新增的特性，包括 jar 包中的 META-INF/web-fragment.xml 的解析以及对 annotations 的支持。 接下去将会将 WebXml 对象中的属性设置到 Context 容器中，这里包括创建 Servlet 对象、filter、listener 等等。这段代码在 WebXml 的 configureContext 方法中。下面是解析 Servlet 的代码片段： 清单 4. 创建 Wrapper 实例 for (ServletDef servlet : servlets.values()) { Wrapper wrapper = context.createWrapper(); String jspFile = servlet.getJspFile(); if (jspFile != null) { wrapper.setJspFile(jspFile); } if (servlet.getLoadOnStartup() != null) { wrapper.setLoadOnStartup(servlet.getLoadOnStartup().intValue()); } if (servlet.getEnabled() != null) { wrapper.setEnabled(servlet.getEnabled().booleanValue()); } wrapper.setName(servlet.getServletName()); Map<String,String> params = servlet.getParameterMap(); for (Entry<String, String> entry : params.entrySet()) { wrapper.addInitParameter(entry.getKey(), entry.getValue()); } wrapper.setRunAs(servlet.getRunAs()); Set<SecurityRoleRef> roleRefs = servlet.getSecurityRoleRefs(); for (SecurityRoleRef roleRef : roleRefs) { wrapper.addSecurityReference( roleRef.getName(), roleRef.getLink()); } wrapper.setServletClass(servlet.getServletClass()); MultipartDef multipartdef = servlet.getMultipartDef(); if (multipartdef != null) { if (multipartdef.getMaxFileSize() != null && multipartdef.getMaxRequestSize()!= null && multipartdef.getFileSizeThreshold() != null) { wrapper.setMultipartConfigElement(new MultipartConfigElement( multipartdef.getLocation(), Long.parseLong(multipartdef.getMaxFileSize()), Long.parseLong(multipartdef.getMaxRequestSize()), Integer.parseInt( multipartdef.getFileSizeThreshold()))); } else { wrapper.setMultipartConfigElement(new MultipartConfigElement( multipartdef.getLocation())); } } if (servlet.getAsyncSupported() != null) { wrapper.setAsyncSupported( servlet.getAsyncSupported().booleanValue()); } context.addChild(wrapper); } 这段代码清楚的描述了如何将 Servlet 包装成 Context 容器中的 StandardWrapper，这里有个疑问，为什么要将 Servlet 包装成 StandardWrapper 而不直接是 Servlet 对象。这里 StandardWrapper 是 Tomcat 容器中的一部分，它具有容器的特征，而 Servlet 为了一个独立的 web 开发标准，不应该强耦合在 Tomcat 中。 除了将 Servlet 包装成 StandardWrapper 并作为子容器添加到 Context 中，其它的所有 web.xml 属性都被解析到 Context 中，所以说 Context 容器才是真正运行 Servlet 的 Servlet 容器。一个 Web 应用对应一个 Context 容器，容器的配置属性由应用的 web.xml 指定，这样我们就能理解 web.xml 到底起到什么作用了。 回页首 创建 Servlet 实例 前面已经完成了 Servlet 的解析工作，并且被包装成 StandardWrapper 添加在 Context 容器中，但是它仍然不能为我们工作，它还没有被实例化。下面我们将介绍 Servlet 对象是如何创建的，以及如何被初始化的。 创建 Servlet 对象 如果 Servlet 的 load-on-startup 配置项大于 0，那么在 Context 容器启动的时候就会被实例化，前面提到在解析配置文件时会读取默认的 globalWebXml，在 conf 下的 web.xml 文件中定义了一些默认的配置项，其定义了两个 Servlet，分别是：org.apache.catalina.servlets.DefaultServlet 和 org.apache.jasper.servlet.JspServlet 它们的 load-on-startup 分别是 1 和 3，也就是当 Tomcat 启动时这两个 Servlet 就会被启动。 创建 Servlet 实例的方法是从 Wrapper. loadServlet 开始的。loadServlet 方法要完成的就是获取 servletClass 然后把它交给 InstanceManager 去创建一个基于 servletClass.class 的对象。如果这个 Servlet 配置了 jsp-file，那么这个 servletClass 就是 conf/web.xml 中定义的 org.apache.jasper.servlet.JspServlet 了。 创建 Servlet 对象的相关类结构图如下： 图 3. 创建 Servlet 对象的相关类结构初始化 Servlet 初始化 Servlet 在 StandardWrapper 的 initServlet 方法中，这个方法很简单就是调用 Servlet 的 init 的方法，同时把包装了 StandardWrapper 对象的 StandardWrapperFacade 作为 ServletConfig 传给 Servlet。Tomcat 容器为何要传 StandardWrapperFacade 给 Servlet 对象将在后面做详细解析。 如果该 Servlet 关联的是一个 jsp 文件，那么前面初始化的就是 JspServlet，接下去会模拟一次简单请求，请求调用这个 jsp 文件，以便编译这个 jsp 文件为 class，并初始化这个 class。 这样 Servlet 对象就初始化完成了，事实上 Servlet 从被 web.xml 中解析到完成初始化，这个过程非常复杂，中间有很多过程，包括各种容器状态的转化引起的监听事件的触发、各种访问权限的控制和一些不可预料的错误发生的判断行为等等。我们这里只抓了一些关键环节进行阐述，试图让大家有个总体脉络。 下面是这个过程的一个完整的时序图，其中也省略了一些细节。 图 4. 初始化 Servlet 的时序图（） 回页首 Servlet 体系结构 我们知道 Java Web 应用是基于 Servlet 规范运转的，那么 Servlet 本身又是如何运转的呢？为何要设计这样的体系结构。 图 5.Servlet 顶层类关联图从上图可以看出 Servlet 规范就是基于这几个类运转的，与 Servlet 主动关联的是三个类，分别是 ServletConfig、ServletRequest 和 ServletResponse。这三个类都是通过容器传递给 Servlet 的，其中 ServletConfig 是在 Servlet 初始化时就传给 Servlet 了，而后两个是在请求达到时调用 Servlet 时传递过来的。我们很清楚 ServletRequest 和 ServletResponse 在 Servlet 运行的意义，但是 ServletConfig 和 ServletContext 对 Servlet 有何价值？仔细查看 ServletConfig 接口中声明的方法发现，这些方法都是为了获取这个 Servlet 的一些配置属性，而这些配置属性可能在 Servlet 运行时被用到。而 ServletContext 又是干什么的呢？ Servlet 的运行模式是一个典型的“握手型的交互式”运行模式。所谓“握手型的交互式”就是两个模块为了交换数据通常都会准备一个交易场景，这个场景一直跟随个这个交易过程直到这个交易完成为止。这个交易场景的初始化是根据这次交易对象指定的参数来定制的，这些指定参数通常就会是一个配置类。所以对号入座，交易场景就由 ServletContext 来描述，而定制的参数集合就由 ServletConfig 来描述。而 ServletRequest 和 ServletResponse 就是要交互的具体对象了，它们通常都是作为运输工具来传递交互结果。 ServletConfig 是在 Servlet init 时由容器传过来的，那么 ServletConfig 到底是个什么对象呢？ 下图是 ServletConfig 和 ServletContext 在 Tomcat 容器中的类关系图。 图 6. ServletConfig 在容器中的类关联图上图可以看出 StandardWrapper 和 StandardWrapperFacade 都实现了 ServletConfig 接口，而 StandardWrapperFacade 是 StandardWrapper 门面类。所以传给 Servlet 的是 StandardWrapperFacade 对象，这个类能够保证从 StandardWrapper 中拿到 ServletConfig 所规定的数据，而又不把 ServletConfig 不关心的数据暴露给 Servlet。 同样 ServletContext 也与 ServletConfig 有类似的结构，Servlet 中能拿到的 ServletContext 的实际对象也是 ApplicationContextFacade 对象。ApplicationContextFacade 同样保证 ServletContex 只能从容器中拿到它该拿的数据，它们都起到对数据的封装作用，它们使用的都是门面设计模式。 通过 ServletContext 可以拿到 Context 容器中一些必要信息，比如应用的工作路径，容器支持的 Servlet 最小版本等。 Servlet 中定义的两个 ServletRequest 和 ServletResponse 它们实际的对象又是什么呢？，我们在创建自己的 Servlet 类时通常使用的都是 HttpServletRequest 和 HttpServletResponse，它们继承了 ServletRequest 和 ServletResponse。为何 Context 容器传过来的 ServletRequest、ServletResponse 可以被转化为 HttpServletRequest 和 HttpServletResponse 呢？ 图 7.Request 相关类结构图上图是 Tomcat 创建的 Request 和 Response 的类结构图。Tomcat 一接受到请求首先将会创建 org.apache.coyote.Request 和 org.apache.coyote.Response，这两个类是 Tomcat 内部使用的描述一次请求和相应的信息类它们是一个轻量级的类，它们作用就是在服务器接收到请求后，经过简单解析将这个请求快速的分配给后续线程去处理，所以它们的对象很小，很容易被 JVM 回收。接下去当交给一个用户线程去处理这个请求时又创建 org.apache.catalina.connector. Request 和 org.apache.catalina.connector. Response 对象。这两个对象一直穿越整个 Servlet 容器直到要传给 Servlet，传给 Servlet 的是 Request 和 Response 的门面类 RequestFacade 和 RequestFacade，这里使用门面模式与前面一样都是基于同样的目的——封装容器中的数据。一次请求对应的 Request 和 Response 的类转化如下图所示： 图 8.Request 和 Response 的转变过程 回页首 Servlet 如何工作 我们已经清楚了 Servlet 是如何被加载的、Servlet 是如何被初始化的，以及 Servlet 的体系结构，现在的问题就是它是如何被调用的。 当用户从浏览器向服务器发起一个请求，通常会包含如下信息：http://hostname: port /contextpath/servletpath，hostname 和 port 是用来与服务器建立 TCP 连接，而后面的 URL 才是用来选择服务器中那个子容器服务用户的请求。那服务器是如何根据这个 URL 来达到正确的 Servlet 容器中的呢？ Tomcat7.0 中这件事很容易解决，因为这种映射工作有专门一个类来完成的，这个就是 org.apache.tomcat.util.http.mapper，这个类保存了 Tomcat 的 Container 容器中的所有子容器的信息，当 org.apache.catalina.connector. Request 类在进入 Container 容器之前，mapper 将会根据这次请求的 hostnane 和 contextpath 将 host 和 context 容器设置到 Request 的 mappingData 属性中。所以当 Request 进入 Container 容器之前，它要访问那个子容器这时就已经确定了。 图 9.Request 的 Mapper 类关系图可能你有疑问，mapper 中怎么会有容器的完整关系，这要回到图 2 中 19 步 MapperListener 类的初始化过程，下面是 MapperListener 的 init 方法代码 : 清单 5. MapperListener.init public void init() { findDefaultHost(); Engine engine = (Engine) connector.getService().getContainer(); engine.addContainerListener(this); Container[] conHosts = engine.findChildren(); for (Container conHost : conHosts) { Host host = (Host) conHost; if (!LifecycleState.NEW.equals(host.getState())) { host.addLifecycleListener(this); registerHost(host); } } } 这段代码的作用就是将 MapperListener 类作为一个监听者加到整个 Container 容器中的每个子容器中，这样只要任何一个容器发生变化，MapperListener 都将会被通知，相应的保存容器关系的 MapperListener 的 mapper 属性也会修改。for 循环中就是将 host 及下面的子容器注册到 mapper 中。 图 10.Request 在容器中的路由图上图描述了一次 Request 请求是如何达到最终的 Wrapper 容器的，我们现正知道了请求是如何达到正确的 Wrapper 容器，但是请求到达最终的 Servlet 还要完成一些步骤，必须要执行 Filter 链，以及要通知你在 web.xml 中定义的 listener。 接下去就要执行 Servlet 的 service 方法了，通常情况下，我们自己定义的 servlet 并不是直接去实现 javax.servlet.servlet 接口，而是去继承更简单的 HttpServlet 类或者 GenericServlet 类，我们可以有选择的覆盖相应方法去实现我们要完成的工作。 Servlet 的确已经能够帮我们完成所有的工作了，但是现在的 web 应用很少有直接将交互全部页面都用 servlet 来实现，而是采用更加高效的 MVC 框架来实现。这些 MVC 框架基本的原理都是将所有的请求都映射到一个 Servlet，然后去实现 service 方法，这个方法也就是 MVC 框架的入口。 当 Servlet 从 Servlet 容器中移除时，也就表明该 Servlet 的生命周期结束了，这时 Servlet 的 destroy 方法将被调用，做一些扫尾工作。 回页首 Session 与 Cookie 前面我们已经说明了 Servlet 如何被调用，我们基于 Servlet 来构建应用程序，那么我们能从 Servlet 获得哪些数据信息呢？ Servlet 能够给我们提供两部分数据，一个是在 Servlet 初始化时调用 init 方法时设置的 ServletConfig，这个类基本上含有了 Servlet 本身和 Servlet 所运行的 Servlet 容器中的基本信息。根据前面的介绍 ServletConfig 的实际对象是 StandardWrapperFacade，到底能获得哪些容器信息可以看看这类提供了哪些接口。还有一部分数据是由 ServletRequest 类提供，它的实际对象是 RequestFacade，从提供的方法中发现主要是描述这次请求的 HTTP 协议的信息。所以要掌握 Servlet 的工作方式必须要很清楚 HTTP 协议，如果你还不清楚赶紧去找一些参考资料。关于这一块还有一个让很多人迷惑的 Session 与 Cookie。 Session 与 Cookie 不管是对 Java Web 的熟练使用者还是初学者来说都是一个令人头疼的东西。Session 与 Cookie 的作用都是为了保持访问用户与后端服务器的交互状态。它们有各自的优点也有各自的缺陷。然而具有讽刺意味的是它们优点和它们的使用场景又是矛盾的，例如使用 Cookie 来传递信息时，随着 Cookie 个数的增多和访问量的增加，它占用的网络带宽也很大，试想假如 Cookie 占用 200 个字节，如果一天的 PV 有几亿的时候，它要占用多少带宽。所以大访问量的时候希望用 Session，但是 Session 的致命弱点是不容易在多台服务器之间共享，所以这也限制了 Session 的使用。 不管 Session 和 Cookie 有什么不足，我们还是要用它们。下面详细讲一下，Session 如何基于 Cookie 来工作。实际上有三种方式能可以让 Session 正常工作： 基于 URL Path Parameter，默认就支持 基于 Cookie，如果你没有修改 Context 容器个 cookies 标识的话，默认也是支持的 基于 SSL，默认不支持，只有 connector.getAttribute("SSLEnabled") 为 TRUE 时才支持 第一种情况下，当浏览器不支持 Cookie 功能时，浏览器会将用户的 SessionCookieName 重写到用户请求的 URL 参数中，它的传递格式如 /path/Servlet;name=value;name2=value2? Name3=value3，其中“Servlet；”后面的 K-V 对就是要传递的 Path Parameters，服务器会从这个 Path Parameters 中拿到用户配置的 SessionCookieName。关于这个 SessionCookieName，如果你在 web.xml 中配置 session-config 配置项的话，其 cookie-config 下的 name 属性就是这个 SessionCookieName 值，如果你没有配置 session-config 配置项，默认的 SessionCookieName 就是大家熟悉的“JSESSIONID”。接着 Request 根据这个 SessionCookieName 到 Parameters 拿到 Session ID 并设置到 request.setRequestedSessionId 中。 请注意如果客户端也支持 Cookie 的话，Tomcat 仍然会解析 Cookie 中的 Session ID，并会覆盖 URL 中的 Session ID。 如果是第三种情况的话将会根据 javax.servlet.request.ssl_session 属性值设置 Session ID。 有了 Session ID 服务器端就可以创建 HttpSession 对象了，第一次触发是通过 request. getSession() 方法，如果当前的 Session ID 还没有对应的 HttpSession 对象那么就创建一个新的，并将这个对象加到 org.apache.catalina. Manager 的 sessions 容器中保存，Manager 类将管理所有 Session 的生命周期，Session 过期将被回收，服务器关闭，Session 将被序列化到磁盘等。只要这个 HttpSession 对象存在，用户就可以根据 Session ID 来获取到这个对象，也就达到了状态的保持。 图 11.Session 相关类图上从图中可以看出从 request.getSession 中获取的 HttpSession 对象实际上是 StandardSession 对象的门面对象，这与前面的 Request 和 Servlet 是一样的原理。下图是 Session 工作的时序图： 图 12.Session 工作的时序图（）还有一点与 Session 关联的 Cookie 与其它 Cookie 没有什么不同，这个配置的配置可以通过 web.xml 中的 session-config 配置项来指定。 回页首 Servlet 中的 Listener 整个 Tomcat 服务器中 Listener 使用的非常广泛，它是基于观察者模式设计的，Listener 的设计对开发 Servlet 应用程序提供了一种快捷的手段，能够方便的从另一个纵向维度控制程序和数据。目前 Servlet 中提供了 5 种两类事件的观察者接口，它们分别是：4 个 EventListeners 类型的，ServletContextAttributeListener、ServletRequestAttributeListener、ServletRequestListener、HttpSessionAttributeListener 和 2 个 LifecycleListeners 类型的，ServletContextListener、HttpSessionListener。如下图所示： 图 13.Servlet 中的 Listener（）它们基本上涵盖了整个 Servlet 生命周期中，你感兴趣的每种事件。这些 Listener 的实现类可以配置在 web.xml 中的 <listener> 标签中。当然也可以在应用程序中动态添加 Listener，需要注意的是 ServletContextListener 在容器启动之后就不能再添加新的，因为它所监听的事件已经不会再出现。掌握这些 Listener 的使用，能够让我们的程序设计的更加灵活。 回页首 总结 本文涉及到内容有点多，要把每个细节都说清楚，似乎不可能，本文试着从 Servlet 容器的启动到 Servlet 的初始化，以及 Servlet 的体系结构等这些环节中找出一些重点来讲述，目的是能读者有一个总体的完整的结构图，同时也详细分析了其中的一些难点问题，希望对大家有所帮助。 参考资料 学习 查看文章 《 Tomcat 系统架构与设计模式》（developerWorks，2010 年 5 月）：了解 Tomcat 中容器的体系结构，基本的工作原理，以及 Tomcat 中使用的经典的设计模式介绍。 Java Servlet技术简介（developerWorks，2004 年 12 月）：介绍并解释 servlet 是什么，它们是如何工作的，如何使用它们来创建您能够想像到的任意复杂度的 Web 应用程序，以及作为一名专业编程人员，您如何才能最有效地使用 servlet。 参考 Apache Tomcat官网，了解 Tomcat 最新动态，以及开发人员参考手册。 Servlet 最新规范，本文是基于 Servlet3.0 规范讲解的，这里有最新的 Servlet 规范，以及 API 的介绍。 HTTP 协议，W3C 关于 HTTP 协议的详细描述。 developerWorks Web development 专区：通过专门关于 Web 技术的文章和教程，扩展您在网站开发方面的技能。 developerWorks Ajax 资源中心：这是有关 Ajax 编程模型信息的一站式中心，包括很多文档、教程、论坛、blog、wiki 和新闻。任何 Ajax 的新信息都能在这里找到。 developerWorks Web 2.0 资源中心，这是有关 Web 2.0 相关信息的一站式中心，包括大量 Web 2.0 技术文章、教程、下载和相关技术资源。您还可以通过 Web 2.0 新手入门 栏目，迅速了解 Web 2.0 的相关概念。 查看 HTML5 专题，了解更多和 HTML5 相关的知识和动向。 讨论 加入 developerWorks 中文社区。 关于作者 许令波，developerWorks 中国网站最佳作者，现就职于淘宝网，是一名 Java 开发工程师。对大型互联网架构设计颇感兴趣，喜欢钻研开源框架的设计原理。有时间将学到的知识整理成文章，也喜欢记录下工作和生活中的一些思考。个人网站是： HYPERLINK "http://xulingbo.net" http://xulingbo.net。
-
发表了文章 2012-04-07
Hot to use typelist of Loki??
最近调试的一段代码,请一起look look! #include<iostream> #include<stdio.h> #include<string.h> using namespace std; namespace MCD { template<class T,class U> struct Typelist{ typedef T Head; typedef U Tail; }; class NullType; #define TYPELIST_1(T1) Typelist<T1,NullType> #define TYPELIST_2(T1,T2) Typelist<T1,TYPELIST_1(T2)> #define TYPELIST_3(T1,T2,T3) Typelist<T1,TYPELIST_2(T2,T3)> #define TYPELIST_4(T1,T2,T3,T4) Typelist<T1,TYPELIST_3(T2,T3,T4)> template<class Tlist> struct Length; template<> struct Length<NullType> { enum{value = 0}; }; template<class T,class U> struct Length<Typelist<T,U> > { enum{value = 1 + Length<U>::value}; }; template<class Tlist,unsigned int index> struct TypeAt; template<class Head,class Tail> struct TypeAt<Typelist<Head,Tail>,0> { typedef Head Result; }; template<class Head,class Tail,unsigned int i> struct TypeAt<Typelist<Head,Tail>,i> { typedef typename TypeAt<Tail,i-1>::Result Result; }; } // End of MCD namespace SolidMCP { class FontTable { public: virtual void Read(){ std::cout << "Reading Font Table" << std::endl; } }; class CMAP_Table:public FontTable { public: CMAP_Table() { std::cout << "Initializing CMAP Table" << std::endl; } void Read() { std::cout << "Read CMAP Table" << std::endl; } static char* GetName() { return "CMAP_Table"; } }; class OS2_Table:public FontTable { public: OS2_Table() { std::cout << "Initializing OS2 Table" << std::endl; } void Read() { std::cout << "Read OS2 Table" << std::endl; } static char* GetName() { return "OS2_Table"; } }; class HEAD_Table:public FontTable { public: HEAD_Table(){ std::cout<< "Initializing HEAD Table" << std::endl; } void Read(){ std::cout << "Read HEAD Table" << std::endl; } static char* GetName(){ return "HEAD_Table"; } }; class HHEA_Table:public FontTable { public: HHEA_Table(){ std::cout << "Initializing HHEA Table" << std::endl; } void Read() { std::cout << "Read HHEA Table" << std::endl; } static char* GetName() { return "HHEA_Table"; } }; class Unknown_Table:public FontTable { public: Unknown_Table(){ std::cout << "Initializing Unknown Table" << std::endl; } void Read() { std::cout << "Read Unknown Table" << std::endl; } static char* GetName() { return "Unknown_Table"; } }; template<class Tlist> struct ReadTable; template<class T> struct ReadTable<MCD::Typelist<T,MCD::NullType> > { static bool Execute() { T table; table.Read(); return true; } }; template<class T,class U> struct ReadTable<MCD::Typelist<T,U> > { static bool Execute() { if(ReadTable<MCD::Typelist<typename U::Head,typename U::Tail> >::Execute()) { T table; table.Read(); } return true; } }; // ReadTaleLoop template<class T,int i> struct ReadTableLoop { static void Execute() { // g递归ui用的很好 ReadTableLoop<T,i-1>::Execute(); typename MCD::TypeAt<T,i-1>::Result table; table.Read(); } }; template<class T> struct ReadTableLoop<T,0> { static void Execute(){} }; // create table class Name template<class Tlist> struct CreateObject; template<class T,class U> struct CreateObject<MCD::Typelist<T,U> > { static void* Execute(char* pName) { if(strcmp(T::GetName(),pName) == 0) { return new T; }else{ return CreateObject<U>::Execute(pName); } } }; template<> struct CreateObject<MCD::NullType> { static void* Execute(char* pName) { return NULL; } }; }// END OF SoildMCP // How to use using namespace SolidMCP; using namespace MCD; int main() { typedef TYPELIST_4(CMAP_Table,OS2_Table,HEAD_Table,HHEA_Table) FontTableList; std::cout << "--Length is " << Length<FontTableList>::value << std::endl; std::cout << "Read the 2nd Table" << std::endl; TypeAt<FontTableList,2>::Result table; table.Read(); std::cout<<"Begin ReadTable" << std::endl; ReadTable<FontTableList>::Execute(); std::cout << "Begin ReadTableLoop" << std::endl; ReadTableLoop<FontTableList,Length<FontTableList>::value>::Execute(); std::cout << "Begin Create Object By Name" << std::endl; FontTable* pTable = (FontTable *)CreateObject<FontTableList>::Execute("HEAD_Table"); pTable->Read(); return 0; }
-
发表了文章 2012-03-11
Policy-based design
One problem which often arises during programming is how to build a base set of functionality which can be extended by the user, while still being modular enough to make it easy to replace only certain parts of an implementation without having to resort to copy & paste techniques. I guess everybody of us has faced this problem at least once, and came up with different solutions. There is a powerful and elegant technique called policy-based design for solving this kind of problem. Simple example #include<iostream> #include<string> using namespace std; template<typename output_policy,typename language_policy> class HelloWorld:public output_policy,public language_policy { using output_policy::Print; using language_policy::Message; public: void Run() { Print(Message()); } }; class HelloWorld_OutputPolicy_WriteToCout { protected: template<typename message_type> void Print(message_type message) { std::cout << message << std::endl; } }; class HelloWorld_LangguagePolicy_English { protected: std::string Message() { return "Hello World!"; } }; class HelloWorld_LanguagePolicy_German { protected: std::string Message() { return "Hello Policy!"; } }; int main() { typedef HelloWorld<HelloWorld_OutputPolicy_WriteToCout,HelloWorld_LangguagePolicy_English> myPolicyClass; myPolicyClass hello1; hello1.Run(); typedef HelloWorld<HelloWorld_OutputPolicy_WriteToCout,HelloWorld_LanguagePolicy_German> myOtherPolicyClass; myOtherPolicyClass hello2; hello2.Run(); } Makefile all:policy # which compiler CC=g++ # Where are include file kept INCLUDE = . # Where to install INSTDIR = /usr/local/bin # Options for development CFLAGS = -g -Wall -ansi policy:policy.o $(CC) -o policy policy.o policy.o:policy.cpp # $(CC) -I$(INCLUDE) $(CFLAGS) -c msgqueue.c # $(CC) -D_REENTRANT -c msgqueue.c -lpthread $(CC) -c policy.cpp -o policy.o clean: -rm policy.o policy install:policy @if [-d $(INSTDIR) ];\ then \ cp policy $(INSTDIR);\ chmod a+x $(INSTDIR)/policy;\ chmod og-w $(INSTDIR)/policy;\ echo "Install in $(INSTDIR)";\ else \ echo "Sorry,$(INSTDIR) does not exist";\ fi Runing result: Hello World Hello Policy
-
发表了文章 2011-12-18
深入理解C++中的mutable关键字
mutalbe的中文意思是“可变的，易变的”，跟constant（既C++中的const）是反义词。 在C++中，mutable也是为了突破const的限制而设置的。被mutable修饰的变量，将永远处于可变的状态，即使在一个const函数中。 我们知道，如果类的成员函数不会改变对象的状态，那么这个成员函数一般会声明成const的。但是，有些时候，我们需要在const的函数里面修改一些跟类状态无关的数据成员，那么这个数据成员就应该被mutalbe来修饰。 下面是一个小例子： class ClxTest{ public: void Output() const;};void ClxTest::Output() const{ cout << "Output for test!" << endl;}void OutputTest(const ClxTest& lx){ lx.Output();} 类ClxTest的成员函数Output是用来输出的，不会修改类的状态，所以被声明为const的。 函数OutputTest也是用来输出的，里面调用了对象lx的Output输出方法，为了防止在函数中调用其他成员函数修改任何成员变量，所以参数也被const修饰。 如果现在，我们要增添一个功能：计算每个对象的输出次数。如果用来计数的变量是普通的变量的话，那么在const成员函数Output里面是不能修改该变量的值的；而该变量跟对象的状态无关，所以应该为了修改该变量而去掉Output的const属性。这个时候，就该我们的mutable出场了——只要用mutalbe来修饰这个变量，所有问题就迎刃而解了。 下面是修改过的代码：class ClxTest{ public: ClxTest(); ~ClxTest(); void Output() const; int GetOutputTimes() const; private: mutable int m_iTimes;};ClxTest::ClxTest(){ m_iTimes = 0;}ClxTest::~ClxTest(){}void ClxTest::Output() const{ cout << "Output for test!" << endl; m_iTimes++;}int ClxTest::GetOutputTimes() const{ return m_iTimes;}void OutputTest(const ClxTest& lx){ cout << lx.GetOutputTimes() << endl; lx.Output(); cout << lx.GetOutputTimes() << endl;} 计数器m_iTimes被mutable修饰，那么它就可以突破const的限制，在被const修饰的函数里面也能被修改。
-
发表了文章 2011-12-18
Think before you code, Virtual Functions in C++
Introduction A few days back, I was doing a job, and unintentionally, I made a mistake in the code (What mistake? That I will explain in the detailed section of the article), and when I was caught by a bug and started de-bugging it, I was amazed how a little mistake can give a programmer a whole lot of pain. Yes, I made a mistake in the virtual function area. How? Let's find out........ Using the code So, why do we need a virtual function? Everyone knows that. Let's say I have a base class and a few derived class as well; and all the derived classes shares a common function, and in the driver program, I do not want to make a big huge switch/if block. I want to iterate through all the derived types and want to execute the common member function. Like this: Collapse | Copy Code #include <iostream> #include <vector> using std::cout; using std::endl; using std::vector; class CommunicationDevices { //Base class has some property, for this article I dont need those public: inline virtual void which(){ cout<<"This is a common device..."<<endl; } }; class MobilePhoneWithGSMSupport:public CommunicationDevices { //Derived class also has some extended property, GSM related public: inline virtual void which(){ cout<<"This is a Mobile Phone...GSM Supported"<<endl; } }; class MobilePhoneWithCDMASupport:public CommunicationDevices { //Derived class also has some extended property, CDMA related public: inline void which(){ cout<<"This is a Mobile Phone....CDMA Supported"<<endl; } }; class Landline:public CommunicationDevices { //Derived class also has some extended property public: inline void which(){ cout<<"This is a Landline Phone..."<<endl; } }; class Iphone:public MobilePhoneWithGSMSupport { //More specific IPhone Feature here public: inline void which(){ cout<<"This is apple Iphone with AT&T connection, GSM Support only..." <<endl; } }; void whichPhoneUserIsUsing(CommunicationDevices &devices){ devices.which(); } int main(){ MobilePhoneWithGSMSupport user1; MobilePhoneWithCDMASupport user2; Landline user3; Iphone user4; whichPhoneUserIsUsing(user1); whichPhoneUserIsUsing(user2); whichPhoneUserIsUsing(user3); whichPhoneUserIsUsing(user4); return 0; } Here, the idea is simple. Since we are using a virtual function in the base class, the “whichPhoneUserIsUsing()” method can take a generic base class argument, and the proper method from the derived class gets accessed depending upon the actual type of the object. This is the beauty of virtual functions. Note that in the method “whichPhoneUserIsUsing()”, we used a reference to the base class as the argument: “CommunicationDevices &devices”, and from the driver (main()), we are passing the derived class' object while calling this function. This is normally called as Upcasting in C++. That is, we are going from the more specific type to the more generic type. And, this casting is type-safe always. As you expected, this code will produce the following o/p: Collapse | Copy Code bash-3.2$ g++ -g -o hello code1.cpp bash-3.2$ ./hello This is a Mobile Phone...GSM Supported This is a Mobile Phone....CDMA Supported This is a Landline Phone... This is apple Iphone with AT&T connection, GSM Support only... Now, consider the following code, only a single character (believe me, just a single character) has been changed here from the previous code: We just modified our method whichPhoneUserIsUsing() like this: Collapse | Copy Code #include <iostream> #include <vector> using std::cout; using std::endl; using std::vector; class CommunicationDevices { //Base class has some property, for this article I dont need those public: inline virtual void which(){ cout<<"This is a common device..."<<endl; } }; class MobilePhoneWithGSMSupport:public CommunicationDevices { //Derived class also has some extended property, GSM related public: inline virtual void which(){ cout<<"This is a Mobile Phone...GSM Supported"<<endl; } }; class MobilePhoneWithCDMASupport:public CommunicationDevices { //Derived class also has some extended property, CDMA related public: inline void which(){ cout<<"This is a Mobile Phone....CDMA Supported"<<endl; } }; class Landline:public CommunicationDevices { //Derived class also has some extended property public: inline void which(){ cout<<"This is a Landline Phone..."<<endl; } }; class Iphone:public MobilePhoneWithGSMSupport { //More specific IPhone Feature here public: inline void which(){ cout<<"This is apple Iphone with AT&T connection, GSM Support only..." <<endl; } }; void whichPhoneUserIsUsing(CommunicationDevices devices){ devices.which(); } int main(){ MobilePhoneWithGSMSupport user1; MobilePhoneWithCDMASupport user2; Landline user3; Iphone user4; whichPhoneUserIsUsing(user1); whichPhoneUserIsUsing(user2); whichPhoneUserIsUsing(user3); whichPhoneUserIsUsing(user4); return 0; } We just modified our method whichPhoneUserIsUsing() like this: Collapse | Copy Code void whichPhoneUserIsUsing(CommunicationDevices devices){ devices.which(); } and bang.................given below is the output: Collapse | Copy Code bash-3.2$ g++ -g -o hello code2.cpp bash-3.2$ ./hello This is a common device... This is a common device... This is a common device... This is a common device... bash-3.2$ vim code2.cpp So, what gets wrong here? Yes, you guessed it correctly, it's a famous copy-constructor problem. When the arguments are just a “CommunicationDevices” instead of a reference to it, the function says: Hey Mr. Programmer, I am bound to create only a temporary object for this function (whichPhoneUserIsUsing()). I am no more responsible to take a reference, so I don't care what kind of actual object you are passing through; I will create a concrete “CommunicationDevices” object, and will copy only those segments from the actual object which are meaningful to me (i.e., which are part of the base class). And, will only invoke the “which” method for this temporary object. And hence, every time you call me, I will call the base class version (i.e., CommunicationDevices version) of the which() method. This famous property is called Object Bisection, or Object Slicing. Cutting down the desired property from one object and copying it to a concrete base class object.
-
发表了文章 2011-11-13
linux操作系统消息队列
所谓消息队列就是指一个消息链表。int msgget(key_t, int flag)：创建和打开队列int msgsnd(int msqid, struct msgbuf *msgp, size_t msgsz, int flag):发送消息，msgid是消息队列的id，msgp是消息内容所在的缓冲区，msgsz是消息的大小，msgflg是标志。int msgrcv(int msqid, struct msgbuf *msgp, size_t msgsz, long msgtyp, int flag):接受消息，msgtyp是期望接收的消息类型。 msgqueue.c文件内容如下； #include<sys/types.h> #include<sys/ipc.h> #include<stdio.h> #include<stdlib.h> #include<unistd.h> #include<string.h> #include<pthread.h> // 增加线程支持 #define BUFSZ 512 struct message{ long msg_type; char msg_text[BUFSZ]; }; #define MSG_SIZE sizeof(struct message) char app_exit = 0; void *thread_funtion(void *parg); int main() { int qid; key_t key; int len; int res; pthread_t a_thread; struct message msg; if((key = ftok(".",'a')) == -1){ // ftok 获得一个key perror("ftok"); exit(1); } if((qid = msgget(key,IPC_CREAT|0666)) == -1){ // 创建一个消息队列 perror("msgget"); exit(1); } printf("opened queue %d\n",qid); puts("Please enter the message to queue:"); if((fgets(msg.msg_text,BUFSZ,stdin)) == NULL){ // 从标准输入获得buffer puts("no message"); exit(1); } msg.msg_type = getpid(); len = strlen(msg.msg_text) + sizeof(msg.msg_type); if((msgsnd(qid,&msg,len,0)) < 0){ // 发送消息 perror("message posted"); exit(1); } /* memset(&msg,0,sizeof(msg)); // 清除内存为0 if(msgrcv(qid,&msg,len,0) < 0){ // 接收消息 perror("message recv"); exit(1); } printf("message is:%s\n",(&msg)->msg_text); if((msgctl(qid,IPC_RMID,NULL))<0){ perror("msgctl"); exit(1); } */ res = pthread_create(&a_thread,NULL,thread_funtion,(void *)&qid); printf("The msgrcv thread is create sucess!\n"); while((app_exit = getchar()) != 'e'){sleep(50);} printf("exit main funtion!\n"); exit(0); } void *thread_funtion(void *parg) { struct message msg; int qid; qid = *((int *)parg); memset(&msg,0,MSG_SIZE); while(app_exit != 'e'){ if(msgrcv(qid,&msg,MSG_SIZE) < 0){ sleep(50); continue; } printf("message is:%s\n",(&msg)->msg_text); if(msgctl(qid,IPC_RMID,NULL) < 0){ perror("msgctl"); } sleep(50); } } Makefile文件类型如下； all:msgqueue # which compiler CC = gcc # Where are include file kept INCLUDE = . # Where to install INSTDIR = /usr/local/bin # Options for development CFLAGS = -g -Wall -ansi msgqueue:msgqueue.o $(CC) -D_REENTRANT -o msgqueue msgqueue.o -lpthread msgqueue.o:msgqueue.c # $(CC) -I$(INCLUDE) $(CFLAGS) -c msgqueue.c # $(CC) -D_REENTRANT -c msgqueue.c -lpthread $(CC) -c msgqueue.c clean: -rm msgqueue.o msgqueue install:msgqueue @if [-d $(INSTDIR) ];\ then \ cp msgqueue $(INSTDIR);\ chmod a+x $(INSTDIR)/msgqueue;\ chmod og-w $(INSTDIR)/msgqueue;\ echo "Install in $(INSTDIR)";\ else \ echo "Sorry,$(INSTDIR) does not exist";\ fi
-
发表了文章 2011-10-24
Comparison of Performance of Different poll implementations
epoll Scalability Web Page Introduction Interface Description Man Pages Testing dphttpd dphttpd SMP results dphttpd UP results pipetest pipetest results Recent comparison results Analysis and Conclusions Acknowledgements Introduction Davide Libenzi wrote an event poll implementation and described it at the /dev/epoll home page here. His performance testing led to the conclusion that epoll scaled linearly regardless of the load as defined by the number of dead connections. However, the main hindrance to having epoll accepted into the mainline Linux kernel by Linus Torvalds was the interface to epoll being in /dev. Therefore, a new interface to epoll was added via three new system calls. That interface will hereafter be referred to as sys_epoll. Download sys_epoll here. sys_epoll Interface int epoll_create(int maxfds);The system call epoll_create() creates a sys_epoll "object" by allocating space for "maxfds" descriptors to be polled. The sys_epoll "object" is referenced by a file descriptor, and this enables the new interface to : Maintain compatibility with the existing interface Avoid the creation of a epoll_close() syscall Reuse 95% of the existing code Inherit the file* automatic clean up code int epoll_ctl(int epfd, int op, int fd, unsigned int events);The system call epoll_ctl() is the controller interface. The "op" parameter is either EP_CTL_ADD, EP_CTL_DEL or EP_CTL_MOD. The parameter "fd" is the target of the operation. The last parameter, "events", is used in both EP_CTL_ADD and EP_CTL_MOD and represents the event interest mask. int epoll_wait(int epfd, struct pollfd * events, int maxevents, int timeout); The system call epoll_wait() waits for events by allowing a maximum timeout, "timeout", in milliseconds and returns the number of events ( struct pollfd ) that the caller will find available at "*events". sys_epoll Man Pages Postscript : epoll.ps epoll_create.ps epoll_ctl.ps epoll_wait.ps ASCI Text : epoll.txt epoll_create.txt epoll_ctl.txt epoll_wait.txt Testing We tested using two applications: dphttpd pipetest dphttpd Software The http client is httperf from David Mosberger. Download httperf here. The http server is dphttpd from Davide Libenzi. Download dphttpd here. The deadconn client is also provided by Davide Libenzi. Download deadconn here. Two client programs (deadconn_last and httperf) run on the client machine and establish connections to the HTTP server (dphttpd) running on the server machine. Connections established by deadconn_last are "dead". These send a single HTTP get request at connection setup and remain idle for the remainder of the test. Connections established by httperf are "active". These continuously send HTTP requests to the server at a fixed rate. httperf reports the rate at which the HTTP server replies to its requests. This reply rate is the metric reported on the Y-axis of the graphs below. For the tests, the number of active connections is kept constant and the number of dead connections increased from 0 to 60000 (X-axis of graphs below). Consequently, dphttpd spends a fixed amount of time responding to requests and a variable amount of time looking for requests to service. The mechanism used to look for active connections amongst all open connections is one of standard poll(), /dev/epoll or sys_epoll. As the number of dead connections is increased, the scalability of these mechanisms starts impacting dphttpd's reply rate, measured by httperf. dphttpd SMP Server Hardware: 8-way PIII Xeon 700MHz, 2.5 GB RAM, 2048 KB L2 cache OS : RedHat 7.3 with 2.5.44 kernel, patched with ONE of: sys_epoll-2.5.44-0.9.diff To reproduce download here.To run the latest sys_epoll patch download here. ep_patch-2.5.44-0.32.diff Download /dev/epoll unpatched kernel Download 2.5.44. /proc/sys/fs/file-max = 131072 /proc/sys/net/ipv4/tcp_fin_timeout = 15 /proc/sys/net/ipv4/tcp_max_syn_backlog = 16384 /proc/sys/net/ipv4/tcp_tw_reuse = 1 /proc/sys/net/ipv4/tcp_tw_recycle = 1 /proc/sys/net/ipv4/ip_local_port_range = 1024 65535 # ulimit -n 131072 # dphttpd --maxfds 20000 --stksize 4096 default size of reply = 128 bytes Client Hardware: 4-way PIII Xeon 500MHz, 3 GB RAM, 512 KB L2 cache OS : RedHat 7.3 with 2.4.18-3smp kernel /proc/sys/fs/file-max = 131072 /proc/sys/net/ipv4/tcp_fin_timeout = 15 /proc/sys/net/ipv4.tcp_tw_recycle = 1 /proc/sys/net/ipv4.tcp_max_syn_backlog = 16384 /proc/sys/net/ipv4.ip_local_port_range = 1024 65535 # ulimit -n 131072 # deadconn_last $SERVER $SVRPORT num_connections where num_connections is one of 0, 50, 100, 200, 400, 800, 1000, 5000, 10000, 15000, 20000, 25000, 30000, 35000, 40000, 45000, 50000, 55000, 60000. After deadconn_last reports num_connections established # httperf --server=$SERVER --port=$SVRPORT --think-timeout 5 --timeout 5 --num-calls 20000 --num-conns 100 --hog --rate 100 Results for dphttpd SMP dphttpd UP Server 1-way PIII, 866MHz, 256 MB RAM OS: 2.5.44 gcc 2.96 (RedHat 7.3) /proc/sys/fs/file-max = 65536 /proc/sys/net/ipv4/tcp_fin_timeout = 15 /proc/sys/net/ipv4/tcp_tw_recycle = 1 # ulimit -n 65536 # dphttpd --maxfds 20000 --stksize 4096 default size of reply = 128 bytes Client 1-way PIII, 866MHz, 256 MB RAM OS: 2.4.18, gcc 2.96 (RedHat 7.3) /proc/sys/fs/file-max = 65536 /proc/sys/net/ipv4/tcp_fin_timeout = 15 /proc/sys/net/ipv4.tcp_tw_recycle = 1 # ulimit -n 65536 # deadconn_last $SERVER $SVRPORT num_connections where num_connections is one of 0, 50, 100, 200, 400, 800, 1000, 2000, 4000, 6000, 8000, 10000, 12000, 14000, 16000. After deadconn_last reports num_connections established # httperf --server=$SERVER --port=$SVRPORT --think-timeout 5 --timeout 5 --num-calls 20000 --num-conns 100 --hog --rate 100 Results for dphttpd UP Pipetest David Stevens added support for sys_epoll to Ben LaHaise's original pipetest.c application. Download Ben LaHaise's Ottawa Linux Symposium 2002 paper including pipetest.c here. Download David Steven's patch to add sys_epoll to pipetest.c here. Pipetest SMP System and Configuration 8-way PIII Xeon 900MHz, 4 GB RAM, 2048 KB L2 cache OS: RedHat 7.2 with 2.5.44 kernel, patched with one of: sys_epoll-2.5.44-0.9.diff To reproduce download here.To run the latest sys_epoll patch download here. async poll ported to 2.5.44 download here. /proc/sys/fs/file-max = 65536 # ulimit -n 65536 # pipetest [poll|aio-poll|sys-epoll] num_pipes message_threads max_generation where num_pipes is one of 10, 100, 500, or 1000-16000 in increments of 1000 message_threads is 1 max_generantion is 300 Results for pipetest on an SMP Results for Pipetest on a UP Same Hardware and Configuration as with the SMP pipetest above with CONFIG_SMP = n being the only change. sys_epoll stability comparisons Oct 30, 2002 Following are performance results comparing version 0.14 of the (Download v0.14 here) to the version, v0.9, originally used for the performance testing outlined above. (Download v0.9 here) Testing was done using two measures: pipetest details here. and dphttpd details here.. Analysis and Conclusion The system call interface to epoll performs as well as the /dev interface to epoll. In addition sys_epoll scales better than poll and AIO poll for large numbers of connections. Other points in favour of sys_epoll are: sys_epoll is compatible with synchronous read() and write() and thus makes it usable with existing libraries that have not migrated to AIO. Applications using poll() can be easily migrated to sys_epoll while continuing to use the existing I/O infrastructure. sys_epoll will be invisible to people who don't want to use it. sys_epoll has a low impact on the existing source code. arch/i386/kernel/entry.S | 4 fs/file_table.c | 4 fs/pipe.c | 36 + include/asm-i386/poll.h | 1 include/asm-i386/unistd.h | 3 include/linux/fs.h | 4 include/linux/list.h | 5 include/linux/pipe_fs_i.h | 4 include/linux/sys.h | 2 include/net/sock.h | 12 net/ipv4/tcp.c | 4 Due to these factors sys_epoll should be seriously considered for inclusion in the mainline Linux kernel. Acknowledgments Thank You to: Davide Libenzi Who wrote /dev/epoll, sys_epoll and dphttpd.He was an all around great guy to work with.Also Thanks to the following people who helped with testing and this web site: Shailabh Nagar, Paul Larson , Hanna Linder, and David Stevens.
-
发表了文章 2011-10-22
Nginx源码分析-Epoll模块
Linux平台上，Nginx使用epoll完成事件驱动，实现高并发；本文将不对epoll本身进行介绍（网上一堆一堆的文章介绍epoll的原理及使用方法，甚至源码分析等），仅看一下Nginx是如何使用epoll的。 Nginx在epoll模块中定义了好几个函数，这些函数基本都是作为回调注册到事件抽象层的对应接口上，从而实现了事件驱动的具体化，我们看如下的一段代码： ngx_event_module_t ngx_epoll_module_ctx = { &epoll_name, ngx_epoll_create_conf, /* create configuration */ ngx_epoll_init_conf, /* init configuration */ { ngx_epoll_add_event, /* add an event */ ngx_epoll_del_event, /* delete an event */ ngx_epoll_add_event, /* enable an event */ ngx_epoll_del_event, /* disable an event */ ngx_epoll_add_connection, /* add an connection */ ngx_epoll_del_connection, /* delete an connection */ NULL, /* process the changes */ ngx_epoll_process_events, /* process the events */ ngx_epoll_init, /* init the events */ ngx_epoll_done, /* done the events */ } }; 这段代码就是epoll的相关函数注册到事件抽象层，这里所谓的事件抽象层在前面的博文中有提过，就是Nginx为了方便支持和开发具体的I/O模型，从而实现的一层抽象。代码后面的注释将功能说明得很详细了，本文就只重点关注ngx_epoll_init和ngx_epoll_process_events两个函数，其他几个函数就暂且忽略了。 ngx_epoll_init主要是完成epoll的相关初始化工作，代码分析如下： static ngx_int_t ngx_epoll_init(ngx_cycle_t *cycle, ngx_msec_t timer) { ngx_epoll_conf_t *epcf; /*取得epoll模块的配置结构*/ epcf = ngx_event_get_conf(cycle->conf_ctx, ngx_epoll_module); /*ep是epoll模块定义的一个全局变量，初始化为-1*/ if (ep == -1) { /*创一个epoll对象，容量为总连接数的一半*/ ep = epoll_create(cycle->connection_n / 2); if (ep == -1) { ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno, "epoll_create() failed"); return NGX_ERROR; } } /*nevents也是epoll模块定义的一个全局变量，初始化为0*/ if (nevents events) { if (event_list) { ngx_free(event_list); } /*event_list存储产生事件的数组*/ event_list = ngx_alloc(sizeof(struct epoll_event) * epcf->events, cycle->log); if (event_list == NULL) { return NGX_ERROR; } } nevents = epcf->events; /*初始化全局变量ngx_io, ngx_os_is定义为： ngx_os_io_t ngx_os_io = { ngx_unix_recv, ngx_readv_chain, ngx_udp_unix_recv, ngx_unix_send, ngx_writev_chain, 0 };（位于src/os/unix/ngx_posix_init.c） */ ngx_io = ngx_os_io; /*这里就是将epoll的具体接口函数注册到事件抽象层接口ngx_event_actions上。 具体是上文提到的ngx_epoll_module_ctx中封装的如下几个函数 ngx_epoll_add_event, ngx_epoll_del_event, ngx_epoll_add_event, ngx_epoll_del_event, ngx_epoll_add_connection, ngx_epoll_del_connection, ngx_epoll_process_events, ngx_epoll_init, ngx_epoll_done, */ ngx_event_actions = ngx_epoll_module_ctx.actions; #if (NGX_HAVE_CLEAR_EVENT) /*epoll将添加这个标志,主要为了实现边缘触发*/ ngx_event_flags = NGX_USE_CLEAR_EVENT #else /*水平触发*/ ngx_event_flags = NGX_USE_LEVEL_EVENT #endif |NGX_USE_GREEDY_EVENT /*io的时候，直到EAGAIN为止*/ |NGX_USE_EPOLL_EVENT; /*epoll标志*/ return NGX_OK; } epoll初始化工作没有想象中的复杂，和我们平时使用epoll都一样，下面看ngx_epoll_process_events，这个函数主要用来完成事件的等待并处理。 static ngx_int_t ngx_epoll_process_events(ngx_cycle_t *cycle, ngx_msec_t timer, ngx_uint_t flags) { int events; uint32_t revents; ngx_int_t instance, i; ngx_uint_t level; ngx_err_t err; ngx_log_t *log; ngx_event_t *rev, *wev, **queue; ngx_connection_t *c; /*一开始就是等待事件，最长等待时间为timer；nginx为事件 专门用红黑树维护了一个计时器。后续对这个timer单独分析。 */ events = epoll_wait(ep, event_list, (int) nevents, timer); if (events == -1) { err = ngx_errno; } else { err = 0; } if (flags & NGX_UPDATE_TIME || ngx_event_timer_alarm) { /*执行一次时间更新, nginx将时间缓存到了一组全局变量中，方便程序高效的获取事件。*/ ngx_time_update(); } /*处理wait错误*/ if (err) { if (err == NGX_EINTR) { if (ngx_event_timer_alarm) { ngx_event_timer_alarm = 0; return NGX_OK; } level = NGX_LOG_INFO; } else { level = NGX_LOG_ALERT; } ngx_log_error(level, cycle->log, err, "epoll_wait() failed"); return NGX_ERROR; } /*wait返回事件数0，可能是timeout返回，也可能是非timeout返回；非timeout返回则是error*/ if (events == 0) { if (timer != NGX_TIMER_INFINITE) { return NGX_OK; } ngx_log_error(NGX_LOG_ALERT, cycle->log, 0, "epoll_wait() returned no events without timeout"); return NGX_ERROR; } log = cycle->log; /*for循环开始处理收到的所有事件*/ for (i = 0; i read; 。。。。。。。。。。。。。 /*取得发生一个事件*/ revents = event_list[i].events; /*记录wait的错误返回状态*/ if (revents & (EPOLLERR|EPOLLHUP)) { ngx_log_debug2(NGX_LOG_DEBUG_EVENT, log, 0, "epoll_wait() error on fd:%d ev:%04XD", c->fd, revents); } if ((revents & (EPOLLERR|EPOLLHUP)) && (revents & (EPOLLIN|EPOLLOUT)) == 0) { /* * if the error events were returned without EPOLLIN or EPOLLOUT, * then add these flags to handle the events at least in one * active handler */ revents |= EPOLLIN|EPOLLOUT; } /*该事件是一个读事件，并该连接上注册的读事件是active的*/ if ((revents & EPOLLIN) && rev->active) { if ((flags & NGX_POST_THREAD_EVENTS) && !rev->accept) { rev->posted_ready = 1; } else { rev->ready = 1; } /*事件放入相应的队列中；关于此处的先入队再处理，在前面的文章中已经介绍过了。*/ if (flags & NGX_POST_EVENTS) { queue = (ngx_event_t **) (rev->accept ? &ngx_posted_accept_events : &ngx_posted_events); ngx_locked_post_event(rev, queue); /*入队*/ } else { rev->handler(rev); } } wev = c->write; /*发生的是一个写事件，和读事件完全一样的逻辑过程*/ if ((revents & EPOLLOUT) && wev->active) { if (flags & NGX_POST_THREAD_EVENTS) { wev->posted_ready = 1; } else { wev->ready = 1; } /*先入队再处理*/ if (flags & NGX_POST_EVENTS) { ngx_locked_post_event(wev, &ngx_posted_events); } else { wev->handler(wev); } } } return NGX_OK; } 本文将关注的两个epoll函数也就这么一点代码了，但整个epoll还有添加事件和删除事件等的相关函数，代码都很简单，本文就不做具体的分析了。
-
发表了文章 2011-07-31
Windows CE系统开发,BSP包中的几个常见文件详解
Windows CE系统开发,BSP包中的几个常见文件详解 BSP介绍（Board Support Package）是介于底层硬件和上层软件之间的底层软件开发包，它主要功能为屏蔽硬件，提供操作系统及硬件驱动，具体功能包括： (1) 单板硬件初始化，主要是CPU的初始化，为整个软件系统提供底层硬件支持； (2) 为操作系统提供设备驱动程序和系统中断服务程序； (3) 定制操作系统的功能，为软件系统提供一个实时多任务的运行环境； (4) 初始化操作系统，为操作系统的正常运行做好准备。 1.1.1 sources.cmn文件 sources.cmn是Common Source文件，BSP内所有的源代码编译都会用到该文件。里面定义一些比较常用的名词或环境变量或路径。 例如在OAMPL138 MCBSP驱动目录下source 文件中“INCLUDES=$(INCLUDES);..\INC”这行伪代码，其中“(INCLUDES)”部分及时引用source.cmn文件中的INCLUDE环境变量，定义如程序清单 0.1所示。 程序清单 0.1 定义 件平台相关的文件，主要 INCLUDE环境变量 INCLUDES=$(_PLATFORMROOT)\common\src\inc; 大家可以看看这段英文介绍； Sources.cmn is a build configuration system file that allows you to set common variables. This can be useful if more than one directory in the build tree need the a variable set to the same value because it can reduce your maintenance efforts. Each build tree can use one sources.cmn file. When build.exe runs, it will determine the root of the build tree by looking for the top most folder with a Dirs file. Build.exe then sets BUILDROOT to the top most folder with a Dirs files. Makefile.def in Public\Common\OAK\Misc will include $(BUILDROOT)\sources.cmn if it exists. Some of the variables that are commonly set in sources.cmn include CDEFINES, ADEFINES and INCLUDES. CDEFINES and ADEFINES are used to set macros that are common to multiple build folders. These might include RAM and ROM sizes, but might also include OEM and CPU specific macros. The include path, INCLUDES, can be set in sources.cmn which is very helpful, especially if and when you change the directory structure. The Platform directory tree should have WINCEOEM set to 1, so sources.cmn is a good place to do this rather than setting it in each sources file. Note: Starting with Windows CE 5.0, sources.cmn is no longer really an option. It is required because sources.cmn needs to at least set _COMMONPUBROOT, _ISVINCPATH, and _OEMINCPATH. These were set by build.exe in prior versions. 1.1.2 XXX.bat文件 DAT文件用于在WinCE启动的时候，定义文件系统的结构，也就是定义有哪些文件夹，哪些文件在什么位置等。每次冷启动的时候，Filesys模块会根据.dat文件中的内容来创建目录以及目录下的文件。 在BSP中我们可以找到platform.dat。在创建一个WinCE的工程以后，可以在工程目录下面找到project.dat。可以在这两个.dat文件中定义我们所需的根目录以及相应的子目录和文件，当然Windows文件夹及其包含的子文件夹除外。用户可以通过定义快捷方式的方法来引用Windows目录下的文件，如程序清单 0.2所示。 程序清单 0.2 DAT文件应用示例 root:-Directory("My Documents") root:-Directory("Program Files") Directory("\Program Files"):-Directory("My Projects") Directory("\Program Files"):-Directory("Accessories") Directory("\Program Files"):-Directory("Communication") Directory("\Program Files\My Projects"):-File("My Project Program", "\Windows\Myproj.exe") root:-File("\control.lnk", "\Windows\control.lnk") 简单解释一下，上面的内容是先创建My Documents和Program Files两个根目录，然后在Program Files下面创建My Projects，Accessories和Communication三个子目录，然后拷贝被定义在Windows目录下的存在于ROM中的Myproj.exe文件到My Projects目录下面，且名字为My Project Program。这里需要说明的一点就是，像Myproj.exe这样的文件都是从ROM中拷贝出来的，所以必须在BIB文件中包含了该文件。最后一行意思是在根路径下创建control.lnk，该文件来自ROM中的control.lnk文件，是一个快捷方式文件。对上面的一些格式做个解释； 1． Root Directory的语法格式 root:[-Directory("dir_name")] [-Permdir("dir_name")] [-File("target_filename", "source_location")] root：表示根目录。 -Directory(“dir_name”)：定义根目录下的目录名。 -Permdir(“dir_name”)：定义一个永久的目录，用户是不能通过RemoveDirectory函数删除的。 -File(" target_filename", " source_location")：定义一个目标文件，该文件从ROM中拷贝过来。target_filename为目标文件的文件名，source_location为ROM中的文件，指Windows目录下的某路径下的文件名。 2． Directory的语法格式： Directory("dir_name"):[-Directory("dir_name")] [-File("target_filename", "source_location")] Directory(“dir_name”)：表示目录名。”\”表示根目录。 -Directory(“dir_name”)：表示目录下的路径，就是子目录。 -File(" target_filename", " source_location")：定义一个目标文件，该文件从ROM中拷贝过来。target_filename为目标文件的文件名，source_location为ROM中的文件，指Windows目录下的某路径下的文件名。 DAT中所使用的语法定义比较简单，看看例子就知道如何修改了，一般我们会通过修改project.dat和platform.dat来改变WinCE启动后的文件路径结构。其中platform.dat是和平台相关的，而project.dat是和WinCE工程相关的。 1.1.3 Config.bib文件 Config.bib文件是BSP中一个重要的内存，他用于指定CPU内存分配，先来看看OAMPL138该部分内容是怎么安排，如程序清单 0.3所示。 程序清单 0.3 CONFIG.BIB文件MEMORY部分 MEMORY NK 80000000 02000000 RAMIMAGE RAM 82000000 0173F800 RAM DISPLAY 8373F800 00600000 RESERVED EMACBUF 83D3F800 000C0000 RESERVED ARGS 83DFF800 00000800 RESERVED #if defined BSP_DSPLINK DSPLINK 83E00000 00200000 RESERVED EXTENSIONRAM 84000000 04000000 RESERVED #else EXTENSIONRAM 83E00000 04200000 RESERVED #endif 简要说明下，在CONFIG.BIB中指定的MEMORY都指的是虚拟内存，如果知道其对应的物理地址，可以参照OEMADDRTAB.INC代码。 在上述的代码中 NK 指定的是Kernel在加载到RAM中存放的地址区域，0x80000000表示的是Kernel的内存起始地址，0x02000000表示的是Kernel的大小，从这里可以看出，内核最大不能超过32MB。 另外一个需要注意的是，在上述代码中ARGS非常重要，它是建立BSP和OS共享参数的区域，一般在BSP中设置的参数都保存在ARGS区域中。如果有需要，在OS启动之后，通过KernelIoControl获取到这段区域，读取其中参数之后进行相关的系统设置。例如在Bootloader启动的过程中，改变了显示屏的参数，则Bootloader将其参数保存在ARGS当中，OS启动之后，LCD驱动获取ARGS区域，解析参数，调整LCD显示屏的设置，这样LCD才能正常显示。 l 小技巧 在 BootLoader 编译的时候，有时候发现nb0文件很难生成，其实就是config.bib 文件未做相关的设置。看看OAMPL138是怎么做，如程序清单 0.4所示，其中的设置参照程序清单 0.3，不做解释。 程序清单 0.4 NB0生成技巧 ROMSTART=80000000 ROMWIDTH=32 ROMSIZE=2000000 另外有时候为了提高RAM的利用率，需要将“NK”区域用于加载Kernel剩余部分的内存释放出来给RAM用，达到扩大程序运行内存的目的。看起来不可思议，其实很简单，只需要在Config.bib文件中加入“AutoSize = ON”即可。 1.1.4 Platform.bib文件 该文件包含了和硬件平台相关的文件，以驱动模块为主。该文件还定义这些模块和文件以什么方式被加载到内存中。具体格式如程序清单 0.5所示； 程序清单 0.5 Platform.bib格式 Name Path Memory Type Name：模块的名字，比如一个dll或者exe文件的文件名。 Path：路径，一般都是WinCE的工程的Release目录。 Memory：指定该模块被放在哪个区域，一般都是NK区域。 Type：定义了文件的类型，具体如下表 0‑1所示； 表 0‑1 TYPE类型 类型 描述 S 系统文件。 H 隐藏文件。 R 只压缩模块的资源部分。 C 压缩模块的所有部分。 D 禁止调试。 N 模块是不可信任的。 P 告诉Romimage.exe不需要检查CPU的类型。 K 告诉Romimage.exe必需固定该模块的内核地址。有该标记的模块只能被LoadKernelLibrary函数加载。 X 告诉Romimage.exe对该模块签名。 M 运行时加载整个模块，不要按需分页。 L 告诉Romimage.exe不要分离ROM DLL。 一般FILES项的Type只支持S，H，N，D几个类型，而MODULES项的Type是都支持的。 举个例子吧： INIT.EXE %_WINCEROOT%RELEASEINIT.EXE NK SH MYDLL.DLL %_WINCEROOT%RELEASEMYDLL.DLL NK SHK 对于BIB文件来说同样支持“条件编译”，我们可以通过设置环境变量来选择性地将某些模块打包到WinCE image中。一般在BSP中，对于一些驱动模块的环境变量我们IF来进行条件判断。而对于WinCE的系统模块来说，一般都是SYSGEN变量，应该使用@CESYSGEN IF来判断。 l 小技巧 通常修改Platform.bib或则Platform.reg文件，都需要系统重新“sysgen”一次，“Sysgen”一次需要耗费比较长的时间，在工作中实在无法忍受。 如此，我们一般采用这种方法来处理。在Platfrom.bib文件同级目录下，建议一个以平台名称命名的bib文件，如OAMPL138.bib，然后在将该文件INCLUDE到Platfrom文件中（如程序清单 0.6所示），需要修改的时候只修改OAMPL138.bib文件，之后Make Run-Time Image即可生效。 程序清单 0.6 INCLUDE #include "$(_TARGETPLATROOT)\FILES\OAMPL138.bib"
-
发表了文章 2011-07-18
WinCE6.0中应用程序如何直接访问物理空间
在实际开发过程中，经常希望能在应用程序中直接读写设备的物理空间。以前在做WinCE6.0下的MEMMgr时通过秘密加载一个内核态驱动实现了这个需求。但这种方式有一个明显的缺陷，每次读写都必须经由它才能完成。如果只是读取GPIO，那问题不算大。如果想通过这种方式实现视频播放的加速就比较困难了。估计非但不能加速，反而会变得更慢。 早先曾与ZL仔细的讨论过这个问题，他当时在WinCE6.0上移植TCPMP，发现播放视频不太流畅，于是想通过直接写显存进行加速。目的很明确，在应用中申请一段虚拟空间，通过某种方法将其映射到显存上，视频解码过程中直接往映射过的虚拟空间上写。这种方法与使用GAPI有一点类似。 实现这个需求，需要用到函数VirtualCopyEx()。看看帮助中关于它的说明，This function dynamically maps a virtual address to a physical address by creating a new page-table entry.This function is callable in kernel mode and in user mode, when the source and destination process handles are the active process.This function is similar to VirtualCopy, except VirtualCopyEx requires handles to the source and destination process. 据此基本可以确定，我们的确可以在应用中申请一段虚拟空间，然后通过这个函数将其映射到某段物理空间上。其中目标进程是我们的应用，而源进程是NK.exe。为了实现在NK.exe中执行VirtualCopyEx()，可以加载一个内核态的驱动。更为方便的方法是移植一个OALIOCTL，并在IOControl()中添加一个case。这样，应用程序在做内存映射时就无需打开某个流驱动，直接调用KernelIoControl()即可。 OALIOCTL中添加的关键代码如下。 1 typedef struct { 2 void* pvDestMem; 3 DWORD dwPhysAddr; 4 DWORD dwSize; 5 } VIRTUAL_COPY_EX_DATA; 6 7 #define IOCTL_VIRTUAL_COPY_EX CTL_CODE (FILE_DEVICE_UNKNOWN,3333,METHOD_BUFFERED,FILE_ANY_ACCESS) 8 9 10 case IOCTL_VIRTUAL_COPY_EX: 11 { 12 VIRTUAL_COPY_EX_DATA *p = (VIRTUAL_COPY_EX_DATA*)pInBuf; 13 HANDLE hDst = (HANDLE)GetDirectCallerProcessId(); 14 HANDLE hSrc = (HANDLE)GetCurrentProcessId(); 15 fRet = VirtualCopyEx(hDst,p->pvDestMem,hSrc,(LPVOID)p->dwPhysAddr,p->dwSize, 16 PAGE_READWRITE|PAGE_PHYSICAL|PAGE_NOCACHE); 17 }break; 应用程序中进行内存映射的关键代码如下。 1 volatile LPVOID GetVirtual(DWORD dwPhyBaseAddress, DWORD dwSize) 2 { 3 volatile LPVOID pVirtual; 4 VIRTUAL_COPY_EX_DATA vced; 5 6 if(dwPhyBaseAddress&0xFFF) 7 { 8 return NULL; 9 } 10 vced.dwPhysAddr = dwPhyBaseAddress>>8; 11 pVirtual = VirtualAlloc(0,dwSize,MEM_RESERVE,PAGE_NOACCESS); 12 vced.pvDestMem = pVirtual; 13 vced.dwSize = dwSize; 14 KernelIoControl(IOCTL_VIRTUAL_COPY_EX,&vced, sizeof(vced), NULL, NULL, NULL); 15 return pVirtual; 16 } 17 18 // WinCE6.0模拟器中应用程序直接写屏 19 PBYTE pLCDBuf = (PBYTE)GetVirtual(0x33f00000,0x100000); 20 memset(pLCDBuf,0,0x100000); 21 这种方法在WinCE6.0的模拟器中测试了一下，能达到预期的效果。
-
发表了文章 2011-07-17
Smart Pointers - What, Why, Which?
Smart Pointers - What, Why, Which? What are they?Smart pointers are objects that look and feel like pointers, but are smarter. What does this mean? To look and feel like pointers, smart pointers need to have the same interface that pointers do: they need to support pointer operations like dereferencing (operator *) and indirection (operator ->). An object that looks and feels like something else is called a proxy object, or just proxy. The proxy pattern and its many uses are described in the books Design Patterns and Pattern Oriented Software Architecture. To be smarter than regular pointers, smart pointers need to do things that regular pointers don't. What could these things be? Probably the most common bugs in C++ (and C) are related to pointers and memory management: dangling pointers, memory leaks, allocation failures and other joys. Having a smart pointer take care of these things can save a lot of aspirin... The simplest example of a smart pointer is auto_ptr, which is included in the standard C++ library. You can find it in the header <memory>, or take a look at Scott Meyers' auto_ptr implementation. Here is part of auto_ptr's implementation, to illustrate what it does: template <class T> class auto_ptr { T* ptr; public: explicit auto_ptr(T* p = 0) : ptr(p) {} ~auto_ptr() {delete ptr;} T& operator*() {return *ptr;} T* operator->() {return ptr;} // ... };As you can see, auto_ptr is a simple wrapper around a regular pointer. It forwards all meaningful operations to this pointer (dereferencing and indirection). Its smartness in the destructor: the destructor takes care of deleting the pointer. For the user of auto_ptr, this means that instead of writing: void foo() { MyClass* p(new MyClass); p->DoSomething(); delete p; }You can write: void foo() { auto_ptr<MyClass> p(new MyClass); p->DoSomething(); }And trust p to cleanup after itself. What does this buy you? See the next section. Why would I use them?Obviously, different smart pointers offer different reasons for use. Here are some common reasons for using smart pointers in C++. Why: Less bugs Automatic cleanup. As the code above illustrates, using smart pointers that clean after themselves can save a few lines of code. The importance here is not so much in the keystrokes saved, but in reducing the probability for bugs: you don't need to remember to free the pointer, and so there is no chance you will forget about it. Automatic initialization. Another nice thing is that you don't need to initialize the auto_ptr to NULL, since the default constructor does that for you. This is one less thing for the programmer to forget. Dangling pointers. A common pitfall of regular pointers is the dangling pointer: a pointer that points to an object that is already deleted. The following code illustrates this situation: MyClass* p(new MyClass); MyClass* q = p; delete p; p->DoSomething(); // Watch out! p is now dangling! p = NULL; // p is no longer dangling q->DoSomething(); // Ouch! q is still dangling!For auto_ptr, this is solved by setting its pointer to NULL when it is copied: template <class T> auto_ptr<T>& auto_ptr<T>::operator=(auto_ptr<T>& rhs) { if (this != &rhs) { delete ptr; ptr = rhs.ptr; rhs.ptr = NULL; } return *this; }Other smart pointers may do other things when they are copied. Here are some possible strategies for handling the statement q = p, where p and q are smart pointers: Create a new copy of the object pointed by p, and have q point to this copy. This strategy is implemented in copied_ptr.h. Ownership transfer: Let both p and q point to the same object, but transfer the responsibility for cleaning up ("ownership") from p to q. This strategy is implemented in owned_ptr.h. Reference counting: Maintain a count of the smart pointers that point to the same object, and delete the object when this count becomes zero. So the statement q = p causes the count of the object pointed by p to increase by one. This strategy is implemented in counted_ptr.h. Scott Meyers offers another reference counting implementation in his book More Effective C++. Reference linking: The same as reference counting, only instead of a count, maintain a circular doubly linked list of all smart pointers that point to the same object. This strategy is implemented in linked_ptr.h. Copy on write: Use reference counting or linking as long as the pointed object is not modified. When it is about to be modified, copy it and modify the copy. This strategy is implemented in cow_ptr.h. All these techniques help in the battle against dangling pointers. Each has each own benefits and liabilities. The Which section of this article discusses the suitability of different smart pointers for various situations. Why: Exception SafetyLet's take another look at this simple example: void foo() { MyClass* p(new MyClass); p->DoSomething(); delete p; }What happens if DoSomething() throws an exception? All the lines after it will not get executed and p will never get deleted! If we're lucky, this leads only to memory leaks. However, MyClass may free some other resources in its destructor (file handles, threads, transactions, COM references, mutexes) and so not calling it my cause severe resource locks. If we use a smart pointer, however, p will be cleaned up whenever it gets out of scope, whether it was during the normal path of execution or during the stack unwinding caused by throwing an exception. But isn't it possible to write exception safe code with regular pointers? Sure, but it is so painful that I doubt anyone actually does this when there is an alternative. Here is what you would do in this simple case: void foo() { MyClass* p; try { p = new MyClass; p->DoSomething(); delete p; } catch (...) { delete p; throw; } }Now imagine what would happen if we had some if's and for's in there... Why: Garbage collectionSince C++ does not provide automatic garbage collection like some other languages, smart pointers can be used for that purpose. The simplest garbage collection scheme is reference counting or reference linking, but it is quite possible to implement more sophisticated garbage collection schemes with smart pointers. For more information see the garbage collection FAQ. Why: EfficiencySmart pointers can be used to make more efficient use of available memory and to shorten allocation and deallocation time. A common strategy for using memory more efficiently is copy on write (COW). This means that the same object is shared by many COW pointers as long as it is only read and not modified. When some part of the program tries to modify the object ("write"), the COW pointer creates a new copy of the object and modifies this copy instead of the original object. The standard string class is commonly implemented using COW semantics (see the <string> header). string s("Hello"); string t = s; // t and s point to the same buffer of characters t += " there!"; // a new buffer is allocated for t before // appending " there!", so s is unchanged. Optimized allocation schemes are possible when you can make some assumptions about the objects to be allocated or the operating environment. For example, you may know that all the objects will have the same size, or that they will all live in a single thread. Although it is possible to implement optimized allocation schemes using class-specific new and delete operators, smart pointers give you the freedom to choose whether to use the optimized scheme for each object, instead of having the scheme set for all objects of a class. It is therefore possible to match the allocation scheme to different operating environments and applications, without modifying the code for the entire class. Why: STL containersThe C++ standard library includes a set of containers and algorithms known as the standard template library (STL). STL is designed to be generic (can be used with any kind of object) and efficient (does not incur time overhead compared to alternatives). To achieve these two design goals, STL containers store their objects by value. This means that if you have an STL container that stores objects of class Base, it cannot store of objects of classes derived from Base. class Base { /*...*/ }; class Derived : public Base { /*...*/ }; Base b; Derived d; vector<Base> v; v.push_back(b); // OK v.push_back(d); // errorWhat can you do if you need a collection of objects from different classes? The simplest solution is to have a collection of pointers: vector<Base*> v; v.push_back(new Base); // OK v.push_back(new Derived); // OK too // cleanup: for (vector<Base*>::iterator i = v.begin(); i != v.end(); ++i) delete *i;The problem with this solution is that after you're done with the container, you need to manually cleanup the objects stored in it. This is both error prone and not exception safe. Smart pointers are a possible solution, as illustrated below. (An alternative solution is a smart container, like the one implemented in pointainer.h.) vector< linked_ptr<Base> > v; v.push_back(new Base); // OK v.push_back(new Derived); // OK too // cleanup is automaticSince the smart pointer automatically cleans up after itself, there is no need to manually delete the pointed objects. Note: STL containers may copy and delete their elements behind the scenes (for example, when they resize themselves). Therefore, all copies of an element must be equivalent, or the wrong copy may be the one to survive all this copying and deleting. This means that some smart pointers cannot be used within STL containers, specifically the standard auto_ptr and any ownership-transferring pointer. For more info about this issue, see C++ Guru of the Week #25. Which one should I use?Are you confused enough? Well, this summary should help. Which: Local variablesThe standard auto_ptr is the simplest smart pointer, and it is also, well, standard. If there are no special requirements, you should use it. For local variables, it is usually the right choice. Which: Class membersAlthough you can use auto_ptr as a class member (and save yourself the trouble of freeing objects in the destructor), copying one object to another will nullify the pointer, as illustrated Below. class MyClass { auto_ptr<int> p; // ... }; MyClass x; // do some meaningful things with x MyClass y = x; // x.p now has a NULL pointerUsing a copied pointer instead of auto_ptr solves this problem: the copied object (y) gets a new copy of the member. Note that using a reference counted or reference linked pointer means that if y changes the member, this change will also affect x! Therefore, if you want to save memory, you should use a COW pointer and not a simple reference counted/linked pointer. Which: STL containersAs explained above, using garbage-collected pointers with STL containers lets you store objects from different classes in the same container. It is important to consider the characteristics of the specific garbage collection scheme used. Specifically, reference counting/linking can leak in the case of circular references (i.e., when the pointed object itself contains a counted pointer, which points to an object that contains the original counted pointer). Its advantage over other schemes is that it is both simple to implement and deterministic. The deterministic behavior may be important in some real time systems, where you cannot allow the system to suddenly wait while the garbage collector performs its housekeeping duties. Generally speaking, there are two ways to implement reference counting: intrusive and non-intrusive. Intrusive means that the pointed object itself contains the count. Therefore, you cannot use intrusive reference counting with 3-rd party classes that do not already have this feature. You can, however, derive a new class from the 3-rd party class and add the count to it. Non-intrusive reference counting requires an allocation of a count for each counted object. The counted_ptr.h is an example of non-intrusive reference counting. Reference linking does not require any changes to be made to the pointed objects, nor does it require any additional allocations. A reference linked pointer takes a little more space than a reference counted pointer - just enough to store one or two more pointers. Both reference counting and reference linking require using locks if the pointers are used by more than one thread of execution. Which: Explicit ownership transferSometimes, you want to receive a pointer as a function argument, but keep the ownership of this pointer (i.e. the control over its lifetime) to yourself. One way to do this is to use consistent naming-conventions for such cases. Taligent's Guide to Designing Programs recommends using "adopt" to mark that a function adopts ownership of a pointer. Using an owned pointer as the function argument is an explicit statement that the function is taking ownership of the pointer. Which: Big objectsIf you have objects that take a lot of space, you can save some of this space by using COW pointers. This way, an object will be copied only when necessary, and shared otherwise. The sharing is implemented using some garbage collection scheme, like reference counting or linking. Which: Summary For this: Use that: Local variables auto_ptr Class members Copied pointer STL Containers Garbage collected pointer (e.g. reference counting/linking) Explicit ownership transfer Owned pointer Big objects Copy on write ConclusionSmart pointers are useful tools for writing safe and efficient code in C++. Like any tool, they should be used with appropriate care, thought and knowledge. For a comprehensive and in depth analysis of the issues concerning smart pointers, I recommend reading Andrei Alexandrescu's chapter about smart pointers in his book Modern C++ Design. Feel free to use my own smart pointers in your code.The Boost C++ libraries include some smart pointers, which are more rigorously tested and actively maintained. Do try them first, if they are appropriate for your needs.
-
发表了文章 2011-07-17
让你的代码变的更加健壮(Making your C++ code robust)
Making your C++ code robust Introduction 在实际的项目中,当项目的代码量不断增加的时候，你会发现越来越难管理和跟踪其各个组件，如其不善，很容易就引入BUG。因此、我们应该掌握一些能让我们程序更加健壮的方法。 这篇文章提出了一些建议，能有引导我们写出更加强壮的代码，以避免产生灾难性的错误。即使、因为其复杂性和项目团队结构，你的程序目前不遵循任何编码规则，按照下面列出的简单的规则可以帮助您避免大多数的崩溃情况。 Background 先来介绍下作者开发一些软件(CrashRpt),你可以http://code.google.com/p/crashrpt/网站上下载源代码。CrashRpt 顾名思义软件崩溃记录软件（库），它能够自动提交你电脑上安装的软件错误记录。它通过以太网直接将这些错误记录发送给你，这样方便你跟踪软件问题，并及时修改，使得用户感觉到每次发布的软件都有很大的提高，这样他们自然很高兴。 图 1、CrashRpt 库检测到错误弹出的对话框 在分析接收的错误记录的时候,我们发现采用下文介绍的方法能够避免大部分程序崩溃的错误。例如、局部变量未初始化导致数组访问越界，指针使用前未进行检测（NULL）导致访问访问非法区域等。 我已经总结了几条代码设计的方法和规则，在下文一一列出，希望能够帮助你避免犯一些错误，使得你的程序更加健壮。 Initializing Local Variables 使用未初始化的局部变量是引起程序崩溃的一个比较普遍的原因，例如、来看下面这段程序片段： // Define local variables BOOL bExitResult; // This will be TRUE if the function exits successfully FILE* f; // Handle to file TCHAR szBuffer[_MAX_PATH]; // String buffer // Do something with variables above... 上面的这段代码存在着一个潜在的错误，因为没有一个局部变量初始化了。当你的代码运行的时候，这些变量将被默认负一些错误的数值。例如bExitResult 数值将被负为-135913245 ，szBuffer 必须以“\0”结尾,结果不会。因此、局部变量初始化时非常重要的，如下正确代码： // Define local variables // Initialize function exit code with FALSE to indicate failure assumption BOOL bExitResult = FALSE; // This will be TRUE if the function exits successfully // Initialize file handle with NULL FILE* f = NULL; // Handle to file // Initialize string buffer with empty string TCHAR szBuffer[_MAX_PATH] = _T(""); // String buffer // Do something with variables above... 注意：有人说变量初始化会引起程序效率降低，是的，确实如此，如果你确实非常在乎程序的执行效率，去除局部变量初始化，你得想好其后果。 Initializing WinAPI Structures 许多Windows API都接受或则返回一些结构体参数，结构体如果没有正确的初始化，也很有可能引起程序崩溃。大家可能会想起用ZeroMemory宏或者memset()函数去用0填充这个结构体(对结构体对应的元素设置默认值)。但是大部分Windows API 结构体都必须有一个cbSIze参数,这个参数必须设置为这个结构体的大小。 看看下面代码，如何初始化Windows API结构体参数: NOTIFYICONDATA nf; // WinAPI structure memset(&nf,0,sizeof(NOTIFYICONDATA)); // Zero memory nf.cbSize = sizeof(NOTIFYICONDATA); // Set structure size! // Initialize other structure members nf.hWnd = hWndParent; nf.uID = 0; nf.uFlags = NIF_ICON | NIF_TIP; nf.hIcon = ::LoadIcon(NULL, IDI_APPLICATION); _tcscpy_s(nf.szTip, 128, _T("Popup Tip Text")); // Add a tray icon Shell_NotifyIcon(NIM_ADD, &nf); 注意:千万不要用ZeroMemory和memset去初始化那些包括结构体对象的结构体,这样很容易破坏其内部结构体,从而导致程序崩溃. // Declare a C++ structure struct ItemInfo { std::string sItemName; // The structure has std::string object inside int nItemValue; }; // Init the structure ItemInfo item; // Do not use memset()! It can corrupt the structure // memset(&item, 0, sizeof(ItemInfo)); // Instead use the following item.sItemName = "item1"; item.nItemValue = 0; 这里最好是用结构体的构造函数对其成员进行初始化. // Declare a C++ structure struct ItemInfo { // Use structure constructor to set members with default values ItemInfo() { sItemName = _T("unknown"); nItemValue = -1; } std::string sItemName; // The structure has std::string object inside int nItemValue; }; // Init the structure ItemInfo item; // Do not use memset()! It can corrupt the structure // memset(&item, 0, sizeof(ItemInfo)); // Instead use the following item.sItemName = "item1"; item.nItemValue = 0; Validating Function Input 在函数设计的时候,对传入的参数进行检测是一直都推荐的。例如、如果你设计的函数是公共API的一部分,它可能被外部客户端调用，这样很难保证客户端传进入的参数就是正确的。 例如，让我们来看看这个hypotethical DrawVehicle() 函数，它可以根据不同的质量来绘制一辆跑车，这个质量数值（nDrawingQaulity ）是0~100。prcDraw 定义这辆跑车的轮廓区域。 看看下面代码，注意观察我们是如何在使用函数参数之前进行参数检测： BOOL DrawVehicle(HWND hWnd, LPRECT prcDraw, int nDrawingQuality) { // Check that window is valid if(!IsWindow(hWnd)) return FALSE; // Check that drawing rect is valid if(prcDraw==NULL) return FALSE; // Check drawing quality is valid if(nDrawingQuality<0 || nDrawingQuality>100) return FALSE; // Now it's safe to draw the vehicle // ... return TRUE; } Validating Pointers 在指针使用之前，不检测是非常普遍的，这个可以说是我们引起软件崩溃最有可能的原因。如果你用一个指针，这个指针刚好是NULL,那么你的程序在运行时,将报出异常。 CVehicle* pVehicle = GetCurrentVehicle(); // Validate pointer if(pVehicle==NULL) { // Invalid pointer, do not use it! return FALSE; } Initializing Function Output 如果你的函数创建了一个对象，并要将它作为函数的返回参数。那么记得在使用之前把他复制为NULL。如不然，这个函数的调用者将使用这个无效的指针，进而一起程序错误。如下错误代码： int CreateVehicle(CVehicle** ppVehicle) { if(CanCreateVehicle()) { *ppVehicle = new CVehicle(); return 1; } // If CanCreateVehicle() returns FALSE, // the pointer to *ppVehcile would never be set! return 0; } 正确的代码如下； int CreateVehicle(CVehicle** ppVehicle) { // First initialize the output parameter with NULL *ppVehicle = NULL; if(CanCreateVehicle()) { *ppVehicle = new CVehicle(); return 1; } return 0; } Cleaning Up Pointers to Deleted Objects 在内存释放之后,无比将指针复制为NULL。这样可以确保程序的没有那个地方会再使用无效指针。其实就是，访问一个已经被删除的对象地址，将引起程序异常。如下代码展示如何清除一个指针指向的对象： // Create object CVehicle* pVehicle = new CVehicle(); delete pVehicle; // Free pointer pVehicle = NULL; // Set pointer with NULL Cleaning Up Released Handles 在释放一个句柄之前，务必将这个句柄复制伪NULL （0或则其他默认值）。这样能够保证程序其他地方不会重复使用无效句柄。看看如下代码，如何清除一个Windows ＡＰＩ的文件句柄： HANDLE hFile = INVALID_HANDLE_VALUE; // Open file hFile = CreateFile(_T("example.dat"), FILE_READ|FILE_WRITE, FILE_OPEN_EXISTING); if(hFile==INVALID_HANDLE_VALUE) { return FALSE; // Error opening file } // Do something with file // Finally, close the handle if(hFile!=INVALID_HANDLE_VALUE) { CloseHandle(hFile); // Close handle to file hFile = INVALID_HANDLE_VALUE; // Clean up handle } 下面代码展示如何清除File *句柄： // First init file handle pointer with NULL FILE* f = NULL; // Open handle to file errno_t err = _tfopen_s(_T("example.dat"), _T("rb")); if(err!=0 || f==NULL) return FALSE; // Error opening file // Do something with file // When finished, close the handle if(f!=NULL) // Check that handle is valid { fclose(f); f = NULL; // Clean up pointer to handle } Using delete [] Operator for Arrays 如果你分配一个单独的对象，可以直接使用new ，同样你释放单个对象的时候,可以直接使用delete . 然而,申请一个对象数组对象的时候可以使用new,但是释放的时候就不能使用delete ,而必须使用delete[]： // Create an array of objects CVehicle* paVehicles = new CVehicle[10]; delete [] paVehicles; // Free pointer to array paVehicles = NULL; // Set pointer with NULL or // Create a buffer of bytes LPBYTE pBuffer = new BYTE[255]; delete [] pBuffer; // Free pointer to array pBuffer = NULL; // Set pointer with NULL Allocating Memory Carefully 有时候，程序需要动态分配一段缓冲区，这个缓冲区是在程序运行的时候决定的。例如、你需要读取一个文件的内容，那么你就需要申请该文件大小的缓冲区来保存该文件的内容。在申请这段内存之前，请注意，malloc() or new是不能申请0字节的内存，如不然，将导致malloc() or new函数调用失败。传递错误的参数给malloc() 函数将导致C运行时错误。如下代码展示如何动态申请内存： // Determine what buffer to allocate. UINT uBufferSize = GetBufferSize(); LPBYTE* pBuffer = NULL; // Init pointer to buffer // Allocate a buffer only if buffer size > 0 if(uBufferSize>0) pBuffer = new BYTE[uBufferSize]; 为了进一步了解如何正确的分配内存，你可以读下Secure Coding Best Practices for Memory Allocation in C and C++这篇文章。 Using Asserts Carefully Asserts用语调试模式检测先决条件和后置条件。但当我们编译器处于release模式的时候，Asserts在预编阶段被移除。因此，用Asserts是不能够检测我们的程序状态,错误代码如下： #include <assert.h> // This function reads a sports car's model from a file CVehicle* ReadVehicleModelFromFile(LPCTSTR szFileName) { CVehicle* pVehicle = NULL; // Pointer to vehicle object // Check preconditions assert(szFileName!=NULL); // This will be removed by preprocessor in Release mode! assert(_tcslen(szFileName)!=0); // This will be removed in Release mode! // Open the file FILE* f = _tfopen(szFileName, _T("rt")); // Create new CVehicle object pVehicle = new CVehicle(); // Read vehicle model from file // Check postcondition assert(pVehicle->GetWheelCount()==4); // This will be removed in Release mode! // Return pointer to the vehicle object return pVehicle; } 看看上述的代码，Asserts能够在debug模式下检测我们的程序,在release 模式下却不能。所以我们还是不得不用if()来这步检测操作。正确的代码如下； CVehicle* ReadVehicleModelFromFile(LPCTSTR szFileName, ) { CVehicle* pVehicle = NULL; // Pointer to vehicle object // Check preconditions assert(szFileName!=NULL); // This will be removed by preprocessor in Release mode! assert(_tcslen(szFileName)!=0); // This will be removed in Release mode! if(szFileName==NULL || _tcslen(szFileName)==0) return NULL; // Invalid input parameter // Open the file FILE* f = _tfopen(szFileName, _T("rt")); // Create new CVehicle object pVehicle = new CVehicle(); // Read vehicle model from file // Check postcondition assert(pVehicle->GetWheelCount()==4); // This will be removed in Release mode! if(pVehicle->GetWheelCount()!=4) { // Oops... an invalid wheel count was encountered! delete pVehicle; pVehicle = NULL; } // Return pointer to the vehicle object return pVehicle; } Checking Return Code of a Function 断定一个函数执行一定成功是一种常见的错误。当你调用一个函数的时候，建议检查下返回代码和返回参数的值。如下代码持续调用Windows API ,程序是否继续执行下去依赖于该函数的返回结果和返回参数值。 HRESULT hres = E_FAIL; IWbemServices *pSvc = NULL; IWbemLocator *pLoc = NULL; hres = CoInitializeSecurity( NULL, -1, // COM authentication NULL, // Authentication services NULL, // Reserved RPC_C_AUTHN_LEVEL_DEFAULT, // Default authentication RPC_C_IMP_LEVEL_IMPERSONATE, // Default Impersonation NULL, // Authentication info EOAC_NONE, // Additional capabilities NULL // Reserved ); if (FAILED(hres)) { // Failed to initialize security if(hres!=RPC_E_TOO_LATE) return FALSE; } hres = CoCreateInstance( CLSID_WbemLocator, 0, CLSCTX_INPROC_SERVER, IID_IWbemLocator, (LPVOID *) &pLoc); if (FAILED(hres) || !pLoc) { // Failed to create IWbemLocator object. return FALSE; } hres = pLoc->ConnectServer( _bstr_t(L"ROOT\\CIMV2"), // Object path of WMI namespace NULL, // User name. NULL = current user NULL, // User password. NULL = current 0, // Locale. NULL indicates current NULL, // Security flags. 0, // Authority (e.g. Kerberos) 0, // Context object &pSvc // pointer to IWbemServices proxy ); if (FAILED(hres) || !pSvc) { // Couldn't conect server if(pLoc) pLoc->Release(); return FALSE; } hres = CoSetProxyBlanket( pSvc, // Indicates the proxy to set RPC_C_AUTHN_WINNT, // RPC_C_AUTHN_xxx RPC_C_AUTHZ_NONE, // RPC_C_AUTHZ_xxx NULL, // Server principal name RPC_C_AUTHN_LEVEL_CALL, // RPC_C_AUTHN_LEVEL_xxx RPC_C_IMP_LEVEL_IMPERSONATE, // RPC_C_IMP_LEVEL_xxx NULL, // client identity EOAC_NONE // proxy capabilities ); if (FAILED(hres)) { // Could not set proxy blanket. if(pSvc) pSvc->Release(); if(pLoc) pLoc->Release(); return FALSE; } Using Smart Pointers 如果你经常使用用享对象指针，如COM 接口等，那么建议使用智能指针来处理。智能指针会自动帮助你维护对象引用记数，并且保证你不会访问到被删除的对象。这样，不需要关心和控制接口的生命周期。关于智能指针的进一步知识可以看看Smart Pointers - What, Why, Which? 和 Implementing a Simple Smart Pointer in C++这两篇文章。 如面是一个展示使用ATL's CComPtr template 智能指针的代码，该部分代码来至于MSDN。 #include <windows.h> #include <shobjidl.h> #include <atlbase.h> // Contains the declaration of CComPtr. int WINAPI wWinMain(HINSTANCE hInstance, HINSTANCE, PWSTR pCmdLine, int nCmdShow) { HRESULT hr = CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE); if (SUCCEEDED(hr)) { CComPtr<IFileOpenDialog> pFileOpen; // Create the FileOpenDialog object. hr = pFileOpen.CoCreateInstance(__uuidof(FileOpenDialog)); if (SUCCEEDED(hr)) { // Show the Open dialog box. hr = pFileOpen->Show(NULL); // Get the file name from the dialog box. if (SUCCEEDED(hr)) { CComPtr<IShellItem> pItem; hr = pFileOpen->GetResult(&pItem); if (SUCCEEDED(hr)) { PWSTR pszFilePath; hr = pItem->GetDisplayName(SIGDN_FILESYSPATH, &pszFilePath); // Display the file name to the user. if (SUCCEEDED(hr)) { MessageBox(NULL, pszFilePath, L"File Path", MB_OK); CoTaskMemFree(pszFilePath); } } // pItem goes out of scope. } // pFileOpen goes out of scope. } CoUninitialize(); } return 0; } Using == Operator Carefully 先来看看如下代码; CVehicle* pVehicle = GetCurrentVehicle(); // Validate pointer if(pVehicle==NULL) // Using == operator to compare pointer with NULL return FALSE; // Do something with the pointer pVehicle->Run(); 上面的代码是正确的,用语指针检测。但是如果不小心用“=”替换了“==”，如下代码； CVehicle* pVehicle = GetCurrentVehicle(); // Validate pointer if(pVehicle=NULL) // Oops! A mistyping here! return FALSE; // Do something with the pointer pVehicle->Run(); // Crash!!! 看看上面的代码,这个的一个失误将导致程序崩溃。 这样的错误是可以避免的，只需要将等号左右两边交换一下就可以了。如果在修改代码的时候，你不小心产生这种失误，这个错误在程序编译的时候将被检测出来。
-
发表了文章 2011-06-17
Intercepting Calls to COM Interfaces
Download sample - 95.21 KB Table of Contents Introduction Some Basic Concepts of COM Practical Example Approach #1: Proxy Object Approach #2: Vtable Patching Conclusion References History Introduction In this article, I’m going to describe how to implement COM interface hooks. COM hooks have something in com mon with the user-mode API hooks (both in goals and in methods), but there are also some significant differences due to the features of COM technology. I’m going to show two of the most often used approaches to the problem, emphasizing advantages and disadvantages of each one. The code sample is simplified as much as possible, so we can concentrate on the most important parts of the problem. Some Basic Concepts of COM Before we start with intercepting calls to COM objects, I’d like to mention some underlying concepts of COM technology. If you know this stuff well, you can just skip this boring theory and move straight to the practical part. All COM classes implement one or several interfaces. All the interfaces must be derived from IUnknown . It’s used for reference counting and obtaining pointers to other interfaces implemented by an object. Every interface has a globally unique interface identifier - IID . Clients use interface pointers to call all methods of COM objects. This feature makes COM com ponents independent on the binary level. It means if a COM server is changed, it doesn’t require its clients to be recom piled (as long as the new version of the server provides the same interfaces). It is even possible to replace COM server with your own implementation. All calls of COM interface methods are executed by means of virtual method table (or simply vtable ). A pointer to vtable is always the first field of every COM class. This table is, in brief, an array of pointers - pointers to the class methods (in order of their declaration). When the client invokes a method, it makes a call by the according pointer. COM servers work either in the context of a client process or in the context of some another process. In the first case, the server is a DLL which is loaded into client process. In the second case, the server executes as another process (maybe even on another com puter). To com municate with the server, the client loads so-called Proxy/Stub DLL. It redirects calls from the client to the server. To be easily accessible, a COM server should be registered in the system registry. There are several functions, which clients can use to create an instance of COM , but usually it is CoGetClassObject , CoCreateInstanceEx or (the most com mon) CoCreateInstance . If you want to get more detailed information, you can use MSDN or one of the sources in the References section. Practical Example Let’s see how we can intercept calls to COM interface. There are several different approaches to this problem. For instance, we can modify registry or use CoTreatAsClass or CoGetInterceptor functions. In this article, two of the most com monly used approaches are covered: use of proxy object and patching of the virtual method table. Each of them has its own advantages and disadvantages, so what to choose depends on the task. A piece of code for this article contains the implementation for the simplest COM server DLL, client application and two samples of COM hooking that demonstrate the approaches I’m going to describe. Let’s run the client application without hooks installed. First, we register the COM server invoking the com mand regsvr32 Com Sample.dll . And then we run Com SampleClient.exe or SrciptCleint.js to see how the client of the sample server works. Now it’s time to set some hooks. Approach #1: Proxy Object COM is pretty much about binary encapsulation. Client uses any COM server via interface and one implementation of the server can be changed to another without rebuilding the client. This feature can be used in order to intercept calls to COM server. The main idea of this method is to intercept the COM object creation request and substitute the newly created instance with our own proxy object. This proxy object is a COM object with the same interface as the original object. The client code interacts with it as it is the original object. The proxy object usually stores a pointer to the original object, so it can call original object’s methods. As I mentioned, proxy object has to implement all interfaces of the target object. In our sample, it will be just one interface: ISampleObject . The proxy class is implemented with ATL: Collapse class ATL_NO_VTABLE CSampleObjectProxy :public ATL::CCom ObjectRootEx< ATL::CCom MultiThreadModel> ,public ATL::CCom CoClass< CSampleObjectProxy, &CLSID_SampleObject> ,public ATL::IDispatchImpl< ISampleObject, &IID_ISampleObject, &LIBID_Com SampleLib, 1 , 0 > {public : CSampleObjectProxy(); DECLARE_NO_REGISTRY() BEGIN_COM _MAP(CSampleObjectProxy)COM _INTERFACE_ENTRY(ISampleObject)COM _INTERFACE_ENTRY(IDispatch) END_COM _MAP() DECLARE_PROTECT_FINAL_CONSTRUCT()public : HRESULT FinalConstruct();void FinalRelease();public : HRESULT static CreateInstance(IUnknown* original, REFIID riid, void **ppvObject);public : STDMETHOD(get_ObjectName)(BSTR* pVal); STDMETHOD(put_ObjectName)(BSTR newVal); STDMETHOD(DoWork)(LONG arg1, LONG arg2, LONG* result); ... }; STDMETHODIMP CSampleObjectProxy::get_ObjectName(BSTR* pVal) {return m_Name.CopyTo(pVal); } STDMETHODIMP CSampleObjectProxy::DoWork(LONG arg1, LONG arg2, LONG* result) { *result = 42 ;return S_OK; } STDMETHODIMP CSampleObjectProxy::put_ObjectName(BSTR newVal) {return m_OriginalObject-> put_ObjectName(newVal); } Notice that if there are methods you’re not interested in (for example, put_ObjectName ), you have to implement them in the proxy anyway. Now, we have to intercept creation of the target object to replace it with our proxy. There are several Windows API functions capable of creating COM objects, but usually CoCreateInstance is used. To intercept target object creation, I’ve used mhook library to hook CoCreateInstance and CoGetClassObject . The technique of setting API hooks is a widely covered topic. If you want to get more detailed information about it you can see, for example, Easy way to set up global API hooks article by Sergey Podobry. Here is the implementation of the CoCreateInstance hook function: Collapse HRESULT WINAPI Hook::CoCreateInstance (REFCLSID rclsid, LPUNKNOWN pUnkOuter, DWORD dwClsContext, REFIID riid, LPVOID* ppv) {if (rclsid == CLSID_SampleObject) {if (pUnkOuter)return CLASS_E_NOAGGREGATION; ATL::CCom Ptr< IUnknown> originalObject; HRESULT hr = Original::CoCreateInstance(rclsid, pUnkOuter, dwClsContext, riid, (void **)&originalObject);if (FAILED(hr))return hr;return CSampleObjectProxy::CreateInstance(originalObject, riid, ppv); }return Original::CoCreateInstance(rclsid, pUnkOuter, dwClsContext, riid, ppv); } To see how the sample of proxy object the approach works, specify the full name of the Com InterceptProxyObj.dll in the AppInit_DLLs registry value (HKEY_LOCAL_MACHINE/Software/Microsoft/Windows NT/CurrentVersion/Windows ). Now you can run Com SampleClient.exe or ScriptClient.js and see that calls to the target object method are intercepted. Approach #2: Vtable Patching The other way of intercepting calls to a COM object is to modify the object’s virtual methods table. It contains pointers to all public methods of a COM object, so they can be replaced with the pointers to hook functions. Unlike the previous one, this approach doesn’t require hooks to be set before the client gets pointer to the target object. They can be set at any place where a pointer to the object is accessible. Here is the HookMethod function code which sets a hook for a COM method: Collapse HRESULT HookMethod(IUnknown* original, PVOID proxyMethod, PVOID* originalMethod, DWORD vtableOffset) { PVOID* originalVtable = *(PVOID**)original;if (originalVtable[vtableOffset] == proxyMethod)return S_OK; *originalMethod = originalVtable[vtableOffset]; originalVtable[vtableOffset] = proxyMethod;return S_OK; } To set hooks for the ISampleObject interface methods, the InstallCom InterfaceHooks function is used: Collapse HRESULT InstallCom InterfaceHooks(IUnknown* originalInterface) {// Only single instance of a target object is supported in the sample if (g_Context.get())return E_FAIL; ATL::CCom Ptr< ISampleObject> so; HRESULT hr = originalInterface-> QueryInterface(IID_ISampleObject, (void **)&so);if (FAILED(hr))return hr; // we need this interface to be present // remove protection from the vtable DWORD dwOld = 0 ;if (!::VirtualProtect(*(PVOID**)(originalInterface),sizeof (LONG_PTR), PAGE_EXECUTE_READWRITE, &dwOld))return E_FAIL;// hook interface methods g_Context.reset(new Context); HookMethod(so, (PVOID)Hook::QueryInterface, &g_Context-> m_OriginalQueryInterface, 0 ); HookMethod(so, (PVOID)Hook::get_ObjectName, &g_Context-> m_OriginalGetObjectName, 7 ); HookMethod(so, (PVOID)Hook::DoWork, &g_Context-> m_OriginalDoWork, 9 );return S_OK; } Virtual method table may be in a write-protected area, so we have to remove the protection with VirtualProtect before setting hooks. Variable g_Context is a structure, which contains data associated with the target object. I’ve made g_Context global to simplify the sample though it supports only one target object to exist at the same time. Here is the hook functions code: Collapse typedef HRESULT (WINAPI *QueryInterface_T) (IUnknown* This, REFIID riid, void **ppvObject); STDMETHODIMP Hook::QueryInterface(IUnknown* This, REFIID riid, void **ppvObject) { QueryInterface_T qi = (QueryInterface_T)g_Context-> m_OriginalQueryInterface; HRESULT hr = qi(This, riid, ppvObject);return hr; } STDMETHODIMP Hook::get_ObjectName(IUnknown* This, BSTR* pVal) {return g_Context-> m_Name.CopyTo(pVal); } STDMETHODIMP Hook::DoWork(IUnknown* This, LONG arg1, LONG arg2, LONG* result) { *result = 42 ;return S_OK; } Take a look at the hook functions definitions. Their prototypes are exactly as the target interface methods prototypes except that they are free functions (not class methods) and they have one extra parameter - this pointer. That’s because COM methods are usually declared as stdcall , this is passed as an implicit stack parameter. When using this approach, you have several things to remember. First of all, when you set a method hook, it will work not only for the current instance of the COM object. It’ll work for all objects of the same class (but not for all the classes implementing the interface hooked). If there are several classes implementing the same interface and you want to intercept calls for all instances of this interface, you will need to patch vtables of all this classes. If you want to store some data, which is specific for every object, you have to store in the static memory area a collection of contexts accessible by the target object pointer value. You also have to watch target object’s lifetime. And if you expect multithreaded access for the target object, you have to provide synchronization for the static collection. If you need to call the target object’s method from your hook function, you have to be careful. You can’t just call a hooked method by an interface pointer because it will cause an access to vtable and a call of the hook function (which is not what you want). So you have to save the pointer to the original method and use it directly to call the method. Here is another tricky thing. When you set a hook, be careful and do not hook the same method twice. If you save a pointer to the original method, it will be rewritten on the second hook attempt. The good news is that in this approach, you don’t have to implement hooks for the methods you don’t need to intercept. And intercepting object’s creation is not necessary too. To see how this part of the sample works, specify the full name of Com InterceptVtablePatch.dll in the AppInit_DLLs registry value, just like you did before, and run the client. Conclusion Both approaches described in the article have advantages and disadvantages. The proxy object approach is much easier to implement, especially if you need sophisticated logic in your proxy. But you have to replace the target object with your proxy before the client gets the pointer to the original object, and it may be difficult or simply impossible in some cases. Also you have to provide in your proxy the same interface as the target object has, even if you have only a couple of methods to intercept. And real-world COM interfaces can be really large. If the target object has several interfaces, you’ll probably need to implement them all. The vtable patching approach demands much more careful implementation and requires a developer to remember a lot of things. It needs some extra amount of code for handling several target object instances of the same interface or calling the target’s methods. But it doesn’t require to set hooks directly after target’s creation, the hooks can be set at any moment. It also allows implementing hooks only for methods you actually need to intercept. Which approach is handier at the moment usually depends on the situation.
-
发表了文章 2011-06-06
内存池完整实现代码及一些思考
为了提高效率和有效的监控内存的实时状态，我们采取了内存池的思想来解决效率与对内存实现监控的问题。 网上查找到了一些方案，根据自己的理解实现了应用。 我们什么时候要调用到内存池， 1，当我们频繁的申请释放同样数据大小的内存空间，我们可以用比动态new更有效方式来管理内存时，我们应该用内存池来提高效率。 2，当我们需要知道内存实时的申请状态，以便于对于服务器内存状态做实时预警时，我们可以用内存池的接口，来给内存增加监控。 实现的特点： 1，内存池内存单元大小可以动态定义，实现多级内存池。 2，申请效率很高，单元测试下是普通new/delete的4倍左右，当然具体性能还应机器类别而异。 MemoryPool.h 的实现 //该内存池理论来自于IBM文章，http://www.ibm.com/developerworks/cn/linux/l-cn-ppp/index6.html//作者冯宏华，徐莹，程远，汪磊享有论文著作权，由2011-06-06 konyel lin根据相关代码和理论进行优化修改。#include <string>#include <malloc.h>//内存对齐值，可以根据机器取指长度进行设置#define MEMPOOL_ALIGNMENT 4#define USHORT unsigned short#define ULONG unsigned longstruct MemoryBlock{ USHORT nSize; USHORT nFree; USHORT nFirst; USHORT nDummyAlign1; MemoryBlock* pNext; char aData[1]; static void* operator new(size_t,USHORT nTypes, USHORT nUnitSize){ return ::operator new(sizeof(MemoryBlock) + nTypes * nUnitSize); } static void operator delete(void *p, size_t){ ::operator delete (p); } MemoryBlock (USHORT nTypes = 1, USHORT nUnitSize = 0); ~MemoryBlock() {}};class MemoryPool{ private: MemoryBlock* pBlock; USHORT nUnitSize; USHORT nInitSize; USHORT nGrowSize; public: MemoryPool( USHORT nUnitSize, USHORT nInitSize = 1024, USHORT nGrowSize = 256 ); ~MemoryPool(); void* Alloc(); void Free( void* p );}; MemoryPool.cpp 的实现 #include "MemoryPool.h"MemoryPool::MemoryPool( USHORT _nUnitSize,USHORT _nInitSize, USHORT _nGrowSize ){ pBlock = NULL; nInitSize = _nInitSize; nGrowSize = _nGrowSize; if ( _nUnitSize > 4 ) nUnitSize = (_nUnitSize + (MEMPOOL_ALIGNMENT-1)) & ~(MEMPOOL_ALIGNMENT-1); else if ( _nUnitSize <= 2 ) nUnitSize = 2; else nUnitSize = 4;}void* MemoryPool::Alloc(){ MemoryBlock* pMyBlock; if ( !pBlock ){ //第一次调用初始化内存块 pMyBlock =new(nGrowSize, nUnitSize) MemoryBlock(nGrowSize, nUnitSize); pBlock = pMyBlock; return (void*)(pMyBlock->aData); } pMyBlock = pBlock; while (pMyBlock && !pMyBlock->nFree ) pMyBlock = pMyBlock->pNext; if ( pMyBlock ){ printf("get a mem from block/n"); char* pFree = pMyBlock->aData+(pMyBlock->nFirst*nUnitSize); //aData记录实际的内存单元标识 pMyBlock->nFirst = *((USHORT*)pFree); pMyBlock->nFree--; return (void*)pFree; } else{ printf("add a new block/n"); if (!nGrowSize) return NULL; pMyBlock = new(nGrowSize, nUnitSize) MemoryBlock(nGrowSize, nUnitSize); if (!pMyBlock ) return NULL; pMyBlock->pNext = pBlock; pBlock = pMyBlock; return (void*)(pMyBlock->aData); }}void MemoryPool::Free( void* pFree ){ MemoryBlock* pMyBlock = pBlock; MemoryBlock* preMyBlock; //确定该待回收分配单元（pFree）落在哪一个内存块的指针范围内,大于起始节点，小于终止节点。 while ( ((ULONG)pMyBlock->aData > (ULONG)pFree) || ((ULONG)pFree >= ((ULONG)pMyBlock->aData + pMyBlock->nSize))){ //不在内存块范围内，则历遍下一个节点 preMyBlock=pMyBlock; pMyBlock=pMyBlock->pNext; } pMyBlock->nFree++; *((USHORT*)pFree) = pMyBlock->nFirst; pMyBlock->nFirst = (USHORT)(((ULONG)pFree-(ULONG)(pBlock->aData)) / nUnitSize); //判断内存块是否全部为自由状态，是则释放整个内存块 if (pMyBlock->nFree*nUnitSize == pMyBlock->nSize ){ preMyBlock->pNext=pMyBlock->pNext; delete pMyBlock; }}MemoryBlock::MemoryBlock (USHORT nTypes, USHORT nUnitSize): nSize (nTypes * nUnitSize), nFree (nTypes - 1), nFirst (1), pNext (0){ char * pData = aData; for (USHORT i = 1; i < nTypes; i++) { //将内存块的前2个字节用来存放内存单元的标识 *reinterpret_cast<USHORT*>(pData) = i; pData += nUnitSize; }}
-
发表了文章 2011-06-05
Handling multiple pending socket read and write operations
The following source was built using Visual Studio 6.0 SP5 and Visual Studio .Net. You need to have a version of the Microsoft Platform SDK installed Note that the debug builds of the code waste a lot of CPU cycles due to the the debug trace output. It's only worth profiling the release builds. JBSocketServer6.zip - a large packet echo server with multiple pending reads and writes - 111 Kb JBSocketShutdown.zip - an off switch for the servers - 42 Kb Overview "How do you handle the problem of multiple pending WSARecv() calls?" is a common question on the Winsock news groups. It seems that everyone knows that it's often a good idea to have more than one outstanding read waiting on a socket and everyone's equally aware that sometimes code doesn't work right when you do that. This article explains the potential problems with multiple pending recvs and provides a solution within the reusable server framework that we've been developing over the last few articles. That's out of order There is a subtle issue when using IO completion ports with multiple threads. Although operations using the IO completion port will always complete in the order that they were submitted, thread scheduling issues may mean that the actual work associated with the completion is processed in an undefined order. For example, if we were to submit three WSARecv requests on a socket then they are guaranteed to complete in the order that we submitted them, however if we have 2 threads servicing the IO completion port two of the completions could be being processed simultaneously. If the thread processing the 'first' WSARecv completion is interrupted the second may be completely processed before the first. This is even more likely to occur on machines with multiple processors where the two threads may really be executing simultaneously, but it's possible on single processor boxes too. As always, this is the kind of subtle problem that probably wont show its face until you release the software to production... The example above is easy to avoid, simply don't have multiple WSARecv requests outstanding on a single socket. This is what we have done so far in the example servers developed in the previous articles. However this reduces performance, it's always more performant to have a receive pending when the data actually arrives on the wire than it is to post a receive after the data has already arrived. Having multiple WSARecv calls outstanding ensures that there's always a call pending. What's more, the problem isn't limited to having multiple WSARecvs. In our server framework we marshal all socket IO calls from the user's threads into our IO thread pool using the IO completion port. This means that there is the potential for a user thread to issue multiple consecutive writes to the socket and for them to be executed in an undefined order. In our example servers so far, code like this: Collapse | Copy Code void CSocketServer::ReadCompleted( Socket *pSocket, CIOBuffer *pBuffer) { // do stuff... pSocket->Write(pBuffer); // echo the command pSocket->Write(pResponse1); // send part 1 response pSocket->Write(pResponse2); // send part 2 response } is potentially unsafe because the writes don't occur synchronously on the user's thread, they are posted to the IO completion port and occur in the IO thread pool. Preserving the order of IO completion operations is relatively straight forward. As you'll remember, the overlapped structure passed to all calls that use IO completion ports represents 'per call' data. We can, and do, extend the overlapped structure to include our own 'per call' data by using the CIOBuffer class. If we add a sequence number to the CIOBuffer we can set the sequence number to the 'next' value in the user's thread and then make sure we process the buffers in order in the IO thread pool. This concept applies to any IO completion port operation and each distinct operation requires its own sequence number. For our server framework that means that our Socket class must now maintain independent sequence numbers for read and write requests. The sequence number management code inside the Socket's Write method could be something like this: Collapse | Copy Code pBuffer->SetSequenceNumber(m_writeSequenceNumber++); To ensure that the sequence numbers actually represent the order that the operations are submitted requires that the setting of the sequence number and the submission of the operation are an atomic operation. For our socket writes this isn't a problem as we only guarentee the order of writes that are performed on a single thread, for socket reads we need to ensure that the allocation of the sequence number and call to WSARecv() occur without another thread having a chance to perform read at the same time. This involves using a critical section to lock access to the socket during the sequence number allocation and WSARecv() call. Failure to lock in this area can lead to the actual order that the WSARecv() calls are made failing to match the ordering of the sequence numbers allocated. Orderly processing The code to ensure that the IO completions are handled in order is a little more complex. For each distinct IO operation we need to keep track of the next sequence number that we can process. When a call to GetQueuedCompletionStatus() returns we need to compare the sequence number in the request with the next sequence number that we can process. If these numbers match then we can process the request. If they don't then the request cannot be processed at this time. If an IO operation cannot be processed it should be stored for later processing. The storage of the out of sequence request needs to be keyed on the sequence number. When an IO thread finds that it can't process the current request it should add the current request to the store and see if there's a request in the store that can be processed. When a request is processed the last thing that the IO thread should do is atomically increment the value representing the next sequence number to process and check to see if there's an IO request in the store that can be processed. The above strategy handles the situation where multiple IO requests complete concurrently. Only one thread can be processing an IO request that meets the criteria of being the next one to process, all other threads will simply add their requests to the store. When the thread that's processing a request finishes processing it can check to see if there are other requests in the store that can now be processed. If a thread needs to store its IO request then it can do so and then check for a request that can be processed in an atomic operation. It's actually more complex to read about than it is to look at, the code to process an operation in order might look like this: Collapse | Copy Code pBuffer = pSocket->m_outOfSequenceWrites.GetNext(pBuffer); while(pBuffer) { DWORD dwFlags = 0; DWORD dwSendNumBytes = 0; if (SOCKET_ERROR == ::WSASend( pSocket->m_socket, pBuffer->GetWSABUF(), 1, &dwSendNumBytes, dwFlags, pBuffer, NULL)) { // handle errors etc. } pBuffer = pSocket->m_outOfSequenceWrites.ProcessAndGetNext(); } The store itself needs to map sequence numbers to CIOBuffers. The obvious choice of data structure is a std::map<> though your performance requirements and profiling may dictate a different choice. GetNext() takes a buffer, compares its sequence number with the next one we can process and either returns the buffer or adds the buffer to the map and then checks the map to see if the first buffer in the map is the one we can process. Remember that the map stores its elements in order of their keys and that we're using the sequence number as the key, so m_list.begin() refers to the element in the map with the lowest sequence number. If this function returns null then we're still waiting for the 'next' buffer to arrive. Collapse | Copy Code CIOBuffer *CIOBuffer::InOrderBufferList::GetNext( CIOBuffer *pBuffer) { CCriticalSection::Owner lock(m_criticalSection); if (m_next == pBuffer->GetSequenceNumber()) { return pBuffer; } BS::value_type value(pBuffer->GetSequenceNumber(), pBuffer)); std::pair<BS::iterator, bool> result = m_list.insert(value); if (result.second == false) { // handle error, element already in map } CIOBuffer *pNext = 0; BufferSequence::iterator it; it = m_list.begin(); if (it != m_list.end()) { if (it->first == m_next) { pNext = it->second; m_list.erase(it); } } return pNext; } After processing a buffer the thread can check to see if there's another buffer that it can handle. It needs to increase the last processed value and perform the check atomically, hence the locking. Collapse | Copy Code CIOBuffer *CIOBuffer::InOrderBufferList::ProcessAndGetNext() { CCriticalSection::Owner lock(m_criticalSection); ::InterlockedIncrement(&m_next); CIOBuffer *pNext = 0; BufferSequence::iterator it; it = m_list.begin(); if (it != m_list.end()) { if (it->first == m_next) { pNext = it->second; m_list.erase(it); } } return pNext; } Handling reads If the CIOBuffer used by every write that occurs contains a sequence number then similar code could be used to ensure that completed read requests are processed in the correct order. However, there's little point in this code being placed in the server framework as different users of the framework may require different functionality. The CSocketServer derived class could use the CIOBuffer::InOrderBufferList class to maintain processing order or it could simply dispatch the read completions to another IO completion port to pass them across to a business logic thread pool. In this case it's the code in the business logic thread pool that actually processes the data and the order should be maintained there. It may even need do both, ensuring packet order in the CSocketServer class itself so that it can successfully break the byte stream into messages and then dispatching the messages to the business logic thread pool and ensuring that these complete messages are also processed in the correct order. Locking granularity Each Socket must now keep track of independent read and write sequence numbers and maintain a map of out of sequence write requests. Manipulation of the map and associated next sequence number counter must be protected. We use a critical section to protect this code. Be aware that allocating a critical section for each Socket connection is potentially resource intensive. Instead we could choose to trade locking granularity for performance. The CSocketServer class already has a critical section that it uses to protect its lists of Sockets, we could pass a reference to this critical section to each Socket rather than have them create their own critical section. The problem with doing this is that we serialise every Socket's map access. This work performed inside the critical section is small, but a better solution might be to create a critical section for every X sockets where X is a value that is determined by profiling your application. Only paying for what you use Including sequence numbers in all buffers used for sending and receiving and ensuring the writes are processed in order adds a little overhead to the work of the IO threads. If you are sure that your server doesn't require this functionality, perhaps because you know that due to your protocol design there will only ever be a single read or write request pending, you can opt not to include this functionality by passing false as the useSequenceNumbers flag in the CSocketServer's constructor. Enabling read or write sequence numbers independently is left as an exercise for the reader. An example To demonstrate the concept of ensuring the ordering of multiple reads and writes we've come up with a rather contrived example. The packet echo server that we developed in the previous article has been changed as follows: It now does its work in a business logic thread pool so that we can demonstrate maintaining receive order in both the socket server and the business logic thread pool when the socket server's worker thread isn't doing the processing itself. It works with larger packets; we use a two byte packet header rather than a one byte header. This two byte header represents the length of the packet using the following format: packetLength = byte1 + (byte2 * 256). The length of the packet includes the two byte header. When the client initially connects it posts a configurable number of reads. As each read completes it posts a new read so that it maintains the number of outstanding reads. It processes the reads in order, and due to the fact that we now have multiple reads outstanding, CSocketServer::ProcessDataStream() has changed so that when more data is required we don't simply reissue a read to read more data into the same buffer. It echoes the packet back to the client in pieces by issuing multiple write requests. The large packet echo server is available for download here in SocketServer6.zip. Testing using telnet is possible, though more complex, you may find it easier to use the test harness that we develop here to test it. As with the previous examples, the server runs until a named event is set and then shuts down. The very simple Server Shutdown program, available here, provides an off switch for the server. Although both the server and thread pool classes are configurable as to whether they use sequence numbers to maintain packet order these settings can only be set in one way for the server to actually work in the way that the test harness expects. All packet ordering flags must be set to true. The purpose of the flags is so that you can turn off the various sequencing required and see the effect on the test. It's not intended that the server can run reliably in any other configuration. Collapse | Copy Code CThreadPool pool( 5, // initial number of threads to create 5, // minimum number of threads to keep in the pool 10, // maximum number of threads in the pool 5, // maximum number of "dormant" threads 5000, // pool maintenance period (millis) 100, // dispatch timeout (millis) 10000, // dispatch timeout for when pool is at max threads 20, // (1) number of reads to post true, // (2) maintain packet order with sequence numbers true); // (3) echo packets with multiple writes pool.Start(); CSocketServer server( INADDR_ANY, // address to listen on 5001, // port to listen on 10, // max number of sockets to keep in the pool 10, // max number of buffers to keep in the pool 1024, // buffer size pool, 65536, // max message size true, // (4) maintain read packet order with sequence numbers true, // (5) maintain write packet order with sequence numbers true); // (6) issue a new read before we've completely processed // this one The configuration flags can be adjusted to witness the following effects: The configuration shown above ensures that packets into the read completion method are processed in sequence - this maintains the validity of the incoming packet data; Packets into the worker thread are maintained in sequence - this maintains the order of echoing the actual packets; and the sequence of write calls is maintained - this maintains the validity of the outgoing packet data. If the number of reads to post (1) is reduced to 1 then there is no need to maintain the read completion sequencing ((4) can be set to false) as long as read completion method doesn't issue another read until it has completed the processing of the current one ((6) should be set to false). If the business logic thread pool doesn't attempt to maintain packet ordering ((2) set to false) then the test will likely report sequence number mismatches - as the packets are echoed out of sequence, and response != message errors as multiple threads in the business logic thread pool attempt to write fragments of the message to the socket in an unsynchronised manner and thus interleave sections of different messages. If (2) is left set to false but (3) is also set to false then the test will only fail with sequence number mismatches as the threads are now echoing their packets in a single write so the data in the individual packets cant be corrupted by being interleaved with sections of other packets. Unfortunately it's currently impossible to simply turn off write packet ordering at the socket server (option (5)) as doing so also turns off read packet sequencing and so makes it impossible to get valid data to the business logic thread so that it can echo it and we can witness the writes being performed out of sequence. If you're interested in seeing this in action then you can hack at the socket server code. Revision history 15th July 2002 - Initial revision. 12th August 2002 - Removed the race condition in socket closure - Thanks to David McConnell for pointing this out. Derived class can receive connection reset and connection error notifications. Socket provides a means to determine if send/receive are connected. Dispatch to the thread pool now uses shared enums rather than hard coded constant values. General code cleaning and lint issues. Adjusted the code and article so that each socket has its own critical section and the resource utilisation optimisation is suggested, rather than imposed. Fixed a bug whereby the critical section that is used to protect the per socket data was owned by the worker thread rather than the per socket data. Other articles in the series A reusable socket server class Business logic processing in a socket server Speeding up socket server connections with AcceptEx Handling multiple pending socket read and write operations Testing socket servers with C# and .Net A high performance TCP/IP socket server COM component for VB
-
发表了文章 2011-05-23
Microsoft Windows CE 5.0 Board Support Package, Boot Loader, and Kernel Startup Sequence
Mark PlaggeMicrosoft Corporation May 2004 Applies To: Microsoft® Windows® CE 5.0 Summary Learn about the initial, low-level startup sequence and the hardware platform functions that are performed when the boot loader and OEM abstraction layer (OAL) are developed and the kernel is run. The startup sequence is an integral part of developing the OAL for a board support package (BSP), and the development process must be implemented correctly for initializing the CPU as well as on-chip and off-chip peripherals. The process described in this article is the minimum set of functions that need to be implemented. It might be necessary for user-defined functions to be implemented where appropriate, depending on the specific device and peripheral set you are developing. Although the startup sequence for the different hardware platform CPU architectures that Windows CE supports is very similar, for the sake of simplicity, this article discusses the ARM kernel startup sequence. Contents IntroductionOverview of the BSP Development ProcessOverview of the Boot Loader and Kernel Startup SequenceBoot Loader Startup SequenceKernel Startup SequenceConclusion Introduction Creating a BSP is the initial development activity required for building a Windows CE-based device. You can develop a BSP on your own or start from a BSP provided by Microsoft or a third party. Leveraging an existing BSP can greatly reduce the amount of development work required. Regardless of the starting point you choose, the development process and startup sequence for the boot loader and the kernel are almost the same. This article describes this startup sequence and the functions you must implement to develop a boot loader and kernel that will boot Windows CE on a device. It also provides information about the recommended functionality that should be implemented in each function that is called by the boot loader and kernel. Because several of the functions calls do very similar actions, the kernel startup work can leverage the work done to develop the boot loader. Overview of the BSP Development Process Like many development tasks, the process for developing a BSP involves high-level and low-level tasks. For example, at the highest level, you must select the hardware platform, and at the lowest level, you must develop a boot loader and the OEM abstraction layer (OAL) for the kernel. The following illustration shows the high-level steps needed to develop the low-level software for a BSP. You begin by selecting or developing the hardware platform that the BSP will target. This includes developing a board with a CPU that can run the Windows CE operating system (OS), selecting the peripheral devices that will interface with the device, and including the interfaces required to support BSP development and debugging. For example, you might not need to include support for serial connections on a device for consumers, but a serial connection is typically important for low-level software development. You should consider populating your device with a debug header that can interface to a debug board for development, but can be depopulated before the device is used for production. After you have identified and created the hardware platform, you must develop a BSP that will run Windows CE on the device. Often, you can modify a BSP from Microsoft or a third party, which will greatly reduce your BSP development work. The process of starting with a BSP from Microsoft is called BSP cloning. You can find information about this process in the Windows CE Help documentation. If a BSP is not available, you will have to develop one entirely on your own. Because developing a BSP entirely on your own is a significant amount of work, Microsoft recommends that you start with a BSP from Microsoft or a third party. At this point, you move to the low-level development tasks by developing a boot loader that can reside in persistent memory on the device. The boot loader's primary function is to initialize enough of the hardware and CPU to allow the hardware to communicate with the development environment for downloading a Windows CE-based run-time image. You can extend the boot loaders functionality to be used for any other purpose that your device requires. Later sections of this paper discuss the low-level startup functions that you must implement in the boot loader. After a boot loader has been developed, you download it and write it into persistent memory on the device using the tools specified by the silicon vendor. You then start development work on the OAL portion of the BSP. Part of the OAL work consists of implementing the bootstrapping kernel functions required to start the Windows CE kernel. Developing the OAL is a step-by-step process of implementing the startup functions and implementing the code that will initialize the hardware on the device in preparation for running the Windows CE kernel. You can leverage or share many of the startup functions that are implemented in the boot loader during OAL development. Once you have developed, downloaded, and debugged the OAL, a minimal Windows CE kernel will be running on the hardware platform. Next, you add support for the peripherals that you have chosen for the device. To do this, you have to iteratively add and debug the drivers for these peripherals one at a time. If you have multiple developers working on drivers, the drivers can be developed in a series and added to the BSP one at a time. Depending on what peripherals you have chosen, or if you are using a System-On-Chip (SOC), you can add drivers from Microsoft or from a third party. You can leverage these drivers as the starting point for each driver that you develop. Once the device drivers have been developed, you should have a BSP with an OAL and the drivers for each peripheral that you intend to support on the hardware platform. Next, you will plan and implement power management. Power management is an important part of the driver and BSP development process. Once you have planned your power management system, which includes deciding to what extent your device will rely on battery power, you will use the Power Manager to help you implement those power management capabilities. Although power management is not the focus of this article, you can find documentation that explicitly covers power management in Windows CE Help and the MSDN Library. There you will find details about how to make each peripheral driver power-management-aware, as well as how drivers interact with the Power Manager and OAL. As you develop the OAL, bring up device drivers, and implement power management, you must test each component. The Windows CE Test Kit (CETK) provides a wide variety of tests to help you with this process. For information about the CETK, see Windows CE Help and the MSDN Library. The final steps in the BSP development process are to create a software development kit (SDK) and package the BSP into an .msi install file so that it can be installed by others. An SDK is a set of headers, libraries, connectivity files, run-time files, OS design extensions, and Help documentation that developers use to write applications for a specific OS design. The contents of an SDK allow developers to create and debug an application on the run-time image built from your OS design. Windows CE provides an SDK Wizard to create an SDK from your BSP and an Export Wizard to package your BSP into an .msi install file. For detailed information on all aspects of the BSP development process, see Windows CE documentation. Overview of the Boot Loader and Kernel Startup Sequence For the most part, the boot loader and kernel share the same Startup functions, as well as some of the code that is called out in the subsequent functions. Consequently, you can leverage or share much of the code initially implemented in the boot loader startup sequence during the kernel startup sequence. The following table shows the startup functions used by the boot loader and the kernel for an ARM-based startup sequence. The startup sequence for the kernel calls into functions that are part of the shared-source license agreement that installs code into the private tree. Because of this, you should sequence the functions as they are shown below. Within each function you might need to customize the order in which components are initialized, depending on your CPU, memory, and peripheral set. Boot Loader Startup Sequence Kernel Startup Sequence Startup() Startup() EbootMain() KernelStart() BootloaderMain() ARMInit() OEMDebugInit() OEMInitDebugSerial() OEMPlatformInit() OEMInit() OEMPreDownload() KernelInit() Download HeapInit() OEMLaunch() InitMemoryPool() ProcInit() SchedInit() FirstSchedule() SystemStartupFunc() IOCTL_HAL_POSTINIT Function calls in bold text denote functions that OEMs need to implement. Documentation is provided in Platform Builder and on MSDN that covers the step-by-step process, with a checklist, for bringing up a boot loader and OAL. Boot Loader Startup Sequence The following table shows the boot loader startup sequence. Boot Loader Startup Sequence Startup() EbootMain() BootloaderMain() OEMDebugInit() OEMPlatformInit() OEMPreDownload() Download occurs OEMLaunch() Functions in bold text denote functions that OEMs need to implement. Startup() -> Startup.s The boot loader Startup function is the first code that is run on the device after a hardware reset or run-time reset, and when the system wakes up. This function performs the following tasks: Sets the device to supervisor mode Performs the necessary initialization for the following hardware: CPU Memory controller System clocks Serial Caches Translation look-aside buffers (TLBs) In the case of a wakeup event, the Startup function reads the saved address for the wakeup event, and then jumps to that location. If it is not a wakeup event, the boot loader is copied to memory and run. The Startup code is typically located in the Startup.s file located in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader or %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader/Eboot directory. The Startup code is typically in assembly language because it is the first code that runs on the device and needs to do low-level hardware initialization and communication with the CPU. Because each CPU and hardware platform requires different initialization, you must modify the Startup.s file, depending on the CPU you are using. The silicon vendor provides the CPU initialization information. The Startup code can be shared with the OAL and can be reused later during that part of the development process. EbootMain() -> Main.c Depending on the CPU you are using for BSP development, EbootMain is typically the entry point for the C code to be run. EbootMain is typically located in the Main.c file in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader directory or in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader/Eboot directory. Although Main.c can perform any initializations on the device, it should end with a call to the BLCOMMON library entry point, BootloaderMain. BootloaderMain() -> Blcommon.c The BLCOMMON library entry point is the BootloaderMain function. BootloaderMain is the primary call in the BLCOMMON library that implements a common and consistent framework for a boot loader. The BLCOMMON library source file is located in the Blcommon.c file in the %_WINCEROOT%/Public/Common/Oak/Drivers/Ethdbg directory. The BLCOMMON library provides the following functionality: Relocates the boot loader to RAM for faster execution Decodes the .bin file contents Verifies checksums Keeps track of load progress The BLCOMMON library calls well-defined OEM functions throughout the process to handle hardware platform or solution-specific customizations. OEMDebugInit() -> Main.c OEMDebugInit is the first function called from the BootloaderMain function when the boot loader is started. This call is typically used to initialize serial UART debugging output. The OEM often chooses to call the OEMInitDebugSerial function to initialize a serial UART for debugging messages. After OEMDebugInit completes, the Windows CE banner appears indicating that this interface is available. The OEM function interface can use OEMWriteDebugString, OEMWriteDebugByte, and OEMReadDebugByte after OEMDebugInit completes. The OAL and the boot loader can share this implementation. OEMDebugInit is located in the Main.c file in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader OR %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader/Eboot directory. OEMPlatformInit() -> Main.c OEMPlatformInit is the next function called from BLCOMMON after the OEMDebugInit function completes and the Windows CE banner is displayed. OEMPlatformInit is an OEM hardware platform initialization function for the clock, PCI interface, or NIC interface. OEMPlatformInit is also where the interruption in the boot up process takes place to display the boot loader menu. The NIC interface for downloading the image is assumed for the following download functions. OEMPlatformInit is located in the Main.c file in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader or %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader/Eboot directory. OEMPreDownload() -> Main.c OEMPreDownload is also called from BLCOMMON before actually downloading a run-time image. This function retrieves the IP address for the device and connects to the development workstation. You can customize OEMPreDownload to prompt the user for feedback to obtain a static IP address or skip the download and jump to an image that might already exist on the device. The preferred method is to use the settings provided in Platform Builder. Return values for OEMPreDownload are typically obtained through a call to EbootInitEtherTransport, which retrieves this information from Platform Builder. You can select downloading or booting from a resident run-time image by options passed from Platform Builder to the boot loader through EbootInitEtherTransport. You can also set up OEMPreDownload to read hardware switch settings to determine the code that should be returned. The following tables shows the possible return codes for the OEMPreDownload function. Return code Description BL_DOWNLOAD (0) The Ethernet transport should be initialized by calling EbootInitEtherTransport. BL_JUMP (1) The download function in the BLCOMMON framework is bypassed. BL_ERROR (-1) Error condition. If a signed run-time image has been downloaded, the signature is checked and verified before calling OEMLaunch. OEMPreDownload is located in the Main.c file in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader or %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader/Eboot directory. OEMLaunch() -> Main.c OEMLaunch is the last boot loader function called out of BLCOMMON and is responsible for jumping to and launching the run-time image. OEMLaunch can use the EbootWaitForHostConnect function, which is defined in Eboot.lib, to wait for returned arguments when you have chosen passive KITL or not to connect to the target device at boot time. Other flags are set to define the transport mechanism as a serial, USB, or Ethernet connection. OEMLaunch jumps to the first instruction specified by the dwLaunchAddr parameter, which is the location of the run-time image Startup function. Startup calls the KernelStart function, as defined below. The launch address is obtained from the .bin file and is stored in the length field of the last block. At this point, the boot loader is done executing code and the Startup function in the OAL is called. OEMLaunch is located in the Main.c file in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader or %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Bootloader/Eboot directory. Kernel Startup Sequence The following table shows the startup functions used by the kernel for an ARM-based startup sequence. The startup sequence for the kernel requires calls into functions that are part of the shared-source license agreement that installs code into the private tree. Because of this you should sequence the functions as they are shown below. Within each function you might need to customize the order in which components are initialized depending on your CPU, memory and peripheral set. Kernel Startup Sequence Startup() KernelStart() ARMInit() OEMInitDebugSerial() OEMInit() KernelInit() HeapInit() InitMemoryPool() ProcInit() SchedInit() FirstSchedule() SystemStartupFunc() IOCTL_HAL_POSTINIT Functions in bold text denote functions that OEMs need to implement. ARM Kernel Startup Overview Startup() -> Startup.s The kernel Startup function is the first code that runs after the boot loader jumps to the run-time image that is now on the device, or when the development boot loader is removed and a production boot loader is flashed to the device and kernel Startup is called from the production boot loader. Although in this example kernel Startup is a different source file than the boot loader Startup function, they have similar code and can be shared. Startup is responsible for detecting a hardware or run-time reset or detecting when the system wakes up. The kernel Startup function performs the necessary hardware initialization, including initializing the CPU, memory controller, system clocks, serial port, and caches, and detecting when the system wakes up. Kernel Startup code is typically located in the Startup.s file in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Kernel/OAL directory. This directory can also share the boot loader Startup.s code. The Startup code is typically in assembly language because this is the first code that runs on the device and needs to perform low-level hardware initialization. Because each CPU and hardware platform requires different initialization, you must modify the file containing the kernel Startup function depending on the CPU you are using. The silicon vendor provides this CPU initialization information. Kernel Startup is responsible for computing the physical address for the OEMAddressTable and loading it into memory for use by the KernelStart function. The Startup function sets up any hardware platform or CPU-specific configurations that are required for the kernel to access ROM and DRAM, and then jumps to KernelStartup. The Startup routine should initialize any processor-specific cache or memory management unit (MMU), but not enable them KernelStart() -> Armtrap.s (Private) The KernelStart function is the main entry point for the kernel. The KernelStart function initializes the first-level page table based upon the contents of the MemoryMap array and enables the MMU and caches. Kernel symbolic addresses are not valid and must be translated through the MemoryMap array to find the correct physical address until the MMU is enabled. KernelStart also initializes the stacks for each mode of operation, and initializes global data for the kernel. The KernelStart function is located in the Armtrap.s file in the %_WINCEROOT%/Private/Winceos/Coreos/Nk/Kernel/ARM directory. ARMInit() -> Mdarm.s (Private) The ArmInit function, which is the main ARM initialization function, is responsible for calling the OEM-supplied initialization functions. ARMInit makes the following function calls: Calls OEMInitDebugSerial to initialize the debug serial port through OEMInitDebugSerial Uses OEMWriteDebugString to display the Windows CE banner Calls OEMInit to perform hardware platform initialization The ARMInit function is located in the Mdarm.c file in the %_WINCEROOT%/Private/Winceos/Coreos/Nk/Kernel/ARM directory. OEMInitDebugSerial() -> Mdarm.s (Private) The OEMInitDebugSerial function initializes the debug serial port on the device. The debugging functions are: OEMDebugInit, OEMWriteDebugString, OEMWriteDebugByte, and OEMReadDebugByte. The code for OEMInitDebugSerial is very similar to the OEMDebugInit function called from the boot loader. Therefore, most of the OEMDebugInit code can be reused or shared. OEMInit() -> Init.c The kernel calls OEMInit after it has performed minimal initialization in ARMInit. When OEMInit is called, interrupts are disabled and the kernel is unable to handle exceptions. OEMInit initializes the following hardware: Cache Interrupts Clock All other board-level hardware peripherals needed to support the hardware platform Note If a KITL connection is being used, then the KITL connection can also be initialized during OEMInit through a call to the OALKitlStart function. The OEMInit function contains much of the functionality present in the OEMPlatformInit function in the boot loader. The OEMInit function is a kernel call from ARMInit and can be found in the Init.c file in the %_WINCEROOT%/Platform/<Hardware Platform Name>/Src/Kernel/OAL directory. KernelInit() -> Kwin32.c (Private) The KernelInit function is where the kernel is initialized. KernelInit calls functions to initialize the following kernel components: Heap Memory pool Kernel process Scheduler When KernelInit returns, the kernel is ready to schedule the first thread through SystemStatupFunc. The KernelInit function can be found in the Kwin32.c file in the %_WINCEROOT%/Private/Winceos/Coreos/Nk/Kernel directory. HeapInit() -> Heap.c (Private) The HeapInit function initializes the kernel heap and is located in the Heap.c file in the %_WINCEROOT%/Private/Winceos/Coreos/Nk/Kernel directory. InitMemoryPool() -> Physmem.c (Private) The InitMemoryPool function initializes the kernel memory pool and is located in the Physmem.c file in the %_WINCEROOT%/Private/Winceos/Coreos/Nk/Kernel directory. ProcInit() -> Schedule.c (Private) The ProcInit function initializes the kernel process and is located in the Schedule.c file in the %_WINCEROOT/Private/Winceos/Coreos/Nk/Kernel directory. SchedInit() -> Schedule.c (Private) The SchedInit function initializes the scheduler and is located in the Schedule.c file in the %_WINCEROOT/Private/Winceos/Coreos/Nk/Kernel directory. SchedInit creates the SystemStatupFunc thread. FirstSchedule() -> Schedule.c (Private) The FirstSchedule function starts the scheduler and is located in the Schedule.c file in the %_WINCEROOT/Private/Winceos/Coreos/Nk/Kernel directory. SystemStartupFunc() -> Schedule.c (Private) The SystemStartupFunc function is called after all the required initialization has been completed and when the system is ready to schedule startup and run kernel threads. SystemStartupFunc is where calls are made to start kernel monitoring through CreateKernelThread, and execute IOCTL_HAL_POSTINIT through the OEMIoControl function. SystemStartupFunc is located in the Schedule.c file in the %_WINCEROOT/Private/Winceos/Coreos/Nk/Kernel directory. Conclusion All Windows CE device development begins by creating a targeted BSP for the device. The BSP also becomes the foundation for applications that run on the device by exporting an SDK from the BSP. Because of this, you must be sure that the high level BSP process is followed and the low-level function calls for the boot loader and kernel are implemented appropriately. With the information from this article, you should have a fundamental understanding of the BSP process and what the development effort is to get a boot loader and kernel running on a device.
-
发表了文章 2011-05-06
OverLapped I/O Socket 的问题
OverLapped I/O Socket 的问题，请教了！-Delphi知识大全 wsasend 异步投递一个发送请求，为了简单lpBuffers 参数只用了一个wsabuf结构，如果一次投递一个50M左右的请求可以成功，但是当我投递一个200M的请求时返回WSAENOBUFS（10055）错误，大概意思是“windows socket提供者报告一个缓冲区死锁”我想应该是请求太大了，我的问题是：1、我是应该把这个大块数据分解，然后投递若干个 wsasend请求呢还是将数据分解到若干个wsabuf数组元素中一次投递？主要还是对lpBuffers这个参数的意义没理解。：（2、不管是那种方法，数据分解成多大合适？最大值是多少？什么地方有相关的解释？ 1:呵呵，都会有这个过程的。可以多次提交多个异步请求，大小自己测一下就知道了，并不是BUF越大速度越快。根据网络情况来测一下吧。2:谁能给解释一下“windows socket提供者报告一个缓冲区死锁”是什么意思，什么原因引起的？我感觉异步i/o好象不应该限制请求的大小，如果有限制的话应该在相关文档中有说明啊感兴趣的朋友帮忙顶一下，谢谢！3:啊，迷糊：怎么我这里 10055 错误的说明是：“系统缓冲区空间不足或队列已满，不能执行套接字上的操作。”呀，有什么区别不？4:我也搞不清楚，我是在wsasend出错后getlasterror返回10055，就是WSAENOBUFS，对于WSAENOBUFS 错误msdn中的解释是这样的WSAENOBUFS The Windows Sockets provider reports a buffer deadlock.5:啊，那你就按我上面的解释处理吧，是同一个错误的不同回答，相对来讲，我这个更明确些。6:异步发送应该是把数据交给系统内核去发送，如果系统缓冲区空间不足或队列已满，那么内核应该等待接受方接受数据腾出系统缓冲区继续发送啊，怎么会返回错误完事呢？是不是我的理解错误，这个概念比较迷糊7:缓冲区死锁在网络中可能形成死锁状态的资源是缓冲区。 • 当一个方向传输的分组占用了太多的缓冲资源时必然影响其他方向的分组有序流动，最终造成死锁。 三种死锁形式： • 最简单的一种死锁是直接存储—转发死锁。解决的方法是：如果不允许结点中的缓冲区全部分配给一个传输方向，或者对每一 3 传输方向都分配固定大小的缓冲区，这种死锁就不会发生。 • 另外一种死锁是间接存储—转发死锁。解决方法：采用结构化的缓冲池技术可防止发生这种死锁。在拥挤的民政部下，“低级的”分组被丢弃，网络尽量把“高级的”分组送往它们的目的地。 • 最后的一种死锁是重装配死锁。这种死锁在 ARPANET 这样的数据报网络中最容易出现。 ARPANET 采用的缓冲区管理方法称为最小分配是最大限制的共享分配法。8:(10055) No buffer space available. An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full.9:asf.sys会把你的缓冲区锁定为未分页内存，未分页的理论最大值就是你的物理内存最大值10:有意思，收藏！11:to 元元她哥：“asf.sys会把你的缓冲区锁定为未分页内存”这句话不知怎么来理解。我的理解因为有so_sndbuf 这个值的限制，asf.sys不应该把我的缓冲区全部缓冲到非分页内存中啊。另外说明一下，我投递的缓冲区是一个映像文件，并且接受端接收到数据之后需要一个确认，不知道这些因素是否有影响。大虾们，是时候出手了！！！12:为了提高高带宽、高延迟网络的性能，Windows 2000 TCP 支持RFC 1323 中定义的TCP 窗口缩放。通过在TCP 三次握手期间商定一个窗口缩放因子，支持T C P 接收窗口的大小可大于64KB 。它支持的接收窗口最大可达1GB 。窗口缩放因子只在三次握手的前两个段中出现。缩放因子是2的s次方，其中s 为商定缩放因子。例如，对缩放因子为3 的通告窗口大小65535，实际接收窗口大小为524280 即8×65535 。 T C P 窗口缩放在缺省时是启用的，并且只要连接的TCP窗口大小值设定为大于6 4 K B 就自动使用 慢启动算法Windows 2000 TCP 支持慢启动和拥塞避免算法。一个连接建立后，T C P 起先只缓慢地发送数据来估计连接的带宽，以避免淹没接收主机或通路上的其他设备和链路。发送窗口大小设为2 个T C P 段，当两个段都被应答后，窗口大小就扩大为三个段。当三个段都被应答后，发送窗口大小再次扩大。如此进行，直到每次突发传输的数据量达到远程主机所声明的接收窗口大小。此时，慢启动算法就不再用了，改用声明的接收窗口进行流控制。 以上是在《Windows 2000 Server资源大全第3卷TCP/IP连网核心技术》摘下来的。 可以用网络监视器监听一下你的通信里TCP SYN 段中的窗口缩放选项。看看它的发送窗口有多大。很可能是发送200M时，系统申请的窗口过大吧。13:补充一句：我不是大虾！ [^][^]14:to painboy:按照你提供的资料理解，窗口的大小是逐渐扩大的，那么出错也应该是在发送过程中发生，但是我这里的现象是wsasend 直接就返回错误了。另外滑窗的大小也应该和接收端的receive的速度有关系，但是我的接收方还没有执行receive，所以我觉得滑窗不可能太大。或者我对异步io流程的理解不对。15:相信以下内容就是你想要的 ：） Socket 体系结构 Winsock2.0规范支持多种协议以及相关的支持服务。这些用户模式服务支持可以基于其他现存服务提供者来扩展他们自己的功能。比如，一个代理层服务支持（LSP）可以把自己安装在现存的TCP/IP服务顶层。这样，代理服务就可以截取和重定向一个对底层功能的调用。 与其他操作系统不同的是，WinNT和Win2000的传输协议层并不直接给应用程序提供socket风格的接口，不接受应用程序的直接访问。而是实现了更多的通用API，称为传输驱动接口(Transport Driver Interface,TDI).这些API把WinNT的子系统从各种各样的网络编程接口中分离出来。然后，通过Winsock内核模式驱动提供了sockets方法（在AFD.SYS里实现）。这个驱动负责连接和缓冲管理，对应用程序提供socket风格的编程接口。AFD.SYS则通过TDI和传输协议驱动层交流数据。 缓冲区由谁来管理 如上所说，对于使用socket接口和传输协议层交流的应用程序来说，AFD.SYS负责缓冲区的管理。也就是说，当一个程序调用send或WSASend函数发送数据的时候，数据被复制到AFD.SYS的内部缓冲里（大小根据SO_SNDBUF设置），然后send和WSASend立刻返回。之后数据由AFD.SYS负责发送到网络上，与应用程序无关。当然，如果应用程序希望发送比SO_SNDBUF设置的缓冲区还大的数据，WSASend函数将会被堵塞，直到所有数据均被发送完毕为止。 同样，当从远地客户端接受数据的时候，如果应用程序没有提交receive请求，而且线上数据没有超出SO_RCVBUF设置的缓冲大小，那么AFD.SYS就把网络上的数据复制到自己的内部缓冲保存。当应用程序调用recv或WSARecv函数的时候，数据即从AFD.SYS的缓冲复制到应用程序提供的缓冲区里。 在大多数情况下，这个体系工作的很好。尤其是应用程序使用一般的发送接受例程不牵涉使用Overlapped的时候。开发人员可以通过使用setsockopt API函数把SO_SNDBUF和SO_RCVBUF这两个设置的值改为0关闭AFD.SYS的内部缓冲。但是，这样做会带来一些后果： 比如，应用程序把SO_SNDBUF设为0,关闭了发送缓冲（指AFD.SYS里的缓冲），并发出一个同步堵塞式的发送操作，应用程序提供的数据缓冲区就会被内核锁定，send函数不会返回，直到连接的另一端收到整个缓冲区的数据为止。这貌似一种挺不错的方法，用来判断是否你的数据已经被对方全部收取。但实际上，这是很糟糕的。问题在于：网络层即使收到远端TCP的确认，也不能保证数据会被安全交到客户端应用程序那里，因为客户端可能发生“资源不足”等情况，而导致应用程序无法从AFD.SYS的内部缓冲复制得到数据。而更重大的问题是：由于堵塞，程序在一个线程里只能进行一次send操作，非常的没有效率。 如果关闭接受缓冲（设置SO_RCVBUF的值为0），也不能真正的提高效率。接受缓冲为0迫使接受的数据在比winsock内核层更底层的地方被缓冲，同样在调用recv的时候进行才进行缓冲复制，这样你关闭AFD缓冲的根本意图（避免缓冲复制）就落空了。关闭接收缓冲是没有必要的，只要应用程序经常有意识的在一个连接上调用重叠WSARecvs操作，这样就避免了AFD老是要缓冲大量的到来数据。 到这里，我们应该清楚关闭缓冲的方法对绝大多数应用程序来说没有太多好处的了。 然而，一个高性能的服务程序可以关闭发送缓冲，而不影响性能。这样的程序必须确保它在同时执行多个Overlapped发送，而不是等待一个Overlapped发送结束之后，才执行另一个。这样如果一个数据缓冲区数据已经被提交，那么传输层就可以立刻使用该数据缓冲区。如果程序“串行”的执行Overlapped发送，就会浪费一个发送提交之后另一个发送执行之前那段时间。16:资源约束 鲁棒性是每一个服务程序的一个主要设计目标。就是说，服务程序应该可以对付任何的突发问题，比如，客户端请求的高峰，可用内存的暂时贫缺，以及其他可靠性问题。为了平和的解决这些问题，开发人员必须了解典型的WindowsNT和Windows2000平台上的资源约束。 最基本的问题是网络带宽。使用UDP协议进行发送的服务程序对此要求较高，因为这样的服务程序要求尽量少的丢包率。即使是使用TCP连接，服务器也必须注意不要滥用网络资源。否则，TCP连接中将会出现大量重发和连接取消事件。具体的带宽控制是跟具体程序相关的，超出了本文的讨论范围。 程序所使用的虚拟内存也必须小心。应该保守的执行内存申请和释放，或许可以使用旁视列表（一个记录申请并使用过的“空闲”内存的缓冲区）来重用已经申请但是被程序使用过，空闲了的内存，这样可以使服务程序避免过多的反复申请内存，并且保证系统中一直有尽可能多的空余内存。（应用程序还可以使用SetWorkingSetSize这个Win32API函数来向系统请求增加该程序可用的物理内存。） 有两个winsock程序不会直接面对的资源约束。第一个是页面锁定限制。无论应用程序发起send还是receive操作，也不管AFD.SYS的缓冲是否被禁止，数据所在的缓冲都会被锁定在物理内存里。因为内核驱动要访问该内存的数据，在访问期间该内存区域都不能被解锁。在大部分情况下，这不会产生任何问题。但是操作系统必须确认还有可用的可分页内存来提供给其他程序。这样做的目的是防止一个有错误操作的程序请求锁定所有的物理RAM，而导致系统崩溃。这意味着，应用程序必须有意识的避免导致过多页面锁定，使该数量达到或超过系统限制。 在WinNT和Win2000中，系统允许的总共的内存锁定的限制大概是物理内存的1/8。这只是粗略的估计，不能作为一个准确的计算数据。只是需要知道，有时重叠IO操作会发生ERROR_INSUFFICIENT_RESOURCE失败，这是因为可能同时有太多的send/receives操作在进行中。程序应该注意避免这种情况。 另一个的资源限制情况是，程序运行时，系统达到非分页内存池的限制。WinNT和Win2000的驱动从指定的非分页内存池中申请内存。这个区域里分配的内存不会被扇出，因为它包含了多个不同的内核对象可能需要访问的数据，而有些内核对象是不能访问已经扇出的内存的。一旦系统创建了一个socket （或打开一个文件），一定数目的非分页内存就被分配了。另外，绑定(binding)和连接socket也会导致额外的非分页内存池的分配。更进一步的说，一个I/O请求，比如send或receive,也是分配了很少的一点非分页内存池的（为了跟踪I/O操作的进行，包含必须信息的一个很小的结构体被分配了）。积少成多，最后还是可能导致问题。因此操作系统限制了非分页内存的数量。在winNT和win2000平台上，每个连接分配的非分页内存的准确数量是不相同的，在未来的windows版本上也可能保持差异。如果你想延长你的程序的寿命，就不要打算在你的程序中精确的计算和控制你的非分页内存的数量。 虽然不能准确计算，但是程序在策略上要注意避免冲击非分页限制。当系统的非分页池内存枯竭，一个跟你的程序完全无关的的驱动都有可能出问题，因为它无法正常的申请到非分页内存。最坏的情况下，会导致整个系统崩溃。比如那些第三方设备或系统本身的驱动。切记：在同一台计算机上，可能还有其他的服务程序在运行，同样在消耗非分页内存。开发人员应该用最保守的策略估算资源，并基于此策略开发程序。 资源约束的解决方案是很复杂的，因为事实上，当资源不足的情况发生时，可能不会有特定的错误代码返回到程序。程序在调用函数时可能可以得到类似WSAENOBUFS或 ERROR_INSUFFICIENT_RESOURCES的这种一般的返回代码。如何处理这些错误呢，首先，合理的增加程序的工作环境设置（Working set，如果想获得更多信息，请参考MSDN里John Robbins关于 Bugslayer的一章）。如果仍然不能解决问题，那么你可能遇上了非分页内存池限制。那么最好是立刻关闭部分连接，并期待情况恢复正常。17:谢谢 painboy,你贴得资料我仔细看了一下，很有帮助。首先声明，我的程序中没有关闭发送缓冲，所以不可能是afd.sys的问题。我现在觉得问题可能使这段描述所说的情况“无论应用程序发起send还是receive操作，也不管AFD.SYS的缓冲是否被禁止，数据所在的缓冲都会被锁定在物理内存里”，但是对这句话的意思还是有些不理解： 当我wsasend时投递的缓冲区会全部被锁定在物理内存中吗？如果是，那我的错误就很好理解了，但是adf.sys的so_sndbound限制是什么？tcp的滑窗又限制的什么？大家在异步发送大的数据块时一般怎么处理？分成小块么？wsasend/wsarecv的lpBuffers这个参数指向一个wsabuf数组，这个数组怎么用？18:Scatter, Gather and MSG_PARTIAL The multiple buffer (WSABUF) input arguments for WSARecv()/WSARecvFrom() and WSASend()/WSASendto() provide support for scatter and gather operations (similar to those in the readv() and writev() functions in BSD Unix). The MSG_PARTIAL flag might also do this, but the specification isn't entirely clear what the intended use of the flag is, and current implementations don't support it (as described below). These operations are designed for datagram sockets that preserve message boundaries, not for stream sockets that do not (so may not fill buffers), though they do seem to work with stream sockets. The advantage that the gather operation provides is that it can assemble a single outgoing datagram from multiple buffers--e.g. put together different fields--and the scatter operation can "parse" fixed field lengths in an incoming datagram into multiple buffers. WSARecv()/WSARecvFrom(): Works fine -- scatters to input buffers on both Win95 and NT4 with datagram sockets. Stream sockets also work on both Win95 and NT4 SP3 in the testing I've done (which I would not recommend, since with a TCP byte stream the spec doesn't indicate that each buffer must be filled to the specified size on scatters, so behavior may be unpredictable under some circumstances with stream sockets). WSASend()/WSASendTo(): Works fine -- gathers from output buffers on both Win95 if you use datagram sockets. It also works with datagram sockets on NT4 SP3, although it failed with 10040 - WSAEMSGSIZE, if the message was larger than the MTU, so required IP fragmentation (e.g. greater than 1472 on Ethernet). This also works with stream sockets, but a similar warning as given for scatters goes for gathers as well (there's no guarantee that all bytes will be sent) 以下是 WSASend在UNIX下的实现： sockapi int __stdcall WSASend(SOCKET s,LPWSABUF lpBuffers,DWORD dwBufferCount,LPDWORD lpNumberOfBytesSent,DWORD dwFlags,LPWSAOVERLAPPED lpOverlapped,LPWSAOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine){int rc; TRACE("WSASend"); if (lpOverlapped != NULL) panic("Overlapped I/O not implemented in WSASend");if (lpCompletionRoutine != NULL) panic("Completion routines not implemented in WSASend"); rc = writev(s, lpBuffers, dwBufferCount);if (rc < 0) return -1; if (lpNumberOfBytesSent) *lpNumberOfBytesSent = rc;return 0;} WRITEV的解释：NAMEreadv, writev - read or write data into multiple buffers SYNOPSIS#include int readv(int filedes, const struct iovec *vector,size_t count); int writev(int filedes, const struct iovec *vector,size_t count); DESCRIPTIONThe readv() function reads count blocks from the fileassociated with the file descriptor filedes into the mul-tiple buffers described by vector. The writev() function writes at most count blocksdescribed by vector to the file associated with the filedescriptor vector. The pointer vector points to a struct iovec defined inas struct iovect{void *iovbase; /* Starting address */size_t iov_len; /* Number of bytes */} ; Buffers are processed in the order vector[0], vector[1],... vector[count]. The readv() function works just like read(2) except thatmultiple buffers are filled. The writev() function works just like write(2) except thatmultiple buffers are written out. 看完了以上说明后，相信你会将WSASEND里面的dwBufferCount设为1吧！：）19:是不是应该这样理解so_sndBuf的限制： 用WSASend发送一个N大小的内容，当N<=so_sndBuf时，ADF.SYS就直接将要发的内容复制到其缓冲区内，函数立即返回。当N>so_sndBuf时，这时系统会堵塞：先锁定要发送的N空间，然后每次从N中读so_sndBuf大小的内容去发送，直到发送完毕函数才返回。 但由于当N=200M时，太“小”了，系统无法把它装进内存并锁定，导致出错。 ……（这里省去6000字） ^_^看过好多例程，缓冲区大小取4至6K 。 最后一个问题……没用过
-
发表了文章 2011-03-15
多线程程序操作共享区域（文件）的一点体会
最近比较忙，很久没有写博客了，持续长时间的编程，使得我完全沦为程序匠人。但是感觉却不是想别人那么糟糕，毕业已经快两年了，我为我的编程兴趣仍然如此强烈而感到欣慰，也对一直以来比较关心的“行业应用软件架构设计”有了更深的了解，这坚定了我的信念！ 今天晚上，终于有了一点点闲暇的时间，就想大家分享下“多线程程序操作共享区域（文件）”的一些体会吧！ 多线程相信大家都陌生吧，多线程程序操作共享区域应该也不陌生吧，但是大家是否经历过多CPU的服务器下同时100个线程，操作离散的文件呢？如果每个线程只负责一个固定的文件，那么问题也就不是问题了，但是如果离散的文件，有可能同时被多线程都读写的，那么是否能保证文件读写的数据一致性、是否能保证数据在存储的时候由于非次序存储而导致数据丢失呢？另外，又怎么解决同一段时间同一个线程操作同一个文件的效率问题呢，如果有方法解决，有怎么保证不出现上述第一个问题呢？ 摆在面前的二个问题，其实都是编写程序时候，尤其是性能要求比较高的时候，特别需要注意的问题。如下将一一介绍我的一些体会； 如何解决多线程多文件操作的数据一致和丢失问题 解决这个问题，我们要先思考一下，究竟是什么原因导致这个问题呢？答案很明显，主要因为多线程多文件存储时候的时候频繁打开和频繁关闭无次序性导致的，这有点想数据库一样，DBMS是怎么来解决这个问题呢？DBMS有两个方法，一是事务、二是双端锁，其实这两个方法是一个样的道理，在这里我们就不做介绍，有兴趣大家可以查寻些其他的资料。这里我介绍实际应用中我的方法。 我的方法是，保证一个文件在操作的时候，只打开一个实例，打开之后只有一次关闭。 首先来看看，我是如何来设计这样的文件处理单元，我们起名为CDataHandlerUint，大家当作其为结构体就可以了，如下代码； typedef struct _DATA_HANDLER_UINT{queue<bool> m_OpenStatusQ; TiXmlDocument m_doc;CString m_strXmlPath;DWORD m_dwOldTicks;}CDataHandlerUint,*PCDataHandlerUint; 由于我们操作XML文件，因此在这个结构体中，我们保存了一个打开的XML文件实例——m_doc。另外一个重要的成员是m_OpenStatusQ这个暂时命名为XML文件打开请求队列，用于记录文件读写次数。其他成员随后会逐一介绍。 如下来看看，这个结构体在线程内部有是怎么用的，我们还需要一个关键的管理这个结构体的成员，CMap<CString,LPCSTR,PCDataHandlerUint,PCDataHandlerUint> m_mapHandlerUint,m_mapHandlerUint就用来管理这个结构体的。其中m_strXmlPath就是这个MAP的KEY，用来直接搜索出CDataHandlerUint的数据处理单元的pointer。 那么在线程中又怎么来应用这样的一个成员呢？看下如下这个线程的处理过程就明白了； UINT32 DataHandlerThread(LPVOID pThis){ (1) 获取要操作的文件路径 (2) 通过文件路径，获取保存在m_mapHandlerUint的Pointer. (3) 如果这个Pointer为NULL,重新NEW，调用m_OpenStatusQ.push(true)，之后并添加到MAP中。 （4）如果这个Pointer不为NULL，调用m_OpenStatusQ.push(true)。 ............ (5) 调用m_OpenStatusQ.pop()，之后检测m_OpenStatusQ的size ,如果size为0，保存并关闭XML文件，否则，在根据m_dwOldTicks判断是否超时，如果超时同样保存并关闭XML文件，否则结束函数 } 这样就应用了MAP加上queue顺利的解决了这个问题，大家看了之后，仔细想想吧！ 如果线程操作文件时，如果文件存在一定顺序，那么怎么提高效率呢？ 如果一个线程序遇到一个这样的文件系列怎么办，如下文件系列； A.xml A.xml B.xml B.xml C.xml ...... 从这里可以看出，文件是有顺序的，那么如何提高效率呢？ 答案就是，尽量减少文件打开和关闭次数，重复利用已经打开的文件句柄！ 这里程序设计的方法其实很简单，只需获取上一次操作文件句柄，并在上次操作的时候不关闭文件，关闭文件的操作放在调用之外，这样当发现两个句柄一样的时候，即调用保存并关闭，当然这里考虑到第一个问题，因此保存并关闭的条件仍然是调用m_OpenStatusQ.pop()，之后检测m_OpenStatusQ的size ,如果size为0，保存并关闭XML文件，否则，在根据m_dwOldTicks判断是否超时，如果超时同样保存并关闭XML文件，否则结束函数。 上述的两个问题，两个方法，只介于本人的体会，肯定还有更好的方法，如大家有兴趣可以联系我本人，方便更深一步的探讨。
-
发表了文章 2011-02-16
C/C++预处理指令#pragma详解
在所有的预处理指令中，#Pragma指令可能是最复杂的了，它的作用是设定编译器的状态或者是指示编译器完成一些特定的动作。#pragma指令对每个编译器给出了一个方法,在保持与C和C ++语言完全兼容的情况下,给出主机或操作系统专有的特征。依据定义,编译指示是机器或操作系统专有的,且对于每个编译器都是不同的。其格式一般为: #Pragma Para其中Para 为参数，下面来看一些常用的参数。 (1)message 参数。 Message 参数是我最喜欢的一个参数，它能够在编译信息输出窗口中输出相应的信息，这对于源代码信息的 控制是非常重要的。其使用方法为：#Pragma message(“消息文本”) 当编译器遇到这条指令时就在编译输出窗口中将消息文本打印出来。当我们在程序中定义了许多宏来控制源代码版本的时候，我们自己有可能都会忘记有没有正确的设置这些宏，此时我们可以用这条指令在编译的时候就进行检查。假设我们希望判断自己有没有在源代码的什么地方定义了_X86这个宏可以用下面的方法#ifdef _X86#Pragma message(“_X86 macro activated!”)#endif当我们定义了_X86这个宏以后，应用程序在编译时就会在编译输出窗口里显示“_X86 macro activated!”。我们就不会因为不记得自己定义的一些特定的宏而抓耳挠腮了。 (2)另一个使用得比较多的pragma参数是code_seg。格式如： #pragma code_seg( [/section-name/[,/section-class/] ] )它能够设置程序中函数代码存放的代码段，当我们开发驱动程序的时候就会使用到它。 (3)#pragma once (比较常用） 只要在头文件的最开始加入这条指令就能够保证头文件被编译一次，这条指令实际上在VC6中就已经有了，但是考虑到兼容性并没有太多的使用它。 (4)#pragma hdrstop表示预编译头文件到此为止，后面的头文件不进行预编译。BCB可以预编译头文件以加快链接的速度，但如 果所有头文件都进行预编译又可能占太多磁盘空间，所以使用这个选项排除一些头文件。有时单元之间有依赖关系，比如单元A依赖单元B，所以单元B要先于单元A编译。你可以用#pragmastartup指定编译优先级，如果使用了#pragma package(smart_init) ，BCB就会根据优先级的大小先后编译。 (5)#pragma resource /*.dfm/表示把*.dfm文件中的资源加入工程。*.dfm中包括窗体外观的定义。 (6)#pragma warning( disable : 4507 34; once : 4385; error : 164 )等价于：#pragma warning(disable:4507 34) // 不显示4507和34号警告信息#pragma warning(once:4385) // 4385号警告信息仅报告一次#pragma warning(error:164) // 把164号警告信息作为一个错误。同时这个pragma warning 也支持如下格式：#pragma warning( push [ ,n ] )#pragma warning( pop )这里n代表一个警告等级(1---4)。#pragma warning( push )保存所有警告信息的现有的警告状态。#pragma warning( push, n)保存所有警告信息的现有的警告状态，并且把全局警告等级设定为n。#pragma warning( pop )向栈中弹出最后一个警告信息，在入栈和出栈之间所作的一切改动取消。例如：#pragma warning( push )#pragma warning( disable : 4705 )#pragma warning( disable : 4706 )#pragma warning( disable : 4707 )//.......#pragma warning( pop )在这段代码的最后，重新保存所有的警告信息(包括4705，4706和4707)。 （7）pragma comment(...) 该指令将一个注释记录放入一个对象文件或可执行文件中。常用的lib关键字，可以帮我们连入一个库文件。 （8）·通过#pragma pack(n)改变C编译器的字节对齐方式 在C语言中，结构是一种复合数据类型，其构成元素既可以是基本数据类型（如int、long、float等）的变量，也可以是一些复合数据类型（如数组、结构、联合等）的数据单元。在结构中，编译器为结构的每个成员按其自然对界（alignment）条件分配空间。各个成员按照它们被声明的顺序在内存中顺序存储，第一个成员的地址和整个结构的地址相同。例如，下面的结构各成员空间分配情况：struct test{ char x1; short x2; float x3; char x4;}; 结构的第一个成员x1，其偏移地址为0，占据了第1个字节。第二个成员x2为short类型，其起始地址必须2字节对界，因此，编译器在x2和x1之间填充了一个空字节。结构的第三个成员x3和第四个成员x4恰好落在其自然对界地址上，在它们前面不需要额外的填充字节。在test结构中，成员x3要求4字节对界，是该结构所有成员中要求的最大对界单元，因而test结构的自然对界条件为4字节，编译器在成员x4后面填充了3个空字节。整个结构所占据空间为12字节。更改C编译器的缺省字节对齐方式。在缺省情况下，C编译器为每一个变量或是数据单元按其自然对界条件分配空间。一般地，可以通过下面的方法来改变缺省的对界条件： · 使用伪指令#pragma pack (n)，C编译器将按照n个字节对齐。 · 使用伪指令#pragma pack ()，取消自定义字节对齐方式。 另外，还有如下的一种方式： · __attribute((aligned(n)))，让所作用的结构成员对齐在n字节自然边界上。如果结构中有成员的长度大于n，则按照最大成员的长度来对齐。 · __attribute__ ((packed))，取消结构在编译过程中的优化对齐，按照实际占用字节数进行对齐。以上的n = 1, 2, 4, 8, 16... 第一种方式较为常见。应用实例 在网络协议编程中，经常会处理不同协议的数据报文。一种方法是通过指针偏移的方法来得到各种信息，但这样做不仅编程复杂，而且一旦协议有变化，程序修改起来也比较麻烦。在了解了编译器对结构空间的分配原则之后，我们完全可以利用这一特性定义自己的协议结构，通过访问结构的成员来获取各种信息。这样做，不仅简化了编程，而且即使协议发生变化，我们也只需修改协议结构的定义即可，其它程序无需修改，省时省力。下面以TCP协议首部为例，说明如何定义协议结构。其协议结构定义如下：#pragma pack(1) // 按照1字节方式进行对齐struct TCPHEADER{ short SrcPort; // 16位源端口号 short DstPort; // 16位目的端口号 int SerialNo; // 32位序列号 int AckNo; // 32位确认号 unsigned char HaderLen : 4; // 4位首部长度 unsigned char Reserved1 : 4; // 保留6位中的4位 unsigned char Reserved2 : 2; // 保留6位中的2位 unsigned char URG : 1; unsigned char ACK : 1; unsigned char PSH : 1; unsigned char RST : 1; unsigned char SYN : 1; unsigned char FIN : 1; short WindowSize; // 16位窗口大小 short TcpChkSum; // 16位TCP检验和 short UrgentPointer; // 16位紧急指针};#pragma pack() // 取消1字节对齐方式
-
发表了文章 2011-02-04
C Programming for Embedded System
Introduction Now for embedded system development people are using operating system to add more features and at the same time reduce the development time of a complex system. This article gives a simple & understandable overview of scheduling technique of embedded system programming for beginners and intermediate programmers. I am considering “C” as the programming language and an application running in a dedicated hardware/device without operating system. The binary output (*.bin file) can be directly running from the device after Power on. In this case, the time scheduling is an important part of system development. We should ensure that the right task should execute at the right time. What is Embedded System? An embedded system is some combination of computer hardware and software, either fixed in capability or programmable - it is specifically designed for a particular kind of application device. Or in short we can say, an embedded system is a special-purpose computer system designed to perform one or a few dedicated functions. All embedded systems need not be a real time system. Real Time systems are those in which timeliness is as important as the correctness of the outputs. Performance estimation and reduction are crucial in real time system. By definition, we can say a real time system is a system that must satisfy explicit (bounded) response time constraints or risk severe consequences, including failure. Embedded system plays an important part in our daily lives. Most of the people around the globe are highly dependent on different types of gadgets like mobile phones, iPods and many more. The embedded systems used in industrial machines, automobiles, medical equipment, airplanes, and vending machines have to be real time. I thought of sharing my experience in Embedded systems. This is my first ever article on CodeProject, so I am expecting your valuable feedback and suggestions for betterment. Background People are using operating system (RTOS) for complex devices to make it more flexible, add more features and minimize the development time. But it will increase the cost of the device for a small application. So for small application firmware development without operating system is very much popular. Time scheduling is an important aspect of real-time system. Real time software are executed in response to external events. This event may be periodic, in which case an appropriate scheduling of events and related task is required to guarantee performance. The scheduling strategy also depends on scheduling facilities that the chosen RTOS offers. This article emphasis on the event based technique when there was no operating system running in the device. Scheduling Two kinds of scheduling techniques are used in Real-Time system: Static Scheduling Dynamic Scheduling Static Scheduling This involves analyzing the tasks statically and determining their timing properties. This timing property can be used to create a fixed scheduling table, according to which tasks will be dispatched for execution at run time. Thus the order of execution of the task is fixed, and it is assumed that their execution time is also fixed. Round-Robin Scheduling Round Robin scheduling by Time Slicing is one of the ways to achieve static scheduling. Round robin is one of the simplest and most widely used scheduling algorithms; in which a small unit of time known as time slice is defined. Schedulers go around the queue of ready-to-run processes and allocate a time slice to each such process. Scheduling with Priority Priority indicates the urgency or importance assigned to a task. There are two approaches of scheduling based on priority based execution – when the processor is idle, the ready task with the highest priority is chosen for execution; once chosen, the task is run to completion. Pre-Emptive Scheduling Preemptive Priority based execution is when the processor is idle, the ready task with highest priority is chosen for execution; at any time, the execution of a task can be preempted if a task of higher priority becomes ready. Thus, at all times, the processor is idle or executing the ready task with the highest priority. Dynamic Scheduling Another kind of scheduling mechanism is known as Dynamic Scheduling – In this case, a real-time program requires a sequence of decisions to be taken during execution of the assignment of resource to transactions. Here each decision must be taken without prior knowledge of the needs of future tasks. Dynamic scheduling is not in the scope of this article, so I am not discussing it in detail here. Perhaps we can discuss it in another article. Code Snippet Let us take a example of a Master-Slave communication system. Master system is connected to n number of slave systems over serial port (RS 485 network) in multi-drop architecture. Figure 1 shows the typical configuration of this system. Here only one system can talk at a time and others are in listen mode. The Master controls the communication. Main Routine Collapse | Copy Code void main(void) { /* Initialise all register of processor and the peripheral devices */ InitMain(); /* Register the event handler */ RegisterTask(MainEventHandler); RegisterTask(CheckDataIntegrity); .............. /* Turn on all the leds for 1 sec as lamp test */ TurnOnLed(LED, 1000); /* Call the application event manager - no return */ EventManager(); } In the above case, the RegisterTast() and EventManager() are two important functions. For any application, we have number of tasks and a function represents the entry point of a task, like 'CheckDataIntegrity' . When a device receives a complete data packet, it goes for data checking. RegisterTask() function creates a link-list of function pointers where each node represents a single task. Here I have passed the function pointer MainEventHandler or CheckDataIntegrity as an argument. Main.h should have the following lines: Collapse | Copy Code /* Application event handler function pointer */ typedef void (*tEventHandler)(unsigned short *); /* Link-list definition */ typedef struct TaskRecord { tEventHandler EventHandler; struct TaskRecord *pNext; }tTaskRecord; static tTaskRecord *mpTaskList = NULL; Here mpTaskList represents a link-list of function pointers. Considering each node of link list as a entry point of a task, this will execute one by one in EventManager() function. Below is the definition of RegisterTask() function which adds the function pointer into the link-list. Collapse | Copy Code void RegisterTask(tEventHandler EventHandlerFunc) { tTaskRecord *pNewTask; /* Create a new task record */ pNewTask = malloc(sizeof(tTaskRecord)); if(pNewTask != NULL) { /* Assign the event handler function to the task */ pNewTask->EventHandler = EventHandlerFunc; pNewTask->pNext = NULL; if(mpTaskList == NULL) { /* Store the address of the first task in the task list */ mpTaskList = pNewTask; } else { /* Move to the last task in the list */ mpActiveTask = mpTaskList; while(mpActiveTask->pNext != NULL) { mpActiveTask = mpActiveTask->pNext; } /* Add the new task to the end of the list */ mpActiveTask->pNext = pNewTask; } } } For this type of application, after initialization there should be an infinite loop for continuous execution. The function EventManager() at the end of the main which is nothing but a infinite loop always checks for active tasks or events. If any event occurs, then it passes that event flag as an argument of the function which is already added into the mpTaskList. So EventManager() function calls MainEventHandler() function with eventID as an argument. MainEventHandler will check the eventId and do the necessary action or execute the corresponding code. Here the event should be unique for each event-handler function, i.e. two event-handler functions should not check the same eventID. Definition of EventManager Function Collapse | Copy Code void EventManager(void) { unsigned short AllEvents; tTaskRecord pActiveTask /* No return */ while(1) { /* Read application events */ AllEvents = mEventID; /* Process any application events */ pActiveTask = mpTaskList; while((AllEvents != 0) && (pActiveTask != NULL)) { if(pActiveTask->EventHandler != NULL) { /* Call the task's event handler function */ (mpActiveTask->EventHandler)(&AllEvents); /* Read application events */ AllEvents = mEventID; } /* Move to the next event handler */ pActiveTask = pActiveTask->pNext; } } } Event can be generated from interrupt service routine or by checking the status of an input pin in polling mode. SerialReceiveISR function generates an event after receiving the complete packet. Since the variable mEventID is modified in the interrupt service routine, it is recommended to disable interrupt while reading. Collapse | Copy Code #pragma interrupt_level 0 void interrupt IService(void) { /* Receive bit is set when a byte is received */ if(Receivebit == 1) { SerialReceiveISR() } ........... /* code for other interrupt */ } SerialReceiveISR Function Collapse | Copy Code #pragma inline SerialReceiveISR void SerialReceiveISR(void) { static char RxMsgDataCount; /* If a framing or overrun error occurs then clear the error */ if(Error == 1) { /* Indicate a problem was seen */ mEventID = mEventID | ATTENTION_REQ_FLG; } else if( RxMsgCount == DataLength) { /* Packet receive complete */ mEventID = mEventID | DATA_RECEIVE_COMPLETE; } else { /* Store data in memory */ Store(RxByte); RxMsgCount++; } } Here ATTENTION_REQ_FLAG & DATA_RECEIVE_COMPLETE flags are two bits of a Global variable mEventID, which is 16 bits and each bit triggers the corresponding event when set. Collapse | Copy Code #define ATTENTION_REQ_FLAG 0x0008 #define DATA_RECEIVE_COMPLETE 0x0001 When the flag is set, the EventManager will call all the registered functions with the eventID as argument. Collapse | Copy Code void MainEventHandler(unsigned short *Event) { if(*Event & DATA_RECEIVE_COMPLETE) { /* Do the corresponding action */ ......... /* Reset the flag */ *Event &= ~DATA_RECEIVE_COMPLETE; } } Now we can change the variable type to increase the number of flags. If you want to generate multiple number of events, then use a structure rather than a single variable. License This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL) About the Author
-
发表了文章 2011-01-06
Windows XP下USB转串口驱动编码实现分析
USB转COM驱动的编写实现有很多中方法,最近在网络上看到一个最常用的方式,即是虚拟一个COM口,在COM初试的时候进行打开一个USB设备. 当USB串COM口驱动处理Write和Read 等IRQ的时候,其实是去读写USB 设备驱动,读写方式是直接采用ZwReadFile和ZwWriteFile函数.但是真正的实现也不是这么简单,真实的实现是开辟一个线程和一段较大的缓冲区,线程用于适时读取USB设备的数据并保存在缓冲区当中.当上层应用程序向这个虚拟的COM口发送READ IRP(IRP_MJ_READ)的时候,则直接从这段缓冲区中读取数据返回即可.当然还不要忘记处理超时的问题处理. 因此USB转COM驱动在INF文件加载的时候实际上需要加载两个驱动程序,一个是USB驱动,对应与USB设备的VID和PID,另外一个就是COM虚拟驱动程序,需要对应设置COM口号.如下将对关键代码列出; 创建COM口设备对象 status = IoCreateDevice(DriverObject, sizeof(VCP4USB_DEVICE_EXTENSION), &ntDeviceName, FILE_DEVICE_SERIAL_PORT, 0, TRUE, // exclusive &fdo); 获取USB设备名称用于打开设备 status = IoGetDeviceInterfaces(pGuid, NULL, 0, &pSymLink); DPrint(DBG_OTHER, ("IoGetDeviceInterface return %d/n", status)); if ((status == STATUS_INVALID_DEVICE_REQUEST) || (*pSymLink == NULL)) return STATUS_UNSUCCESSFUL; pCur = pSymLink; instanceCur = 0; status = STATUS_INVALID_INFO_CLASS; while (*pCur != NULL) { p = pCur; for (size = 0; *p != NULL; size ++) p ++; DPrint(DBG_OTHER, ("No.%d: size=%d/n", instanceCur, size)); DPrint(DBG_OTHER, ("name:%ws/n", pCur)); if (instance == instanceCur) { if (RtlCompareMemory(pCur, pPrefix, prefixLength) == prefixLength) { DPrint(DBG_OTHER, ("Find OK/n")); devName->MaximumLength = size * sizeof(WCHAR); devName->Buffer = (PWSTR)ExAllocatePool(NonPagedPool, (size + 1) * sizeof(WCHAR)); if (devName->Buffer == NULL) { DPrint(DBG_OTHER, ("Allocate devName error./n")); status = STATUS_INSUFFICIENT_RESOURCES; break; } RtlCopyMemory(devName->Buffer, pCur, (size+1) * sizeof(WCHAR)); RtlInitUnicodeString(devName, devName->Buffer); status = STATUS_SUCCESS; break; // find ok and break } } pCur += size + 1; // skip last NULL instanceCur ++; if (instanceCur >= 2) // for debug break; } ExFreePool(pSymLink); return status; 打开USB设备端口 curStatus = GetUsbDeviceName(&usbDeviceName, &GUID_CLASS_XXXX_BULK, &XXXX_SYMLINK_CMPSTR, XXXX_SYMLINK_STRLEN, 0); if (!hRead) rdStatus = createFile(&hRead, &usbDeviceName, L"//PIPE0", 6, GENERIC_READ); if (!hWrite) wtStatus = createFile(&hWrite, &usbDeviceName, L"//PIPE1", 6, GENERIC_WRITE); 其他的驱动程序处理都是常规处理,在这里就不多做介绍!
-
发表了文章 2010-12-23
__HW_VTBL 结构体神奇!
typedef struct __HW_VTBL { PVOID (*HWInit)(ULONG Identifier, PVOID pMDDContext, PHWOBJ pHWObj); BOOL (*HWPostInit)(PVOID pHead); ULONG (*HWDeinit)(PVOID pHead); BOOL (*HWOpen)(PVOID pHead); ULONG (*HWClose)(PVOID pHead); INTERRUPT_TYPE (*HWGetIntrType)(PVOID pHead); ULONG (*HWRxIntrHandler)(PVOID pHead, PUCHAR pTarget, PULONG pBytes); VOID (*HWTxIntrHandler)(PVOID pHead, PUCHAR pSrc, PULONG pBytes); VOID (*HWModemIntrHandler)(PVOID pHead); VOID (*HWLineIntrHandler)(PVOID pHead); ULONG (*HWGetRxBufferSize)(PVOID pHead); BOOL (*HWPowerOff)(PVOID pHead); BOOL (*HWPowerOn)(PVOID pHead); VOID (*HWClearDTR)(PVOID pHead); VOID (*HWSetDTR)(PVOID pHead); VOID (*HWClearRTS)(PVOID pHead); VOID (*HWSetRTS)(PVOID pHead); BOOL (*HWEnableIR)(PVOID pHead, ULONG BaudRate); BOOL (*HWDisableIR)(PVOID pHead); VOID (*HWClearBreak)(PVOID pHead); VOID (*HWSetBreak)(PVOID pHead); BOOL (*HWXmitComChar)(PVOID pHead, UCHAR ComChar); ULONG (*HWGetStatus)(PVOID pHead, LPCOMSTAT lpStat); VOID (*HWReset)(PVOID pHead); VOID (*HWGetModemStatus)(PVOID pHead, PULONG pModemStatus); VOID (*HWGetCommProperties)(PVOID pHead, LPCOMMPROP pCommProp); VOID (*HWPurgeComm)(PVOID pHead, DWORD fdwAction); BOOL (*HWSetDCB)(PVOID pHead, LPDCB pDCB); BOOL (*HWSetCommTimeouts)(PVOID pHead, LPCOMMTIMEOUTS lpCommTO); BOOL (*HWIoctl)(PVOID pHead, DWORD dwCode,PBYTE pBufIn,DWORD dwLenIn, PBYTE pBufOut,DWORD dwLenOut,PDWORD pdwActualOut);} HW_VTBL, *PHW_VTBL;
-
发表了文章 2010-12-22
STL之二分查找 (Binary search in STL)
Section I 正确区分不同的查找算法count,find,binary_search,lower_bound,upper_bound,equal_range 本文是对Effective STL第45条的一个总结，阐述了各种查找算法的异同以及使用他们的时机。 首先可供查找的算法大致有count,find,binary_search,lower_bound,upper_bound,equal_range。带有判别式的如count_if,find_if或者binary_search的派别式版本，其用法大致相同，不影响选择，所以不作考虑。 注意这些查找算法需要序列式容器，或者数组。关联容器有相应的同名成员函数except binary_search。 首先,选择查找算法时，区间是否排序是一个至关重要的因素。 可以按是否需要排序区间分为两组: A. count,find B. binary_search,lower_bound,upper_bound,equal_range A组不需排序区间， B组需要排序区间。 当一个区间被排序，优先选择B组，因为他们提供对数时间的效率。而A则是线性时间。 另外A组B组所依赖的查找判断法则不同，A使用相等性法则(查找对象需要定义operator==), B使用等价性法则(查找对象需要定义operator<,必须在相等时返回false)。 A组的区别 count:计算对象区间中的数目。 find:返回第一个对象的位置。 查找成功的话，find会立即返回，count不会立即返回（直到查找完整个区间），此时find效率较高。 因此除非是要计算对象的数目，否则不考虑count。 B组的区别 {1，3，4，5，6} binary_search：判断是否存在某个对象 lower_bound: 返回>=对象的第一个位置，lower_bound(2)=3, lower_bound(3)=3 目标对象存在即为目标对象的位置,不存在则为后一个位置. upper_bound: 返回>对象的第一个位置, upper_bound(2)=3,upper_bound(3)=4 无论是否存在都为后一个位置. equal_bound: 返回由lower_bound和upper_bound返回值构成的pair,也就是所有等价元素区间。 equal_bound有两个需要注意的地方: 1. 如果返回的两个迭代器相同，说明查找区间为空，没有这样的值 2. 返回迭代器间的距离与迭代器中对象数目是相等的，对于排序区间，他完成了count和find的双重任务 Section II binary search in STL 如果在C++ STL容器中包含了有序的序列，STL提供了四个函数进行搜索，他们是利用二分查找实现的(Binary search). 其中： 假定相同值的元素可能有多个 lower_bound 返回第一个符合条件的元素位置 upper_bound 返回最后一个符合条件的元素位置 equal_range 返回所有等于指定值的头/尾元素的位置，其实就是lower_bound和upper_bound binary_search 返回是否有需要查找的元素。 Section II Effect STL #45 条款45：注意count、find、binary_search、lower_bound、upper_bound和equal_range的区别 你要寻找什么，而且你有一个容器或者你有一个由迭代器划分出来的区间——你要找的东西就在里面。你要怎么完成搜索呢？你箭袋中的箭有这些：count、count_if、find、find_if、binary_search、lower_bound、upper_bound和equal_range。面对着它们，你要怎么做出选择？ 简单。你寻找的是能又快又简单的东西。越快越简单的越好。 暂时，我假设你有一对指定了搜索区间的迭代器。然后，我会考虑到你有的是一个容器而不是一个区间的情况。 要选择搜索策略，必须依赖于你的迭代器是否定义了一个有序区间。如果是，你就可以通过binary_search、lower_bound、upper_bound和equal_range来加速（通常是对数时间——参见条款34）搜索。如果迭代器并没有划分一个有序区间，你就只能用线性时间的算法count、count_if、find和find_if。在下文中，我会忽略掉count和find是否有_if的不同，就像我会忽略掉binary_search、lower_bound、upper_bound和equal_range是否带有判断式的不同。你是依赖默认的搜索谓词还是指定一个自己的，对选择搜索算法的考虑是一样的。 如果你有一个无序区间，你的选择是count或着find。它们分别可以回答略微不同的问题，所以值得仔细去区分它们。count回答的问题是：“是否存在这个值，如果有，那么存在几份拷贝？”而find回答的问题是：“是否存在，如果有，那么它在哪儿？” 假设你想知道的东西是，是否有一个特定的Widget值w在list中。如果用count，代码看起来像这样： list<Widget> lw; // Widget的list Widget w; // 特定的Widget值 ... if (count(lw.begin(), lw.end(), w)) ...{ ... // w在lw中 } else ...{ ... // 不在 } 这里示范了一种惯用法：把count用来作为是否存在的检查。count返回零或者一个正数，所以我们把非零转化为true而把零转化为false。如果这样能使我们要做的更加显而易见： if (count(lw.begin(), lw.end(), w) != 0) ... 而且有些程序员这样写，但是使用隐式转换则更常见，就像最初的例子。 和最初的代码比较，使用find略微更难懂些，因为你必须检查find的返回值和list的end迭代器是否相等： if (find(lw.begin(), lw.end(), w) != lw.end()) { ... // 找到了 } else { ... // 没找到 } 如果是为了检查是否存在，count这个惯用法编码起来比较简单。但是，当搜索成功时，它的效率比较低，因为当找到匹配的值后find就停止了，而count必须继续搜索，直到区间的结尾以寻找其他匹配的值。对大多数程序员来说，find在效率上的优势足以证明略微增加复杂度是合适的。 通常，只知道区间内是否有某个值是不够的。取而代之的是，你想获得区间中的第一个等于该值的对象。比如，你可能想打印出这个对象，你可能想在它前面插入什么，或者你可能想要删除它（但当迭代时删除的引导参见条款9）。当你需要知道的不止是某个值是否存在，而且要知道哪个对象（或哪些对象）拥有该值，你就得用find： list<Widget>::iterator i = find(lw.begin(), lw.end(), w); if (i != lw.end()) { ... // 找到了，i指向第一个 } else { ... // 没有找到 } 对于有序区间，你有其他的选择，而且你应该明确的使用它们。count和find是线性时间的，但有序区间的搜索算法（binary_search、lower_bound、upper_bound和equal_range）是对数时间的。 从无序区间迁移到有序区间导致了另一个迁移：从使用相等来判断两个值是否相同到使用等价来判断。条款19由一个详细地讲述了相等和等价的区别，所以我在这里不会重复。取而代之的是，我会简单地说明count和find算法都用相等来搜索，而binary_search、lower_bound、upper_bound和equal_range则用等价。 要测试在有序区间中是否存在一个值，使用binary_search。不像标准C库中的（因此也是标准C++库中的）bsearch，binary_search只返回一个bool：这个值是否找到了。binary_search回答这个问题：“它在吗？”它的回答只能是是或者否。如果你需要比这样更多的信息，你需要一个不同的算法。 这里有一个binary_search应用于有序vector的例子（你可以从条款23中知道有序vector的优点）： vector<Widget> vw; // 建立vector，放入 ... // 数据， sort(vw.begin(), vw.end()); // 把数据排序 Widget w; // 要找的值 ... if (binary_search(vw.begin(), vw.end(), w)) ...{ ... // w在vw中 } else ...{ ... // 不在 } 如果你有一个有序区间而且你的问题是：“它在吗，如果是，那么在哪儿？”你就需要equal_range，但你可能想要用lower_bound。我会很快讨论equal_range，但首先，让我们看看怎么用lower_bound来在区间中定位某个值。 当你用lower_bound来寻找一个值的时候，它返回一个迭代器，这个迭代器指向这个值的第一个拷贝（如果找到的话）或者到可以插入这个值的位置（如果没找到）。因此lower_bound回答这个问题：“它在吗？如果是，第一个拷贝在哪里？如果不是，它将在哪里？”和find一样，你必须测试lower_bound的结果，来看看它是否指向你要寻找的值。但又不像find，你不能只是检测lower_bound的返回值是否等于end迭代器。取而代之的是，你必须检测lower_bound所标示出的对象是不是你需要的值。 很多程序员这么用lower_bound： vector<Widget>::iterator i = lower_bound(vw.begin(), vw.end(), w); if (i != vw.end() && *i == w) ...{ // 保证i指向一个对象； // 也就保证了这个对象有正确的值。 // 这是个bug！ ... // 找到这个值，i指向 // 第一个等于该值的对象 } else ...{ ... // 没找到 } 大部分情况下这是行得通的，但不是真的完全正确。再看一遍检测需要的值是否找到的代码： if (i != vw.end() && *i == w) ... 这是一个相等的测试，但lower_bound搜索用的是等价。大部分情况下，等价测试和相等测试产生的结果相同，但就像条款19论证的，相等和等价的结果不同的情况并不难见到。在这种情况下，上面的代码就是错的。 要完全完成，你就必须检测lower_bound返回的迭代器指向的对象的值是否和你要寻找的值等价。你可以手动完成（条款19演示了你该怎么做，当它值得一做时条款24提供了一个例子），但可以更狡猾地完成，因为你必须确认使用了和lower_bound使用的相同的比较函数。一般而言，那可以是一个任意的函数（或函数对象）。如果你传递一个比较函数给lower_bound，你必须确认和你的手写的等价检测代码使用了相同的比较函数。这意味着如果你改变了你传递给lower_bound的比较函数，你也得对你的等价检测部分作出修改。保持比较函数同步不是火箭发射，但却是另一个要记住的东西，而且我想你已经有很多需要你记的东西了。 这儿有一个简单的方法：使用equal_range。equal_range返回一对迭代器，第一个等于lower_bound返回的迭代器，第二个等于upper_bound返回的（也就是，等价于要搜索值区间的末迭代器的下一个）。因此，equal_range，返回了一对划分出了和你要搜索的值等价的区间的迭代器。一个名字很好的算法，不是吗？（当然，也许叫equivalent_range会更好，但叫equal_range也非常好。） 对于equal_range的返回值，有两个重要的地方。第一，如果这两个迭代器相同，就意味着对象的区间是空的；这个只没有找到。这个结果是用equal_range来回答“它在吗？”这个问题的答案。你可以这么用： vector<Widget> vw; ... sort(vw.begin(), vw.end()); typedef vector<Widget>::iterator VWIter; // 方便的typedef typedef pair<VWIter, VWIter> VWIterPair; VWIterPair p = equal_range(vw.begin(), vw.end(), w); if (p.first != p.second) ...{ // 如果equal_range不返回 // 空的区间... ... // 说明找到了，p.first指向 // 第一个而p.second // 指向最后一个的下一个 } else ...{ ... // 没找到，p.first和 // p.second都指向搜索值 } // 的插入位置 这段代码只用等价，所以总是正确的。 第二个要注意的是equal_range返回的东西是两个迭代器，对它们作distance就等于区间中对象的数目，也就是，等价于要寻找的值的对象。结果，equal_range不光完成了搜索有序区间的任务，而且完成了计数。比如说，要在vw中找到等价于w的Widget，然后打印出来有多少这样的Widget存在，你可以这么做： VWIterPair p = equal_range(vw.begin(), vw.end(), w); cout << "There are " << distance(p.first, p.second) << " elements in vw equivalent to w."; 到目前为止，我们所讨论的都是假设我们要在一个区间内搜索一个值，但是有时候我们更感兴趣于在区间中寻找一个位置。比如，假设我们有一个Timestamp类和一个Timestamp的vector，它按照老的timestamp放在前面的方法排序： class Timestamp { ... }; bool operator<(const Timestamp& lhs, // 返回在时间上lhs const Timestamp& rhs); // 是否在rhs前面 vector<Timestamp> vt; // 建立vector，填充数据， ... // 排序，使老的时间 sort(vt.begin(), vt.end()); // 在新的前面 现在假设我们有一个特殊的timestamp——ageLimit，而且我们从vt中删除所有比ageLimit老的timestamp。在这种情况下，我们不需要在vt中搜索和ageLimit等价的Timestamp，因为可能不存在任何等价于这个精确值的元素。 取而代之的是，我们需要在vt中找到一个位置：第一个不比ageLimit更老的元素。这是再简单不过的了，因为lower_bound会给我们答案的： Timestamp ageLimit; ... vt.erase(vt.begin(), lower_bound(vt.begin(), // 从vt中排除所有 vt.end(), // 排在ageLimit的值 ageLimit)); // 前面的对象 如果我们的需求稍微改变了一点，我们要排除所有至少和ageLimit一样老的timestamp，也就是我们需要找到第一个比ageLimit年轻的timestamp的位置。这是一个为upper_bound特制的任务： vt.erase(vt.begin(), upper_bound(vt.begin(), // 从vt中除去所有 vt.end(), // 排在ageLimit的值前面 ageLimit)); // 或者等价的对象 如果你要把东西插入一个有序区间，而且对象的插入位置是在有序的等价关系下它应该在的地方时，upper_bound也很有用。比如，你可能有一个有序的Person对象的list，对象按照name排序： class Person { public: ... const string& name() const; ... }; struct PersonNameLess: public binary_function<Person, Person, bool> { // 参见条款40 bool operator()(const Person& lhs, const Person& rhs) const { return lhs.name() < rhs.name(); } }; list<Person> lp; ... lp.sort(PersonNameLess()); // 使用PersonNameLess排序lp 要保持list仍然是我们希望的顺序（按照name，插入后等价的名字仍然按顺序排列），我们可以用upper_bound来指定插入位置： Person newPerson; ... lp.insert(upper_bound(lp.begin(), // 在lp中排在newPerson lp.end(), // 之前或者等价 newPerson, // 的最后一个 PersonNameLess()), // 对象后面 newPerson); // 插入newPerson 这工作的很好而且很方便，但很重要的是不要被误导——错误地认为upper_bound的这种用法让我们魔术般地在一个list里在对数时间内找到了插入位置。我们并没有——条款34解释了因为我们用了list，查找花费线性时间，但是它只用了对数次的比较。 一直到这里，我都只考虑我们有一对定义了搜索区间的迭代器的情况。通常我们有一个容器，而不是一个区间。在这种情况下，我们必须区别序列和关联容器。对于标准的序列容器（vector、string、deque和list），你应该遵循我在本条款提出的建议，使用容器的begin和end迭代器来划分出区间。 这种情况对标准关联容器（set、multiset、map和multimap）来说是不同的，因为它们提供了搜索的成员函数，它们往往是比用STL算法更好的选择。条款44详细说明了为什么它们是更好的选择，简要地说，是因为它们更快行为更自然。幸运的是，成员函数通常和相应的算法有同样的名字，所以前面的讨论推荐你使用的算法count、find、equal_range、lower_bound或upper_bound，在搜索关联容器时你都可以简单的用同名的成员函数来代替。 调用binary_search的策略不同，因为这个算法没有提供对应的成员函数。要测试在set或map中是否存在某个值，使用count的惯用方法来对成员进行检测： set<Widget> s; // 建立set，放入数据 ... Widget w; // w仍然是保存要搜索的值 ... if (s.count(w)) { ... // 存在和w等价的值 } else { ... // 不存在这样的值 } 要测试某个值在multiset或multimap中是否存在，find往往比count好，因为一旦找到等于期望值的单个对象，find就可以停下了，而count，在最遭的情况下，必须检测容器里的每一个对象。（对于set和map，这不是问题，因为set不允许重复的值，而map不允许重复的键。） 但是，count给关联容器计数是可靠的。特别，它比调用equal_range然后应用distance到结果迭代器更好。首先，它更清晰：count 意味着“计数”。第二，它更简单；不用建立一对迭代器然后把它的组成（译注：就是first和second）传给distance。第三，它可能更快一点。 要给出所有我们在本条款中所考虑到的，我们的从哪儿着手？下面的表格道出了一切。 你想知道的 在无序区间 在有序区间 在set或map上 在multiset或multimap上 期望值是否存在？ find binary_search count find 期望值是否存在？如果有，第一个等于这个值的对象在哪里？ find equal_range find find或lower_bound（参见下面） 第一个不在期望值之前的对象在哪里？ find_if lower_bound lower_bound lower_bound 第一个在期望值之后的对象在哪里？ find_if upper_bound upper_bound upper_bound 有多少对象等于期望值？ count equal_range，然后distance count count 等于期望值的所有对象在哪里？ find（迭代） equal_range equal_range equal_range 上表总结了要怎么操作有序区间，equal_range的出现频率可能令人吃惊。当搜索时，这个频率因为等价检测的重要性而上升了。对于lower_bound和upper_bound，它很容易在相等检测中退却，但对于equal_range，只检测等价是很自然的。在第二行有序区间，equal_range打败了find还因为一个理由：equal_range花费对数时间，而find花费线性时间。 对于multiset和multimap，当你在搜索第一个等于特定值的对象的那一行，这个表列出了find和lower_bound两个算法作为候选。 已对于这个任务find是通常的选择，而且你可能已经注意到在set和map那一列里，这项只有find。但是对于multi容器，如果不只有一个值存在，find并不保证能识别出容器里的等于给定值的第一个元素；它只识别这些元素中的一个。如果你真的需要找到等于给定值的第一个元素，你应该使用lower_bound，而且你必须手动的对第二部分做等价检测，条款19的内容可以帮你确认你已经找到了你要找的值。（你可以用equal_range来避免作手动等价检测，但是调用equal_range的花费比调用lower_bound多得多。） 在count、find、binary_search、lower_bound、upper_bound和equal_range中做出选择很简单。当你调用时，选择算法还是成员函数可以给你需要的行为和性能，而且是最少的工作。按照这个建议做（或参考那个表格），你就不会再有困惑。
-
发表了文章 2010-12-15
程序访问物理地址之MmMapIoSpace 与 OALPAtoVA
（1）MmMapIoSpace 用法 在程序中使用的都是虚拟地址，如果要对物理地址进行操作，需要用到MmMapIoSpace把物理地址映射到虚拟地址，如：pBaseAddress = (PUCHAR)MmMapIoSpace(ioPhysicalBase, Size, FALSE);访问pBaseAddress的指向地址，就是访问被映射后ioPhysicalBase定义的物理地址。 PVOID MmMapIoSpace ( PHYSICAL_ADDRESS PhysicalAddress, 参数1：需要映射的物理地址 ULONG NumberOfBytes, 参数2：映射的地址长度 BOOLEAN CacheEnable 参数3：是否使用cache（驱动中要使用uncached）); 在使用MmMapIoSpace后，必须使用MmUnmapIoSpace。 VOID MmUnmapIoSpace ( PVOID BaseAddress, 参数1：被映射后的虚拟地址 ULONG NumberOfBytes 参数2：映射的地址长度); MmMapIoSpace这个函数是在ceddk里面实现的，所以只要link了ceddk.lib就可以使用了。它的源码实现在：X：/WINCE600/PUBLIC/COMMON/OAK/DRIVERS/CEDDK/DDK_MAP，源码中是调用了virtualalloc来预留虚拟内存，然后virtualcopy来做映射的。这两个函数都不需要Kernel权限，所以没有Full Kernel也可以用。 （2）OALPAtoVA 用法 该函数也是将要访问的物理地址映射成虚拟地址。共有三个类似函数： OALPAtoUA：This function returns the uncached virtual address for a specified physical address.VOID* OALPAtoUA(UINT32 pa); OALPAtoCA：Returns the cached virtual address.VOID* OALPAtoCA(UINT32 pa); OALPAtoVA：Indicates whether the virtual address is cached. A value of TRUE indicates that it is a cached address. A value of FALSE indicates that it is uncached.VOID* OALPAtoVA(UINT32 pa,BOOL cached); 在OAL层， OALPAtoVA() 功能= VirtualAlloc() + VirtualCopy()功能； 本文来自CSDN博客，转载请标明出处：http://blog.csdn.net/zhandoushi1982/archive/2009/09/22/4581226.aspx
-
发表了文章 2010-12-05
金山安全卫士软件开源了！
金山安全卫士软件开源了！ 前几天还只听别人提起，当时不以为然，心中觉得“怎么可能呢？”。 今天在google上搜索以下，果然，金山开源了，下载代码瞧瞧，果真是货真价实的源代码啊！这对于中国软件界的同仁们该是多大好事啊~， 就个人而言，但从技术难度来看，当属信息安全类的软件编码较难实现。现在金山开源了，这已经不是什么难题了！ 金山果真是大企业，大手笔。自从360免费以来，整个信息安全行业都被绞的乱七八糟，这也是新的商业模式给就的商业模式带来的冲击。今天金山开源给这新的商业模式增加了一片活力，从此就得商业模式将彻底被打垮。估计不彻底的360用不了多久就要开始开源，至此新的信息安全行业标准将如期出台，就如andirod操作系统统一了移动操作系统平台UI一样。 如此开源代码,怕是很多同仁都望而却步，故此，决定写一本诠释金山安全卫士软件开源代码，书名就是《诠释金山安全卫士软件开源代码》。
-
发表了文章 2010-11-17
Generic P2P Architecture, Tutorial and Example - CodeProject
Generic P2P Architecture, Tutorial and Example By dzzxyz | 15 Mar 2004 Generic P2P architecture, tutorial and example that covers basic P2P strategy. Upon completing this tutorial, you will be proficient in basic P2P design and coding. Is your email address OK? You are signed up for our newsletters but your email address has not been reconfirmed in a long time. To make this warning go away please click here to have a confirmation email sent so we can confirm your email address and continue sending you your newsletters. Alternatively, you can update your subscriptions. Download source code - 16.8 Kb This generic P2P architecture tutorial was brought to you by Planet API � Searching thousands of ASPX / ASP.NET Public Webmethods / Public Webservices, Search Engine APIs, Novelty APIs, B2B APIs, Game APIs, P2P APIs, Fun APIs, AI Chatbot APIs, etc. Overview of P2P Culture P2P (Peer To Peer) is when multiple computers think collectively towards a shared objective. Computer programs that use less central servers and rely on a collection of computers such as Gnutella, distributed media streaming, networks of DCC based IRC fservers etc. tend to be referred to as being more P2P. Computer programs where many end-users communicate with few central services tend to be referred to as being less P2P or not P2P. To fully understand and leverage P2P technology, one must separate his or her self from the dogma that our computer programs must be united by servers in our physical possession to synchronize activities. Rather, think of our computer programs from a more digital-life oriented perspective and break the computer software up over multiple machines and make no single part of the software critical to the collective objective. P2P Philosophy �Single servants are less powerful then a single server but the collective of many servants is more powerful then any single server� - Daniel Stephen Rule. For example, a large software company gives each employee a very small amount of responsibility. Even if this means you get your month�s coding done in a few days, it is more beneficial to the company as a whole to not rely on any single employee too much and allows the company more overall stability and to ultimately write larger more complex software packages than any single person is capable of. Your software is more P2P if you leverage this same principle to achieve more bandwidth and computing speed. Basic P2P Terminology Peer or Servant A computer program that acts as both a client and a server for the entire P2P network. Connection Manager A light server application that provides a starting point for applications which enter a P2P network. The less the connection manager is involved in the objective of your overall application, the more P2P your application is. The more P2P your application is, the less strain on your own hardware. Simple P2P Chat Example This example demonstrates a very simple but highly P2P application. This example consists of two fundamental P2P parts: a connection manager and a servant. The connection manager should be compiled and executed once. The servant should be compiled and its config.xml�s connectionmgr tag should be set to the IP address or domain name of the computer that is running the connection manager. Make multiple copies of the servant�s executable and config.xml and place them on multiple computers. Execute each servant on a different machine and they will contact the connection manager to resolve each other�s location and network with each other. Each servant will frequently ask the connection manager who is on the P2P network and keep their own publish list up to date. When a servant leaves the network, an updated list is published to all the other servants and they discontinue attempting to communicate with the servant who left. Can I still try this out if I only have one computer to work with? Yes. The connection manager assigns a new port number to each servant so each servant listens on a unique port number. You can run the servant executable as many times as you want on a single machine. For your first test, I would suggest running the connection manager once and then run the servant a couple of times on the same machine as the connection manager. Then chat in the servant windows to verify that the P2P network has been constructed on your computer. If you have other computers to work with, simply execute the servant on the other computers and chat to verify that they successfully joined the P2P network. The configuration file: Put the IP address of the server that has the connection manager here. 127.0.0.1 Leave this 85 unless you change the port that the connection manager sits on: 85 List the IP addresses or domain names that you wish this servant to ignore: 1.1.1.1 fooUser234.fooISP23423.com What are the ban tags for? The ban tags allow each servant to list the IP addresses or domain names that they do not wish to get data from. The Connection Manager Add a peer to the connection manager�s knowledge of the P2P network. Collapse | Copy Code Private Sub p2p_ConnectionRequest(Index As Integer, ByVal requestID As Long) iPortMax = iPortMax + 1 Dim a As Integer For a = 1 To p2p.UBound Dim istate As Integer istate = p2p(a).State If istate = 0 Or istate = 8 Then Call EnterCriticalSection(tCritSection) RemovePeerByOffset CStr(a) p2p(a).Close p2p(a).Accept requestID AddPeer p2p(a).RemoteHostIP, CStr(iPortMax), CStr(a) Call LeaveCriticalSection(tCritSection) Exit Sub End If DoEvents Next a DoEvents Dim i As Integer i = p2p.UBound Call EnterCriticalSection(tCritSection) Load p2p(i + 1) p2p(i + 1).Accept requestID AddPeer p2p(i + 1).RemoteHostIP, CStr(iPortMax), CStr(i + 1) Call LeaveCriticalSection(tCritSection) End Sub A servant wants a list of all its peers. Collapse | Copy Code Private Sub p2p_DataArrival(Index As Integer, ByVal bytesTotal As Long) Dim a As String If p2p(Index).State <> 7 Then p2p(Index).Close: Exit Sub p2p(Index).GetData a If a = "needs peer list" Then On Error GoTo exit_critical_list Call EnterCriticalSection(tCritSection) Dim pPersonalPeerDoc As MSXML2.DOMDocument Set pPersonalPeerDoc = New MSXML2.DOMDocument pPersonalPeerDoc.loadXML pDoc.xml pPersonalPeerDoc.selectSingleNode("./peers/peer[offset = '" &_ Index & "']/me").Text = "TRUE" p2p(Index).SendData pPersonalPeerDoc.xml exit_critical_list: On Error Resume Next Call LeaveCriticalSection(tCritSection) Else MsgBox Index & " sent: " & a & " to the connection manager" End If End Sub A servant left the network. Collapse | Copy Code Private Sub p2p_Close(Index As Integer) Call EnterCriticalSection(tCritSection) RemovePeerByOffset CStr(Index) Call LeaveCriticalSection(tCritSection) End Sub The Servants The connection manager has a new list of this servant�s peers. Collapse | Copy Code Private Sub Winsock1_DataArrival(ByVal bytesTotal As Long) Dim document As String If Winsock1.State <> 7 Then Winsock1.Close: Exit Sub Winsock1.GetData document pDoc1.loadXML document Dim pPeerList As MSXML2.IXMLDOMNodeList Set pPeerList = pDoc1.selectNodes("./peers/peer/port") userList1.Clear Dim i As Integer For i = 0 To pPeerList.length - 1 If pPeerList(i).Text = _ pDoc1.selectSingleNode("./peers/peer[me = 'TRUE']/port").Text Then userList1.AddItem "*" & pPeerList(i).Text Else userList1.AddItem pPeerList(i).Text End If Next servants1(0).Close servants1(0).LocalPort = _ CInt(pDoc1.selectSingleNode("./peers/peer[me = 'TRUE']/port").Text) servants1(0).Listen End Sub This servant is connecting to all of its peers and publishing some data to all of them. Collapse | Copy Code Private Sub txtSend1_KeyPress(KeyAscii As Integer) On Error Resume Next If KeyAscii = 13 Then iSendsLeft1 = pDoc1.selectNodes("./peers/peer").length Dim i As Integer For i = 0 To pDoc1.selectNodes("./peers/peer").length - 1 Dim iIp As String Dim iPort As Integer iIp = _ pDoc1.selectNodes("./peers/peer").Item(i).selectSingleNode("./ip").Text iPort = _ CInt(pDoc1.selectNodes("./peers/peer").Item(i).selectSingleNode("./port").Text) Dim strState As String While send1.State = 6 DoEvents Wend send1.Close send1.Connect iIp, iPort Next End If DoEvents End Sub A peer of this servant wants to connect. Collapse | Copy Code Private Sub servants1_ConnectionRequest(Index As Integer,_ ByVal requestID As Long) Dim remoteip As String Dim remoteaddy As String remoteip = servants1(Index).RemoteHostIP remoteaddy = servants1(Index).RemoteHost If (pConfig.selectNodes("./config/bans/ban[target = '"_ & remoteip & "']").length = 0) _ And (pConfig.selectNodes("./config/bans/ban[target = '" _ & remoteaddy & "']").length = 0) Then Dim a As Integer For a = 1 To servants1.UBound If servants1(a).State = 0 Or servants1(a).State = 8 Then Call EnterCriticalSection(tCritSection) servants1(a).Close servants1(a).Accept requestID Call LeaveCriticalSection(tCritSection) Exit Sub End If DoEvents Next a DoEvents Call EnterCriticalSection(tCritSection) Dim i As Integer i = servants1.UBound Load servants1(i + 1) servants1(i + 1).Accept requestID Call LeaveCriticalSection(tCritSection) End If End Sub A peer of this servant has some data for it. Collapse | Copy Code Private Sub servants1_DataArrival(Index As Integer,_ ByVal bytesTotal As Long) On Error Resume Next Dim a As String If servants1(Index).State <> 7 Then servants1(Index).Close: Exit Sub servants1(Index).GetData a txtChat1.Text = txtChat1.Text & vbCrLf & a End Sub Why don�t my P2P servants communicate with other servants on the Internet when they are behind a router? Some routers have default communication restrictions called a �Firewall�. These restrictions are intended to prevent a virus from misusing your computer and to force you to explicitly disable them if and when you need more access to the Internet. One of the most common restrictions that harm P2P networks is when the router blocks most outgoing ports by default. You can test to see if your router is blocking a port by: Run a copy of the connection manager on a computer behind your router. On a computer outside the router on the Internet, open a DOS box and type in �telnet 85�. Your connection manager on the computer behind the firewall should display: Collapse | Copy Code ip of peer 2224 1 FALSE If not then you need to enable port 85 in your router for the connection manager. This also has to be done for each peer behind a router with a built in firewall. You can still leverage P2P technology on your personal network (behind your router) as long as all of your peers and your connection manager are behind your firewall. Each peer must be configured with the IP address of the connection manager that the router has assigned to it. I would highly suggest having the router assign the connection manager machine a static IP. This way, peers do not have to be reconfigured each time you reboot the box with the connection manager on it. The servants, however, can just get a dynamic IP address from your router every time they boot up because they will use the connection manager to resolve each other's location. But I really need both servants behind routers and servants on the Internet to all be part of my P2P network. This topic is outside the scope of this article, but in short here is one common solution used by other P2P technologies: Set your connection manager on a box that is intended to be a web server. If your connection manager is behind a router then configure your router to block all incoming ports but make the router forward port 85 connections to the machine that is running the connection manager. Have each servant report to the connection manager any peers that they are unable to contact. Have the connection manager determine if they are behind a port blocking router by seeing if the suspicious servant sends another heart beat. If most peers are complaining that they can�t connect to a suspicious servant but the suspicious servant continuously asks the connection manager for a user list then the connection manager can conclude that the servant is probably behind a port blocking router. If the connection manager makes this determination, have it notify the suspicious servant that it needs to pull from another peer. For the life of the suspicious servant instance, it pulls from another peer who publishes any data that it gets. The chosen peer must not itself be a suspicious peer behind a router. This will slow the P2P network down but will allow peers behind port blocking routers to join the P2P network. I would not suggest having the connection manager just assume that a peer is suspicious instantly when the other peers complain about it, rather it should have a threshold of 2 or 3 complaints from each of the other peers before telling it that it is suspicious and assigning it a chosen peer to pull from. License This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below. A list of licenses authors might use can be found here
-
发表了文章 2010-11-16
谁才是移动互联网的赢家？
近日听一朋友介绍了一款Andriod软件，集导航、聊天功能以一身。试分析了下这款软件实现其实不难，主要应用了google map方面的技术。但是反过来想想，如果用硬生生的C++实现，那难度可不小，可以说根本不是一个人能完成的。然而听朋友后续介绍，发布这个软件的开发人员正开始创业之旅，依托Andriod平台软件作为创业之基本。这让我回想10年前，中国互联网兴起之时，那时候一片HTML网页足以价值百元，然而正是因为新技术的兴起，ASP、ASP.NET、Java高级语言的诞生，使得原本需要很大难度、很长周期完成的软件工程一下子变得那么轻松实现，可以说根本不是要什么技术。这种情况下，对那些一直想搀和互联网但又没有过硬技术的人员来说，这给了他们机会，于是基于互联网为基本的公司如雨后春训般兴起，从而导致谁都不想看到的情况反生了——“网络泡沫”。 然而在来看看今日，自从google推出Andriod 和开放Map API技术之后，原本足以倾其公司之力完成的导航软件，现在却可以在10分钟之类实现，原本在一个linux平台下需要费大量时间构建的UI现在都已经标准化了，直接使用即可，这是不是与10年前的情形有些类似呢？ 现在很多人都开始看好移动互联网这个行业，李开复博士创建的“创新工厂”的大多数产品都是基于移动互联网，加之“云计算”大力宣传，使得每一个IT人都想在中占据一席之地。这难道不和10年钱的情形又极其相似吗？ 现在我们可以大胆的预测，正如10年前，用不了10年的时间，"移动互联网泡沫"定将发生。尤其、如果大家还期待“云计算”将为移动互联网带来进一步的商机的话，那么我可以告诉你，“云计算”成熟之日就是移动互联网软件行业破灭之时。 然而、谁才是移动互联网的赢家？ 可以这样说，移动互联网，开源社区是大赢家，开源的营业模式（如google）永远是大赢家，但是更适合大家追随创业的“高效通用行业应用平台（软件+专业硬件）”才是大赢家。
-
发表了文章 2010-11-14
在完成端口中使用GetAcceptExSockaddrs
MSDN：When using AcceptEx, the GetAcceptExSockaddrs function must be called to parse the buffer into its three distinct parts (data, local socket address, and remote socket address). On Windows XP and later, once the AcceptEx function completes and the SO_UPDATE_ACCEPT_CONTEXT option is set on the accepted socket, the local address associated with the accepted socket can also be retrieved using the getsockname function. Likewise, the remote address associated with the accepted socket can be retrieved using the getpeername function. 和accept不一样的是，AcceptEx是非阻塞的，而且在接受客户端请求前，可以发起n个AcceptEx，当连接建立时(三次握手结束时刻)，会有一个接受完成包加入IOCP的完成队列，按照MSDN上说，就可以在工作线程中，通过调用GetAcceptExSockaddrs解析(parse)1）客户端发送的第一块数据，2）本地(Server)套接字地址，3）远程(Client)套接字地址 工作线程： // 取得客户地址 int nLocalLen, nRmoteLen; LPSOCKADDR pLocalAddr, pRemoteAddr; m_lpfnGetAcceptExSockaddrs( pBuffer->buff, pBuffer->nLen - ((sizeof(sockaddr_in) + 16) * 2), sizeof(sockaddr_in) + 16, sizeof(sockaddr_in) + 16, (SOCKADDR **)&pLocalAddr, &nLocalLen, (SOCKADDR **)&pRemoteAddr, &nRmoteLen); 正如上面说当AcceptEx完成时，并且SO_UPDATE_ACCEPT_CONTEXT选项被设置时，也可以通过getsockname返回local address ，通过调用getpeername返回remote address 设置SO_UPDATE_ACCEPT_CONTEXT原因是： When the AcceptEx function returns, the socket sAcceptSocket is in the default state for a connected socket. The socket sAcceptSocket does not inherit the properties of the socket associated with sListenSocket parameter until SO_UPDATE_ACCEPT_CONTEXT is set on the socket. 设置： Use the setsockopt function to set the SO_UPDATE_ACCEPT_CONTEXT option, specifying sAcceptSocket as the socket handle and sListenSocket as the option value. For example: err = setsockopt( sAcceptSocket, SOL_SOCKET, SO_UPDATE_ACCEPT_CONTEXT, (char *)&sListenSocket, sizeof(sListenSocket) );
-
发表了文章 2010-11-12
嵌入式通用行业应用平台的灵魂和搭建
嵌入式通用行业应用平台的灵魂和搭建 嵌入式通用行业应用平台的灵魂和搭建 机会总是伴随着市场需求的到来，如今嵌入式行业的发展如日中天。有些单靠做流媒体行业应用发家的，有些单靠做手持机行业产品发家的。从市场分析来看，所有的这些应用都是基于一个很小的行业发展起来的，深入研究数年就小有成就，正如我去年发表的一片文章中介绍的，如今的嵌入式行业应该定位一个行业，深挖这个行业的需求，并专注于这个行业，致力做到该行业的领导品牌。但是反过来看看，在嵌入式行业，基于行业应用的产品也不乏小数，成功的例子又有几人? 如此、不禁引起我们的反思，如何构建嵌入式通用行业应用平台呢？让我们从下面这几个问题来慢慢阐述。 什么是嵌入式通用行业应用平台的灵魂？ 这是一个困挠着无数嵌入式通用行业应用平台的开发项目经理的大难问题。这个群体中到多数人是从事硬件开发的，由于他们一直以来在硬件技术的沉淀和积累，无形中使得他们产生思维定时，从而一味的追求硬件技术的创新和实现，他们认为硬件平台是嵌入式通用行业应用平台的灵魂。孰不知，正是这种定势在悄悄的扼杀了平台的灵魂，导致最终的产品像一堆废铁一样堆在仓库当中，接下来整个团队就开始不停的接收硬件定制项目，接收之时、才惊讶的发现这个硬件平台还能应用这样的行业，孰不知这整个行业的发展机会已经拱手让给了别人，自己还拼命的兴奋与下一个定制项目，如此、整个团队的创新、激情、活力就将断送在定制项目，这也是为什么嵌入式行业人次流动颇大的原因。 什么才是嵌入式通用行业应用平台的灵魂呢？我可以毫不夸张的告诉大家，硬件平台只是基础，真正灵魂是软件平台。在中国，软件的发展要早于硬件，在嵌入式行业，软件的规范和管理流程要优硬件平台，软件是正真提些行业应用的需求，是摆在客户面的直接印象，如果把嵌入式通用行业应用产品进行分解，“模具”是产品衣服，“软件”是产品的中枢，硬件是产品的裸体。举个例子，相信很多人都用过凯立德导航软件，凯立德软件以其独特的界面风格、精确的地理信息著称，从而被应用绝大部分的终端设备上。现如今有谁能记住，导航产品的硬件结构呢？可以这样说，凯立德公司是完全可以做到硬件外包，或则直接兼容其他硬件平台。试问硬件平台还是嵌入式通用行业应用平台的核心吗？ 怎样进行软件平台的搭建？ 如果大家对软件平台是嵌入式通用行业应用平台的灵魂没有疑意，那么如何来进行软件平台的搭建呢？ 首先、需求是整个产品的关键所在，没有需求的产品是肯定的没有投资的必要。因此软件平台的第一份需求材料应该来自于销售和市场人员，因此搭建软件平台首先应该完善销售和市场人员捕捉需求的机制，应该建立研发人员和市场、销售人员需求互相的平台，使得研发人员能够第一时间获取需求信息，调整产品的开发方向。 其次、采用快速原型开发模式进行初期的软件开发，在如今的中国软件行业，为抢夺市场正确进一步捕捉需求的时间，我想不到第二种模式能够跟适合他们的。因此在构建嵌入式通用应用平台的初期应该迅速根据当前的需求构建出于一个相对完善的软件平台，这个初期版本可以当作整个平台的技术指标，也可以直接参与项目演示，尽量争取软件平台与这个特定行业打交道的机会，这也正是进一步捕捉需求的机会。大家都知道一旦软件的需求完善了，软件的灵魂就开始孕育了，不管是重新构建软件，还是在原型的基础之上继续修改开发，最终的软件都将给整个产品带来无限活力。 最后、将整个软件产品化，由于原型开发阶段获取了大量的需求材料，这时候正是考虑产品的时候了，就像凯立德一样，完全脱离硬件平台。软件的产品化需要对整个需求进行筛选、分析，最终根据需求分析说明书制定相应的详细软件设计方案，最后参照软件原型开始进行再次开发，并进行最终的需求确认性测试，如此整个软件平台的设计才算完成。 因此，我建议在通用行业应用平台设计之初，应该同时制定硬件和软件开发团队，软硬件平台协同开发进行，软件开发团队主要的作用就是捕捉硬件平台适合应用的行业需求，并开发出软件原型。 怎样进行软件平台的测试？ 如果是做过软件开发的人员都会发现，软件测试在整个开发流程中都占据着重要的作用。有时候会发现软件的测试时间要比软件开发的时间高出两倍甚至更多。那么在嵌入式行业中如何做到软件平台的测试呢？ 测试不是一成不变的，根据各个行业需求的不同测试的要求也不同，例如军工、医疗行业就不同，他们对测试的要求就极其之高。但是有一点我们可以肯定，不管那个行业他们对性能的要求总是有个指标的，因此我觉得软件平台的测试应该制定测试指标，让测试指标贯穿整个测试过程，不管是功能测试、单元测试、系统测试、集成测试还是确认性测试。测试指标可以如下定义； rps：response rate(响应速度)接口响应性能参数，表示每秒最少响应次数 eot：error’s count of thousand (错误次数)接口性能参数，千次中出现错误的最多次数 fps：frame per sercond软件功能性能参数，指定每秒最少获取视频帧数 可以在具体的行业测试可以根据具体的需求规定这些参数，例如在视频监控行业，可以根据一些标准规定，如下; 服务连接接口响应性能指标为：0.3 rps 客户端传输过程错误次数指标: 10 eot 客户与服务器传输速度指标: 15 fps 如果规定的这些测试指标一旦获得了客户的确认，那么这个整个测试人员来说测试将是如此明了的事情，只需要根据规定编写测试用例进行测试即可。 最终、嵌入式通用行业应用平台必定是嵌入式行业的发展方向，构建嵌入式通用行业应用平台确实不是一件容易的事情，尤其对于项目负责人来说是多么大挑战啊！每次平台的搭建就好像一次创业，稍有不慎产品的市场就将荡然无存，整个团队就将处于定制项目的无效挣扎当中，但是只要我们坚持不遗余力的进行产品的演变 、软件需求捕捉和重构，我相信行业最终将属于我们的团队。
-
发表了文章 2010-11-08
（DirectX系列09）DirectShow EDS应用编码分析
（DirectX系列09）DirectShow EDS应用编码分析 DES (DirectShow Editing Services)，是一套基于DirectShow核心框架的编程接口。DES的出现，简化了视频编辑任务，弥补了DirectShow对于媒体文件非线性编辑支持的先天性不足。但是，就技术本身而言，DES并没有超越DirectShow Filter架构，而只是DirectShow Filter的一种增强应用。 本章通过DirectX 下timelinetest为例子详细介绍Eds应用编码实现，其中主要包括创建事件线对象、在轨道上加入音频过度、在轨道上加入视频过度、使用控制引擎等。 创建事件线对象 这是在所有操作之前必须要做的事情，同样也贯穿这个编码过程，如下代码； hr = CoCreateInstance( CLSID_AMTimeline, NULL, CLSCTX_INPROC_SERVER, IID_IAMTimeline, (void**) &pTimeline ); if(FAILED( hr )) { Err(_T("Could not create timeline")); return hr; } 创建组，并标记为音频组或视频组 DirectShow 通过CreateEmptyNode来创建EDS组，并采用SetMediaType函数来标识音视频组，设置为MEDIATYPE_Video标识为视频组，设置为MEDIATYPE_Audio为音频组，如下代码； // 创建视频组 hr = pTimeline->CreateEmptyNode( &pVideoGroupObj, TIMELINE_MAJOR_TYPE_GROUP ); CComQIPtr< IAMTimelineGroup, &IID_IAMTimelineGroup > pVideoGroup( pVideoGroupObj ); CMediaType VideoGroupType; // all we set is the major type. The group will automatically use other defaults VideoGroupType.SetType( &MEDIATYPE_Video ); hr = pVideoGroup->SetMediaType( &VideoGroupType ); // 创建音频组 hr = pTimeline->CreateEmptyNode( &pAudioGroupObj, TIMELINE_MAJOR_TYPE_GROUP ); CComQIPtr< IAMTimelineGroup, &IID_IAMTimelineGroup > pAudioGroup( pAudioGroupObj ); CMediaType AudioGroupType; // all we set is the major type. The group will automatically use other defaults AudioGroupType.SetType( &MEDIATYPE_Audio ); hr = pAudioGroup->SetMediaType( &AudioGroupType ); 在轨道上加入视频过渡处理，并实现过渡子对象的设置 CComPtr< IAMTimelineObj > pTrack2Obj; hr = pTimeline->CreateEmptyNode( &pTrack2Obj, TIMELINE_MAJOR_TYPE_TRACK ); if(FAILED( hr )) { Err(_T("Could not create second track")); return hr; } hr = pRootComp->VTrackInsBefore( pTrack2Obj, -1 ); if(FAILED( hr )) { Err(_T("Could not insert second track")); return hr; } TLStart = 0 * UNITS; TLStop = 8 * UNITS; MediaStart = 0 * UNITS; MediaStop = 8 * UNITS; (void)StringCchCopyW( pClipname, NUMELMS(pClipname), T2W( tBasePath ) ); (void)StringCchCatW( pClipname, NUMELMS(pClipname), L"///0" ); (void)StringCchCatW( pClipname, NUMELMS(pClipname),wszVideo2Name ); // create the timeline source // CComPtr<IAMTimelineObj> pSource2Obj; hr = pTimeline->CreateEmptyNode( &pSource2Obj, TIMELINE_MAJOR_TYPE_SOURCE ); if(FAILED( hr )) { Err(_T("Could not create the second timeline source")); return hr; } // set up source right // hr = pSource2Obj->SetStartStop( TLStart, TLStop ); CComQIPtr< IAMTimelineSrc, &IID_IAMTimelineSrc > pSource2Src( pSource2Obj ); hr |= pSource2Src->SetMediaTimes( MediaStart, MediaStop ); hr |= pSource2Src->SetMediaName( pClipname ); if(FAILED( hr )) { Err(_T("Could not configure second media source")); return E_FAIL; } //-------------------------------------------- // tell the track about the source //-------------------------------------------- CComQIPtr< IAMTimelineTrack, &IID_IAMTimelineTrack > pTrack2( pTrack2Obj ); hr = pTrack2->SrcAdd( pSource2Obj ); if(FAILED( hr )) { Err(_T("Could not add second track")); return hr; } CComQIPtr< IAMTimelineTransable, &IID_IAMTimelineTransable > pTransable( pTrack2 ); #ifdef DO_TRANSITION REFERENCE_TIME TransStart = 0 * UNITS; REFERENCE_TIME TransStop = 4 * UNITS; // create the timeline effect // CComPtr<IAMTimelineObj> pTrackTransObj; hr = pTimeline->CreateEmptyNode(&pTrackTransObj, TIMELINE_MAJOR_TYPE_TRANSITION ); if(FAILED( hr )) { Err(_T("Could not create transition effect")); return hr; } //-------------------------------------------- // set up filter right //-------------------------------------------- // we set the CLSID of the DXT to use instead of a pointer to the // actual object. We let the DXT have it's default properties. // hr = pTrackTransObj->SetSubObjectGUID( CLSID_DxtJpeg ); hr |= pTrackTransObj->SetStartStop( TransStart, TransStop ); CComQIPtr< IAMTimelineTrans, &IID_IAMTimelineTrans > pTrackTrans( pTrackTransObj ); hr |= pTransable->TransAdd( pTrackTransObj ); if(FAILED( hr )) { Err(_T("Could not configure transition object")); return E_FAIL; } #ifdef CUTS_ONLY //--------------------------------------------- // turn the transition into a cut by doing this //--------------------------------------------- hr = pTrackTrans->SetCutsOnly( TRUE ); if(FAILED( hr )) { Err(_T("Could not SetCutsOnly to TRUE")); return hr; } #endif // CUTS_ONLY //--------------------------------------------- // create an transition on the track from B 2 A //--------------------------------------------- TransStart = 4 * UNITS; TransStop = 8 * UNITS; // create the timeline effect // pTrackTransObj.Release( ); hr = pTimeline->CreateEmptyNode(&pTrackTransObj, TIMELINE_MAJOR_TYPE_TRANSITION ); ASSERT( !FAILED( hr ) ); // set up filter right // hr = pTrackTransObj->SetSubObjectGUID( CLSID_DxtJpeg ); hr |= pTrackTransObj->SetStartStop( TransStart, TransStop ); pTrackTrans = pTrackTransObj; hr |= pTrackTrans->SetSwapInputs( TRUE ); hr |= pTransable->TransAdd( pTrackTransObj ); ASSERT( !FAILED( hr ) ); 在轨道中加入声音效果 CComPtr< IAMTimelineObj > pTrack4FxObj; hr = pTimeline->CreateEmptyNode( &pTrack4FxObj, TIMELINE_MAJOR_TYPE_EFFECT ); ASSERT( !FAILED( hr ) ); // set up effect riht // hr = pTrack4FxObj->SetStartStop( TLStart, TLStop ); hr |= pTrack4FxObj->SetSubObjectGUID( CLSID_AudMixer ); ASSERT( !FAILED( hr ) ); // add the effect // CComQIPtr< IAMTimelineEffectable , &IID_IAMTimelineEffectable > pTrack4Fable( pTrack4 ); hr = pTrack4Fable->EffectInsBefore( pTrack4FxObj, -1 ); ASSERT( !FAILED( hr ) ); 控制引擎的使用 hr = CoCreateInstance( CLSID_RenderEngine, NULL, CLSCTX_INPROC_SERVER, IID_IRenderEngine, (void**) &pRenderEngine ); ASSERT( !FAILED( hr ) ); // tell the render engine about the timeline it should look at // hr = pRenderEngine->SetTimelineObject( pTimeline ); ASSERT( !FAILED( hr ) ); //-------------------------------------------- // connect up the front end, then the back end //-------------------------------------------- hr = pRenderEngine->ConnectFrontEnd( ); hr |= pRenderEngine->RenderOutputPins( ); ASSERT( !FAILED( hr ) ); //-------------------------------------------- // get a bunch of pointers, then run the graph //-------------------------------------------- hr = pRenderEngine->GetFilterGraph( &pGraph ); hr |= pGraph->QueryInterface( IID_IMediaEvent, (void**) &pEvent ); hr |= pGraph->QueryInterface( IID_IMediaControl, (void**) &pControl ); hr |= pGraph->QueryInterface( IID_IMediaSeeking, (void**) &pSeeking ); hr |= pGraph->QueryInterface( IID_IVideoWindow, (void**) &pVidWindow ); ASSERT( !FAILED( hr ) ); //-------------------------------------------- // give the main window a meaningful title //-------------------------------------------- BSTR bstrCaption; WriteBSTR(&bstrCaption, wszTitle); hr = pVidWindow->put_Caption(bstrCaption); FreeBSTR(&bstrCaption); //-------------------------------------------- // since no user interaction is allowed, remove // system menu and maximize/minimize buttons //-------------------------------------------- long lStyle=0; hr = pVidWindow->get_WindowStyle(&lStyle); lStyle &= ~(WS_MINIMIZEBOX | WS_MAXIMIZEBOX | WS_SYSMENU); hr = pVidWindow->put_WindowStyle(lStyle); //-------------------------------------------- // run it //-------------------------------------------- hr = pControl->Run( ); ASSERT( !FAILED( hr ) ); //-------------------------------------------- // wait for it //-------------------------------------------- long EventCode = 0; hr = pEvent->WaitForCompletion( -1, &EventCode ); ASSERT( !FAILED( hr ) ); REFERENCE_TIME Start = 0; #ifdef DO_RENDERRANGE // seek the timeline back to 0 // hr = pSeeking->SetPositions( &Start, AM_SEEKING_AbsolutePositioning, NULL, AM_SEEKING_NoPositioning ); ASSERT( !FAILED( hr ) ); hr = pRenderEngine->SetRenderRange2( 2.0, 6.0 ); ASSERT( !FAILED( hr ) ); //------------------------------------------------------ // connect up the front end, then the back end if needed //------------------------------------------------------ hr = pRenderEngine->ConnectFrontEnd( ); ASSERT( !FAILED( hr ) ); if(hr == (HRESULT) S_WARN_OUTPUTRESET) { hr |= pRenderEngine->RenderOutputPins( ); } ASSERT( !FAILED( hr ) ); //-------------------------------------------- // run it //-------------------------------------------- hr = pControl->Run( ); ASSERT( !FAILED( hr ) ); //-------------------------------------------- // wait for it //-------------------------------------------- hr = pEvent->WaitForCompletion( -1, &EventCode ); ASSERT( !FAILED( hr ) ); #endif // DO_RENDERRANGE #ifdef DO_RECONNECT //--------------------------------------------- // make a change to the timeline, however small //--------------------------------------------- CComPtr< IAMTimelineObj > pTransObj; REFERENCE_TIME InOut = -1; pTransable->GetNextTrans( &pTransObj, &InOut ); CComQIPtr< IAMTimelineTrans, &IID_IAMTimelineTrans > pTrans( pTransObj ); pTrans->SetCutsOnly( TRUE ); pTransObj.Release( ); pTrans.Release( ); hr = pTransable->GetNextTrans( &pTransObj, &InOut ); pTrans = pTransObj; hr |= pTrans->SetCutsOnly( TRUE ); ASSERT( !FAILED( hr ) ); // seek the timeline back to 0 // hr = pSeeking->SetPositions( &Start, AM_SEEKING_AbsolutePositioning, NULL, AM_SEEKING_NoPositioning ); ASSERT( !FAILED( hr ) ); //------------------------------------------------------ // connect up the front end, then the back end if needed //------------------------------------------------------ hr = pRenderEngine->ConnectFrontEnd( ); if(hr == (HRESULT) S_WARN_OUTPUTRESET) { hr |= pRenderEngine->RenderOutputPins( ); } ASSERT( !FAILED( hr ) ); //-------------------------------------------- // run it //-------------------------------------------- hr = pControl->Run( ); ASSERT( !FAILED( hr ) ); //-------------------------------------------- // wait for it //-------------------------------------------- hr = pEvent->WaitForCompletion( -1, &EventCode );
-
发表了文章 2010-11-08
点对点视频会议程序:VideoNet
该程序可以用于两个人在LAN/Intranet(或者Internet)上进行视频会议。现在有许多视频会议程序，每个都有各自的性能提升技术。主要的问题是视频会议视频帧的尺寸对于传输来说太大。因此，性能依赖于对帧的编解码。我使用快速h263编码库来达到更好的压缩率提高速度。该程序做些小改动也可以在Internet上使用。音频的录制与播放我在以前的语音会议程序中使用了RecordSound和PlaySound类，这里我将提供摘要说明RecordSound和PlaySound类的使用。 // Create and Start Recorder Thread record=new RecordSound(this); record->CreateThread(); // Create and Start Player Thread play=new PlaySound1(this); play->CreateThread(); // Start Recording record->PostThreadMessage(WM_RECORDSOUND_STARTRECORDING,0,0); // Start Playing play->PostThreadMessage(WM_PLAYSOUND_STARTPLAYING,0,0); // During audio recording, data will be available in the OnSoundData // callback function of the RecordSound class. Here, you can place // your code to send the data to remote host... // To play the data received from the remote host play->PostThreadMessage(WM_PLAYSOUND_PLAYBLOCK,size,(LPARAM)data); // Stop Recording record->PostThreadMessage(WM_RECORDSOUND_STOPRECORDING,0,0); // Stop Playing play->PostThreadMessage(WM_PLAYSOUND_STOPPLAYING,0,0); // At last, to Stop the Recording Thread record->PostThreadMessage(WM_RECORDSOUND_ENDTHREAD,0,0); // To stop playing thread... play->PostThreadMessage(WM_PLAYSOUND_ENDTHREAD,0,0); 视频捕获使用VFW(Video For Windows)API进行视频捕获，它提供了通过webcam进行视频捕获。 VideoCapture.h 和VideoCapture.cpp包含了处理视频捕获的代码。如下代码说明了如何使用该类： // Create instance of Class vidcap=new VideoCapture(); // This is later used to call display function of the main // dialog class when the frame is captured... vidcap->SetDialog(this); // This does lot of work, including connecting to the driver // and setting the desired video format. Returns TRUE if // successfully connected to videocapture device. vidcap->Initialize(); // If successfully connected, you can get the BITMAPINFO // structure associated with the video format. This is later // used to display the captured frame... this->m_bmpinfo=&vidcap->m_bmpinfo; // Now you can start the capture.... vidcap->StartCapture(); // Once capture is started, frames will arrive in the "OnCaptureVideo" // callback function of the VideoCapture class. Here you call the // display function to display the frame. // To stop the capture vidcap->StopCapture(); // If your job is over....just destroy it.. vidcap->Destroy(); 要使以上代码通过编译，你应该链接适当的库： #pragma comment(lib,"vfw32") #pragma comment(lib,"winmm") 显示捕获的视频帧有许多方法和API可以显示捕获的视频。你可以使用SetDIBitsToDevice()方法直接显示，但给予GDI的函数非常的慢。更好的方法是使用DrawDib API 显示。DrawDib函数为设备无关位图(DIBs)提供了高性能的图形绘制能力。DrawDib函数直接写入视频内存，因此性能更好。以下代码摘要演示了使用DrawDib API显示视频帧。 // Initialize DIB for drawing... HDRAWDIB hdib=::DrawDibOpen(); // Then call this function with suitable parameters.... ::DrawDibBegin(hdib,...); // Now, if you are ready with the frame data, just invoke this // function to display the frame ::DrawDibDraw(hdib,...); // Finally, termination... ::DrawDibEnd(hdib); ::DrawDibClose(hdib); 编解码库编码器： 我使用快速h.263编码库进行编码。该库是使其实时编码更快的 Tmndecoder 修改版。我已经将该库从C转换到C++，这样可以很容易用于任何Windows应用程序。我移除了快速h263编码库中一些不必要的代码与文件，并在.h和.cpp文件中移除了一些定义与申明。以下是H263编码库的使用方法： // Initialize the compressor CParam cparams; cparams.format = CPARAM_QCIF; InitH263Encoder(&cparams); //If you need conversion from RGB24 to YUV420, call this InitLookupTable(); // Set up the callback function // OwnWriteFunction is the global function called during // encoding to return the encoded data... WriteByteFunction = OwnWriteFunction; // For compression, data must be in the YUV420 format... // Hence, before compression, invoke this method ConvertRGB2YUV(IMAGE_WIDTH,IMAGE_HEIGHT,data,yuv); // Compress the frame..... cparams.format = CPARAM_QCIF; cparams.inter = CPARAM_INTRA; cparams.Q_intra = 8; cparams.data=yuv; // Data in YUV format... CompressFrame(&cparams, &bits); // You can get the compressed data from the callback function // that you have registerd at the begining... // Finally, terminate the encoder // ExitH263Encoder(); 解码器：这是tmndecoder(H.263解码器)的修改版。使用ANSI C编写，我将它转换到C++使其方便在Windows应用程序中使用。我移除了一些用于显示和文件处理的文件，移除了不必要的代码并增加了一些新文件。原始的库中一些文件不适合于实时的解码。我已经做了修改使其适合实时的解码处理。现在，可以使用该库来解码H263帧，该库非常快，性能不错。解码的使用方法： //Initialize the decoder InitH263Decoder(); // Decompress the frame.... // > rgbdata must be large enough to hold the output data... // > decoder produces the image data in YUV420 format. After // decoding, it is converted into RGB24 format... DecompressFrame(data,size,rgbdata,buffersize); // Finaly, terminate the decoder ExitH263Decoder(); 如何运行程序拷贝可执行文件到局域网上两台不同的机器中：A和B，运行他们。在机器A(或B）中选择connect菜单条，在弹出的对话框中输入机器B的名字或IP地址然后按connect按钮，在另外一台机器（B）显示出accept/reject对话框，按accept按钮。在机器A将显示一个通知对话框，按OK后开始会议。That''''s it....Enjoy......!!! 致谢：我感谢 Paul Cheffers 提供了他的音频录制播放类。因为有了开源人士奉献的开源库才有你所看到的videonet程序，我感激Tmndecoder的开发者Karl Lillevold和h.263快速编码库的开发者Roalt Aalmoes 免费提供这些开发库。
-
发表了文章 2010-11-02
Wiondows 查询卷设备信息代码实现
NTSTATUS DPQueryVolumeInformation( PDEVICE_OBJECT DevObj, LARGE_INTEGER * TotalSize, DWORD * ClusterSize, DWORD * SectorSize ) { #define _FileSystemNameLength 64 //定义FAT16文件系统签名的偏移量 #define FAT16_SIG_OFFSET 54 //定义FAT32文件系统签名的偏移量 #define FAT32_SIG_OFFSET 82 //定义NTFS文件系统签名的偏移量 #define NTFS_SIG_OFFSET 3 //这是FAT16文件系统的标志 const UCHAR FAT16FLG[4] = {'F','A','T','1'}; //这是FAT32文件系统的标志 const UCHAR FAT32FLG[4] = {'F','A','T','3'}; //这是NTFS文件系统的标志 const UCHAR NTFSFLG[4] = {'N','T','F','S'}; //返回值 NTSTATUS ntStatus = STATUS_SUCCESS; //用来读取卷DBR扇区的数据缓冲区 BYTE DBR[512] = { 0 }; //DBR扇区有512个bytes大小 ULONG DBRLength = 512; //以下是三个指针，统一指向读取的DBR数据，但是这三个指针的类型分别代表FAT16，FAT32和NTFS类型文件系统的DBR数据结构 PDP_NTFS_BOOT_SECTOR pNtfsBootSector = (PDP_NTFS_BOOT_SECTOR)DBR; PDP_FAT32_BOOT_SECTOR pFat32BootSector = (PDP_FAT32_BOOT_SECTOR)DBR; PDP_FAT16_BOOT_SECTOR pFat16BootSector = (PDP_FAT16_BOOT_SECTOR)DBR; //读取的偏移量，对于DBR来说是卷的起始位置，所以偏移量为0 LARGE_INTEGER readOffset = { 0 }; //读取时的io操作状态 IO_STATUS_BLOCK ios; //为了同步读取所设置的同步事件 KEVENT Event; //为了同步读取所需要构建的irp指针 PIRP pIrp = NULL; //下面我们首先从指定的卷设备上读取偏移量为0的一个扇区，也就是这个卷的DBR扇区，准备加以分析 //因为我们要同步读取，所以先初始化一个为了同步读取设置的事件 KeInitializeEvent(&Event, NotificationEvent, FALSE); //构造一个irp用来发给卷设备来读取信息 pIrp = IoBuildAsynchronousFsdRequest( IRP_MJ_READ, DevObj, DBR, DBRLength, &readOffset, &ios ); if (NULL == pIrp) { goto ERROUT; } //设置完成函数，并且将同步事件作为完成函数的参数传入 IoSetCompletionRoutine( pIrp, DPQueryVolumeInformationCompletionRoutine, &Event, TRUE, TRUE, TRUE ); //调用目标设备去处理这个irp ntStatus = IoCallDriver(DevObj, pIrp); if(ntStatus = STATUS_PENDING) { //如果下层设备一时不能完成这个irp请求，我们就等 ntStatus = KeWaitForSingleObject( &Event, Executive, KernelMode, FALSE, NULL ); //将返回值设置为这个io操作的状态 ntStatus = pIrp->IoStatus.Status; if (!NT_SUCCESS(ntStatus)) { goto ERROUT; } } if (*(DWORD*)NTFSFLG == *(DWORD*)&DBR[NTFS_SIG_OFFSET]) { //通过比较标志发现这个卷是一个ntfs文件系统的卷，下面根据ntfs卷的DBR定义来对各种需要获取的值进行赋值操作 *SectorSize = (DWORD)(pNtfsBootSector->BytesPerSector); *ClusterSize = (*SectorSize) * (DWORD)(pNtfsBootSector->SectorsPerCluster); TotalSize->QuadPart = (LONGLONG)(*SectorSize) * (LONGLONG)pNtfsBootSector->TotalSectors; } else if (*(DWORD*)FAT32FLG == *(DWORD*)&DBR[FAT32_SIG_OFFSET]) { //通过比较标志发现这个卷是一个ntfs文件系统的卷，下面根据ntfs卷的DBR定义来对各种需要获取的值进行赋值操作 *SectorSize = (DWORD)(pFat32BootSector->BytesPerSector); *ClusterSize = (*SectorSize) * (DWORD)(pFat32BootSector->SectorsPerCluster); TotalSize->QuadPart = (LONGLONG)(*SectorSize) * (LONGLONG)(pFat32BootSector->LargeSectors + pFat32BootSector->Sectors); } else if (*(DWORD*)FAT16FLG == *(DWORD*)&DBR[FAT16_SIG_OFFSET]) { //通过比较标志发现这个卷是一个ntfs文件系统的卷，下面根据ntfs卷的DBR定义来对各种需要获取的值进行赋值操作 *SectorSize = (DWORD)(pFat16BootSector->BytesPerSector); *ClusterSize = (*SectorSize) * (DWORD)(pFat16BootSector->SectorsPerCluster); TotalSize->QuadPart = (LONGLONG)(*SectorSize) * (LONGLONG)(pFat16BootSector->LargeSectors + pFat16BootSector->Sectors); } else { //走到这里，可能是其它任何文件系统，但是不是windows认识的文件系统，我们统一返回错 ntStatus = STATUS_UNSUCCESSFUL; } ERROUT: if (NULL != pIrp) { IoFreeIrp(pIrp); } return ntStatus; } NTSTATUS DPVolumeOnLineCompleteRoutine( IN PDEVICE_OBJECT DeviceObject, IN PIRP Irp, IN PVOLUME_ONLINE_CONTEXT Context ) { //返回值 NTSTATUS ntStatus = STATUS_SUCCESS; //这个卷设备的dos名字，也就是C，D等 UNICODE_STRING DosName = { 0 }; //在这里Context是不可能为空的，为空就是出错了 ASSERT(Context!=NULL); //下面调用我们自己的VolumeOnline处理 //获取这个卷的dos名字 ntStatus = IoVolumeDeviceToDosName(Context->DevExt->PhyDevObj, &DosName); if (!NT_SUCCESS(ntStatus)) goto ERROUT; //将dos名字变成大写形式 Context->DevExt->VolumeLetter = DosName.Buffer[0]; if (Context->DevExt->VolumeLetter > L'Z') Context->DevExt->VolumeLetter -= (L'a' - L'A'); //我们只保护“F”盘 if (Context->DevExt->VolumeLetter == L'F') { //获取这个卷的基本信息 ntStatus = DPQueryVolumeInformation( Context->DevExt->PhyDevObj, &(Context->DevExt->TotalSizeInByte), &(Context->DevExt->ClusterSizeInByte), &(Context->DevExt->SectorSizeInByte)); if (!NT_SUCCESS(ntStatus)) { goto ERROUT; } //建立这个卷对应的位图 ntStatus = DPBitmapInit( &Context->DevExt->Bitmap, Context->DevExt->SectorSizeInByte, 8, 25600, (DWORD)(Context->DevExt->TotalSizeInByte.QuadPart / (LONGLONG)(25600 * 8 * Context->DevExt->SectorSizeInByte)) + 1); if (!NT_SUCCESS(ntStatus)) goto ERROUT; //对全局量赋值，说明我们找到需要保护的那个设备了 gProtectDevExt = Context->DevExt; } ERROUT: if (!NT_SUCCESS(ntStatus)) { if (NULL != Context->DevExt->Bitmap) { DPBitmapFree(Context->DevExt->Bitmap); } if (NULL != Context->DevExt->TempFile) { ZwClose(Context->DevExt->TempFile); } } if (NULL != DosName.Buffer) { ExFreePool(DosName.Buffer); } //设置等待同步事件，这样可以让我们等待的DeviceIoControl处理过程继续运行 KeSetEvent( Context->Event, 0, FALSE); return STATUS_SUCCESS; }
-
发表了文章 2010-10-29
Windows 操作系统页面文件原理
页面文件，是指操作系统反映构建并使用虚拟内存的硬盘空间大小所使用的文件。要整理页面文件，首先将页面文件从原先所在的驱动器移动到其他驱动器，然后对原来驱动器进行整理，最后再将页面文件移回到原驱动器上，此时页面文件就会存放在连续的磁盘空间中了。具体来说，在 windows操作系统下（Windows 2000/XP）pagefile.sys这个文件，它就是系统页面文件(也就是大家熟知的虚拟内存文件)，它的大小取决于打开的程序多少和你原先设置页面文件的最小最大值，是不断变化的，有时可能只有几十M，有时则达到600M以上。 一些大型软件由于对内存的消耗比较大，也单独开辟一部分硬盘空间作为缓冲，这部分硬盘空间的实体就是软件创建的页面文件，比如Photoshop。这种设计可以缓解系统调度内存的压力，并提高软件运行速度。 内存在计算机中的作用很大，电脑中所有运行的程序都需要经过内存来执行，如果执行的程序很大或很多，就会导致内存消耗殆尽。为了解决这个问题，Windows中运用了虚拟内存技术，即拿出一部分硬盘空间来充当内存使用，当内存占用完时，电脑就会自动调用硬盘来充当内存，以缓解内存的紧张。举一个例子来说，如果电脑只有128MB物理内存的话，当读取一个容量为200MB的文件时，就必须要用到比较大的虚拟内存，文件被内存读取之后就会先储存到虚拟内存，等待内存把文件全部储存到虚拟内存之后，跟着就会把虚拟内存里储存的文件释放到原来的安装目录里了。下面，就让我们一起来看看如何对虚拟内存进行设置吧。 前面这段话有误导，虚拟内存并非在物理内存用完之后才使用，他们是同时进行的;虚拟内存只是暂时存储物理内存的内容，任何使用中的内存都只从物理内存中读取，如果物理内存都占完，还怎么再把虚拟内存中的内容放入到物理内存中去呢？ 即使物理内存没有耗尽，系统同样会为新打开的程序指定一个位置，只不过此时在页面文件夹中这个制定位置的文件是空的而已。 对于虚拟内存主要设置两点，即内存大小和分页位置，内存大小就是设置虚拟内存最小为多少和最大为多少；而分页位置则是设置虚拟内存应使用那个分区中的硬盘空间。对于内存大小的设置，如何得到最小值和最大值呢？你可以通过下面的方法获得：选择“开始→程序→附件→系统工具→系统监视器”（如果系统工具中没有，可以通过“添加/删除程序”中的Windows安装程序进行安装）打开系统监视器，然后选择“编辑→添加项目”，在“类型”项中选择“内存管理程序”，在右侧的列表选择“交换文件大小”。这样随着你的操作，会显示出交换文件值的波动情况，你可以把经常要使用到的程序打开，然后对它们进行使用，这时查看一下系统监视器中的表现值，由于用户每次使用电脑时的情况都不尽相同，因此，最好能够通过较长时间对交换文件进行监视来找出最符合您的交换文件的数值，这样才能保证系统性能稳定以及保持在最佳的状态。 找出最合适的范围值后，在设置虚拟内存时，用鼠标右键点击“我的电脑”，选择“属性”，弹出系统属性窗口，选择“性能”标签，点击下面“虚拟内存”按钮，弹出虚拟内存设置窗口，点击“用户自己指定虚拟内存设置”单选按钮，“硬盘”选较大剩余空间的分区，然后在“最小值”和“最大值”文本框中输入合适的范围值。如果您感觉使用系统监视器来获得最大和最小值有些麻烦的话，这里完全可以选择“让Windows管理虚拟内存设置”。 Windows 9x的虚拟内存分页位置，其实就是保存在C盘根目录下的一个虚拟内存文件（也称为交换文件）Win386.swp，它的存放位置可以是任何一个分区，如果系统盘C容量有限，我们可以把Win386.swp调到别的分区中，方法是在记事本中打开System.ini（C:/Windows下）文件，在[386Enh]小节中，将“PagingDrive=C:WindowsWin386.swp”，改为其他分区的路径，如将交换文件放在D:中，则改为“D:Win386.swp=PagingDrive”如没有上述语句可以直接键入即可。 而对于使用Windows 2000和Windows XP的，可以选择“控制面板→系统→高级→性能”中的“设置→高级→更改”，打开虚拟内存设置窗口，在驱动器[卷标]中默认选择的是系统所在的分区，如果想更改到其他分区中，首先要把原先的分区设置为无分页文件，然后再选择其他分区。 或者，WinXP一般要求物理内存在256M以上。如果你喜欢玩大型3D游戏，而内存（包括显存）又不够大，系统会经常提示说虚拟内存不够，系统会自动调整（虚拟内存设置为系统管理）。 如果你的硬盘空间够大，你也可以自己设置虚拟内存，具体步骤如下：右键单击“我的电脑”→属性→高级→性能 设置→高级→虚拟内存 更改→选择虚拟内存（页面文件）存放的分区→自定义大小→确定最大值和最小值→设置。一般来说，虚拟内存为物理内存的1.5倍，稍大一点也可以，如果你不想虚拟内存频繁改动，可以将最大值和最小值设置为一样。 对于虚拟内存如何设置的问题，微软已经给我们提供了官方的解决办法，对于一般情况下，我们推荐采用如下的设置方法： (1)在Windows系统所在分区设置页面文件，文件的大小由你对系统的设置决定。具体设置方法如下：打开"我的电脑"的"属性"设置窗口，切换到"高级"选项卡，在"启动和故障恢复"窗口的"写入调试信息"栏，如果你采用的是"无"，则将页面文件大小设置为2MB左右，如果采用"核心内存存储"和"完全内存存储"，则将页面文件值设置得大一些，跟物理内存差不多就可以了。
-
发表了文章 2010-10-27
SQL Server 2008开始支持MERGE
merge主要用于两表之间的关联操作 oracle中 merge: 从oracle 9i开始支持merge用法，10g有了完善 create table a (id_ integer,count_ integer); insert into a values(1,3); insert into a values(3,6); create table b (id_ integer,count_ integer); insert into b values(1,7); insert into b values(2,4); MERGE INTO a USING b ON (a.id_ = b.id_) WHEN MATCHED THEN UPDATE SET count_ = b.count_+a.count_ /* 注意指名count_属于的表 */ WHEN NOT MATCHED THEN INSERT VALUES (b.id_,b.count_); commit; select * from a; 结果： id_ count_ 1 10 3 6 2 4 SQL Server 2008开始支持merge: 有两张结构一致的表：test1,test2 create table test1 (id int,name varchar(20)) go create table test2 (id int,name varchar(20)) go insert into test1(id,name) values(1,'boyi55'),(2,'51cto'),(3,'bbs'),(4,'fengjicai'),(5,'alis') insert into test2(id,name) values(1,'boyi'),(2,'51cto') 将test1同步到test2中，没有的数据进行插入，已有数据进行更新 merge test2 t --要更新的目标表 using test1 s --源表 on t.id=s.id --更新条件（即主键） when matched --如果主键匹配，更新 then update set t.name=s.name when not matched then insert values(id,name);--目标主未知主键，插入。此语句必须以分号结束 运行以下查询查看更新结果 select a.id,a.name as name_1,b.name as name_2 from test1 as a,test2 as b where a.id=b.id id name_1 name_2 ----------- -------------------- -------------------- 1 boyi55 boyi55 2 51cto 51cto 3 bbs bbs 4 fengjicai fengjicai 5 alis alis
-
发表了文章 2010-10-27
(DirectX系列08）DirectShow WavDest编码分析
在DirectShow 中有很多Samples，WavDest就是其中一个，这个Fliter主要用于将采集到的视频流写入到指定的文件，文件格式是.WAV。如下来看看具体的代码实现。 对于每一个Filter都有一个固定的注册区代码，这是必不可少，如下代码； // {3C78B8E2-6C4D-11d1-ADE2-0000F8754B99} static const GUID CLSID_WavDest = { 0x3c78b8e2, 0x6c4d, 0x11d1, { 0xad, 0xe2, 0x0, 0x0, 0xf8, 0x75, 0x4b, 0x99 } }; const AMOVIESETUP_FILTER sudWavDest = { &CLSID_WavDest, // clsID L"WAV Dest", // strName MERIT_DO_NOT_USE, // dwMerit 0, // nPins 0 // lpPin }; // Global data CFactoryTemplate g_Templates[]= { {L"WAV Dest", &CLSID_WavDest, CWavDestFilter::CreateInstance, NULL, &sudWavDest}, }; int g_cTemplates = sizeof(g_Templates) / sizeof(g_Templates[0]); 这个Filter主要用于将采集的数据保存在指定的文件当中，因此它的最佳的父类应该是CTransformFilter，还有几个函数也是必须需要实现的如CreateInstance、Transform等，如下来看看CreateInstance是如何实现的。 if(SUCCEEDED(*phr)) { // Create an output pin so we can have control over the connection // media type. CWavDestOutputPin *pOut = new CWavDestOutputPin(this, phr); if(pOut) { if(SUCCEEDED(*phr)) { m_pOutput = pOut; } else { delete pOut; } } else { *phr = E_OUTOFMEMORY; } // // NOTE!: If we've created our own output pin we must also create // the input pin ourselves because the CTransformFilter base class // will create an extra output pin if the input pin wasn't created. // CTransformInputPin *pIn = new CTransformInputPin(NAME("Transform input pin"), this, // Owner filter phr, // Result code L"In"); // Pin name // a failed return code should delete the object if(pIn) { if(SUCCEEDED(*phr)) { m_pInput = pIn; } else { delete pIn; } } else { *phr = E_OUTOFMEMORY; } } 再来看看Transform代码是如何实现,还有WavDest是如何将数据保存到文件当中;在WavDest中数据处理流程是Transform函数-> Copy函数->StopStreaming函数,其中StopStreaming函数用于保存数据,该函数代码如下; IStream *pStream; if(m_pOutput->IsConnected() == FALSE) return E_FAIL; IPin * pDwnstrmInputPin = m_pOutput->GetConnected(); if(!pDwnstrmInputPin) return E_FAIL; HRESULT hr = ((IMemInputPin *) pDwnstrmInputPin)->QueryInterface(IID_IStream, (void **)&pStream); if(SUCCEEDED(hr)) { BYTE *pb = (BYTE *)_alloca(m_cbHeader); RIFFLIST *pRiffWave = (RIFFLIST *)pb; RIFFCHUNK *pRiffFmt = (RIFFCHUNK *)(pRiffWave + 1); RIFFCHUNK *pRiffData = (RIFFCHUNK *)(((BYTE *)(pRiffFmt + 1)) + m_pInput->CurrentMediaType().FormatLength()); pRiffData->fcc = FCC('data'); pRiffData->cb = m_cbWavData; pRiffFmt->fcc = FCC('fmt '); pRiffFmt->cb = m_pInput->CurrentMediaType().FormatLength(); CopyMemory(pRiffFmt + 1, m_pInput->CurrentMediaType().Format(), pRiffFmt->cb); pRiffWave->fcc = FCC('RIFF'); pRiffWave->cb = m_cbWavData + m_cbHeader - sizeof(RIFFCHUNK); pRiffWave->fccListType = FCC('WAVE'); LARGE_INTEGER li; ZeroMemory(&li, sizeof(li)); hr = pStream->Seek(li, STREAM_SEEK_SET, 0); if(SUCCEEDED(hr)) { hr = pStream->Write(pb, m_cbHeader, 0); } pStream->Release(); } 其他代码都是常规实现,具体代码可以查看Directshow C++目录下的Samples.
-
发表了文章 2010-10-26
（DirectX系列07）DirectShow MFC下Filter编码分析
MFC下Filter的编写和Win32下Filter的编写极其相似，但又存在很多不同点，在Win32中需要去实现CreateInstance函数，而在MFC直接用NEW 动态产生，不过这并不代表不需要去维护Filter对象计数，因此在MFC 下Filter中依然需要调用AddRef函数来维护这个平衡；其它还不需要实现注册表注册等功能，如下代码； ////////////////////////////////////////////////////////////////////////////////CAppTransform::CAppTransform(LPUNKNOWN pUnkOuter, HRESULT *phr) : CTransInPlaceFilter(NAME("App Transform"), pUnkOuter, GUID_NULL, phr){} HRESULT CAppTransform::Transform(IMediaSample *pSample){ // Override to do something inside the application // Such as grabbing a poster frame... // ... return S_OK;} // Check if we can support this specific proposed type and formatHRESULT CAppTransform::CheckInputType(const CMediaType *pmt) { // We accept a series of raw media types if (pmt->majortype == MEDIATYPE_Video && (pmt->subtype == MEDIASUBTYPE_RGB32 || pmt->subtype == MEDIASUBTYPE_RGB24 || pmt->subtype == MEDIASUBTYPE_RGB565 || pmt->subtype == MEDIASUBTYPE_RGB555 || pmt->subtype == MEDIASUBTYPE_UYVY || pmt->subtype == MEDIASUBTYPE_YUY2)) { return NOERROR; } return E_FAIL;}
-
发表了文章 2010-10-25
（DirectX系列06）DirectShow 字符叠加Filter编码分析
（DirectX系列06）DirectShow 字符叠加Filter编码分析 在很多视频播放的软件当中，字幕的处理是免不了的，有些可能本身已经加载到图像当中未做处理，但大部分都是通过字符叠加来进行处理的。DirectShow 的字符叠加Filter在这些软件中都扮演这绝佳的作用。这一节来介绍DirectShow字符叠加Filter编码的实现，如下详细介绍； 这个Filter的大概作用是在视频流指定的一段时间内进行字符叠加，字符字体、大小、颜色都进行控制，普遍支持RGB 的各种编码格式，同时实现字符的其他效果例如滚动的。如下来看看具体的编码实现； 注册表配置 注册表的配置对于Filter的开发者来说都大同小异，主要对g_Templates进行配置，如下代码； // {E3FB4BFE-8E5C-4aec-8162-7DA55BE486A1} DEFINE_GUID(CLSID_HQTitleOverlay, 0xe3fb4bfe, 0x8e5c, 0x4aec, 0x81, 0x62, 0x7d, 0xa5, 0x5b, 0xe4, 0x86, 0xa1); // {E70FE57A-19AA-4a4c-B39A-408D49D73851} DEFINE_GUID(CLSID_HQTitleOverlayProp, 0xe70fe57a, 0x19aa, 0x4a4c, 0xb3, 0x9a, 0x40, 0x8d, 0x49, 0xd7, 0x38, 0x51); …… // List of class IDs and creator functions for the class factory. This // provides the link between the OLE entry point in the DLL and an object // being created. The class factory will call the static CreateInstance CFactoryTemplate g_Templates[] = { { L"HQ Title Overlay Std.", &CLSID_HQTitleOverlay, CFilterTitleOverlay::CreateInstance, NULL, &sudFilter }, { L"HQ Title Overlay Property Page", &CLSID_HQTitleOverlayProp, CTitleOverlayProp::CreateInstance } }; CFilterTitleOverlay类的实现 CFilterTitleOverlay类设计是整个Filter的关键所在，其中最重要的是父类的选择。如下代码； class CFilterTitleOverlay : public CTransInPlaceFilter , public ISpecifyPropertyPages , public ITitleOverlay CTransInPlaceFilter类是CFilterTitleOverlay功能实现的关键只关键，该类可以提供在视频传输的过程中截获数据流，方便于字符叠加； ISpecifyPropertyPages类是CFilterTitleOverlay提供属性页面支持功能，ITitleOverlay是一个Interface接口的纯虚类，也就是所谓的接口，如下来看看这个接口是如何实现； // ITitleOverlay // 原型如下 // interface __declspec(novtable) ITitleOverlay : public IUnknown DECLARE_INTERFACE_(ITitleOverlay, IUnknown) { // 设置Filter进行叠加的类型，如果需要改变类型，这个函数必须第一个设置， // 调用这个函数成功后，才能调用其他的函数进行参数设置。 // 可以设置的叠加类型参见枚举类型OVERLAY_TYPE的定义。 // 如下原型 // virtual HRESULT _stcall put_TitleOverlayType(long inOverlayType) = 0 STDMETHOD(put_TitleOverlayType) (THIS_ long inOverlayType ) PURE; …… }; 构造函数 先从CFilterTitleOverlay类的构造函数说起，在CFilterTitleOverlay构造函数中有一个对象是必须创建的，这个对象就是COverlayController，COverlayController是用来控制叠加效果的通用类，如下代码； CFilterTitleOverlay::CFilterTitleOverlay(TCHAR *tszName, LPUNKNOWN punk, HRESULT *phr) : CTransInPlaceFilter(tszName, punk, CLSID_HQTitleOverlay, phr) { mOverlayType = OT_STATIC; mOverlayController = new COverlayController(); mNeedEstimateFrameRate = FALSE; char szTitle[] = "Hello, DirectShow!"; put_Title(szTitle, sizeof(szTitle)); } CreateInstance函数 这个函数不用多说了，是CUnknown中的一个虚函数，其中这个函数主要用来创建CFilterTitleOverlay对象，是COM组件必须具备的函数，否则就失去COM组件的意义，如下代码详解； CUnknown * WINAPI CFilterTitleOverlay::CreateInstance(LPUNKNOWN punk, HRESULT *phr) { #if 1 // 做指定应用程序验证 char szCreatorPath[256], szCreatorName[256]; ::strcpy(szCreatorPath, ""); ::strcpy(szCreatorName, ""); HMODULE hModule = ::GetModuleHandle(NULL); ::GetModuleFileName(hModule, szCreatorPath, 256); char * backSlash = ::strrchr(szCreatorPath, '//'); if (backSlash) { strcpy(szCreatorName, backSlash); } ::_strlwr(szCreatorName); // Please specify your app name with lowercase if (::strstr(szCreatorName, "graphedt") == NULL && ::strstr(szCreatorName, "ourapp") == NULL) { *phr = E_FAIL; return NULL; } #endif // 创建CFilterTitleOverlay对象 CFilterTitleOverlay *pNewObject = new CFilterTitleOverlay(NAME("TitleOverlay"), punk, phr); return pNewObject; } Transform 函数 Transform函数是整个字符叠加处理的关键，再这个函数中可以捕获需要处理的数据（RGB格式）如下来看看具体的实现； HRESULT CFilterTitleOverlay::Transform(IMediaSample *pSample) { // If we cann't read frame rate info from input pin's connection media type, // We estimate it from the first sample's time stamp! …… if (mOverlayType != OT_NONE) { PBYTE pData = NULL; pSample->GetPointer(&pData); mOverlayController->DoTitleOverlay(pData); } return NOERROR; } 代码中最为关键的DoTitleOverlay函数就是实现字符叠加的函数，这个函数是COverlayController类中的一个成员函数，如下来看看它是如何实现的； if (mImageHeight > mTitleSize.cy && mTitleSize.cx > 0 && mTitleSize.cy > 0) { …… PBYTE pStartPos = pTopLine + mStartPos.y * strideInBytes + mStartPos.x * mImageBitCount / 8; for (DWORD dwY = 0; dwY < (DWORD)mTitleSize.cy; dwY++) { PBYTE pbTitle = mTitleDIBBits + mDIBWidthInBytes * ((DWORD)mTitleSize.cy - dwY - 1); // Point to the valid start position of title DIB pbTitle += (mValidTitleRect.left >> 3); long startLeft = mValidTitleRect.left % 8; long endRight = startLeft + mValidTitleRect.right - mValidTitleRect.left; for (long dwX = startLeft; dwX < endRight; dwX++) { if ( !((0x80 >> (dwX & 7)) & pbTitle[dwX >> 3]) ) { PBYTE pbPixel = mPixelConverter->NextNPixel(pStartPos, dwX - startLeft); if (mIsOverlayByCover) { // 进行RGB数据复值，24三占用三字节 mPixelConverter->ConvertByCover(pbPixel); } else { mPixelConverter->ConvertByReverse(pbPixel); } } } pStartPos += strideInBytes; } } ActualCreateTitleDIB函数 这个函数用于创建字符位图，创建一个DIB位图的虚拟内存空间，保存RGB数据格式，再通过GetDIBits函数获取数据缓冲区用于字符叠加之用，如下代码； HBITMAP COverlayController::ActualCreateTitleDIB(HDC inDC) { // DIB info we used to create title pixel-mapping. // The system default color policy is: // Initial Whole Black, while output area White-background and Black-text. struct { BITMAPINFOHEADER bmiHeader; DWORD rgbEntries[2]; } bmi = { { sizeof(BITMAPINFOHEADER), 0, 0, 1, 1, BI_RGB, 0, 0, 0 }, { 0x00000000, 0xFFFFFFFF } }; …… // Set proper DIB size here! Important! bmi.bmiHeader.biHeight = mTitleSize.cy; bmi.bmiHeader.biWidth = mTitleSize.cx; HBITMAP hbm = CreateDIBitmap(inDC, &bmi.bmiHeader, 0, NULL, NULL, 0); BOOL pass = (hbm != NULL); // Draw title after selecting DIB into the DC if (pass) { HGDIOBJ hobj = SelectObject(inDC, hbm); pass = ExtTextOut(inDC, 0, 0, ETO_OPAQUE | ETO_CLIPPED, NULL, mTitle, lstrlen(mTitle), NULL); SelectObject(inDC, hobj); } // Get the title-drew DIB bits if (pass) { ReleaseTitleDIB(); // Attention: To get bitmap data from the DIB object, // the scan line must be a multiple of 4 (DWORD)! // If the actual bitmap data is not exactly fit for DWORD, // The rest of DWORD bits will be filled automatically. // So we should expand to bytes and round up to a multiple of 4. mDIBWidthInBytes = ((mTitleSize.cx + 31) >> 3) & ~3; mTitleDIBBits = new BYTE[mDIBWidthInBytes * mTitleSize.cy]; memset(mTitleDIBBits, 0, mDIBWidthInBytes * mTitleSize.cy); LONG lLines = GetDIBits(inDC, hbm, 0, mTitleSize.cy, (PVOID)mTitleDIBBits, (BITMAPINFO *)&bmi, DIB_RGB_COLORS); pass = (lLines != 0); } …… return hbm; } CompleteConnect函数 CompleteConnect函数是用来完成output pin 与 下一个input pin连接只用，也是构建Fiters 链的必备函数，如下代码； HRESULT CFilterTitleOverlay::CompleteConnect(PIN_DIRECTION direction, IPin *pReceivePin) { HRESULT hr = CTransInPlaceFilter::CompleteConnect(direction, pReceivePin); if (SUCCEEDED(hr) && direction == PINDIR_INPUT) { hr = SetInputVideoInfoToController(); } return hr; } 属性页设置（CTitleOverlayProp） 这个类从CBasePropertyPage直接继承的，用于配置和观察这个Filters 属性之用。CTitleOverlayProp其实是一个窗体，类似一个应用程序，不过我们可以直接对其进行消息捕捉，如下代码； BOOL CTitleOverlayProp::OnReceiveMessage(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam) { switch (uMsg) { case WM_INITDIALOG: { // Get windows' handles m_hOverlayType = GetDlgItem(hwnd, IDC_COMBO_OVERLAY_TYPE); m_hEditTilte = GetDlgItem(hwnd, IDC_EDIT_TITLE); m_hEditStartX = GetDlgItem(hwnd, IDC_EDIT_STARTX); m_hEditStartY = GetDlgItem(hwnd, IDC_EDIT_STARTY); m_hEditStartTime = GetDlgItem(hwnd, IDC_EDIT_STARTTIME); m_hEditEndTime = GetDlgItem(hwnd, IDC_EDIT_ENDTIME); m_hEditColorR = GetDlgItem(hwnd, IDC_EDIT_COLORR); m_hEditColorG = GetDlgItem(hwnd, IDC_EDIT_COLORG); m_hEditColorB = GetDlgItem(hwnd, IDC_EDIT_COLORB); break; } case WM_COMMAND: { if (HIWORD(wParam) == BN_CLICKED) { switch (LOWORD(wParam)) { case IDC_BUTTON_CHANGE_FONT: OnButtonChangeFont(); break; } } SetDirty(); break; } } return CBasePropertyPage::OnReceiveMessage(hwnd,uMsg,wParam,lParam); } // OnReceiveMessage 有了这个消息捕捉函数当然就可以直接对所需要配置的参数进行配置了。同时在继承CBasePropertyPage的时候为了方便CBasePropertyPage还提供了其他几个接口如CreateInstance、OnConnect、OnActivate等。 整个字符叠加的Fiters编码就基本上完成了，其中几个地方还是需要在次提醒，第一、对基类的选择，一定要选择正确的基类，这样才能达到事半工倍。第二、处理字符叠加时一定要注意帧频率，否则会产生错位。第三、字符需要绘制到一段虚拟的内存当中，不能直接绘制。字符叠加的应用非常广泛，估计暴风影音的字符叠加功能就是这样做的！
-
发表了文章 2010-10-18
static_cast、dynamic_cast、reinterpret_cast、和const_cast
关于强制类型转换的问题，很多书都讨论过，写的最详细的是C++ 之父的《C++ 的设计和演化》。最好的解决方法就是不要使用C风格的强制类型转换，而是使用标准C++的类型转换符：static_cast, dynamic_cast。标准C++中有四个类型转换符：static_cast 、dynamic_cast 、reinterpret_cast 、 和const_cast 。下面对它们一一进行介绍。static_cast 用 法：static_cast < type-id > ( expression ) 该运算符把 expression转换为type-id类型，但没有运行时类型检查来保证转换的安全性。它主要有如下几种用法： 用于类层次结构中基类和子类之间指针或引用的转换。进行上行转换（把子类的指针或引用转换成基类表示）是安全 的；进行下行转换（把基类指针或引用转换成子类表示）时，由于没有动态类型检查，所以是不安全的。 用于基本数据类型之间的转换，如把 int转换成char，把int转换成enum。这种转换的安全性也要开发人员来保证。 把空指针转换成目标类型的空指针。 把 任何类型的表达式转换成void类型。 注意：static_cast不能转换掉expression的const、 volitale、或者__unaligned属性。dynamic_cast 用 法：dynamic_cast < type-id > ( expression ) 该运算符把 expression转换成type-id类型的对象。Type-id必须是类的指针、类的引用或者void *；如果type-id是类指针类型，那么expression也必须是一个指针，如果type-id是一个引用，那么expression也必须是一个 引用。 dynamic_cast主要用于类层次间的上行转换和下行转换，还可以用于类之间的交叉转换。 在类层次间进行上行转换时，dynamic_cast和static_cast的效果是一样的；在进行下行转换时，dynamic_cast具有类型检查 的功能，比static_cast更安全。 class B{ public: int m_iNum; virtual void foo(); }; class D:public B{ public: char *m_szName[100]; }; void func(B *pb){ D *pd1 = static_cast<D *>(pb); D *pd2 = dynamic_cast<D *>(pb); } 在上面的代码段中，如果pb指向一个D类型的对象，pd1和pd2是一样的，并且对这两个指 针执行D类型的任何操作都是安全的；但是，如果pb指向的是一个B类型的对象，那么pd1将是一个指向该对象的指针，对它进行D类型的操作将是不安全的 （如访问m_szName），而pd2将是一个空指针。另外要注意：B要有虚函数，否则会编译出错；static_cast则没有这个限制。这是由于运行 时类型检查需要运行时类型信息，而这个信息存储在类的虚函数表（关于虚函数表的概念，详细可见<Inside c++ object model>）中，只有定义了虚函数的类才有虚函数表，没有定义虚函数的类是没有虚函数表的。 另外，dynamic_cast 还 支持交叉转换（cross cast）。如下代码所示。 class A{ public: int m_iNum; virtual void f(){} }; class B:public A{ }; class D:public A{ }; void foo(){ B *pb = new B; pb->m_iNum = 100; D *pd1 = static_cast<D *>(pb); //copile error D *pd2 = dynamic_cast<D *>(pb); //pd2 is NULL delete pb; } 在 函数foo中，使用static_cast 进行转换是不被允许的，将在编译时出错；而使用 dynamic_cast 的转换 则是允许的，结果是空指针。reinpreter_cast 用 法：reinpreter_cast <type-id> (expression) type-id必须是一个 指针、引用、算术类型、函数指针或者成员指针。它可以把一个指针转换成一个整数，也可以把一个整数转换成一个指针（先把一个指针转换成一个整数，在把该整 数转换成原类型的指针，还可以得到原先的指针值）。 该运算符的用法比较多。const_cast 用法：const_cast <type_id> (expression) 该运算符用来修改类型的const或volatile属性。除了const 或volatile修饰之外， type_id和expression的类型是一样的。 常量指针被转化成非常量指针，并且仍然指向原来的 对象；常量引用被转换成非常量引用，并且仍然指向原来的对象；常量对象被转换成非常量对象。 Voiatile和const类试。举如下一 例： class B{ public: int m_iNum; } void foo(){ const B b1; b1.m_iNum = 100; //comile error B b2 = const_cast<B>(b1); b2. m_iNum = 200; //fine } 上面的代码编译时会报错，因为b1是一个常量对象，不能对它进行改变；使用const_cast把它转换成一个常量对象，就可以对它的数据成员任意改变。 注意：b1和b2是两个不同的对象。
-
发表了文章 2010-10-17
Windows CE S3C2440A显示驱动编码分析
在Windows CE下显示驱动是一个比较复杂的驱动，不仅仅设计到硬件的操作，还有上层驱动的GDI接口支持，有时候还需要支持DirectDraw等绘图接口。如果所有的编码工作都重新做一遍的话，难度还是挺大的，庆幸的是微软已经把大部分的接口都提供好了，DDI中包含的20个接口函数（以函数指针的方式表示），需要我们实现的也仅仅只有GPEEnableDriver。闲话不多说，我们先来看看S3C2440A_LCD_REG结构体吧，如下所示； typedef struct { UINT32 LCDCON1; // 0x00 UINT32 LCDCON2; // 0x04 UINT32 LCDCON3; // 0x08 UINT32 LCDCON4; // 0x0C UINT32 LCDCON5; // 0x10 UINT32 LCDSADDR1; // 0x14 UINT32 LCDSADDR2; // 0x18 UINT32 LCDSADDR3; // 0x1C UINT32 REDLUT; // 0x20 UINT32 GREENLUT; // 0x24 UINT32 BLUELUT; // 0x28 UINT32 PAD[8]; // 0x2C - 0x48 UINT32 DITHMODE; // 0x4C UINT32 TPAL; // 0x50 UINT32 LCDINTPND; // 0x54 UINT32 LCDSRCPND; // 0x58 UINT32 LCDINTMSK; // 0x5C UINT32 TCONSEL; // 0x60 } S3C2440A_LCD_REG, *PS3C2440A_LCD_REG; 其中这里面的成员分别对应着LCD控制器中的相关的寄存器，一旦映射成功就可以对这些寄存器进行直接性质的操作了。先来看看Eboot是如何对其进行初始化操作的，如下代码； static void InitDisplay(void) { volatile S3C2440A_IOPORT_REG *s2440IOP = (S3C2440A_IOPORT_REG *)OALPAtoVA(S3C2440A_BASE_REG_PA_IOPORT, FALSE); volatile S3C2440A_LCD_REG *s2440LCD = (S3C2440A_LCD_REG *)OALPAtoVA(S3C2440A_BASE_REG_PA_LCD, FALSE); // Set up the LCD controller registers to display a power-on bitmap image. s2440IOP->GPCUP = 0xFFFFFFFF; s2440IOP->GPCCON = 0xAAAAAAAA; s2440IOP->GPDUP = 0xFFFFFFFF; s2440IOP->GPDCON = 0xAAAAAAAA; s2440IOP->GPGCON &= ~(3 << 8); // Set LCD_PWREN as output s2440IOP->GPGCON |= (1 << 8); s2440IOP->GPGDAT |= (1 << 4); // LCD3V3 ON //clkval_calc = (WORD)((float)(S3C2440A_HCLK)/(2.0*5000000)+0.5)-1; s2440LCD->LCDCON1 = (CLKVAL_TFT << 8) |/* VCLK = HCLK /((CLKVAL + 1) * 2) -> About7Mhz */ (LCD_MVAL_USED << 7) | /* 0 : Each Frame */ (3 << 5) | /* TFT LCD Pannel */ (12 << 1) | /* 16bpp Mode */ (0 << 0) ; /* Disable LCD Output */ s2440LCD->LCDCON2 = (LCD_VBPD << 24) | /* VBPD : 1 */ (LCD_LINEVAL_TFT << 14) | /* Vertical Size : 320 - 1 */ (LCD_VFPD << 6) | /* VFPD : 2 */ (LCD_VSPW << 0) ; /* VSPW : 1 */ s2440LCD->LCDCON3 = (LCD_HBPD << 19) | /* HBPD : 6 */ (LCD_HOZVAL_TFT << 8) | /* HOZVAL_TFT : 240 - 1 */ (LCD_HFPD << 0) ; /* HFPD : 2 */ s2440LCD->LCDCON4 = (LCD_MVAL << 8) | /* MVAL : 13 */ (LCD_HSPW << 0) ; /* HSPW : 4 */ s2440LCD->LCDCON5 = (0 << 12) | /* BPP24BL : LSB valid */ (1 << 11) | /* FRM565 MODE : 5:6:5 Format */ (0 << 10) | /* INVVCLK : VCLK Falling Edge */ (1 << 9) | /* INVVLINE:Inverted Polarity */ (1 << 8) | /* INVVFRAME : Inverted Polarity */ (0 << 7) | /* INVVD : Normal */ (0 << 6) | /* INVVDEN : Normal */ (0 << 5) | /* INVPWREN : Normal */ (0 << 4) | /* INVENDLINE : Normal */ (1 << 3) | /* PWREN : Disable PWREN */ (0 << 2) | /* ENLEND: Disable LEND signal */ (0 << 1) | /* BSWP : Swap Disable */ (1 << 0) ; /* HWSWP : Swap Enable */ s2440LCD->LCDSADDR1 = ((IMAGE_FRAMEBUFFER_DMA_BASE_eboot >> 22) << 21) | ((M5D(IMAGE_FRAMEBUFFER_DMA_BASE_eboot >> 1)) << 0); s2440LCD->LCDSADDR2 = M5D((IMAGE_FRAMEBUFFER_DMA_BASE_eboot + (LCD_XSIZE_TFT * LCD_YSIZE_TFT * 2)) >> 1); s2440LCD->LCDSADDR3 = (((LCD_XSIZE_TFT - LCD_XSIZE_TFT) / 1) << 11) | (LCD_XSIZE_TFT / 1); //s2440LCD->TCONSEL |= 0x3; s2440LCD->TCONSEL &= (~7); //s2440LCD->TCONSEL |= (0x1<<4); s2440LCD->TPAL = 0x0; s2440LCD->LCDCON1 |= 1; // Display a bitmap image on the LCD... //memcpy((void *)IMAGE_FRAMEBUFFER_UA_BASE, ScreenBitmap, LCD_ARRAY_SIZE_TFT_16BIT); memset((void *)IMAGE_FRAMEBUFFER_UA_BASE_eboot, 0xef, LCD_ARRAY_SIZE_TFT_16BIT); } 上面的代码已经注释的很清楚，其中IMAGE_FRAMEBUFFER_UA_BASE_eboot指向的是显示屏幕的内存区域，其中主要可以装载RGB256 4MB的图片数据；OALPAtoVA函数的作用是将物理地址转化为虚拟地址，物理地址和虚拟地址的映射表可以在oemaddrtab_cfg.inc文件中可以看到，对应的数组为g_oalAddressTable二维数组。 然而驱动模式下的内存映射API和Eboot完全不同，正如前几节中介绍，驱动的内存映射方式常采用VirtualAlloc、VirtualCopy等函数，如下代码； v_s2440CLKPWR = (volatile S3C2440A_CLKPWR_REG *)VirtualAlloc(0, sizeof(S3C2440A_CLKPWR_REG), MEM_RESERVE, PAGE_NOACCESS); VirtualCopy((PVOID)v_s2440CLKPWR, (PVOID)(S3C2440A_BASE_REG_PA_CLOCK_POWER>>8), sizeof(S3C2440A_CLKPWR_REG), PAGE_READWRITE | PAGE_NOCACHE | PAGE_PHYSICAL )) vm_pIOPreg = (volatile S3C2440A_IOPORT_REG *)VirtualAlloc(0, sizeof(S3C2440A_IOPORT_REG), MEM_RESERVE, PAGE_NOACCESS); VirtualCopy((PVOID)vm_pIOPreg, (PVOID)(S3C2440A_BASE_REG_PA_IOPORT >> 8), sizeof(S3C2440A_IOPORT_REG), PAGE_PHYSICAL | PAGE_READWRITE | PAGE_NOCACHE)) CursorOn操作 显示的操作对于上层应用来说，通过微软的封装GDI接口可以实现各种各样的图形绘制，包括简单的线、点、矩形等操作，然而对于系统来说就比较简单了，其中只需要实现最基本的点操作，线操作即可，当然需要复杂的操作还可以扩展。如下介绍CursorOn的操作，这个函数主要实现对鼠标类型的各个操作绘制，相反的还有CursorOff函数操作，如下代码； if (!m_CursorForcedOff && !m_CursorDisabled && !m_CursorVisible) { RECTL cursorRectSave = m_CursorRect; int iRotate; RotateRectl(&m_CursorRect); for (y = m_CursorRect.top; y < m_CursorRect.bottom; y++){ if (y < 0){ continue; } if (y >= m_nScreenHeightSave){ break; } ptrLine = &ptrScreen[y * m_pPrimarySurface->Stride()]; cbsLine = &m_CursorBackingStore[(y - m_CursorRect.top) * (m_CursorSize.x * (m_colorDepth >> 3))]; for (x = m_CursorRect.left; x < m_CursorRect.right; x++){ if (x < 0){ continue; } if (x >= m_nScreenWidthSave){ break; } // x' = x - m_CursorRect.left; y' = y - m_CursorRect.top; // Width = m_CursorSize.x; Height = m_CursorSize.y; switch (m_iRotate){ case DMDO_0: iRotate = (y - m_CursorRect.top)*m_CursorSize.x + x - m_CursorRect.left; break; case DMDO_90: iRotate = (x - m_CursorRect.left)*m_CursorSize.x + m_CursorSize.y - 1 - (y - m_CursorRect.top); break; case DMDO_180: iRotate = (m_CursorSize.y - 1 - (y - m_CursorRect.top))*m_CursorSize.x + m_CursorSize.x - 1 - (x - m_CursorRect.left); break; case DMDO_270: iRotate = (m_CursorSize.x -1 - (x - m_CursorRect.left))*m_CursorSize.x + y - m_CursorRect.top; break; default: iRotate = (y - m_CursorRect.top)*m_CursorSize.x + x - m_CursorRect.left; break; } cbsLine[(x - m_CursorRect.left) * (m_colorDepth >> 3)] = ptrLine[x * (m_colorDepth >> 3)]; ptrLine[x * (m_colorDepth >> 3)] &= m_CursorAndShape[iRotate]; ptrLine[x * (m_colorDepth >> 3)] ^= m_CursorXorShape[iRotate]; if (m_colorDepth > 8) { cbsLine[(x - m_CursorRect.left) * (m_colorDepth >> 3) + 1] = ptrLine[x * (m_colorDepth >> 3) + 1]; ptrLine[x * (m_colorDepth >> 3) + 1] &= m_CursorAndShape[iRotate]; ptrLine[x * (m_colorDepth >> 3) + 1] ^= m_CursorXorShape[iRotate]; if (m_colorDepth > 16) { cbsLine[(x - m_CursorRect.left) * (m_colorDepth >> 3) + 2] = ptrLine[x * (m_colorDepth >> 3) + 2]; ptrLine[x * (m_colorDepth >> 3) + 2] &= m_CursorAndShape[iRotate]; ptrLine[x * (m_colorDepth >> 3) + 2] ^= m_CursorXorShape[iRotate]; } } } } m_CursorRect = cursorRectSave; m_CursorVisible = TRUE; } 其他还有如电源操作（需要初始化话屏幕，其原理有点像Windows XP的驱动开发模式），绘制各种图形操作等，显示的编码与硬件操作的这块不是很难，只需要继承DDGPE，实现相应的虚拟函数即可，可以说这也是C++高妙之处。随后几节将介绍DDGPE的实现；
-
发表了文章 2010-10-17
快速原型开发模式在实际开发过程中的应用
【摘要】本文以作者的实践开发经验为主线，从理论和实际的角度探讨快速原型开发模式在实践开发中的应用，并从软件开发的各个角度、各个时期剖析快速开发模式的优缺点和应该注意的问题。 【关键字】软件工程、开发模式、快速开发、软件开发、原型模式 快速原型开发模式的基本思想是在系统开发的初期，在对用户需求初步了解的基础之上，以快速的方法先构造一个可以工作的系统原型。将这个原型提供给用户使用，听取他们的意见。然后修正原型，补充新的数据、数据结构和应用模型，形成新的原型。经过几次迭代以后，可以达到用户与开发者之间的完美沟通，消除各种误解，形成明确的系统定义以及用户界面要求。 了解快速原型开发模式后，下面结合我开发过学分制收费管理系统项目经验来谈一下我是如何在实际开发过程中实施快速原型开发模式。 【项目背景】 随着国家教育事业的发展，很多高校纷纷引进学分制教学体制模式。我所在的学校为跟上时代的步伐，经市教委批准，从2008-2009学年试行学分制改革，2009-2010正式运行。以传统的教学模式相比学分制教学模式有很多明显的优势，学生的自由度也得到了很大的提高。然而一种新的教学模式要取代传统的教学模式，势必会存在很多麻烦和问题。其中学分制教学模式下的收费方式与传统的收费方式就存在很大的差异，任然沿用传统的收费方式已经无法满足学分制改革的要求，因此为推动学分制改革，制定一套符合学分制教学要求的收费管理系统势在必行。有幸这个项目由我们团队负责开发。 然而事情远没有想象那么简单，学分制改革是学校的大事情，需要财务处、教务处、学工部等行政部门的支持和各二级学院的配合。学分制收费更是与各个部门、学院和学生兮兮相关。试分析可以发现：待制定的学分制收费管理系统必须做到把财务处的各项收费标准信息、教务处的学生选课信息和学工部的助学贷款、缓缴学费、参军等学生信息紧密的糅合起来，并计算出学生预缴费用。由于涉及到的部门比较多，各部门领导又并不具备专业的软件知识，提出的需求并不明确或则是根本无法系统化。如果采用瀑布模式或则是演变模式进行开发，显然会存在着很大的风险，介于此、经项目组研讨决定采用快速原型开发模式进行项目开发。 【具体实施】 ㈠ 开发工具选择 经项目研讨后我们决定选择.NET平台采用ASP.NET+AJAX+SQL SERVER2000技术进行开发。主要原因是.NET平台具有一下优势： ⑴、技术领先 .Net技术于2001年由微软公司推出，与Java构成当前最主流的开发平台，.Net对XML、Web Service、AJAX提供很好的支持，而且，提供了更为便捷的开发、调试、部署环境，同时，与微软的BizTalk、Office、SQL Server2000等系统可以无缝衔接。 ⑵、安全性 .Net是构建于操作系统之上的虚拟平台，提供了更为强健的安全系统。在系统当中，提供集成Windows验证、基于角色的权限管理机制、SSL传输加密、MD5数据加密等多种安全手段，以提高系统的安全性。 ⑶、稳定性 作为24*7运行的系统，除了提供良好的性能之外，系统的稳定性也非常重要，系统采用如下方法提高系统性能及稳定性： ①Web服务器采用Windows 2003+IIS6 ②模板系统：更新不频繁的数据使用模板系统生成静态页面，减少数据库压力 ③站点缓冲：频繁更新的数据，使用缓冲以提高访问速度，减少数据库压力 ④系统日志：再好的设计都会有bug，系统日志记录程序运行过程中产生的异常，以方便调试系统，发现潜在的bug ⑷、扩展性 采集4层结构，分为数据访问层、业务逻辑层、业务外观层、表现层，各层之间严格遵守"高内聚、低耦合"原则，使系统具备较好的扩展性。 数据访问层：完成基本的CRUD（Create/Read/Update/Delete）操作。 业务逻辑层：完成各种业务规则和逻辑的实现，调用数据访问层完成CRUD操作。 业务外观层：为表示层提供统一的访问接口，分离界面和具体的业务功能。 表示层：分为B/S和C/S两中表现形式（暂时只实现了B/S一种模式）。 多层分布式设计，当业务和访问量增大时，可以在中间层部署更多的应用服务器，前端部署更多的Web服务器，提高对客户端的响应，而所有的变化对客户端是透明的。 ㈡ 项目组成员以及分工 我们项目组由一个项目负责人、一个测试工程师、一个文档管理员、三个编码员（其中一个软件设计师和两个程序员）。具体分工如下表： 成员 任务 输出文档 项目负责人 需求采集①、控制进度、协调用户关系 学分制收费研究报告 测试工程师 集成测试、总体测试 测试报告 文档管理员 编写用户手册、编写操作手册、软件服务制定 用户手册、操作手册 软件服务说明书 编码员 软件设计师：需求分析、数据库设计、软件架构、核心代码编写、配合集成测试和总体设计、任务划分、编码质量控制 需求分析报告、系统设计书、详细设计、软件规范说明书 其他两个编码员：单元代码的编码、单元测试 很荣幸我担任的是软件设计师的职务，在此感谢项目组对我的信任。另外在项目研讨的时候，根据项目开发时间紧迫、需求不好把握、需不断的构造软件原型等特点，我们打破常规，将原本属于编码员完成的集成测试任务全部划分给了测试工程师，测试工程师也只需将每次测试结果当做一种需求的方式返回给我们，我们再根据返回的需求微调程序，微调后的程序就基本上能满足要求。但这样做有个很大的前提就是测试工程师要对需求相当成熟。 ①项目负责人通过与各部门领导沟通和软件演示的方式来采集用户需求。 ㈢工作流程以时间安排 项目负责人通过与各部门领导的沟通和实际调查，初步确定了软件需求，并提交学分制收费研究报告，同时把软件的核心功能定位于“计算学生的预缴费用，并将这些数据提供给财务收费系统（以.XLS文件导入、导出）”。随后经各部门领导协商，定于4.20日正式提交软件，如果软件能满足要求则立即投入使用。时间很紧迫，为保证第二次原型开发具有充足的时间，经项目组讨论决定制定了以下的工作安排。 项目名称：学分制收费管理系统 任务安排表 任务代码 / 名称 交付的文档 人员 计划 开始 结束 工期(天) 学分制收费管理系统 2009.2.9 2009.4.19 69 T1 确定初步需求 学分制收费研究报告 项目负责人 2009.2.14 2009.2.28 14 T2 项目研讨会 项目组成员 2009.2.28 2009.3.1 1 T3系统设计 需求分析、系统设计、详细设计、系统规范说明 软件设计师 2009.3.2 2009.3.9 7 T4第一次原型构建 编码员 2009.2.9 2009.2.16 7 T5集成测试 测试报告 测试工程师 2009.3.16 2009.3.17 1 T6程序微调 编码员 2009.3.17 2009.3.18 1 T7软件演示 需求分析报告 项目负责人 2009.4.18 2009.4.19 1 T8项目研讨会 项目组全体成员 2009.3.19 2009.3.20 1 T9需求调整 软件设计师 2009.3.20 2009.3.21 1 T10第二次原型构建 编码员 2009.3.21 2009.4.4 14 T11集成测试 测试报告 测试工程师 2009.4.4 2009.4.5 1 T12程序微调 编码员 2009.4.5 2009.4.6 1 T13第二次演示 需求分析报告 项目负责人 2009.4.6 2009.4.7 1 T14项目研讨 项目组全体成员 2009.4.8 2009.4.9 1 T15软件完善 编码员 2009.4.9 2009.4.14 5 T16集成测试 测试报告 测试工程师 2009.4.14 2009.4.15 1 T17程序微调 编码员 2009.4.15 2009.4.16 1 T18总体测试 测试报告 测试工程师 2009.4.16 2009.4.18 2 T19测试微调 编码员 2009.4.18 2009.4.19 1 T20文档整理 用户手册、操作手册、软件服务说明书 文档员 2009.3.22 2009.4.12 21 T21软件提交 项目负责人 2009.4.19 2009.4.20 1 在具体的实施的工程当中，我们依照任务安排表严格执行，经过两个多月的开发，学分制收费管理系统终于完成，并在第二次软件演示的时候得到了各部门领导的一致好评。 【存在的不足】 虽然开发的系统得到了各部门领导的好评，但在整个开发工程当中仍然存在很多不足。我总结了主要有以下几点： ①某些关键的细节⊕在最开始就被忽略，这导致了后期为弥补这个细节花费了大量的时间，同时影响了队员的信心。 ②程序员经验不足，对需求的理解能力稍差，有时候开发出来的某些复杂的模块根本不能满足要求，这无形中增加了需求沟通和程序修改的时间。 ③对捕捉程度不够清晰，有时候需求过大，需要的开发时间较长，很难在预定时间内完成，只得加班加点。但有时需求较少，需要的开发时间较少，预先安排的时间有空余。这种情况使得程序员作息正常的作息时间被打乱，虽然开发进度能被很好的把握，但其实开发效率并不高。 ④第一次原型开发初期，由于时间比较紧，对编码质量没有进行很好的控制，这导致后期的开发当中常常出现一些莫名其妙的错误（比如某个模块运行时间过长）。 ⊕数据表中的某一个字段不清楚到底该如何处理时将其忽略。 【后期总结】 虽然在开发的过程当中存在一些不足，但我仍然学到了很多东西，同时也第一次正真的体验了快速原型开发模式在实践当中的应用，这次的经验在我今后的工作当中也都将产生深远的影响。在项目结束时，关于快速原型开发模式在实践当中的应用，我总结以下几点值得参考性的意见。 ①在选择项目组成员时，应该本着“少而精”的原则。 ②在软件开发之前，必须提出核心需求，进而确定软件的核心功能。 ③在软件开发之前，对开发需时进行认真评估，制定一张符合实际的任务安排表，保证队员正常作息时间。 ④在软件开发的过程当中，应严格控制原型的构建次数（建议只构建三次），一般在第一次软件演示后就应该基本确定用户需求，第二次软件演示的时就应该基本满足用户需求，第三次软件演示后再通过一些细节方面的修改就可以交付。 ⑤对于某些暂时模糊的关键性细节应予以认真记录和分析，影响力大、需及时解决的细节必须及时解决，暂时不忙解决的应将涉及到这个细节的所有功能模块放在下一次原型构造时才进行开发与解决。 ⑥如果开发时间很紧迫，测试工程师应跟踪测试，确保测试与开发同步。
-
发表了文章 2010-10-14
Windows CE6.0 S3C2440A IIC驱动编码解析
S3C2440A RISC微处理器可以支持多主设备IIC总线串行接口。专用串行总线（SDA）和串行时钟线（SCL）承载总线主机设备和连接IIC总线的外围设备之间的信息。SDA和SCL线都是双向的。本章采用TQ2440开发板进行分析，我们先来看看其硬件电路图； 从这里可以看的出 TQ2440 采用的是AT24C02A IIC器件，其中I2CSCL和I2CSDA分别表示时钟线和数据线。接下来看看IIC寄存器的相关结构体； typedef struct _I2C_CONTEXT { DWORD Sig; // Signature volatile S3C2440A_IICBUS_REG *pI2CReg; // I2C Registers volatile S3C2440A_IOPORT_REG *pIOPReg; // GPIO Ports volatile S3C2440A_CLKPWR_REG *pCLKPWRReg; // Clock / Power CRITICAL_SECTION RegCS; // Register CS I2C_MODE Mode; // State I2C_STATE State; int Status; FLAGS Flags; // Data PUCHAR Data; // pointer to R/W data buffer int DataCount; // nBytes to R/W to/from data buffer UCHAR WordAddr; // slave word address UCHAR RxRetAddr; // returned slave address on Rx DWORD SlaveAddress; // Our I2C Slave Address HANDLE DoneEvent; // I/O Done Event HANDLE ISTEvent; // IST Event HANDLE IST; // IST Thread DWORD OpenCount; DWORD LastError; HANDLE hProc; CEDEVICE_POWER_STATE Dx; } I2C_CONTEXT, *PI2C_CONTEXT; 在这个结构体中pI2CReg、pIOPReg和pCLKPWRReg都扮演着非常重要的角色，如下来看看如何实现这上个寄存器初始化工作，后面将介绍IIC驱动的几个关键部分代码； bMapReturn = VirtualCopy( pVMem,(LPVOID)(S3C2440A_BASE_REG_PA_IICBUS>>8), PAGE_SIZE,PAGE_READWRITE | PAGE_NOCACHE |PAGE_PHYSICAL); pI2C->pI2CReg = (volatile S3C2440A_IICBUS_REG*)(pVMem); pVMem += PAGE_SIZE; VirtualCopy(pVMem,(LPVOID)(S3C2440A_BASE_REG_PA_IOPORT>>8), PAGE_SIZE,PAGE_READWRITE | PAGE_NOCACHE |PAGE_PHYSICAL); pI2C->pIOPReg = (volatile S3C2440A_IOPORT_REG*)(pVMem); pVMem += PAGE_SIZE; VirtualCopy(pVMem,(LPVOID)(S3C2440A_BASE_REG_PA_CLOCK_POWER>>8), PAGE_SIZE,PAGE_READWRITE | PAGE_NOCACHE |PAGE_PHYSICAL); pI2C->pCLKPWRReg = (volatile S3C2440A_CLKPWR_REG*)(pVMem); 中断模式处理 在Windows CE当中中断处理的常用的几个函数有KernelIoControl、InterruptInitialize、InterruptDisable和InterruptDone函数，如下所示是其处理过程。 Irq = IRQ_IIC; KernelIoControl(IOCTL_HAL_REQUEST_SYSINTR, &Irq, sizeof(UINT32), &gIntrIIC, sizeof(UINT32), NULL)) // initialize the interrupt InterruptInitialize(gIntrIIC, pI2C->ISTEvent, NULL, 0) ) ; InterruptDone(gIntrIIC); // create the IST pI2C->IST = CreateThread(NULL, 0, I2C_IST, (LPVOID)pI2C, 0, NULL)); CeSetThreadPriority(pI2C->IST, I2C_THREAD_PRIORITY)); IST函数 PI2C_CONTEXT pI2C = (PI2C_CONTEXT)Context; DWORD i2cSt; BOOL bDone = FALSE; do { if (pI2C->Mode == INTERRUPT) { DWORD we; bDone = FALSE; we = WaitForSingleObject(pI2C->ISTEvent, INFINITE); // clear the interrupt here because we re-arm another below InterruptDone(gIntrIIC); switch(pI2C->State) { case OFF: DEBUGMSG(ZONE_IST|ZONE_TRACE,(TEXT("I2C_IST: ExitThread /r/n"))); ExitThread(ERROR_SUCCESS); break; case IDLE: DEBUGMSG(ZONE_IST|ZONE_TRACE,(TEXT("I2C_IST: IDLE /r/n"))); continue; break; default: if (pI2C->State != WRITE_ACK && pI2C->State != RESUME && pI2C->DataCount == INVALID_DATA_COUNT) { continue; } break; } } // EnterCriticalSection(&pI2C->RegCS); __try { switch(pI2C->State) { case IDLE: case SUSPEND: continue; break; case RESUME: InitRegs(pI2C); pI2C->LastError = ERROR_OPERATION_ABORTED; SetEvent(pI2C->DoneEvent); break; case SET_READ_ADDR: if ( (pI2C->DataCount--) == 0 ) { bDone = TRUE; break; } // write word address // For setup time of SDA before SCL rising edge, rIICDS must be written // before clearing the interrupt pending bit. if (pI2C->Flags.WordAddr) { rIICDS = pI2C->WordAddr; // clear interrupt pending bit (resume) rIICCON = RESUME_IIC_CON; pI2C->Flags.WordAddr = FALSE; } break; case READ_DATA: ASSERT(pI2C->Data); if ( (pI2C->DataCount--) == 0 ) { bDone = TRUE; *pI2C->Data = (UCHAR)rIICDS; pI2C->Data++; rIICSTAT = MRX_STOP; rIICCON = RESUME_IIC_CON; // resume operation. break; } // Drop the returned Slave WordAddr? if ( pI2C->Flags.DropRxAddr ) { pI2C->RxRetAddr = (UCHAR)rIICDS; pI2C->Flags.DropRxAddr = FALSE; } else { *pI2C->Data = (UCHAR)rIICDS; pI2C->Data++; } // The last data is read with no ack. if ( pI2C->DataCount == 0 ) { rIICCON = RESUME_NO_ACK; // resume operation with NOACK. DEBUGMSG(ZONE_READ|ZONE_TRACE,(TEXT("R1:0x%X /r/n"), r)); } else { rIICCON = RESUME_IIC_CON; // resume operation with ACK DEBUGMSG(ZONE_READ|ZONE_TRACE,(TEXT("R2:0x%X /r/n"), r)); } break; case WRITE_DATA: ASSERT(pI2C->Data); if ( (pI2C->DataCount--) == 0 ) { bDone = TRUE; rIICSTAT = MTX_STOP; rIICCON = RESUME_IIC_CON; // resume operation. //The pending bit will not be set after issuing stop condition. break; } rIICDS = (UCHAR)*pI2C->Data; pI2C->Data++; } rIICCON = RESUME_IIC_CON; // resume operation. break; } } _except(EXCEPTION_EXECUTE_HANDLER) { rIICSTAT = (pI2C->State == READ_DATA) ? MRX_STOP : MTX_STOP; rIICCON = RESUME_IIC_CON; pI2C->DataCount = INVALID_DATA_COUNT; pI2C->LastError = GetExceptionCode(); } if (bDone) { DEBUGMSG(ZONE_IST, (TEXT("SetEvent DONE/r/n"))); SetEvent(pI2C->DoneEvent); } } while (pI2C->Mode == INTERRUPT); return ERROR_SUCCESS; } 读取数据操作 驱动采用的是中断方式读取数据，其中数据指针保存在pI2C->pData当中，其中为了保证用户区缓冲和驱动内核区缓冲一致，还必须调用GetCallerProcess()、 MapPtrToProcess()和GetCurrentProcessID()函数，这几个函数的具体用法可以查询MSDN帮助即可，这部分代码如下； pI2C->State = WRITE_DATA; pI2C->DataCount = 1 + Count; // slave word address + data pI2C->WordAddr = WordAddr; pI2C->Flags.WordAddr = TRUE; pI2C->Data = pData; // write slave address rIICDS = (UCHAR)SlaveAddr; rIICSTAT = MTX_START; // IST writes the slave word address & data if (WAIT_OBJECT_0 != SyncIst(pI2C, TX_TIMEOUT)) { goto _done; } 写数据操作 写操作和读操作方法很类似，是其反过程，代码很简单，这里就不多讲了，具体在S3C2440A BSP中可以看到。 这个IIC驱动是一个典型的Windows CE流接口驱动程序，是一个很好的学习范例，特写至此，希望对来客有写帮助。
-
发表了文章 2010-10-13
用纯 C 编写 WINDOWS 控件
/* my_button.h */#ifndef _MY_BUTTON_H_#define _MY_BUTTON_H_#define BUTTON_POSE_DOWN -1#define BUTTON_POSE_NORMAL 0#define BUTTON_POSE_HOVER 1typedef struct button { RECT rect; int state; WNDPROC old_wndproc; void *old_userdata;} button_t;int button_attach_win (button_t *button, HWND hwnd);#endif my_button.c#include <windows.h>#include "my_button.h"#pragma warning(disable: 4312)#pragma warning(disable: 4311)#define WM_ATTACH_WIN(_obj,_hwnd,_new_userdata,_new_wndproc) / (_obj)->old_userdata = (void *)GetWindowLong ( (_hwnd), GWL_USERDATA); / (_obj)->old_wndproc = (WNDPROC)GetWindowLong ( (_hwnd), GWL_WNDPROC); / SetWindowLong ( (_hwnd), GWL_USERDATA, (LONG)_new_userdata); / SetWindowLong ( (_hwnd), GWL_WNDPROC, (LONG)_new_wndproc)#define WM_DETACH_WIN(_obj,_hwnd) / SetWindowLong ( (_hwnd), GWL_USERDATA, (LONG) (_obj)->old_userdata); / SetWindowLong ( (_hwnd), GWL_WNDPROC, (LONG) (_obj)->old_wndproc)LRESULT CALLBACK button_wndproc (HWND hwnd, UINT message, WPARAM wparam, LPARAM lparam){ button_t *button; PAINTSTRUCT ps; HDC hdc; button = (button_t *)GetWindowLong (hwnd, GWL_USERDATA); switch (message) { case WM_KEYDOWN: case WM_LBUTTONDOWN: button->state = BUTTON_POSE_DOWN; InvalidateRect (hwnd, NULL, FALSE); return 0; case WM_KEYUP: case WM_LBUTTONUP: // MessageBox (NULL, L"Clicked", NULL, 0); button->state = BUTTON_POSE_NORMAL; InvalidateRect (hwnd, NULL, FALSE); return 0; case WM_MOUSEMOVE: button->state = BUTTON_POSE_HOVER; InvalidateRect (hwnd, NULL, FALSE); return 0; case WM_PAINT: hdc = BeginPaint (hwnd, &ps); switch (button->state) { case BUTTON_POSE_HOVER: FillRect (hdc, &button->rect, (HBRUSH)GetStockObject (LTGRAY_BRUSH)); break; case BUTTON_POSE_NORMAL: FillRect (hdc, &button->rect, (HBRUSH)GetStockObject (DKGRAY_BRUSH)); break; case BUTTON_POSE_DOWN: FillRect (hdc, &button->rect, (HBRUSH)GetStockObject (BLACK_BRUSH)); break; default: break; } EndPaint (hwnd, &ps); break; default: break; } return button->old_wndproc (hwnd, message, wparam, lparam);}int button_attach_win (button_t *button, HWND hwnd){ WM_ATTACH_WIN (button, hwnd, button, button_wndproc); return 1;} extern "C" {#include "my_button.h"}button_t button;void CAboutDlg::OnBnClickedButton1(){ button.rect.left = 5; button.rect.top = 5; button.rect.right = 100; button.rect.bottom = 100; button_attach_win (&button, m_hWnd); Invalidate ();}
-
发表了文章 2010-10-03
IO_STACK_LOCATION 结构和处理过程
IO_STACK_LOCATION 结构 The IO_STACK_LOCATION structure defines an I/O stack location , which is an entry in the I/O stack that is associated with each IRP. Each I/O stack location in an IRP has some common members and some request-type-specific members. typedef struct _IO_STACK_LOCATION { UCHAR MajorFunction; UCHAR MinorFunction; UCHAR Flags; UCHAR Control; // // The following user parameters are based on the service that is being // invoked. Drivers and file systems can determine which set to use based // on the above major and minor function codes. // union { // // System service parameters for: NtCreateFile // struct { PIO_SECURITY_CONTEXT SecurityContext; ULONG Options; USHORT POINTER_ALIGNMENT FileAttributes; USHORT ShareAccess; ULONG POINTER_ALIGNMENT EaLength; } Create; // // System service parameters for: NtReadFile // struct { ULONG Length; ULONG POINTER_ALIGNMENT Key; LARGE_INTEGER ByteOffset; } Read; // // System service parameters for: NtWriteFile // struct { ULONG Length; ULONG POINTER_ALIGNMENT Key; LARGE_INTEGER ByteOffset; } Write; // // System service parameters for: NtQueryInformationFile // struct { ULONG Length; FILE_INFORMATION_CLASS POINTER_ALIGNMENT FileInformationClass; } QueryFile; // // System service parameters for: NtSetInformationFile // struct { ULONG Length; FILE_INFORMATION_CLASS POINTER_ALIGNMENT FileInformationClass; PFILE_OBJECT FileObject; union { struct { BOOLEAN ReplaceIfExists; BOOLEAN AdvanceOnly; }; ULONG ClusterCount; HANDLE DeleteHandle; }; } SetFile; // // System service parameters for: NtQueryVolumeInformationFile // struct { ULONG Length; FS_INFORMATION_CLASS POINTER_ALIGNMENT FsInformationClass; } QueryVolume; // // System service parameters for: NtFlushBuffersFile // // No extra user-supplied parameters. // // // System service parameters for: NtDeviceIoControlFile // // Note that the user's output buffer is stored in the UserBuffer field // and the user's input buffer is stored in the SystemBuffer field. // struct { ULONG OutputBufferLength; ULONG POINTER_ALIGNMENT InputBufferLength; ULONG POINTER_ALIGNMENT IoControlCode; PVOID Type3InputBuffer; } DeviceIoControl; // end_wdm // // System service parameters for: NtQuerySecurityObject // struct { SECURITY_INFORMATION SecurityInformation; ULONG POINTER_ALIGNMENT Length; } QuerySecurity; // // System service parameters for: NtSetSecurityObject // struct { SECURITY_INFORMATION SecurityInformation; PSECURITY_DESCRIPTOR SecurityDescriptor; } SetSecurity; // begin_wdm // // Non-system service parameters. // // Parameters for MountVolume // struct { PVPB Vpb; PDEVICE_OBJECT DeviceObject; } MountVolume; // // Parameters for VerifyVolume // struct { PVPB Vpb; PDEVICE_OBJECT DeviceObject; } VerifyVolume; // // Parameters for Scsi with internal device contorl. // struct { struct _SCSI_REQUEST_BLOCK *Srb; } Scsi; // // Parameters for IRP_MN_QUERY_DEVICE_RELATIONS // struct { DEVICE_RELATION_TYPE Type; } QueryDeviceRelations; // // Parameters for IRP_MN_QUERY_INTERFACE // struct { CONST GUID *InterfaceType; USHORT Size; USHORT Version; PINTERFACE Interface; PVOID InterfaceSpecificData; } QueryInterface; // end_ntifs // // Parameters for IRP_MN_QUERY_CAPABILITIES // struct { PDEVICE_CAPABILITIES Capabilities; } DeviceCapabilities; // // Parameters for IRP_MN_FILTER_RESOURCE_REQUIREMENTS // struct { PIO_RESOURCE_REQUIREMENTS_LIST IoResourceRequirementList; } FilterResourceRequirements; // // Parameters for IRP_MN_READ_CONFIG and IRP_MN_WRITE_CONFIG // struct { ULONG WhichSpace; PVOID Buffer; ULONG Offset; ULONG POINTER_ALIGNMENT Length; } ReadWriteConfig; // // Parameters for IRP_MN_SET_LOCK // struct { BOOLEAN Lock; } SetLock; // // Parameters for IRP_MN_QUERY_ID // struct { BUS_QUERY_ID_TYPE IdType; } QueryId; // // Parameters for IRP_MN_QUERY_DEVICE_TEXT // struct { DEVICE_TEXT_TYPE DeviceTextType; LCID POINTER_ALIGNMENT LocaleId; } QueryDeviceText; // // Parameters for IRP_MN_DEVICE_USAGE_NOTIFICATION // struct { BOOLEAN InPath; BOOLEAN Reserved[3]; DEVICE_USAGE_NOTIFICATION_TYPE POINTER_ALIGNMENT Type; } UsageNotification; // // Parameters for IRP_MN_WAIT_WAKE // struct { SYSTEM_POWER_STATE PowerState; } WaitWake; // // Parameter for IRP_MN_POWER_SEQUENCE // struct { PPOWER_SEQUENCE PowerSequence; } PowerSequence; // // Parameters for IRP_MN_SET_POWER and IRP_MN_QUERY_POWER // struct { ULONG SystemContext; POWER_STATE_TYPE POINTER_ALIGNMENT Type; POWER_STATE POINTER_ALIGNMENT State; POWER_ACTION POINTER_ALIGNMENT ShutdownType; } Power; // // Parameters for StartDevice // struct { PCM_RESOURCE_LIST AllocatedResources; PCM_RESOURCE_LIST AllocatedResourcesTranslated; } StartDevice; // begin_ntifs // // Parameters for Cleanup // // No extra parameters supplied // // // WMI Irps // struct { ULONG_PTR ProviderId; PVOID DataPath; ULONG BufferSize; PVOID Buffer; } WMI; // // Others - driver-specific // struct { PVOID Argument1; PVOID Argument2; PVOID Argument3; PVOID Argument4; } Others; } Parameters; // // Save a pointer to this device driver's device object for this request // so it can be passed to the completion routine if needed. // PDEVICE_OBJECT DeviceObject; // // The following location contains a pointer to the file object for this // PFILE_OBJECT FileObject; // // The following routine is invoked depending on the flags in the above // flags field. // PIO_COMPLETION_ROUTINE CompletionRoutine; // // The following is used to store the address of the context parameter // that should be passed to the CompletionRoutine. // PVOID Context; } IO_STACK_LOCATION, *PIO_STACK_LOCATION; IO_STACK_LOCATION 处理过程 define IoSkipCurrentIrpStackLocation( Irp ) / (Irp)->CurrentLocation++; / (Irp)->Tail.Overlay.CurrentStackLocation++; #define IoCopyCurrentIrpStackLocationToNext ( Irp ) Value: { / PIO_STACK_LOCATION irpSp; / PIO_STACK_LOCATION nextIrpSp; / irpSp = IoGetCurrentIrpStackLocation( (Irp) ); / nextIrpSp = IoGetNextIrpStackLocation( (Irp) ); / RtlCopyMemory( nextIrpSp, irpSp, FIELD_OFFSET(IO_STACK_LOCATION, CompletionRoutine)); / nextIrpSp->Control = 0; } NTSTATUS IoCallDriver( IN PDEVICE_OBJECT DeviceObject, IN OUT PIRP Irp ) { return IofCallDriver (DeviceObject, Irp); } NTSTATUS FASTCALL IofCallDriver( IN PDEVICE_OBJECT DeviceObject, IN OUT PIRP Irp ) { // // This routine will either jump immediately to IopfCallDriver, or rather // IovCallDriver. // return pIofCallDriver(DeviceObject, Irp); } NTSTATUS FASTCALL IopfCallDriver( IN PDEVICE_OBJECT DeviceObject, IN OUT PIRP Irp ) /*++ Routine Description: This routine is invoked to pass an I/O Request Packet (IRP) to another driver at its dispatch routine. Arguments: DeviceObject - Pointer to device object to which the IRP should be passed. Irp - Pointer to IRP for request. Return Value: Return status from driver's dispatch routine. --*/ { PIO_STACK_LOCATION irpSp; PDRIVER_OBJECT driverObject; NTSTATUS status; // // Ensure that this is really an I/O Request Packet. // ASSERT( Irp->Type == IO_TYPE_IRP ); // // Update the IRP stack to point to the next location. // Irp->CurrentLocation--; if (Irp->CurrentLocation <= 0) { KeBugCheckEx( NO_MORE_IRP_STACK_LOCATIONS, (ULONG_PTR) Irp, 0, 0, 0 ); } irpSp = IoGetNextIrpStackLocation( Irp ); Irp->Tail.Overlay.CurrentStackLocation = irpSp; // // Save a pointer to the device object for this request so that it can // be used later in completion. // irpSp->DeviceObject = DeviceObject; // // Invoke the driver at its dispatch routine entry point. // driverObject = DeviceObject->DriverObject; PERFINFO_DRIVER_MAJORFUNCTION_CALL(Irp, irpSp, driverObject); status = driverObject->MajorFunction[irpSp->MajorFunction]( DeviceObject, Irp ); PERFINFO_DRIVER_MAJORFUNCTION_RETURN(Irp, irpSp, driverObject); return status; }
-
发表了文章 2010-09-30
盗梦空间现实版：Linux正盗走Windows美梦
Linux一直试图在桌面操作系统上与Windows一决高低，但是多年来赢取的市场份额非常有限。事实证明，除去PC桌面，Linux还有很多可 以争取的市场空间。随着近期，诺基亚Maemo与英特尔Moblin项目合并后推出MeeGo操作系统，谷歌Android操作系统在智能手机市场取得的 进展，这些基于Linux平台的自由软件接连迸发，无疑都让人欢欣鼓舞。 同时，KDE这个14年来伴随着Linux走进普 通用户电脑桌面的图形桌面，也迎来了革命性的时刻。在2010年，KDE每年一度的Akademy国际会议上，KDE确立了两个重要的主题：扩大KDE软 件的使用而不仅限于桌面电脑上；通过社交网和云计算链接用户和数据。如今的KDE早已不局限在Linux桌面，它甚至支持世界最大的移动电话和处理器供应 商。 我们可以看见，MeeGo、Android、KDE等的灵动性正在被激活，作用正在被放大，这让基于自由软件研发 的技术人员有了用武之地，也看见光明的前景。不过，对于Linux向来的对手Windows来说可就没有那么轻松，他们会感觉原本自己坐拥的市场被瓜分 了。如果用当前流行的一部电影《盗梦空间》来比拟，Windows的美梦Linux被盗走了。 就连诺基亚公司MeeGo软件主管Valtteri Halla也声称，他不但在幻想着不满一岁的MeeGo如何将接管世界的愿景，而且还邀请KDE一道去实现这个梦想。 值得注意的是，谷歌Document、Dropbox、Facebook、Twitter以及即将发布的谷歌Chrome OS等，都已经将目光聚焦在KDE社区的互动性、操控能力和在线服务等方面。那么，KDE正在发生着哪些你意想不到的变化呢？下面，让我们一探究竟吧。 KDE走入移动设备 事实上，KDE已经将移动设备作为目标，甚至计划从KDE 4.0就开始了。随着Valtteri Halla的出现，连同其他诺基亚和英特尔的代表都对MeeGo有兴趣，这就提供了一个绝好的机会，让KDE炫耀其现有的移动通信技术，展示其未来的计划 并分享他们的经验。 MeeGo得以协同工作和促成在逆境中的项目，得益于KDE的良好性能。诺基亚甚至已经改编KDE 的Koffice应用程序以打造一款移动办公浏览程序，同时，还修复了Koffice中的Bugs和完善KOffice处理微软文件格式的功能，这实际上 也节省了诺基亚的大量时间和从头开发解决方案的费用。 KDE个人信息的管理团队的Till Adam表示，自己已经使用Kontact（Kontact是KDE里的个人信息管理软件）在自己的手机上了，在推向普通大众之前还需要拿出一个最佳版 本。对于Kontact来说，最主要的挑战就是要适应从界面到小屏幕的转换，降低处理器和内存需求，并将网络交通降到最低。 当然，最终目标是要远远超出目前可见的移动电子邮件客户端的能力。Adam认为如果电池寿命的负面影响能够进一步降低，它将可以让用户用他们的手机 做更多事情。 用桌面地球仪找到自己的方向 随着KDE 4.5版本发布，桌面地球仪（Marble）将包括用OpenStreetMap数据进行先进的本地搜索，以便用户能够找到并通过打印地址可以快速移动到 目的地。 Marble也正在获得路径规划的能力，展示在地图上点之间的航线而且提供循序渐进的指示。结合GPS定位意识，这将使Marble可以作为一个完 全自由的导航系统软件。 另外一个特性是实现让地图追踪用户当前的位置，这是目前谷歌夏季代码项目正在开发的一部分。 Marble的开发者也一直在努力使Marble移动起来。它已经可以运行在Linux、Mac OS X、Windows和MeeGo等多种系统上。Marble的开发者已经实现了这种功能，下载OpenStreetMap数据以供将来使用，特别是当一个 互联网连接是无效的或昂贵的时候，使本地地图在高分辨率下依然是可用的。 在手机上的等离子 听说过等离子电视，但是没有听说过手机与等离子拉上关系。研发人员正在研究一个新的领域，等离子正在被开发放在智能手机上使用。Artur Duque de Souza和Alexis Menard德斯·索萨声称，这样的状况距离我们还远。 虽然，为手机制作KDE界面的一些实验性工作始于2009年，但是，官方开始等离子手机项目是在2月份。 等离子体手机的目标是在界面中，拥有更强大的定位和上下文感知。例如，当用户在步行、用社交网络的时候，游戏将不会在主页面上出现，从而避免你在工 作的时候受到干扰。 Marco Martin对当前用作移动网络设备的等离子上网本作了一个概述。等离子上网本带着KDE4.4甚至是，更稳定、更加完美的KDE 4.5亮相，伴随着流畅的动画和更多的数据缓存地广泛采用，能够减少处理和网络的要求。 Alessandro Diaferia展示他在即将到来的等离子媒体中心的工作，这个中心的目的是要在基于KDE技术的统一界面里，完成看视频、听音乐和查看照片等基本功能。 所有在等离子媒体中心的东西都是一个等离子部件，这意味着任何组件都可以根据个人喜好更换，包括播放器。多种部件能够同时使用，所以，你可以很容易 地边看假日快照边听最喜爱的音乐专辑。 元数据将用KDE的语义桌面层来跟踪，这样它就可以在传统KDE桌面应用和媒体中心之间分享标签和等级信息。网络服务也将被整合，使用户在不离开界 面的情况就能够直接访问来自YouTube和Flickr的内容。 最新的一项技术预演应该可以在2010年的秋季与我们见面，而且1.0版本也将在2011年的KDE 4.6中见到。 将Web带到桌面上 Sebastian Kügler参加了Akademy网站和云计算的集会，参与讨论要怎样做，才能够让KDE软件，在不丢弃14年来工作的桌面应用软件，还可以充分利用网络 服务 他解释一些基于Web的应用程序当前面临的问题：他们依赖于网络连接而工作，并有一致的用户界面，而且因为他们需要工作在更大范围的设备类型，所以 他们未能充分利用其力量和大部分家用电脑的大屏幕。 要克服这些问题，Sebastian的建议是将数据从外观中分离出来。数据可以存储在云中，但应使用本地缓存以支持脱机使用，而外观可能被KDE应 用程序处理。 在KDE里，这些概念被称为丝绸项目而且统一存在KDE技术里。这些包括用Nepomuk（KDE的语义桌面层）的索引网页和用 Akonadi(KDE的数据存储引擎)使RSS解析可以关闭。 Sebastian也提议，通过KDE的Dragon视频播放器读取YouTube视频，并且在Gwenview图像查看器中浏览和标注 Flickr图象。这些幻想已经开始成为现实。 在KDE应用与网络服务的互动是好的，但对于自由软件的倡导者来说，封闭的服务是一个问题。 Frank Karlitschek，在社交网络和内容共享网站的OpenDesktop家庭背后的力量 (包括KDE-Apps，KDE-Look和Gnome-Look)，已将注意力转向了云。 他最近一个在KDE内的项目，如我们所知是OwnCloud，旨在提供任何人都可以安装的免费的云计算系统。 你可能想知道为什么会有人不怕麻烦去安装自己的服务器，但对于Karlitschek来说，动机是清晰的。它不仅促进自由软件，防止自由桌面对于网 页服务的所有者来说变成了一个界面而已，但它也给用户完全控制他们的数据和特征的加密选项。 对于个人来说，保护他们的隐私是很重要，但处理敏感数据对于公司来说又是必不可少的。它也容易掌握变化，因为OwnCloud采用 FreeDesktop.org开放合作服务标准的通知，结合KDE的通知系统。 Koffice开发基于Web的开放文档格式编辑器的工作已经在进行中了，所以它将有可能在常见的Koffice界面中编辑文档或经由浏览器。协同 编辑的功能也计划在未来加进来。 OwnCloud 1.1的发布时间最晚在2010年，共享的数据将被激活，用来整合基于网页的画廊和音乐服务的插件都将是可用的。在未来的发展中，进一步文件的版本(可能 是基于Git)将会增加。 第4页：呼吁社区去寻求统一 虽然移动应用程序和网络服务一体化是KDE发展令人兴奋的新领域，贡献者仍然没有失去将关注的焦点放在传统的桌面计算机。 5月份，KOffice发布其2.2版本，第一个KDE 4.x版本被认为适合实际工作。Inge Wallin提出了未来KOffice的研究方向而且展望了即将来临的特点，包括对1.2版本OpenDocument格式的支持。 他还讨论了在Koffice中，对于 特性的利用，这使得任何一种应用对于所有人都是可以使用的。用这种方法，KPresenter简单地从KWord嵌入到文本中。从Krita(像素编辑图 片)和Karbon14(矢量图)。KOffice应用程序也被演示运行在Windows环境下。 呼吁社区去寻求统一 Koffice的绘画应用程序，Krita，特点是以一个隔离的状态显示。Lukas Tvrdy做了一个产品演示，用刷子引擎做自然绘画。Tvrdy一直广泛地致力于此，他非常感谢收到来自于社区吸引来的资金。 长期支持KDE的开发商以及福音传道者Aaron Seigo表示，KDE已经受欢迎多年，然后列举了一些它的成功之处，包括持续部署5千万在巴西学校电脑桌面上，以及获得数万额外部署在大学校园里。 KDE桌面的部署在葡萄牙几乎翻了一倍从4到7万笔记本电脑，还有一大堆KDE部署在委内瑞拉，而且在德国全球范围内的大使馆里，KDE软件用于计 算机已达11000个。 Seigo呼吁社区去寻求共识，而不是总是努力追求一致意见，为了达成一致的行动路线而迫使他们追求快捷。最重要的是，在每件事情上，KDE都要做 得简洁。 他敦促应用程序开发者检查其接口，让它们更直观和一致。图书馆的开发者应该在API上做同样的事情，他补充说。应该减少“行话”和尽量少得弹出错误 警告（除非最紧急的错误）。 在短短的时间内，Seigo主要思想产生的影响是可以看见的，作为开发人员调整他们的接口，探讨最佳的方式做事，经常请Seigo评估他们的作品是 否简洁。 Michael Leupold曾提出KDE和Gnome之间合作开发一个统一标准来存储机密。目的是KDE和Gnome应用程序都能够分享一个公共的机密结构，但是至 今，仍然是分开的图形界面。 如果他们需要揭开一个在Empathy的账户，一个KDE用户将被提供一个KDE界面，而Gnome用户将看到Gnome界面用于密码管理，即使他 们喜欢用KDE的Kopete聊天。我们希望这个标准也能吸引其他软件的供应商，例如，Mozilla。 小结： Linux源源不断的技术和功能的创新皆来自于社区，所以，一旦Linux能够获得更大范围的统一，那么 将焕发出巨大的能量，Linux想要盗取Windows的美梦将不再困难。
暂无更多信息
-
发表了文章
2008-12-13
在c#使用Windows IOCP（完成端口）编程研究
-
发表了文章
2008-12-20
关于C#-委托和事件研究
-
发表了文章
2008-12-20
使用异步服务器套接字编程指南（微软MSDN）
-
发表了文章
2009-01-09
[朱总]My Inferiority Complex
-
发表了文章
2009-01-14
SQL SERVER2000中订阅与发布的具体操作收藏
-
发表了文章
2009-01-14
MFC线程处理概述
-
发表了文章
2009-02-04
Window Mobile RDA数据库操作解决方案研究
-
发表了文章
2009-02-04
eMbedded Visual C++开发入门
-
发表了文章
2009-02-04
一名风险投资人总结的创业者们十大迷思
-
发表了文章
2009-02-04
给同学们了一趟C#数据库操作课程
-
发表了文章
2009-02-13
Mathematical Contest in Modeling---Traffic Circle Problem
-
发表了文章
2009-02-21
堆和栈的区别在哪里？
-
发表了文章
2009-02-23
[罗振辉]MFC中菜单命令路由方式
-
发表了文章
2009-02-24
u盘编程技巧
-
发表了文章
2009-02-24
C#中文件与文件夹相关的操作
-
发表了文章
2009-02-24
Win32串口编程(VC++6.0)
-
发表了文章
2009-03-27
重载与覆盖的区别
-
发表了文章
2009-04-08
航天定票管理系统(上千行代码)
-
发表了文章
2009-04-08
完善的学生成绩计算统计
-
发表了文章
2009-04-08
C/C++模仿小灵通系统
滑动查看更多
暂无更多信息
暂无更多信息