Overview
With the transformation of the financial services market, the demand for core technology for conducting financial transactions has grown. The financial transaction model has changed from traditional manual bidding to matchmaking orders through an electronic transaction system. This article aims to give a detailed background on the changing financial transaction, and the matchmaking technology that is driving these changes.
Background: The Evolving Financial Transactions
The traditional financial transactions mainly occur in the tangible financial market. The contracting parties of financial transactions reach an agreement through bidding, price negotiation, and other methods. Meanwhile, the transaction is complete upon making a physical delivery at the designated point as stipulated in the trade order.
Since humans primarily execute the entire transaction process, the disadvantages of traditional financial transactions include low efficiency and speed, restrictions on transaction time venue, high transaction costs, and security threats like insider transactions, poor transaction expansibility, and man-made errors.
With the development of technology, electronic transactions are preferred for financial trades. They have not only eliminated the challenges associated with the traditional transaction methods but also led to the rapid development of the modern financial industry. Key advantages of electronic financial transactions include high transaction efficiency and speed, high system security, no limits on transaction time and venue, and multi-directional expansion.
Therefore, the electronic transaction has become the mainstream financial transaction method. With the constant increase of transaction population and quantity, the system is subject to growing pressure. In the case of system faults during the transaction, the resulting loss is often inestimable. As such, a more reliable and efficient electronic transaction system is urgently needed.
Matchmaking transactions play a very important role in the financial transaction system. To design a matchmaking system, it is crucial to understand the nature and business of matchmaking transactions.
Financial Transactions: Overall System Design
Hierarchical Design
The financial transaction matchmaking system includes the following core modules:
●User: the end user is entrusted with providing the quotation and quantity, with the next being order generation, and sending the order to the transaction platform.
●Gateway: collects and distributes the user's order to the matchmaking engine.
●Matchmaking engine: it is the core of the transaction system. It receives and fulfills the order by following the business logic, matches and generates the transaction record, and feeds the transaction results back to the user.
●Database: used to save order and transaction data in the transaction process, and realize data persistence.
Matchmaking modules exist in different business divisions depending on the type of financial transactions. Each division is subject to independent matchmaking and hence immune from other divisions.
Hierarchical Design 1: Matchmaking Transaction Algorithm
As shown in Figure 1, the core business module of a matchmaking engine is the matchmaking transaction algorithm. Its goal is to execute the matchmaking function for the customer's order, and to ensure fairness, high efficiency and expandability of the algorithm. Since the matchmaking business varies for different financial transaction systems, this section outlines the commonly used algorithms.
Figure 1
Hierarchical Design 2: Order Queue
An important part of transaction matchmaking is the sale or purchase of orders. Creation of transaction records occurs by matching the selling and buying orders. Therefore, if the orders are not matched immediately, the system should save the data in a buying queue and a selling queue. The queue shall adhere to the principle of "price first," and "time first" in the case of the same price. The buying queue arrangement follows the sequence from low to high according to the entrusted prices, and the selling queue arrangement follows the sequence from low to high according to the assigned prices as shown in the figure below.
Figure 2
Matchmaking Sequence
When receiving a new buying order, the matchmaking engine will search from the beginning of the selling queue to confirm if any selling orders comply with the price discipline. All orders with a selling price lower or equal to the buying price are extracted from the queue and matched into a transaction. If the selling queue is empty or the queue head fails to meet the price relationship, the system inserts the buying order into the buying queue. It will then arrange and sort the newly inserted order in the buying queue.
Correspondingly, when receiving a new selling order, the matchmaking engine will search the beginning of the buying queue to confirm if any buying orders comply with the price discipline. All orders with buying prices higher or equal to the selling price are extracted from the order queue and matched into a transaction. If the buying queue is empty or the queue head fails to meet the price relationship, the system inserts the selling order into the selling queue. Since the selling queue has two parameters, price and time, both do sorting and insert the new order accordingly [23].
Figure 3
In combination with the selling or buying orders, Figure 3 shows the process followed by the matchmaking algorithm. According to the sequence illustrated in Figure 3, the order of the buying and selling queues is the basis for ensuring certainty in the algorithm's output.
Hierarchical Design 3: Memory Matchmaking
The poor performance of the database matchmaking technology is due to excessive interactions with the database, which leads to high I/O transactions. Meanwhile, the database transaction logic seems to restrict the overall processing speed of the system.
Hence this paper discusses the memory-based matchmaking technology. After minimizing database interaction, the system implements the matchmaking logic in the memory (as shown in Figure 4). Therefore, there are less I/O interactions in comparison with database matchmaking, and the performance is improved significantly. Memory volatility is the main disadvantage of memory matchmaking. In the case of accidental shutdown of the server, all transaction data is lost, and the reliability and consistency of the system reduce dramatically. Consequently, multi-machine hot standby and distributed consistency technology improve the reliability of memory matchmaking technology. They realize the high performance of memory matchmaking and data persistence of database matchmaking technology.
Figure 4
Hierarchical Design 4: Multi-Machine Hot Standby
The multi-machine hot standby technology improves fault tolerance because memory matchmaking shows indigent reliability and consistency when the matchmaking engine is abnormal. Furthermore, the financial transaction system has extremely low tolerance for interrupted service and data loss due to business operations being affected.
With the help of multi-machine hot standby, the system deploys a group of matchmaking engines into a mutual standby matchmaking engine cluster. Additionally, there is only one engine in service at any given time. When one of the operating engines breaks down and ceases to work, the engine cluster detects the fault and selects a standby engine to take over the task of the faulty engine. The adoption of multi-machine hot standby technology stems from the need to avoid long-term matchmaking interruption to ensure reliable services of the system. As shown in Figure 5, several matchmaking engines conduct hot standby to make sure that when the host matchmaking engine breaks down, the switchover between the host and the standby units occur within an acceptable time, with the standby unit guaranteeing seamless service.
Figure 5
The risk of an unavailable system from a single engine fault has significantly reduced. However, it remains hard to provide hundred percent availability due to service unavailability during large-scale failures in the server cluster. In the actual production environment, three mutual backup servers can provide higher reliability.
Hierarchical Design 5: Memory State Machine Replication
Since several mutual hot standby matchmaking engines are involved, it is necessary to ensure data consistency among servers according to the matchmaking system design and logic requirements. Therefore, consistency among several servers is essential.
The matchmaking algorithm acts as a certainty state machine, tailored into multiple copies and deployed to numerous matchmaking engines in the system. Each copy operates from the same initial status. While the system receives the order sent by the gateway, it also matches such orders, successively producing the transaction record, and updating the independent state of the algorithm.
Through this method, when the system operates normally, each matchmaking engine copy has the same result. In the event of a fault or exception in the system, the engine will is inconsistent. In other words, the system deems any inconsistent results as an exception.
Key Technologies
Following are the major technology divisions of the matchmaking system:
1.The state machine service for the certainty matchmaking algorithm is often deployed into several independent engines.
2.Received gateway orders serve as the input of the state machine for the certainty matchmaking algorithm.
3.According to its demand, you should select the order sorting method.
4.Each engine will match the order already sorted.
5.The transaction record output by the state machine for certainty matchmaking algorithm will serve as the response to the user or the database.
6.Monitor the status or output difference of the engine copy.
Implementation Scheme
To implement the matchmaking system replicated based on the memory status, the following schemes help realize the key technologies of state machine replication:
●The atomic multicast will solve the reliable multicast and global ordering of the engine order.
●The use of assembly line matchmaking technology based on the unlocked order queue to realize rapid order matchmaking.
●The realization of database interactions by asynchronous consistent persistence technology [3]
●The use of invalid redundancy technology to monitor the status of the engine cluster to ensure the fault tolerance of the system; [24] [3115) progress catching-up technology is used to solve the problems of recovering the fault matchmaking engine or adding a new engine.
Hierarchical Design 6: System Architecture
System Hardware Architecture
Figure 6 illustrates a standard performance matchmaking model's hardware architecture. The system is composed of N server sets, N gateway sets, X product clusters (each cluster consists of between 2 to 3 matchmaking engines, and responsible for response processing of product orders), one transaction record database and optional monitoring system. The client has an established connection to the corresponding gateway. The gateway is responsible for receiving orders submitted by the client and forwarding them to the relevant product cluster according to the financial product category of the order. All engines in the project cluster will receive the order sent by the gateway, match the order according to the matchmaking business rules, and relay the feedback message to the gateway and the client. Meanwhile, the transaction record generated by matchmaking will become part of the transaction record database.
Figure 6
By enlarging the product cluster and increasing the quantity of the engine, you can strengthen the reliability of the product cluster. It will forward different financial products to different matchmaking product clusters to realize efficient parallel matchmaking.
System Software Architecture
Figure 7
As shown in Figure 7, the elements that make up the high reliability and performance matchmaking model include the presentation layer, the forwarding layer, the business layer and the data layer. Matchmaking engine clustering mainly has the business layer at its core. Each engine is subject to the matchmaking process after order sequencing is done by the atomic multicast. A subsequent combination with unlocked order queues then delivers efficient assembly line matchmaking. Finally, the local log displays the results. In the entire business process, the messaging bus plays the role of returning the message to the forwarding layer. The system forwards the order to the corresponding engine cluster according to the product forwarding rules. The engine then reads the transaction record in the local log, writes it into process communication along with the asynchronous persistence on the data layer, and finally makes it persistent in the database. The local log strengthens the reliability of the system data. After a fault occurs, it is possible to recover the data from the local log; asynchronous persistence mechanisms improve the data persistence throughput rate.
Matchmaking Engine Architecture
Figure 8
To improve system maintenance, the matchmaking engine has an atomic multicast order sequencing module, a matchmaking processor module, and a transaction record log module along with memory data. Each module has inbuilt divisions designed according to the functional business. Each part has the following functions:
●Transaction order receiving thread: receive orders from the gateway and finish the multicast order sequencing process.
●Sending thread of trade order: send the sequenced order to relevant matchmaking business thread.
●Sending thread of trade information: provide the feedback about the status of order transaction to the gateway.
●Logic thread of peripheral activities: conduct standby processes for matchmaking data, and update memory order data.
●Logic thread of the matchmaking business: match the received orders according to the certainty matchmaking algorithm.
●Release the threads of exchange quotation: process memory exchange quotation information and release it to the gateway.
●Writing thread of synchronization logs: synchronously make the transaction record produced by order matchmaking persistent into a local log file.
●Agent process of asynchronous persistence: asynchronously read out the data in the log document and make the data persistent into a database.
●Order information: save the relevant price, quantity, user, limitation, type, status and other information of the order.
●Exchange quotation information: exchange quotation information in the process of the transaction.
System Interface
The matchmaking system mainly provides ordering and inquiry service, real-time feedback function of the exchange quotation and monitoring and checking service for the system status. As such, the reserved interfaces that the system ought to realize mainly include the ordering interface, order inquiry interface, quotation inquiry interface, system control interface, and operating state inquiry interface.
Conclusion
Starting with the overall design, the system transfers the matchmaking transaction processing from the database into the memory. Meanwhile, the multi-machine hot standby technology solves the volatility problem of memory matchmaking technology. Finally, a different approach suggests that the memory state machine replication scheme can serve as the route to realizing a high-reliability and high-performance matchmaking system.
With the transformation of the financial services market, the demand for core technology for conducting financial transactions has grown. The financial transaction model has changed from traditional manual bidding to matchmaking orders through an electronic transaction system. This article aims to give a detailed background on the changing financial transaction, and the matchmaking technology that is driving these changes.
