MySQL的CSV引擎应用实例解析

本文涉及的产品
RDS MySQL Serverless 基础系列,0.5-2RCU 50GB
云数据库 RDS MySQL,高可用系列 2核4GB
云数据库 RDS PostgreSQL,高可用系列 2核4GB
简介:
如果您想把EXCEL的数据或者CSV格式的数据导入到MySQL中,MySQL的CSV引擎再适合不过了。
MySQL的CSV引擎在5.0后开始提供,不过不支持WINDOWS,到了5.1才支持。
今天我测试的版本号是5.0.45
一、注意几点:
1、没有索引,跟MySQL5的数据字典库一样。
2、可以直接用任何文本编辑器来编辑数据文件。
3、非英文编码问题。
我的字符终端和表都是UTF-8的,所以要把上传的CSV文件保存为UTF-8的编码。
4、编码转化工具,我这边在WINDOWS下用EDITPLUS来转化,在LINUX下可以用ICONV命令行工具来转化编码。
二、示例数据文件。
"1","Designed for 99.999% Availability","MySQL Cluster provides a fault tolerant architecture that ensures your organization's mission critical applications achieve 99.999% availability. This means less than 5 minutes downtime per year, including scheduled maintenance time. MySQL Cluster implements automatic node recoverability to ensure an application automatically fails over to another database node that contains a consistent data set, if one or more database nodes fail. Should all nodes fail due to hardware faults for example, MySQL Cluster ensures an entire system can be safely recovered in a consistent state by using a combination of checkpoints and log execution. Furthermore, MySQL Cluster ensures systems are available and consistent across geographies by enabling entire clusters to be replicated across regions."
"2","High Performance Only a Main Memory Database Can Deliver","MySQL Cluster provides the response time and throughput to meet the most demanding high volume enterprise applications. MySQL Cluster achieves its performance advantage by being a main memory clustered database solution, which keeps all data in memory and limits IO bottlenecks by asynchronously writing transaction logs to disk. MySQL Cluster also enables servers to share processing within a cluster, taking full advantage of all hardware. Typical response times for MySQL Cluster are in the range of a few milliseconds and MySQL Cluster has been proven to handle tens of thousands of distributed transactions per second that are also replicated across database nodes."
"3","Extremely Fast Automatic Failover","MySQL delivers extremely fast failover time with sub-second response so your applications can recover quickly in the event of application, network or hardware failure. MySQL Cluster uses synchronous replication to propagate transaction information to all the appropriate database nodes so applications can automatically fail over to another node extremely quickly. This eliminates the time consuming operation of recreating and replaying log files required by 'Shared-Disk' architectures to fail over successfully. Plus, MySQL Cluster database nodes are able to automatically restart, recover, and dynamically reconfigure themselves in case of failures without having to program advanced features into the application."
"4","Flexible Distributed Architecture with No Single Point of Failure","The parallel server architecture combines database nodes, management server nodes, and application nodes that can be distributed across computers and geographies to ensure there is no single point of failure. Any node can be stopped or started without stopping the applications that use the database. And MySQL Cluster is highly configurable so you can implement the appropriate level of performance, scalability and fault tolerance to match your application requirements."
"5","Significantly Reduce Costly Downtime","MySQL Cluster not only lowers up-front license costs with affordable commercial licensing under a dual licensing mechanism, but it also significantly reduces system downtime - the number one contributor to the Total Cost of Ownership (TCO) of database software. Furthermore, a highly portable standards-based environment allows you to cost-effectively distribute your applications using commodity hardware and open source software infrastructure."
"6","Lower Maintenance Costs","MySQL Cluster is designed to be largely self-governing so very few system parameters actually need fine-tuning, further reducing the risk of costly errors. As a result, there are typically fewer conflicts with other software and hardware, and less need for manual intervention. This also means that MySQL Cluster will have a much lower maintenance costs, with less fine tuning required by Database Administrators."
"7","Easy-to-use Administration","MySQL Cluster includes easy to use and powerful tools for administering your clustered environment. Command line tools enable you to monitor database nodes, control access to applications, and create and restore backups."
"8","Services and Support","MySQL provides extensive consulting, training and technical support services to ensure the success of your next mission-critical database application project. MySQL has a proven track record gained through millions of successful customer deployments that can lower your risk and maximize return on investment."
"9","MySQL CLUSTER","以上是集群的特性概括"
"10","月亮他爸","我博客地址:[url]http://yueliangdao0608.cublog.cn[/url]"
"11","最要注意的一点","最后的一行必须有个空格!如果是中文,好奇怪!"


三、直接COPY数据文件到MySQL数据库目录下。
1、用EDITPLUS转码,然后进入MySQL命令行。
mysql> \C gbk 
Charset changed
mysql> flush tables;
Query OK, 0 rows affected (0.00 sec)
mysql> select sql_calc_found_rows * from ytt order by id desc limit 2\G
*************************** 1. row ***************************
     id: 11
  title: 最要注意的一点
summary: 最后的一行必须有个空格!如果是中文,好奇怪!"
*************************** 2. row ***************************
     id: 10
  title: 月亮他爸
summary: 我博客地址:[url]http://yueliangdao0608.cublog.cn[/url]"
2 rows in set (0.00 sec)

mysql> select found_rows();
+--------------+
| found_rows() |
+--------------+
|           11 | 
+--------------+
1 row in set (0.00 sec)
2、用ICONV转码
现在可以直接用EXCEL来编辑数据文件,不过可惜的是EXCEL的编码是CP936的,所以传上去后要用ICONV来转码。

[root@localhost t_girl]# iconv -f CP936 -t UTF-8 ytt.CSV -o ytt.new
[root@localhost t_girl]# chown mysql:mysql ytt.new 
[root@localhost t_girl]# cp -uf ytt.new ytt.CSV 
[root@localhost t_girl]# ls -sihl
total 56K
13797305 8.0K -rw-rw---- 1 mysql mysql   61 Apr 30 15:19 db.opt
30539777 8.0K -rw-rw---- 1 mysql mysql  383 Jun 19 09:54 show_user.frm
13797308  12K -rw-r--r-- 1 mysql mysql 4.5K Aug  9 14:59 ytt.CSV
13797306  16K -rw-rw---- 1 mysql mysql 8.5K Aug  8 17:39 ytt.frm
13797307  12K -rw-r--r-- 1 mysql mysql 4.5K Aug  9 14:57 ytt.new
[root@localhost t_girl]# sed -i 's/^/"/g' ytt.CSV 
[root@localhost t_girl]# sed -i 's/,/","/g' ytt.CSV 
[root@localhost t_girl]# sed -i 's/$/"/g' ytt.CSV 

用SED替换是为了用双引号来分割各个字段。






本文转自 david_yeung 51CTO博客,原文链接:http://blog.51cto.com/yueliangdao0608/92045,如需转载请自行联系原作者

相关实践学习
每个IT人都想学的“Web应用上云经典架构”实战
本实验从Web应用上云这个最基本的、最普遍的需求出发,帮助IT从业者们通过“阿里云Web应用上云解决方案”,了解一个企业级Web应用上云的常见架构,了解如何构建一个高可用、可扩展的企业级应用架构。
MySQL数据库入门学习
本课程通过最流行的开源数据库MySQL带你了解数据库的世界。   相关的阿里云产品:云数据库RDS MySQL 版 阿里云关系型数据库RDS(Relational Database Service)是一种稳定可靠、可弹性伸缩的在线数据库服务,提供容灾、备份、恢复、迁移等方面的全套解决方案,彻底解决数据库运维的烦恼。 了解产品详情: https://www.aliyun.com/product/rds/mysql 
相关文章
|
1月前
|
SQL 存储 关系型数据库
MySQL内存引擎:Memory存储引擎的适用场景
MySQL Memory存储引擎将数据存储在内存中,提供极速读写性能,适用于会话存储、临时数据处理、高速缓存和实时统计等场景。但其数据在服务器重启后会丢失,不适合持久化存储、大容量数据及高并发写入场景。本文深入解析其特性、原理、适用场景与限制,并提供性能优化技巧及替代方案比较,助你合理利用这一“内存闪电”。
|
1月前
|
SQL 监控 关系型数据库
MySQL事务处理:ACID特性与实战应用
本文深入解析了MySQL事务处理机制及ACID特性,通过银行转账、批量操作等实际案例展示了事务的应用技巧,并提供了性能优化方案。内容涵盖事务操作、一致性保障、并发控制、持久性机制、分布式事务及最佳实践,助力开发者构建高可靠数据库系统。
|
3月前
|
关系型数据库 MySQL 分布式数据库
安全可靠的PolarDB V2.0 (兼容MySQL)产品能力及应用场景
PolarDB分布式轻量版采用软件输出方式,能够部署在您的自主环境中。PolarDB分布式轻量版保留并承载了云原生数据库PolarDB分布式版技术团队深厚的内核优化成果,在保持高性能的同时,显著降低成本。
|
1月前
|
关系型数据库 MySQL 数据库
阿里云数据库RDS费用价格:MySQL、SQL Server、PostgreSQL和MariaDB引擎收费标准
阿里云RDS数据库支持MySQL、SQL Server、PostgreSQL、MariaDB,多种引擎优惠上线!MySQL倚天版88元/年,SQL Server 2核4G仅299元/年,PostgreSQL 227元/年起。高可用、可弹性伸缩,安全稳定。详情见官网活动页。
|
1月前
|
关系型数据库 分布式数据库 数据库
阿里云数据库收费价格:MySQL、PostgreSQL、SQL Server和MariaDB引擎费用整理
阿里云数据库提供多种类型,包括关系型与NoSQL,主流如PolarDB、RDS MySQL/PostgreSQL、Redis等。价格低至21元/月起,支持按需付费与优惠套餐,适用于各类应用场景。
|
1月前
|
存储 关系型数据库 MySQL
介绍MySQL的InnoDB引擎特性
总结而言 , Inno DB 引搞 是 MySQL 中 高 性 能 , 高 可靠 的 存 储选项 , 宽泛 应用于要求强 复杂交易处理场景 。
77 15
|
2月前
|
存储 关系型数据库 MySQL
MYSQL数据加密压缩函数应用实战指南。
总的来说,加密和压缩是维护MySQL数据库中数据安全性和效率的有效手段。使用时需权衡性能与安全,合理应用加密和压缩函数。在设计数据库架构时要考虑到加密字段的查询性能,因为加密可能使得一些索引失效。压缩数据能有效减少存储空间的占用,但在服务器负载较高时应避免实时压缩和解压,以免影响总体性能。
113 10
|
2月前
|
存储 SQL 关系型数据库
MySQL中binlog、redolog与undolog的不同之处解析
每个都扮演回答回溯与错误修正机构角色: BinLog像历史记载员详细记载每件大大小小事件; RedoLog则像紧急救援队伍遇见突發情況追踪最后活动轨迹尽力补救; UndoLog就类似时间机器可倒带历史让一切归位原始样貌同时兼具平行宇宙观察能让多人同时看见各自期望看见历程而互不干扰.
173 9
|
1月前
|
关系型数据库 MySQL 数据库
阿里云数据库RDS支持MySQL、SQL Server、PostgreSQL和MariaDB引擎
阿里云数据库RDS支持MySQL、SQL Server、PostgreSQL和MariaDB引擎,提供高性价比、稳定安全的云数据库服务,适用于多种行业与业务场景。
|
3月前
|
存储 SQL 关系型数据库
MySQL 核心知识与索引优化全解析
本文系统梳理了 MySQL 的核心知识与索引优化策略。在基础概念部分,阐述了 char 与 varchar 在存储方式和性能上的差异,以及事务的 ACID 特性、并发事务问题及对应的隔离级别(MySQL 默认 REPEATABLE READ)。 索引基础部分,详解了 InnoDB 默认的 B+tree 索引结构(多路平衡树、叶子节点存数据、双向链表支持区间查询),区分了聚簇索引(数据与索引共存,唯一)和二级索引(数据与索引分离,多个),解释了回表查询的概念及优化方法,并分析了 B+tree 作为索引结构的优势(树高低、效率稳、支持区间查询)。 索引优化部分,列出了索引创建的六大原则
102 2

热门文章

最新文章

推荐镜像

更多