《MongoDB极简教程》第一章 NoSQL简史 & MongoDB安装&环境配置

本文涉及的产品
云数据库 MongoDB,通用型 2核4GB
简介: MongoDB 是一款开源的文档数据库,并且是业内领先的 NoSQL 数据库,用 C++ 编写而成。NoSQL(NoSQL = Not Only SQL ),意即"不仅仅是SQL"。

MongoDB 是一款开源的文档数据库,并且是业内领先的 NoSQL 数据库,用 C++ 编写而成。

NoSQL

(NoSQL = Not Only SQL ),意即"不仅仅是SQL"。

在现代的计算系统上每天网络上都会产生庞大的数据量。
这些数据有很大一部分是由关系数据库管理系统(RDMBSs)来处理。 1970年 E.F.Codd's提出的关系模型的论文 "A relational model of data for large shared data banks",这使得数据建模和应用程序编程更加简单。

通过应用实践证明,关系模型是非常适合于客户服务器编程,远远超出预期的利益,今天它是结构化数据存储在网络和商务应用的主导技术。

NoSQL 是一项全新的数据库革命性运动,早期就有人提出,发展至2009年趋势越发高涨。NoSQL的拥护者们提倡运用非关系型的数据存储,相对于铺天盖地的关系型数据库运用,这一概念无疑是一种全新的思维的注入。

RDBMS vs NoSQL
RDBMS

  • 高度组织化结构化数据
  • 结构化查询语言(SQL) (SQL)
  • 数据和关系都存储在单独的表中。
  • 数据操纵语言,数据定义语言
  • 严格的一致性
  • 基础事务
    NoSQL
  • 代表着不仅仅是SQL
  • 没有声明性查询语言
  • 没有预定义的模式
    -键 - 值对存储,列存储,文档存储,图形数据库
  • 最终一致性,而非ACID属性
  • 非结构化和不可预知的数据
  • CAP定理
  • 高性能,高可用性和可伸缩性

NoSQL 简史

NoSQL一词最早出现于1998年,是Carlo Strozzi开发的一个轻量、开源、不提供SQL功能的关系数据库。

2009年,Last.fm的Johan Oskarsson发起了一次关于分布式开源数据库的讨论[2],来自Rackspace的Eric Evans再次提出了NoSQL的概念,这时的NoSQL主要指非关系型、分布式、不提供ACID的数据库设计模式。

2009年在亚特兰大举行的"no:sql(east)"讨论会是一个里程碑,其口号是"select fun, profit from real_world where relational=false;"。因此,对NoSQL最普遍的解释是"非关联型的",强调Key-Value Stores和文档数据库的优点,而不是单纯的反对RDBMS。

CAP定理(CAP theorem)

在计算机科学中, CAP定理(CAP theorem), 又被称作 布鲁尔定理(Brewer's theorem), 它指出对于一个分布式计算系统来说,不可能同时满足以下三点:

一致性(Consistency) (所有节点在同一时间具有相同的数据)
可用性(Availability) (保证每个请求不管成功或者失败都有响应)
分隔容忍(Partition tolerance) (系统中任意信息的丢失或失败不会影响系统的继续运作)

CAP理论的核心是:一个分布式系统不可能同时很好的满足一致性,可用性和分区容错性这三个需求,最多只能同时较好的满足两个。
因此,根据 CAP 原理将 NoSQL 数据库分成了满足 CA 原则、满足 CP 原则和满足 AP 原则三 大类:

CA - 单点集群,满足一致性,可用性的系统,通常在可扩展性上不太强大。
CP - 满足一致性,分区容忍性的系统,通常性能不是特别高。
AP - 满足可用性,分区容忍性的系统,通常可能对一致性要求低一些。
cap-theoram-image

NoSQL的优点/缺点

优点:

  • 高可扩展性
  • 分布式计算
  • 低成本
  • 架构的灵活性,半结构化数据
  • 没有复杂的关系

缺点:

  • 没有标准化
  • 有限的查询功能(到目前为止)
  • 最终一致是不直观的程序

BASE

BASE:Basically Available, Soft-state, Eventually Consistent。 由 Eric Brewer 定义。

CAP理论的核心是:一个分布式系统不可能同时很好的满足一致性,可用性和分区容错性这三个需求,最多只能同时较好的满足两个。
BASE是NoSQL数据库通常对可用性及一致性的弱要求原则:

Basically Availble --基本可用
Soft-state --软状态/柔性事务。 "Soft state" 可以理解为"无连接"的, 而 "Hard state" 是"面向连接"的
Eventual Consistency --最终一致性 最终一致性, 也是是 ACID 的最终目的。

MongoDB 特性&优势

MongoDB 特性 优势
事务支持 MongoDB 目前只支持单文档事务,需要复杂事务支持的场景暂时不适合
灵活的文档模型 JSON 格式存储最接近真实对象模型,对开发者友好,方便快速开发迭代
高可用复制集 满足数据高可靠、服务高可用的需求,运维简单,故障自动切换
可扩展分片集群 海量数据存储,服务能力水平扩展
高性能 mmapv1、wiredtiger、mongorocks(rocksdb)、in-memory 等多引擎支持满足各种场景需求
强大的索引支持 地理位置索引可用于构建 各种 O2O 应用、文本索引解决搜索的需求、TTL索引解决历史数据自动过期的需求
Gridfs 解决文件存储的需求
aggregation & mapreduce 解决数据分析场景需求,用户可以自己写查询语句或脚本,将请求都分发到 MongoDB 上完成

文档参考

中文社区

MongoDB 极简实践入门

安装&环境配置

下载:https://www.mongodb.com/download-center#community

安装:
https://docs.mongodb.com/manual/administration/install-community/

解压,配置环境 变量PATH即可。

MONGODB_HOME=/Users/jack/soft/mongodb-osx-x86_64-3.4.2
export PATH=$PATH:$MONGODB_HOME/bin

启动脚本

cat start_mongodb.sh 
#!/bin/bash
source ~/.bashrc

mongod --dbpath ~/dbdata

启动服务进程

./start_mongodb.sh 

2017-03-19T02:50:44.491+0800 I CONTROL  [initandlisten] MongoDB starting : pid=67062 port=27017 dbpath=/Users/jack/dbdata 64-bit host=ChenJackdeMacBook-Pro.local
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] db version v3.4.2
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] allocator: system
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] modules: none
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] build environment:
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten]     distarch: x86_64
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] options: { storage: { dbPath: "/Users/jack/dbdata" } }
2017-03-19T02:50:44.494+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3584M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-03-19T02:50:45.352+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2017-03-19T02:50:45.748+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/Users/jack/dbdata/diagnostic.data'
2017-03-19T02:50:46.073+0800 I INDEX    [initandlisten] build index on: admin.system.version properties: { v: 2, key: { version: 1 }, name: "incompatible_with_version_32", ns: "admin.system.version" }
2017-03-19T02:50:46.073+0800 I INDEX    [initandlisten]      building index using bulk method; build may temporarily use up to 500 megabytes of RAM
2017-03-19T02:50:46.123+0800 I INDEX    [initandlisten] build index done.  scanned 0 total records. 0 secs
2017-03-19T02:50:46.124+0800 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.4
2017-03-19T02:50:46.124+0800 I NETWORK  [thread1] waiting for connections on port 27017



2017-03-19T02:51:24.533+0800 I NETWORK  [thread1] connection accepted from 127.0.0.1:49370 #1 (1 connection now open)
2017-03-19T02:51:24.534+0800 I NETWORK  [conn1] received client metadata from 127.0.0.1:49370 conn1: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.4.2" }, os: { type: "Darwin", name: "Mac OS X", architecture: "x86_64", version: "16.4.0" } }


Terminal连接客户端

$ mongo
MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.2
Server has startup warnings: 
2017-03-19T02:50:45.352+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> 
> 
> show dbs;
admin  0.000GB
local  0.000GB

强大的 MongoDB 管理工具

MongoDB Shell Enhancements

https://github.com/Jason-Chen-2017/mongo-hacker

Installation

npm install -g mongo-hacker

Enhancements

Basic UX

  • Sort document keys by default
  • Highlight querytime if verboseShell is enabled
    • In green if querytime is at or below slowms
    • In red if query time is above slowms
  • Default indent is 2 spaces instead of tab
    • Customizable by setting indent key of config
  • Verbose shell is enabled by default -- to disable: setVerboseShell(false)
  • Disable notfication of "Type 'it' for more"
  • Custom prompt: hostname(process-version)[rs_status:set_name] db>
  • Always pretty print. You can still use default format by appending .ugly() to the end of db statement.
  • Show DBs has aligned columns, is sorted by database name and shows less significant digits (in master for Mongo 2.5/2.6)
  • Nicer sh.status() output (remove lastmod, take up less space, colorize chunk's shard)
  • Colorized query output for console/terminal windows supporting ANSI color codes.


    Colorized Output

Additional shell commands

The MongoDB shell offers various "shell commands" (sometimes referred to as "shell helpers" as well) that make interactive use of the shell much more convenient than proper, Javascript-only scripted use of the shell.

To make interactive use of the MongoDB shell even more convenient, mongo-hacker adds the following shell commands:

  • count collections/count tables: count the number of collections in each of the mongo server's databases - by @pvdb
  • count documents/count docs: count the number of documents in all (non-system) collections in the database - by @pvdb
  • count indexes: list all collections and display the size of all indexes - by @cog-g

Some of these commands have hidden features that can be enabled in the mongo-hacker config, to make the command output even more useful:

  • by changing the count_deltas setting to true in config.js, the count documents command will also print out the change in the number of documents since the last count - by @pvdb

API Additions

Scripting

Get a list of database names: (by @pvdb)

db.getMongo().getDatabaseNames()

(note that this method is similar - functionality-wise and usage-wise - to the existing db.getCollectionNames() API method and allows for advanced, cross-database scripting in the MongoDB shell)

General

Filter for a collection of documents:

db.collection.filter(<criteria>)

One for finding a single document:

db.collection.find({ ... }).one() == db.collection.findOne({ ... })

Select for selecting fields to return (projection):

db.collection.find({ ... }).select({ name: 1 })

Reverse for descending sort by insertion order (default) or arbitrary field:

db.collection.find({ ... }).reverse()
db.collection.find({ ... }).reverse('createDate')

Last for finding last inserted document (default) or document last by given field:

db.collection.find({ ... }).last()
db.collection.find({ ... }).last('createDate')

Update, Replace, Upsert and Remove can be called on a DBQuery Object:

db.collection.find({ ... }).update({ ... })  // multi update
db.collection.find({ ... }).replace({ ... }) // single replacement
db.collection.find({ ... }).upsert({ ... })  // single upsert
db.collection.find({ ... }).remove()         // multi remove

Sort, limit, and skip through multi updates and removes:

db.collection.find({ ... }).limit(7).update({ ... })
db.collection.find({ ... }).sort({ ... }).skip(1).limit(3).update({ ... })
db.collection.find({ ... }).limit(3).remove()

Note: The performance of multi updates involving a skip or limit may be worse than those without those specifications due to there being absolutely no native support for this feature in MongoDB itself. It should be understood by the user of this software that use of this feature (by calling update on a cursor rather than a collection) is advanced and experimental. The option to do this sort of operation is purely additive to the MongoDB experience with MongoHacker and usage of it is in no way required. Furthermore, its inclusion in this enhancement does not effect the operation of updates invoked through collections and, in practice, is insanely useful.

Aggregation Framework

The aggregation framework is now fluent as well. You can use it as currently documented or via the chainable methods.

Calling aggregate without an array of operations or $operations will make it a match.

// matches every document
db.collection.aggregate()
db.collection.aggregate({})

// matches documents where the "a" is equal to 1
db.collection.aggregate({a: 1})

// matches documents where "a" is greater than 7
db.collection.aggregate({a: {$gt: 7}})

Additional methods can then be chained on top of the inital match in order to make more complicated aggregations.

// Match and project
db.collection.aggregate(<querydoc>).project(<projection>)
db.collection.aggregate({a: 1}).project({a: 1, _id: 0})

// Match, group and sort
db.collection.aggregate({<match>}).group({<group>}).sort({<sort>})
db.test.aggregate().group({_id: '$a', 'sum': {'$sum': 1}}).sort({sum: -1})

Data Generation

For easy and simple random data generation you can utilise these methods below. You can use any of these functions in a loop. For example:

// Inserts 20 documents with random data. 
for (i=1; i<21; i++) { 
    db.collection.insert(
            {
             word: randomWord(), 
             number: randomNumber(), 
             date: randomDate() 
            }
    ); 
}
randomWord

You can specify the length of each word, the number of words, and an optional seeded word in a sentence randomly. Use the optional seed parameter for testing text search.

randomWord(length=5, words=1, seed=undefined)

// Inserts a random sentence consisting of 5 letters per word, 5 words in total, 
// with a probability to insert the word 'needle' in the sentence
db.collection.insert( { words: randomWord(5, 5, 'needle') } )

// Inserts a random word consisting of 16 letters
db.collection.insert( { words: randomWord(16) } )
randomNumber

You can specify maximum number to be randomly generated (exclusive)

randomNumber(max=100)

// Inserts a random number in the range of 0 or 1. 
db.collection.insert( { number: randomNumber(2) } )

// Inserts a random number in the range of 0 or 999. 
db.collection.insert( { number: randomNumber(1000) } )

randomDate

You can specify start and end dates range to be randomly generated. (exclusive)

randomDate(start= <2 years ago> , end=Date() )

// Inserts a random date object in the range of 1st January 2016 to 1st February 2016
db.collection.insert( { date: randomDate(ISODate("2016-01-01T00:00:00"), ISODate("2016-02-01T00:00:00")) })

// If today is 19th May 2016 and you specify only the start of the day, 
// this will generate random date object between 00:00:00 to current time.  
db.collection.insert( { date: randomDate(ISODate("2016-05-19T00:00:00")) })

Helpers

General Shell Helpers

  • findCommand('search') list commands that match the search string

Aggregation Framework Helpers -- on collections

  • Group and Count: gcount(group_field, filter)
  • Group and Sum: gsum(group_field, sum_field, filter)
  • Group and Average: gavg(group_field, avg_field, filter)

Run function on some/all databases

runOnDbs(/db_names_regexp/, function(db) {
    // callback is ran for each database which name matches regular expression
    // db is that selected database
});

mongo-express

https://github.com/Jason-Chen-2017/mongo-express

Web-based MongoDB admin interface, written with Node.js and express

终端查看的效果:

jack@jacks-MacBook-Air:~$ mongo
MongoDB shell version: 3.2.4
connecting to: test
Mongo-Hacker 0.0.14
Server has startup warnings: 
2017-03-13T10:31:13.023+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2017-03-13T10:31:13.023+0800 I CONTROL  [initandlisten] 
jacks-MacBook-Air(mongod-3.2.4) test> 
jacks-MacBook-Air(mongod-3.2.4) test> 
jacks-MacBook-Air(mongod-3.2.4) test> db
test
jacks-MacBook-Air(mongod-3.2.4) test> show dbs;
admin      → 0.000GB
local      → 0.000GB
restfiddle → 0.001GB
jacks-MacBook-Air(mongod-3.2.4) test> use restfiddle;
switched to db restfiddle
jacks-MacBook-Air(mongod-3.2.4) restfiddle> show collections;
EntityAuth        → 0.000MB / 0.004MB
activityLog       → 0.001MB / 0.035MB
baseNode          → 0.021MB / 0.043MB
config            → 0.000MB / 0.016MB
conversation      → 0.017MB / 0.035MB
httpRequestHeader → 0.005MB / 0.016MB
oAuth2            → 0.002MB / 0.016MB
project           → 0.002MB / 0.031MB
rfRequest         → 0.017MB / 0.035MB
rfResponse        → 0.036MB / 0.047MB
runnerLog         → 0.000MB / 0.016MB
tag               → 0.000MB / 0.016MB
user              → 0.000MB / 0.035MB
workspace         → 0.001MB / 0.031MB
jacks-MacBook-Air(mongod-3.2.4) restfiddle> l
2017-03-21T14:18:55.202+0800 E QUERY    [thread1] ReferenceError: l is not defined :
@(shell):1:1

jacks-MacBook-Air(mongod-3.2.4) restfiddle> db.runnerLog.find()
{
  "_id": ObjectId("575fad7dba637304fffbae85"),
  "_class": "com.restfiddle.entity.RunnerLog",
  "status": "ACTIVE",
  "version": NumberLong("0")
}
{
  "_id": ObjectId("575fad92ba637304fffbae86"),
  "_class": "com.restfiddle.entity.RunnerLog",
  "status": "ACTIVE",
  "version": NumberLong("0")
}
{
  "_id": ObjectId("575fad9cba637304fffbae87"),
  "_class": "com.restfiddle.entity.RunnerLog",
  "status": "ACTIVE",
  "version": NumberLong("0")
}
Fetched 3 record(s) in 7ms
jacks-MacBook-Air(mongod-3.2.4) restfiddle> 

mongo-express的效果:

螢幕快照 2017-03-21 14.25.03.png
螢幕快照 2017-03-21 14.25.57.png
相关实践学习
MongoDB数据库入门
MongoDB数据库入门实验。
快速掌握 MongoDB 数据库
本课程主要讲解MongoDB数据库的基本知识,包括MongoDB数据库的安装、配置、服务的启动、数据的CRUD操作函数使用、MongoDB索引的使用(唯一索引、地理索引、过期索引、全文索引等)、MapReduce操作实现、用户管理、Java对MongoDB的操作支持(基于2.x驱动与3.x驱动的完全讲解)。 通过学习此课程,读者将具备MongoDB数据库的开发能力,并且能够使用MongoDB进行项目开发。 &nbsp; 相关的阿里云产品:云数据库 MongoDB版 云数据库MongoDB版支持ReplicaSet和Sharding两种部署架构,具备安全审计,时间点备份等多项企业能力。在互联网、物联网、游戏、金融等领域被广泛采用。 云数据库MongoDB版(ApsaraDB for MongoDB)完全兼容MongoDB协议,基于飞天分布式系统和高可靠存储引擎,提供多节点高可用架构、弹性扩容、容灾、备份回滚、性能优化等解决方案。 产品详情: https://www.aliyun.com/product/mongodb
相关文章
|
1月前
|
NoSQL Linux MongoDB
linux麒麟系统安装mongodb7.0
linux麒麟系统安装mongodb7.0
69 10
|
12天前
|
NoSQL 网络协议 MongoDB
docker安装mongodb(单点)图文详解
该文档提供了一个快速搭建MongoDB环境的指南,适用于开发和学习,但不适用于生产。主要步骤包括:1) 使用Docker创建数据卷`mongo_data_db`和`mongo_data_configdb`。2) 拉取`mongo`镜像并运行名为`mymongo`的容器,映射端口并挂载数据卷。3) 初始化管理员账号,创建具有`root`权限的用户`admin`,密码为`adminpwd`。4) 防火墙开放端口27017。最后,提到了使用第三方客户端进行连接。
|
12天前
|
存储 NoSQL Go
【Go语言专栏】Go语言中的MongoDB操作与NoSQL应用
【4月更文挑战第30天】本文介绍了Go语言中操作MongoDB的方法和NoSQL应用的优势。MongoDB作为流行的NoSQL数据库,以其文档型数据模型、高性能和可扩展性被广泛应用。在Go语言中,通过mongo-go-driver库可轻松实现与MongoDB的连接及插入、查询、更新和删除等操作。MongoDB在NoSQL应用中的优点包括灵活的数据模型、高性能、高可用性和易于扩展,使其成为处理大规模数据和高并发场景的理想选择。
|
12天前
|
NoSQL Java 关系型数据库
Java基础教程(21)-Java连接MongoDB
【4月更文挑战第21天】MongoDB是开源的NoSQL数据库,强调高性能和灵活性。Java应用通过MongoDB Java驱动与之交互,涉及MongoClient、MongoDatabase、MongoCollection和Document等组件。连接MongoDB的步骤包括:配置连接字符串、创建MongoClient、选择数据库和集合。伪代码示例展示了如何建立连接、插入和查询数据。
|
13天前
|
弹性计算 NoSQL Shell
一键安装 MongoDB 数据库脚本
【4月更文挑战第29天】
16 4
|
14天前
|
NoSQL Linux 网络安全
【专栏】在 RHEL 8 或者 CentOS 8 上顺利安装并运行MongoDB,以处理非结构化数据和扩展技术栈
【4月更文挑战第28天】本文档介绍了如何在RHEL或CentOS 8上安装MongoDB,包括环境准备(系统更新、依赖安装、硬件需求和sudo用户)、导入MongoDB GPG公钥、创建Yum仓库、安装MongoDB社区版,以及后续的基本配置和验证(启动服务、防火墙设置和连接验证)。通过这些步骤,用户可以顺利安装并运行MongoDB,以处理非结构化数据和扩展技术栈。
|
19天前
|
存储 分布式计算 NoSQL
MongoDB的简介和安装(在服务器上)
MongoDB的简介和安装(在服务器上)
30 0
|
23天前
|
NoSQL MongoDB Redis
Python与NoSQL数据库(MongoDB、Redis等)面试问答
【4月更文挑战第16天】本文探讨了Python与NoSQL数据库(如MongoDB、Redis)在面试中的常见问题,包括连接与操作数据库、错误处理、高级特性和缓存策略。重点介绍了使用`pymongo`和`redis`库进行CRUD操作、异常捕获以及数据一致性管理。通过理解这些问题、易错点及避免策略,并结合代码示例,开发者能在面试中展现其技术实力和实践经验。
312 8
Python与NoSQL数据库(MongoDB、Redis等)面试问答
|
1月前
|
存储 NoSQL Linux
CentOS7安装MongoDB
CentOS7安装MongoDB
18 0
|
2月前
|
NoSQL MongoDB Docker
docker安装MongoDB
docker安装MongoDB
24 0