基于容器引擎的Titan1.0入门指南

本文涉及的产品
容器镜像服务 ACR,镜像仓库100个 不限时长
简介: 分布式图数据库[Titan](http://titan.thinkaurelius.com)是基于大数据的关系分析可选的底层引擎。本文主要介绍如何借助Docker快速实现Titan的入门。

分布式图数据库Titan是基于大数据的关系分析可选的底层引擎。

本文主要介绍如何借助Docker快速实现Titan的入门。Titan支持多种存储引擎和搜索引擎,本文选用Cassandra和Elasticsearch。最新版本(1.0)的Titan使用的Elasticsearch是1.5、Cassandra是2.1,由于引擎选用的版本有些老,所以不定制Dockerfile,而是直接使用hub上的镜像。

1 启动引擎容器

Elasticsearch

1.1 搜索引擎Elasticsearch
docker run -d --name es1.5 --net=host elasticsearch:1.5
1.2 存储引擎Cassandra
  • 10.101.95.23 c1(seed node)

    sudo docker run -d --name c1 \
     -e CASSANDRA_BROADCAST_ADDRESS=10.101.95.23 \
    --net=host \
    cassandra:2.1
  • 10.189.193.225 c2(seed node)

    sudo docker run -d --name c2 \
    -e CASSANDRA_BROADCAST_ADDRESS=10.189.193.225 \
    --net=host \
    -e CASSANDRA_SEEDS=10.101.95.23,10.189.193.225 \
    cassandra:2.1
  • 10.101.110.3 c3

    sudo docker run -d --name c3 \
    -e CASSANDRA_BROADCAST_ADDRESS=10.101.110.3 \
    --net=host \
    -e CASSANDRA_SEEDS=10.101.95.23,10.189.193.225 \
    cassandra:2.1
  • 100.81.0.123 c4

    sudo docker run -d --name c4 \
    -e CASSANDRA_BROADCAST_ADDRESS=100.81.0.123 \
    --net=host \
    -e CASSANDRA_SEEDS=10.101.95.23,10.189.193.225 \
    cassandra:2.1

集群启动完毕后,检查节点状态

  sudo docker exec -ti c1 nodetool status
  Datacenter: datacenter1
  =======================
  Status=Up/Down
  |/ State=Normal/Leaving/Joining/Moving
  --  Address         Load       Tokens  Owns (effective)  Host ID                               Rack
  UN  10.101.95.23    152.12 KB  256     48.3%             7bab2c1d-91f4-40c0-a2ba-2c41c6e6e78d  rack1
  UN  100.81.0.123    19.3 KB    256     50.4%             0256243a-7434-4a1d-83db-58505d894bcc  rack1
  UN  10.189.193.225  152.56 KB  256     50.2%             62b461ed-18f8-4268-8a6f-2ce2ad126646  rack1
  UN  10.101.110.3    167.72 KB  256     51.1%             c1825c5e-5d22-4cba-9350-ffa5c73ea881  rack1

2 编辑配置并启动Titan

首先从https://github.com/thinkaurelius/titan/wiki/Downloads下载并解压缩titan-1.0.0-hadoop2.zip到ECS。

2.1 conf/eric-titan.properties
gremlin.graph=com.thinkaurelius.titan.core.TitanFactory

storage.backend=cassandra
storage.hostname=10.101.110.3

cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-time = 180000
cache.db-cache-size = 0.25

index.search.backend=elasticsearch
index.search.hostname=127.0.0.1
index.search.elasticsearch.client-only=true
2.2 conf/eric-gremlin-server.yaml
host: 10.101.91.65
port: 18182
threadPoolWorker: 1
gremlinPool: 8
scriptEvaluationTimeout: 30000
serializedResponseTimeout: 30000
channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer
graphs: {
  graph: conf/eric-titan.properties}
plugins:

- aurelius.titan
    scriptEngines: {
    gremlin-groovy: {
    imports: [java.lang.Math],
    staticImports: [java.lang.Math.PI],
    scripts: [scripts/empty-sample.groovy]},
    nashorn: {
      imports: [java.lang.Math],
      staticImports: [java.lang.Math.PI]}}
    serializers:
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { useMapperFromGraph: graph }}
    - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
    - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { useMapperFromGraph: graph }}
    - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { useMapperFromGraph: graph }}
      processors:
    - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}
      metrics: {
      consoleReporter: {enabled: true, interval: 180000},
      csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
      jmxReporter: {enabled: true},
      slf4jReporter: {enabled: true, interval: 180000},
      gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
      graphiteReporter: {enabled: false, interval: 180000}}
      threadPoolBoss: 1
      maxInitialLineLength: 4096
      maxHeaderSize: 8192
      maxChunkSize: 8192
      maxContentLength: 65536
      maxAccumulationBufferComponents: 1024
      resultIterationBatchSize: 64
      writeBufferHighWaterMark: 32768
      writeBufferHighWaterMark: 65536
      ssl: {
      enabled: false}
2.3 启动Titan服务
bin/gremlin-server.sh conf/eric-gremlin-server.yaml

3 构造图

3.1 conf/eric-remote.yaml
hosts: 127.0.0.1
port: 18182
serializer: { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { custom: [com.thinkaurelius.titan.graphdb.relations.RelationIdentifier] }}
3.2 客户端连接Titan
bin/gremlin.sh :remote conf/eric-remote.yaml
3.3 构造神家谱
gremlin> graph = TitanFactory.open('conf/eric-titan.properties')
==>standardtitangraph[cassandra:[10.101.110.3]]

gremlin> GraphOfTheGodsFactory.load(graph)
==>null

实现类:https://github.com/thinkaurelius/titan/blob/titan10/titan-core/src/main/java/com/thinkaurelius/titan/example/GraphOfTheGodsFactory.java

4 查询图

gremlin> g = graph.traversal()
==>graphtraversalsource[standardtitangraph[cassandra:[10.101.110.3]], standard]

gremlin> saturn = g.V().has('name', 'saturn').next()
==>v[4160]

gremlin> g.V(saturn).valueMap()
==>[name:[saturn], age:[10000]]

gremlin> g.V(saturn).in('father').in('father').values('name')
==>hercules

gremlin> g.E().has('place', geoWithin(Geoshape.circle(37.97, 23.72, 50)))
==>e[4r0-38g-9hx-6e0][4192-battled->8280]
==>e[4cs-38g-9hx-6i0][4192-battled->8424]

gremlin> g.E().has('place', geoWithin(Geoshape.circle(37.97, 23.72, 50))).as('source').inV().as('god2').select('source').outV().as('god1').select('god1', 'god2').by('name')
==>[god1:hercules, god2:hydra]
==>[god1:hercules, god2:nemean]

gremlin> hercules = g.V(saturn).repeat(__.in('father')).times(2).next()
==>v[4192]

gremlin> g.V(hercules).out('father', 'mother').values('name')
==>jupiter
==>alcmene

gremlin> g.V(hercules).out('father', 'mother').label()
==>god
==>human

gremlin> hercules.label()
==>demigod

gremlin> g.V(hercules).outE('battled').has('time', gt(1)).inV().values('name')
==>cerberus
==>hydra

gremlin> g.V(hercules).outE('battled').has('time', gt(1)).inV().values('name').toString()
==>[GraphStep([v[4192]],vertex), VertexStep(OUT,[battled],edge), HasStep([time.gt(1)]), EdgeVertexStep(IN), PropertiesStep([name],value)]

gremlin> pluto = g.V().has('name', 'pluto').next()
==>v[4272]

gremlin> g.V(pluto).out('lives').in('lives').values('name')
==>pluto
==>cerberus

gremlin>  g.V(pluto).out('brother').out('lives').values('name')
==>sky
==>sea

gremlin> g.V(pluto).out('brother').as('god').out('lives').as('place')
==>v[4232]
==>v[8328]

gremlin> g.V(pluto).outE('lives').values('reason')
==>no fear of death

gremlin> g.E().has('reason', textContains('loves'))
==>e[3kb-388-b2t-39k][4184-lives->4232]
==>e[36l-3c8-b2t-6fc][4328-lives->8328]

gremlin>  g.E().has('reason', textContains('loves')).as('source').values('reason').as('reason').select('source').outV().values('name').as('god').select('source').inV().values('name').as('thing').select('god', 'reason', 'thing')
==>[god:jupiter, reason:loves fresh breezes, thing:sky]
==>[god:neptune, reason:loves waves, thing:sea]

5 查看存储

5.1 使用容器内cassandra-cli查看存储
sudo docker exec -ti c1 bash
cassandra-cli
use titan;
show schema;
5.2 查看列存储详情
[default@titan] list titan_ids limit 3;
-------------------
RowKey: 0000000000000003
=> (name=fffffffffffec77f00053a400cd0f0f8306136353562343132383133382d653031303130313039313036352d7a6d6631, value=, timestamp=1471421650891001)
-------------------
RowKey: a800000000000000
=> (name=ffffffffffffd8ef00053a400d2e7278306136353562343132383133382d653031303130313039313036352d7a6d6631, value=, timestamp=1471421657019001)
-------------------
RowKey: b000000000000003
=> (name=fffffffffffec77f00053a400d28d8e0306136353562343132383133382d653031303130313039313036352d7a6d6631, value=, timestamp=1471421656652001)

[default@titan] list graphindex limit 3;
-------------------
RowKey: 10a5a072741e6d6f746865f2
=> (name=00, value=5895, timestamp=1471421652566001)
-------------------
RowKey: 0489a07361747572ee
=> (name=00, value=20c0, timestamp=1471421658055001)
-------------------
RowKey: 10a5a072741e62726f746865f2
=> (name=00, value=010095, timestamp=1471421652566001)
[default@titan] list graphindex_lock_ limit 3;
-------------------
RowKey: 0000000910a5a072741e6167e500
-------------------
RowKey: 0000000910a5a0766c1e676fe400
-------------------
RowKey: 0000000b10a5a072741e6c697665f300

[default@titan] list edgestore limit 3;
RowKey: 0000000000003415
=> (name=02, value=0001045080, timestamp=1471421652566001)
=> (name=10c0, value=a072741e31323330393a626174746c6573427954696de5044c80, timestamp=1471421652566001)
=> (name=10c2846000, value=8f01018e008080, timestamp=1471421652566001)
=> (name=10c2846400, value=99820000000000001805018e008180, timestamp=1471421652566001)
=> (name=10c2846800, value=ad81018e008280, timestamp=1471421652566001)
=> (name=10c2846c00, value=9981018e008380, timestamp=1471421652566001)
=> (name=10c2847000, value=ae80018e008480, timestamp=1471421652566001)
=> (name=10c2847400, value=b082018e008680, timestamp=1471421652566001)
=> (name=10c2847800, value=b382018e008780, timestamp=1471421652566001)
=> (name=10c4, value=0080045480, timestamp=1471421652566001)
=> (name=10c8, value=0080053a400ce4d6b8045c80, timestamp=1471421652566001)
=> (name=30c9806015847c00, value=009180, timestamp=1471421652566001)
-------------------
RowKey: 0000000000002815
=> (name=02, value=0001034080, timestamp=1471421652566001)
=> (name=10c0, value=a072741e6661746865f2033c80, timestamp=1471421652566001)
=> (name=10c2835000, value=8f00018e008080, timestamp=1471421652566001)
=> (name=10c2835400, value=9981018e008180, timestamp=1471421652566001)
=> (name=10c2835800, value=ad80018e008280, timestamp=1471421652566001)
=> (name=10c2835c00, value=9981018e008380, timestamp=1471421652566001)
=> (name=10c2836000, value=ae83018e008480, timestamp=1471421652566001)
=> (name=10c2836400, value=b082018e008680, timestamp=1471421652566001)
=> (name=10c2836800, value=b382018e008780, timestamp=1471421652566001)
=> (name=10c4, value=0080034480, timestamp=1471421652566001)
=> (name=10c8, value=0080053a400ce48898034c80, timestamp=1471421652566001)
-------------------
RowKey: 0000000000003815
=> (name=02, value=0001050480, timestamp=1471421652566001)
=> (name=10c0, value=a072741e6c697665f3050080, timestamp=1471421652566001)
=> (name=10c2851400, value=8f00018e008080, timestamp=1471421652566001)
=> (name=10c2851800, value=9981018e008180, timestamp=1471421652566001)
=> (name=10c2851c00, value=ad80018e008280, timestamp=1471421652566001)
=> (name=10c2852000, value=99820000000000001c05018e008380, timestamp=1471421652566001)
=> (name=10c2852400, value=ae80018e008480, timestamp=1471421652566001)
=> (name=10c2852800, value=b082018e008680, timestamp=1471421652566001)
=> (name=10c2852c00, value=b382018e008780, timestamp=1471421652566001)
=> (name=10c4, value=0080050880, timestamp=1471421652566001)
=> (name=10c8, value=0080053a400ce4f210051080, timestamp=1471421652566001)
[default@titan] list edgestore_lock_ limit 3;
0 Row Returned.
[default@titan] list system_properties limit 3;
-------------------
RowKey: 636f6e66696775726174696f6e
=> (name=63616368652e64622d6361636865, value=8f01, timestamp=1471415029987001)
=> (name=63616368652e64622d63616368652d636c65616e2d77616974, value=8ca8, timestamp=1471415029982001)
=> (name=63616368652e64622d63616368652d73697a65, value=943fd0000000000000, timestamp=1471415029892001)
=> (name=63616368652e64622d63616368652d74696d65, value=8d800000000002bf20, timestamp=1471415029965001)
=> (name=67726170682e74696d657374616d7073, value=b681, timestamp=1471415030003001)
=> (name=67726170682e746974616e2d76657273696f6e, value=92a0312e302eb0, timestamp=1471415030000001)
=> (name=68696464656e2e66726f7a656e, value=8f01, timestamp=1471415030159001)
=> (name=696e6465782e7365617263682e6261636b656e64, value=92a0656c61737469637365617263e8, timestamp=1471415029972001)
=> (name=696e6465782e7365617263682e656c61737469637365617263682e636c69656e742d6f6e6c79, value=8f01, timestamp=1471415029977001)
=> (name=696e6465782e7365617263682e686f73746e616d65, value=9e84a031302e3130312e38392eb3a031302e3130312e39302eb9a031302e3130312e38352e3230b8, timestamp=1471415029992001)
=> (name=73797374656d2d726567697374726174696f6e2e306136353562343131383235332d653031303130313039313036352d7a6d66312e737461727475702d74696d65, value=c18000000057b419ce0119452980, timestamp=1471420878163001)
[default@titan] list system_properties_lock_ limit 3;
0 Row Returned.
[default@titan] list systemlog limit 3;
-------------------
RowKey: ffffffffa0306136353562343131333933362d653031303130313039313036352d7a6d66b1
=> (name=01, value=0000000000000000, timestamp=1471420623958001)
-------------------
RowKey: 000000000000000000e08568
=> (name=00053a400cecf8c0a0306136353562343132383133382d653031303130313039313036352d7a6d66b10000000000000001, value=8081810489, timestamp=1471421652730001)
=> (name=00053a400d151648a0306136353562343131383235332d653031303130313039313036352d7a6d66b10000000000000001, value=81a0306136353562343132383133382d653031303130313039313036352d7a6d66b181, timestamp=1471421655362001)
=> (name=00053a400d433eb0a0306136353562343132383133382d653031303130313039313036352d7a6d66b10000000000000002, value=81a0306136353562343132383133382d653031303130313039313036352d7a6d66b181, timestamp=1471421658382001)
-------------------
RowKey: ffffffffa0306136353562343132383133382d653031303130313039313036352d7a6d66b1
=> (name=01, value=0000000000000002, timestamp=1471423313062001)
[default@titan] list txlog limit 3;
-------------------
RowKey: ffffffffa0306136353562343131333933362d653031303130313039313036352d7a6d66b1
=> (name=01, value=0000000000000000, timestamp=1471420623962001, ttl=604800)
-------------------
RowKey: ffffffffa0306136353562343132383133382d653031303130313039313036352d7a6d66b1
=> (name=01, value=0000000000000000, timestamp=1471423313067001, ttl=604800)

6 查看索引

6.1 查看Titan创建了哪些索引
$curl localhost:9200/_cat/indices
yellow open titan 5 1 12 0 13.7kb 13.7kb
6.2 查看titan索引下的Type
$curl localhost:9200/titan?pretty
{
  "titan" : {

    "aliases" : { },
    "mappings" : {
      "vertices" : {
        "_ttl" : {
          "enabled" : true
        },
        "properties" : {
          "age" : {
            "type" : "integer"
          }
        }
      },
      "edges" : {
        "_ttl" : {
          "enabled" : true
        },
        "properties" : {
          "place" : {
            "type" : "geo_point"
          },
          "reason" : {
            "type" : "string"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1471420330925",
        "number_of_shards" : "5",
        "uuid" : "WSAMXWioQX6_JAZZ4E_RWw",
        "version" : {
          "created" : "1050199"
        },
        "number_of_replicas" : "1"
      }
    },
    "warmers" : { }
  }
}
6.3 查看type=edges的文档数量
$curl localhost:9200/titan/edges/_count?pretty
{
  "count" : 6,
  "_shards" : {

    "total" : 5,
    "successful" : 5,
    "failed" : 0
  }
}
6.4 查看type=vertices的文档数量
$curl localhost:9200/titan/vertices/_count?pretty
{
  "count" : 6,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  }
}
相关实践学习
使用阿里云Elasticsearch体验信息检索加速
通过创建登录阿里云Elasticsearch集群,使用DataWorks将MySQL数据同步至Elasticsearch,体验多条件检索效果,简单展示数据同步和信息检索加速的过程和操作。
ElasticSearch 入门精讲
ElasticSearch是一个开源的、基于Lucene的、分布式、高扩展、高实时的搜索与数据分析引擎。根据DB-Engines的排名显示,Elasticsearch是最受欢迎的企业搜索引擎,其次是Apache Solr(也是基于Lucene)。 ElasticSearch的实现原理主要分为以下几个步骤: 用户将数据提交到Elastic Search 数据库中 通过分词控制器去将对应的语句分词,将其权重和分词结果一并存入数据 当用户搜索数据时候,再根据权重将结果排名、打分 将返回结果呈现给用户 Elasticsearch可以用于搜索各种文档。它提供可扩展的搜索,具有接近实时的搜索,并支持多租户。
目录
相关文章
|
13天前
|
Kubernetes Cloud Native Docker
云原生时代的容器化实践:Docker和Kubernetes入门
【10月更文挑战第37天】在数字化转型的浪潮中,云原生技术成为企业提升敏捷性和效率的关键。本篇文章将引导读者了解如何利用Docker进行容器化打包及部署,以及Kubernetes集群管理的基础操作,帮助初学者快速入门云原生的世界。通过实际案例分析,我们将深入探讨这些技术在现代IT架构中的应用与影响。
55 2
|
15天前
|
Cloud Native 持续交付 Docker
Docker容器化技术:从入门到实践
Docker容器化技术:从入门到实践
|
22天前
|
Cloud Native 持续交付 云计算
云原生入门指南:从容器到微服务
【10月更文挑战第28天】在数字化转型的浪潮中,云原生技术成为推动现代软件开发的关键力量。本篇文章将带你了解云原生的基本概念,探索它如何通过容器化、微服务架构以及持续集成和持续部署(CI/CD)的实践来提升应用的可伸缩性、灵活性和可靠性。你将学习到如何利用这些技术构建和部署在云端高效运行的应用,并理解它们对DevOps文化的贡献。
45 2
|
27天前
|
运维 Kubernetes Cloud Native
云原生入门:Kubernetes和容器化的未来
【10月更文挑战第23天】本文将带你走进云原生的世界,探索Kubernetes如何成为现代软件部署的心脏。我们将一起揭开容器化技术的神秘面纱,了解它如何改变软件开发和运维的方式。通过实际的代码示例,你将看到理论与实践的结合,感受到云原生技术带来的革命性影响。无论你是初学者还是有经验的开发者,这篇文章都将为你开启一段新的旅程。让我们一起踏上这段探索之旅,解锁云原生技术的力量吧!
|
1月前
|
Ubuntu Shell 开发者
Docker入门:轻松开始容器化之旅
【10月更文挑战第17天】Docker 是一种开源的应用容器引擎,它让开发者能够“一次构建、到处运行”。Docker 通过容器化技术将应用程序及其依赖打包在一起,从而确保应用在任何环境中都能一致地运行。本文将为新手用户提供一个全面的Docker入门指南,包括基本概念、优势、安装配置以及如何创建和管理容器。
46 2
|
1月前
|
存储 运维 云计算
探索Docker容器化:从入门到实践
在这个快速发展的云计算时代,Docker容器化技术正在改变应用的开发、部署和管理方式。本文旨在为初学者提供一个关于Docker的全面入门指南,并通过实践案例展示Docker在实际开发中的应用。我们将一起了解Docker的核心概念、基本操作、网络和存储,以及如何构建和部署一个简单的Web应用。无论你是开发者还是运维人员,本文都会帮助你快速掌握Docker的核心技能。
|
1月前
|
Kubernetes Cloud Native Docker
云原生时代的容器化实践:Docker与Kubernetes入门
【9月更文挑战第30天】在云计算的浪潮中,云原生技术正以前所未有的速度重塑着软件开发和运维领域。本文将通过深入浅出的方式,带你了解云原生的核心组件——Docker容器和Kubernetes集群,并探索它们如何助力现代应用的构建、部署和管理。从Docker的基本命令到Kubernetes的资源调度,我们将一起开启云原生技术的奇妙之旅。
|
2月前
|
运维 Cloud Native Docker
云原生技术入门:Docker容器化实战
【9月更文挑战第20天】本文将引导你走进云原生技术的世界,通过Docker容器化技术的实战演练,深入理解其背后的原理和应用。我们将一起探索如何在云平台上利用Docker简化部署、扩展和管理应用程序的过程,并揭示这一技术如何改变现代软件的开发和运维模式。
|
2月前
|
Kubernetes 监控 Cloud Native
云原生入门:从传统应用到容器化部署的旅程
【9月更文挑战第19天】在数字化浪潮中,企业正迅速将目光投向云原生技术,以实现更快的应用开发和更灵活的资源管理。本文将通过一个简单示例引导读者理解如何将传统应用转变为云原生应用,并部署至云端。我们将探索容器化技术的基础,以及它如何帮助企业解锁现代软件交付的速度和效率。准备好让你的应用乘上云原生的快车了吗?让我们开始这段令人兴奋的旅程吧!
|
2月前
|
Cloud Native 持续交付 Docker
云原生技术入门与实践:Docker容器化部署示例
【9月更文挑战第25天】在数字化转型的浪潮下,云原生技术成为推动企业创新的重要力量。本文旨在通过浅显易懂的语言,为初学者揭示云原生技术的核心概念及其应用价值。我们将以Docker容器为例,逐步引导读者了解如何将应用程序容器化,并在云端高效运行。这不仅是对技术趋势的跟随,更是对资源利用和开发效率提升的探索。
61 4
下一篇
无影云桌面