Spring Boot2集成Elasticsearch、PostgreSQL遇到的问题

项目背景

  在描述和还原事故之前，简单说明下相关环境：

• spring boot v2.0.4.RELEASE
• spring-boot-starter-data-elasticsearch (以前做项目的时候，Spring Data ES跟ES服务存在版本匹配关系，但目前在spring boot v2.0.4.RELEASE中使用未发现有版本不兼容情况)
• spring-boot-starter-data-jpa (用于操作PostgreSQL)

PostgreSQL启动连接报错

  启动项目的时候出现错误，具体异常信息如下：

2018-08-29 21:33:18,397 INFO  org.hibernate.dialect.Dialect[157]: HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL95Dialect
2018-08-29 21:33:21,479 INFO  o.h.e.j.e.i.LobCreatorBuilderImpl[124]: HHH000424: Disabling contextual LOB creation as createClob() method threw error : java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException: null
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:45009)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:45012)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.hibernate.engine.jdbc.env.internal.LobCreatorBuilderImpl.useContextualLobCreation(LobCreatorBuilderImpl.java:113)
    at org.hibernate.engine.jdbc.env.internal.LobCreatorBuilderImpl.makeLobCreatorBuilder(LobCreatorBuilderImpl.java:54)
    at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentImpl.<init>(JdbcEnvironmentImpl.java:271)
    at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:114)
    at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
    at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:259)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:233)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:210)
    at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:51)
    at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:94)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:242)
    at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:210)
    at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.handleTypes(MetadataBuildingProcess.java:352)
    at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:111)
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:861)
    at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:888)
    at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.__createEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:57)
    at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:40002)
    at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.__createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
    at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:42002)
    at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:390)
    at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:377)
    at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1758)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1695)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:573)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
    at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$109/1509220174.getObject(Unknown Source)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1089)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:859)
    at org.springframework.context.support.AbstractApplicationContext.__refresh(AbstractApplicationContext.java:550)
    at org.springframework.context.support.AbstractApplicationContext.jrLockAndRefresh(AbstractApplicationContext.java:40002)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:41008)
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:762)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:398)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:330)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246)
    at cn.mariojd.nearjob.NearjobApplication.main(NearjobApplication.java:19)
Caused by: java.sql.SQLFeatureNotSupportedException: 这个 org.postgresql.jdbc.PgConnection.createClob() 方法尚未被实作。
    at org.postgresql.Driver.notImplemented(Driver.java:688)
    at org.postgresql.jdbc.PgConnection.createClob(PgConnection.java:1269)
    ... 51 common frames omitted

  这个错误确实有点奇怪，不过好在Github上已经有相关Issue，有兴趣的可以去看看，该问题的解决方法是添加配置项：spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation: true

JPA实体继承映射数据表

  当多个实体间有多个属性相同时，可以考虑抽取抽象实体类的方式复用属性定义，并在抽象父类上使用@MappedSuperclass注解(注意此父类不能再标注@Entity或@Table注解)：

• BaseEntity

@Data
@MappedSuperclass
public abstract class BaseEntity {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;

    private Integer gender;

    @Column(name = "username")
    private String name;

}

• Student

@Entity
@Table
@Data
public class Student extends BaseEntity {

    private Integer score;

}

• Teacher

@Entity
@Table
@Data
public class Teacher extends BaseEntity {

    private String phone;

}

  此外，JPA中还有不同的遗传策略来解决多实体间的继承映射关系，同样可以实现上述一样的效果(@Inheritance(strategy = InheritanceType.TABLE_PER_CLASS))：

JPA InheritanceType

Spirng Data抽取抽象Repository

  这种情况跟上面那种情况有一定的关联，我们借助泛型来解决，首先建立一个BaseDao(需指定为@NoRepositoryBean)：

@NoRepositoryBean
public interface BaseDao<T extends BaseEntity> extends JpaRepository<T, Integer> {

    // 部分字段查询需添加相对应构造器
    @Query(value = "SELECT new #{#entityName}(id,username) FROM #{#entityName} WHERE username=?1")
    T findByUsername(String username);
    
    T findByIdAndUsername(int id,String username);

}

  这样多个子Dao只需要继承这个BaseDao便可以拥有这些扩展方法。

Reactive Web集成ES启动冲突

  该问题出现在使用webflux集成elasticsearch启动项目的时候，异常信息打印如下：

2018-08-30 08:43:20.286  INFO 16636 --- [           main] o.elasticsearch.plugins.PluginsService   : loaded plugin [org.elasticsearch.transport.Netty4Plugin]
2018-08-30 08:43:22.999  WARN 16636 --- [           main] onfigReactiveWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'elasticsearchClient' defined in class path resource [org/springframework/boot/autoconfigure/data/elasticsearch/ElasticsearchAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.elasticsearch.client.transport.TransportClient]: Factory method 'elasticsearchClient' threw exception; nested exception is java.lang.IllegalStateException: availableProcessors is already set to [4], rejecting [4]

  具体解决方案可参考如下：

@SpringBootApplication
public class DemoApplication {

    public static void main(String[] args) {
        // 添加该配置项即可
        System.setProperty("es.set.netty.runtime.available.processors", "false");
        SpringApplication.run(DemoApplication.class, args);
    }
    
}

Spring Data Elasticsearch与ES mapping字段不一致

  如果没有主动创建mapping，Spring Data ES默认会在第一次添加数据的时候创建，对应mapping的字段名跟实体属性保持一致。如果原本已经创建好mapping，或是想自定义mapping字段跟实体属性的对照关系，这里有两种解决方案：

• 方案1

  借助@JsonProperty更改ES字段与实体属性的映射关系

@Data
@Document(indexName = "school", type = "primary_school")
public class Student {

    @Id
    private String id;

    private Integer gender;

    @JsonProperty("student_name")
    private String studentName;

}

Elasticsearch _mpping

• 方案2

  使用@JsonNaming注解并指定相应的映射策略。如果当前实体需要使用多个@JsonProperty才能定义这种关系，可以考虑使用这种更快捷的方式

@Data
@Document(indexName = "school", type = "primary_school")
@JsonNaming(PropertyNamingStrategy.SnakeCaseStrategy.class)
public class Student {

    @Id
    private String id;

    private Integer gender;
    
    private String studentName;

}

PropertyNamingStrategy

  从上图可以看到jackson包中已经定义好有五种不同的映射策略，如果都不满足实际需求的话还可以自行扩展，只需要继承PropertyNamingStrategyBase这个抽象类并重写它的translate()方法即可。

ES一个Index对应多个type问题

  如果出现下面这个错误信息，说明定义了多个Type对应在一个Index。实际上在ES6.0之后，官方已经不推荐这种映射关系。按以前那种思路，Index对应Database，然后type对应table的关系(所以一个database中有多个table，那么一个index也就可以有多个type)是不严谨的，在官网上有这个Reference。

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.elasticsearch.repository.support.SimpleElasticsearchRepository]: Constructor threw exception; nested exception is java.lang.IllegalArgumentException: Rejecting mapping update to [school] as the final mapping would have more than 1 type: [teacher, student]

参考链接

Spring-data自定义Repository
elasticsearch常见的问题
JPA实体继承实体的映射策略
SpringData ES 关于字段名和索引中的列名字不一致导致的查询问题