Architecture

Elasticsearch Flow

Small-sized Solution

ELK Stack

Elasticsearch / Lucene

• Lucene

  Elasticsearch is an open-source search engine built on top of Apache Lucene, as the rest of the ELK Stack, including Logstash and Kibana.

Componenets

• Index

  An index consists of one or more Documents

  • Shard, Replicas 2개 유형으로 구분 가능

• Document

  Document consists of one or more Fields

• Cluster

  ES내 가장 큰 시스템 단위

  • 서로 다른 cluster는 데이터의 접근, 교환을 할 수 없는 독립적인 시스템
  • 여러 대의 server가 하나의 cluster를 구성 가능 / 한 서버에 여러 cluster가 존재할 수 있음

• Node

  ES를 구성하는 하나의 단위 프로세스 (데이터를 저장하고 indexing을 하는 등의 ES Instance)

• Shards

  split up indices horizontally into pieces called shards

  • index를 여러 shard로 쪼갬 for scaling out in ES
  • Sharding : 데이터를 분산해서 저장하는 방법

• Replica

  One or More copy of index

  • 또 다른 형태의 shard
  • node를 손실했을 경우의 데이터의 신뢰성을 위해 shard를 복제
  • 권장사항 : replica는 서로 다른 노드에 존재할 것

Inverted index

• Elasticsearch uses a data structure called an inverted index that supports very fast full-text searches. An inverted index lists every unique word that appears in any document and identifies all of the documents each word occurs in.

Example

Source Example

Blog.java @Entity

• ES 7.0.0 부터는 Type 이 deprecated 되어 indexName만 설정가능

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

@Setter
@Document(indexName = "blog")
public class Blog {

    @Id
    @Field(type = FieldType.Long)
    private String id;

    @Field(type = FieldType.Text)
    private String title;

    @Field(type = FieldType.Text)
    private String content;

    @Field(type = FieldType.Date)
    private Date log_date;

    @Field(type = FieldType.Text)
    private String log_text;

    @Field(type = FieldType.Long)
    private Long price;
}

BlogEsRepository.java

1
2
3

@Repository
public interface BlogEsRepository extends ElasticsearchRepository<Blog, String> {
}

BlogEsService.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

@Service
public class BlogEsService {

   @Autowired
   private ElasticsearchOperations operations;

   @Autowired
   private RestHighLevelClient client;


  public serviceMethod() {

    // BoolQueryBuilder
    // NativeSearchQueryBuilder
    // SearchHits
    // PageRequest
    // SearchRequest
    // SearchResponse
    // ...
    // Reference : Classes

  }

}

Test

1
2
3
4
5
6
7
8
9
10
11

@Resource
BlogEsRepository blogEsRepository;

@Test
void test(){
    Blog blog = new Blog();
    blog.setId("1");
    blog.setContent("내용입니다.");
    blog.setTitle("제목입니다.");
    blogEsRepository.save(blog);
}

bulid.gradle

1
2
3

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-data-elasticsearch'
}

application.yml

1
2
3

elasticsearch:
  host: localhost
  port: 9200

Classes

• SearchRequest
• SearchResponse
• SearchSourceBuilder
• SearchHits
• Document
  • @Document(indexName = “noresults_#{indexNames.getYMD()}”, createIndex = false)
• RequestOptions
• NativeSearchQueryBuilder
• QueryBuilders
  • termsQuery
  • boolQuery
  • matchAllQuery
  • wildcardQuery
  • prefixQuery
• SortBuilders
  • fieldSort

• IndexQueryBuilder

• ElasticsearchOperations
  • search
  • getIndexCoordinatesFor
  • bulkUpdate
• RestHighLevelClient
• IndexCoordinates

Search API : response

[Reference]

https://esbook.kimjmin.net/

https://bgpark.tistory.com/158

https://velog.io/@sunghwancode/TIL-%EA%B2%80%EC%83%89-%EC%8B%9C%EC%8A%A4%ED%85%9C-Elastic-Search-Redis

https://yeminyi.github.io/myblog/elk/2020/10/22/what-is-elk.html

https://victorydntmd.tistory.com/308

https://idea-sketch.tistory.com/59

https://esbook.kimjmin.net/06-text-analysis/6.1-indexing-data

https://engineering.mercari.com/en/blog/entry/20220311-operational-tips-on-using-elasticsearch/

https://juntcom.tistory.com/137

https://victorydntmd.tistory.com/313