ES查询语句

发表于 | 分类于 ，

range match agg 排序 查询ES
使用 scan 获取查询结果大于 10000 的数据

时间查询

1
2
3
4
5
6
7
8
9
10
{
    "query": {
        "range" : {
            "@timestamp" : {
                "gte" : 1505534400000,
                "lte" : 1505620800000
            }
        }
    }
}

条件查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{
  "query": {
    "bool": {
      "must": [
        {"match": {"event": "2"}},
        {"range": {
          "@timestamp": {
            "gte": 1505534400000,
            "lte": 1505620800000
          }
        }}
      ]
    }
  }
}

聚合

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
  "query": {
    "bool": {
      "must": [
        {"match": {"event": "2"}},
        {"range": {
          "@timestamp": {
            "gte": 1505534400000,
            "lte": 1505620800000
          }
        }}
      ]
    }
  },
    "aggs": {
    "classroom_ids": {
      "terms": {
        "field": "classroom"
      }
    }
  }
}

聚合排序取最大

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
GET index*/_search
{
  "query": {
    "bool": {
      "must": [
        {"range": {
          "@timestamp": {
            "gte": 1517270400000,
            "lte": 1517488758000
          }
        }},
        {"match": {"user_type": "STUDENT"}},
        {"match": {"version": "6.3.0"}}
      ]
    }
  }, 
  "aggs": {
    "class_ids": {
      "terms": {
        "field": "class_id",
        "size": 20
      },
      "aggs": {
        "max_timestamp": {
          "max": {
            "field": "@timestamp"
          }
        }
      }
    }
  }, 
  "size": 0
}

查询所有的记录 > 2000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from elasticsearch_dsl import Search, Q

def build_range(begin, end):
    return {'@timestamp': dict(gte=begin, lte=end)}

timestamp_dict = build_range(start_tp, end_tp)
search = Search(using=es, index='INDEX*', doc_type='DOCTYPE')\
        .filter('range', **timestamp_dict)\
        .query(Q('bool', must=[Q('match', field='python')]))

for hit in search.scan():
source = hit.to_dict()
source = {key: value
                 for key, value in source.items()
                 if key in [filed1, filed2 ...]}