Yc-Ma Blog

自动解析图模型并抽取ES中指标生成虚拟图

 
  • Yc-Ma
  • May 25, 2021

Here’s the table of contents:

  1. 图模型自动解析并通过过滤器获取指标数据生成虚拟图
    1. 查询三度担保金额超过10万的担保子图，并返回挂到ES指标的虚拟图
    2. 查询与上海康达化工新材料集团股份有限公司同属于化工行业的其它公司三度担保金额超过10万的担保子图，并返回挂到ES指标的虚拟图
    3. 【指定指标的返回字段_source】查询与上海康达化工新材料集团股份有限公司同属于化工行业的其它公司三度担保金额超过10万的担保子图，并返回挂到ES指标的虚拟图
    4. ES过滤器转换为查询语句【vFMap中ES过滤器的解析函数】
    5. 预装函数【OLAB依赖APOC的预装存储过程】
      1. custom.es.result.bool
      2. custom.es.result

图模型自动解析并通过过滤器获取指标数据生成虚拟图

通过指标过滤查询图结构并返回过滤器与实体ID的绑定，然后生成挂好指标数据的虚拟图【三维数据模型和二维数据模型的聚合结果】

查询三度担保金额超过10万的担保子图，并返回挂到ES指标的虚拟图

WITH '{"graph":{"nodes":[{"id":"1","labels":["公司"]},{"id":"2","labels":["公司"]},{"id":"3","labels":["公司"]},{"id":"4","labels":["公司"]}],"relationships":[{"id":"1","type":"担保","startNode":"1","endNode":"2","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}}}"}]},{"id":"2","type":"担保","startNode":"2","endNode":"3","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}}}"}]},{"id":"3","type":"担保","startNode":"3","endNode":"4","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}}}"}]}]}}' as json
WITH olab.schema.auto.cypher(json,-1,3,false,true) AS cypher
CALL apoc.cypher.run(cypher,{}) YIELD value WITH randomUUID() AS uniqueGraphID,value.graph AS graph,value.graph.graph AS paths,value.vFMap AS vFMap
UNWIND paths AS path
WITH RELATIONSHIPS(path) AS rels,vFMap,graph,uniqueGraphID
UNWIND rels AS r
// 生成虚拟图并挂上指标数据【获取图的指标数据】
// path：虚拟路径，vFMap：过滤器与实体ID的绑定，graph：以图搜图的原始匹配结果，uniqueGraphID：子图与虚拟子图的UUID唯一标记
CALL olab.schema.loop.vpath.ind(r,-1,vFMap) YIELD from,rel,to RETURN (from)-[rel]->(to) AS path,vFMap,graph,uniqueGraphID

查询与上海康达化工新材料集团股份有限公司同属于化工行业的其它公司三度担保金额超过10万的担保子图，并返回挂到ES指标的虚拟图

WITH '{"graph":{"nodes":[{"id":"1","labels":["公司"],"properties_filter":[{"name":"{var}.name=\'上海康达化工新材料集团股份有限公司\'"}]},{"id":"2","labels":["行业"],"properties_filter":[{"name":"{var}.name=\'化工\'"}]},{"id":"3","labels":["公司"]},{"id":"4","labels":["公司"]},{"id":"5","labels":["公司"]},{"id":"6","labels":["公司"]}],"relationships":[{"id":"1","type":"属于","startNode":"1","endNode":"2"},{"id":"2","type":"属于","startNode":"3","endNode":"2"},{"id":"3","type":"担保","startNode":"3","endNode":"4","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}}}"}]},{"id":"4","type":"担保","startNode":"4","endNode":"5","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}}}"}]},{"id":"5","type":"担保","startNode":"5","endNode":"6","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}}}"}]}]}}' as json
WITH olab.schema.auto.cypher(json,-1,3,false,true) AS cypher
CALL apoc.cypher.run(cypher,{}) YIELD value WITH randomUUID() AS uniqueGraphID,value.graph AS graph,value.graph.graph AS paths,value.vFMap AS vFMap
UNWIND paths AS path
WITH RELATIONSHIPS(path) AS rels,vFMap,graph,uniqueGraphID
UNWIND rels AS r
// 生成虚拟图并挂上指标数据【从ES获取图的指标数据】
// path：虚拟路径，vFMap：过滤器与实体ID的绑定，graph：以图搜图的原始匹配结果，uniqueGraphID：子图与虚拟子图的UUID唯一标记
CALL olab.schema.loop.vpath.ind(r,-1,vFMap) YIELD from,rel,to RETURN (from)-[rel]->(to) AS path,vFMap,graph,uniqueGraphID

【指定指标的返回字段_source】查询与上海康达化工新材料集团股份有限公司同属于化工行业的其它公司三度担保金额超过10万的担保子图，并返回挂到ES指标的虚拟图

WITH '{"graph":{"nodes":[{"id":"1","labels":["公司"],"properties_filter":[{"name":"{var}.name=\'上海康达化工新材料集团股份有限公司\'"}]},{"id":"2","labels":["行业"],"properties_filter":[{"name":"{var}.name=\'化工\'"}]},{"id":"3","labels":["公司"]},{"id":"4","labels":["公司"]},{"id":"5","labels":["公司"]},{"id":"6","labels":["公司"]}],"relationships":[{"id":"1","type":"属于","startNode":"1","endNode":"2"},{"id":"2","type":"属于","startNode":"3","endNode":"2"},{"id":"3","type":"担保","startNode":"3","endNode":"4","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}},_source:[\'defineDate\',\'amount\']}"}]},{"id":"4","type":"担保","startNode":"4","endNode":"5","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}},_source:[\'defineDate\',\'amount\']}"}]},{"id":"5","type":"担保","startNode":"5","endNode":"6","es_filter":[{"es_url":"10.20.13.130:9200","index_name":"gh_ind_rel_company_guarantee_company","query":"{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}}}]}},_source:[\'defineDate\',\'amount\']}"}]}]}}' as json
WITH olab.schema.auto.cypher(json,-1,3,false,true) AS cypher
CALL apoc.cypher.run(cypher,{}) YIELD value WITH randomUUID() AS uniqueGraphID,value.graph AS graph,value.graph.graph AS paths,value.vFMap AS vFMap
UNWIND paths AS path
WITH RELATIONSHIPS(path) AS rels,vFMap,graph,uniqueGraphID
UNWIND rels AS r
// 生成虚拟图并挂上指标数据【从ES获取图的指标数据】
// path：虚拟路径，vFMap：过滤器与实体ID的绑定，graph：以图搜图的原始匹配结果，uniqueGraphID：子图与虚拟子图的UUID唯一标记
CALL olab.schema.loop.vpath.ind(r,-1,vFMap) YIELD from,rel,to RETURN (from)-[rel]->(to) AS path,vFMap,graph,uniqueGraphID

ES过滤器转换为查询语句【vFMap中ES过滤器的解析函数】

// * @param esFilter:ES返回布尔值的过滤器
// * @param entityUniqueCode:实体ID与ES绑定的唯一代码
RETURN olab.es.filter.transfer({es_filter},{entityUniqueCode}) AS esQuery
RETURN olab.es.filter.transfer('custom.es.result.bool(\'10.20.13.130:9200\',\'gh_ind_rel_company_guarantee_company\',{size:1,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:{var}.entity_unique_code}}]}}})','47a9007e73d24417142253c206b9667e') AS esQuery

预装函数【OLAB依赖APOC的预装存储过程】

使用OLAB插件时部分函数和过程依赖下面的函数和过程，需要提前安装到集群【在CORE节点执行即可】

custom.es.result.bool

  • 功能 通过判断ES查询结果返回FALSE或者TRUE 
    @param {es-url}-ES地址
@param {index-name}-索引名称
@param {query-dsl}-ES查询
【返回值类型：BOOLEAN】
【结果集大于0返回TRUE】
RETURN custom.es.result.bool('10.20.13.130:9200','dl_default_indicator_def',{size:1,query:{term:{product_code:"PF0020020104"}}}) AS boolean
  • 安装方式 
    CALL apoc.custom.asFunction(
  'es.result.bool',
  'CALL apoc.es.query($esuUrl,$indexName,\'\',null,$queryDsl) YIELD value WITH value.hits.total.value AS count CALL apoc.case([count>0,\'RETURN TRUE AS countBool\'],\'RETURN FALSE AS countBool\') YIELD value RETURN value.countBool AS bool',
  'BOOLEAN',
  [['esuUrl','STRING'],['indexName','STRING'],['queryDsl','MAP']],
  false,
  '通过判断ES查询结果返回FALSE或者TRUE【结果集大于0返回TRUE】'
);

    custom.es.result

  • 功能 返回ES查询结果 
    @param {es-url}-ES地址
@param {index-name}-索引名称
@param {query-dsl}-ES查询
【返回值类型：MAP】
RETURN custom.es.result({esuUrl},{indexName},{queryDsl}) AS result
RETURN custom.es.result('10.20.13.130:9200','gh_ind_rel_company_guarantee_company',{size:100,query:{bool:{filter:{bool:{must:[{range:{amount:{gte:100000}}}]}},must:[{term:{entity_unique_code:'852c1ea85d8dd6b1354aa1a786dbc1db'}}]}}}) AS result
  • 安装方式 
    CALL apoc.custom.asFunction(
  'es.result',
  'CALL apoc.es.query($esuUrl,$indexName,\'\',null,$queryDsl) YIELD value RETURN value',
  'MAP',
  [['esuUrl','STRING'],['indexName','STRING'],['queryDsl','MAP']],
  false,
  '返回ES查询结果'
);
Search