Utilizing Powerful Extensions for Analytics and Operations

Utilizing Powerful Extensions
for Analytics and Operations

~ 30 cool Neo4j extensions
in ~30 minutes

Neo4j Developer Surface
2000-2010 0.x Embedded Java API
2010-2014 1.x REST
2014-2015 2.x Cypher over HTTP
2016 3.0.x Bolt, Official Language Drivers, User Defined Procedures
2016 3.1.x User Defined Functions
2017 3.2.x User Defined Aggregation Functions

Procedures
Functions
Aggregate Functions

Can be written in any JVM language

Callable Standalone
and in
Cypher Statements

CALL example.search('User','name:Brook*')

public class FullTextIndex {
@Context
public GraphDatabaseService db;
@Procedure( name = "example.search", mode = Procedure.Mode.READ )
public Stream<SearchHit> search( @Name("index") String index,
@Name("query") String query ) {
if( !db.index().existsForNodes( index )) {
return Stream.empty();
}
return db.index().forNodes( index ).query( query ).stream()
.map( SearchHit::new );
}
public static class SearchHit {
public final Node node;
SearchHit(Node node) { this.node = node; }
}
}

try ( Driver driver = GraphDatabase.driver( "bolt://localhost",
Config.build().toConfig() ) ) {
try ( Session session = driver.session() ) {
String call = "CALL example.search('User',$query)";
Map<String,Object> params = singletonMap( "query", "name:Brook*");
StatementResult result = session.run( call, params);
while ( result.hasNext() {
// process results
}
}
}

Useable in any Cypher
expression or lightweight
computation

RETURN example.join(['Hello', 'World'],' ')
=> "Hello World"

public class Join {
@UserFunction
@Description("example.join(['s1','s2',...], delimiter)
- join the given strings with the given delimiter.")
public String join(
@Name("strings") List<String> strings,
@Name(value = "delimiter", defaultValue = ",") String delimiter ) {
if ( strings == null || delimiter == null ) {
return null;
}
return String.join( delimiter, strings );
}
}

Config.build().toConfig() ) )
{
try ( Session session = driver.session() )
{
String query = "RETURN example.join(['Hello', 'World']) AS result";
String result = session.run( query )
.single().get( "result" ).asString();
}
}

User Defined
Aggregation Functions

Custom, efficient aggregations
for Data Science and BI

UNWIND ['abc', 'abcd', 'ab'] AS string
RETURN example.longestString(string)
=> 'abcd'

public class LongestString {
@UserAggregationFunction
@Description( "aggregates the longest string found" )
public LongStringAggregator longestString() {
return new LongStringAggregator();
}
public static class LongStringAggregator {
private int longest;
private String longestString;
@UserAggregationUpdate
public void findLongest( @Name( "string" ) String string ) {
if ( string != null && string.length() > longest) {
longest = string.length();
longestString = string;
}
}
@UserAggregationResult
public String result() { return longestString; }
}
}

Config.build().toConfig() ) ) {
try ( Session session = driver.session() ) {
String query = "UNWIND ['abc', 'abcd', 'ab'] AS string " +
"RETURN example.longestString(string) AS result";
String result = session.run(query).single().get("result").asString();
}
}

> 400
procedures and functions

Load from a relational database

apoc.load.jdbc
WITH "jdbc:mysql://localhost:3306/northwind?user=root" AS url
CALL apoc.load.jdbc(url,"products")
YIELD row
MERGE (p:Product {id: row.ProductID})
SET p.name = row.ProductName, p.unitPrice = row.UnitPrice

YIELD row
Execute procedure

Apply Cypher transformation
YIELD row

apoc.load.xml
CALL apoc.load.xml('http://overpass.osm.rambler.ru/cgi/xapi_meta?*[bbox=11.54,48.14,11.543,48.145]')
YIELD value
UNWIND value["_children"] AS child
WITH child WHERE child["_type"] = "node"
WITH child.id AS id,
child.lat AS latitude,
child.lon AS longitude,
child["user"] AS userName
MERGE (point:Point {id: id})
SET point.latitude = latitude,
point.longitude = longitude
MERGE (user:User {name: userName})
MERGE (user)-[:EDITED]->(point)

Execute procedure
YIELD value

UNWIND the array of elements
YIELD value

Filter rows
YIELD value

Apply Cypher transformation
YIELD value

apoc.load.json
WITH "https://api.stackexchange.com/2.2/questions?pagesize=100&order=desc&sort=creation&tagged=neo4j&site=stackoverflow&filter=!5-i6Zw8Y)4W7vpy91PMYsKM-k9yzEsSC1_Uxlf" AS url
CALL apoc.load.json(url) YIELD value
UNWIND value.items AS q
MERGE (question:Question {id:q.question_id})
ON CREATE SET question.title = q.title,
question.share_link = q.share_link,
question.favorite_count = q.favorite_count
MERGE (owner:User {id:q.owner.user_id})
ON CREATE SET owner.display_name = q.owner.display_name
MERGE (owner)-[:ASKED]->(question)
FOREACH (tagName IN q.tags |
MERGE (tag:Tag {name:tagName}) MERGE (question)-[:TAGGED]->(tag))
FOREACH (a IN q.answers |
MERGE (question)<-[:ANSWERS]-(answer:Answer {id:a.answer_id})
MERGE (answerer:User {id:a.owner.user_id})
ON CREATE SET answerer.display_name = a.owner.display_name
MERGE (answer)<-[:PROVIDED]-(answerer)
)

WITH "https://api.stackexchange.com/2.2/questions?pagesize=100&order=desc&sort=creation&tagged=neo4j&site=stackoverflow&filter=!5-i6Zw8Y)4W7vpy91PMYsKM-k9yzEsSC1_Uxlf" AS url
CALL apoc.load.json(url) YIELD value
UNWIND value.items AS q
MERGE (question:Question {id:q.question_id})
ON CREATE SET question.title = q.title,
question.share_link = q.share_link,
question.favorite_count = q.favorite_count
MERGE (owner:User {id:q.owner.user_id})
ON CREATE SET owner.display_name = q.owner.display_name
MERGE (owner)-[:ASKED]->(question)
)
Use FOREACH for arrays within a row
)

apoc.refactor.mergeNodes
MATCH (n:Person)
WITH n.email AS email, collect(n) as people
WHERE size(people) > 1
CALL apoc.refactor.mergeNodes(people)
YIELD node
RETURN node

apoc.create.addLabels
MATCH (n:Movie)
CALL apoc.create.addLabels( id(n), [ n.genre ] ) YIELD node
REMOVE node.genre
RETURN node

Run large scale updates
CALL apoc.periodic.iterate(
'MATCH (n:Person) RETURN n',
'SET n.name = n.firstName + " " + n.lastName',
{batchSize:10000, parallel:true})

Compute soundex encoding of a string
CALL apoc.text.phonetic('Hello, dear User!')
YIELD value
RETURN value
// will return 'H436'

How similar do two strings sound?
CALL apoc.text.phoneticDelta(
'Hello Mr Rabbit', 'Hello Mr Ribbit')
// will return '4' (very similar)

Extract domain names
WITH 'http://www.example.com/all-the-things' AS url
RETURN apoc.data.domain(url)
// will return 'www.example.com'

Date to Timestamp
RETURN apoc.date.parse(
'2015/03/25 03:15:59',
's',
'yyyy/MM/dd HH:mm:ss'
)
// will return 1427253359

Timestamp to Date
RETURN apoc.date.format(
1427253359,
's',
'yyyy/MM/dd HH:mm:ss'
)
// will return "2015/03/25 03:15:59"

semantics.importRDF
CALL semantics.importRDF("file:///industry.ntriples","N-Triples", {})

Thomson Reuters' OpenPermID Graph

com.graphaware.runtime.enabled=true
#ES becomes the module ID:
com.graphaware.module.ES.2=com.graphaware.module.es.ElasticSearchModuleBootstrapper
#URI of Elasticsearch
com.graphaware.module.ES.uri=localhost
#Port of Elasticsearch
com.graphaware.module.ES.port=9201
conf/neo4j.conf

CALL ga.es.queryNode('{"query":
{"match":{"name":"alessandro"}}}')
YIELD node, score
RETURN node, score
Find nodes

CALL ga.es.queryRelationship('{"query":
{"match":{"city":"paris"}}}')
YIELD relationship, score
RETURN relationship, score
Find relationships

dzone.com/refcardz/graph-powered-search-neo4j-amp-elasticsearch

CALL graph.versioner.init('Person',
{ssn: 123456789, name: 'Marco'},
{address: 'Via Roma 11'}
)
Create node

{ssn: 123456789, name: 'Marco'},
)
Immutable properties

{ssn: 123456789, name: 'Marco'},
)
State properties

MATCH (p:Person {name: "Marco"})
WITH p
CALL graph.versioner.update(p, {address: 'Via Roma 12'})
YIELD node
RETURN node
Update state

MATCH (p:Person {name: "Marco"})
WITH p
CALL graph.versioner.update(p, {address: 'Via Roma 12'})
YIELD node
RETURN node
Pass in the new state

lyonwj.com/2016/08/09/neo4j-spatial-procedures-congressional-boundaries/

CALL spatial.addWKTLayer('geom', 'wkt')
Create spatial index

CREATE (d:District {...})
SET d.wkt = 'MULTIPOLYGON(((23.22, ... )))'
Add spatial property to nodes

MATCH (d:District)
WITH collect(d) AS districts
CALL spatial.addNodes('geom', districts) YIELD node
RETURN count(*)
Add nodes to spatial index

CALL spatial.withinDistance('geom',
{latitude: 37.563440, longitude: -122.322265}, 1)
YIELD node AS d
WITH d, d.wkt AS wkt, d.state AS state, d.district AS district
LIMIT 1
MATCH (d)<-[:REPRESENTS]-(l:Legislator)
MATCH (l)-[:SERVES_ON]->(c:Committee)
MATCH (c)<-[:REFERRED_TO]-(b:Bill)
MATCH (b)-[:DEALS_WITH]->(s:Subject)
RETURN *
Query nodes by location

YIELD node AS d
LIMIT 1
RETURN *
Finds nodes within 1km

YIELD node AS d
LIMIT 1
RETURN *
Continue with the rest of the query

CREATE (n:News)
SET n.text = "Scores of people were already lying dead or
injured inside a crowded Orlando nightclub,
and the police had spent hours trying to connect with the
gunman and end the situation without further violence.
But when Omar Mateen threatened to set off explosives, the
police decided to act, and pushed their way through a
wall to end the bloody standoff."
Annotating text

MATCH (n:News)
CALL ga.nlp.annotate({text: n.text, id: id(n)})
YIELD result
MERGE (n)-[:HAS_ANNOTATED_TEXT]->(result)
RETURN result
Annotating text

MATCH (n:News)
YIELD result
RETURN result
Execute procedure

MATCH (n:News)
YIELD result
RETURN result
Relate to text node

Graph of Thrones
● Community Detection
● Centrality

Neo4j
Native Graph
Database
Analytics
Integrations
Cypher Query
Language
Wide Range of
APOC Procedures
Optimized
Graph Algorithms

Finds the optimal
path or evaluates
route availability and
quality
Evaluates how a
group is clustered
or partitioned
Determines the
importance of distinct
nodes in the network

Single Source Short Path
All-Nodes SSP
Parallel BFS / DFS
Strongly Connected
Components
Union Find / WCC
Label Propagation
Louvain
Triangle-Count /
Clustering Coefficent
PageRank (baseline)
Betweeness
Closeness
Degree

1.Call as Cypher procedure
2.Pass in specification (Label, Prop, Query) and configuration
3.~.stream variant returns (a lot) of results
CALL algo.<name>.stream('Label','TYPE', {conf})
YIELD nodeId, score
4.non-stream variant writes results to graph
and returns statistics
CALL algo.<name>('Label','TYPE', {conf})
Usage

Pass in Cypher statement for node and relationship lists.
CALL algo.<name>(
'MATCH ... RETURN id(n)',
'MATCH (n)-->(m)
RETURN id(n) as source, id(m) as target',
{graph:'cypher'})
Cypher Projection

CALL algo.pageRank.stream('Page', 'Link', {iterations:5})
YIELD node, score
WITH *
ORDER BY score DESC
LIMIT 5
RETURN node.title, score
Important pages
+--------------------------------------+
| node.title | score |
+--------------------------------------+
| "United States" | 13349.2 |
| "Animal" | 6077.77 |
| "France" | 5025.61 |
| "List of sovereign states" | 4913.92 |
| "Germany" | 4662.32 |
+--------------------------------------+
5 rows 20 seconds

CALL algo.labelPropagation()
Largest Clusters

CALL algo.labelPropagation()
MATCH (n:Page)
WITH n ORDER BY n.pagerank DESC
LIMIT 1000000
WITH n.partition AS partition,
count(*) AS clusterSize,
collect(n.title) AS pages
RETURN pages[0] AS mainPage,
pages[1..10] AS otherPages
ORDER BY clusterSize DESC
LIMIT 20
Largest Clusters

Graph Algorithms Sandbox
neo4j.com/sandbox

var viz;
function draw() {
var config = {
container_id: "viz",
server_url: "bolt://localhost:7687",
server_user: "neo4j",
server_password: "sorts-swims-burglaries",
labels: {
"Character": {
"caption": "name",
"size": "pagerank",
"community": "community"
}
},
relationships: {
"INTERACTS": {
"thickness": "weight",
"caption": false
}
},
initial_cypher: "MATCH (n)-[r:INTERACTS]->(m) RETURN *"
};
viz = new NeoVis.default(config);
viz.render();
}

GraphQL is a query language for your API, and a
server-side runtime for executing queries by using a
type system you define for your data.
What is it?

GraphQL is a query language for your API, and a
server-side runtime for executing queries by using a
type system you define for your data.
What is it?
type Planet {
name: String
climate: String
}
type Character {
name: String
friends: [Character]
homeWorld: Planet
species: Species
}
type Species {
name: String
lifespan: Int
origin: Planet
}

Auto translating GraphQL → Cypher
github.com/neo4j-graphql/neo4j-graphql
github.com/neo4j-graphql/neo4j-graphql-js

Neo4j-GraphQL Database Extension
Neo4j serves GraphQL endpoint directly

dbms.unmanaged_extension_classes=org.neo4j.graphql=/graphql
dbms.security.procedures.whitelist=graphql.*
conf/neo4j.conf

CALL graphql.idl('
type Movie {
title: String!
released: Int
actors: [Person] @relation(name:"ACTED_IN",direction:IN)
}
type Person {
name: String!
born: Int
movies: [Movie] @relation(name:"ACTED_IN")
}'
)

WITH '{ Person(born: 1961) { name, born } }' as query, {} as params
CALL graphql.execute(query,params)
YIELD result
UNWIND result.Person as p
RETURN p.name, p.born

import { neo4jgraphql } from 'neo4j-graphql-js';
const resolvers = {
Query: {
Movie(object, params, ctx, resolveInfo) {
return neo4jgraphql(object, params, ctx, resolveInfo);
}
}
};

The GRAND stack
GraphQL
React
Apollo
Neo4j Database

dzone.com/refcardz/an-overview-of-graphql

How do I find out
about more cool stuff?

This Week in Neo4j - Developer Update
neo4j.com/tag/twin4j

We want to hear from YOU
bit.ly/neo4j-survey

Get Dressed: Neo4j Graph Gear Store
graphgearstore.com

Toronto Meetup - Tuesday May 8th
meetup.com/Women-Who-Code-Toront
o

(you)-[:HAVE]->(:Question)<-[:ANSWERS]-(will)

Utilizing Powerful Extensions for Analytics and Operations

More Related Content

What's hot (18)

Similar to Utilizing Powerful Extensions for Analytics and Operations (20)

More from Neo4j (20)

Recently uploaded (20)

Utilizing Powerful Extensions for Analytics and Operations