Merge pull request #75 from graphfoundation/3.5-int

3.5 - Integrating latest from neo4j/3.5

Author: bradnussbaum

CONTRIBUTING.adoc

@@ -53,6 +53,23 @@ If there are multiple commits, and the PR is fine to merge online, use *Squash a
 
 Using the issue id in the summary line will allow us to keep track of commits belonging together.
 
+=== Branching Policy
+
+We do have a branch for each major version of Neo4j, e.g. `3.5`, `4.0` and `4.1`. 
+As a contributor you're asked to use newest possible branch for you PR.
+
+When your PR is accepted and merged it's the responsibility of the maintainers who merged it to cherry-pick that changes to any newer branch. 
+Once cherry-picking is done, the maintainers should mark that PR with the `cherry-picked` label. 
+
+Please indicate in your PR message text if your PR needs a different behaviour - e.g.  if the feature you're fixing has been removed in a newer branch or APIs have changed too much and you have a separate PR for the newer branch.
+
+EXAMPLE: 
+
+1. You're fixing a bug being reported for 3.5.x.x. 
+2. You're choosing the 3.5 branch as base for your PR branch. 
+3. Once you're done you send a PR.
+4. When a maintainer merges that PR, they also take care to cherry-pick it to 4.0 and any more recent branches.
+
 === Handling pull requests
 
 Be polite. 
@@ -84,4 +101,4 @@ curl $PULL_REQUEST_URL.patch | git am --ignore-whitespace
 Make sure to push changes to a PR to the original remote branch.
 This will cause the pull request UI in GitHub show and link those commits.
 
-This guideline document is based on the https://github.com/spring-projects/spring-data-build/blob/master/CONTRIBUTING.adoc[spring-data guidelines], thanks @olivergierke.
+This guideline document is based on the https://github.com/spring-projects/spring-data-build/blob/master/CONTRIBUTING.adoc[spring-data guidelines], thanks @olivergierke.

build.gradle

@@ -8,6 +8,7 @@ plugins {
     id "io.codearte.nexus-staging" version "0.20.0"
     id 'maven-publish'
     id 'antlr'
+    id "org.sonarqube" version "2.7"
 }
 asciidoctorj {
     version = '1.6.1'
@@ -39,7 +40,7 @@ description = """neo4j-apoc-procedures"""
 
 ext {
     // NB: due to version.json generation by parsing this file, the next line must not have any if/then/else logic
-    neo4jVersion = "3.5.16"
+    neo4jVersion = "3.5.17"
     // instead we apply the override logic here
     neo4jVersionEffective = project.hasProperty("neo4jVersionOverride") ? project.getProperty("neo4jVersionOverride") : neo4jVersion
     testContainersVersion = '1.11.0'
@@ -157,7 +158,7 @@ dependencies {
     testCompile group: 'com.couchbase.client', name: 'java-client', version: '2.7.2'
 
     compileOnly group: 'org.neo4j', name: 'neo4j', version: neo4jVersionEffective
-    compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.10.3'
+    compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.9.10.4'
     compile group: 'com.opencsv', name: 'opencsv', version: '4.2' 
     compileOnly group: 'org.ow2.asm', name: 'asm', version: '5.0.2'
     compile group: 'com.github.javafaker', name: 'javafaker', version: '0.10'

docs/asciidoc/graph-updates/ttl.adoc

@@ -3,69 +3,179 @@
 
 [abstract]
 --
-This section describes procedures that can be used to remove nodes from the database once a time limit has been reached.
+This section describes procedures that can be used to remove nodes from the database once a time or time limit has been reached.
 --
 
 Some nodes are not meant to live forever.
 That's why with APOC you can specify a time by when they are removed from the database, by utilizing a schema index and an additional label.
-A few convenience procedures help with that.
+A few procedures help with that.
 
-ifdef::backend-html5[]
-++++
-<iframe width="560" height="315" src="https://www.youtube.com/embed/e9aoQ9xOmoU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
-++++
-endif::[]
+This section includes:
 
-Enable TTL with setting in `neo4j.conf` : `apoc.ttl.enabled=true`
+* <<ttl-available-procedures, Available Procedures>>
+* <<ttl-config-parameters, Configuration and Parameters>>
+//* <<ttl-handson-video, Time-To-Live Hands-On>>
+* <<ttl-examples, Examples>>
+    ** <<ttl-expireAt, Expire at Specified Time>>
+    ** <<ttl-expireIn, Expire after Amount of Time>>
+* <<ttl-process, Manual Process: How TTL Works>>
 
-There are some convenience procedures to expire nodes.
+[[ttl-available-procedures]]
+== Available Procedures
 
-You can also do it yourself by running
+The table below describes the available procedures:
+
+[separator=¦,opts=header,cols="1,1m,1m,5"]
+|===
+include::../../../build/generated-documentation/apoc.ttl.csv[]
+|===
+
+[[ttl-config-parameters]]
+== Configuration and Parameters
+
+For configuration, you will need to enable time-to-live functionality with the following settings in `neo4j.conf`:
+
+.neo4j.conf
+[source,properties]
+----
+apoc.ttl.enabled=true
+
+# Optional: controls the repeat frequency
+# apoc.ttl.schedule=5
+----
+
+In the available procedures listed above, there are several parameters with specific values.
+The table below outlines values and formats for the valid parameters.
+
+[options="header"]
+|===
+| Parameter | Description | Possible Values | Examples
+| `node` | The entity or entities to add the label and property of time-to-live (previous selection statement needed) | Any node or group of nodes fitting desired criteria | `n`, `person`, `group`
+| `epochTime` | The datetime value of when the node(s) should expire | Any value in epoch seconds or millisecond format | `1540944000`, `1582209630000`
+| `time-unit` | Measurement of units for input value | `ms, s, m, h, d` (long forms: `millis, milliseconds, seconds, minutes, hours, days`) | `milliseconds`, `h`
+|===
+
+//[[ttl-handson-video]]
+//== Time-To-Live Hands-On
+//
+//ifdef::backend-html5[]
+//++++
+//<iframe width="560" height="315" src="https://www.youtube.com/embed/e9aoQ9xOmoU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+//++++
+//endif::[]
+
+[[ttl-examples]]
+== Examples: Time-To-Live
+
+This section includes examples showing how to use the time-to-live procedures.
+These examples are based on a movies dataset, which can be imported by running the following Cypher query:
+
+include::../export/createExportGraph.adoc[]
+
+The Neo4j Browser visualization below shows the imported graph:
+
+image::play-movies.png[title="Movies Graph Visualization"]
+
+[[ttl-expireAt]]
+=== Expire node(s) at specified time
+
+The `apoc.ttl.expireAtInstant` procedure deletes a node or group of nodes after the datetime specified.
+
+To remove a single node or set of nodes, we can use a selection query prior to calling the procedure that defines which nodes we want to apply the time-to-live label and property.
+We then call the procedure and pass in the selected node(s), the future datetime at which we want the nodes to be removed, and the specificity of the datetime (seconds, milliseconds, etc).
 
 [source,cypher]
 ----
-SET n:TTL
-SET n.ttl = timestamp() + 3600
+MATCH (movie:Movie)<-[produced:PRODUCED]-(person:Person)
+CALL apoc.ttl.expireAtInstant(person,1585176720,'s')
+RETURN movie, produced, person
 ----
 
-[cols="1m,5"]
+.Results
+[opts="header"]
 |===
-| CALL apoc.date.expire.in(node,time,'time-unit') | expire node in given time-delta by setting :TTL label and `ttl` property
-| CALL apoc.date.expire(node,time,'time-unit') | expire node at given time by setting :TTL label and `ttl` property
+| "movie" | "produced" | "person"
+| {"title":"The Matrix","tagline":"Welcome to the Real World","released":1999} | {} | {"name":"Joel Silver","ttl":1585176720000,"born":1952}
 |===
 
-Optionally set `apoc.ttl.schedule=5` as repeat frequency.
+After the point in time specified (in this case, after `2020-03-25 17:52:00`), the node(s) will be expired and deleted from the graph.
+Running the statement below will return no results for our example graph.
+
+[source,cypher]
+----
+MATCH (movie:Movie)<-[produced:PRODUCED]-(person:Person)
+RETURN movie, produced, person
+----
 
-== Process
+[[ttl-expireIn]]
+=== Expire node(s) after specified time period
 
-30s after startup an index is created:
+The `apoc.ttl.expireAfterTimeLength` procedure deletes a node or group of nodes after the length of time specified.
+//LEFT OFF HERE!
+Just as with the similar procedure above, we can use a selection query prior to calling the procedure that defines which nodes we want to apply the time-to-live label and property.
+We then call the procedure and pass in the selected node(s), the time delta from current time at which we want the nodes to be removed, and the specificity of the time amount (seconds, milliseconds, etc).
 
 [source,cypher]
 ----
-CREATE INDEX ON :TTL(ttl)
+MATCH (movie:Movie)<-[produced:PRODUCED]-(person:Person)
+CALL apoc.ttl.expireAfterTimeLength(person,1585176720,'s')
+RETURN movie, produced, person
 ----
 
-At startup a statement is scheduled to run every 60s (or configure in `neo4j.conf` -  `apoc.ttl.schedule=120`)
+.Results
+[opts="header"]
+|===
+| "movie" | "produced" | "person"
+| {"title":"The Matrix","tagline":"Welcome to the Real World","released":1999} | {} | {"name":"Joel Silver","ttl":120000,"born":1952}
+|===
+
+After the length of time specified has passed (in this case, after `120 seconds`), the node(s) will be expired and deleted from the graph.
+Running the statement below will return no results for our example graph.
 
 [source,cypher]
 ----
-MATCH (t:TTL) where t.ttl < timestamp() WITH t LIMIT 1000 DETACH DELETE t
+MATCH (movie:Movie)<-[produced:PRODUCED]-(person:Person)
+RETURN movie, produced, person
 ----
 
-The `ttl` property holds the *time when the node is expired in milliseconds since epoch*.
+[[ttl-process]]
+== Manual Process: How TTL Works
 
-You can expire your nodes by setting the :TTL label and the ttl property:
+You can also do the time-to-live process manually by running the following steps:
 
+* Set the `:TTL` label and `ttl` property on the node(s) you want to expire.
 
 [source,cypher]
 ----
-MATCH (n:Foo) WHERE n.bar SET n:TTL, n.ttl = timestamp() + 10000;
+SET n:TTL
+SET n.ttl = timestamp() + 3600
 ----
 
-There is also a procedure that does the same:
+The `ttl` property holds the *time when the node is expired in milliseconds since epoch*.
+
+* Create an index on the time-to-live label and property.
 
 [source,cypher]
 ----
-CALL apoc.date.expire(node,time,'time-unit');
-CALL apoc.date.expire(n,100,'s');
+CREATE INDEX ON :TTL(ttl)
+----
+
+When using the procedure, the index is created 30 seconds after startup.
+
+* Remove node(s) that have passed the expiration time or length of time
+
+[source,cypher]
+----
+MATCH (t:TTL) where t.ttl < timestamp() WITH t LIMIT 1000 DETACH DELETE t
+----
+
+When using the procedure, the deletion statement to remove nodes past expiration will run every 60 seconds.
+You can also configure the schedule by adding the following setting in `neo4j.conf`:
+
+
+.neo4j.conf
+[source,properties]
 ----
+# Optional: controls the repeat frequency
+apoc.ttl.schedule=120
+----

docs/asciidoc/import/loadjson.adoc

@@ -195,3 +195,28 @@ You will receive the following response:
   "weight": 0.02548018842935562
 }
 ----
+
+== Import JSON
+
+If you used the `apoc.export.json.all` to export the graph, we provide the `apoc.import.json` to reimport that data.
+
+[source,cypher]
+----
+CALL apoc.import.json($file, $config)
+----
+
+The `$config` parameter is a map
+
+[opts=header,cols="m,m"]
+|===
+| name | default | description
+| unwindBatchSize | 5000 | the batch size of the unwind
+| txBatchSize | 5000 | the batch size of the transacttion
+| importIdName | neo4jImportId | the name of the "id" field into the used for the import it refers to the "id" field into the root object of the json.
+| nodePropertyMappings | {} | The mapping label/property name/property type for Custom Neo4j types (point date). I.e. { User: { born: 'Point', dateOfBirth: 'Datetime' } }
+| relPropertyMappings | {} | The mapping rel type/property name/property type for Custom Neo4j types (point date). I.e. { KNOWS: { since: 'Datetime' } }
+|===
+
+N.b. Supported Neo4j types via config mappings are:
+
+Point, Localdate, Localtime, Localdatetime, Duration, offsettime, Zoneddatetime

docs/asciidoc/path-finding/config-params.adoc

@@ -6,8 +6,12 @@ The procedures support the following config parameters:
 | name | type | default | description
 | minLevel | Long | -1 | the minimum number of hops in the traversal. Must be 0 or 1 if specified
 | maxLevel | Long | -1 | the maximum number of hops in the traversal
-| relationshipFilter | String | null | the relationship types and directions to traverse. See <<expand-subgraph-relationship-filters>>.
-| labelFilter | String | null | the node labels to traverse. See <<expand-subgraph-label-filters>>.
+| relationshipFilter | String | null | the relationship types and directions to traverse.
+
+See <<expand-subgraph-relationship-filters>>.
+| labelFilter | String | null | the node labels to traverse.
+
+See <<expand-subgraph-label-filters>>.
 | beginSequenceAtStart | Boolean | true | starts matching sequences of node labels and/or relationship types (defined in `relationshipFilter`, `labelFilter`, or `sequences`) one node away from the start node.
 | bfs | Boolean | true | use Breadth First Search when traversing. Uses Depth First Search if set to `false`
 | filterStartNode | Boolean | false | whether the `labelFilter` and `sequence` apply to the start node of the expansion.