This repository was archived by the owner on May 6, 2021. It is now read-only.
This repository was archived by the owner on May 6, 2021. It is now read-only.
Error when indexing infinite/nan values #158
Description
Pushed by a customer;
- happens during bulk indexation
Versions
- Neo4j 3.5.8 Enterprise
- graphaware-neo4j-to-elasticsearch-3.5.1.53.11.jar
- graphaware-server-enterprise-all-3.5.1.53.jar
- graphaware-uuid-3.5.1.53.17.jar
What happens:
- the value stored in Java is
Double.NEGATIVE_INFINITY - the Elasticsearch client tries to serialize the value to send it to Elasticsearch
- the serializer complains that
-Infinityis not standard JSON, i.e. it does not know how to serialize the value.
How to fix:
- set up the HttpClient to use a custom Gson serializer to avoid the issue
Option 1:
- use
GsonBuilder.serializeSpecialFloatingPointValues()to tolerate-Infinityetc - problem: this is not valid JSON and migh break in ES
Option 2:
- create a serialization adapter to convert
-Infinityinto a"-Infinity"(a valid JSON string)
Error log:
2020-02-17 12:32:54.266+0000 WARN Runtime exception occurred during transaction execution. -Infinity is not a valid double value as per JSON specification. To override this behavior, use GsonBuilder.serializeSpecialFloatingPointValues() method.
java.lang.IllegalArgumentException: -Infinity is not a valid double value as per JSON specification. To override this behavior, use GsonBuilder.serializeSpecialFloatingPointValues() method.
at com.google.gson.Gson.checkValidFloatingPoint(Gson.java:317)
at com.google.gson.Gson$3.write(Gson.java:285)
at com.google.gson.Gson$3.write(Gson.java:271)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:208)
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:145)
at com.google.gson.Gson.toJson(Gson.java:661)
at com.google.gson.Gson.toJson(Gson.java:640)
at com.google.gson.Gson.toJson(Gson.java:595)
at com.google.gson.Gson.toJson(Gson.java:575)
at io.searchbox.action.AbstractAction.getData(AbstractAction.java:155)
at io.searchbox.core.Bulk.getData(Bulk.java:102)
at io.searchbox.client.http.JestHttpClient.prepareRequest(JestHttpClient.java:81)
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:46)
at com.graphaware.module.es.executor.BulkOperationExecutor.executeBulk(BulkOperationExecutor.java:83)
at com.graphaware.module.es.executor.BulkOperationExecutor.flush(BulkOperationExecutor.java:63)
at com.graphaware.module.es.ElasticSearchWriter.flushBatch(ElasticSearchWriter.java:147)
at com.graphaware.module.es.ElasticSearchWriter.processOperations(ElasticSearchWriter.java:143)
at com.graphaware.module.es.ElasticSearchModule.lambda$reindexNodes$2(ElasticSearchModule.java:184)
at com.graphaware.tx.executor.batch.IterableInputBatchTransactionExecutor.lambda$processQueue$1(IterableInputBatchTransactionExecutor.java:115)
at com.graphaware.tx.executor.single.SimpleTransactionExecutor.doExecuteInTransaction(SimpleTransactionExecutor.java:69)
at com.graphaware.tx.executor.single.SimpleTransactionExecutor.executeInTransaction(SimpleTransactionExecutor.java:58)
at com.graphaware.tx.executor.batch.IterableInputBatchTransactionExecutor.processQueue(IterableInputBatchTransactionExecutor.java:103)
at com.graphaware.tx.executor.batch.IterableInputBatchTransactionExecutor.doExecute(IterableInputBatchTransactionExecutor.java:76)
at com.graphaware.tx.executor.batch.DisposableBatchTransactionExecutor.execute(DisposableBatchTransactionExecutor.java:35)
at com.graphaware.module.es.ElasticSearchModule.reindexNodes(ElasticSearchModule.java:189)
at com.graphaware.module.es.ElasticSearchModule.lambda$reindex$0(ElasticSearchModule.java:146)
at java.lang.Thread.run(Thread.java:748)
.....
2020-02-17 12:32:54.267+0000 WARN Rolled back transaction for batch number 79540