OpenGIN

💡 Note (α)
Name needs to be proposed, voted and finalized.

🚀 Running Services

1. Run CORE API Service

-Read about running the CORE Service

2. Run Read API Service

-Read about running the Read API

3. Run Ingestion API Service

-Read about running the Ingestion API

4. Run Swagger-UI

-Read about running the Swagger UI

5. Database Cleanup Service

The cleanup service provides a way to clean all databases (PostgreSQL, MongoDB, Neo4j) before and after running tests or services.

Usage:

# Clean databases before starting services
docker-compose --profile cleanup run --rm cleanup /app/cleanup.sh pre

# Clean databases after services complete
docker-compose --profile cleanup run --rm cleanup /app/cleanup.sh post

# Clean databases anytime you need
docker-compose --profile cleanup run --rm cleanup /app/cleanup.sh pre

What it cleans:

• PostgreSQL: attribute_schemas, entity_attributes, and all attr_* tables
• MongoDB: metadata and metadata_test collections
• Neo4j: All nodes and relationships

Note: The cleanup service uses the cleanup profile, so it won't start automatically with docker-compose up.

6. Database Backup and Restore

The system provides comprehensive backup and restore capabilities for all databases.

Local Backup Management:

# Create backups
./deployment/development/init.sh backup_mongodb
./deployment/development/init.sh backup_postgres
./deployment/development/init.sh backup_neo4j

# Restore from local backups
./deployment/development/init.sh restore_mongodb
./deployment/development/init.sh restore_postgres
./deployment/development/init.sh restore_neo4j

GitHub Integration:

# Restore from GitHub releases
./deployment/development/init.sh restore_from_github 0.0.1
./deployment/development/init.sh list_github_versions

For detailed backup and restore documentation, see Backup Integration Guide.

---

Run a sample query with CURL

Ingestion API

Create

curl -X POST http://localhost:8080/entities \
-H "Content-Type: application/json" \
-d '{
  "id": "12345",
  "kind": {
    "major": "example",
    "minor": "test"
  },
  "created": "2024-03-17T10:00:00Z",
  "terminated": "",
  "name": {
    "startTime": "2024-03-17T10:00:00Z",
    "endTime": "",
    "value": {
      "typeUrl": "type.googleapis.com/google.protobuf.StringValue",
      "value": "entity-name"
    }
  },
  "metadata": [
    {"key": "owner", "value": "test-user"},
    {"key": "version", "value": "1.0"},
    {"key": "developer", "value": "V8A"}
  ],
  "attributes": [],
  "relationships": []
}'

Read

curl -X GET http://localhost:8080/entities/12345

Update

TODO: The update creates a new record and that's a bug, please fix it.

curl -X PUT http://localhost:8080/entities/12345 \
  -H "Content-Type: application/json" \
  -d '{
    "id": "12345",
    "kind": {
      "major": "example",
      "minor": "test"
    },
    "created": "2024-03-18T00:00:00Z",
    "name": {
      "startTime": "2024-03-18T00:00:00Z",
      "value": "entity-name"
    },
    "metadata": [
      {"key": "version", "value": "5.0"}
    ]
  }'

Delete

curl -X DELETE http://localhost:8080/entities/12345

Read API

Retrieve Metadata

curl -X GET "http://localhost:8081/v1/entities/12345/metadata"

Run E2E Tests

Make sure the CORE server and the API server are running.

Note when making a call to ReadEntity, the ReadEntityRequest must be in the following format (output can be one or more of metadata, relationships, attributes):

ReadEntityRequest readEntityRequest = { entity: { id: entityId, kind: {}, created: "", terminated: "", name: { startTime: "", endTime: "", value: check pbAny:pack("") }, metadata: [], attributes: [], relationships: [] }, output: ["relationships"] };

Run Ingestion API Tests

cd opengin/tests/e2e
python basic_core_tests.py

Run Read API Tests

cd opengin/tests/e2e
python basic_read_tests.py

Implementation Progress

Track Progress