Components

Logical components of droonga network

• network
• dataset
• message
• proxy
• migrator

network

• consists of one or more datasets
• connected with fluentd
• interact with clients via proxies

dataset

• made of groonga database
• partitioned by time-span and/or hash(key)
• can have one or more mirrors in the network
• datasets in a network is managed in a dictionary

proxy

• every fluentd in the network can be a proxy
• interprets every incoming message
• emit internal message according to description in the dictionary
• emitted messages are processed in workers
• workers may also emit messages which propagate in the network

migrator

• maintain the network to satisfy the description of the resource dictionary

message

• delivered as log events of fluentd
• queued and processed asynchronously
• 3 types (incoming/outgoing/internal)

incoming message

• 4 types (DDL/DML/find/collect)
• find offers simple key/value lookup function
• collect offers aggregative operation
• DML like insert/update operation are not supposed to return result respectively, but error/status info can be reported or stored in another dataset
• adapter can be defined freely which transforms incoming message in front of the proxy
• handler procedure of each message can be customized and may emit messages recursively

outgoing message

• 2 types (reply/alert)
• reply corresponds to incoming message
• alert can be emitted autonomically
• destination of alert messages must be registered and tied to the client in a dataset

collect operation

• parameters (collect, map, reduce, finalize, out)
• map and collect are exclusive
• map is applied for each row
• collect is supposed to do set oriented operation
• reduce is a procedure to aggrregate the result recursively
• finalize can modify the result after all reduce operations are done
• out can control whether the result is stored as a dataset itself or sent to the client
• result dataset may be maintained automatically when the source datasets are modified