-
Notifications
You must be signed in to change notification settings - Fork 99
Minor docs corrections plus single-side-join docs rewrite #8229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: staging
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -77,7 +77,7 @@ components { | |
| namePattern | false | .* | Regexp for filtering operations by operationId or by created service name (concatenation of HTTP method, path and parameters) | | ||
| security | false | | Configuration for [authentication](https://swagger.io/docs/specification/authentication/) for each `securitySchemas` defined in the *OpenAPI interface definition* | | ||
| security.*.type | false | | Type of security configuration for a given security schema. Currently only `apiKey` is supported | | ||
| security.*.apiKeyValue | false | | API key that will be passed into the service via header, query parameter or cookie (depending on definition provided in OpenAPI) | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I find it difficult to understand statement like "depending on definition provided in OpenAPI". OpenAPI is too vague for me, it can mean standard defitinition, a particular service (interface) definition or maybe something else. I prefer to be more helpful here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. imho previous description is simpler (it doesn't use huge world "interface") and points out that behaviour is dependent on specific service configuration There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| security.*.apiKeyValue | false | | API key that will be passed to the service via header, query parameter or cookie, as indicated in the *OpenAPI interface definition* | | ||
| secret | false | | Configuration for [authentication](https://swagger.io/docs/specification/authentication/) which matches any `securitySchemas` defined in the *OpenAPI interface definition*. This config entry has the same structure as values in `security` object (see above) | | ||
|
||
## Operations | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -193,40 +193,42 @@ The `subscriberId` will be available in a `#totalTransfers.subscriberId` variabl | |
|
||
## Single-side-join | ||
|
||
Single-side-join component is conceptually similar to components computing aggregates in time windows, so it is convenient to discuss it here. Conceptually Single-side-join is an equivalent of the [left (or right) join](https://www.w3schools.com/sql/sql_join.asp) . In SQL case, the left join returns all records from the left table, and the matched records from the right table. In Nussknacker's case the Single-side-join will join two ‘branches’ of a scenario - the Main branch and the Joined branch and will **return exactly as many events as there were in the Main branch**. Even if no events will be matched in the Joined branch, an event will be emitted, with the value corresponding to the aggregator selected - null for List and Set, 0 for Sum, null for Min and Max. **The time window boundaries will be determined by the event coming from the main branch** and will be in the range of \[main-branch-event-event-time, main-branch-event-event-time + windowLength\]. | ||
|
||
Under the hood single-side-join uses sliding-window on the JOINED branch to deliver events to the MAIN branch. You can choose which aggregate function to use on the JOINED branch; yet `Last` is probably the most natural choice if you want the most recent value seen in the JOINED branch. | ||
|
||
Single-side-join can be an attractive and very fast alternative to database lookup's if you have enough memory to stream your whole lookup table and (if needed) you are able to stream changes of the look up table records. To use single-side-join as a very fast lookup, configure the topic containing the lookup table values as a **JOINED** branch. Make sure that you set window length to value high enough to ensure that there are always events which qualify to the window in the JOINED branch. | ||
Single-side-join component is conceptually similar to components computing aggregates in time windows, so it is convenient to discuss it here. Conceptually Single-side-join is an equivalent of the [left (or right) join](https://www.w3schools.com/sql/sql_join.asp) . In SQL case, the left join returns all records from the left table, and the matched records from the right table. | ||
|
||
 | ||
|
||
In Nussknacker's case the Single-side-join joins two ‘branches’ of a scenario - the MAIN branch and the JOINED branch. For every event coming from the MAIN branch, the single-side-join will attempt to enrich it with the result of the `aggregator` function, which will act on events from the JOINED branch. If no events are matched with the JOINED branch, an event will be emitted, with the aggregator result corresponding to the aggregator selected - null for List and Set, 0 for Sum, null for Min and Max. **The time window boundaries will be determined by the event coming from the MAIN branch** and will be in the range of \[main-branch-event-event-time - windowLength, main-branch-event-event-time\]. | ||
|
||
Because there are no tables and table names to refer to, Nussknacker will derive names of the branches to join from the names of nodes taking part in the Single-side-join. Let’s consider an example where there is a topic containing alerts about subscribers; for every alert generated for the subscriber we want to track all events generated by this subscriber in the 24 hours **preceding** the alert event. The Nussknacker scenario would look like in the picture below. | ||
Under the hood single-side-join uses sliding-window on the JOINED branch to deliver events to the MAIN branch. Consequently, the events coming from the MAIN branch are enriched with the aggregator result and events coming from the JOINED branch are terminated. | ||
|
||
 | ||
Single-side-join can be an attractive and very fast alternative to database lookup's if you have enough memory to stream your whole lookup table and (if needed) you are able to stream changes of the look up table records. To use single-side-join as a very fast lookup, configure the topic containing the lookup table values as a JOINED branch. Make sure that you set window length to value high enough to ensure that there are always events which qualify to the window in the JOINED branch. | ||
|
||
|
||
The configuration of the Single-side-join would be as in the picture below; note how Nussknacker Designer helps you to decide which branch is which. | ||
### Parameters | ||
|
||
* **branchType** - either `MAIN` or `JOINED` | ||
* **key** - value used to join branches; separate value for each branch. The expression in this field can refer to events and their associated variables (including `#input`) from their respective branches only. | ||
raphaelsolarski marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* **aggregateBy** - the input to the `aggregator` function. Only variables associated with the event from the JOINED branch can be used in the `aggregateBy` expression. | ||
* **Ouput variable name** - the name of the variable which will hold the results of the `aggregator` function | ||
* **aggregator** - aggregator function. | ||
|
||
 | ||
### Example | ||
|
||
There are couple fine points to make here: | ||
Let’s consider an example where there is a topic containing alerts about subscribers; for every alert generated for the subscriber we want to track all events generated by this subscriber in the 24 hours **preceding** the alert event. The Nussknacker scenario would look like in the picture below. | ||
|
||
* The time window (of 1 day in our case) will be **closed** upon arrival of the event with the given `#input.subscriber` value. | ||
* The `#input` variable used in the aggregateBy field holds the content of the event “arriving” from the joined branch. This variable will be available downstream. | ||
* The events which arrive from the joined branch are terminated. | ||
* The `#outputVar` will be available downstream of the outer-join aggregate. | ||
 | ||
|
||
The configuration of the Single-side-join would be as in the picture below; note how Nussknacker Designer helps you to decide which branch is which. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
 | ||
|
||
## Full-outer-join | ||
|
||
Full-outer-join is Nussknacker's version of SQLs full outer join. It works much like single-side-join, | ||
but it has aggregates for both branches and emits a new event for every event it receives. Every time | ||
a new event is received, it is matched with events with the same key, then the aggregate for the appropriate | ||
branch is updated, and values of aggregates for both branches are returned. If an event cannot be matched, | ||
then a new event is still emitted, but some aggregates have a value of zero. | ||
then a new event is still emitted, but the the aggregate for the branch with no matched events has value of null or zero, depending on the selected aggregator function. | ||
|
||
 | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mby just "typing" instead of
"Typing information" system
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thia, "system" is too much probably, I will remove it. However the document uses "Typing information" all over the place, so it is probably better to keep it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
type system