traderx blog post tweaks

Author: refset

src/pages/blog/bitemporal-traderx.mdx

@@ -4,7 +4,7 @@ title: 'Bitemporal TraderX - augmenting a sample trading system'
 description: 'Adding XTDB and Clojure into a sample microservices trading application for the FINOS 2024 Tech Sprint'
 category: 'database'
 layout: '../../layouts/BlogPost.astro'
-publishedDate: '2024-11-19'
+publishedDate: '2024-11-20'
 heroImage: 'bitemporal-traderx.jpg'
 tags:
   - 'SQL'
@@ -30,9 +30,9 @@ Here's a short video that the team put together which gives an overview of the s
 In summary, we:
 
 - Lightly modified the TraderX microservice architecture to implement blotter history and reporting capabilities based on a central [XTDB](https://xtdb.com) database that stores bitemporal versions of all data, allowing for strong auditability and correction/adjustment facilities
-- Extended the blotter dashboards to show pricing data, and add a 'sliders' to control the as-of reporting windows, giving users an ability to trivially understand the evolving state of their portfolio
-- Used XTDB's bitemporal SQL features to easily model the TraderX domain with limited code changes
-- Opened a [PR](TODO) to propose these changes back to the main TraderX repository
+- Extended the blotter dashboards to show pricing data, and added 'sliders' to control the as-of reporting windows, giving users an ability to trivially understand the evolving state of their portfolio of positions
+- Used XTDB's bitemporal SQL features to easily model the TraderX domain with limited code changes. The underlying data model now allows for all reference and pricing data to be auditably adjusted, and for all trades to be amended or cancelled retroactively, while providing complete visibility to the end user
+- Opened a [PR](https://github.com/finos/traderX/pull/237) to propose these changes back to the main TraderX repository
 
 This work was showcased at the FINOS 'OSFF' conference in New York last month, where my colleague Jeremy (XTDB Head of Product) also presented on the importance of bitemporality more broadly - you can read/watch more about this [here](/blog/reconciliation-risk-ml-and-use-cases-for-bitemporal-systems/).
 
@@ -68,7 +68,7 @@ This tab also adds two sliders:
 
 ### A re-imagined 'Reference Service'
 
-In supporting of all these UI additions we created a new microservice to replaced the existing 'reference-service'. It is written in Clojure and is split across several clearly delineated namespaces. This service now does a few things:
+In supporting of all these UI additions we created a new microservice to replace the existing 'reference-service'. It is written in Clojure and is split across several clearly delineated namespaces. This service now does a few things:
 - Fully replaces and supersedes the existing 'reference-data' microservice for serving up ‘static’ / lookup data (securities and accounts)
 - Provides bitemporal storage for trades, positions and prices
 - Generates a year’s worth of daily prices (initial value randomly generated, thereafter - last price varied by 10% so that we have an approximation of how prices change over time on the market - prices rise and fall)
@@ -82,7 +82,7 @@ The service interfaces with XTDB via vanilla [pgJDBC](https://jdbc.postgresql.or
 ## XTDB experience report - first impressions
 
 This was my first time using XTDB so I had some learning to do myself. It takes a little time to get used to the temporal syntax for making ‘updates’ with valid_time (but then - you only have to learn it once).
-My first reaction was to add dedicated columns for handling various applicaion and auditing timestamps like `traded_at` and `updated_at` - but what a pleasant surprise it was to find out there’s little need for many of these use-cases - timestamping is already ubiquitous and all done under the covers.
+My first reaction was to add dedicated columns for handling various application and auditing timestamps like `traded_at` and `updated_at` - but what a pleasant surprise it was to find out there’s little need for many of these use-cases - timestamping is already ubiquitous and all done under the covers.
 
 *For all intents and purposes how you interact with the database does not change, compared to traditional SQL databases* - until you need to get the historical data or introduce retrospective updates (where you now know that some data was incorrect in the past and you can update that fact specifying the correct value and period during which it was the case). This still leaves you with a perfectly auditable history - you can see that the retrospective change for a particular record was created at `system time`.
 
@@ -102,22 +102,23 @@ Getting data from a couple of joined tables (when you want to see it ‘as of’
 
 It also took me a couple iterations to get correct `time points` for the ‘event’ slider (which shows ‘history’ of trades - and therefore positions).
 
-My initial approach was to get a collection of time periods with period start being valid_from and period end being valid_to - for all trades in ascending order. This however has yielded unwanted results and seemed quite confusing.
-At that point I changed the ‘period dropdown’ to a slider where the points are all valid_from values for all trades (and there’s no restriction on valid_to). This resulted in a clear history of trades and their lifecycle - from their creation in pending state to settlement.
+My initial thought was to naively query for a collection of time periods, with the period start being `_valid_from` and period end being `_valid_to`, for all trades in ascending order. This however yielded unwanted results and seemed quite confusing (because `_valid_from` and `_valid_to` would be interleaved and not correlated between trades, as some trades ‘disappeared’ while others concurrently ‘changed state’ from settled to pending).
 
-In addition to the main blotter slider I also added a price slider. This was much easier, just a query for price ‘as of’ some given day, using regular daily steps.
+At that point I changed the ‘period dropdown’ to a slider where the points are all `_valid_from` values for all trades (and there’s no restriction on `_valid_to`). This resulted in a clear history of trades and their lifecycle - from their creation in pending state to settlement.
+
+In addition to the main blotter slider I also added a price slider. This was much easier, essentially a basic view of price ‘as of’ some given day, using regular daily steps. The blotter query then effortlessly joins across these two underlying table sources using their respective as-of timestamps.
 
 ## My XTDB Takeaways
 
 I have worked on several applications that incorporate reporting requirements previously, and I can safely say XTDB brings a lot of benefits over the traditional SQL database paradigm.
 
 **Having added bitemporal storage unlocks incredible reporting capabilities** - not only over past changes, but also opening the door to speculative analysis (e.g. "what would my account PnL look like with prices from a given day in the past" / "how much could I have gained/lost if I had bought stock X on date Y").
 
-**A first-class history close to hand is a game changer in terms of application development** - a whole range of time related complexities are taken care of by the bitemporal model and XTDB makes working with bitemporal data very easy. You could compare it to what happened when you could relegate transaction management from application logic into the database layer. There is a little bit to learn about in order to using [SQL:2011](https://en.wikipedia.org/wiki/SQL:2011)’s temporal operators - but this is really nothing compared to benefits you get from just using it.
+**A first-class history close to hand is a game changer in terms of application development** - a whole range of time related complexities are taken care of by the bitemporal model and XTDB makes working with bitemporal data very easy. You could compare it to what happened when you could relegate transaction management from application logic into the database layer. There is a little bit to learn about in order to start using [SQL:2011](https://en.wikipedia.org/wiki/SQL:2011)’s temporal operators - but this is really nothing compared to benefits you get from just using it.
 
 **XTDB provides bitemporal storage with virtually no barrier to entry** - DML (SQL statements for storing / updating entities) simply does not change existing data, while the retrieval of historical data is trivial using SQL2011 queries (leveraging ‘FOR VALID TIME’ to get a point/period in the past). The only caveat in this project was that XTDB doesn't yet fully work with JPA style persistence due to minor differences with real Postgres, but working with plain SQL or simpler abstractions (e.g. [jOOQ](/blog/15-years-of-jooq-with-lukas-eder/)) is my preference anyway.
 
-**There is no more need for audit tables** or manual addition of more timestamped fields (such as ‘updated_at’) - history is transparently preserved along with the two timelines - system time and application time.
+**No more need for audit tables** or manual addition of more timestamped fields (such as ‘updated_at’) - history is transparently preserved along with the two timelines - system time and valid time.
 
 **XTDB drastically reduces complexity** otherwise introduced by hand-rolled reporting solutions - these usually come in later as post production release requirements (dictated by reporting/auditing/legal needs).
 You’re free to focus on adding competitive value to your application - unique features that will set you out from your competition - without expending effort on things which, whilst often absolutely required, do not necessarily give end users visible benefit.
@@ -130,4 +131,4 @@ JUXT is currently collaborating with several Design Partners using XTDB who help
 
 ## XTDB webinar on 11th December
 
-Finally, the XTDB team is running a webinar in conjunction with FINOS on 11th December, you can learn more and register for that [here](TODO).
+Finally, the XTDB team is running a webinar in conjunction with FINOS on 11th December, you can learn more and register for that [here](https://zoom.us/webinar/register/4617283986911/WN_3b4DvHhvQbCtt98DOvjbDQ).