How to get graph runtime in csp.graph?

[jasonbourne13]

Thank you for open-sourcing this excellent package. I want to ask what's the best practice for getting graph runtime in csp.graph? csp.engine_start_time() can directly return a datetime object. I want to access runtime timestamp as well. csp.times gives a csp.impl type but not datetime object. I can certainly add additional complexity to use csp.now in csp.node, but that won't be as convenient. Appreciate any help!

[AdamGlustein]

If I understand your question correctly, you would like to access the current time that the graph is executing at each cycle. To do this, you would use csp.times as you mentioned. csp.times returns a time-series of datetime values, so you can pass that downstream to any other nodes (or just print it, return it as output, etc.).

Here's a basic example:

import csp
from datetime import datetime, timedelta

@csp.graph
def print_runtimes():
    x = csp.timer(timedelta(seconds=1), value=42.0)
    csp.print('val', csp.times(x))

csp.run(print_runtimes, starttime=datetime.utcnow(), endtime=timedelta(seconds=5), realtime=True)

Gives:

>> python ex_csp.py
2025-03-17 16:14:32.378832 val:2025-03-17 16:14:32.378832
2025-03-17 16:14:33.378832 val:2025-03-17 16:14:33.378832
2025-03-17 16:14:34.378832 val:2025-03-17 16:14:34.378832
2025-03-17 16:14:35.378832 val:2025-03-17 16:14:35.378832

[jasonbourne13]

Thanks for the help Adam! I thought it should work, too. But when I try to wrap a pd.Timestamp around the output of csp.times and got the following error. I think this is because it returns a csp.ts object so that we cannot access it with normal timestamp operations.

      1 @csp.graph
      2 def test_get_graph_runtime(resample_seconds: int = 60) -> csp.ts[pd.Timestamp]:
      3     cur_time = get_graph_runtime(resample_seconds)
----> 5     return pd.Timestamp(cur_time)

File timestamps.pyx:1865, in pandas._libs.tslibs.timestamps.Timestamp.__new__()

File conversion.pyx:425, in pandas._libs.tslibs.conversion.convert_to_tsobject()

TypeError: Cannot convert input [Edge( tstype=csp.impl.types.tstype.TsType[datetime.datetime], nodedef=<csp.baselib.times object at 0x7ff16b5b9950>, output_idx=0, basket_idx=-1 )] of type <class 'csp.impl.wiring.edge.Edge'> to Timestamp

Here's the code to replicate this error:

@csp.graph
def get_graph_runtime(resample_seconds: int = 60) -> csp.ts[datetime]:
    resample_interval = timedelta(seconds=resample_seconds)
    timer = csp.timer(interval=resample_interval, value=True)
    
    return csp.times(timer)

@csp.graph
def test_get_graph_runtime(resample_seconds: int = 60) -> csp.ts[pd.Timestamp]:
    cur_time = get_graph_runtime(resample_seconds)
    
    return pd.Timestamp(cur_time)

csp.run(test_get_graph_runtime,
        starttime=pd.Timestamp('2023-02-13 20:55:00.000'),
        endtime=pd.Timestamp('2023-02-13 21:10:00.000'),
        output_numpy=True)

[AdamGlustein]

Ok, I see what your issue is. In csp (and a lot of other computation graph libraries) there is a key distinction between wiring time (graph-time) and runtime (node-time).

Any code under csp.graph is graph-time code: this means that there are no values available yet, all that you are doing is describing your computation. This amounts to wiring the graph together, so when you call csp.times(...) in the graph it returns to you an Edge object that you can connect to other nodes. This is an Edge of datetime values, not a datetime itself. The actual datetimes will only be available at runtime.

Any code under csp.node is node-time code: the values accessed as inputs to a node and returned from a node are those that propagate throughout the computation. If you want to convert the datetimes at runtime to pd.Timestamp, you'd need to do so in a node. For simple one-liners like this we have a csp.apply node so you can minimize boilerplate: https://github.com/Point72/csp/wiki/Base-Nodes-API#cspapply.

Thus, your corrected code would look something like:

import csp
from datetime import datetime, timedelta
import pandas as pd

@csp.graph
def get_graph_runtime(resample_seconds: int = 60) -> csp.ts[datetime]:
    resample_interval = timedelta(seconds=resample_seconds)
    timer = csp.timer(interval=resample_interval, value=True)
    
    return csp.times(timer)

@csp.graph
def test_get_graph_runtime(resample_seconds: int = 60) -> csp.ts[pd.Timestamp]:
    cur_time = get_graph_runtime(resample_seconds)
    as_pd = csp.apply(cur_time, lambda x: pd.Timestamp(x), pd.Timestamp) # This is the only change!!!
    return as_pd

res = csp.run(test_get_graph_runtime,
        starttime=pd.Timestamp('2023-02-13 20:55:00.000'),
        endtime=pd.Timestamp('2023-02-13 21:10:00.000'),
        output_numpy=True)
print(res)

We have a description of the differences between nodes and graphs here (https://github.com/Point72/csp/wiki/Common-Mistakes#graphs-vs-nodes) which I hope you find useful. I'd also recommend watching some of the CSP talks we have on YouTube too where we address this distinction.

[jasonbourne13]

Thanks for the explanation Adam. You are amazing!