Allowing data batching when poll frequency < sensor frequency

[tobie]

Even when doing realtime processing (eg. say at 60 Hz), there is a benefit to having more than one new value every time you process (eg. Oculus polls a gyroscope at 1000 Hz for head tracking). In this case, every time you process, you'll have [1000/60] ~= 16 new sensor data.

This seems like a desirable feature. How would you handle this though?

Is this (yet) an extra option you set? How does that work with sensors with multiple values, etc.?

/cc @borismus

Proposed resolutions: Some use cases have high data frequency requirements which might cause performance and/or memory constraints. Further research is needed to assess and determine how to satisfy those requirements.

Further actions:

• List use cases with high data frequency requirements.
• Research the performance costs of crossing the JS context to justify batching requirements (talk with implementors).
• Decide on a solution to expose the batching in the Sensor API.
• Devise a strategy to handle huge backlog of historical data.
• Decide whether performance and/or memory constraints justify making the batch mode opt-in or whether it can just be the normal operational mode.

[borismus]

I described a possible approach at the end of my blog post:

// Handle sensor events.
function onMagnetometer(event) {
  var data = event.data[0];
  // Get the timestamp (in millis).
  var t = data.timestamp;
  // Get the data (in this case µT, as per spec).
  var x = data.values[0];
  var y = data.values[1];
  var z = data.values[2];
  // Process the data.
  superAdvancedSensorFusionThing.addData(t, x, y, z);
}

[tobie]

OK, so have event.data always be a reverse-chronological array of sensor values (all values since the previous event, I suppose?). Isn't that unnecessarily wasteful in case only the last value is used/needed? Also, do you only expose the last value on the sensor object itself? Or do you also expose an array of values there? Or nothing at all?

[rwaldron]

I don't understand how the example illustrates an approach that matches the title of this issue.

As a side note, I'm curious to know what magnetometer device (or any sensor, really) actually supports data register history. Generally, the IC will write to the data register at whatever frequency it's been programmed to do so, but if the data isn't read before the next register write, it will be overwritten by that write and no longer accessible.

[borismus]

@tobie Yes in this snippet event.data is reverse-chronological array of sensor values since the previous event. Are you saying it's wasteful to have an array with one value in it?

Perhaps there are two modes here: batched (you get all data since the last callback), and sampled (you get only one data point).

@rwaldron Do you need IC support for this? A native thread can poll at sensor rate and cache the values, exposing batches of data to the JS context. My concern is that for very fast sensors (ie. sensor frequency >> 60 Hz), entering JS for every single value is going to really harm performance.

[tobie]

Yes in this snippet event.data is reverse-chronological array of sensor values since the previous event. Are you saying it's wasteful to have an array with one value in it?

No, I was saying that passing around an array with all the batched data for the cases where only the current value was needed seemed wasteful.

Perhaps there are two modes here: batched (you get all data since the last callback), and sampled (you get only one data point).

As a constructor option? (And maybe even .value and .values distinction with only the latter returning an array in such cases?) Could work if the use cases justify this added complexity.

[rwaldron]

Do you need IC support for this?

No, that's not what I meant—that's way too low level

cache the values

Got it, this is interesting and I agree with the motivation. A constructor options param could switch on a "batch" feature where the event object delivered to change events would contain a Set of readings (readings can be single or multiple value data) since the last change event.

[tobie]

There seems to be solid use cases and broad agreement to provide batches of sensor values when the frequency of onchange events is below the frequency at which the data is emitted by the sensor (for example, onchange events emitted at 60 Hz and a sensor emitting values at 1,000 Hz). This seems to be the good compromise between the need for fine grained data and perf requirements.

There are various means to expose these data batches but none that jump out as being the obvious, most elegant and efficient way to do so. There's also the possibility of making that an opt-in process for performance reasons.

Proposed resolutions: A batch mode must be specified. The use cases justifying said batch mode must be documented.

Further actions: understand the performance constraints better to see if this batch mode should be opt-in or always on.

[rwaldron]

(for example, onchange events emitted at 60 Hz and a sensor emitting values at 1,000 Hz)

So for this example, there would be at least ~17 values between each ondata(or whatever it's called) event. A 200 Hz accelerometer would have ~3 values between each ondata. Is it smart to expose a feature that will inevitably lead to code that does something like this:

let { Accelerometer } = sensors;
let accel = new Accelerometer({ frequency: 60, batch: true });
accel.on("data", function(event) {
  // For illustration only...
  // "event.batch" is an array of objects containing frozen "snapshots" of 
  // all the readings in the last 16.666666ms
  event.batch.length; // 17 for a 1000 Hz sensor
  event.batch.length; // 3 for a 200 Hz sensor
});

Ok, that's fine right? But what happens when it's an onchange event and the user sets the phone down on a table and has the screen sleep set to 30 minutes? 29 minutes later, they pick up the phone, which triggers the onchange which now delivers an array containing 104400000 (1000ms * 60s * 60m * 29m) frozen snapshot objects.

This will lock up your browser

var batch = [];
for (var i = 0; i < 104400000; i++) {
  batch[i] = {x: 1, y: 2, z: 3};
}
console.log(batch);

Another issue is that developers may write code that's somehow dependent on the number of records in a batch for the ondata event, eg. 17 for the 60 Hz reporting of the 1000 Hz sensor, or 3 for the 60 Hz reporting of the 200 Hz sensor.

Unless the batching period is only for the last period (16.66--ms in our example)?

What if there is no batching included, but design in a way that allows later introduction if there is demand. I suggest this because batching could be done in user code:

let { Accelerometer } = sensors;
let accel = new Accelerometer(); // Important!! This will default to the sensor's read resolution
let batch = [];
let start, end;
accel.on("data", function(event) {
  if (batch.length === 0) {
    start = event.timestamp;
    end = start + (1000 / 60);
  }
  if (event.timestamp < end) {
    // I can pick what information is useful to me...
    batch.push(this.acceleration);
  } else {
    doSomethingWithBatch(batch.slice());
    batch.length = 0;
  }
});

If this becomes common, we can extend the api to alleviate the pain.

[tobie]

My understanding from #9 (comment) is that a batch mode is necessary for performance reasons (due to the cost of re-entering the JS context every time the sensor emits a value). This implies that the batching is done in native code and then transferred to the JS context all at once. Now whether that batched data is then emitted all together in a single ondata event (e.g. sensor.emit("data", batch);) or separately in n different events (e.g. batch.forEach(value => sensor.emit("data", value));) can be debated, though there might also be performance considerations there.

[rwaldron]

This implies that the batching is done in native code and then transferred to the JS context all at once.

Yes, no dispute.

My concern still stands: what if the batch is 104400000 accelerometer snapshot objects?