Skip to content
This repository was archived by the owner on Aug 14, 2024. It is now read-only.

Commit 58f8ded

Browse files
philipphofmannphilipphofmann
authored
Add frames delay (#1153)
Add docs for the performance metric frames delay.
1 parent 930640a commit 58f8ded

File tree

3 files changed

+163
-0
lines changed

3 files changed

+163
-0
lines changed

‎src/components/sidebar.tsx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,9 @@ export default () => {
246246
<SidebarLink to="/sdk/performance/time-to-initial-full-display/">
247247
Time to Initial/Full Display
248248
</SidebarLink>
249+
<SidebarLink to="/sdk/performance/frames-delay/">
250+
Frames Delay
251+
</SidebarLink>
249252
<SidebarLink to="/sdk/performance/dynamic-sampling-context/">
250253
Dynamic Sampling Context
251254
</SidebarLink>

‎src/docs/sdk/performance/frames-delay.mdx‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
---
2+
title: "Frames Delay"
3+
---
4+
5+
Frames Delay is the user-perceived delayed duration of rendered frames. SDKs can attach this
6+
information to spans. We recommend implementing this feature for mobile and desktop SDKs.
7+
8+
## Background
9+
10+
This section explains why frames delay is a better metric than slow and frozen frames. If you
11+
already know why, you can skip this section.
12+
13+
On Mobile, slow and frozen frames are well-established metrics for tracking the responsiveness of
14+
applications. A phone or tablet typically renders 60 frames per second (fps). At 60 fps, every frame
15+
has 16 or 16.67 ms to render:
16+
17+
* __Slow Frames__: Using 60 fps, slow frames are frames that take more than 16.67 ms to render.
18+
* __Frozen Frames__: Frozen frames are frames that take longer than 700 ms to render.
19+
20+
The major downside is that they lack essential information on how long they lasted, which is a
21+
significant problem when prioritizing which unresponsive application parts need improvement. The following
22+
scenarios highlight the problem when prioritizing by looking at slow and frozen frames.
23+
24+
Scenario | Slow Frames | Frozen Frames |
25+
-- | -- | -- | --
26+
1 | 5 all 500ms | 0 |
27+
2 | 10 all 20ms | 0 |
28+
3 | 0 | 1 with 800ms |
29+
30+
When comparing the different scenarios, a user might want to fix scenario 3 first because it contains one frozen
31+
frame. When comparing scenarios 1 and 2, they would pick scenario 2 cause it has more slow frames. While scenario
32+
2 looks worse, the user experience is way worse for scenario 1 because the slow frames last longer.
33+
34+
That's where a new metric called __frames delay__ comes to the rescue. Frames delay represents the delayed frame
35+
render duration of all frames. The frame delay for one frame is `actual frame duration - expected frame duration`.
36+
For example, with 60 fps the expected frame duration is 16.67 ms. When a frame takes 20 ms to render, the frame
37+
delay is `20ms - 16.67ms = 3.33ms`.
38+
39+
Let's calculate the frames delay for scenarios 1 - 3.
40+
41+
Scenario | Calculation | Frames Delay
42+
-- | -- | --
43+
1 | (500ms - 16.67ms) * 5 | 2416.65ms
44+
2 | (20ms - 16.67 ms) * 10 | 33.3ms
45+
3 | 800ms - 16.67ms | 783.33ms
46+
47+
Now, when prioritizing the user-perceived degraded responsiveness by frames delay, we get 1, 3, 2 instead of
48+
3, 2, 1 when looking at slow and frozen frames.
49+
50+
## Specification
51+
52+
Frames delay represents the delayed frame render duration of all frames. The frame delay for one frame is
53+
`actual frame duration - expected frame duration`. For example, with 60 fps the expected frame duration is 16.67 ms.
54+
When a frame takes 20 ms to render, the frame delay is `20ms - 16.67ms = 3.33ms`. As the expected frame rate can
55+
change at any time, SDKs must check the expected frame rate for every frame. The frame delay for a span is the sum
56+
of frame delays for all slow and frozen frames during the start and end time of the span.
57+
58+
SDKs should attach frames delay in seconds to spans via <Link to="/sdk/performance/span-data-conventions/#mobile">span data</Link>
59+
and __attach frames delay to all spans__, regardless of on which thread a span runs, and as well to concurrently running spans.
60+
For transactions SDKs should send frames delay via the root span with span data.
61+
While it makes sense to calculate frames delay in milliseconds, __SDKs must send frames delay in seconds__, as SDKs already
62+
use seconds for most timestamps. For the specification here, we use milliseconds cause they are easier to read.
63+
64+
The specification is written in the [Gherkin syntax](https://cucumber.io/docs/gherkin/reference/).
65+
66+
67+
```Gherkin
68+
Scenario: Slow and frozen frames
69+
Given a frame rate of 60 fps
70+
And 2 frames with 300 ms
71+
And 1 frame with 900 ms
72+
And 10 frames with 16.76 ms
73+
When calculating the frames delay
74+
Then the frames delay is (300 - 16.67) * 2 + 900 - 16.67 = 1449.99 ms
75+
76+
Scenario: Frame rate changes
77+
Given 1 frame with 200 ms with 60 fps
78+
And 2 frame with 500 ms with 120 fps
79+
And 10 frames with 8.33 ms with 120 fps
80+
When calculating the frames delay
81+
Then the frames delay is 200 - 16.67 + (500 - 8.33) * 2 = 1166.67 ms
82+
83+
Scenario: Ongoing delayed frame
84+
"""
85+
[nf][ ------- ?? ------ ] | nf = normal frame, ?? = No frame information
86+
[---- span duration ----]
87+
When there is an ongoing delayed frame, the logic doesn't know how long it
88+
will last and, therefore, has no frame information.
89+
"""
90+
Given a frame rate of 60 fps
91+
And 1 frame with 16.67 ms
92+
And 200 ms pass after without any frame being drawn
93+
And no recorded delayed frame
94+
When calculating the frames delay
95+
Then the frames delay 200 ms - 16.67 ms = 183.33 ms
96+
97+
98+
Scenario: Delayed frame starts before span
99+
"""
100+
[---- delayed frame ---- ]
101+
[- span duration -]
102+
"""
103+
Given a frame rate of 60 fps
104+
And a recorded delayed frame with a duration of 300 ms
105+
When calculating the frames delay for the span 100 ms to 300 ms
106+
Then the frames delay is the full duration of the span of 200ms
107+
108+
109+
Scenario: Delayed frame starts shortly before span
110+
"""
111+
[| e | delayed frame ---- ] e = the expected frame duration
112+
[--- span duration --- ]
113+
"""
114+
Given a frame rate of 60 fps
115+
And a recorded delayed frame with a duration of 300 ms
116+
And the delayed frame started at 0 ms shortly before the span
117+
When calculating the frames delay for the span 10 ms to 300 ms
118+
Then the frames delay is 300 ms - 16.67 ms = 283.33 ms, which is only the delayed part
119+
of the delayed frame
120+
And the expected frame duration part of the delayed frame is not added as a frame delay
121+
122+
Scenario: Delayed frame starts and ends before the span
123+
"""
124+
[| e | delayed frame ------ ] e = the expected frame duration
125+
[--- span duration --- ]
126+
"""
127+
Given a frame rate of 60 fps
128+
And a recorded delayed frame with a duration of 310 ms
129+
And the delayed frame started at 0 ms shortly before the span
130+
When calculating the frames delay for the span duration of 10 ms to 300 ms
131+
Then the frames delay is 300 ms - 6.67 ms = 293.33 ms, which is only the delayed part of
132+
the delayed frame having an intersection with the span duration
133+
And the expected frame duration part of the delayed frame is not added as a frame delay
134+
135+
136+
Scenario: One delayed frame starts shortly before the end of a span
137+
"""
138+
[| e | delayed frame ][| e | delayed frame - ] e = the expected frame duration
139+
[---- span duration ---- ]
140+
"""
141+
Given a frame rate of 60 fps
142+
And a recorded delayed frame with a duration of 290 ms
143+
And a recorded delayed frame with a duration of 200 ms
144+
When calculating the frames delay for the span duration of 0 ms to 300 ms
145+
Then the frames delay is 290 ms - 16.67 ms = 273.33 ms
146+
And the expected frame duration part of the second delayed frame is not added as a frame delay
147+
148+
Scenario: Two concurrent spans
149+
"""
150+
[| e | --- delayed frame--- ] e = the expected frame duration
151+
[----- span 1 -----]
152+
[---- span 2 ----]
153+
"""
154+
Given a frame rate of 60 fps
155+
And a recorded delayed frame with a duration of 300 ms
156+
When calculating the frames delay
157+
Then the frames delay for span 1 is it's full duration of 180 ms
158+
Then the frames delay for span 2 is it's full duration of 160 ms
159+
```

‎src/docs/sdk/performance/span-data-conventions.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ Below describes the conventions for the Span interface for the `data` field on t
4242
| `frames.total` | int | The number of total frames rendered during the lifetime of the span. | `60` |
4343
| `frames.slow` | int | The number of slow frames rendered during the lifetime of the span. | `2` |
4444
| `frames.frozen` | int | The number of frozen frames rendered during the lifetime of the span. | `1` |
45+
| `frames.delay` | number | The sum of all delayed frame durations in seconds during the lifetime of the span. For more information see <Link to="/sdk/performance/frames-delay/">frames delay</Link>. | `1.3246` |
4546

4647
## Browser
4748

0 commit comments

Comments
 (0)