resourcetiming-compression
Advanced tools
v1.4.0
Licensed under the MIT license
resourcetiming-compression.js compresses data from ResourceTiming. A
companion script, resourcetiming-decompression.js, converts the compressed data back to the original form.
ResourceTiming is a W3C WebPerf API that exposes all of the page's resources' network timing information (and additional details) to JavaScript, and is available in most modern browsers.
The interface performance.getEntriesByType('resource') returns a list of resources with information about each resource's URL, inititor, timestamps, size information and more.
Collecting this information is easy, but beaconing all of the data back to a data warehouse can be a challenge because of the amount of data available for each resource.
On a typical page, which might have over 100 resources, the browser could easily generate 50 KB+ of ResourceTiming data per page-load.
resourcetiming-compression.js applies several data-compression techniques to reduce the size of your serialized
ResourceTiming data to about 15% of it's original size in many cases. See
this nicj.net blog post for a description of these techniques.
resourcetiming-decompression.js is a companion script that will take the compressed ResourceTiming data and
rebuilds the ResourceTiming structures (eg. performance.getEntriesByType('resource')) for analysis.
NOTE: resourcetiming-compression.js is the same code that drives the restiming.js plugin for
Boomerang, but also includes the resourcetiming-decompression.js component.
Releases are available for download from GitHub.
Development: resourcetiming-compression.js - 57kb
Production: resourcetiming-compression.min.js - 4.7kb (minified / gzipped)
Development: resourcetiming-decompression.js - 40kb
Production: resourcetiming-decompression.min.js - 3.3kb (minified / gzipped)
resourcetiming-compression.js is also available as the npm resourcetiming-compression module. You can install using Node Package Manager (npm):
npm install resourcetiming-compression
resourcetiming-compression.js is also available via bower. You can install using:
bower install resourcetiming-compression
Please see the W3C ResourceTiming API Reference for details on how to use the ResourceTiming API.
To include resourcetiming-compression.js, simply include it via a script tag:
<script type="text/javascript" src="resourcetiming-compression.min.js"></script>
Once included in the page, a top-level ResourceTimingCompression object is available on window. If AMD
or CommonJS environments are detected, it will simply expose itself via those methods.
From the NPM module:
var ResourceTimingCompression = require("resourcetiming-compression").ResourceTimingCompression;
To get all of the compressed resources, you can simply call:
var {restiming, servertiming} = ResourceTimingCompression.getResourceTiming();
To include resourcetiming-decompression.js, simply include it via a script tag:
<script type="text/javascript" src="resourcetiming-decompression.min.js"></script>
Once included in the page, a top-level ResourceTimingDecompression object is available on window. If AMD
or CommonJS environments are detected, it will simply expose itself via those methods.
From the NPM module:
var ResourceTimingDecompression = require("resourcetiming-compression").ResourceTimingDecompression;
To decompress your resources, you can simply call:
var original = ResourceTimingDecompression.decompressResources(restiming, servertiming);
Please see this blog post for a more detailed description of the compression format.
Each ResourceTiming entry (e.g. URL) is first combined into an optimized Trie:
{
"http://": {
"foo.com/": {
"|": "0,a",
"js/foo.js": "370,1z,1c|390,1,2",
"css/foo.css": "48c,5k,14*0a"
}
}
}
The above Trie contains data for the following URLs:
http://foo.com/ (the | leaf node)http://foo.com/js/foo.js (contains two entries as designated by | in the middle of the value)http://foo.com/css/foo.cssSome notes:
| means an entry for the Trie up to that point.
This may happen if a URL is a common prefix of other entries.
The URL would not contain the |.| means the URL has multiple entries (e.g. the same URL has multiple hits)*[n] means "special data" is encoded for that URL, such as dimensions, sizes, Server Timing, etc.
See below for more details.The very first character for every ResourceTiming entry is its initiatorType.
For example:
{
"http://foo.com/js/foo.js": "370,1z,1c"
}
3 is the initiator of foo.js, which maps to script.
The mapping is defined as:
INITIATOR_TYPES = {
"other": 0,
"img": 1,
"link": 2,
"script": 3,
"css": 4,
"xmlhttprequest": 5,
"html": 6,
"image": 7,
"beacon": 8,
"fetch": 9,
"iframe": "a",
"subdocument": "a",
"body": "b",
"input": "c",
"frame": "a",
"object": "d",
"video": "e",
"audio": "f",
"source": "g",
"track": "h",
"embed": "i",
"eventsource": "j",
"navigation": 6,
"early-hints": "k",
"ping": "l",
"font": "m"
}
After the initiatorType, each entry contains the resource's timestamps in the following order:
[initiatorType][startTime],[responseEnd],[responseStart],
[requestStart],[connectEnd],[secureConnectionStart],[connectStart],
[domainLookupEnd],[domainLookupStart],[redirectEnd],[redirectStart]
Each entry is Base36 encoded.
startTime is the Base36 value of the startTime timestamp.
All of the other timestamps are the Base36 value of their offset from startTime.
All trailing ,0s are removed and can be assumed to be the same as 0 or missing when decoding.
For example:
{
"http://foo.com/js/foo.js": "370,1z,1c"
}
Results in:
{
"name": "http://blah.com/js/foo.js",
"initiatorType": "script",
"startTime": 252,
"responseEnd": 323,
"responseStart": 300
}
With every other timestamp being 0.
Additional information about each resource is encoded with special "special data" delimiters (*[type]).
The special data types are:
0: Dimensions1: Sizes2: <script> attributes3: ServerTiming4: <link rel=> value5: Namespaced data6: Service Worker Start time7: nextHopProtocol data8: contentType data9: deliveryType dataa: renderBlockingStatus datab: responseStatus dataDetails about each special data follows.
If available, when compressing the resources via compressResourceTiming(),
any resource that has a visible component on the page (such as an IMG element) will have its height width top
and left values captured and included in the compressed data as well.
This information is encoded as a "special value" in the resource's timings array. They will be appended to the
list of timings with a special prefix of *0.
For each resource, multiple hits to the same URL are separated by a pipe (|) character:
{
// this resource was loaded twice with timings 70,1z,1c and 90,1,2
"http://blah.com/js/foo.js": "370,1z,1c|390,1,2"
}
If the resource has visible elements on the page, they will be appended to this list of timings with a special prefix
of *0 (Base36 encoded):
{
// this resource was loaded twice with timings 70,1z,1c and 90,1,2 and had
// dimensions of height = 1, width = 5, top = 10 and left = 11
"http://blah.com/js/foo.js": "370,1z,1c|390,1,2|*01,5,a,b"
}
If available via ResourceTiming2, when compressing the resources
via compressResourceTiming(), the resource's transferSize, encodedBodySize and decodedBodySize
will be captured and included in the compressed data as well.
This information is encoded as a "special value" in the resource's timings array. They will be appended to the
list of timings with a special prefix of *1.
The data will be stored in the order of: [e, t, d].
e:
e is the value of encodedBodySize (encodedBodySize = parseInt(e, 36))encodedBodySize was 0 (no body, just headers)t:
t is the transferSize increase in size over encodedBodySize
transferSize = parseInt(e, 36) + parseInt(t, 36))"_", transferSize is 0 (cached)transferSize was either 0 (cached) or undefined (unknown)d:
d is the decodedBodySize increase in size over encodedBodySize
decodedBodySize = parseInt(e, 36) + parseInt(d, 36))decodedBodySize was equal to encodedBodySize (the request was not encoded)For example:
{
// this resource was loaded with timings 70,1z,1c and had sizes set
"http://blah.com/js/foo.js": "370,1z,1c*1a,b,c"
}
Results in:
encodedBodySize = parseInt("a", 36) = 10transferSize = parseInt("a", 36) + parseInt("b", 36) = 21decodedBodySize = parseInt("a", 36) + parseInt("c", 36) = 22<script> elements have attributes such as async and defer. These attributes, plus its location (<head>
or <body>) will be included for any <script> elements.
This information is encoded as a "special value" in the resource's timings array. They will be appended to the
list of timings with a special prefix of *2.
The data is encoded as a bitmask:
async attribute has a value of 0x1defer attribute has a value of 0x2<script> was in the body, the location will be set to 0x4. Otherwise, the script was in the <head>For example:
{
// this resource was loaded with timings 70,1z,1c and had script attributes of async/defer/body set
"http://blah.com/js/foo.js": "370,1z,1c*27"
}
Server Timing entries are included on Resource- and NavigationTiming entries as serverTiming.
They must have a name, might have a non-empty description, and will likely have a non-zero duration.
This compression is built on the presumption that resources will have Server Timing entries with unique durations
pointing mostly to the same name and descriptions.
This information is encoded as a "special value" in the resource's timings array. They will be appended to the
list of timings with a special prefix of *3.
There are two parts to this compression:
name and description pairs (an array of arrays, sorted with most-common first)duration of the
Server Timing entry and the key maps to the name and description in (1)For example:
performance.getEntriesByName(<path/to/resource1>)[0].serverTiming === [{
name: 'm1',
duration: 1,
description: 'desc1'
}, {
name: 'm2',
duration: 2,
description: 'desc3'
}]
performance.getEntriesByName(<path/to/resource2>)[0].serverTiming === [{
name: 'm1',
duration: 3,
description: 'desc1'
}, {
name: 'm1',
duration: 4,
description: 'desc2'
}]
getResourceTiming() will return a servertiming "lookup" will all of the unique pairs of name and description, equal to:
[[m1, desc1, desc2], [m2, desc3]]
We supplement the compressed resource timing data with comma-separated list of the form: <duration>:<entryIndex>.<descriptionIndex>.
1:0.0,2:1.03:0.0,4:0.1To save bytes, we will omit the zeroes, and irrelevant separators. So, from our example:
1,2:13,4:.1And lastly, were there only one description for a given name and it was empty-string, then we simplify that array entry:
[["description1", ""], ...] would become ["description1", ...]
For example:
{
// this resource was loaded with timings 70,1z,1c and had ServerTimings
"http://blah.com/js/foo.js": "370,1z,1c*3100,:1"
}
With servertiming data of:
["edge", ["cdn-cache", "HIT", "MISS"], "origin"]
Results in two ServerTiming entries:
[
{
name: "edge",
description: "",
duration: 100
},
{
name: "cdn-cache",
description: "HIT",
duration: 0
}
]
<link> elements have attributes such as rel. Known rel types will be included for any <link> elements.
This information is encoded as a "special value" in the resource's timings array. They will be appended to the
list of timings with a special prefix of *4.
REL_TYPES = {
"prefetch": 1,
"preload": 2,
"prerender": 3,
"stylesheet": 4
};
For example:
{
// this resource was loaded with timings 70,1z,1c and had a known link rel
"http://blah.com/js/foo.js": "270,1z,1c*41"
}
Results in:
foo.js was a <link> node (initiatorType = 2) and was a prefetchNamespaced data can be used to extend ResourceTiming compression with any other data that is desired.
If the ResourceTiming entry itself has a _data attribute, the key/value pairs will be included in the compressed
ResourceTiming data.
This information is encoded as a "special value" in the resource's timings array. They will be appended to the
list of timings with a special prefix of *5.
For example:
{
// this resource was loaded with timings 70,1z,1c and had a known link rel
"http://blah.com/js/foo.js": "270,1z,1c*5abc:123,def:z"
}
Results in:
{
"name": "http://blah.com/js/foo.js",
"initiatorType": "script",
"startTime": 252,
"responseEnd": 323,
"responseStart": 300,
"_data": {
"abc": "123",
"def": "z"
}
}
ResourceTiming's workerStart
timestamp is when the Service Worker was started, if active. The difference between workerStart and fetchStart
is how long it took the Service Worker to startup.
workerStart was not included in the original ResourceTiming Compression timestamp array, so it is included after
the regular timestamps as a "special value" if available. The data will be appended to the list of timings with
a special prefix of *6:
*6[Base36(workerStart-startTime)],[Base36(fetchStart-startTime)]
Note that both workerStart and fetchStart are included in this data, as fetchStart is not normally included
in the ResourceTiming Compression timestamp array: it can be inferred to be startTime or redirectEnd if there
was no Service Worker active.
For example:
{
// this resource has Service Worker workerStart timing
"abc": "31,b*62,3"
}
Results in:
initiatorType = 3 = scriptstartTime = parseInt("1", 36) = 1responseEnd = parseInt("b", 36) + startTime = 12workerStart = parseInt("2", 36) + startTime = 3fetchStart = parseInt("3", 36) + startTime = 4nextHopProtocol dataResourceTiming's nextHopProtocol
is the final connection's ALPN negotiated protocol, such as http/1.1, h2 or h3.
Note: http/ is replaced with h so http/1 is consistent with H2 and H3.
The data will be appended to the list of timings with
a special prefix of *7:
*7[h1|h1.1|h2|h3|...]
Initial versions of nexHopProtocol data were encoded inline as the string (per above). If the value is 2+ characters, it can be assumed the data is the nextHopProtocol string itself (possibly with http/ replaced as just h).
Recent versions no longer include the string, and instead use a lookup map to minimize bytes. If the value is a single character, it maps to one of the values below (or new values, if seen in the wild):
"h2": 0,
"h0.9": 1,
"h1.0": 2,
"h1.1": 3,
"h2c": 4,
"h3": 5
*7[0|1|2...] // possible values
*7 // h2
*73 // h3
*76 // unknown value, value included
// in another array on the beacon
Note if no value is provided, it is assumed to be index 0 (h2).
ResourceTimingDecompression can decode both versions.
contentTypeThe HTTP Content-Type is included for some assets as .contentType.
This information is encoded as a "special value" in the resource's timings array. It will be appended to the
list of timings with a special prefix of *8.
The value is a map to one of the pre-defined Content-Types below. If the Content-Type for a resource doesn't match one of those in the pre-defined list below,
it will generate a new index (and that list would be included on the beacon for lookup).
The pre-defined Content-Types are:
"application/json": 0,
"application/xml": 1,
"font/woff": 2,
"font/woff2": 3,
"image/avif": 4,
"image/gif": 5,
"image/jpeg": 6,
"image/png": 7,
"image/svg+xml": 8,
"image/webp": 9,
"image/x-icon": a,
"text/css": b,
"text/html": c,
"text/javascript": d,
"text/plain": e,
Note if no value is provided, it is assumed to be index 0 (application/json).
Example:
*8[0|1|2...e] // possible values
*8 // application/json
*8d // text/javascript
*8f // unknown value, value included
// in another array on the beacon
*8g // second unknown value, etc
deliveryTypeThe Delivery Type is included for some assets as .deliveryType.
This information is encoded as a "special value" in the resource's timings array. It will be appended to the
list of timings with a special prefix of *9.
The value is a map to one of the pre-defined Delivery Types below. If the Delivery Type for a resource doesn't match one of those in the pre-defined list below, it will generate a new index (and that list would be included on the beacon for lookup).
The pre-defined Delivery Types are:
"cache": 0,
"navigational-prefetch": 1
Note if no value is provided, it is assumed to be index 0 (cache).
Example:
*9[0|1...] // possible values
*9 // cache
*91 // navigational-prefetch
*92 // unknown value, value included
// in another array on the beacon
renderBlockingStatusThe Render Blocking Status is included for some assets as .renderBlockingStatus.
This information is encoded as a "special value" in the resource's timings array. It will be appended to the
list of timings with a special prefix of *a.
If set, it means the Render Blocking Status was blocking.
Example:
*a // blocking
responseStatusThe HTTP Response Status Code is included for some assets as .responseStatus.
This information is encoded as a "special value" in the resource's timings array. It will be appended to the
list of timings with a special prefix of *b.
If set, it is the Base36 encoded HTTP Response Status Code.
If included but no value is set, HTTP 200 is assumed.
Example:
*b[base36 code] // possible values
*b // HTTP 200
*bb8 // HTTP 404
We call contribution of a resource to a page the proportion of the total load time that can be blamed on that resource. We want contribution scores to encourage parallelization and not only short resources.
It enables us to study resource impact in a more meaningful way over simply looking at raw load times.
Here is an example with only 2 resources to get some intuition into how it works:
0 100ms
A |-----------------------------------------|
50ms 100ms
B |----------------------|
The contribution of resource A is 50ms for the 0-50ms time range and 50ms/2 for the 50ms to 100ms time range because it overlaps with B. We get:
contribution_A = (50 + 50 / 2) / 100 = .75
contribution_B = (50 / 2) / 100 = .25
This is computed based on all the resources in a single beacon. It is not done by default because it is computationally expensive.
Here is a code example to add contributions to your own resources:
var original = ResourceTimingDecompression.decompressResources(rtData);
// "original" is now an array of resource timings.
ResourceTimingDecompression.addContribution(original);
// Resources in "original" now have a field "contribution".
Tests are provided in the test/ directory, and can be run via mocha:
mocha test/*
Or via gulp:
gulp test
nextHopProtocol datafetchStart for Service Worker Startup dataskipDimensions for getResourceTiming() and compressResourceTiming()initiatorType above 9 properlyinitiatorType valuescompression and decompression exportsResourceTimingCompression.getVisibleEntries: Fixed onload bug, updated to work with <PICTURE> elementsResourceTimingCompression: Dimensions work with HOSTNAMES_REVERSEDLINK rel and SCRIPT attributesINITIATOR_TYPESnaturalHeight and naturalWidth to dimensionData (#20)getResourceTiming() now returns an object of { restiming, servertiming }
instead of just restiming (#17)beacon, fetchasync, body and defer flags for SCRIPT types (#16)naturalHeight and naturalWidth for dimensions (#16)src attribute capture and HREF handling (3796c2ae)rect.height|width for dimensions (3796c2ae)FAQs
ResourceTiming compression and decompression
The npm package resourcetiming-compression receives a total of 410 weekly downloads. As such, resourcetiming-compression popularity was classified as not popular.
We found that resourcetiming-compression demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
This episode explores the hard problem of reachability analysis, from static analysis limits to handling dynamic languages and massive dependency trees.
Security News
/Research
Malicious Nx npm versions stole secrets and wallet info using AI CLI tools; Socket’s AI scanner detected the supply chain attack and flagged the malware.
Security News
CISA’s 2025 draft SBOM guidance adds new fields like hashes, licenses, and tool metadata to make software inventories more actionable.