Research
Security News
Malicious npm Package Targets Solana Developers and Hijacks Funds
A malicious npm package targets Solana developers, rerouting funds in 2% of transactions to a hardcoded address.
@angular/benchpress
Advanced tools
Benchpress is a framework for e2e performance tests. See here for an example project.
The sources for this package are in the main Angular repo. Please file issues and pull requests against that repo.
License: MIT
There are so called "micro benchmarks" that essentially use a stop watch in the browser to measure time
(e.g. via performance.now()
). This approach is limited to time, and in some cases memory
(Chrome with special flags), as metric. It does not allow to measure:
This kind of data is already available in the DevTools of modern browsers. However, there is no standard way to use those tools in an automated way to measure web app performance, especially not across platforms.
Benchpress tries to fill this gap, i.e. allow to access all kinds of performance metrics in an automated way.
Benchpress uses webdriver to read out the so called "performance log" of browsers. This contains all kinds of interesting data, e.g. when a script started/ended executing, gc started/ended, the browser painted something to the screen, ...
As browsers are different, benchpress has plugins to normalizes these events.
console.time()
/ console.timeEnd()
console.time()
/ console.timeEnd()
mark the timeline in the DevTools, so it makes sense
to use them in micro benchmark to visualize and understand them, with or without benchpress.A benchmark in benchpress is made by an application under test and a benchmark driver. The application under test is the actual application consisting of html/css/js that should be tests. A benchmark driver is a webdriver test that interacts with the application under test.
Let's assume we want to measure the script execution time, as well as the render time that it takes to fill a container element with a complex html string.
The application under test could look like this:
index.html:
<button id="reset" onclick="reset()">Reset</button>
<button id="fill" onclick="fill()">fill innerHTML</button>
<div id="container"></div>
<script>
var container = document.getElementById('container');
var complexHtmlString = '...'; // TODO
function reset() { container.innerHTML = ''; }
function fill() {
container.innerHTML = complexHtmlString;
}
</script>
A benchmark driver could look like this:
// A runner contains the shared configuration
// and can be shared across multiple tests.
var runner = new Runner(...);
driver.get('http://myserver/index.html');
var resetBtn = driver.findElement(By.id('reset'));
var fillBtn = driver.findElement(By.id('fill'));
runner.sample({
id: 'fillElement',
// Prepare is optional...
prepare: () {
resetBtn.click();
},
execute: () {
fillBtn.click();
// Note: if fillBtn would use some asynchronous code,
// we would need to wait here for its end.
}
});
If the application under test would like to, it can measure on its own. E.g.
index.html:
<button id="measure" onclick="measure()">Measure document.createElement</button>
<script>
function measure() {
console.time('createElement*10000');
for (var i=0; i<10000; i++) {
document.createElement('div');
}
console.timeEnd('createElement*10000');
}
</script>
When the measure
button is clicked, it marks the timeline and creates 10000 elements.
It uses the special names createElement*10000
to tell benchpress that the
time that was measured is for 10000 calls to createElement and that benchpress should
take the average for it.
A test driver for this would look like this:
driver.get('.../index.html');
var measureBtn = driver.findElement(By.id('measure'));
runner.sample({
id: 'createElement test',
microMetrics: {
'createElement': 'time to create an element (ms)'
},
execute: () {
measureBtn.click();
}
});
When looking into the DevTools Timeline, we see a marker as well:
console.time
It's also possible to measure any "user metric" within the browser
by setting a numeric value on the window
object. For example:
bootstrap(App)
.then(() => {
window.timeToBootstrap = Date.now() - performance.timing.navigationStart;
});
A test driver for this user metric could be written as follows:
describe('home page load', function() {
it('should log load time for a 2G connection', done => {
runner.sample({
execute: () => {
browser.get(`http://localhost:8080`);
},
userMetrics: {
timeToBootstrap: 'The time in milliseconds to bootstrap'
},
providers: [
{provide: RegressionSlopeValidator.METRIC, useValue: 'timeToBootstrap'}
]
}).then(done);
});
});
Using this strategy, benchpress will wait until the specified property name,
timeToBootstrap
in this case, is defined as a number on the window
object
inside the application under test.
Benchpress can also measure the "smoothness" of scrolling and animations. In order to do that, the following set of metrics can be collected by benchpress:
frameTime.mean
: mean frame time in ms (target: 16.6ms for 60fps)frameTime.worst
: worst frame time in msframeTime.best
: best frame time in msframeTime.smooth
: percentage of frames that hit 60fpsTo collect these metrics, you need to execute console.time('frameCapture')
and console.timeEnd('frameCapture')
either in your benchmark application or in you benchmark driver via webdriver. The metrics mentioned above will only be collected between those two calls and it is recommended to wrap the time/timeEnd calls as closely as possible around the action you want to evaluate to get accurate measurements.
In addition to that, one extra provider needs to be passed to benchpress in tests that want to collect these metrics:
benchpress.sample(providers: [{provide: bp.Options.CAPTURE_FRAMES, useValue: true}], ... )
Benchpress can also record the number of requests sent and count the received "encoded" bytes since window.performance.timing.navigationStart:
receivedData
: number of bytes received since the last navigation startrequestCount
: number of requests sent since the last navigation startTo collect these metrics, you need the following corresponding extra providers:
benchpress.sample(providers: [
{provide: bp.Options.RECEIVED_DATA, useValue: true},
{provide: bp.Options.REQUEST_COUNT, useValue: true}
], ... )
Use normalized environments
Use relative comparisons
Assert post-commit for commit ranges
Repeat benchmarks multiple times in a fresh window
Use force gc with care
Open a new window for every test
Definitions:
Components:
Runner
Sampler
Metric
Validator
scriptTime
through the last 10 measure values is >=0, i.e. the values for the scriptTime
metric are no more decreasingReporter
WebDriverAdapter
WebDriverExtension
FAQs
Benchpress - a framework for e2e performance tests
We found that @angular/benchpress demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
A malicious npm package targets Solana developers, rerouting funds in 2% of transactions to a hardcoded address.
Security News
Research
Socket researchers have discovered malicious npm packages targeting crypto developers, stealing credentials and wallet data using spyware delivered through typosquats of popular cryptographic libraries.
Security News
Socket's package search now displays weekly downloads for npm packages, helping developers quickly assess popularity and make more informed decisions.