tracerbench
CLI for TracerBench
Last updated 25 days ago by lynchbomb .
BSD-2-Clause · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install tracerbench 
SYNC missed versions from official npm registry.

TracerBench: Automated Chrome Tracing For Benchmarking

Build Status Version License

TracerBench Core

https://github.com/TracerBench/tracerbench/blob/master/README.md

Usage

$ npm install -g tracerbench
$ tracerbench COMMAND
running command...
$ tracerbench (-v|--version|version)
tracerbench/3.1.1 darwin-x64 node-v12.16.1
$ tracerbench --help [COMMAND]
USAGE
  $ tracerbench COMMAND
...

Cookies

The cookies.json format takes an array of cookies: https://chromedevtools.github.io/devtools-protocol/tot/Network#type-Cookie. The required keys are name, value, domain and path.

{
  "cookies": [
    {
      "name": "foo",
      "value": "COOKIE-VALUE-HERE",
      "domain": "www.foo.com",
      "path": "/"
    }
  ]
}

Optional Config

The optional existance of a "tbconfig.json" file in the project root will be consumed by TracerBench and specifies default command flag options. Please note this file is optional, however is strongly recommended as this drastically speeds up running TracerBench tests succinctly Example:

{
  "$schema": "https://raw.githubusercontent.com/TracerBench/tracerbench/master/packages/cli/tb-schema.json",
  // the title of the report pdf file
  "plotTitle": "tbconfig_base file",
  "tbResultsFolder": "../tracerbench-results",
  "controlURL": "https://www.tracerbench.com/",
  "experimentURL": "https://www.tracerbench.com/",
  "url": "https://www.tracerbench.com/",
  "tracingLocationSearch": "?tracing",
  "regressionThreshold": "100",
  "appName": "tracerbench",
  "cpuThrottleRate": 1,
  "network": "fast-3g",
  "emulateDevice": "iphone-x",
  "fidelity": "high",
  "markers": [
    {
      "start": "fetchStart",
      "label": "jquery"
    },
    {
      "start": "jqueryLoaded",
      "label": "ember"
    },
    {
      "start": "emberLoaded",
      "label": "application"
    },
    {
      "start": "startRouting",
      "label": "routing"
    },
    {
      "start": "willTransition",
      "label": "transition"
    },
    {
      "start": "didTransition",
      "label": "render"
    },
    {
      "start": "renderEnd",
      "label": "afterRender"
    }
  ],
  "socksPorts": [8880, 8881],
  "browserArgs": [
    "--disable-features=site-per-process,TranslateUI,BlinkGenPropertyTrees",
    "--disable-ipc-flooding-protection",
    "--v8-cache-options=none"
  ],
  "servers": [
    {
      "name": "control-tracerbench-build_1.0.1234",
      "url": "https://www.tracerbench.com/",
      "dist": "./relative-path-to-control-dist-files/",
      "har": "./relative-path-location-to-har-file/",
      "socksPort": 8880
    },
    {
      "name": "experiment-tracerbench-build_1.0.1234_v78gh95",
      "url": "https://www.tracerbench.com/",
      "dist": "./relative-path-to-experiment-dist-files/",
      "har": "./relative-path-location-to-har-file/",
      "socksPort": 8881
    }
  ]
}

FAQ

What exactly is contained within the tracerbench compare "tracerbench-results.json"? An overview of "tracerbench-results.json" is available here

What exactly is a HAR file? HAR (HTTP Archive) is a file format used by several HTTP session tools to export the captured data. The format is basically a JSON object with a particular field distribution. In any case, please note that not all the fields are mandatory, and many times some information won't be saved to the file "Additional insight on Google's HAR Analyzer"

What exactly is contained within the output file "trace.json"? The file "trace.json" is leveraged by TracerBench to capture an array of trace events. The interface of an individual trace event is essentially:

// process id
pid: number;
// thread id
tid: number;
// timestamp in μs
ts: number;
// event phase
ph: TRACE_EVENT_PHASE;
// event categories (comma delimited)
cat: string;
// event name
name: string;
// event key/value pairs
args: { [key: string]: any } | ARGS.STRIPPED;
// ?timestamp in μs for trace event phase complete
dur?: number;
// ?thread clock timestamp in μs
tts?: number;
// ?thread clock duration in μs for trace event phase complete
tdur?: number;
// ?thread clock timestamp for related async events
use_async_tts?: number;
// ?scope of id
scope?: string;
// ?event id. optionally serialized as int64
id?: string;
// ?scoped event ids
id2?: | { local: string; } | { global: string; };
// ?async event/event associations
bp?: string;
// ?flow binding id optionally serialized as int64
bind_id?: string;
// ?incoming flow flag
flow_in?: boolean;
// ?outgoing flow flag
flow_out?: boolean;
// ?scope for TRACE_EVENT_PHASE_INSTANT events
s?: TRACE_EVENT_SCOPE;

Commands

tracerbench compare

Compare the performance delta between an experiment and control

USAGE
  $ tracerbench compare

OPTIONS
  --browserArgs=browserArgs
      (required) [default: 
      --crash-dumps-dir=./tmp,--disable-background-timer-throttling,--disable-dev-shm-usage,--disable-cache,--disable-v8-i
      dle-tasks,--disable-breakpad,--disable-notifications,--disable-hang-monitor,--safebrowsing-disable-auto-update,--ign
      ore-certificate-errors,--v8-cache-options=none] (Default Recommended) Additional chrome flags for the TracerBench 
      render benchmark. TracerBench includes many non-configurable defaults in this category.

  --config=config
      Specify an alternative directory rather than the project root for the tbconfig.json. This explicit config will 
      overwrite all.

  --controlURL=controlURL
      Control URL to visit for compare command

  --cpuThrottleRate=cpuThrottleRate
      (required) [default: 2] CPU throttle multiplier

  --debug
      Debug flag per command. Will output noisy command

  --emulateDevice=iphone-4|iphone-5se|iphone-678|iphone-678-plus|iphone-x|blackberry-z30|nexus-4|nexus-5|nexus-5x|nexus-
  6|nexus-6p|pixel-2|pixel-2-xl|lg-optimus-l70|nokia-n9|nokia-lumia-520|microsoft-lumia-550|microsoft-lumia-950|galaxy-s
  -iii|galaxy-s5|kindle-fire-hdx|ipad-mini|ipad|ipad-pro|blackberry-playbook|nexus-10|nexus-7|galaxy-note-3|galaxy-note-
  ii|laptop-with-touch|laptop-with-hidpi-screen|laptop-with-mdpi-screen
      Emulate a mobile device screen size.

  --emulateDeviceOrientation=horizontal|vertical
      [default: vertical] Expected to be either "vertical" or "horizontal". Dictates orientation of device screen.

  --experimentURL=experimentURL
      Experiment URL to visit for compare command

  --fidelity=fidelity
      (required) [default: low] Directly correlates to the number of samples per trace. eg. test,low,medium,high OR any 
      number between 2-100

  --headless
      Run with headless chrome flags

  --hideAnalysis
      Hide the the analysis output in terminal

  --isCIEnv=isCIEnv
      Provides a drastically slimmed down stdout report for CI workflows. However does NOT hide analysis.

  --markers=markers
      (required) [default: domComplete] User Timing Markers

  --network=none|offline|dialup|slow-2g|2g|slow-edge|edge|slow-3g|dsl|3g|fast-3g|4g|cable|LTE|FIOS
      (required) [default: none] Simulated network conditions.

  --regressionThreshold=regressionThreshold
      [default: 100] The upper limit the experiment can regress slower in milliseconds. eg 100

  --report
      Generate a PDF report directly after running the compare command.

  --runtimeStats
      Compare command output deep-dive stats during run.

  --socksPorts=socksPorts
      Specify a socks proxy port as browser option for control and experiment

  --tbResultsFolder=tbResultsFolder
      (required) [default: ./tracerbench-results] The output folder path for all tracerbench results

  --tracingLocationSearch=tracingLocationSearch
      (required) [default: ?tracing] The document location search param.

See code: dist/src/commands/compare/index.ts

tracerbench compare:analyze RESULTSFILE

Generates stdout report from the "tracerbench compare" command output

USAGE
  $ tracerbench compare:analyze RESULTSFILE

ARGUMENTS
  RESULTSFILE  The "tracerbench compare" command json output file

OPTIONS
  --fidelity=fidelity                        (required) [default: low] Directly correlates to the number of samples per
                                             trace. eg. test,low,medium,high OR any number between 2-100

  --isCIEnv=isCIEnv                          (required) Provides a drastically slimmed down stdout report for CI
                                             workflows. However does NOT hide analysis.

  --regressionThreshold=regressionThreshold  (required) [default: 100] The upper limit the experiment can regress slower
                                             in milliseconds. eg 100

See code: dist/src/commands/compare/analyze.ts

tracerbench compare:report

Generates report files (PDF/HTML) from the "tracerbench compare" command output

USAGE
  $ tracerbench compare:report

OPTIONS
  --config=config                    Specify an alternative directory rather than the project root for the
                                     tbconfig.json. This explicit config will overwrite all.

  --tbResultsFolder=tbResultsFolder  (required) [default: ./tracerbench-results] The output folder path for all
                                     tracerbench results

ALIASES
  $ tracerbench report

See code: dist/src/commands/compare/report.ts

tracerbench help [COMMAND]

display help for tracerbench

USAGE
  $ tracerbench help [COMMAND]

ARGUMENTS
  COMMAND  command to show help for

OPTIONS
  --all  see all commands in CLI

See code: @oclif/plugin-help

tracerbench profile HARPATH

Parses a CPU profile and aggregates time across heuristics.

USAGE
  $ tracerbench profile HARPATH

ARGUMENTS
  HARPATH  The path to the HTTP Archive File (HAR)

OPTIONS
  --cookiespath=cookiespath
      (required) The path to a JSON file containing cookies to authenticate against the correlated URL

  --cpuThrottleRate=cpuThrottleRate
      (required) [default: 2] CPU throttle multiplier

  --hideUsertimings
      Hide stdout of user-timings.

  --network=none|offline|dialup|slow-2g|2g|slow-edge|edge|slow-3g|dsl|3g|fast-3g|4g|cable|LTE|FIOS
      [default: none] Simulated network conditions.

  --tbResultsFolder=tbResultsFolder
      (required) [default: ./tracerbench-results] The output folder path for all tracerbench results

  --url=url
      (required) URL to visit for record-har, timings & trace commands

ALIASES
  $ tracerbench trace

See code: dist/src/commands/profile.ts

tracerbench record-har

Generates a HAR file from a URL.

USAGE
  $ tracerbench record-har

OPTIONS
  --config=config            Specify an alternative directory rather than the project root for the tbconfig.json. This
                             explicit config will overwrite all.

  --cookiespath=cookiespath  (required) The path to a JSON file containing cookies to authenticate against the
                             correlated URL

  --dest=dest                (required) The destination path for the generated file

  --filename=filename        (required) [default: tracerbench] The filename for the generated file

  --headless                 Run with headless chrome flags

  --marker=marker            (required) [default: loadEventEnd] The last marker before ending a HAR recording

  --url=url                  (required) URL to visit for record-har, timings & trace commands

See code: dist/src/commands/record-har.ts

Current Tags

  • 2.0.0-beta.25                                ...           beta (a year ago)
  • 3.1.1                                ...           latest (25 days ago)

57 Versions

  • 3.1.1                                ...           25 days ago
  • 3.1.0                                ...           a month ago
  • 3.0.17                                ...           a month ago
  • 3.0.14                                ...           3 months ago
  • 3.0.13                                ...           4 months ago
  • 3.0.12                                ...           4 months ago
  • 3.0.11                                ...           5 months ago
  • 3.0.10                                ...           5 months ago
  • 3.0.9                                ...           5 months ago
  • 3.0.8                                ...           6 months ago
  • 3.0.7                                ...           6 months ago
  • 3.0.6                                ...           6 months ago
  • 3.0.5                                ...           6 months ago
  • 3.0.4                                ...           6 months ago
  • 3.0.3                                ...           6 months ago
  • 3.0.2                                ...           6 months ago
  • 3.0.1                                ...           7 months ago
  • 3.0.0                                ...           7 months ago
  • 2.3.0                                ...           7 months ago
  • 2.2.4                                ...           7 months ago
  • 2.2.3                                ...           7 months ago
  • 2.2.2                                ...           8 months ago
  • 2.2.1                                ...           8 months ago
  • 2.1.0                                ...           9 months ago
  • 2.0.7                                ...           9 months ago
  • 2.0.6                                ...           9 months ago
  • 2.0.5                                ...           9 months ago
  • 2.0.4                                ...           9 months ago
  • 2.0.3                                ...           10 months ago
  • 2.0.2                                ...           10 months ago
  • 2.0.1                                ...           10 months ago
  • 2.0.0                                ...           a year ago
  • 2.0.0-beta.26                                ...           a year ago
  • 2.0.0-beta.25                                ...           a year ago
  • 2.0.0-beta.18                                ...           a year ago
  • 2.0.0-beta.11                                ...           a year ago
  • 2.0.0-beta.10                                ...           a year ago
  • 2.0.0-beta.7                                ...           a year ago
  • 2.0.0-beta.6                                ...           a year ago
  • 2.0.0-beta.5                                ...           a year ago
  • 2.0.0-beta.4                                ...           a year ago
  • 2.0.0-beta.3                                ...           a year ago
  • 1.0.0                                ...           a year ago
  • 1.0.0-alpha.11                                ...           a year ago
  • 1.0.0-alpha.10                                ...           a year ago
  • 1.0.0-alpha.9                                ...           a year ago
  • 1.0.0-alpha.8                                ...           a year ago
  • 1.0.0-alpha.7                                ...           a year ago
  • 1.0.0-alpha.6                                ...           a year ago
  • 1.0.0-alpha.5                                ...           a year ago
  • 1.0.0-alpha.4                                ...           a year ago
  • 1.0.0-alpha.3                                ...           a year ago
  • 1.0.0-alpha.2                                ...           a year ago
  • 1.0.0-alpha.1                                ...           a year ago
  • 1.0.0-alpha.0                                ...           a year ago
  • 0.14.0                                ...           2 years ago
  • 0.0.1                                ...           2 years ago
Downloads
Today 0
This Week 0
This Month 1
Last Day 0
Last Week 0
Last Month 93
Dependencies (24)

Copyright 2014 - 2016 © taobao.org |