Files
cherrypick/benchmark_cherrypick

benchmark_cherrypick

Benchmark suite for cherrypick DI container and its features.

Overview

This package provides comprehensive benchmarks for the cherrypick dependency injection core and comparable DI scenarios. It includes a CLI tool for running a matrix of synthetic scenarios—covering depth and breadth, named resolutions, scope overrides, async chains, memory usage and more.

Key Features:

  • Declarative matrix runs (chain count, nesting depth, scenario, repeats)
  • CLI tool with flexible configuration
  • Multiple report formats: pretty table, CSV, JSON, Markdown
  • Memory and runtime statistics (mean, median, stddev, min, max, memory diffs)
  • Built-in and extensible scenarios (singletons, factories, named, async, overrides)
  • Easy to extend with your own modules/adapters

Benchmark Scenarios

  • RegisterSingleton: Registers and resolves a singleton dependency
  • ChainSingleton: Resolves a deep chain of singleton dependencies (A→B→C...)
  • ChainFactory: Resolves a deep chain using factory bindings (new instance each time)
  • AsyncChain: Resolves an async dependency chain (async providers)
  • Named: Resolves a named dependency from several implementations
  • Override: Resolves a dependency overridden in a child scope

How to Run

  1. Get dependencies:

    dart pub get
    
  2. Run all benchmarks (default single configuration, 2 warmups, 2 repeats):

    dart run bin/main.dart
    
  3. Show available CLI options:

    dart run bin/main.dart --help
    

CLI Parameters

  • --benchmark, -b — Benchmark scenario:
    registerSingleton, chainSingleton, chainFactory, asyncChain, named, override, all (default: all)
  • --chainCount, -c — Comma-separated chain counts, e.g. 10,100
  • --nestingDepth, -d — Comma-separated chain depths, e.g. 5,10
  • --repeat, -r — Number of measurement runs per scenario (default: 2)
  • --warmup, -w — Warmup runs before measuring (default: 1)
  • --format, -f — Output format: pretty, csv, json, markdown (default: pretty)
  • --help, -h — Show usage

Examples

  • Matrix run:

    dart run bin/main.dart --benchmark=chainSingleton --chainCount=10,100 --nestingDepth=5,10 --repeat=5 --warmup=2 --format=markdown
    
  • Run just the named scenario:

    dart run bin/main.dart --benchmark=named --repeat=3
    

Example Output (Markdown)

| Benchmark         | Chain Count | Depth | Mean (us) | ... | PeakRSS(KB) |
|------------------|-------------|-------|-----------| ... |-------------|
| ChainSingleton   | 10          | 5     | 2450000   | ... | 200064      |

Report Formats

  • pretty — Tab-delimited table (human-friendly)
  • csv — Machine-friendly, for spreadsheets/scripts
  • json — For automation, data pipelines
  • markdown — Markdown table for docs/wikis/issues

How to Add Your Own Benchmark

  1. Implement a class extending BenchmarkBase (sync case) or AsyncBenchmarkBase.
  2. Configure scenario modules/services using the DI adapter interface.
  3. Add scenario selection logic if needed (see bin/main.dart).
  4. Optionally extend reporters or adapters for new DI libraries.

Example minimal benchmark:

class MyBenchmark extends BenchmarkBase {
  MyBenchmark() : super('My custom');
  @override void setup() { /* setup test DI modules */ }
  @override void run()   { /* resolve or invoke dependency chain */ }
  @override void teardown() { /* cleanup if needed */ }
}

To plug in a new DI library, implement DIAdapter and register it in CLI.


Metrics Collected

All benchmarks record:

  • Time (microseconds): mean, median, stddev, min, max, timings
  • Memory:
    • memory_diff_kb — change in RSS (KB)
    • delta_peak_kb — change in peak RSS (KB)
    • peak_rss_kb — absolute peak RSS (KB)

License

MIT