benchmark_cherrypick

Benchmark suite for cherrypick DI container and its features.

Overview

This package provides comprehensive benchmarks for the cherrypick dependency injection core and comparable DI scenarios. It includes a CLI tool for running a matrix of synthetic scenarios—covering depth and breadth, named resolutions, scope overrides, async chains, memory usage and more.

Key Features:

Declarative matrix runs (chain count, nesting depth, scenario, repeats)
CLI tool with flexible configuration
Multiple report formats: pretty table, CSV, JSON, Markdown
Memory and runtime statistics (mean, median, stddev, min, max, memory diffs)
Built-in and extensible scenarios (singletons, factories, named, async, overrides)
Easy to extend with your own modules/adapters

Benchmark Scenarios

RegisterSingleton: Registers and resolves a singleton dependency
ChainSingleton: Resolves a deep chain of singleton dependencies (A→B→C...)
ChainFactory: Resolves a deep chain using factory bindings (new instance each time)
AsyncChain: Resolves an async dependency chain (async providers)
Named: Resolves a named dependency from several implementations
Override: Resolves a dependency overridden in a child scope

How to Run

Get dependencies:
```
dart pub get
```
Run all benchmarks (default single configuration, 2 warmups, 2 repeats):
```
dart run bin/main.dart
```
Show available CLI options:
```
dart run bin/main.dart --help
```

CLI Parameters

--benchmark, -b — Benchmark scenario:
registerSingleton, chainSingleton, chainFactory, asyncChain, named, override, all (default: all)
--chainCount, -c — Comma-separated chain counts, e.g. 10,100
--nestingDepth, -d — Comma-separated chain depths, e.g. 5,10
--repeat, -r — Number of measurement runs per scenario (default: 2)
--warmup, -w — Warmup runs before measuring (default: 1)
--format, -f — Output format: pretty, csv, json, markdown (default: pretty)
--help, -h — Show usage

Examples

Matrix run:

dart run bin/main.dart --benchmark=chainSingleton --chainCount=10,100 --nestingDepth=5,10 --repeat=5 --warmup=2 --format=markdown

Run just the named scenario:

dart run bin/main.dart --benchmark=named --repeat=3

Example Output (Markdown)

| Benchmark         | Chain Count | Depth | Mean (us) | ... | PeakRSS(KB) |
|------------------|-------------|-------|-----------| ... |-------------|
| ChainSingleton   | 10          | 5     | 2450000   | ... | 200064      |

Report Formats

pretty — Tab-delimited table (human-friendly)
csv — Machine-friendly, for spreadsheets/scripts
json — For automation, data pipelines
markdown — Markdown table for docs/wikis/issues

How to Add Your Own Benchmark

Implement a class extending BenchmarkBase (sync case) or AsyncBenchmarkBase.
Configure scenario modules/services using the DI adapter interface.
Add scenario selection logic if needed (see bin/main.dart).
Optionally extend reporters or adapters for new DI libraries.

Example minimal benchmark:

class MyBenchmark extends BenchmarkBase {
  MyBenchmark() : super('My custom');
  @override void setup() { /* setup test DI modules */ }
  @override void run()   { /* resolve or invoke dependency chain */ }
  @override void teardown() { /* cleanup if needed */ }
}

To plug in a new DI library, implement DIAdapter and register it in CLI.

Metrics Collected

All benchmarks record:

Time (microseconds): mean, median, stddev, min, max, timings
Memory:
- memory_diff_kb — change in RSS (KB)
- delta_peak_kb — change in peak RSS (KB)
- peak_rss_kb — absolute peak RSS (KB)

License

MIT

3.9 KiB Raw Blame History