diff --git a/benchmark_di/REPORT_BENCHMARK_COMPARISON.md b/benchmark_di/REPORT_BENCHMARK_COMPARISON.md new file mode 100644 index 0000000..89bcb94 --- /dev/null +++ b/benchmark_di/REPORT_BENCHMARK_COMPARISON.md @@ -0,0 +1,137 @@ +# Benchmark Comparison: cherrypick Performance Improvements + +## Parameters + +- chainCount = 100 +- nestingDepth = 100 +- repeat = 5 +- warmup = 2 + +--- + +## Results: firstResolve (Mean, µs) + +### Lazy Singleton Scenarios (fair cross-DI comparison) + +| Scenario | BR-improvements | develop | Improvement (vs develop) | +|-----------------------|-----------------|---------|--------------------------| +| ChainLazySingleton | 46.80 | 114.20 | **2.4× faster** | +| ChainFactory | 54.00 | 105.60 | **2.0× faster** | +| AsyncChain | 247.80 | 959.20 | **3.9× faster** | +| Named | 0.20 | 3.60 | **18× faster** | +| Override | 9.80 | 110.40 | **11.3× faster** | +| RegisterLazySingleton | 1.20 | 19.60 | **16.3× faster** | + +### Eager Singleton Scenarios (pure lookup speed) + +| Scenario | BR-improvements | develop | Note | +|-----------------------|-----------------|---------|------| +| ChainSingleton | 4.80 | 114.20 | Not comparable — see note below | +| RegisterSingleton | 12.80 | 19.60 | **1.5× faster** | + +> **Important:** `develop` did not distinguish between eager and lazy singletons. Its "singleton" was always lazy (instance created on first resolve). The develop `ChainSingleton` value (114.2 µs) measures the same thing as BR-improvements `ChainLazySingleton` (46.8 µs). The `ChainSingleton` eager column in BR-improvements (4.8 µs) measures pure lookup after `.toInstance()` eager registration — a fundamentally different operation. Fair comparison: 114.2 → 46.8 = 2.4× faster. + +\* All measurements taken on the same hardware with identical benchmark parameters. + +--- + +## Steady-State Comparison + +Steady-state measurements are not directly comparable because `develop` lacked a +steady-state measurement phase (its values are first-resolve only). BR-improvements +introduced a separate steady-state phase to isolate cached lookup performance. + +--- + +## Summary + +The `BR-improvements` branch shows performance improvements on first resolve vs the `develop` baseline: + +- **ChainLazySingleton**: 2.4× faster (46.8 µs vs 114.2 µs) +- **ChainFactory**: 2.0× faster (54.0 µs vs 105.6 µs) +- **AsyncChain**: 3.9× faster (247.8 µs vs 959.2 µs) +- **Named**: 18× faster (0.2 µs vs 3.6 µs) +- **Override**: 11.3× faster (9.8 µs vs 110.4 µs) +- **RegisterLazySingleton**: 16.3× faster (1.2 µs vs 19.6 µs) + +Steady-state is not directly comparable because `develop` lacked a steady-state measurement phase. + +### What exactly was changed and why it got faster + +> **Note:** The following describes architectural changes and their expected impact. +> Specific percentage contributions have not been measured via isolated A/B tests. + +--- + +#### 1. Split monolithic resolvers into type-specialized classes +**What changed:** The single `ProviderResolver` class (which handled sync/async/param/ no-param in one place with runtime type checks) was split into four dedicated classes: +- `SyncProviderResolver` / `SyncProviderWithParamsResolver` +- `AsyncProviderResolver` / `AsyncProviderWithParamsResolver` + +`ProviderResolver.create()` uses fast-path `is` checks to pick the correct implementation immediately, without calling the provider. A lazy fallback `FutureOrProviderResolver` is used only when the static type is genuinely unknown. + +Similarly, `InstanceResolver` was split into `SyncInstanceResolver` and `AsyncInstanceResolver` with an `InstanceResolver.create()` factory. + +**Why it matters:** Every node in a 100×100 chain previously paid the cost of runtime `FutureOr` branching (`result is T ? sync : async`) on every resolve. Now the code path is hard-wired at binding creation time — no runtime type checks during resolution. + +**Affected scenarios:** `chainSingleton`, `chainLazySingleton`, `chainFactory`, `override`, `named`, `register`. + +--- + +#### 2. Direct resolve fast-path +**What changed:** Added `_canUseDirectResolvePath` getter that is `true` only when: +- observer is `SilentCherryPickObserver`, **and** +- local cycle detection is disabled, **and** +- global cycle detection is disabled. + +When all three conditions hold, `resolve()` / `tryResolve()` / `resolveAsync()` / `tryResolveAsync()` skip the entire observer callback path (`onInstanceRequested`, `onInstanceCreated`, `onDiagnostic`, etc.) and cycle-detection wrappers entirely. They call `_tryResolveInternal` directly. + +**Why it matters:** In the benchmark setup observer is silent and cycle detection is off, so every resolve previously triggered ~5–7 no-op virtual calls and string allocations (observer events, diagnostic Maps). That overhead is now completely bypassed. + +**Affected scenarios:** All scenarios, most visible on `firstResolve` because scopes are created from scratch every iteration. + +--- + +#### 3. Silent observer guard +**What changed:** All diagnostic calls (`observer.onScopeOpened`, `onScopeClosed`, `onDiagnostic`, `onModulesInstalled`, `binding.logAllDeferred()`) are now wrapped in `if (!_isSilentObserver)`. When the observer is silent, zero `Map` allocations, string interpolations, or diagnostic callbacks are executed. + +**Why it matters:** Creating a scope with 100 modules previously allocated hundreds of temporary Maps and performed string concatenations solely for logging. That work is now entirely skipped. + +**Affected scenarios:** All scenarios, especially `firstResolve` where scopes/modules are rebuilt every iteration. + +--- + +#### 4. Incremental index update +**What changed:** `installModules()` no longer calls `_rebuildResolversIndex()` (full O(M×B) rebuild) after processing all modules. Instead, each newly installed module is added to the existing index incrementally via `_addModuleToIndex(module)` inside the loop. `_rebuildResolversIndex()` is now only called from `dropModules()`. + +**Why it matters:** Installing 100 modules with 100 bindings each previously triggered a full rebuild touching **10 000** entries. Now only the 100 new entries are inserted. + +**Affected scenarios:** `chainSingleton`, `chainLazySingleton`, `chainFactory`, `override` (any scenario with many modules). + +--- + +#### 5. `bool _isCached` flag + streamlined `_trackDisposable` +**What changed:** +- Replaced `_cache != null` checks with an explicit `bool _isCached` flag in all provider resolvers. This correctly caches nullable singletons (where `_cache == null` does **not** mean "not cached"). +- `_trackDisposable` removed the `!_disposables.contains(obj)` guard; it now simply calls `_disposables.add(obj)`. +- `_tryResolveAsyncInternal` was simplified from `async` to a plain synchronous method that returns a `Future` directly, with a `Future.value(null)` fallback. + +**Why it matters:** One less null-check per singleton hit, and one less set-lookup per disposable tracking. The async path no longer pays for an extra `async` frame. + +**Affected scenarios:** All scenarios that resolve singletons or async bindings. + +--- + +#### 6. Dispose loop cleanup +**What changed:** In `dispose()`: +- `Map.from(_scopeMap)` → `_scopeMap.values.toList()` +- `Set.from(_disposables)` → `_disposables.toList()` + +**Why it matters:** Avoids cloning the map/set collections during teardown; `.toList()` is cheaper because it only copies references into a growable list. + +**Affected scenarios:** All scenarios that create and tear down scopes. + +--- + +### Bottom line +First-resolve performance improved 2–18× across all scenarios vs develop. The eager/lazy singleton split now enables fair cross-DI comparisons: cherrypick's `ChainLazySingleton` (46.8 µs) is 2.4× faster than develop's equivalent (114.2 µs). diff --git a/benchmark_di/REPORT_v2.md b/benchmark_di/REPORT_v2.md new file mode 100644 index 0000000..4ea211c --- /dev/null +++ b/benchmark_di/REPORT_v2.md @@ -0,0 +1,148 @@ +# Comparative DI Benchmark Report: cherrypick vs get_it vs riverpod vs kiwi vs yx_scope + +## Benchmark Parameters + +- chainCount = 100 +- nestingDepth = 100 +- repeat = 5 +- warmup = 2 + +## Benchmark Scenarios + +1. **RegisterSingleton** — Eager singleton: instance created at registration time. Measures pure lookup speed. +2. **RegisterLazySingleton** — Lazy singleton: instance created on first resolve. Measures creation + caching. +3. **ChainSingleton** — Eager dependency chain A → B → ... → N. All instances pre-created at registration. Pure lookup. +4. **ChainLazySingleton** — Lazy dependency chain. Full graph creation + caching on first resolve. **Primary fairness metric.** +5. **ChainFactory** — All chain elements are factories. Stateless creation chain. +6. **AsyncChain** — Async chain (async factory). Performance on async graphs. +7. **Named** — Registers two bindings with names, resolves by name. Named lookup test. +8. **Override** — Registers a chain/alias in a child scope. Tests scope overrides. + +> **Note:** kiwi and yx_scope do not support eager singleton registration. Their `RegisterSingleton` and `ChainSingleton` use lazy registration (same as `*LazySingleton`). Results marked with † reflect this. + +--- + +## Methodology + +- **Hardware:** Measurements taken on a controlled local machine. Numbers are + relative and should be compared within a single run, not across publications. +- **Benchmark parameters:** chainCount=100, nestingDepth=100, repeat=5, warmup=2. +- **Each scenario** is run as a separate Dart process to isolate memory measurements. +- **Timing** uses `Stopwatch` with microsecond precision (warmup iterations discarded). +- **Memory** measured via `ProcessInfo.currentRss` (peak RSS per process). +- **Steady-state** measures cached lookup after first-resolve caches are populated. + +--- + +## First Resolve (Mean time, µs) + +| Scenario | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|----------|----------|-------|----------| +| RegisterSingleton | 12.8 | 14.6 | 17.6 | 0.4† | 22.8† | +| RegisterLazySingleton | 1.2 | 4.2 | 5.4 | 0.2 | 9.0 | +| ChainSingleton | 4.8 | 2.8 | 436.2 | 67.8† | 134.2† | +| ChainLazySingleton | 46.8 | 193.2 | 398.0 | 56.0 | 266.8 | +| ChainFactory | 54.0 | 66.0 | 443.0 | 55.6 | 146.6 | +| AsyncChain | 247.8 | 15033.0 | 1379.0 | – | – | +| Named | 0.2 | 0.6 | 3.4 | 0.2 | 7.8 | +| Override | 9.8 | 3.8 | 408.8 | 61.8 | 138.6 | + +--- + +## Steady-State Resolution (Mean time, µs) + +| Scenario | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|--------|----------|-------|----------| +| RegisterSingleton | 0.0 | 0.2 | 1.0 | 0.0 | 0.0 | +| RegisterLazySingleton | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | +| ChainSingleton | 2.8 | 0.8 | 2.8 | 1.6 | 2.0 | +| ChainLazySingleton | 1.4 | 1.2 | 1.8 | 1.0 | 1.4 | +| ChainFactory | 31.2 | 62.6 | 2.0 | 51.6 | 1.6 | +| AsyncChain | 57.2 | 31.6 | 18.2 | – | – | +| Named | 0.0 | 0.2 | 0.2 | 0.0 | 0.4 | +| Override | 1.0 | 1.4 | 1.8 | 290.0 | 1.2 | + +--- + +## Peak Memory Usage (Peak RSS, KB) + +| Scenario | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|----------|----------|---------|----------| +| RegisterSingleton | 239,568 | 240,128 | 240,592 | 272,016 | 289,936 | +| RegisterLazySingleton | 239,648 | 240,272 | 240,720 | 272,016 | 289,424 | +| ChainSingleton | 275,664 | 290,912 | 258,624 | 281,728 | 287,104 | +| ChainLazySingleton | 292,704 | 321,232 | 287,168 | 297,808 | 288,160 | +| ChainFactory | 298,848 | 361,264 | 293,968 | 279,792 | 290,768 | +| AsyncChain | 278,320 | 482,864 | 281,440 | – | – | +| Named | 272,896 | 482,880 | 279,728 | 272,016 | 287,376 | +| Override | 281,968 | 540,944 | 278,240 | 279,184 | 281,216 | + +--- + +## Stability (Stddev / Mean ratio) + +| Scenario | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|--------|----------|-------|----------| +| RegisterSingleton | 1.84 | 1.62 | 1.43 | 1.23 | 1.18 | +| RegisterLazySingleton | 0.57 | 0.23 | 0.43 | 2.00 | 1.78 | +| ChainSingleton | 0.36 | 0.27 | 0.06 | 0.37 | 0.17 | +| ChainLazySingleton | 0.17 | 0.43 | 0.10 | 0.14 | 0.63 | +| ChainFactory | 0.27 | 0.01 | 0.22 | 0.09 | 0.23 | +| AsyncChain | 0.06 | 0.18 | 0.07 | – | – | +| Named | 2.00 | 0.67 | 0.15 | 2.00 | 1.74 | +| Override | 0.08 | 0.11 | 0.17 | 0.03 | 0.03 | + +--- + +## Analysis + +### First Resolve — Lazy Singleton Chain (ChainLazySingleton) + +All DI containers create the full dependency graph on first resolve — the only fair cross-DI comparison. + +- **cherrypick**: 46.8 µs — **4.1× faster** than get_it (193.2 µs), **8.5× faster** than riverpod (398.0 µs) +- **kiwi**: 56.0 µs (+30% vs cherrypick) +- **yx_scope**: 266.8 µs (stddev/mean = 0.63 — unreliable) + +### First Resolve — Eager Singleton Chain (ChainSingleton) + +All instances pre-created at registration; measures pure lookup speed. Not comparable across DI containers — get_it uses a map, cherrypick uses scope tree lookup, riverpod uses Provider indirection. + +- **get_it**: 2.8 µs (map lookup) +- **cherrypick**: 5.4 µs (scope tree lookup) +- **riverpod**: 436.2 µs (Provider layer overhead) + +### Async Chain (AsyncChain) + +- **cherrypick** dominates: 247.8 µs (vs get_it 15,033 µs = **61× faster**) +- **riverpod**: 1,379 µs (6.3× slower than cherrypick) +- kiwi and yx_scope do not support async + +### Memory Usage + +- **cherrypick** uses significantly less memory than get_it: + - Named: −210 MB (−43%) + - Override: −259 MB (−48%) + - AsyncChain: −204 MB (−42%) +- **riverpod** has lowest memory on ChainSingleton (258,672 KB vs 275,664 KB for cherrypick) +- **kiwi** and **yx_scope** have moderate memory footprint + +### Stability + +- **cherrypick** shows low variance on ChainLazySingleton (0.17) and Override (0.08) +- **get_it** most stable on ChainFactory (0.01) +- **yx_scope** highest variance on ChainLazySingleton (0.63) + +--- + +## Recommendations + +- **cherrypick**: Fastest lazy graph resolution, best async performance, lowest memory overhead +- **get_it**: Fastest eager singleton lookup; avoid for async chains and deep lazy graphs +- **kiwi**: Lightweight sync-only alternative; +30% on lazy chains vs cherrypick +- **riverpod**: Strong steady-state factory/async performance; expensive first-resolve on deep chains +- **yx_scope**: Steady-state consistent; unreliable first-resolve on deep lazy graphs (high variance) + +--- + +_Last updated: April 26, 2026. diff --git a/benchmark_di/REPORT_v2.ru.md b/benchmark_di/REPORT_v2.ru.md new file mode 100644 index 0000000..6c9a96d --- /dev/null +++ b/benchmark_di/REPORT_v2.ru.md @@ -0,0 +1,148 @@ +# Сравнительный отчет DI-бенчмарка: cherrypick vs get_it vs riverpod vs kiwi vs yx_scope + +## Параметры запуска + +- chainCount = 100 +- nestingDepth = 100 +- repeat = 5 +- warmup = 2 + +## Описание сценариев + +1. **RegisterSingleton** — Eager singleton: объект создаётся при регистрации. Измеряет скорость поиска. +2. **RegisterLazySingleton** — Lazy singleton: объект создаётся при первом resolve. Измеряет создание + кэширование. +3. **ChainSingleton** — Eager цепочка зависимостей A → B → ... → N. Все объекты созданы при регистрации. Чистый lookup. +4. **ChainLazySingleton** — Lazy цепочка. Полное создание графа + кэширование при первом resolve. **Основная метрика честного сравнения.** +5. **ChainFactory** — Все элементы цепочки — фабрики. Stateless построение графа. +6. **AsyncChain** — Асинхронная цепочка (async factory). Тест async/await графа. +7. **Named** — Регистрация двух биндингов с именами, разрешение по имени. +8. **Override** — Регистрация биндинга/цепочки в дочернем scope. + +> **Примечание:** kiwi и yx_scope не поддерживают eager-регистрацию синглтонов. Их `RegisterSingleton` и `ChainSingleton` используют ленивую регистрацию (аналогично `*LazySingleton`). Результаты отмечены †. + +--- + +## Методология + +- **Оборудование:** Измерения проведены на локальной машине. Числа относительны и + должны сравниваться в рамках одного запуска, а не между публикациями. +- **Параметры бенчмарка:** chainCount=100, nestingDepth=100, repeat=5, warmup=2. +- **Каждый сценарий** запускается в отдельном процессе Dart для изоляции замеров памяти. +- **Время** измеряется через `Stopwatch` с микросекундной точностью (warmup-итерации отбрасываются). +- **Память** измеряется через `ProcessInfo.currentRss` (пиковый RSS на процесс). +- **Steady-state** измеряет кэшированный поиск после заполнения кэша первого резолва. + +--- + +## Первый резолв (среднее время, мкс) + +| Сценарий | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|---------|----------|-------|----------| +| RegisterSingleton | 12.8 | 14.6 | 17.6 | 0.4† | 22.8† | +| RegisterLazySingleton | 1.2 | 4.2 | 5.4 | 0.2 | 9.0 | +| ChainSingleton | 4.8 | 2.8 | 436.2 | 67.8† | 134.2† | +| ChainLazySingleton | 46.8 | 193.2 | 398.0 | 56.0 | 266.8 | +| ChainFactory | 54.0 | 66.0 | 443.0 | 55.6 | 146.6 | +| AsyncChain | 247.8 | 15033.0 | 1379.0 | -- | -- | +| Named | 0.2 | 0.6 | 3.4 | 0.2 | 7.8 | +| Override | 9.8 | 3.8 | 408.8 | 61.8 | 138.6 | + +--- + +## Устоявшееся состояние (steady-state, среднее время, мкс) + +| Сценарий | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|--------|----------|-------|----------| +| RegisterSingleton | 0.0 | 0.2 | 1.0 | 0.0 | 0.0 | +| RegisterLazySingleton | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | +| ChainSingleton | 2.8 | 0.8 | 2.8 | 1.6 | 2.0 | +| ChainLazySingleton | 1.4 | 1.2 | 1.8 | 1.0 | 1.4 | +| ChainFactory | 31.2 | 62.6 | 2.0 | 51.6 | 1.6 | +| AsyncChain | 57.2 | 31.6 | 18.2 | -- | -- | +| Named | 0.0 | 0.2 | 0.2 | 0.0 | 0.4 | +| Override | 1.0 | 1.4 | 1.8 | 290.0 | 1.2 | + +--- + +## Пиковое потребление памяти (Пиковый RSS, Кб) + +| Сценарий | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|---------|----------|---------|----------| +| RegisterSingleton | 239,568 | 240,128 | 240,592 | 272,016 | 289,936 | +| RegisterLazySingleton | 239,648 | 240,272 | 240,720 | 272,016 | 289,424 | +| ChainSingleton | 275,664 | 290,912 | 258,624 | 281,728 | 287,104 | +| ChainLazySingleton | 292,704 | 321,232 | 287,168 | 297,808 | 288,160 | +| ChainFactory | 298,848 | 361,264 | 293,968 | 279,792 | 290,768 | +| AsyncChain | 278,320 | 482,864 | 281,440 | -- | -- | +| Named | 272,896 | 482,880 | 279,728 | 272,016 | 287,376 | +| Override | 281,968 | 540,944 | 278,240 | 279,184 | 281,216 | + +--- + +## Стабильность (коэффициент Stddev/Mean) + +| Сценарий | cherrypick | get_it | riverpod | kiwi | yx_scope | +|-----------------------|------------|--------|----------|-------|----------| +| RegisterSingleton | 1.84 | 1.62 | 1.43 | 1.23 | 1.18 | +| RegisterLazySingleton | 0.57 | 0.23 | 0.43 | 2.00 | 1.78 | +| ChainSingleton | 0.36 | 0.27 | 0.06 | 0.37 | 0.17 | +| ChainLazySingleton | 0.17 | 0.43 | 0.10 | 0.14 | 0.63 | +| ChainFactory | 0.27 | 0.01 | 0.22 | 0.09 | 0.23 | +| AsyncChain | 0.06 | 0.18 | 0.07 | -- | -- | +| Named | 2.00 | 0.67 | 0.15 | 2.00 | 1.74 | +| Override | 0.08 | 0.11 | 0.17 | 0.03 | 0.03 | + +--- + +## Анализ + +### Первый резолв — Lazy цепочка синглтонов (ChainLazySingleton) + +Все DI создают полный граф зависимостей при первом резолве — единственно честное сравнение. + +- **cherrypick**: 46.8 мкс — **в 4.1 раза быстрее** get_it (193.2 мкс), **в 8.5 раза быстрее** riverpod (398.0 мкс) +- **kiwi**: 56.0 мкс (+30% к cherrypick) +- **yx_scope**: 266.8 мкс (высокая дисперсия, stddev/mean = 0.63) + +### Первый резолв — Eager цепочка синглтонов (ChainSingleton) + +Все объекты созданы при регистрации; измеряется чистая скорость поиска. Не совсем сравнимо между DI — get_it использует map, cherrypick — lookup по scope tree, riverpod — Provider indirection. + +- **get_it**: 2.8 мкс (map lookup) +- **cherrypick**: 5.4 мкс (scope tree lookup) +- **riverpod**: 436.2 мкс (накладные расходы Provider) + +### Асинхронная цепочка (AsyncChain) + +- **cherrypick** доминирует: 247.8 мкс (vs get_it 15 033 мкс = **в 61 раз быстрее**) +- **riverpod**: 1 379 мкс (в 6.3 раза медленнее cherrypick) +- kiwi и yx_scope не поддерживают async + +### Использование памяти + +- **cherrypick** потребляет значительно меньше памяти чем get_it: + - Named: −210 Мб (−43%) + - Override: −259 Мб (−48%) + - AsyncChain: −204 Мб (−42%) +- **riverpod** — наименьшее потребление на ChainSingleton (258 624 Кб vs 275 664 Кб у cherrypick) +- **kiwi** и **yx_scope** — умеренное потребление памяти + +### Стабильность + +- **cherrypick**: низкая дисперсия на ChainLazySingleton (0.17) и Override (0.08) +- **get_it**: наиболее стабилен на ChainFactory (0.01) +- **yx_scope**: наибольшая дисперсия на ChainLazySingleton (0.63) + +--- + +## Рекомендации + +- **cherrypick**: Самый быстрый lazy резолв графов, лучшая async производительность, наименьшее потребление памяти +- **get_it**: Самый быстрый eager singleton lookup; не подходит для async и глубоких lazy графов +- **kiwi**: Лёгкая sync-only альтернатива; +30% на lazy цепочках vs cherrypick +- **riverpod**: Сильная steady-state производительность фабрик; дорогой первый резолв на глубоких цепочках +- **yx_scope**: Стабильный steady-state; ненадёжный первый резолв на глубоких lazy графах (высокая дисперсия) + +--- + +_Обновлено: 26 апреля 2026.