Projects
Portfolio build 0.00%

Selected work · 2015 – 2022

The Art ofGrayness

The first things I ever made seriously were graphite portraits. This section begins there.

View repository
Graphite portrait of Albert Einstein

Albert Einstein

2015

QA case study · 02

Testing Scope Framework

Risk-based test design · production practice

Can we expect two independent QA engineers to complete the same task with the same result?

Imagine two identical parallel universes with one exception: the QA engineer assigned to the task. How can we shape the QA process for an AAA system with a complex Core Gameplay structure so that we can expect the same result in both universes?

My experience in AAA suggests that the individual specialist has too much influence on the outcome. So what can we do about it?

Identical inputSame change request
Universe AQA APersonal interpretationScope A
Universe BQA BPersonal interpretationScope B
Same task. Different reasoning.Experience and individual interpretation can produce two valid-looking but materially different test scopes.

Individuality is valuable – until it leads to a poor outcome.

Individuality can create frameworks like this. It can also create the reasons such frameworks are needed.

When we talk about quality, we mean bringing the expected and actual results into alignment. We cannot accept a situation in which we do not know what to expect.

The original problem

A universal AAA checklist does not exist.

A check is not inherently mandatory or optional.

This makes it important not to settle for the apparently safest conclusion: test everything. In an AAA environment, deciding what does not need to be tested is the hardes – and perhaps the most importan – question.

This framework builds that decision around three parameters:

Depth defines vertical coverage.

It determines how many rules, states, boundaries, combinations and failure paths must be investigated inside the changed functionality.

Distrust is the space for individuality and QA expertise. It is always applied on top of the depth and breadth calculation – and it remains visible, explainable and open to discussion.

Maximum coverage is not automatically maximum quality.When every layer is treated as equally necessary, scope becomes expensive but the reasoning behind it remains invisible.

The framework helps explain what does not need to be tested – and why.

Where did the idea come from?The initial question grew from my work at Strikerz on UFL, specifically within Core Gameplay QA. Independent public workThe framework was substantially redesigned and discloses no internal data, processes or implementation details.

Interactive model

Estimate the minimum test scope

Choose what the change can affect and how difficult it is to implement safely.

01
Impact measures consequence.

Player exposure, affected systems, persistence, business risk and the cost of an unnoticed failure. It answers how far regression can travel.

02
Complexity measures uncertainty.

Rules, states, branches, dependencies, timing and technical novelty. It answers how many ways the implementation can behave unexpectedly.

Why 60 / 40? Impact is weighted higher because testing scope should primarily protect the product from consequences. Complexity increases uncertainty, but difficult code with a tiny blast radius should not automatically receive release-wide regression.

01 Impact

How widely can a failure affect the product or player?

02 Complexity

How many rules, states, dependencies or technical risks change?

Recommended minimumM
Deep Validation

Verify the complete mechanic as a system of calculations, states, transitions and dependencies.

ceil(2 × .6 + 3 × .4) = 3
Recommendation matrixRows: development complexity · Columns: impact
ImpactComplexityXSSMLXL
XSXSSMML
SSSMLL
MSMMLXL
LMMLLXL
XLMLLXLXL
The ceiling is deliberate: an intermediate score is rounded upward because uncertainty should not reduce the minimum recommendation. The matrix starts a review; QA can raise the result when evidence or context requires it.

One change · five possible scopes

What the recommendation changes

qalab example · level M

Ratio-dependent safe temperature

The safe temperature now depends on the AmountA/AmountB ratio, creating combinations and a dynamic boundary.

  1. DepthCalculations, combinations, boundaries, states and data consistency
  2. BreadthThe whole Reaction Experiment mechanic
  3. BoundaryTesting does not expand beyond the mechanic

Inherited verification

What becomes required?

Every level includes the required checks from the levels below it.

Requirement complianceChange pointPositive scenariosNegative scenariosBoundary valuesState transitionsAction combinationsFormula checksData consistencyMechanic end-to-endSystem interactionsLocal regressionAdjacent regressionCross-domain regressionExtreme conditionsPerformanceCross-team involvement
RequiredOutside minimum10 of 17 check groups required at M

The critical validation

A consistent formula can still produce a consistently wrong scope.

01 · TaskWhat actually changes?

Goal, requirements, dependencies and credible failure modes.

→
02 · AssignedImpact + Complexity

Values proposed by the development process.

→
03 · QA gateValidate the inputs

QA checks that the assigned values actually hold up.

→
04 · ScopeUse the matrix

Only validated inputs produce a defensible minimum.

Correct inputs → Comparable scopeIncorrect inputs → Consistently wrong scope

Scope and workflow

One framework does not fit every delivery path.

New functionalityBug fixesRefactoringTechnical changesR&D hypotheses
Base workflow

Understand → Validate → Calculate → Test → Finalize

The framework applies when the team can validate inputs and build a reasoned minimum scope.

Explicit exclusion: emergency and hotfix branches require a separate faster process.

Production evidence library

A decision needs more than PASS

Select a source to see what question it answers.

01 · Logic & state

Blueprints show what the feature can do.

Inspect execution paths, state ownership and dependencies before deciding which conditions and transitions need direct validation.

QuestionWhich paths, states and connected systems can this change reach?
Understand the implementation → map test conditions

Common failure modes

Most failures begin before testing.

Input failure · goal

The team cannot agree on success.

If the intended result is vague or no longer relevant, neither the matrix nor a detailed checklist can make the conclusion reliable.

Before testing continuesRestate the current goal, expected behavior and acceptance boundary in the task.

The actual framework

Use the model, not a reconstruction.

This is the working self-contained tool from the public repository. Change its inputs and copy the resulting qalab checklist.

View source ↗

The decision artifact

A level is not the deliverable.

Another engineer must understand what is included, excluded and still uncertain.

Recommended levelM

Impact S · Complexity M

Expert override: none
INCLUDED
✓ Complete mechanic
✓ Boundaries and state transitions
✓ Formula and data consistency

EXCLUDED
 – Adjacent-system regression
 – Performance validation

EVIDENCE
Automated runs · Logs · Baseline comparison

Calibration, not dogma

The values may be calibrated.
The reasoning must remain shared.

01Start with 60 / 40

Impact is weighted higher because consequence and regression breadth take priority over implementation difficulty.

02Validate historically

Compare recommendations with escaped defects, over-testing costs and real tasks from the product.

03Change as a team

Weights may evolve, but they cannot vary invisibly from one engineer or task to another.

QA strategy · Unreal Engine · production-informed
Concept developed in 2025Repository ↗

QA case study · 03

Analytics-driven QA

Reading guideQuestionMethodEvidenceDecision

The question

How do you test what the interface cannot show?

In a mobile game, a tap can trigger analytics, experiment assignment, advertising callbacks and cloud-save transitions. The UI shows the outcome; ClickHouse shows the evidence.

Synthetic funnel7 days

Start 4,293
Complete 4,064
Fail 3
A failed ad event is an outcome signal – not automatically an ANR or crash.

The method

01ActionReproduce a controlled test step → 02EventLocate the emitted telemetry → 03QueryCheck sequence and parameters → 04DecisionConfirm, investigate or prioritize

01 · Event trail

Rebuild a user session

Read only

Trace ordered analytics events and inspect experiment context without touching production state.

How to read this

This baseline trace reconstructs the session. Device-owned install and session rows can leave product fields empty, so they are filtered before event order is assessed.
Context ClickHouse · OLAP · columnar 
SELECT
    created_at,
    activity_kind,
    event_name,
    JSONExtractString(base_parameters, 'ab_group') AS ab_group
FROM analytics.realtime_events
WHERE user_id = {user_id:String}
  AND toDate(created_at) = today()
  AND activity_kind NOT IN ('install', 'session')
ORDER BY created_at DESC;
Signal ordered event trailUse reproduce & verifyGuard synthetic identifiers

The evidence

Some of the QA questions

Choose a scenario to update the explanation, query and interpretation.

Conclusion · decision

01

The event sequence is the evidence

The logging contract is healthy when the emitted order and parameters match the controlled actions performed on the device.

Decision

Confirm the scenario, or investigate the first missing, duplicated or malformed event.

Article conclusions

  • Use telemetry to verify what the interface cannot expose.
  • Interpret counts against an expected resul – not in isolation.
  • Keep diagnostics read-only; perform recovery only through approved tooling.

QA case study · 01

qalab – quality assurance lab for Preventive Regression Signaling

MechanicAutomateMeasureDecide

Analytical Chemistry × Quality Assurance × Data

Functional tests vs. Behavioral tests

Functional checks may produce the expected result, but that does not guarantee that behavioral checks will do the same. In other words, testing a feature canno – and should no – end with a binary true/false or passed/failed result. What happens afterward is just as important.

Unreal Engine laboratory prototype with the player character at the reaction station
Actual Unreal Engine prototype

One core mechanic

Five testing views
Manual testing via UI Automated tests Test stands Analytics tests Blueprint debug panel
The same production mechanic is exercised through five complementary testing views.
Functional PASS·Behavioral WARNINGReactionTime crept −2% → −6% over 5 days – every single day passed, only the trend crossed ±5%.

01 · VALIDATE

33 / 33 PASS

Functional gameplay automation

Domain rules, state transitions, interaction range, parameter sweeps, exports and regression logic are exercised through Unreal Automation Framework.

Controlled fixtureday 5 of 5
Versioned baseline12.42 sReactionTime mean
→
Friday's batch11.67 sReactionTime mean
Regression analyzerReady

› Waiting for baseline and current batch.

29 simple tests + 1 parameterized test with 4 cases · Editor-only qalabTests module

The connection

Standardize→Validate→Analyze

Graphite portrait

Albert Einstein

Graphite portrait of Albert Einstein
Original artwork · graphite on paper 2015