This section refines the Quality Goals into testable quality scenarios.
Each scenario follows the arc42 form: a source triggers a stimulus in a given environment, the system responds, and the response measure makes the response testable. Scenarios are usage scenarios (the system in operation), change scenarios (the system or its surroundings are modified), or failure scenarios (a part of the system or a neighbouring system fails).
Targets marked [to be agreed] are not set yet. The baseline is production data from September 2026.
| ID | Type | Quality | Source and stimulus | Environment | Response | Response measure | Baseline |
|---|---|---|---|---|---|---|---|
| QS-1 | Usage | Security: Confidentiality | Anyone on the internet, or a signed-in agent without the required role, tenant, or context, calls a backend endpoint directly, bypassing the UI. | Production | The request is rejected; no customer data is returned. | 100% of backend endpoints require a signed-in agent and check a feature that exists in the feature model. Exceptions: sign-in, sign-out, error pages, CSP reports, and the external API for ASI. | Not met: the e-mail create endpoint checks no feature (manual check, October 2026). No automated check yet. |
| QS-2 | Usage | Security: Accountability | The data protection officer asks who changed a customer’s data. | Up to [to be agreed] months after the change | The changes are listed with the agent and the time of each change. | 100% of changes are attributable to an agent; the answer is available within [to be agreed] working days. | Not measured. Changes are tracked with the agent identity (Logging). |
| QS-3 | Usage | Reliability: Availability | Agents use KBP. | Production, working hours (06:00–22:30) | KBP answers requests. | Available for at least 99.5% of working hours per calendar month, measured by the availability test. | 100% over the last 90 days. |
| QS-4 | Change | Reliability: Availability | Operations deploys a new release. | Production, working hours | The release is swapped in from the staging slot while agents keep working. | 0 failed availability probes and no rise in failed requests during the swap. | Not measured. |
| QS-5 | Usage | Performance efficiency: Time behaviour | An agent opens a page. | Production, normal load in working hours | The page is ready for input. | Within 3 s for 95% of page loads over 30 days, for every page (pageLoadTime_* metrics). |
Overall p95 2.6 s; Kontodaten-Übersicht p95 3.6 s misses the target. |
| QS-6 | Failure | Reliability: Fault tolerance | One backend system (e.g. APA) becomes unavailable. | Production, working hours | Features that do not depend on that backend keep working; the affected features show a message instead of an error page. | 0 failures in features that do not depend on the failed backend. | Not measured; no outage drill yet. |
| QS-7 | Failure | Reliability: Fault tolerance | A backend call fails transiently (timeout, HTTP 5xx). | Production, normal operation | The call is retried before the agent sees an error. | At most [to be agreed] % of calls per backend fail per month. | At most 0.11% per backend (dependency failure rate). |
| QS-8 | Change | Compatibility: Interoperability | A backend system releases a new API version. | Development | KBP is adapted to the new version. | Code changes stay within that backend’s client in the infrastructure layer; effort at most [to be agreed] person-days. | Not measured. |
| QS-9 | Usage | Usability: Operability | A newly trained agent performs a tariff change for the first time. | Production, live customer call | The agent completes the task without help. | Within [to be agreed] minutes; less than [to be agreed] % of these journeys are abandoned. | Not evaluated. Interaction tracking records time to action and abandoned journeys. |
| QS-10 | Usage | Usability: User error protection | An agent enters invalid data (e.g. an IBAN or address). | Production, live customer call | The error is shown at the field and the entered data is kept. | Less than [to be agreed] % of backend rejections are caused by invalid input. | Not measured. |
| Quality Criteria | Description |
|---|