Product Analytics and System Monitoring

Context & Background

The Digital Lock On system requires visibility into user behaviour and system performance to ensure the application meets operational needs and maintains reliability in safety-critical rail corridor environments.

Without product analytics, the team is unable to validate whether features are being used as intended, where users encounter friction, or how field conditions affect workflows. Without system monitoring, performance issues and errors may go undetected, potentially impacting safety-critical operations.

This requirement addresses the need for:

• Performance monitoring to ensure backend services remain responsive and reliable
• Error tracking to proactively identify and resolve issues before they impact users
• Usage insights to inform product decisions and validate development priorities

Given the offline capability, monitoring must account for intermittent connectivity, and delayed data synchronisation.

Application Architecture Context

The Digital Lock On PWA uses a modal-heavy interface where most user actions occur within modals and overlays rather than through traditional page navigation. Users typically remain on a single page (e.g., worksite view) while performing multiple workflows through modal dialogs.

This architectural pattern requires analytics to focus on:

• Event-based tracking rather than pageview-centric analytics
• Modal workflow tracking to understand user journeys within a single page context
• Interaction events (button clicks, form submissions, QR scans) as primary engagement metrics
• Frontend event logging as the primary source of product insights, given most user interactions occur client-side

Analytics implementation must account for:

• Minimal URL changes despite multiple user interactions
• Multiple workflows initiated and completed from a single page view
• Modal open/close/completion events as key funnel stages
• Frontend-heavy event generation, with backend events primarily for technical monitoring

Related Decisions

Architecture Decisions

• [ ] ADR needed: Analytics and monitoring tool selection
• [ ] ADR needed: Analytics event taxonomy and naming conventions
• [ ] ADR needed: PII handling in analytics and monitoring systems

Pending Decisions

• [ ] Pending: Data retention periods for analytics and monitoring data
• [ ] Pending: Alert notification channels and escalation procedures

User Story

As a product and development team managing the Digital Lock On application

I want product analytics and performance monitoring

So that I can understand how users interact with the system, detect performance issues proactively, and make data-driven decisions about future development while ensuring system reliability for safety-critical operations.

Business Rules

1. Analytics and monitoring SHALL NOT interfere with application performance or block user workflows
2. Data collection SHALL function both online and offline, with queued data syncing when connectivity returns
3. Personal identifiable information (PII) SHALL be handled in accordance with privacy policies and shall not be collected unnecessarily
4. The team SHALL be alerted to critical errors, performance degradation, and system availability issues
5. Analytics SHALL track user journeys through critical workflows including modal-based interactions
6. Performance monitoring SHALL track both frontend user experience and backend API performance
7. Implementation SHALL be maintainable without requiring specialist analytics or DevOps engineers
8. Modal workflows SHALL be tracked as discrete events with lifecycle visibility (open, interact, complete, abandon)

Acceptance Criteria

Event-Driven Requirements

When a user completes a critical workflow (worker lock-on, protection activation, handover), the system SHALL capture analytics events with relevant context.

When a user opens a modal dialog, the system SHALL capture the modal type, trigger source, and relevant context.

When a user completes or abandons a modal workflow, the system SHALL capture the outcome and duration.

When a user encounters an error or exception, the system SHALL capture the error details, and context leading to the error.

When an API request is processed, the system SHALL automatically log performance metrics (response time, status, dependencies called).

When the PWA loses network connectivity, the system SHALL queue analytics data locally without blocking user workflows.

When network connectivity is restored, the system SHALL synchronise queued analytics and monitoring data.

When system performance degrades beyond defined thresholds, the system SHALL alert the development team.

When critical errors occur, the system SHALL notify the team with sufficient context for diagnosis and resolution.

State-Driven Requirements

While the application is running, the system SHALL automatically track user sessions and interactions.

While a user remains on a single page view, the system SHALL track modal interactions and workflow progressions.

While backend services are operating, the system SHALL continuously monitor API performance, database query times, and dependency health.

While offline, the system SHALL continue collecting analytics data without degrading application performance.

While users interact with the application, the system SHALL maintain user identity context across analytics events.

Ubiquitous Requirements

The system SHALL provide visibility into:

• Feature usage and adoption rates
• User workflow completion and abandonment
• Session duration and engagement patterns
• API endpoint performance and failure rates
• Database query performance
• Exception and error rates
• System availability and uptime

The system SHALL respect user privacy by:

• Minimising collection of personal information
• Redacting sensitive data from recordings and logs
• Adhering to applicable privacy regulations

Conditional Requirements

Where multiple users access the same worksite, the system SHALL track collaborative workflows.

Where offline operation results in delayed data synchronisation, the system SHALL track sync performance.

Unwanted Behaviour

If analytics or monitoring services are unavailable, then the system SHALL NOT block user workflows or degrade application functionality.

If data collection fails, then the system SHALL fail silently and log errors without alerting users.

If monitoring systems detect issues, then alerts SHALL NOT generate false positives or alert fatigue.

Key Areas to Monitor

Product Analytics (Frontend/User Behaviour)

Critical User Journeys:

• Protection activation and deactivation flows
• Worker lock-on and lock-off processes (+ manual and force lock-off)
• Protector shift handover workflows
• Plant lock-on and handover operations
• Manual sign-on and on-the-fly onboarding

Usage Metrics:

• Active users (daily, weekly, monthly)
• Session duration and frequency
• Feature adoption and usage rates
• Workflow completion and abandonment rates
• Error rates

User Experience:

• Page load times and interaction responsiveness
• Navigation patterns and user journeys
• Offline operation frequency and duration
• Points of user confusion or friction

System Performance Monitoring (Backend/Infrastructure)

API Performance:

• Endpoint response times (percentiles: p50, p95, p99)
• Request volume and throughput
• Success vs failure rates by endpoint
• Dependency call performance (database, external services)

Database Performance:

• Query execution times
• Slow query identification
• Timeout occurrences

System Health:

• Service availability and uptime
• Resource utilisation (CPU, memory, storage)
• Error and exception rates
• Background job performance (sync, notifications)

Critical Business Operations:

• Handover session success and failure rates
• Heartbeat processing performance
• Offline sync processing times
• QR code generation and email delivery rates

Error Tracking

Application Errors:

• JavaScript exceptions in PWA
• API exceptions and unhandled errors
• Database connection failures
• Integration failures (email, external services)

User Impact:

• Number of users affected by errors
• Error frequency and patterns

Performance Requirements

Response Times

• Analytics event capture SHALL NOT add noticeable delays to user interactions
• Monitoring telemetry SHALL NOT noticeably impact API response times
• Offline data queuing SHALL be non-blocking

Resource Usage

• Analytics SDK SHALL add minimal size to PWA bundle
• Monitoring instrumentation SHALL add minimal overhead to API processing
• Local event storage SHALL NOT exceed 10MB per device

Scalability

• System SHALL support analytics for 100+ concurrent users
• System SHALL handle 10,000+ events per day
• Monitoring SHALL capture all API requests without sampling (at current scale)

Privacy & Compliance

Data Minimisation

• Collect only data necessary for stated purposes
• Exclude authentication tokens, passwords, and unnecessary PII

User Privacy

• Provide transparency about data collection practices
• Enable data deletion requests if required by policy

Data Retention

• Retention periods should be aligned with operational needs and regulations
• Archive or delete data beyond retention period
• Maintain audit trail of data access and deletion

Dependencies

Requires:

• PWA frontend application
• .NET backend API services
• User authentication system
• Offline storage capabilities
• Azure cloud infrastructure

Enables:

• Data-driven product development decisions
• Proactive issue identification and resolution
• UX optimisation based on real usage patterns
• Performance optimisation
• Feature validation and prioritisation

Linked Specifications

• [ ] To be specified: Frontend analytics implementation (PostHog or similar)
• [ ] To be specified: Backend monitoring implementation (Application Insights or similar)
• [ ] To be specified: Analytics event schema and naming conventions
• [ ] To be specified: Alert rules and notification procedures
• [ ] To be specified: Privacy configuration and data redaction rules

Notes

Phase 1 Priorities

• Core event tracking for critical workflows
• API performance monitoring
• Error and exception tracking
• Basic user identification and session tracking