Distributed systems is a field with a deep published literature and excellent open-source implementations. That cuts both ways for the R&D Tax Incentive. The literature defines what a competent engineer already knows, so most adoption work is routine. It also marks clearly where knowledge stops, which is where a genuine core activity can begin.
What counts as already known
A competent distributed systems engineer is taken to know the published consensus protocols and their proofs, the documented semantics of mainstream brokers and databases, the consistency models and their trade-offs, and the behaviour of mature replication and CRDT libraries. Using any of these as documented is applying existing knowledge. So is putting services on an orchestrator, adding a service mesh or sharding a database along a documented key. The ATO's software alert TA 2017/5 lists using existing technologies as they were intended to be used among the marks of an ineligible claim.
Where the unknowns are
In Rand's reading of AusIndustry's core criteria, a typical core shape is getting components to interoperate when their combined behaviour cannot be known in advance. In distributed work that usually means behaviour under conditions the published evaluations did not test:
- Real networks. Loss, asymmetric latency, intermittent links and long-haul routes rather than a data-centre LAN.
- Specific failure combinations. Partitions during leader change, clock skew during recovery, correlated device failures.
- Scale and topology together. Tens of thousands of participants, or a geography the protocol was not designed for.
The question is whether the outcome could be predicted, not whether it was hard to build. If a competent engineer could work a property out from known theory with standard analysis, it was knowable in advance. That does not rule out analysis as a method: AusIndustry says experiments take many forms, and model checking or simulating a new design can be how the unknown is resolved. But models rest on assumptions about the network, so measured behaviour under real conditions is often where a genuine core R&D activity remains.
Scope it narrowly. "Our replication layer" is a component, not a knowledge gap. One activity per unanswered coordination question, with each run's result shaping the next, reads as an experiment and keeps the routine build work around it out of the core.
Edge cases peculiar to distributed work
- Chaos engineering. In Active Sports Management [2023] AATA 4078 the Tribunal found trial and error without isolable variables is not a repeatable experiment. Terminating instances to see what happens is that. A planned design is not, whether it changes one condition at a time or randomises several, as long as the conditions are recorded and results can be attributed to them.
- Formal specifications. Whether a TLA+ or similar specification is supporting work, the experiment itself or routine depends on what it was written to find out; the FAQ below sets out the three cases.
- Consistency bugs. Finding and fixing a race in your own code is usually debugging, even when it takes weeks.
- Re-platforming. Moving onto managed services or containers is covered on the cloud migration page.
- Pure throughput limits. When the question is how fast one component can go rather than how nodes coordinate, see performance and scale engineering.
Companies whose product is the infrastructure itself will find more on scoping activities on the data infrastructure page. Pull requests whose descriptions record the protocol question and each result are usually the best record; the GitHub integration page explains what Rand reads from them.