Data infrastructure R&D sits in storage, query and streaming guarantees, not in pipelines built from known tools
Data infrastructure companies can claim the R&D Tax Incentive where a storage, query or streaming design must meet a latency, correctness or cost target whose outcome published designs cannot predict for that data, and experiments settle it. Pipelines assembled from managed tools, migrations and a company's own reporting warehouse are not core.
Reviewed by George WalchFacts last verified
Where the R&D usually sits
The unknowns are in guarantees under load: whether an encoding or index meets a query latency and storage budget on a data shape published formats were not tested on; whether change data capture can deliver exactly once into several sinks through failover without breaking order; whether a vector index holds recall at a memory ceiling across thousands of tenants. The competent professional is an experienced database or distributed systems engineer who reads the database research literature and the design documents of the major open-source engines.
What usually qualifies
Data infrastructure activities and how they usually fare
Fact
Usually
Why
Designing an encoding or index that meets latency and storage targets on a data shape no published format handles
Likely core
Performance on that data cannot be derived from published results
Delivering change data capture exactly once into several sinks through failover
Often core
Core where ordering and duplication behaviour under failure is genuinely unproven
Generating synthetic datasets and replay rigs at production scale for those experiments
Supporting
Directly enables the core measurements
Standing up pipelines with managed warehouse and orchestration tools per their documentation
No
Not core; known tools, predictable result. Pipelines built only to feed a core experiment can be supporting
Migrating data between platforms with established tools
No
Usually not core; listed as routine in AusIndustry's software guide
Building management reporting and dashboards over the company's own data
No
Excluded from core as internal administration software (s 355-25(2)(h)); supporting only where it directly serves a core activity
Benchmarking vendors or tuning configuration flags to pick a setup
No
Usually not core; choosing between known options. A comparison designed to test a stated unknown can be part of an experiment
Illustrative example
An illustrative vector search company's index claim
A hypothetical vector database company hypothesised that product quantisation with a learned codebook per tenant would hold recall at 10 of at least 0.95 under 20 ms p99 for 500 million vectors on nodes with 64 GB of memory, across 5,000 tenants of very different sizes. Its review of published approximate nearest neighbour work found no benchmark at that tenant count, and, more to the point, nothing that showed how recall for small tenants behaves when codebooks compete for a fixed memory budget. A shared codebook, tried first, lost recall on small tenants. The index experiment is the core activity, and the synthetic multi-tenant corpus and load rig form one supporting activity. Its infrastructure-as-code modules, billing dashboards and warehouse connectors are not claimed. This is an illustration, not a client.
The evidence you already have
Data teams already keep the raw material: benchmark scripts in version control, query plans, load test reports with p50 and p99 latency, storage and cost metrics per run, incident reviews for consistency failures, and design documents comparing approaches. Two habits make it claimable: pin the dataset and hardware for each run so results are comparable, and write the target and the reason no published system meets it before the first run.
Watch outs
The internal administration exclusion is the trap specific to data work: AusIndustry names information management and management information systems as internal administration, so a warehouse built for your own reporting is excluded as core however hard it was. Vendor bake-offs are usually choosing, not experimenting. Configuration tuning on a managed service is routine. And whole-platform claims ("the new data platform") fail the per-activity test.
This page serves two kinds of company, and Division 355 treats them differently.
A company that sells a database, streaming platform, storage engine or observability product is building software for unconnected customers. Its hard problems (latency, durability, consistency, cost at scale) are product questions, and the experimental ones can be core R&D activities.
A company whose data team builds a warehouse, pipelines and dashboards to run the business is in a different position. Section 355-25(2)(h) excludes software developed for the dominant purpose of the company's own internal administration, and AusIndustry's examples include information management and management information systems. A reporting stack that tells management how the business is going is that, however much engineering it took. AusIndustry notes such work may still be supporting R&D where it directly serves a core activity, but it cannot be the core. See internal administration software.
Pipelines that are part of what customers buy are not internal administration. The deciding fact is who the software is for.
Is it really unknown?
Three tests come up again and again in data claims.
Published systems first. The research literature and the design documents of open-source engines are where a competent professional would look. If a documented design already meets your target on comparable data, the work is applying it. Record what you checked before you started.
Unpublished cuts both ways. Knowledge must be publicly available or reasonably accessible. A closed-source vendor's internals are neither, so their private solution does not close the gap. Nor does a missing benchmark at your exact scale open one: if published designs let a competent engineer predict the result, the work is applying them.
New data shapes can be a real unknown. Moreton Resources held that applying an existing technology in a new context can be experimental if the outcome could not be known. A known index on a very different distribution or tenancy pattern can qualify, but only with a stated hypothesis about why published results do not transfer.
Cloud spend and the claim
Benchmark and replay clusters can make compute a material R&D cost for a data company. The share used for registered experiments can be claimed; production capacity serving customers cannot, even when an experiment runs alongside it. Separate accounts, projects or tags for experiment workloads turn that share into a record rather than an estimate. More on cloud compute costs.
How data registrations usually split
One core activity per guarantee being tested (an index's recall and memory frontier, a delivery semantic under failover, a compaction strategy's read latency).
Supporting activities for synthetic data generation, replay rigs and benchmark harnesses.
Left out: connectors, managed-service configuration, migrations, dashboards and the company's own reporting stack.
Name each core activity for the guarantee, not the component: "holding recall above 0.95 at a fixed memory ceiling across uneven tenants" tells an AusIndustry reviewer what was unknown, where "vector index v2" does not.
For pipeline-specific questions, see data pipelines and distributed systems.
Frequently asked questions
A closed-source competitor already does this. Does that rule us out?
Not by itself. The test is against knowledge that is publicly available or reasonably accessible worldwide, and a competitor's unpublished internals are not. But their silence does not create a gap either: if a competent engineer could work out the answer from published designs, there is no gap. If the competitor published how, that changes the answer too.
Is moving our platform to a lakehouse architecture R&D?
Usually not. Re-platforming onto documented tools is routine, and AusIndustry lists migrating data with known tools as an example. A specific question inside the move can still qualify, for example whether a new compaction strategy holds read latency on your write pattern.
Can we claim the cloud bill for our benchmark cluster?
The part used to run registered experiments can be R&D expenditure. Keep experiment workloads on separately tagged accounts or clusters so the share is evidence rather than an estimate, and do not count production capacity that serves customers.
The R&D Tax Incentive is a self-assessment program. This page is general information, not tax, legal or financial advice; eligibility depends on your circumstances and you should seek independent advice for them.
