Glossary

Experiment: the procedure that tests the hypothesis

An experiment is the scientific procedure a core R&D activity runs to test its hypothesis: it varies some parameters, holds others constant and measures the result, so the outcome can be observed and evaluated. Related experiments testing the same idea can be grouped in one core activity. Routine testing that confirms an expected result is not an experiment.

Reviewed by George WalchFacts last verified

Example

For example, an illustrative SaaS team tests three caching strategies against a replay of recorded production traffic at 5,000 requests per second. It holds hardware and data size constant, varies only the eviction policy, and measures p99 latency and cache hit rate. Writing each strategy's code is part of the experiment; hardening the winner for release is not.

Legislative reference: Section 355-25(1), Income Tax Assessment Act 1997 ("experimental activities")

Related terms: hypothesis, systematic progression, core rd activity, supporting rd activity, excluded activities

Section 355-25 defines core R&D activities as "experimental activities". AusIndustry describes the experiment itself as "a scientific procedure that you undertake to test your hypothesis".

What the records should show

For each experiment, AusIndustry expects records of:

  • the parameters you changed;
  • the parameters you held constant;
  • what you observed or measured, and when.

Related experiments that test the same technical or scientific idea can sit under one core activity, which keeps the registration readable. A run log or benchmark output with a timestamp is worth more than a summary written months later, because it shows the test happened as planned. The design comes from the hypothesis: an experiment that cannot confirm or disprove it is not testing it.

What belongs inside the experiment

Where the activity is a genuine core experiment, building the version under test is part of it, to the extent needed to test it. So is running it behind a feature flag in the live product for a study group. Writing code or shipping behind a flag does not by itself make work an experiment; the unknown outcome and the hypothesis do. Comparing variants and analysing results are core work. Environments, test harnesses and data preparation around the experiment can be supporting R&D activities when they are directly related to it. Polishing the winning variant for general release is product work, not experimentation.

What is not an experiment

  • Routine testing. Quality testing, characterisation and imaging that confirm an expected result. See materials testing for where testing becomes experimentation.
  • Routine tuning. Trying available settings when a competent professional could predict that one of them will work. Searching a configuration space can still be an experiment where the outcome could not be known in advance and the search is planned and evaluated.
  • Commercial tests. Measuring conversion or revenue, which is excluded market research; see A/B tests.

Physical builds raise the same question: a prototype is part of the experiment when the build is the test, and supporting when it only enables one (see prototypes and tooling).

Frequently asked questions

Can an A/B test be an experiment for R&D purposes?
It depends on what it measures. A controlled test of a pre-stated hypothesis on a technical or clinical endpoint can be, if the rest of the core test is met. A test measured on conversion, churn or revenue is market testing, which is excluded from core R&D.
Is our QA and regression testing an experiment?
Usually not. It checks that known behaviour still works, so the outcome is expected. It can sometimes be supporting work where it exists to enable a specific core experiment.

Sources

  1. AusIndustry (business.gov.au), Conducting core R&D activities
  2. AusIndustry (business.gov.au), Biotechnology and medical sciences sector guide
  3. Income Tax Assessment Act 1997, s 355-25 (Federal Register of Legislation)

The R&D Tax Incentive is a self-assessment program. This page is general information, not tax, legal or financial advice; eligibility depends on your circumstances and you should seek independent advice for them.