The quality of autism intervention studies must improve to ensure validity

Last updated 18 December 2023

The number of studies focusing on early detection of autism spectrum disorder and interventions has increased over recent years. Ensuring consistency between studies and finding a consensus as to the most effective intervention strategies, however, remains a challenge. Earlier this year, Jonathan Green and Shruti Garg compiled an Annual Research Review on the state of autism intervention science for the Journal of Child Psychology and Psychiatry. Here, the researchers discuss their key findings and outline where progress needs to be made.

The past decade has seen a rise in the number of international reviews of autism spectrum disorder (ASD) interventions, but confusingly, many of these reviews come to different conclusions or only assess selected forms of intervention. Such discrepancies derive, in part, from the review of different selections of small and/or poor quality studies, and as such, the results can be a hindrance rather than a help to practitioners. “To address these issues, we conducted a wide-ranging review based solely on source papers that were of a reasonable size and quality”, describe Green and Garg. “In doing so, we built on the very valuable 2018 systematic review by French and Kennedy (also published in the Journal of Child Psychology and Psychiatry) that independently looked at the quality of methods used in trials for early intervention in ASD”.

A key finding of Green and Garg’s review was that the quality of research into autism intervention is, unfortunately, often poor. This finding was also highlighted by French and Kennedy, who found that only 6/48 studies reviewed met reasonable quality criteria. “Many of the treatments in common use around the world for autism (for example, behavioural interventions such as Applied Behaviour Analysis or Early Intensive Behavioural Intervention) have no robust evidence base for their effectiveness as judged by accepted healthcare standards”, explain Green and Garg. “This finding is really worrying in a condition as important as autism”.

The researchers looked at intervention targets across psychosocial domains and neurological processes, to clarify exactly what kind of effect each intervention is having in relation to ASD development at the mechanistic level.

In terms of psychosocial treatments, the researchers found that most only targeted immediate social interactions, such as those between an affected child and their caregiver. Fewer studies assessed the evidence for generalising such treatments into other contexts or everyday functioning, and only a handful looked at how treatment effects were sustained over time. “We argue that if an intervention is truly to be considered a ‘treatment for autism’, it needs to demonstrate evidence of an effect on autism-specific outcomes in a more generalised way across contexts and time: many claims of ‘effectiveness’ don’t do this”, state Green and Garg. “We did identify, however, that some interventions (mainly those involving social communication) are beginning to show a sustained effect over time, for instance on reducing the severity of autism symptoms when measured in a standardised way”.

Regarding neurological mechanisms, the researchers reviewed a range of new studies aiming to find biological treatments for the underlying neurodevelopmental difficulties experienced in autism. “To the best of our knowledge, this is the first time such an analysis of biological interventions for ASD has been performed”, say the researchers. “The information gathered here gives us a good vantage point to consider neurodevelopmental intervention for autism in an integrated way”.

The researchers found that the number of interventions based on neurological targets (such as neurotransmitters) is increasing and studies of these interventions have promise to inform about the underlying causes of autism. However, as none of these interventions have yet shown efficacy in large-scale trials, they cannot be implemented in health practice.

The researchers expect that continued work on neurological and neurodevelopmental targets for ASD intervention will ultimately help improve neural functioning in affected patients. “We also expect to learn a lot more in the years to come as to how targeted psychosocial interventions given at the right time can impact brain development”, suggest Green and Garg. “The notion of ‘brain plasticity’ is commonly invoked in ASD intervention studies, but as yet, this concept has not been studied in depth – we will likely find both exciting possibilities but also limitations here”.

This Annual Research Review emphasized that many questions remain to be answered before evidence-based practice in ASD can move forward. Although practitioners are getting an idea of the long-term effects of psychosocial treatments in ASD, understanding their eventual limits for improvement is lacking. In addition, there are many other interventions promoted for ASD that have not yet been properly tested in rigorous trials. “We do not yet know which interventions that act at the neural developmental level may improve outcomes in ASD; initial progress has been made, but there remains much work to do”, say Green and Garg. “We hope that it may be possible to combine the interventions that target the brain and cognition with those that target social communication and the environment: this would be an exciting next step”.

An important consideration that the researchers highlight is that autism is an enduring developmental condition; as such, a “one-off” intervention provided at any point in development is unlikely to be sufficient. Consequently, the researchers propose that an evidenced programme of sequenced interventions delivered throughout childhood development, at the right time and according to need, is required to optimise desired developmental outcomes. “Testing combinations of interventions over time is a major challenge”, admit Green and Garg. “But building up this type of evidence step-by-step will ultimately allow us to produce transformative change for autistic children and their families”.

Green and Garg are also actively researching the effectiveness of novel interventions for different groups of patients affected by ASD. “We are currently working to further adapt and implement our psychosocial intervention known as PACT, into low and middle-income countries”, describe Green and Garg. “We have already done quite a lot of work in this area and are now starting a big scale-up study called COMPASS, together with our colleagues in India”.  In addition, the researchers are testing the effects of biological interventions (including statins) in those with syndromic autism caused by defects in the gene neurofibromatosis 1 (Nf1). Other work includes further analysis of an early infancy intervention for babies with high likelihood of autism, previously described by Green et al., in the Journal of Child Psychology and Psychiatry in 2017. “We hope that data from this research project will give us a good sense as to whether intervening in ASD very early in development really adds value”, explains Green. “If it does, there could be major public-health implications as to where we focus our ASD intervention efforts”. Green notes, however, that this does not imply a sole focus on earliest intervention measures, as evidence shows that interventions can also be effective later in childhood and will always be needed in some form throughout development.

In summary, Green and Garg found that many claims of effectiveness for particular ASD interventions are based on poor quality evidence – for instance deriving conclusions from small numbers of participants, or non-randomised or otherwise non-rigorously designed trials.  “This issue matters because we know that the results from studies of that kind aren’t always reliable and often don’t replicate”, say Green and Garg. “The literature shows that it is quite possible to perform a good quality randomised controlled trial in this area, and that such trials are helpful in making progress”.

The researchers hope that the future will see a move towards much larger intervention studies that will take the common, effective elements of interventions analysed in small, initial trials and test them on larger adolescent populations throughout their childhood development. Such studies will help identify some of the factors that moderate treatment success in different patient groups and will move the field towards more personalised treatment plans in ASD. In addition, the researchers hope that the level of dialogue between professionals, clinicians, families and the autism community about what autism interventions should be aiming for will increase. Improving this level of communication between all parties should positively affect the kind of outcomes analyzed in intervention studies.

Referring to: Green, J. & Garg, S.  (2018), Annual Research Review: The state of autism intervention science: progress, target psychological and biological mechanisms and future prospects. J Child Psychol Psychiatr, 59: 424-443. doi:10.1111/jcpp.12892

Further reading:

Stivaros, S. et al. (2018), Randomised controlled trial of simvastatin treatment for autism in young children with neurofibromatosis type 1 (SANTA). Mol Autism, 9:12. doi:10.1186/s13229-018-0190-z

Green, J. et al. (2017), Randomised trial of a parent-mediated intervention for infants at high risk for autism: longitudinal outcomes to age 3 years. J Child Psychol Psychiatr, 58: 1330-1340. doi:10.1111/jcpp.12728

French, L. & Kennedy, E.M.M. (2018), Annual Research Review: Early intervention for infants and young children with, or at-risk of, autism spectrum disorder: a systematic review. J Child Psychol Psychiatr, 59:444-456. doi: 10.1111/jcpp.12828

Pickles, A. et al. (2016) Parent-mediated social communication therapy for young children with autism (PACT): long-term follow-up of a randomised controlled trial.  Lancet, 388:2501-2509


Dr Jessica Edwards
Jessica received her MA in Biological Sciences and her DPhil in Neurobehavioural Genetics from the University of Oxford (Magdalen College). After completing her post-doctoral research, she moved into scientific editing and publishing, first working for Spandidos Publications (London, UK) and then moving to Nature Publishing Group. Jessica is now a freelance editor and science writer, and started writing for “The Bridge” in December 2017.

Add a comment

Your email address will not be published. Required fields are marked *