Seen the “98% of studies were ignored!” one doing the rounds on social media. The editorial in the BMJ put it in much better terms:

“One emerging criticism of the Cass review is that it set the methodological bar too high for research to be included in its analysis and discarded too many studies on the basis of quality. In fact, the reality is different: studies in gender medicine fall woefully short in terms of methodological rigour; the methodological bar for gender medicine studies was set too low, generating research findings that are therefore hard to interpret.”

  • streetlights@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    2
    ·
    3 months ago

    It’s not used for “dismissal” it’s used to score studies on their likelihood of bias. Studies without appropriate controls for example are more susceptible to bias than those with.

    98% of the data was dismissed in the synthesis

    Demonstrably false, only low quality studies were excluded from the synthesis which account for less than half of the 103 reviewed. A lie is a lie no matter how often repeated.

    and were not used in the conclusion that there wasn’t enough scientific evidence to support transition when 98% of the science says that is wrong.

    That’s not what the conclusions say, for example:

    Synthesis of moderate-quality and high-quality studies showed consistent evidence demonstrating efficacy for suppressing puberty

    And

    Evidence from mainly pre–post studies with 12-month follow-up showed improvements in psychological outcomes

    “They dismissed 98% of data” remains a lie.

      • streetlights@lemmy.worldOP
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        1
        ·
        3 months ago

        That was published a month before Cass came out and so hasn’t anything to do with the two systematic reviews being discussed above. It doesn’t even mention them.

        I’m uncertain what expertise a business graduate can bring to assessing the quality of a systematic review in medicine.

        Readers are free to Google the author and subsequently make a judgement on their objectivity on the subject matter.

        • Cogency@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          3 months ago

          And yet you have no scientific reason other than an ad hominem fallacy with the author with which to dismiss the criticism with. That like the Cass report are not scientifically sufficient reasons to disclude the criticism or the data respectively.

          And I can garuntee you that the Cass report was not peer reviewed like all of the studies they dismissed were because it would have been torn apart. That’s the real litmus test of scientific debate.

          • streetlights@lemmy.worldOP
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            1
            ·
            3 months ago

            And yet you have no scientific reason other than an ad hominem fallacy with the author with which to dismiss the criticism with.

            If they made a scientific argument about these review papers under discussion I might but this is just a polemic using unscientific language like “cis-supremacy” in a low impact obscure journal.

            That like the Cass report are not scientifically sufficient reasons to disclude the criticism or the data respectively.

            Newcastle-Ottawa scoring is a scientific method for weighting the methodical quality of scientific studies.

            And I can garuntee you that the Cass report was not peer reviewed like all of the studies they dismissed were because it would have been torn apart.

            It was peer reviewed since thats BMJ policy, unless you have evidence to the contrary. There is even a link on the online edition of both reviews for you to submit a rapid response pointing out all their flaws which I would encourage you to do.

            That’s the real litmus test of scientific debate.

            Interestingly some nice fellow DM’d me with a link to “Patient Zero” of the “they dismissed 98% of the data” myth.

            https://twitter.com/benryanwriter/status/1779671152148857212

            And of course, everyone has doubled down rather than admit they read the wrong paper. A better “litmus test” of scientific debate is humbly correcting yourself when shown to be wrong.

            “They dismissed 98% of the data” remains a lie.

            • Cogency@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              edit-2
              3 months ago

              98% of the data could be summarized in one sentence. Trans healthcare and hrt works. 98% of the data comes to that conclusion with vast consensus across multi disciplines and fields comes to that conclusion and that was ignored. 98% of the data was discarded. Most of those studies discarded already had a statistical analysis backing up their efficacy while the Cass report doesn’t. Nor does the Cass report include a nearly mandatory implicit bias report.

              Those peer reviews are most likely selected and not randomized selections or contestations as most peer reviews are required to be, they are ok for initial release irc. But it is an outgoing process that doesn’t have an endpoint. They were most likely provided prior to release and the normal peer review process won’t be completed for years to undo the damage. But it is not considered peer reviewed yet.

              Again you have not proven that the new castle Ottawa scale has any efficacy or scientific merit as a disqualifying tool No one has as far as I know.

              • streetlights@lemmy.worldOP
                link
                fedilink
                English
                arrow-up
                1
                ·
                edit-2
                3 months ago

                98% of the data was discarded

                Liar.

                Those peer reviews are most likely selected and not randomized selections or contestations

                So now they were peer reviewed but by people you don’t trust based on the same evidence you used to assert it wasn’t peer reviewed in the first place I.e. zero.

                Again you have not proven that the new castle Ottawa scale has any efficacy or scientific merit as a disqualifying tool No one has as far as I know.

                You’d better tell the Cohcrane library to bin every systematic review they’ve ever done which used this system then. I’d be eager to hear their reply to you

                • Cogency@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  edit-2
                  3 months ago

                  98% of the data was discarded

                  No I’m just explaining the process and why it isn’t complete yet. Or even valid yet

                  And show me that the Cochrane library ever discarded a study using the criteria even once yet alone with the same level as the Cass report and I’ll write them

                  For something that illustrates the problem with the Cass report read https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/

                  • streetlights@lemmy.worldOP
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    ·
                    edit-2
                    3 months ago

                    98% of the data was discarded

                    It was not. All studies that scored high or moderate quality made it into the synthesis. That’s 60 out of the 103 looked at, that’s not 2%.

                    No I’m just explaining the process and why it isn’t complete yet. Or even valid yet

                    You are speculating, based on nothing.

                    And show me that the Cochrane library ever discarded a study using the criteria even once yet alone with the same level as the Cass report and I’ll write them

                    Here’s one I found in <7 seconds

                    https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013256.pub2/full

                    If you want to find more simply search the Cochrane library for reviews with “Newcastle-Ottawa” in the main body of text. It seems like this is new to you.

                    For something that illustrates the problem with the Cass report read https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/

                    The relevance of a joke paper from 2003 to a systematic review published last week is certainly questionable but it seeks you’re trying to imply that Cass discarded anything except RCT’s. The didn’t and that’'s myth #2 from the original Quackometer article.

                    Where will the goalposts move next?