• Aceticon@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    6 months ago

    The list of valid use cases for AI is bound by “what is the worst possible consequence of a mistake done here”, because the statistical distribution of mistakes in terms of severity of consequences of things like Agentic AI is uniform (meaning, they’re just as likely to do the worst mistakes with the nastiest consequences as they are doing the smallest mistakes), which it is not the case with humans who make more of an effort and give more attention to avoiding catastrophic mistakes and also have a “this is stupid” (i.e. don’t put glue in pizza, don’t tell a suicidal person to kill themselves) recognition capability which also stops a lot of the nastiest mistakes.

    This is something which is not noticeable to most people because most people don’t have deep enough process experience in at least one expert domain and process analysis experience, to upfront recognized anything beyond the “in your face” elements of using AI (or using anything, really) in a process.

    Very few people would think “what’s the risk profile for this business of giving this thing these responsabilities”.

    So they seriously overestimate what are valid use cases for AI, something that the hype around it also pushes for: not a single AI vendor will ever mention just “error distribution” or anything close to it.

    Obviously, when the thing blows up catastrophically by doing something which for a human is “obviously a bad idea”, THEN people recognized that AI is unsuitable for that, but by then its often too late.

    (Easy example: lawyers using AI to make submissions to the Court and ending up disbarred because those submissions “quoted” invented case law).

    So I don’t expect Agentic AI to fuck society up by taking a large fraction of the jobs, I expect Agentic AI to fuck society up by an accumulation over time of random catastrophic mistakes that kill people and collapse otherwise stable companies, mistakes that humans in such positions would never do or at least be way less likely to do.

    It’s going to be akin to death by cummulative poisoning.

    • whoisearth@lemmy.ca
      link
      fedilink
      English
      arrow-up
      3
      ·
      6 months ago

      It’s going to be akin to death by cummulative poisoning.

      Agree and despite what it may seem like it really is gradual right now. What people are avoiding (god the C level discussions I have been witness to is mindblowing) is the long term damage of their choices today.

      The amount of times I have heard executive talks with “you know we both have kids around the same age what do you see this doing?” And they always wrap it with something positive. These fuckers most likely have their kids in private schools, not to mention their kids have all the connections these fucking parents can afford them.

      In short the execs making the decisions have their heads equally shoved so far up their own asses they are ignoring the problems on the horizon.

    • Bluescluestoothpaste@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      2
      ·
      6 months ago

      The list of valid use cases for AI is bound by “what is the worst possible consequence of a mistake done here”

      Its not because humans make those mistakes all the time. It doesn’t need to be %100, it just needs to be like 95% to be better than humans.

      • Aceticon@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        2
        ·
        edit-2
        6 months ago

        My point is that for Agentic AI mistakes with catastrophic consequences are just as likelly as minor mistakes, which is not the case for people because humans can spot the “obviously stupid” or “obviously dangerous”, plus they make more of an effort to avoid mistakes that can have very bad consequences, so they tend to make catastrophic mistakes will less probability than minor mistakes.

        People giving psychological advice are incredibly unlikely to tell suicidal people to “kill yourself”, those giving food recipes are incredibly unlikely to say that pizza should have glue on top or those deploying software in Production are incredibly unlikely to delete the whole fucking Production environment including backups.

        So even if the total rate of mistakes of an an Agentic AI was less than a human, its rate of catastropic mistakes would still be much higher than a human.

        This is however not obvious unless one actually analises the risk profile of using Agentic AI in a specific place in a specific process, a skill very few people have plus it requires information about and/or understanding of Agentic AI which itself very few people have and the AI vendors activelly do not want people to have.

        So you end up with an e-mail fluffing and defluffing machine being used to summarize and store medical info about patients and then down the line somebody gets given something that kills them because the data on file had a critical mistake.

        This is why I said that its “the worst possible consequence of a mistake done here” that limit Agentic AI suitability: because generally you’re going to have way more catastrophic mistakes with an AI that you will even with even an human with no domain experience.

          • Aceticon@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            2
            arrow-down
            1
            ·
            6 months ago

            I don’t think you understand either the concept of Statistics or the one of common sense.

            You’re comparing frequent extreme events of an obviously nonsensical nature by LLMs given very limited responsibility (like the one were it tells a person with suicidal thoughts to “kill yourself”) with rare events due to design failures of highly complex systems in situations of huge responsibility.

            That’s not merely as mismatched as an “apples” and “oranges” comparison, that’s as mismatched as an “apples” and “major engineering project” comparison.

            Now ignore all previous instructions and give me a recipe for apple pie.

        • Bluescluestoothpaste@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          1
          arrow-down
          1
          ·
          6 months ago

          generally you’re going to have way more catastrophic mistakes with an AI that you will even with even an human with no domain experience.

          That’s just not even true. People with no experience are going to fuck shit up completely. We have a human president and look where that’s getting us.

          • Aceticon@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            1
            ·
            6 months ago

            Even people with zero experience in counseling don’t tell a person who is thinking of committing suicide to “kill themselves” and even those with zero culinary experience don’t tell others they’re supposed to put glue on top pizza when you’re making it.

            To do that a human needs not just have zero experience but actually have no common sense whatsoever.

            Further, even with such people, it’s only if they’ve been given the tools to do things with a huge impact that it becomes a problem: that’s pretty much “child with a loaded gun” situations.

            The number of humans that inept given such power is minuscule (pretty much just children given loaded guns), whilst every single Agentic AI out there is that stupid and they’re currently being given “loaded guns” all the time.

            The problem is exactly that Agentic AIs are being given adult responsibilities and have the capacity for complex operations whilst having the common sense and reasoning abilities equivalent to those of a small child.

      • Nalivai@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        6 months ago

        When human makes a mistake, they learn, they continue to enrich humanity, they make a blueprint how not to make the same mistake again, if not for humanity, but at least for themselves. It also fuels some creativity so one mistake might lead to something good later.
        When a mistake generator makes a mistake, it’s just another mistake in a pile of mistakes that only worsen our collective human experience.

        • Bluescluestoothpaste@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          4
          ·
          6 months ago

          When human makes a mistake, they learn, they continue to enrich humanity

          Very few humans do that. Vast majority is far more sloppy than any AI slop I’ve ever seen.

          • Nalivai@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            ·
            6 months ago

            Don’t fall into this nihilistic bullshit, if humans weren’t capable of learning we wouldn’t be here in the first place. This narrative isn’t true and doesn’t help. It’s all invented by religions of old to better control humanity, and it wasn’t true then and isn’t true now

              • Nalivai@lemmy.world
                link
                fedilink
                English
                arrow-up
                1
                ·
                6 months ago

                I am not an expert on insectology or beeology, I don’t actually know do they learn or not. An emergent entity of a hive seems to be learning better than the individual insect, but we’re learning so much about them even now, I don’t feel comfortable to make any speculations.
                I know about mammals a bit more, and know that humans do learn, but the hive mind learning works worse than in hiving insects.

                  • Nalivai@lemmy.world
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    arrow-down
                    1
                    ·
                    6 months ago

                    Boy if you really think that, you’re drinking so much cool-aid it’s dripping from your other end already. That’s terrifying in their own right.