• Socsa@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    57
    arrow-down
    4
    ·
    3 months ago

    Bayesian purist cope and seeth.

    Most machine learning is closer to universal function approximation via autodifferentiation. Backpropagation just lets you create numerical models with insane parameter dimensionality.

      • hotsox@lemmy.blahaj.zone
        link
        fedilink
        English
        arrow-up
        23
        arrow-down
        5
        ·
        edit-2
        3 months ago

        Universal function approximation - neural networks.

        Auto-differentiation - algorithmic calculation of partial derivatives (aka gradients)

        Backpropagation - when using a neural network (or most ML algorithms actually), you find the difference between model prediction and original labels. And the difference is sent back as gradients (of the loss function)

        Parameter dimensionality - the “neurons” in the neural network, ie, the weight matrices.

        If thats your argument, its worse than Statistics imo. Atleast statistics have solid theorems and proofs (albeit in very controlled distributions). All DL has right now is a bunch of papers published most often by large tech companies which may/may not work for the problem you’re working on.

        Universal function approximation theorem is pretty dope tho. Im not saying ML isn’t interesting, some part of it is but most of it is meh. It’s fine.

  • The Snark Urge@lemmy.world
    link
    fedilink
    English
    arrow-up
    40
    arrow-down
    2
    ·
    3 months ago

    Eh. Even heat is a statistical phenomenon, at some reference frame or another. I’ve developed model-dependent apathy.

  • FaceDeer@fedia.io
    link
    fedilink
    arrow-up
    65
    arrow-down
    36
    ·
    3 months ago

    The meme would work just the same with the “machine learning” label replaced with “human cognition.”

    • wizardbeard@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      84
      arrow-down
      10
      ·
      3 months ago

      Have to say that I love how this idea congealed into “popular fact” as soon as peoples paychecks started relying on massive investor buy in to LLMs.

      I have a hard time believing that anyone truly convinced that humans operate as stochastic parrots or statistical analysis engines has any significant experience interacting with others human beings.

      Less dismissively, are there any studies that actually support this concept?

      • essell@lemmy.world
        link
        fedilink
        English
        arrow-up
        51
        arrow-down
        2
        ·
        edit-2
        3 months ago

        Speaking as someone whose professional life depends on an understanding of human thoughts, feelings and sensations, I can’t help but have an opinion on this.

        To offer an illustrative example

        When I’m writing feedback for my students, which is a repetitive task with individual elements, it’s original and different every time.

        And yet, anyone reading it would soon learn to recognise my style same as they could learn to recognise someone else’s or how many people have learned to spot text written by AI already.

        I think it’s fair to say that this is because we do have a similar system for creating text especially in response to a given prompt, just like these things called AI. This is why people who read a lot develop their writing skills and style.

        But, really significant, that’s not all I have. There’s so much more than that going on in a person.

        So you’re both right in a way I’d say. This is how humans develop their individual style of expression, through data collection and stochastic methods, happening outside of awareness. As you suggest, just because humans can do this doesn’t mean the two structures are the same.

        • darthelmet@lemmy.world
          link
          fedilink
          English
          arrow-up
          20
          arrow-down
          3
          ·
          3 months ago

          Idk. There’s something going on in how humans learn which is probably fundamentally different from current ML models.

          Sure, humans learn from observing their environments, but they generally don’t need millions of examples to figure something out. They’ve got some kind of heuristics or other ways of learning things that lets them understand many things after seeing them just a few times or even once.

          Most of the progress in ML models in recent years has been the discovery that you can get massive improvements with current models by just feeding them more and data. Essentially brute force. But there’s a limit to that, either because there might be a theoretical point where the gains stop, or the more practical issue of only having so much data and compute resources.

          There’s almost certainly going to need to be some kind of breakthrough before we’re able to get meaningful further than we are now, let alone matching up to human cognition.

          At least, that’s how I understand it from the classes I took in grad school. I’m not an expert by any means.

          • Match!!@pawb.social
            link
            fedilink
            English
            arrow-up
            13
            ·
            3 months ago

            I would say that what humans do to learn has some elements of some machine learning approaches (Naive Bayes classifier comes to mind) on an unconscious level, but humans have a wild mix of different approaches to learning and even a single human employs many ways of capturing knowledge, and also, the imperfect and messy ways that humans capture and store knowledge is a critical feature of humanness.

          • oce 🐆@jlai.lu
            link
            fedilink
            English
            arrow-up
            6
            ·
            3 months ago

            I think we have to at least add the capacity to create links that were not learned through reasoning.

          • CeeBee_Eh@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            arrow-down
            2
            ·
            3 months ago

            The difference in people is that our brains are continuously learning and LLMs are a static state model after being trained. To take your example about brute forcing more data, we’ve been doing that the second we were born. Every moment of every second we’ve had sound, light, taste, noises, feelings, etc, bombarding us nonstop. And our brains have astonishing storage capacity. AND our neurons function as both memory and processor (a holy grail in computing).

            Sure, we have a ton of advantages on the hardware/wetware side of things. Okay, and technically the data-side also, but the idea of us learning from fewer examples isn’t exactly right. Even a 5 year old child has “trained” far longer than probably all other major LLMs being used right now combined.

        • CeeBee_Eh@lemmy.world
          link
          fedilink
          English
          arrow-up
          7
          arrow-down
          2
          ·
          3 months ago

          The big difference between people and LLMs is that an LLM is static. It goes through a learning (training) phase as a singular event. Then going forward it’s locked into that state with no additional learning.

          A person is constantly learning. Every moment of every second we have a ton of input feeding into our brains as well as a feedback loop within the mind itself. This creates an incredibly unique system that has never yet been replicated by computers. It makes our brains a dynamic engine as opposed to the static and locked state of an LLM.