656
you are viewing a single comment's thread
view the rest of the comments
[-] V0ldek@awful.systems 21 points 5 months ago* (last edited 5 months ago)

I'm not even going to engage in this thread cause it's a tar pit, but I do think I have the appropriate analogy.

When taking certain exams in my CS programme you were allowed to have notes but with two restrictions:

  1. Have to be handwritten;
  2. have to fit on a single A4 page.

The idea was that you needed to actually put a lot of work into making it, since the entire material was obviously the size of a fucking book and not an A4 page, and you couldn't just print/copy it from somewhere. So you really needed to distill the information and make a thought map or an index for yourself.

Compare that to an ML model that is allowed to train on data however long it wants, as long as the result is a fixed-dimension matrix with parameters that helps it answer questions with high reliability.

It's not the same as an open book, but it's definitely not closed book either. And the LLMs have billions of parameters in the matrix, literal gigabytes of data on their notes. The entire text of War and Peace is ~3MB for comparison. An LLM is a library of trained notes.

load more comments (11 replies)
this post was submitted on 15 May 2024
656 points (100.0% liked)

TechTakes

1384 readers
138 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 1 year ago
MODERATORS