a computer must never be allowed to make a decision because a computer cannot be held accountable for its decision. I'm not not a technopagan, but roko's basilisk is not going to torture things that are indistinguishable from the decaying system of energy and memes that i consider to be my self. there is no god in the machine, and likely no ghost hidden in these gigabytes and terabytes of non-random numbers. Also, Roko Basilisk is just a character in a webcomic.

These are not things people should be using for automated decision-making or "agentic" capabilities, the things people are most excited to be using them for. You might be able to ape some of this stuff with an unacceptable error rate, or throw enough spaghetti at the wall to make it work, but these are massive probabilistic shell-script runners and search engines, basically, not some sentient butler and financial advisor and pair programmer.

but you can listen to what it says about the similarity and "vector space" of your documents and personal data and the correlations it spits out and ask yourself if you agree with it, and augment what it spits out with things you have previously agreed with. This meta-cognition feedback loop is already how I use my journal, to read old or related elements and ask myself what i think about them, i want to super-power this loop and build out the Fediverse Second Brain in the process.

I think in that loop, that loop of having a thought and rather than losing it forever to the next one, but writing it down or persisting it and then later presenting it to yourself and others and asking what we think about it now, is like one of the fundamental things that make us human. LLMs don't have true memory yet, this RAG stuff is the closest thing you can get and you can see the edges of it.

rather than memories, the thing has that semantic compression space. Your jaunt through it can be primed with high-context information from your own notes, then, or wikipedia or another data source and used as a sort of intuition tool to surface and compare documents in ways you might not intuit yourself. you can't trust anything that comes out of these systems, and as soon as you're working with your own notes, and texts, and journals, you'll feel this tension directly as it makes logical leaps across canyons and returns invalid summaries of unrelated documents that you know like the back of your hand, only because some of the words in them share semantic foot-holds with the things you're asking about. this can be a boon or it could be bad, we'll see.

You have to cultivate your own wisdom and intuition to see through bullshit. this is not only true in LLMs but in life in general. this is another tool i am building in trying to cultivate a deeper intuition and sort of internal self-understanding. Remember samadhi, and cultivate emptiness or un-born-ness when working with these programs as in everything else.

Once upon a time, an oracle was known to speak truth, but a dangerous and nuanced truth with often hidden dangers or tragic under-tones applied to them. People would consult Delphi and receive an answer they'd take months to ponder on and worry over and consider as they unknowing slide their way in to its future. if it's true anyhow that we're creating these powerful agents, why should we not treat these weird little daemons we've created the same way?

NEXT have someone hit me with a cluebat here

:Project: