A mind is a device for shunting around concepts. The more intelligent it is, the more abstract a concept it can shunt around. If you’re manipulating mental concepts, streamlining them into an argument, you need to make sure you understand them completely (a technical understanding, not a verbal understanding) otherwise your outputs will be bad. Garbage in, garbage out.

A concept is like a puzzle piece – it has a distintive shape that lets it mesh with other pieces. The more you understand a concept, the more completely you can define its shape. Disagreements occur because people disagree on the shape of the pieces, or on whether certain pieces fit together or not. The danger of an incomplete, overly simplified understanding is that the concepts “shape” will seem to be much simpler than it actually is. Complex topics might seem to easily mesh together because their shapes all seem simple. But when you understand concepts better, the edges become much rougher, and it’s much much harder to make them mesh properly. A little knowledge can be a dangerous thing – knowing just enough for things to seem simple can take you far off course.

This is the problem with Chris Anderson, Kevin Kelly, Malcolm Gladwell, and other populizers who aren’t experts in what they’re talking about – they’ve overly simplified the concepts they’re working with, so they’re meshing them together and shunting them around in ways that a more complete understanding would prevent.

This is what I’m trying to avoid – this is why I post on Lesswrong, to have my ideas relentlessly dissected by extremely smart people. It’s why for anything of substance, I try to pepper it with as many supporting links as possible. A simplified understanding is like a poorly designed building – it collapses under the slightest breeze of an argument. A complete understanding might be difficult and time-consuming to construct, but it’s something that’s built to last.

(part 1 of my essay on salience. The first paragraph will probably be substantially re-written, on account of it sucking)

From Kahneman and Tversky:

“A person is said to employ the availability heuristic whenever he estimates frequency or probability by the ease with which instances or associations could be brought to mind”

I doubt that’s news to any LessWrong readers – the availability heuristic isn’t exactly cutting edge psychology. But the degree to which our minds rely on mental availability goes far beyond estimating probabilities. In a sense, every conscious thought is determined by how available it is – by whether it pops into our heads or not. It’s not something we even have the illusion of control over – we can’t steer our train of thought. (If you spend time actually looking directly at how thoughts proceed through your head, the idea of free will becomes more and more unrealistic. But I digress). What does and doesn’t come to mind has an enormous impact on our mental functioning – our conscious minds can only work with what enters working memory.

Whether it’s called salience, availability, or vividness, marking certain things important relative to other things is key to the proper functioning of our brains. Schizophrenia (specifically psychosis) has been described as “a state of aberrant salience”[link to paper], where the brain incorrectly assigns importance to what it processes. And a quick perusal of the list of cognitive biases[link] reveals a large number directly tied to mental availability – there’s the obvious availability heuristic, but there’s also the simulation heuristic (a special case of the availability heuristic), base-rate neglect (abstract probabilities aren’t salient, so aren’t taken into account), hyperbolic discounting (the present is more salient than the future), the conjunction fallacy (ornate, specific descriptions make something less likely but more salient), the primacy/recenty bias, the false consensus effect, the halo effect, projection bias, etc etc etc. Even consciousness seems to be based on things hitting a certain availability threshhold and entering working memory. And it’s not particularly suprising that A) we tend to process only what we mark as important and B) this causes us to get things wrong. Our minds are efficient, not perfect – a “good enough” solution to the problem of finding food and avoiding lions.

That doesn’t mean that availability isn’t a complex system. It doesn’t seem to be a fixed number that gets assigned when a memory is written – it’s highly dependent on the context of the situation. A perfect example of this is priming [lesswrong link]. Simply seeing a picture is enough to make certain things more available, and that small change in availability is all that’s needed to change how you vote. In state-dependent memory, information that’s been absorbed while under a certain state can only be retrieved under that same state – the context of the situation is needed for activation. It’s why students are told to study under the same conditions that the test will be taken, and why musicians are told not to always practice sitting in the same position, to avoid setting up the wrong context dependent state. Anecdotally, I notice that my mind tends to slide between things that make me happy when I’m happy, and things that make me upset when I’m angry (moods are thought to be important context cues). In general, the more available something is, the less context is needed to activate it. The less available, the more context dependent it becomes. Some things are so available that they’re activated in improper contexts – this is how figurative language is thought to work [link] But some context is always required, or our minds would be nothing a but a greatest hits of our most salient thoughts, on a continuous loop.

(the high points of the essay on salience. This is probably 80% similar to what the final draft will look like. There’s still quite a bit that needs to be tweaked, and it needs a proper conclusion)

Intro: availability heuristic, how it’s much more widespread – everything we think is determined by whether it comes to mind or not. It’s not something we even have the illusion of control over – thoughts just pop into our brains, and we slide between them without knowing where we’re going next. (if you spend time and actually LOOK directly at how thoughts proceed through your head, the idea of free will becomes more and more unrealistic. But I digress…) What most easily comes to mind has an enormous impact on how we process information, as does what DOESN’T come to mind.

Paragraph 1: Saliency, availability, vividness – key to the function of our minds is marking certain things as more important relatively to other things. Schizophrenia/psychosis has been described as a state of aberrant salience, where abnormal things become extremely important/noticed. And a large number of heuristics and biases are based on what our minds mark as important (is consciousness one?) Not particularly suprising – it’s probably a relatively efficient mind design to only have to process the most salient connections out of the hundreds possible that exist between thoughts.

Paragraph 2: Availability isn’t some fixed number – it’s determined by the context of the situation. Priming, state dependent memory, retrieval cues – context matters for how something is retrieved. The more salient something is, the less context is needed for it to be activated. The less salient, the more context dependent it is. It also seems dependent on mood (it’s easier to remember sad things when you’re sad, and happy things when you’re happy)

Paragraph 3: The dark side of this is that it’s possible to perform an action, or have a thought, without having access to the beliefs that sit below it because, for whatever reason, they aren’t salient enough to achieve access. What we have conscious access to is whatever we can fit in working memory – if something doesn’t, we don’t see it. Perception exceeds working memory, so we can see something but not have cognitive access to it. Landmark test, dots test. visuo-spacia disease – recognizing faces but not seeing them (and yet still knowing if you’re seeing one or not!) Golf and learning a swing. Xbox and knowing what buttons you’re hitting. Disliking a movie/person/restaurant but not knowing why (the dislike is salient, the reasons why aren’t). You can know something without knowing you know it; you can believe something without having access to WHY you believe it.

Conclusion: This is bad news for a rationalist (or anyone really). If you’re building complex interconnected belief structures (and you are), you need to be sure you’re working on firm ground. The good news is that memory never seems to be deleted – we can lose access, but the memory itself doesn’t fade. So the problem is one of keeping access open. An easy way to do this is to keep your memory sharp – there are many basic techniques for improving memory. It’s not perfect, but it’s a start. A better way might be to leverage your mind’s propensity for habituation – force yourself to trace your network of belief down to the base, and eventually it will start to become something you do automatically. This isn’t perfect either – it’s not something you can do for fast pace day-to-day life, and it’s probably subject to a whole series of biases. it might even be worth it to write them down – dual benefits of creating a hard copy for you to reference later, and increasing the salience of each through the act of writing and recalling. (concluding sentences)

(working on a new essay about salience, availability, and brain function. Here’s the bones of it that I’ll hopefully assemble into a working skeleton tomorrow)

Basic Points:

We can do/perceive things that we don’t have cognitive access to: actions that have become habits (sports/videogames) slip below conscious access. Visual perception exceeds working memory (and thus conscious access)

Priming, state-dependent memory, retrieval cues – context matters for remembering (consciously accessing something)

Saliency is an extremely important facet of brain function – psychosis has been described as a state of aberrant salience, where abnormal things become extremely important/noticed.

Saliency is availability, and the availability heuristic powers our brains.

The more salient an item, the more retrievable it is – the less salient, the more context is needed to activate it.

Abstract things are less salient (available) than direct experience – it’s why analogies/metaphors/examples are such useful instruction tools.

Things that increase (linguistic) salience – frequency, familiarity, conventionality, prototypicality.

Relation between linguistic salience, perception, memory retrieval?

Salience and ’sliding’ between thoughts – seems affected by mood/state (again, context matters)

Key Point 1: How you think is almost completely dependent on what comes to mind, what you have cognitive access to, how you slide between thoughts.

Key Point 2: It’s perfectly possible (not to mention sub-optimal) to form beliefs without having access to the foundation those beliefs are built on – you can believe something without knowing WHY you believe it. (May be more important/more widespread than I first estimated – we form explanations after we’ve made a judgement, not before. Find that link.)

Something I find interesting about language is how compressed it is – how many concepts and ideas can be attached to a single word or phrase. Often all you need to make an accurate judgement of something is a single word taken in the proper context. Some examples:

The ‘Woo’ Heuristic: If, in a movie trailer, any character shouts the word ‘Woo!’, the movie is completely awful. Examples: GI Joe, Every Martin Lawrence movie (note: the word ‘Damn!’ can be used in the same way).

The ‘lol’ Heuristic: A person who uses LOL in a non-ironic manner when typing is not a person who will startle you with their intelligence, or someone you will learn a great deal from.

The *slur* Heuristic: A person who uses a racial slur in a non-sarcastic manner is a person you should distance yourself from.

And these aren’t even words being used to summarize a great deal of data, like the words ‘rapist’ or ‘murderer’. They’re individual linguistic datapoints, from which we try to extrapolate the entire set of salient features – the style of a movie, the personality of a person. It’s akin to determining the contents of a photograph from the color of a single pixel.

Part of why we can do this is that even a single word contains an amazing amount of data. Hearing a word spoken gives you the word, but it also gives you tone, inflection, emotion – all things that can aid you in making an inference. The same word written down contains much less informational content. And words aren’t heard in a vacuum – they’re used in a situation that creates a context for how the word is interpreted. Even though we can judge based on one word, extra information leaks in from the surrounding situation.

Our minds are designed to process a huge amount of information, but most of that processing is hidden to us. The end result pops into our minds ‘He’s smart! She’s a ditz! He’s an asshole!” and maybe the last few inferential steps. But unless we really look, the incredible complexity of our minds behavior stays silent.

If you’re on the internet, you’ve probably heard the phrase “10,000 hours of practice to be an expert on something”. You can thank Malcolm Gladwell and his ridiculous hair for that. It’s very popular for what I assume are the same reason that role playing games are popular – it means you don’t need to be particularly skilled at something, you just need to put in the time. You don’t need to be good at videogames to beat Final Fantasy VII – you just need to grind away till your whole party is level 99. Likewise, you don’t need to be skilled to be a racecar driver, or a chef, or a painter – you just need to spend 10,000 hours grinding away at it.

I haven’t read Outliers (the book where Gladwell makes this point) but I have no reason to believe this isn’t at least partially true. There’s quite a bit of data that suggests tenacity and hard work are much more predictive of success than talent or skill. The problem is that people tend to focus on the ‘10,000 hours’ part, and not the ‘practice’ part. That practice means PRACTICE – hard work that pushes the boundaries of your current skill. It means you can’t just punch the clock, sit down, and turn into an expert when the counter rolls over to 10,000. You have to deliberately work to improve, not just sit back and let wisdom arrive.

This is why, even though we’ve put in the time, we aren’t all driving like Mario Andretti, or typing 150 words per minute, or becoming experts at any of the other things we tend to do day in and day out. We spend time on them, but we’re not deliberately practicing, and we’re not challenging ourselves, so at some point we stop improving. Our skills at driving level off once we can handle the freeway, and our skills at typing level off when we stop trying to do it faster.

I think this same sort of thing happens on a larger scale, with lives as a whole. At a certain point, people get good enough at living their lives, solving their problems, getting through the day, that they stop getting any better. They don’t practice, so they don’t improve. So you see people moving from crappy office job to crappy office job until they retire, or complaining about the same things at age 60 that they were complaining about at age 20. They never get any better at living their lives – they just start coasting along.

I shuffled through a bunch of different adjectives to describe how I feel about this, and finally settled on one – tragic. It’s a tragedy that people can take the ONE life they have to live, and decide that it’s ‘good enough’. That they can reach a certain point and never try to get beyond it. In the abstract, it’s hard to fathom. If you only had one hamburger to eat for the rest of your life, would you settle for anything less than the best damn hamburger in the world? If you won’t settle for a mediocre burger, why the fuck would you settle for a mediocre life? It doesn’t make any goddamn sense.

Of course, I know why it happens – it’s all too easy to get bogged down in day to day living and to forget that, in the words of Tyler Durden, “This is your life, and it’s ending one second at a time”. That’s why I have that quote taped to my wall at work – to constantly remind myself that I want so much more than to spend my life inside a beige box, and that if I don’t bust my ass trying to improve, I won’t ever get it.

Directly related to the concept of flux is the idea of the swarm. It’s a simple concept – a large group of relatively inexpensive agents deployed against a target. A group of honeybees, a group of missiles, a group of 4chan script kiddies are all swarms. A swarm lets you achieve high flux while forcing your enemy to have low flux.

From Wikipedia:

Military Swarming involves the use of a decentralized force against an opponent, in a manner that emphasizes mobility, communication, unit autonomy and coordination/synchronization.

A swarm takes your destructive/disruptive capability and breaks it into many small parts, and arrays them all against your opponent. And it works by focusing on one specific limiting constraint: attention. It relies on the enemy being able to focus on only a limited number of things at the same time, and then overwhelming it's defense capabilities with shear numbers. How many hornets buzzing around you could you effectively fight at once? Three? Five? Limited attention is more of a controlling factor for our actions than we realize. And the decentralized nature of the swarm makes it difficult to fight against - you can't bring it down by hitting a few targets.

A great example of swarming tactics can be found in the Traveler TCS competition. It was a war game designed to find the most efficient way to build a fleet of warships. Contestants were given a hypothetical trillion dollars, a rulebook, and tasked with creating the best fleet that they could. The fleets then competed against each other to find the winner. When the competition was over, the winner was an AI called EURISKO designed by a computer scientist – not an economist or a strategist – named Doug Lenat. And the winning fleet wasn’t a traditional group of warships like the other teams fielded – it was an enormous group of incredibly weak ships. They were essentially floating gun platforms – they weren’t even capable of movement – but there were so many of them that they were able to overwhelm every other fleet. A battleship may be incredible powerful, but it has a limited number of guns, and they take time to reload, so the number of targets it can engage at a time is limited. EURISKO ruthlessly exploited this weakness, and won decisively 2 years in a row (it was ultimately banned from competition for violating the ’spirit’ of the rules.)

Such a strategy is far from purely theoretical; Iran used an enormous fleet of patrol boats as it’s navy in the Strait of Hormuz. In fact, it’s fairly well known that U.S. ships are incredibly vulnerable to these sorts of attacks.

For the swarm to work, a few conditions have to be filled:

Economies of scale: you need to produce an enormous mass of weak agents. For this to be a good investment, you need to be able to produce them cheaply.

Mission Knowledge: swarms tend to fail when they’re tried to be used like normal forces. For them to work each agent must have a knowledge of it’s goal, and be able to operate independently of the other agents. Otherwise, it lacks the decentralized nature that makes swarms so devastatingly effective. Theoretically, this could be replaced with instantaneous communication between agents, but I don’t see this as feasible.

Agent Effectiveness: Each agent has to be able to inflict some minimum amount of damage that you’ll accomplish your mission if only a few of them get through. A machinegun works on this principle – it doesn’t matter if only 1 out of 100 bullets his a target, since each one is deadly. But a machinegun won’t be very effective against a tank or a battleship – the bullets aren’t damaging enough, no matter how many of them you can shoot.

Size: the swarm can’t be too big, or each agent will be unable to move effectively. Since each agent is so weak, a small decrease in effectiveness makes a big difference.

Like everything else in strategy, the use or disuse of a swarm boils down to whether or not it’s an efficient use of your limited resources. Swarms aren’t a one-size-fits-all plan for victory, but they are a useful tool given the right situation.

There are many like it, but this one is mine.

Anyway.

Think of our minds as an enormous network of interconnected nodes, where each node is a concept/piece of information/thought/whatever. We’re only capable of looking at around seven nodes at the same time – that’s how big our working memory is (give or take). So every node except the seven you currently have in mind is grayed out. Now, your mind progresses from thought to thought along a network path, with older thoughts gradually fading out until they’re removed from your working memory. Moving to a new thought is determined by salience – whichever thought that’s closes to the surface of your mind that’s connected to the node you’re currently on is the one your train of thought will move to next.

A ‘thought’ isn’t really an atomic item – it’s a high level idea mapped to various lower-level concepts. Each node in your mind breaks into a network of smaller nodes when you zoom in closer. For example, you might be thinking of buying a new computer. If you then take the thought ‘computer’, and zoom in, you’ll come up with a microprocessor, and Microsoft Windows, and computer programs, and hard drives, and whatever other concepts are most salient when you think of computer. Each one of those, in turn, can be broke up into even lower-level concepts. So thoughts exist as a hierarchy, with higher level concepts being built up from lower level concepts, bottoming out at direct perception and experience. (This is an oversimplification – for one thing, thoughts don’t always progress neatly downward from complex to simple. And it ignores an enormous range of brain function. But it’s close enough for government work.)

The more powerful a brain is, the farther it can climb up this hierarchy. Most animal brains are only one or two levels above direct experience. Human minds can go quite a few levels farther, though the farther you get away direct experience, the more difficult the subject. It’s why abstract concepts like philosophy and mathematics can be difficult to understand – they’re extremely far removed from direct experience. And it’s why examples and analogies can be so instructive – they provide a shortcut from a high level concept to a direct experience (albeit an imagined one).

This model needs some work, but at the very least it has many of the features that a model of the brain requires. And it makes falsifiable predictions (that the more abstract a concept, the more difficult it will be to understand) though not enough of them. I plan on using it as my starting point for an understanding of how the brain works.

In the commotion surrounding the life and death death of the daily newspaper, one of the best arguments I’ve heard on the ‘nay’ side is “if they didn’t exist today, nobody would bother to invent them.” Newspapers don’t fill a unique niche or solve an important problem anymore – they continue to exist purely on the momentum of enormous media companies.

Necessity is the mother of invention, and usually when something is no longer necessary it slowly fades away. You don’t see too many cassette players around these days, or 8-track players, or fountain pens, or gas lanterns, or vacuum tubes, or CRT monitors, or any of a million other technological relics. But they don’t just disappear. Depending on how long they’ve been around, how ingrained they’ve become in our society, they can linger for years while slowly being replaced by something stronger, better, and faster.

Looking around today, it’s not hard to find things that no one would bother to create if they didn’t already exist: Video Rental Stores. Pay phones/home phones. The Post Office. (maybe) Fax Machines. Globes. Watches. Vinyl Records. Cigarettes.

Some of these will certainly disappear within the next decade or so (I’m looking at you fax machine). But some of these became so ingrained in the cultural zeitgeist that they’ll continue on under a new purpose. Watches have become an element of men’s style, even though the type of man that would need a watch almost certainly has a cell phone. Globes too have become decorative elements. Vinyl records serve as a signal of music collecting status. It’s hard to kill technological artifacts off completely.

Knowing this, we can extrapolate what things in the future will be the same as today, based on how prevalent they are in our society. For example, cars are such an enormous part of it that it might take more than 100 years for them to disappear after their need has been obviated – the systems based around them are just too big to dismantle quickly. And how long will people still smoke cigarettes? My guess is longer than we think. Wooden pencils, and the pencil sharpeners to go with them, probably have quite a few years left in them too. The bigger something gets, and the longer it exists, the harder it is to kill it. The future may be completely different in ways we can’t predict, but it will probably also be similar in ways we aren’t expecting.

Infinity isn’t a number – it’s a concept, It’s the idea of a chain of digits without end. It works differently than normal numbers, which is why you have to invent special math to deal with it.

But it’s hard to define “an endless chain of digits”, because every time you try to, you come up with something thats finite. You can say the words, but to actually be able to manipulate the concept – for example, in a computer program – requires a little more work.

I’m not what you’d call well-read in the field of computer programming. But it seems to me that any reasonable definition of infinity has to be recursive. n = n+1 is a basic (if slow) way of getting there. n = (n+1) * (n+1) will grow pretty goddamn quick. Ackermann’s function will also do the trick. The problem with these is that it really just pushes the issue back a level. These will all reach infinity, provided you iterate an infinite number of times. So to define a concept of infinity, you need infinity.

Even colloquial definitions of infinity are in essence recursive. Take, for example “the biggest number you can imagine” or better yet “bigger than you can imagine”. For every number you think of, you immediately have to use that number to think of an even greater one. This encounters the same problem – you need to iterate forever for it to actually be infinity.

So the definition of the infinite is a diffcult one, which is why a) we encapsulate it in a special symbol, and simply try to manipulate the symbol and b) why infinite math is so goddamn non-intuitive. There’s at least some people who, given that any math that actually applies to the world will be finite, that mathematical infinities are bullshit. As a dilettante I don’t have any worthwhile opinions on the subject, but I can appreciate it’s difficulty – how can you use something that matches neither your intuitions or anything that actually exists in the world?