The other night I got into a discussion with a nerd friend of mine about what might be the best keyboard ever. We agreed that it should give auditory and tactile feedback. It ought to need little force, but also enough to make it sure that miskeying was unlikely. She even agreed to a point with my idea that the classical grand piano's touch could have something to do with a good keyboard touch, but I think I sort of lost her about there with the consequences. Thus, I'll dump the whole of my reasoning here.
A proper keyboard needs to have both tactile and auditory feedback. Those two modalities work together well when you're inputing information that will be interpreted by your visual sense -- the segregation of the two modalities heightens both at the same time. Your tactile sense helps you keep track of the keys, your auditory sense provides exceedingly time-accurate feedback at the same time, and all of that is separate from your visual sense which serves to check and guide the overall picture of what was said. Given that our hearing is accurate downto the microseconds, it probably makes sense that the primary auditory feedback should be purely mechanical as well, since the roundabout through a computer would introduce a delay and a temporal uncertainty that is far worse.
Thus they keyboard should be quiet, but it should nevertheless sound/click.
Then, one of the worst things a keyboard can do is to require lifting your fingers off it. To any touchtypist, that sort of thing seems like a swamp that swallows your fingers. Thankfully that sort of thing is rare today.
Third, it's also rather tiresome to have to push down the keys against a viscous, dissipative load, plus a string force. The string force which negates the previous need to explicitly lift your fingers off, but at the same time necessitates quite a lot of force to be used on your part for every downward keypress. And since laptops derive the balancing force from a nonlinear and viscously damped rubber pad, any work you expend on the key in either direction is rapidly dissipated. This is already a workable balance, but not an enjoyable one, and it's what laptop keyboards pretty much always do.
Fourth, older keyboards, and the better quality current, full-width keyboards do something quite different. They dissipate the end-of-movement energy via a plastic collision at either end, in average they exhibit linear tension, and then they have a little bit of die-hard hysteresis in the middle when they click. In here, the design parameter was traditionally about where the click happened with respect to the common typing motion. In here, we're finally narrowing downto the proper ergonomics.
But then, fifth, even the company my nerd friend pointed to hasn't really gone beyond this paradigm. They haven't really considered whether the traditional design is the best one; they've just gone with the best that we had in the past.
So the final, sixth, longest point of my is about keyboard idealism. Bear with me here, because this is gonna be wild.
Clearly we can't get all of those different nice aspects in a mechanical keyboard at the same time, unless we resort to very convoluted mechanical designs. Those will then break under no time. So, let's ditch the mechanical design everywhere we can. The plastic, mechanical stops will have to be there, of course, both going up and down. But the rest? Not likely.
Otherwise than that, every key could contain a lightweight, very tighthly wound, fed with very high voltage (from a buck-boost-fed power supply) moving coil. Set in resin, and moving against both a set Neodymiym magnet at the bottom, and a similar set coil further out. With enough current (plus regeneration, plus a big enough reserve capacitor), and especially high enough scanning frequency and proper signal processing, this sort of thing could mimic any and all touchfeels via algorithm. All of the so called "moving parts", i.e. coils, could be set in resin and be totally isolated wrt each other, for automatic military grade assurance of functioning. Pour a bottle of coke on it and it just keeps working, without any stickiness.
Then, it's not just about force or feedback when you work a keyboard to its fullest. Here, my favourite example is the grand piano. Pianists are able to expend even full Watts in steady state on their keyboard. Us touchtypists are not, so we get tired, and our output dropps off as our hands tire. What's the reason? Obviously it is because our power output isn't properly impendance matched to the keyboard, and the current keyboard is such a thoroughly dissipative thing. So why not make the above electromagnetic coupling work both ways, so that every keypress leads to electrical input, and then the circuit recouples that energy right back into the typist's fingers to bounce them off the keyboard? In that way we could go with "staccato typing" with diminished loss of energy, the way pianists do.
Old school electronic circuits could not have done that. They would have incurred heavy losses because of low voltages, too low magnetic fields, and resonant circuits which were only designed for a single resonant frequency/speed of typing. They would have been massive and dissipative as hell, as well. But today, we could easily implement all of this using a movable coil, a fixed coil, and if we wanted to save energy, a fixed high-flux permanent magnet, per key. With high frequency active control and the right power circuits per key. No, it wouldn't be cheap. But yes, it could be military grade in more than one sense, and even in the civil society that sort of thing could save insane amounts of money from treatment in carpal tunnel syndrome, lost productivity due to programmers struggling with their keyboards, and so on.
2010-01-29
2010-01-25
Physical steganography
Suppose you want to bug a room and you know somebody's going to sweep it. It's well known that pretty much all radio frequency emitters can be detected using a careful RF analysis. Basically, if it's emitting energy, it doesn't much matter under real scrutiny how you try to divide that energy. It's radiant energy, and as such it can be intercepted and detected.
Unless, that is, it can't be told apart from what the sweeper is expecting already, or alternatively is so utterly different from what se's expecting that se doesn't even have the equipment to efficiently detect it.
From there, I can come up with two examples already. First, powerline hum. If all you put out is something very close to the extremely strong hum always present in any modern building, you're not going to get caught. And even if people start to pay attention to this sort of thing, you can always lower your transmission rate, driving the effort of detection upwards by as much.
And second, since that leads to a diminishing channel, you can play the age old game of finding new modes of conveying the information out. One such means that I haven't seen detected yet is coherent polarization modulation. Say, you place your transmission right on top of some prominent microwave emission line of a compound that is widely present and put out a low level, constant envelope additive signal. Only your signal encodes your stuff via varying polarization, not amplitude of frequency. Just try and find the equipment which can deal with this sort of stuff at low levels...
Unless, that is, it can't be told apart from what the sweeper is expecting already, or alternatively is so utterly different from what se's expecting that se doesn't even have the equipment to efficiently detect it.
From there, I can come up with two examples already. First, powerline hum. If all you put out is something very close to the extremely strong hum always present in any modern building, you're not going to get caught. And even if people start to pay attention to this sort of thing, you can always lower your transmission rate, driving the effort of detection upwards by as much.
And second, since that leads to a diminishing channel, you can play the age old game of finding new modes of conveying the information out. One such means that I haven't seen detected yet is coherent polarization modulation. Say, you place your transmission right on top of some prominent microwave emission line of a compound that is widely present and put out a low level, constant envelope additive signal. Only your signal encodes your stuff via varying polarization, not amplitude of frequency. Just try and find the equipment which can deal with this sort of stuff at low levels...
2010-01-05
High-rise fire fighting
I was just reading about a famous firefighting operation in Los Angeles. As always, a couple of ideas came to mind. Rather than waste them, I thought I'd write them down -- pretty much why I blog at all. Much of the problem of firefighting boils down to water, so that's what's guiding me here as well.
First, high-rise buildings present the precise same, gravity bound predicament to emergency personnel that they do to the architect, or the lazy person. Going up is unduly hard. One important aspect of this is standpipe usage. It's hard labour to get the water upto the fire, and even more importantly, you lose pressure on every floor going up. You have to have all sorts of engineering safeguards so that the upper and the lower floors receive even adequate service in time of need. And in an emergency that manages to break the standpipe, all hell breaks loose. You have about zero pressure for the floor above the break, and considerably less even for the floors directly beneath, because then the pressure only comes from the fountain above to the breakpoint (plus friction, viscosity and upward momentum, which should mostly be neglected because in the "normal" situation any such thing would hinder the service to the upper floors, which are no less valuable).
So my first thought... One way to combat this would be to include low-pressure shutouts on every floor which would cut out the flow below the break. Difficult to do in a manner that doesn't rely unduly on fallible technology or impede pressure/flow under "normal" conditions, I know, but worthy of thought especially in very high rise buildings where the probability and the span of potentially inaccessible floors compound. The solution would probably take the form of mechanical devices designed to slowly and gradually activate upon the presence of significant flow and low pressure, and which furthermore draw power from velocity alone. That sort of thing is eminently constructible, and at least in time, it could be refined to a level where it also "works itself out", so that jamming becomes unlikely even in extended use. As for illicit use of such devices, they would obviously be in more or less plain sight, any control of them would be rapidly overridden by a firesquad, and so any tampering and anybody hanging about the switch would be detected pretty much the first or second time around over an operation.
Second, I wonder whether the water and the inherent power its rapid, pressured flow carries with it could be used to extra advantage, by varying how not just the water, but the power carried by it, is actually used. I mean, power is Power; why waste the huge power that is in the flowing water so that it dissipates in splashes and heat? Why not do something useful with it?
From an alternative viewpoint, this boils down to the basic thermochemical fundaments of sustained combustion or the lack of it: what it takes for a fire to sustain itself is something combustible, usually a mixture of an oxidizer (oxygen, anybody?), an otherwise energy rich material that is held back from spontaneous decomposition by a molecular energy threshold (quite often an organic material, but purified metals and the like serve more than well), and then enough temperature that the average molecular kinetic energy transcends the chemical reaction threshold, giving rise to an exothermic reaction at a wide enough scale, and then positive feedback (usually limited by the amount of the most common oxidizer, air, available).
Ergo, a fire. Which tends to spread. And tends to consume all of those valuable organic and/or otherwise energetically unstable, purified materials we hold as economically valuable. We don't like uncontrolled fire much, then, so we fight it.
From those basics we see that fighting spreading fire means taking one of the necessary elements away. Suffocate the fucker for the lack of an oxidizer. Atomize and isolate it to limit the positive feedback/"contagion". Cool it down to recontain the original combustible so that it is energetically favourable for it to stay in shape and not burn. And in the last part especially, make sure that it don't remain hot enough to restart the reaction spontaneously, no matter what temporary extinguishing measures you've done before that. (Unless, of course, those measures aren't that temporary, which is why we also have foam, powder, zero-ventilation upon Halon release, and so on. But that is another story.)
As a general rule, you must not only stop the reaction, but maintain the reacting system in a state that somehow stops it from reacting again before it's reached its "normal", "reaction- blockaded" state. I believe every firefighter knows this, both theoretically, and practically, and is exceedingly good at it. So what I'm talking about is not what firefighters do, can do or should do. It's about the technology at their disposal.
Here I'd like to take the example of the (very badly named and kitsch-sounding) IFEX 3000. Here the idea is to carry compressed air for power, and little water. You use the air to atomize a small amount of water, and to propel it right into the "heart" (i.e. the surface of the base combustible) of the fire very rapidly. The stated idea of the invention is to maximize the usage of water, by driving it into the fire by force, and maximizing both the proportion of it that is eventually vaporized completely (100%, drawing away the maximum amount of heat water can carry per volume, which is a lot, and which is why we also primarily fight fire with water), and also doing this rapidly (the rationale for this is not explained too well, but it's sound: it's about dividing the fire and bringing large surface areas of it back into chemical stability at once; the spatial backspread is then so slow that multiple blasts can bring even large fires back under control; every fire fighter does this when swinging the hose around over a centralized, expanding cold-zone, and is encouraged to do so in training, instead of just "spraying around"). This innovation does work, and is quite an idea as such...but only upto a point.
There are at least two problems. In the case of the fire I looked at, one principal source of nastiness was burning aluminium, along with updraft over the outer surface of a high-riser which enabled rapid vertical spread.
Aluminium, that stuff reacts with water when hot. So if you feed in small amounts of high surface area water mist, the fire will not go out; even if you were able to crowd out oxygen altogether, factually you would still be supplying enough oxidizer to the fire to keep it going. In that case, if water is all you have (and with aluminium you'd have to use something like pure nitrogen to actually suffocate it, which is not feasible in practical firefighting work, especially at that scale), your only hope is to cool it down enough. So that it stops reacting/burning.
And that's also going to be a challenge to the fine-mist-little-water-camp, because, although aluminium has a limited heat capacity by any metallurgical standard, it also has less than intuitive thermal conductivity for a metal, so that you can cool it down at the surface, but it would still have plenty of internal heat to reignite at the surface if it had burned long enough beforehand. So that sort of thing clearly calls out for conventional, quite heavy water cooling at the surface, well in excess of what you'd use with other sorts of fire, and also dogged persistence -- which is squarely what IFEX cannot give you with the limited amount of water it carries.
(By the way, I suspect that is why they always demonstrate the system with volatile gas/liquid fires. There it's highly efficient, because the total heat capacity of the coolant needed is limited. You only have to extinguish the gases in the flame to bring the fire under control. But were you to put a couple of coal bricks, or a red-hot cube of iron above the liquid surface, or heaven bid an active flare into the mix, it just wouldn't work.)
So, is there a middle ground? A practical one? I think there is. The main point of IFEX is a) rapid delivery and b) high surface area of c) water, which is really, really good at c1) carrying away heat and c2) once vaporized, usually extinguishing the source of oxidizer (atmospheric oxygen) as well.
You don't need compressed air to atomize your water. You only need energy. And every firefighter knows there's plenty of that right in the pressure hose; with enough pressure and men to hold the hose down, plaster walls and the like are gone in no time. Water it also has, quite evidently. The only thing missing is the capability of efficiently forming mist instead of spray or a run. So why not build long, turbulent (even cavitating) , extremely high inlet pressure, asymmetrical, vortex-inducing, sideflow-turbine -like nozzles for firefighting use? That sort of thing would:
Sure, this intuition comes from a lesson already learnt; namely the fire I'm now looking at. But it is also informed by the systemic perspective on things, much magnified by the failure of etc. the WTC towers -- which were also very, *very* well engineered, and in fact eventually stood much more than they were even designed to. So what I'm asking here is, did we forget some essential element in our social science, engineering science, economical science, chemical/combustion, the actuarial one, whatever, about the distinct character of high-rise? That is, that they're *high*; as in "gravitationally challenged" in so many ways?
I know that the intuitive aspect of this has been thoroughly grasped within the rescue community. The eventual, surprisingly fast extinguishment of it by the LAFD bears witness to this. But still, do we really, *really* have any hard science underlying our responses to high rise stuff? Something that explains what we do there, and at the same time suggests verifiably better ways to improve our response?
I don't really think so. And that distresses me. As someone who'd very much like to acquaint oneself with at least one of these marvellous products of the human mind and spirit.
First, high-rise buildings present the precise same, gravity bound predicament to emergency personnel that they do to the architect, or the lazy person. Going up is unduly hard. One important aspect of this is standpipe usage. It's hard labour to get the water upto the fire, and even more importantly, you lose pressure on every floor going up. You have to have all sorts of engineering safeguards so that the upper and the lower floors receive even adequate service in time of need. And in an emergency that manages to break the standpipe, all hell breaks loose. You have about zero pressure for the floor above the break, and considerably less even for the floors directly beneath, because then the pressure only comes from the fountain above to the breakpoint (plus friction, viscosity and upward momentum, which should mostly be neglected because in the "normal" situation any such thing would hinder the service to the upper floors, which are no less valuable).
So my first thought... One way to combat this would be to include low-pressure shutouts on every floor which would cut out the flow below the break. Difficult to do in a manner that doesn't rely unduly on fallible technology or impede pressure/flow under "normal" conditions, I know, but worthy of thought especially in very high rise buildings where the probability and the span of potentially inaccessible floors compound. The solution would probably take the form of mechanical devices designed to slowly and gradually activate upon the presence of significant flow and low pressure, and which furthermore draw power from velocity alone. That sort of thing is eminently constructible, and at least in time, it could be refined to a level where it also "works itself out", so that jamming becomes unlikely even in extended use. As for illicit use of such devices, they would obviously be in more or less plain sight, any control of them would be rapidly overridden by a firesquad, and so any tampering and anybody hanging about the switch would be detected pretty much the first or second time around over an operation.
Second, I wonder whether the water and the inherent power its rapid, pressured flow carries with it could be used to extra advantage, by varying how not just the water, but the power carried by it, is actually used. I mean, power is Power; why waste the huge power that is in the flowing water so that it dissipates in splashes and heat? Why not do something useful with it?
From an alternative viewpoint, this boils down to the basic thermochemical fundaments of sustained combustion or the lack of it: what it takes for a fire to sustain itself is something combustible, usually a mixture of an oxidizer (oxygen, anybody?), an otherwise energy rich material that is held back from spontaneous decomposition by a molecular energy threshold (quite often an organic material, but purified metals and the like serve more than well), and then enough temperature that the average molecular kinetic energy transcends the chemical reaction threshold, giving rise to an exothermic reaction at a wide enough scale, and then positive feedback (usually limited by the amount of the most common oxidizer, air, available).
Ergo, a fire. Which tends to spread. And tends to consume all of those valuable organic and/or otherwise energetically unstable, purified materials we hold as economically valuable. We don't like uncontrolled fire much, then, so we fight it.
From those basics we see that fighting spreading fire means taking one of the necessary elements away. Suffocate the fucker for the lack of an oxidizer. Atomize and isolate it to limit the positive feedback/"contagion". Cool it down to recontain the original combustible so that it is energetically favourable for it to stay in shape and not burn. And in the last part especially, make sure that it don't remain hot enough to restart the reaction spontaneously, no matter what temporary extinguishing measures you've done before that. (Unless, of course, those measures aren't that temporary, which is why we also have foam, powder, zero-ventilation upon Halon release, and so on. But that is another story.)
As a general rule, you must not only stop the reaction, but maintain the reacting system in a state that somehow stops it from reacting again before it's reached its "normal", "reaction- blockaded" state. I believe every firefighter knows this, both theoretically, and practically, and is exceedingly good at it. So what I'm talking about is not what firefighters do, can do or should do. It's about the technology at their disposal.
Here I'd like to take the example of the (very badly named and kitsch-sounding) IFEX 3000. Here the idea is to carry compressed air for power, and little water. You use the air to atomize a small amount of water, and to propel it right into the "heart" (i.e. the surface of the base combustible) of the fire very rapidly. The stated idea of the invention is to maximize the usage of water, by driving it into the fire by force, and maximizing both the proportion of it that is eventually vaporized completely (100%, drawing away the maximum amount of heat water can carry per volume, which is a lot, and which is why we also primarily fight fire with water), and also doing this rapidly (the rationale for this is not explained too well, but it's sound: it's about dividing the fire and bringing large surface areas of it back into chemical stability at once; the spatial backspread is then so slow that multiple blasts can bring even large fires back under control; every fire fighter does this when swinging the hose around over a centralized, expanding cold-zone, and is encouraged to do so in training, instead of just "spraying around"). This innovation does work, and is quite an idea as such...but only upto a point.
There are at least two problems. In the case of the fire I looked at, one principal source of nastiness was burning aluminium, along with updraft over the outer surface of a high-riser which enabled rapid vertical spread.
Aluminium, that stuff reacts with water when hot. So if you feed in small amounts of high surface area water mist, the fire will not go out; even if you were able to crowd out oxygen altogether, factually you would still be supplying enough oxidizer to the fire to keep it going. In that case, if water is all you have (and with aluminium you'd have to use something like pure nitrogen to actually suffocate it, which is not feasible in practical firefighting work, especially at that scale), your only hope is to cool it down enough. So that it stops reacting/burning.
And that's also going to be a challenge to the fine-mist-little-water-camp, because, although aluminium has a limited heat capacity by any metallurgical standard, it also has less than intuitive thermal conductivity for a metal, so that you can cool it down at the surface, but it would still have plenty of internal heat to reignite at the surface if it had burned long enough beforehand. So that sort of thing clearly calls out for conventional, quite heavy water cooling at the surface, well in excess of what you'd use with other sorts of fire, and also dogged persistence -- which is squarely what IFEX cannot give you with the limited amount of water it carries.
(By the way, I suspect that is why they always demonstrate the system with volatile gas/liquid fires. There it's highly efficient, because the total heat capacity of the coolant needed is limited. You only have to extinguish the gases in the flame to bring the fire under control. But were you to put a couple of coal bricks, or a red-hot cube of iron above the liquid surface, or heaven bid an active flare into the mix, it just wouldn't work.)
So, is there a middle ground? A practical one? I think there is. The main point of IFEX is a) rapid delivery and b) high surface area of c) water, which is really, really good at c1) carrying away heat and c2) once vaporized, usually extinguishing the source of oxidizer (atmospheric oxygen) as well.
You don't need compressed air to atomize your water. You only need energy. And every firefighter knows there's plenty of that right in the pressure hose; with enough pressure and men to hold the hose down, plaster walls and the like are gone in no time. Water it also has, quite evidently. The only thing missing is the capability of efficiently forming mist instead of spray or a run. So why not build long, turbulent (even cavitating) , extremely high inlet pressure, asymmetrical, vortex-inducing, sideflow-turbine -like nozzles for firefighting use? That sort of thing would:
- be powered only by the water pressure differential to ambient, making extra supplies unnecessary,
- under current standpipes, it might reduce flow, but instead it could be optimized towards optimum dissipation of heat, and would obviously be under the control of the firefighter as well, for expert effect,
- could yield many of the benefits of the IFEX design, without tripping any patents,
- would have no more moving parts, though perhaps some extra length and weight, compared to today's standard nozzle (perhaps they would also only be used when burning metals are involved, or something, so that my idea is an addition to, not a replacement of, the well-tested arsenal of today), so that they could be easy to use and also robust,
- unlike IFEX-type systems, would ensure continual replenishing, so that even high heat capacity targets could be fought continually until they truly give up,
- unlike conventional type systems, would minimize drainage, which is a hazard not only to firefighters in heavy action (water clutters up the scene, and any water spilled is also a step away from water security in tight spots, exposing firefighters to unnecessary personal risk), but also to the property and the lives that are being protected, and finally,
- they would be a logical step beyond the flashover/hot-ceiling spray techniques that are already commonplace, thus providing a physically well-founded step beyond simple spray, and perhaps even an impetus for going towards spray instead of splash in ordinary scenarios where high heat capacity of the combustible is not an issue.
Sure, this intuition comes from a lesson already learnt; namely the fire I'm now looking at. But it is also informed by the systemic perspective on things, much magnified by the failure of etc. the WTC towers -- which were also very, *very* well engineered, and in fact eventually stood much more than they were even designed to. So what I'm asking here is, did we forget some essential element in our social science, engineering science, economical science, chemical/combustion, the actuarial one, whatever, about the distinct character of high-rise? That is, that they're *high*; as in "gravitationally challenged" in so many ways?
I know that the intuitive aspect of this has been thoroughly grasped within the rescue community. The eventual, surprisingly fast extinguishment of it by the LAFD bears witness to this. But still, do we really, *really* have any hard science underlying our responses to high rise stuff? Something that explains what we do there, and at the same time suggests verifiably better ways to improve our response?
I don't really think so. And that distresses me. As someone who'd very much like to acquaint oneself with at least one of these marvellous products of the human mind and spirit.
2010-01-01
Equilibrium/gun-kata
Ever since I first bumped into the movie Equilibrium, I deeply liked it. Aesthetically, narratively, and for its dystopian social commentary. The critics didn't; they wrote it off as just one more dark, run-of-the-mill dystopia, saved only by the sleek outfits, the overblown commentary, and, of course, a hefty dose of noveau martial arts with guns.
Once more I don't know what this movie was supposed to tell us. Interpretation of just about any artistic work is bound to be a fickle, unsure and highly subjective process. Given my own eccentric leanings in aesthetics, I'd even say that the whole business is completely subjective -- what the critics say is good rarely coincides with what I deem to be so. But still...
Art stilll has its internal invariances. The internal symmetry that pervades the work, in one form or another. When that symmetry involves societal issues, it still challenges us. In art, you can bring on any symmetry you like, and that symmetry is going to be readable across the board when done right. No matter whether you like it or not.
In this case, I think the relevant symmetry is the tradeoff between emotion and order. The tradeoff between a neatly functioning, deterministic, safe society, and the one where individuality makes things complex, chaotic and, in the rhetoric of the film, "too human and emotional". That tradeoff and dynamic is pressed beyond all recognition, perhaps then making this flick a bit too overbearing for an analyst that goes at it with the usual, neutral starting point.
But if you *really* look at it, the so-called dystopia depicted could in fact be a kind of utopia as well. Indeed, there is no war there. No love triangles, no aggression, nothing personally or impersonally threatening. Sure, there is the control mechanism to keep things as such, and that gives rise to a certain dramatical dystopian imagery like maladroits continually being removed by force from the community, and incinerated. But the real philosophical touchstone is still the basic, personal existence of the majority, and the feelings of the protagonist as a (pivotal) part of it. If you think about it, that sort of existence is alien, but conceivable, and not necessarily all that bad.
This ambivalence is what makes Equilibrium so compelling to me. Also, it does not hurt the work that there is an amount of high-caliber action and martial arts going on: that could equally well be interpreted as an artistic means to get the real point across. Thoughtful people could or should read it as such as well. Then they would laud the work as something that encapsulates deeper philosophical contrapositions into a form which actually captivates the common audience. I aim at a win-win interpretation, and I think I have one here, despite common critical acclaim.
Finally, and simply personally, the visual style and composition in this film simply rocks. It would have been enough if they just invented gun-kata for it. But the cinematography goes *so* far beyond that. The sets are amazingly genre-consistent and particular, the consistent high contrast draws my eye, there is a definite visual reference to the monochromatic era, perspective is being used rather liberally to give a sense of space to the scenes, and the handling of photographic time takes on a quality I've *never* seen before or since (i.e. it hints at the kind of bullet-time Matrix used, but it completely stays away from the special effects used there, instead opting for lighting, perspective and well-placed visual cuts).
That all is the hallmark of an inspired photographer at work. The audio track, it's the genre-specific, which makes it rather muted, choppy, mostly non-dialogue. Just as the visual scenery is: a sequence of impressionistic sketches. Very much true to the cyberpunk form of telling futuristic stories.
As I already said, I really, *really* like the result. Perhaps you now understand why, and also why I think this particular movie is in my estimation under-appreciated.
Once more I don't know what this movie was supposed to tell us. Interpretation of just about any artistic work is bound to be a fickle, unsure and highly subjective process. Given my own eccentric leanings in aesthetics, I'd even say that the whole business is completely subjective -- what the critics say is good rarely coincides with what I deem to be so. But still...
Art stilll has its internal invariances. The internal symmetry that pervades the work, in one form or another. When that symmetry involves societal issues, it still challenges us. In art, you can bring on any symmetry you like, and that symmetry is going to be readable across the board when done right. No matter whether you like it or not.
In this case, I think the relevant symmetry is the tradeoff between emotion and order. The tradeoff between a neatly functioning, deterministic, safe society, and the one where individuality makes things complex, chaotic and, in the rhetoric of the film, "too human and emotional". That tradeoff and dynamic is pressed beyond all recognition, perhaps then making this flick a bit too overbearing for an analyst that goes at it with the usual, neutral starting point.
But if you *really* look at it, the so-called dystopia depicted could in fact be a kind of utopia as well. Indeed, there is no war there. No love triangles, no aggression, nothing personally or impersonally threatening. Sure, there is the control mechanism to keep things as such, and that gives rise to a certain dramatical dystopian imagery like maladroits continually being removed by force from the community, and incinerated. But the real philosophical touchstone is still the basic, personal existence of the majority, and the feelings of the protagonist as a (pivotal) part of it. If you think about it, that sort of existence is alien, but conceivable, and not necessarily all that bad.
This ambivalence is what makes Equilibrium so compelling to me. Also, it does not hurt the work that there is an amount of high-caliber action and martial arts going on: that could equally well be interpreted as an artistic means to get the real point across. Thoughtful people could or should read it as such as well. Then they would laud the work as something that encapsulates deeper philosophical contrapositions into a form which actually captivates the common audience. I aim at a win-win interpretation, and I think I have one here, despite common critical acclaim.
Finally, and simply personally, the visual style and composition in this film simply rocks. It would have been enough if they just invented gun-kata for it. But the cinematography goes *so* far beyond that. The sets are amazingly genre-consistent and particular, the consistent high contrast draws my eye, there is a definite visual reference to the monochromatic era, perspective is being used rather liberally to give a sense of space to the scenes, and the handling of photographic time takes on a quality I've *never* seen before or since (i.e. it hints at the kind of bullet-time Matrix used, but it completely stays away from the special effects used there, instead opting for lighting, perspective and well-placed visual cuts).
That all is the hallmark of an inspired photographer at work. The audio track, it's the genre-specific, which makes it rather muted, choppy, mostly non-dialogue. Just as the visual scenery is: a sequence of impressionistic sketches. Very much true to the cyberpunk form of telling futuristic stories.
As I already said, I really, *really* like the result. Perhaps you now understand why, and also why I think this particular movie is in my estimation under-appreciated.
2009-12-31
One more way to detect bugs
The intelligence community is cool stuff. Since I could never be a G-man, I then tend to be interested in the SIGINT side of things. At least I can relate to what those folks are doing, and the technology side is rather deep as well.
Now that I then ventured onto one of the better known technical surveillance counter-measures (TSCM, "bug sweeping") site and started reading through the material, one simple but potentially quite powerful idea came to me. In fact, it's so embarassingly simple that I can't really understand why I'm not able to find any online references to existing implementations already.
The basic premise of bugs is to capture and retransmit data, most commonly audio or video, where remote surveillance does not suffice. Essentially they are about capturing, amplifying and retransmitting data that is otherwise too weak to be detected at a distance. The processing also has to be achieved in a manner which is surreptitious enough not to be noticed. The problem of the bug sweeper is to detect whether that sort of thing is going on, which then boils down to a) identifying and enumerating the possible covert channels which could be used to retransmit the information, and b) systematically, physically going through the weak spots to verify the presence or absence of machinery ("bugs") that could be doing the job. The problem is that there are incredible numbers of such channels, so that even checking for the presence of unwanted radio transmissions is very laborious. Especially when the adversary is actively trying to hide his presence.
Still, there are two distinct advantages on the side of the bug hunter. The first is that he's almost by definition closer to the transmitting device than the adversary, since because range extension is what bugs are installed for in the first place. That means that any signals they send out are going to be stronger at the source, and thus in theory easier to detect. This advantage is commonly used in the TSCM community, in the form of frequency scanners and the like. The adversary then tries to hide his presence into the complexity of the problem, and spread out, encrypt and randomize the outward information flow. Cutting through such counter-counter-measures becomes a well-known arms-race.
Then there is the second advantage, which is that by definition, bugs are about retransmitting a signal produced by the party that is bugged. This means that the signal is under the control of the target, and so potentially under the control of the TSCM expert as well. At some level, this idea has been exploited as well -- the typical warning sign that bugging is going on is when outsiders seem to know stuff that they shouldn't, and of course no spy novel is ever complete without at least some amount of deliberately planted misinformation.
However, at the purely technical level, I've never seen this idea utilized to its fullest. My point is that at one level or another, the functioning and often even the transmissions of a bug of any kind must show some kind of correlation to the message being retransmitted. Since you can control the message, sorting out whether a particular information channel is retransmitting becomes tantamount to measuring some form of cross-correlation or cross-information measure. And at least as far as cross-correlation goes, there are highly efficient means of computing it, especially with input sequences such as maximum length sequences. So why not start by (ideally) convolving/correlating the entire useful radio spectrum against such a low-level noise signal that you inserted? Quite a number of continuous retransmission schemes could be uncovered in that way, after the matched filter presented by your sequences brought out dependent structure from the RF chaos.
The beauty of this kind of thinking is that it is bug-agnostic and employs your natural advantage to its fullest. As a systems level concept it is equally applicable to video -- just perturb the lighting conditions instead, and then proceed the same. It is well-suited for continuous monitoring, because spread spectrum probes can be quiet/dim enough not to be disturbing (or even noticeable) to anybody in the space. Algorithms and hardware to accomplish the feat are generally available, and can easily stretch to hundreds of millions of samples per second, which covers quite a bandwidth already. Narrowband scanning techniques are equally as adaptable to this kind of treatment as are wider band ones. High speed entropy estimation algorithms are available which can help sort out whether there is something anomalous ("I hear structure") in the filter output. Sure, store-and-burst type bugs would not be easily detected this way -- but then, even they have to record continuously, which means that their input stage will probably be noisy in the RF range in a way that is correlated to the acoustics, so that local detection is possible. (In the longer term, their aggregate RF output will also have to be correlated over time with the envelope of the bits actually captured and retransmitted.) And perhaps most beautifully, when you're only trying to detect the presence of anomalous transmissions and not trying to decode them in any particular way, correlations show even in heavily aliased signals, which means that the complexity of the correlator stage can theoretically be dropped into a much lower intermediate sampling frequency; all that is really needed is a wideband sample-and-hold circuit without antialiasing of any kind, and a long enough correlation length.
This sort of thing is not fool-proof; nothing ever is. But I have a hunch that it could catch a relatively high proportion of low to middle tier intercept devices in a completely automated way. And because of the systems level approach, it would also catch many spurious and passive covert channels as well.
Now that I then ventured onto one of the better known technical surveillance counter-measures (TSCM, "bug sweeping") site and started reading through the material, one simple but potentially quite powerful idea came to me. In fact, it's so embarassingly simple that I can't really understand why I'm not able to find any online references to existing implementations already.
The basic premise of bugs is to capture and retransmit data, most commonly audio or video, where remote surveillance does not suffice. Essentially they are about capturing, amplifying and retransmitting data that is otherwise too weak to be detected at a distance. The processing also has to be achieved in a manner which is surreptitious enough not to be noticed. The problem of the bug sweeper is to detect whether that sort of thing is going on, which then boils down to a) identifying and enumerating the possible covert channels which could be used to retransmit the information, and b) systematically, physically going through the weak spots to verify the presence or absence of machinery ("bugs") that could be doing the job. The problem is that there are incredible numbers of such channels, so that even checking for the presence of unwanted radio transmissions is very laborious. Especially when the adversary is actively trying to hide his presence.
Still, there are two distinct advantages on the side of the bug hunter. The first is that he's almost by definition closer to the transmitting device than the adversary, since because range extension is what bugs are installed for in the first place. That means that any signals they send out are going to be stronger at the source, and thus in theory easier to detect. This advantage is commonly used in the TSCM community, in the form of frequency scanners and the like. The adversary then tries to hide his presence into the complexity of the problem, and spread out, encrypt and randomize the outward information flow. Cutting through such counter-counter-measures becomes a well-known arms-race.
Then there is the second advantage, which is that by definition, bugs are about retransmitting a signal produced by the party that is bugged. This means that the signal is under the control of the target, and so potentially under the control of the TSCM expert as well. At some level, this idea has been exploited as well -- the typical warning sign that bugging is going on is when outsiders seem to know stuff that they shouldn't, and of course no spy novel is ever complete without at least some amount of deliberately planted misinformation.
However, at the purely technical level, I've never seen this idea utilized to its fullest. My point is that at one level or another, the functioning and often even the transmissions of a bug of any kind must show some kind of correlation to the message being retransmitted. Since you can control the message, sorting out whether a particular information channel is retransmitting becomes tantamount to measuring some form of cross-correlation or cross-information measure. And at least as far as cross-correlation goes, there are highly efficient means of computing it, especially with input sequences such as maximum length sequences. So why not start by (ideally) convolving/correlating the entire useful radio spectrum against such a low-level noise signal that you inserted? Quite a number of continuous retransmission schemes could be uncovered in that way, after the matched filter presented by your sequences brought out dependent structure from the RF chaos.
The beauty of this kind of thinking is that it is bug-agnostic and employs your natural advantage to its fullest. As a systems level concept it is equally applicable to video -- just perturb the lighting conditions instead, and then proceed the same. It is well-suited for continuous monitoring, because spread spectrum probes can be quiet/dim enough not to be disturbing (or even noticeable) to anybody in the space. Algorithms and hardware to accomplish the feat are generally available, and can easily stretch to hundreds of millions of samples per second, which covers quite a bandwidth already. Narrowband scanning techniques are equally as adaptable to this kind of treatment as are wider band ones. High speed entropy estimation algorithms are available which can help sort out whether there is something anomalous ("I hear structure") in the filter output. Sure, store-and-burst type bugs would not be easily detected this way -- but then, even they have to record continuously, which means that their input stage will probably be noisy in the RF range in a way that is correlated to the acoustics, so that local detection is possible. (In the longer term, their aggregate RF output will also have to be correlated over time with the envelope of the bits actually captured and retransmitted.) And perhaps most beautifully, when you're only trying to detect the presence of anomalous transmissions and not trying to decode them in any particular way, correlations show even in heavily aliased signals, which means that the complexity of the correlator stage can theoretically be dropped into a much lower intermediate sampling frequency; all that is really needed is a wideband sample-and-hold circuit without antialiasing of any kind, and a long enough correlation length.
This sort of thing is not fool-proof; nothing ever is. But I have a hunch that it could catch a relatively high proportion of low to middle tier intercept devices in a completely automated way. And because of the systems level approach, it would also catch many spurious and passive covert channels as well.
On divine intervention
I've been browsing over Contact, which is one of the most evocative movies for me. For some reason.
In this and quite a number of other movies I continually wonder why divinity intersects the plot as strongly, without ever making itself known. I mean, in the fully extraphysical, miraculous sense. Sure, they talk about miraculous "voices from the sky"...but then those have been detected by purpose built radio telescopes. In earlier lore the same thing often takes the form of other means of communication through...stuff that was physically well-known, but not popularly well-understood at the time.
Even the stuff that *really* stretches even the functioning scientist's mind is always left out. We never see the kinds of mental gymnastics exemplia gratis Greg Egan imposes upon us. And that's just self-limiting although experimental, hard scifi, then. It reaches nowhere *near* the kind of awe divinity is supposed to inspire in us'll.
I find just about all descriptions of the preternatural in fictional wanting. They fail to awe. And I don't just about talk about the Bible -- the lousy piece of pseudo-history that tome is -- I mean real pieces of God's honest, meant-to-be awesome pieces of hardcore fiction. The kinds which make me both laugh and cry, possibly at the same time.
Even they fail to carry any sense of divinity. As much as they perhaps try.
In this and quite a number of other movies I continually wonder why divinity intersects the plot as strongly, without ever making itself known. I mean, in the fully extraphysical, miraculous sense. Sure, they talk about miraculous "voices from the sky"...but then those have been detected by purpose built radio telescopes. In earlier lore the same thing often takes the form of other means of communication through...stuff that was physically well-known, but not popularly well-understood at the time.
Even the stuff that *really* stretches even the functioning scientist's mind is always left out. We never see the kinds of mental gymnastics exemplia gratis Greg Egan imposes upon us. And that's just self-limiting although experimental, hard scifi, then. It reaches nowhere *near* the kind of awe divinity is supposed to inspire in us'll.
I find just about all descriptions of the preternatural in fictional wanting. They fail to awe. And I don't just about talk about the Bible -- the lousy piece of pseudo-history that tome is -- I mean real pieces of God's honest, meant-to-be awesome pieces of hardcore fiction. The kinds which make me both laugh and cry, possibly at the same time.
Even they fail to carry any sense of divinity. As much as they perhaps try.
2009-12-15
Accounting wonders
Once upon a time I have (unsuccessfully) worked as the fund-keeper of a small private organization. It was my first time, and I had very little practical understanding of accounting. Sure, I knew precisely how it was done in theory, and that wasn't the problem; my eventual failure had more to do with motivation than technique. Still, I never had an intuitive grasp of what accounting is about. Recently I've also started wondering whether accountants themselves really know why they're doing it the way they do, either.
My favourite example is double-entry bookkeeping. Just about every source I've ever consulted tells me that its raison d'etre is that it serves as a kind of primitive error correction scheme. Well, it sure does that, but why do people keep on using that sort of primitive in the age of computers? Some sources additionally suggest that there is a practical benefit to being able to subtract from an account by using the credit column or to repeating numbers across accounts without sign changes, but those are really just another way to say the same thing. It doesn't get us around subtracting from one account and adding to another, which is the more basic description of double-entry.
In this one, precise case the likeliest explanation dawned on me thanks to my ample use of alcohol.
At the time I was dating someone who lived in a commune. That commune had a common liquor cabinet, with more or less free access as long as any drain was eventually cleared with the owner. The result was a much more varied, practically inexhaustible and eminently available/convenient cabinet for everybody.
As I became to be accepted as a permanent fixture of the communal life, I was granted my own box in the diagram, and due to my thirst, quickly became adept at keeping tabs on my usage. That bookkeeping went through a couple of revisions, but eventually settled in a form where you tallied a standardized shot ("ravintola-annos") on 1) whose bottles you touched, and also 2) your own name. That way both who suffered the blow and who should be the one to compensate were recorded. On one tipsy night I then finally grasped the connection: what we had there was a picture-perfect double-entry accounting using a single account, which just happened to be in base one so that the analogy didn't show as easily as it otherwise would have. And still we pretty much never balanced the book, so that that couldn't have been why we really went with to columns to begin with. (In case you wonder, yes, there was a price premium above buying cost; this commune had little to do with communism.)
The real reason was that we had a multiple input, multiple output stock, where only the aggregate balances accrued mattered. Each transaction needed to go from a single owner of a bottle to a single consumer of a drink, but that was only a mechanism which maintained the constraint that every shot eventually had to be paid for by someone. The real beef was that the aggregate contribution to and withdrawal from the cabinet needed to be kept in check; every transaction basically lost its meaning after those sums had been updated. The stock, it was there in order to permit us to share efficiencies of scale, and the numbers were there to account for aggregate inflows and outflows which helped maintain that beneficial scale.
This is exactly what happens in firms. They have multiple inputs and multiple outputs. From the accounting perspective, all that matters are the sums total of incoming and outgoing money flows. (Or vice versa material flows, if you want to do some kind of materials accounting as well.) In theory you could treat the finance of a large multinational using a matrix of all-inputs vs. all-outputs, sure, but you really don't want to be doing that with even a dozen accounts when what you're actually dealing withis an additive, homogeneous, non-perishable thing like money. You're only interested in what goes in and what comes out in toto. So that is the basis you will choose, input-output=balance, and since every singular transaction you do influences both, you'll have to incrementally maintain those numbers in two different places. Voila: double-entries.
P.S. There are still at least three things unaccounted for, if you pardon the pun. First, journls do exist. They do so for, let's say, forensic purposes. They record the transactions so that if the abstraction that only sum inputs and outputs matter somehow falls apart, we can still get at the details. This could be if the inherent error check double-entry affords us signals an error, doubly so in case there has been intentional fraud, and then on the other hand so that we can also perform closer analysis on the disaggregated flow of transactions (todays called OLAP).
Second, why have multiple accounts? That' s because only homogeneous stocks can be reasonably tracked in the sum-in, sum-out manner: you can have an account in money, but not one in money plus steel, or one in yens plus dollars. When we want to make finer distinctions, we usually require that certain sorts of transactions going out from account a have to go to account b and not account c. That's a rudimentary means of business analytics: the sums on each account tell us how we're doing with its underlying stock, even if the account does not denote anything physical at all. Double-entry then helps us even further, there: you only need to consider the accounts/stocks-tracked relevant to your current task.
And third, why do we still have this system when in principle we could calculate all of the data on the fly from the general journal? This can be explained as a form of precalculation and caching, which enables 1) distributed, asynchronous and write-only maintenance of the relevant balances, 2) early availability of the aggregates because of that, and 3) enables local analysis of aggregate inflow and outflow in the absence of access to a central repository of data.
My favourite example is double-entry bookkeeping. Just about every source I've ever consulted tells me that its raison d'etre is that it serves as a kind of primitive error correction scheme. Well, it sure does that, but why do people keep on using that sort of primitive in the age of computers? Some sources additionally suggest that there is a practical benefit to being able to subtract from an account by using the credit column or to repeating numbers across accounts without sign changes, but those are really just another way to say the same thing. It doesn't get us around subtracting from one account and adding to another, which is the more basic description of double-entry.
In this one, precise case the likeliest explanation dawned on me thanks to my ample use of alcohol.
At the time I was dating someone who lived in a commune. That commune had a common liquor cabinet, with more or less free access as long as any drain was eventually cleared with the owner. The result was a much more varied, practically inexhaustible and eminently available/convenient cabinet for everybody.
As I became to be accepted as a permanent fixture of the communal life, I was granted my own box in the diagram, and due to my thirst, quickly became adept at keeping tabs on my usage. That bookkeeping went through a couple of revisions, but eventually settled in a form where you tallied a standardized shot ("ravintola-annos") on 1) whose bottles you touched, and also 2) your own name. That way both who suffered the blow and who should be the one to compensate were recorded. On one tipsy night I then finally grasped the connection: what we had there was a picture-perfect double-entry accounting using a single account, which just happened to be in base one so that the analogy didn't show as easily as it otherwise would have. And still we pretty much never balanced the book, so that that couldn't have been why we really went with to columns to begin with. (In case you wonder, yes, there was a price premium above buying cost; this commune had little to do with communism.)
The real reason was that we had a multiple input, multiple output stock, where only the aggregate balances accrued mattered. Each transaction needed to go from a single owner of a bottle to a single consumer of a drink, but that was only a mechanism which maintained the constraint that every shot eventually had to be paid for by someone. The real beef was that the aggregate contribution to and withdrawal from the cabinet needed to be kept in check; every transaction basically lost its meaning after those sums had been updated. The stock, it was there in order to permit us to share efficiencies of scale, and the numbers were there to account for aggregate inflows and outflows which helped maintain that beneficial scale.
This is exactly what happens in firms. They have multiple inputs and multiple outputs. From the accounting perspective, all that matters are the sums total of incoming and outgoing money flows. (Or vice versa material flows, if you want to do some kind of materials accounting as well.) In theory you could treat the finance of a large multinational using a matrix of all-inputs vs. all-outputs, sure, but you really don't want to be doing that with even a dozen accounts when what you're actually dealing withis an additive, homogeneous, non-perishable thing like money. You're only interested in what goes in and what comes out in toto. So that is the basis you will choose, input-output=balance, and since every singular transaction you do influences both, you'll have to incrementally maintain those numbers in two different places. Voila: double-entries.
P.S. There are still at least three things unaccounted for, if you pardon the pun. First, journls do exist. They do so for, let's say, forensic purposes. They record the transactions so that if the abstraction that only sum inputs and outputs matter somehow falls apart, we can still get at the details. This could be if the inherent error check double-entry affords us signals an error, doubly so in case there has been intentional fraud, and then on the other hand so that we can also perform closer analysis on the disaggregated flow of transactions (todays called OLAP).
Second, why have multiple accounts? That' s because only homogeneous stocks can be reasonably tracked in the sum-in, sum-out manner: you can have an account in money, but not one in money plus steel, or one in yens plus dollars. When we want to make finer distinctions, we usually require that certain sorts of transactions going out from account a have to go to account b and not account c. That's a rudimentary means of business analytics: the sums on each account tell us how we're doing with its underlying stock, even if the account does not denote anything physical at all. Double-entry then helps us even further, there: you only need to consider the accounts/stocks-tracked relevant to your current task.
And third, why do we still have this system when in principle we could calculate all of the data on the fly from the general journal? This can be explained as a form of precalculation and caching, which enables 1) distributed, asynchronous and write-only maintenance of the relevant balances, 2) early availability of the aggregates because of that, and 3) enables local analysis of aggregate inflow and outflow in the absence of access to a central repository of data.
2009-12-10
The improperness of improper priors
Over the past ten of days or so I've once again been diving into Bayesian statistics; let us say I'm experiencing a serious, personal paradigm shift wrt statistical inference. My main starting point as a practicing database guy is the easy correspondence between embedded multivalued dependency (EMVD) on the one hand and conditional independence (CI) in Bayesian networks on the other—EMVD has for the first time made the basic ideas of BN's accessible to me at the intuitive level, and of course it's pretty nice in other ways, since we're now only talking about the easy, finite, always continuous cases.
Still, you have to mind the infinite frame of mind as well. And since I happen to think like I do, I'm from the start strongly attached to the objective Bayesian viewpoint, in preference to the more conventional subjective one. In fact, my revival is intimately tied with my finally having learned that the Bayesian framework can also be described in purely objective, information theoretical, measurement-of-ignorance terms instead of vague references to "degrees of belief".
Here, the nastiest counter-example seems to be the problem with unnormalized (improper) priors, and the subsequent marginalization paradox which is absent from the theory dealing purely with proper priors/normalized probability measures. At least to me it implies that there might not be a coherent description of complete uncertainty with respect to the Bayesian framework. That is rather bad, because evenas we already know that Bayes's theorem is just about the neatest framework for consodlidating uncertain information in a provably coherent way (no Dutch Book! eventual stabilization among a group of Bayesian learners with shared priors, modulo common knowledge concerns and the like! a neat generalization of Popperian falsificationism!!!), we do always need a coherent starting point which has an objective, not a purely subjective, interpretation.
Now, I had a little bit of an intuitive flash there, as I'm prone to. While it is true that we cannot normalize many of the most natural "distributions" that would go along with common improper priors, perhaps that has less to do with the impossibility of bringing formal rigor to bear to them than we might think. I mean, sure, we cannot even make the simplest of such priors, the flat one, normalize into a probability measure in the infinite base set case; not even using the theory of general distributions. But sure enough we can make it work if we let the prior become a general linear functional, restricted to a class of arguments where it is continuous.
So, maybe the marginalization problem, or the common use of improper priors, wasn't so much about the impossibility of representing noninformative priors "cleanly", after all. Maybe it was just about our choice of representation?
I haven't gone through the particularities, but I have a strong sense that this sort of approach could lead to a natural, structural encoding of the special nature of not-knowing-shit versus sorta-probabilistically-knowing-something—structural restrictions wrt the resulting operator algebra would simply make certain kinds of calculations inadmissible, those restrictions would propagate in the intuitively proper manner through things like Bayes's rule, and things that manifest themselves as the marginalization paradox (or others alike it?) would probably be forced out into the open as topological limitations in the compatibility of operators which act not only on distributions proper, but the dual space as well. In particular, Bayes's rule with the usual pointwise limit interpretation would only apply to the density function side of things, and would probably be qualified in case of functional/function, and certainly functional/functional, interactions (e.g. you can't reasonably deal with products of functions and functionals, you always have to go to the inner product, i.e. in conventional terms marginalize when both are present; and in the case of functionals, the topology of the dual function space is rather different from the conventional one, seriously affecting the theory of integration).
Still, you have to mind the infinite frame of mind as well. And since I happen to think like I do, I'm from the start strongly attached to the objective Bayesian viewpoint, in preference to the more conventional subjective one. In fact, my revival is intimately tied with my finally having learned that the Bayesian framework can also be described in purely objective, information theoretical, measurement-of-ignorance terms instead of vague references to "degrees of belief".
Here, the nastiest counter-example seems to be the problem with unnormalized (improper) priors, and the subsequent marginalization paradox which is absent from the theory dealing purely with proper priors/normalized probability measures. At least to me it implies that there might not be a coherent description of complete uncertainty with respect to the Bayesian framework. That is rather bad, because evenas we already know that Bayes's theorem is just about the neatest framework for consodlidating uncertain information in a provably coherent way (no Dutch Book! eventual stabilization among a group of Bayesian learners with shared priors, modulo common knowledge concerns and the like! a neat generalization of Popperian falsificationism!!!), we do always need a coherent starting point which has an objective, not a purely subjective, interpretation.
Now, I had a little bit of an intuitive flash there, as I'm prone to. While it is true that we cannot normalize many of the most natural "distributions" that would go along with common improper priors, perhaps that has less to do with the impossibility of bringing formal rigor to bear to them than we might think. I mean, sure, we cannot even make the simplest of such priors, the flat one, normalize into a probability measure in the infinite base set case; not even using the theory of general distributions. But sure enough we can make it work if we let the prior become a general linear functional, restricted to a class of arguments where it is continuous.
So, maybe the marginalization problem, or the common use of improper priors, wasn't so much about the impossibility of representing noninformative priors "cleanly", after all. Maybe it was just about our choice of representation?
I haven't gone through the particularities, but I have a strong sense that this sort of approach could lead to a natural, structural encoding of the special nature of not-knowing-shit versus sorta-probabilistically-knowing-something—structural restrictions wrt the resulting operator algebra would simply make certain kinds of calculations inadmissible, those restrictions would propagate in the intuitively proper manner through things like Bayes's rule, and things that manifest themselves as the marginalization paradox (or others alike it?) would probably be forced out into the open as topological limitations in the compatibility of operators which act not only on distributions proper, but the dual space as well. In particular, Bayes's rule with the usual pointwise limit interpretation would only apply to the density function side of things, and would probably be qualified in case of functional/function, and certainly functional/functional, interactions (e.g. you can't reasonably deal with products of functions and functionals, you always have to go to the inner product, i.e. in conventional terms marginalize when both are present; and in the case of functionals, the topology of the dual function space is rather different from the conventional one, seriously affecting the theory of integration).
2008-09-30
A libertarian appeal to anchoring bias
In our own lifetime, we've used to living in a liberal democratic state, where the rule of law, personal morals, prudence and the like are held in high regard. At the same time it's well known from cognitive psychology that such a life-long status quo is bound to serve as a cognitive frame and an anchor against which we reflect our lives. Policy proposals included.
Hence we're used to thinking about taxes as just a minor nuisance, compared to the void that would be left if the nation state couldn't finance itself. Quite a number of regulations seem quite normal to us, and not particularly bad, since they were always there, and we got used to them. A little bit more in either department doesn't seem like such a bad deal to us.
But what if we lifted the anchor, and instead set it at the utmost of regulation and taxation? Suppose we were living in a society where every last bit of human gesture was regulated, and every last penny of our income was forfeit, to be distributed as the state seems fit. Would we then think the same about all of the less restrictive policies we're accustomed to as we did before?
I think not.
Suddenly we'd see that yes, if I made a euro in free exchange with my fellow man, it'd be quite wrong to punish our mutual fortune -- effectively stopping it from happening beforehand because of the lack of a profit motive -- by forfeiting the bilateral gains of the interaction. Suddenly the idea that I should perhaps be able to voice my opposition to the state -- or in fact any ideology, opinioin, or soul at all -- freely would carry a whole lot more weight.
Starting from our current viewpoint, framing, and cognitive anchoring, we'd quite likely be able to tolerate a whole lot of intervention. But at the same time, starting from total interventionism, we'd quite likely be willing to go right down to zero intervention.
So, to sum it up, my point is that thinking about liberty starting from our current reasonably wide liberty is bound to make us value it less than would be the case if we started from a relative lack of it. I also think this constitutes a neat rationale for why liberty is not valued as high as it used to be, in the era when my liberal/libertarian ideology developed, and why the most fervent proponents of my idealogy tend to be detractors from the opposite side. It also sheds some light on the relative unstability of political ideologies -- different starting points might well lead to countervailing shifts in popular opinion, and they also might overshoot to the degree of cyclical revolution, at worst.
Hence we're used to thinking about taxes as just a minor nuisance, compared to the void that would be left if the nation state couldn't finance itself. Quite a number of regulations seem quite normal to us, and not particularly bad, since they were always there, and we got used to them. A little bit more in either department doesn't seem like such a bad deal to us.
But what if we lifted the anchor, and instead set it at the utmost of regulation and taxation? Suppose we were living in a society where every last bit of human gesture was regulated, and every last penny of our income was forfeit, to be distributed as the state seems fit. Would we then think the same about all of the less restrictive policies we're accustomed to as we did before?
I think not.
Suddenly we'd see that yes, if I made a euro in free exchange with my fellow man, it'd be quite wrong to punish our mutual fortune -- effectively stopping it from happening beforehand because of the lack of a profit motive -- by forfeiting the bilateral gains of the interaction. Suddenly the idea that I should perhaps be able to voice my opposition to the state -- or in fact any ideology, opinioin, or soul at all -- freely would carry a whole lot more weight.
Starting from our current viewpoint, framing, and cognitive anchoring, we'd quite likely be able to tolerate a whole lot of intervention. But at the same time, starting from total interventionism, we'd quite likely be willing to go right down to zero intervention.
So, to sum it up, my point is that thinking about liberty starting from our current reasonably wide liberty is bound to make us value it less than would be the case if we started from a relative lack of it. I also think this constitutes a neat rationale for why liberty is not valued as high as it used to be, in the era when my liberal/libertarian ideology developed, and why the most fervent proponents of my idealogy tend to be detractors from the opposite side. It also sheds some light on the relative unstability of political ideologies -- different starting points might well lead to countervailing shifts in popular opinion, and they also might overshoot to the degree of cyclical revolution, at worst.
2008-08-27
Detecting record structure in a file
As a signal processing kinda person, I'm well acquainted with the properties of the Fourier transform. It is usually thought of as a tool that has mostly to do with the discretized continuous time signals. But in the abstract, it's founded in shift-(in)variance. That makes it pretty much the ultimate tool for detecting cyclic structures, even if they're discrete at the origin.
I'm also a file/data format freak. A data representation enthusiast, if you can call anybody that. That makes me think, how about simply taking a long term FFT of a data file, in order to bring about any periodicity/record structure in it? Perhaps that sort of operation could serve as a reverse engineering tool when repetitive, record structures are present. And, looking into the Fourier statistics, perhaps it could even help calculate some sort of implicit measure of fractal dimension, which could help figure out the dimensionality of the record structure being looked at.
I'm also a file/data format freak. A data representation enthusiast, if you can call anybody that. That makes me think, how about simply taking a long term FFT of a data file, in order to bring about any periodicity/record structure in it? Perhaps that sort of operation could serve as a reverse engineering tool when repetitive, record structures are present. And, looking into the Fourier statistics, perhaps it could even help calculate some sort of implicit measure of fractal dimension, which could help figure out the dimensionality of the record structure being looked at.
Subscribe to:
Posts (Atom)