We Are Really Unhappy with Our Operating Systems, and Don’t Know It

September 24th, 2016

Linux has won. It is taking over everything, from tiny devices to the biggest super-computers. Apple’s operating systems are all pretty much on the same model, and Microsoft always seems to be trotting along in roughly this direction, too.

The idea is pretty cool: Give each program a uniform view of the machine, keep them from interfering with each other. Not only can each program mostly pretend it owns the entire machine, the model is good enough to be extended to multiple users, all running on the same machine.

Yes, there will be resource limits with all this sharing going on, but that is a necessary limitation, the larger sharing model is great.

So why are we so unhappy with it? Why do we have this big virtualization fad? The operating system was supposed to let multiple users share the same physical machine, why an extra layer of multiple operating systems sharing the same hardware? If these multiple operating systems were different kinds of operating systems (needed to be compatible with different kinds of programs) that would make sense, but mostly we run multiple virtual copies of the same operating system. Frequently the same version of the same operating system. The popularity of hypervisors for providing multiple uniform views of the hardware, keeping them from interfering with each other, seems a big indictment of what the OS was supposed to do. Something is wrong with the API offered by the OS if we prefer the API offered by BIOS. Something is wrong.

And inside the OS, different programs were supposed to do the different things. So why are we now inventing enormous container facilities like Docker and Kubernetes for supplying the features we want? Isn’t that what the OS was supposed to orchestrate?

I don’t see much questioning of the role of the OS, but I see an awful lot of ad hoc reinventing of OS-like services.

Part of this is clearly a limitation of the OS model: Individual programs are isolated from each other, but it seems not isolated enough, we want more isolation, so we fire up new OS instances. Also, individual programs have complicated and conflicting dependencies to shared libraries that the old OS model isn’t good at mediating. Finally, individual programs are not where the action is, we run different programs in concert with both dependency confusions between them, and contradicting desires to be isolated from other programs (so they don’t interfere) but not isolated from other programs (so they can cooperate). It seems these are all issues the OS should handle, and it doesn’t, that’s why we have so many VMs, and these container facilities.

Recently I ran across the various name-spaces that the Linux kernel offers. (Linus is very pedantic that the kernel just be the kernel, but that doesn’t mean it isn’t still freaking gigantic and bursting with features.) These name spaces provide a lot of granularity for controlling what is isolated and what is shared between different programs. It seems they make it possible to completely isolate software, as if you were running completely different operating systems. I say “it seems” because I don’t know that I am right, I don’t know that these different name spaces cover all the bases. And, even if I knew they did cover all the bases, how would anyone ever trust that they did it in a bugfree way? How would anyone ever know that there isn’t some unintended leaking between spaces, security holes hidden in the confusion.

I think this gets to the point: The confusion. The old OS model was simple, that was a virtue. The model implicit in Linux name spaces is so complicated that I almost don’t want to call it a model: if almost no one can understand the implications of all those features, can it be called a “model”? Does it instead become an “artifact”? Something to be studied, as opposed to a model, something clear enough to be understood?

Maybe I am just being overwhelmed and demonstrating my ignorance. But something tells me that the simplicity of hypervisors, presenting a near bare-metal model, isn’t about to lose its appeal as everybody starts to grok Linux name spaces.

I think we are choking on unmanaged complexity, that we are building systems that are more complicated than we know, that not only are they riddled with conventional bugs, but attackers are waltzing though our systems via the security holes made possible by that complexity. But that’s another topic.

My conclusion here is we have run out the old OS model to the point of absurdity, that we need to rethink what abstractions an OS should offer. The old OS model was both powerful and simple, but look at the layers of baroque filigree we are accumulating, it is time to revisit our assumptions about what an OS is.


©2016 Kent Borg

Kent’s Super-Simple, Excellent Password Advice

September 22nd, 2016

This excellent advice is simple, in fact its excellence depends upon being simple. Complicated is the enemy of security. If you follow this advice you will be among a very rare elite in how secure your passwords will be.

Four parts:

1. Write down your passwords. On real paper, with a real pen or pencil, and keep the list safe. If you want to get fancy, maybe don’t quite tell the truth, at least not the whole truth, maybe leave something off each password (something you will remember), so if someone finds the list they won’t quite know any of the passwords on the list. And keep the list safe.

2. Now that you can keep track of what your passwords are, never recycle passwords between accounts. So, if someone breaks into one site, your other accounts aren’t at risk. (Today’s news, as I write this, is information on 500,000,000 accounts were stolen from Yahoo.) Don’t reuse passwords in different places.

3. When you make up a new password, dream up something you think no one will guess. (I know, you already do that.) Now, to be extra secure, add something even you couldn’t guess. Maybe look at the time, exactly how many minutes past the hour? Include that in the password. Or look around you, pick something else—but pick something you could not anticipate—and include it as part of the password.

4. Keep this entirely manual, the whole approach is low-tech for a reason. Computers are usually pretty insecure. (Ask Yahoo…) Don’t automate any of it, because that’s really hard to do safely (ask Yahoo), keep it manual. Don’t even photocopy your password list, because copiers are really computers these days. Don’t take a picture of the list, because cameras are also computers these days. Yes, backups are good, but sorry that has to be manual. The benefit is, as long as you keep all of this manual, you can trust your common sense, because you will understand every aspect, you have real expertise manual stuff because you can see it.

That’s it. Low-tech as hell, which means most techies will hate it, but who cares that it’s controversial as hell? It’s smart. Because it is simple.


P.S. And I really am so very sorry you can’t use a password manager program, but they are just too complicated, they will have security problems, admit it, you know it in your heart they will. Don’t trust them.

Snowden, the Movie

September 16th, 2016

I went to one of the first Boston matinees of the movie Snowden today.

It was all very familiar territory: it could have been boring or–as with any subject I know a lot about–it could have been excruciating in its errors. It was neither. It held my attention, it did not disappoint.

But was it a good movie? I usually have tons of opinions, I fret over whether a movie hits the ten-minute mark right, whether the script is “economical”, whether characters are compelling, whether the plot is interesting. In this case I can’t say, I am not unbiased: I am an American. And this is really important material–important to any American.

I do know it was at least a competent movie, because it had me wanting to cry. I knew Edward Snowden was a hero, but Oliver Stone tugs for tears. At least from me.

Is it a great movie? Probably not, just because great movies are rare. But I don’t know. Ask me in a few years, I’ll know better. But right now I am kinda choked up over a man whose illusions were shattered, followed by his world being shattered as he followed his conscience with selfless acts.

Another bit of praise: Usually it is painful to see a movie on a topic that I know something about, worse if the movie is technical, and far worse if it is about a technical topic I know something about. This movie did well by that measure.

-kb, the Kent who thinks the three branches of government should not be secret legislative measures, implemented by secret executive orders and agencies, overseen by secret courts.

©2016 Kent Borg

Why We Got Ourselves a Trump: 4 Crazy Tricks

May 4th, 2016

Four things came together to give us Donald J. Trump as the all-but-official GOP nominee.

First, the so-called cable news channels in the US are, indeed, for-profit businesses, looking for ratings, looking for an edge. Thump was ratings gold for them. He got enormous free publicity in exchange for supplying viewers. He did a very good clown-act. Maybe professional “wrestling” is a closer approximation, but either way they have airtime to fill and he provided them with riveting free content.

Second, the GOP has been pandering to, and fanning the flames of, a very dissatisfied base. They have been promoting a “reality” that is remote from actual facts. And, they have not delivered on their promises. This left the GOP a hollow party, dependent on a narrowing base, that no one could reason with. They created a monster. A monster that apparently watches cable TV.

Third, due to something the GOP has accomplished, the 1% have done great in this otherwise tepid recovery, but vast areas of this country (it goes zip code, by zip code) have been left behind and are still where George II’s Great Recession left them. They have not seen the recovery at all. They are rightfully afraid, and unfortunately very angry. And they watch TV.

Fourth, The Donald turned out to be a political genius. The man is nothing if not arrogant, but I don’t think even he had a clue how good he would be at this. He took his entertainment TV experience and turned it into an entertaining campaign, driven by free TV.

I thought the Trump phenomenon would burn out. I think he guessed it would, too, that he would come away with a bigger “brand”. Well, the second part sure came true.


©2016 Kent Borg

Touchscreen Password Idea

February 1st, 2016

Passwords are a problem, and lots of people say they are doomed, but I have seen no good alternatives, so I sometimes think about making them better.

Touchscreens are important yet really hard to enter good passwords.

Also, I would like to do more of a “key exchange” when entering my password. I use different computers and I don’t reuse passwords between these computers, which means I sometimes enter a password for the wrong computer. Oops! Some sort of richer interaction with the other end would prevent this.

So here is my (embrionic) idea.

Have the password be a location in a virtual 3D space. Use the 3D hardware capabilities of phones and tablets and have the user drag around the screen to drive to the location that is the password. By having different randomly chosen starting points in the 3D space for each login attempt a simple “key logger” is made more difficult as is reading screen smudges. By having more of the space revealed as the user navigates the computer has to reveal more information in response to the user’s input, making it more of a “key exchange” and making the space richer and so lengthing the password.

Put another way: a complex 3D space, uniquely generated for each user. The password is a “secret button” somewhere is the space. To authenticate the computer starts the user in some random location and the user flies through the space and touches the secret button but no other.

Shoulder surfing is a problem, but once the user gets good at it s/he might be swooping through so fast that a casual observer might have a hard time realizing what just happened. Particularly if there were a needle-threading aspect where some routes are good and other are not.

By using the full power of the GPU it also puts a limit on how far away a man-in-the-middle could be. (Which makes remote authentication tricky.)

By drawing on the user’s motor skills there might be a way to drop the password down in the brain so the user doesn’t know it in a way that can be told to others. Make the password more like a customized motor skill.


©2016 Kent Borg

An Idea for Doing Background Removal from a Sequence of Stationary Images, Manual-Style [Updated]

January 11th, 2016

Update: Finally looking at implementing this and I realize that thinking of that fully populated tree is probably good for understanding it, I don’t need to store anything but the left edge. When a new frame comes in, I will calculate a new left edge based on the new frame and the previous left edge.

My memory requirements for a size N triangle are then N-1 (I don’t need to save the result if no one will ask for it again) and while calculating I need to store N-frames plus whatever my image processing library uses, etc. The fact this scales linearly with the length of my background history is nice, I can go long for cheap. The time to calculate does scale with the length of the history, but still linear.

Another thought: natural vision systems pretty much only see change, make something stand still long enough and it will go away. It might make sense to spend the linear time and memory to compute a long history, but allow the caller to choose how quickly stationery objects disappear; compute the whole left edge to maintain the chain of history, but choose to look at a more recent step.

A final correction: This is not really parallelizible, the library doing the underlying image processing could well parallelize, but these steps need to be done in sequence.

Back to the original post…

[Warning, this completely techie, musing about computer vision by someone who doesn't really know much about computer vision. But heck, sometimes those who don't know the right way to do something occasionally come up with something cool.]

How about something like this. Maintain a triangular poly tree where at the base is a history of recent frames.

                             / \
                            x   x
                           / \ / \
                          x   x   x
                         / \ / \ / \
                        x   x   x   x
                       / \ / \ / \ / \
                      x   x   x   x   x
                     / \ / \ / \ / \ / \
                    x   x   x   x   x   x
Newer              / \ / \ / \ / \ / \ / \              Older
 <-               x   x   x   x   x   x   x               ->
                 / \ / \ / \ / \ / \ / \ / \
                x   x   x   x   x   x   x   x
               / \ / \ / \ / \ / \ / \ / \ / \
              x   x   x   x   x   x   x   x   x
             / \ / \ / \ / \ / \ / \ / \ / \ / \
            x   x   x   x   x   x   x   x   x   x
           / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
          x   x   x   x   x   x   x   x   x   x   x
         / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
        x   x   x   x   x   x   x   x   x   x   x   x
       / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
      x   x   x   x   x   x   x   x   x   x   x   x   x
     / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
    x   x   x   x   x   x   x   x   x   x   x   x   x   x
   / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
  x   x   x   x   x   x   x   x   x   x   x   x   x   x   x
 / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \ / \
x   x   x   x   x   x   x   x   x   x   x   x   x   x   x   x
A 16-base tree then has 120 vertices in it. The 16 Xs along the base are 16 historical frames. At 5fps, this covers a history of three seconds.

The first row of Xs above the base is made by taking the two images below it and doing:

  • an absdiff;
  • a threshold of the result to make contours of the areas in common; and
  • using those contours, a masking of one of the frames to make masked   image of what is in common between the two.

This is a reduction operation. We start with whole frames and we produce new frames that are at most whole frames, but quite likely reduced (masked) frame areas.

At this point every pixel that has made it up to the second row is in some sense a good pixel, it has matched some other pixel.

We create the rest of the tree by continuing to do pair-wise operations on the images below, but the operation for the rest of the tree is a bit different from the first of operations.

  • To begin with we do do the same operation, we do the matching and reduction (for any area in both masks, if the pixels match the they get added to the output mask sent up to the next level).
  • But then we do a supplementing operation: for any pixels in one input mask but not the other input mask, they get added to the output mask and included in the output sent up to the next level.This continues at each layer to yield one masked image at the top.

I won’t know what this looks like until I see it, but imagine something moving through time, casting a shadow from under the pyramid, maybe tampering with, say 6-frames. Looking up the tree, it can only influence the triangle above it for 5-layers up, then is gets out-voted by constant stuff from before or after it in time.

This poly tree scheme is expensive and so can only go back a short distance in time. The number of operations in that whole triangle, to compute the apex is great, too much of a cheap CPU to calculate per frame at any reasonable frame rates. So instead we trade memory for CPU. We keep most all the output data from each new frame’s computation, and just computer the change for each. What is that computation of change?

- Age out the oldest frame: remove the 16 Xs that go down the right edge;

- Add the newest frame: a new X on the left at the bottom row; and

- Do the 15 calculations necessary to put 15 new Xs up the left edge above that new frame.

Motivation: I have played with OpenCV and the cv2.BackgroundSubtractorMOG() and cv2.BackgroundSubtractorMOG2() background removal functions and I don’t like them.

First, they aren’t working for me: old background information never ages out, a big change in scene continues to be included in the foreground and never displaces the old background.

Second, they are too slow. Particularly MOG2. I can’t keep up with a reasonable frame rate on a Raspberry Pi 2.
Falling back on a simple absdiff for motion detection I discovered I can, in place of the MOG2 that was falling behind, do a stupid loop of 30 absdiff’s and not fall behind. With this scheme I estimate I will have to do about that much work. And, unlike MOG2, this can be parallelized to multiple threads which can run on the multiple CPUs of a Raspberry Pi 2.
There are probably better ways, but it was easier to think up this one than to go read a couple books on computer vision. And it looks pretty easy to build. I just need to find the time to try it. And how do I efficiently represent a polytree in Python without breaking my brain. Will it be easy or hard?Will a nice recursive model work…?


©2016 Kent Borg

Our Founding Fathers–Eating

January 5th, 2016

I have occasionally imagined a piece of historical fiction, a Rip van Winkle story where John Adams (a grumpy, wise, philosopher) or Ben Franklin (a gourmand, party animal, and scientist) or maybe Tom Jefferson (a million contradictions who liked liberty and revolution and food and wine and books and women of all colors) is dropped into the present day to make sense of it, with the help of someone to be a clumsy guide and to keep our time traveler out of jail. (Me! Me! Picke me!) We learn about ourselves and our history as our lab rat tries to make sense of our era.

Fascinating to think about. It makes me ashamed I know so little history (and cultural history) to have a decent guess at what the poor lab rat would see.

Then I became completely distracted by eating. Our food bears so little resemblance to what these men knew that I think any one of them would be both impressed by the Taste Sensations of McDonalds, and ornery and out-of-sorts after a day here because our “food” is not food.

The result? Every time I eat something really good (Tonight: local lamb from Walden Local Meat, here in Boston) I feel like I am eating like a Founding Father.


©2016 Kent Borg

What Makes Anyone Think Physics is Not a Religion?

December 16th, 2015

[I am sure this is not original, on so many levels, but my fingers are compelled to try to capture these thoughts. Forgive me.]

At a fundamental level physicists appear to be deeply religious. Their Articles of Faith are something like this:

  • There is order,
  • This order can be understood by us,
  • It is predictive, has temporal properties, and likely practical implications,
  • (Might be isotropic).

Snide remarks aside, there is a deep faith here: What I do in my little life doesn’t seem to have a great and deep order to it, why should the far larger universe constrain itself to being so precisely ordered that we can make exact equations about it? Why couldn’t the universe be capricious and random and arbitrary and I’ll-do-this-here and I’ll-do-that-there? I don’t know.

But physicists have this deep faith that they will understand if they only keep looking, that there is a fundamental order to the universe, that there is a simplicity under all these chaotic details we see when we look about.

As the world I see certainly has a lot of confusion in it, isn’t this a religious perspective by physicists? More creeping Secular Humanism? Isn’t it just another religion?


The difference between the faith of a physicist and that of a religious person is that the physicist wants data what will displace his/er faith. The physicist wants observations that will explain the mechanisms of why we see what we see–even if they are mind-bending and paradoxical–the physicist wants to expose his/er faith and dispel mystery.

The religious person wants faith, wants to hold on to mystery.

If I might get all meta: The physicist has faith that there will always be plenty mystery; that there is no risk in explaining things.


©2015 Kent Borg

Will Donald Trump Split the Republican Party?

November 24th, 2015

Splitting the GOP: Refused

A few years ago, when the GOP first lurched right and so seemed to be dismissing any reasonable chance of winning the presidency, I figured they needed to split in two, they needed to kick out the crazies.

But they didn’t, instead they all adopted this crazy attitude, and I forgot about my constructive idea. The GOP wasn’t following my suggestion.

Now it Might Happen

Not saying it will, but I can see a way it might: Donald Trump.

Observers have puzzled to figure out what Trump’s ideology is, and other than being a brash showman, it has been hard. I think it is because he didn’t much have any political ideology. But now, as he shifts right–shifts far right–he might be finding his ideology. For the last week The Donald is having a good time with fascism; he might decide he really likes it.

So if a GOP split happens it won’t be because crazies were pushed out but it might be that they leave to follow Trump.

His chance of winning the GOP nomination is not zero, but it is low. If he looses he might go third-party. He said he won’t, but he could change his mind. If he walks, and is the pied piper who leads the brownshirts and tea baggers to form a white-wing party, er, right-wing party, what is left behind?

A quieter and rather shaken GOP, free to be sensible and reality-based again.

Not saying it will happen, but it could. And it would be good for the country, good for the world. The GOP will win the presidency again–if for no other reason than the Democrat might trip and fall. I want a reasonable GOP to return before that happens.


©2015 Kent Borg

Benefits of The Donald

October 19th, 2015

I missed the first Democratic debate of the 2016 election. I wanted to see it and being stuck on an airplane, with CNN, while it was on, seemed a fine coincidence. Except my Virgin America flight from LA to Boston was missing a quarter of the channels it was supposed to have, including CNN. And they didn’t want to reset too much of their entertainment equipment because the Dodger’s playoff game was coming in loud-and-clear and for some reason my fellow passengers seemed more interested in that.

From what I gather, the dog didn’t bark: Hillary is alive and Sanders is real. But that’s news. Likely it keeps Biden undecided until it is too late to get in.

Oh, and the guy from Maryland maybe has some bite.

The other two?

I’ve always liked Chafee, but he is disqualified for being funny looking, he can’t be elected president in 2016 without being more attractive on TV. A shame. I also saw a spin-room photo where he was being interviewed, surrounded by…one lone reporter. In the same room where Bernie was mobbed. Sad.

I used to like Webb, but I forget what earlier silliness from him was my letdown. But I am over him.

On to The Donald

As I have often repeated, I don’t dare root for a crazy-but-weak GOP candidate because in a two horse race the most embarrassing and lagging nag can win–if the other horse trips and falls. But that reticence doesn’t mean I can’t appreciate some aspects of GOP infighting and stunts.

In this case The Donald is taking on W’s reputation. It has been almost as if George II wasn’t even president on September 11th, 2001. Until Jeb! said “At least he kept us safe.”. A few liberals jumped on that and no one notices them.

But now The Donald is hitting Jeb! with the fact of what his brother’s job was at the time: President of the United States of America. And in true Donald-style, he isn’t being gentle about it. Legions of Ignorant Americans (who think the movie The Martian is based on a true story) are learning that George W. Bush maybe was a bad president. (“Really!?”)

Now if The Donald would only take on W’s torture, kidnapping, and arbitrary imprisonment.

Okay if he wants to leave alone W’s hollowing out of the federal government by driving scads of competent talent into retirement. (A dozen or so years from now, ask Barack about the consequences of that–he might not give a straight answer, but watch him take a deep breath and try not to roll is eyes.) Also okay if The Donald doesn’t want to touch W’s horrible deficits, before he very nearly plunged us into another Great Depression.

At some point it becomes implausible that George II was that bad, so best keep it believable…

But I would like the torture, kidnapping, and arbitrary imprisonment trotted out by a showman of The Donald’s calibre. Please?

©2015 Kent Borg