Category Archives: Clandestine Activity

IG and the Quantification of Privacy

Posted on by under america, Clandestine Activity, Innovation, Privacy, Technology

A while back, I talked about computing IG–information gain–by clandestine methods via an otherwise secret(personal) email. I will point to some other prior blogs entries about what can we reasonably consider private and some reasons why I think it’s bad (Because it removes competition….

The basic challenge is this: If your competitor can spy on what you do (unilaterally) then they will never be motivated to innovate. Their key strength will be their ability to hack your secrets and they will work hard on that, but not on how to build a better product or cure a disease or solve a new problem. If you can both spy on each other with perfect information then there is no need to innovate, just calculate the equilibrium and aim for that. If you can disinform your opponent then all your effort will go into disinformation instead of innovation. Basically it is much easier to do something sneaky and cheat than to do the right thing and innovate. This is why the government, a non-competing body whose interest is to make sure everyone compete (at least in America government this is the case), should provide for information security.

)

I realize in retrospect that IG may not make sense to most people based on the formulation I laid out. Let’s review. IG is the change in entropy from a state without additional knowledge to a state with knowledge

IG = H(secret) – H(secret | private email)

This measurement seem to be of a quite abstract concept of entropy–a unitless measurement. Why would I think this useful for any reason other than that it is called “Information Gain?” Well truth be told, what I had in mind was more of the IG from machine learning literature: Class purity after conditioning on some private information. It is actually used more as a measurement of correctness of predicting discrete output than abstract change in entropy of distribution after conditioning. I will refer reader to these excellent introductory books regarding “classification” algorithms.

… Some days passes and the books will hopefully have arrived on your desks…

So the example is if my secret is the probability that I will have Chinese food tonight. Let’s throw in several more classes, say Italian, Mexican cover 99.9% of all possibilities. This probability may be internal to me. Or it may be an externalizable model like I will toss a three-sided die and figure out what I will eat tonight.

Actually, this system forces us to think of a new class. I will call this new class the innovation class. It covers all cases where something new might happen, such as tonight when I went off on a tangent and forgot to eat dinner completely. Or I might be abducted by Aliens for demanding privacy, Japanese paramilitary for blogging, or God for thinking all these awful things. The fact is, I do not know what will happen, but what I do know is that things I don’t know will happen. So the class is called IC, Innovation Class–now we have a 4 sided die: Chinese, Mexican, Italian, IC; Let’s write naively that the probability for each class is:

Chinese Mexican Italian IC
33% 33% 33% 1%

The formula for the entropy of these classes is written as:

-H(Dinner)= p(Chinese) * log(p(Chinese)) + p(Mexican) * log(p(Mexican)) + p(Italian) * log(p(Italian)) + p(IC)*log(p(IC))

the above evaluates to almost the maximum possible entropy in three-class situation: H(Dinner)= 1.6499060116098556

that’s it. that’s the formula for calculating entropy that we will use repeatedly. Now, suppose that you have read my email to my wife saying “oh man, look at this great deal on groupon, 50% off on Indian food right near our home” What is the right thing to think about the distribution of my dinner?

P(IC)=99%

Indian food is not Chinese or Mexican or Italian, but we have thought of that and put in IC to account for it.

Chinese Mexican Italian IC
10% 10% 10% 70%

-H(Dinner|private email to wife) = p(Chinese|private email to wife) * log(p(Chinese|private email to wife)) + p(Mexican|private email to wife) * log(p(Mexican|private email to wife)) + p(Italian|private email to wife) * log(p(Italian|private email to wife)) + p(IC|private email to wife)*log(p(IC|private email to wife))

gives us the conditional entropy of probability of dinner after reading my private email. This entropy H(Dinner|private email to wife)=0.09596342477405478

IG(Dinner; private email to wife) = H(Dinner) – H(Dinner|private email to wife) = 1.6499060116098556-0.09596342477405478=1.5539425868358008. This corresponds to an IGR of 1619.31%, that is, 15X more information after you saw the email than before.

 

Great! so now we know how much information is gained by reading that one private email of mine. This number, I think quantifies my loss of privacy.

 

Btw, this innocent example contain some hand waving. H(Dinner) for example is something that we may or may not know. Most people have trouble writing down a distribution for dinner choices. also, P(Dinner|private email to wife) here written as a table contain assumed values. What if after reading my private email you feel that P(IC)=85%? Who is to say what the reality of this probability is? This is why I felt that this model will not make to main stream legal system because the link between private email and the actual secret itself is not so obvious. You might use naive Bayes as the definitive of reality (refer to chapter in books or wiki), logistic regression, decision trees, or you might use something else… You may even use a distributions system like SVM or god forbid rule based systems…

If you understand this computation above, then it will be easy for you to understand the continuous version. Let dinner be a continuous variable, we can still write the same expression

IG(Dinner; private email to wife) = H(Dinner) – H(Dinner|private email to wife)

and it would have the same meaning. How far are we from the truth. This idea, btw, is indeed partially inspired by the name Information Gain, which also goes by Kullback-Leibler divergence when computed over distributions. The above formation exactly with the exception that “private email to wife” is a distribution, say, perhaps, my emails are generated randomly.

KL( Dinner|private email || Dinner )

But KL divergence does point us to some other interesting characterizations. Divergence–distance without some properties of distance. Namely that it is not a metric distance:

* Nonnegative dl(x,y)>=0:  yes

* Indiscernability: dl(x,y)=0 iff x==y: yes

* Symmetric dl(x,y)==dl(y,x): NO

* Triangle inequality dl(x,y)+dl(y,z) >= dl(x,z): NO

This has some serious implications regarding this formulation of privacy. Somethings that we naturally think should make sense do not.

Let’s say I have two emails, e1 and e2, and let’s say dinner is still the subject of intense TLA investigation:

KL(d;e1) + KL(d;e2) != KL(d;e1,e2)

All private information must be considered together, because considering them separately would yield inconsistent measurement of privacy loss

Let’s say there’re two secrets, d1 is my dinner choose and d2 is my wife’s dinner choose

KL(d1;e1,e2) + KL(d2;e1,e2) != KL(d1,d2; e1,e2)

All secrets must be computed together, because computing IG separately and adding is not equal to the total information gain.

Let’s say we have an intermediate decision called Mode of Transportation (mt), and it is a secret just like my dinner choice.

KL(mt;e1,e2) + KL(d ; mt) != KL(d; e1,e 2)

The intermediate secret can be calculated, but again, it must be calculated carefully and not by additive increase of IG.

Bummer, but fascinating!! But we we must make some choice about how to proceed. Knowledge about the nature of information (and especially electronic information), I believe, informs us about how we make choice in our privacy laws:

 

  • Should the whole data be analyzed all at once?
  • or should we only allow each individual’s data be processed all at once?
  • or should we only allow daily data of everyone to be processed together?
  • or should we only allow daily data  of each individual to be processed separately?

Each of these choice (and many other) impact the private information loss due to clandestine activities.

 

 

Activities of a Clandestine Nature (4 of…

Posted on by under Clandestine Activity, Privacy

Recently I heard a really great argument against clandestine activities: It perpetuates the practice, the habits, the policies, and the systems that facilitate clandestine activities. Being something that we don’t want, systematic clandestine activities should be pointed out, certainly be strictly live-audited by unbiased third parties.

Why is clandestine activities bad? The truth of the matter is that knowledge begotten of clandestine activities are inherently out of context and incomplete information. Why spy on my computer, when you can walk up to me and ask? When you take a small slice of what happens, you will surely miss the whole as the whole is not represented by some of the things that you are able to see as a clandestine agent.

Previously suggested problem that those taking part in clandestine activities will as all things in nature fall into the path of least resistance. Some day, we will just water board every person we suspect, I mean why not? I’m sure there’s an email I sent once that says “I hate you” or “I’m gonna kill you” or “I hope you die”. And my constant opposition of clandestine activities is surely sign that I plan something and desire that no one sees it.

What is the difference between these series acts: passing a secret law that permits some person unknown to me at a time unknown to me read my emails, gather all my past school and employment records, find copies of all emails I’ve ever sent by USPS, and analyze all information about all my past employment and my family and friends, and these second series of acts: passing a secret law that permits some person unknown to me at a time unknown to me knock me out (perhaps it’s already happening in my sleep ? or even on flights, god knows how often I fall asleep quite inexplicably moments before push-off, with two air jets blowing cold air at me and two reading lights shining down! and only to come to quite suddenly for no reason), and torture me and get that information?

Well, you say, there is collateral damage, you feel pain when you are tortured but you do not feel pain when your email is being scanned. This ought to be the most humane way of getting the information from you. Why are you not on your knees thanking all the people whose hard work went into making it so that you are not water boarded? (rightfully or not)

Aha, thank you President Obama! The constitution should save us… Let’s see, according to wiki it implicitly presumes innocent for US citizens until proven guilty, but it provides wide leeway for authorities to investigate when suspicion is arouse.

We cannot pursue it through cruel and unusual punishments(8th amendment) as reading my email can hardly be construed as cruel and unusual… even in my interpretation. Although I can imagine some feel it is cruel.

It appears in the Fourth Amendment against unreasonable search and seizure:

The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

It also fall under Fifth Amendment of due process:

No person shall be held to answer for a capital, or otherwise infamous crime, unless on a presentment or indictment of a Grand Jury, except in cases arising in the land or naval forces, or in the Militia, when in actual service in time of War or public danger; nor shall any person be subject for the same offence to be twice put in jeopardy of life or limb; nor shall be compelled in any criminal case to be a witness against himself, nor be deprived of life, liberty, or property, without due process of law; nor shall private property be taken for public use, without just compensation.

There should be a Grand Jury of my peers selected uniformly at random who when presented with evidence agree to the search and seizure of my information. I should not be deprived of my liberty and (privacy) property without due process of law. And of course the Ninth Amendment says that we may have rights beyond those listed

The enumeration in the Constitution, of certain rights, shall not be construed to deny or disparage others retained by the people.

I should cover my behind and say, you guys in law enforcement are doing a heck of a job, which is much appreciated by present author. And I really hate all those other people who invade my privacy. It’s just that I might have a small chance by conventional means (law suite, legal protests, policies, etc.) of changing those things you do that I don’t like, and I do not have methods to affect those others.

Everyone who do take part in clandestine activities all feel absolute righteousness as they proceed in invasion of privacy that I do not want them to. Their feeling and their intention absolutely annoys me in addition to their act of invasion. Perhaps we should define invasion of privacy more formally so that these feelings about their feelings are processed rationally. If they can define information theoretic brain death, why can we not define more precisely what invasion of privacy is? What is personal privacy beyond those facts(bits, characters, words, sentences…) whose association with me is information that may cause me harm? regardless of harm, can we take the entropy of those bits and say that is the quantity of privacy lost? Actually, we should take information gain from a representative population and that is the information I lose–those that you gain. The privacy loss as defined (the negative of your information gain by reading my email from knowledge of all emails) actually only qualifies the privacy. It actually does not quantify it sufficiently.

Sadly, this very truthful and fundamental definition takes us a short ways. If you were an English major trying to find new phrasing of something, or if you are a VC looking for new cute company names, this will definitely find information detrimental to those trying to keep it private. But if I am someone plotting next Taliban attack, or someone discussing 21st century is a Marxist century, then the naïve information loss does not help as much as you would like it to (Certainly my email would give away less information under this definition than XYXYXZZZ.com inc) If everyone writes emails using words representing their true meaning equally and every one has same amount of total information(private+public) associated with them then reading your email and reading my email decreases our privacy equally. So we have parameters I_pr for private information, I_pu for public information.

We should compute using Bayes’ rule to compute

P(I_pr|my emails, others’ emails, I_pu) =

P(my emails | I_pr,I_pu, others’ emails)*P(I_pr,I_pu, others’ emails)/P(my emails, others’ emails, I_pu)

and

P(my emails|Others’ emails, I_pu)

and we can then calculate the information

IG(I_pr; my emails|others’ emails, I_pu)

based on these distributions, pending specification of relevant linking functions or mechanisms. But the problem with this much more convincing information gain is that you will never convince anyone that the link functions is representative of you. Too complicated for constitutional purposes for sure, and the courts will surely not be empathetic enough to follow the math… Maybe next century when everyone’s played with IG and done some modeling in grammar school.

For another example the number $54,102,299.14 and the number $14,541,022.99 relieves me of the same character-wise entropy privacy, however are quantitatively different. We need to rely on some oracle magic. Suppose there is a most concise way to describe the entirety of my privacy, say H containing a series of bits an oracle produced. Your knowledge of H would be your complete knowledge about me. ergmum, we should have a vocabulary of engrams, minimal cognitive elements… H is a series of engrams that is the complete knowledge about me–it’s finiteness is not specified. Let’s also suppose that my emails (the thing that you use to access my privacy) is encoded by the same oracle using the same engram language producing E the complete knowledge about my emails. |H| is the theoretic maximum privacy I can lose, H*E is the information that I actually lost (inner product like operation for vector space, TBD for strings, perhaps LCS for a special oracle). It remains only to calculate distance(such as edit_distance(H,E) for strings and euclidian_distance(H,E) for euclidian spaces) which is disinformation you gained by reading my email. H*E/|H| is the ratio of my privacy lost, H*E/|E| is the truthfulness of my emails.

It remains to be seen how to find an oracle, the definition of the engram language, operations over it, campaign to enact law to account and compensate us for the privacy lost, etc. However, I am really really wishing that all these clandestine activities are like zits in the face of growing humanity reaching adulthood and will blow away as our vitalities settle into their respective places.

Things of a Clandestine Nature (3 of…

Posted on by under Clandestine Activity, Privacy, Uncategorized

Money
There should be money value to losses of privacy. Every time an organized clandestine action is done onto me and that their actions is proven wrong, there should be consequence.

Having suspicion is a right, a duty of these law enforcement folks. But acting on an incorrect suspicion(whether justified or not) should carry consequence. Just as they are rewarded for following a hunch and catching a crook, there must be punishment for following a wrong hunch and negatively impacting a person’s life.

In fact, I feel that even the access and analysis of my private information (email, files, my personal space such as my home, the airspace above my head, signals sent into my person and my possessions) these invasions of privacy must be punished when proven to be wrong.

Each violation must state hypothesis and the condition of test requiring invasion of privacy. If test proves hypothesis wrong then a punishment is assessed. If it is proven right then a reward is given.

Every kilobyte of my email you read, you should be paying me $x. If you retain the data then you will be charged $y/year.

This belittles human privacy rights, but it is one way that we can use to quantify, regulate and monitor the clandestine sector.

Things of a Clandestine Nature (2 of…)

Posted on by under Clandestine Activity, Privacy, Uncategorized

What is power? What is privacy?

I am not a very forceful thinker. I find myself thinking of every matter from multiple perspectives. More often than not I argue myself out of my own position.

The organized clandestine activities are organized for a reason. A large enough and powerful enough intelligence has recognized its necessity and has facilitated it’s existence. That originator of this clandestine organization mandates that the clandestine activities be clandestine in all ways I have described in my previous post.

The consequence of this is that the clandestine organization must satisfy a certain level of service. Similar to the concept of an SLA, the clandestine organization must obtain information (in stealth) within a certain time period of it becoming known. When it masquerades, it must succeed in fooling all involved into believing that the party being masqueraded is truly doing it.

This ability is a power. And in effect it is not a power if it cannot be wielded freely. Similar to my right of speech. If I speak, and my voice is interfered with every time, or if my blog is unsearchable by some search engine trick then effectively my right of speech has been impinged upon. Conversely if here is a regulation that restrained the clandestine activity, then it decreases it’s usefulness.

Let me give a simpler example to illustrate. If we are to guarantee the power of these clandestine organizations the ability to masquerade as me typing into my computer, then they can always type into my computer. If the operator (the member of clandestine organization) is have a bad day, if he is having a seizure, if he is unhappy that I oppose the existence of his job, if he made a typo and affect what I type into the computer as I work, then that is allowed.

Because if we regulate these abilities to try to prevent his foul mood or desire to keep his job in affecting my job performance then we have not given these clandestine organizations absolute power, and consequently they cannot function effectively, right? Consequently they are not liable for major failures like 911 right?

If any of the theatrical depiction of clandestine organizations are any where close to reality, it is safe to assume that all of them are insane. They are all paranoid and all able to argue for 100% absolute clandestine power!

Oh, and let me be obnoxious for once, having be target of obnoxious behavior so often… The aurora shooting, why don’t the people watching my screen and analyzing my blog and causing typos in my keyboards and making my mouse fly around weirdly at work and home, why don’t they go and wreck some havoc in an actual bad person’s life? Why don’t you go and prevent a really bad thing from happening? instead of secretly watching my activity?