Author Archives: Me

drunken poste

Posted on by under Uncategorized

I’m drunk.

I just took my prescription pills. One of these pills is a soft-gel that I found to work the best when I chew it before swallowing (probly works better in the upper digestive system than lower). Maybe someday, we’ll have nutrition pills that require chewing like real food. And we’ll just chew on pills for sustenance.

Anyways, I wanted to post something I was discussing with a really good friend last night. The models and algorithms that we were interested in require analysis and optimization in these properties:

  • Latency and Throughput (Needs to respond fast enough, and supply solutions fast continuously)
  • Relevance. (This is often ignored because most of the time designers of algorithms will not deviate with from the problem)
  • Performance. (Actual performance in the sense of accuracy, coverage, generalizability, stability and robustness, and min-max properties, etc.)
  • Cost: how much is it goina cost to apply an algorithm satisfying the above three types of specifications. This should also include implementation cost. Something like TCO, possibly minus the O at some point. 😦
  • Long term performance: What it will do in the long term. This includes every thing else that is more global than what is required to satisfy the above four and contain things like environmental issues, game theoretic problems, and the likes.

This may seem completely out of context to everybody, but that one friend who’s email system suddenly all barfed, here they are. the properties that I want to keep track of.

goina drink more now. wanted to get that down on electrons before I pass out.

Documentaries and Boardwalk Empire

Posted on by under Uncategorized
I just watched the premier of HBO’s new series Boardwalk Empire(Wiki). The first episode was directed by Martin Scorsese.

I enjoyed watching it. However, I’m trying to come to terms with the drama, the history of the 1920’s America, and Big Business. The show masterfully puts us in the right mood with Enoch speaking at a women’s temperance movement. He speaks for universal suffrage, as any sane politician would do in a room full of women just around the time when women gained the right to vote. In matter of minutes, the audience is completely enveloped in the period, the sounds, the smells, and the taste of alcohol, the politics, the money, and the crime. Gotta hand it to martin Scorsese, master film maker, awesome show!! Oh, and the Italian mobster, the

However, entering into this show, I had just watched a documentary called The End of America hosted by Naomi Wolf, author of a book by the same name. For a moment, I won’t talk about all the parallels she draw between post-911-USA and the Nazi’s (the Swarzwasser private police force and the modern day Black Water), and I won’t think about how the guy who had business rights to the world trade center was(and still is) a Jewish person(Larry Silverstein). It all suddenly seem to me that 911 is mostly a war to destroy Jewish properties; and as a second purpose to enforce a police state, justify secret and illegal spying on citizens;and as a third reason to play out all kinds of sex-kinked tortures; and lastly a white-dominance thing…

But most importantly, from the documentaries, I learned that there is the possibility that the prohibition, was put in place not for practical/moral/ethical/social reasons, but for the exclusive purpose of forcing people to use Gasoline. The wiki entry for John D. Rockefeller does not mention the prohibition. But apparently Ford (the founder of Ford) had decided to make his cars(Model A,T) operate both on gasoline AND alcohol. For Ford, this technological feature was something that was very useful at the time. Farmers can pull up to any farm and ask for some home distilled alcohol and the car would run. For Standard Oil it would mean the end of lucrative business, Rockefeller’s oil company lobbied to pass prohibition laws, and that’s where Al Capon makes his cameo in Boardwalk Empire…

The prohibition ended, in fact, the very year after Henry Ford gave up and stopped making his cars compatible with alcohol. Coincidence?? I think not!!

So what?! you say, what does it matter that these dead people did some lobbying to get their ways?

Well, it puts things in perspective… It also makes a manufactured 911 much more believable. If Rockefeller could pass a law in a democratic country full of the freest people in the world so that they can’t drink any more, I can imagine those people jumping from the world trade centers volunteering to be victims. This must be the power of… no, not power of the devil, the power of money at work!

And knowing that Ford was an innovator who gained money by building technologically superior quality products and not through entirely selfish politicking, I feel very proud to be driving a Ford to work tomorrow morning!! Designed and built in America by Americans!


I did not know about Bhopal

Posted on by under Uncategorized

omg, I did not know about Bhopal. Apparently, a pesticide company Union Carbide India Limited had a plant in Bhopal India, a city located smack dab in the middle of India. The pesticide company had an accident December of 1984, killing more than 8000 people with in one week, and up to 16k people in the following time, 550k injuries, 38k severe, 4k permanent. The release hurt so many people because it was high temperature, high pressure gas released into atmosphere which covered the entire city. It also killed a lot of trees within the first week.

Wow, the poor Indian people.
I learned of this from a documentary The Yes Men Fix the World. The company Union Carbide is now owned by Dow Chemicals(DOW). The Indian people are continuing to pass aid package for the people affected by this disaster. The United States, on pains of economic consequences, refuses Indian any additional aid or payment.
I am at odds with myself on this. I mean, yes, really, the original people who caused this should pay for it. Subsequently, the current owner (as the movie demonstrates) should pick up the tab to improve their own corporate image. (Because China is watching, and Europe, and Africa, and South America… are all places that can afford to refuse Dow Chemicals) Lastly, the United States government should restrict it’s corporations and make finally remediation.
As an American, I feel sorry for the poor people of India and wish that they can be helped. But at the same time, I feel that their government was most negligent, and continues to be friendly with US corporations to an unnecessary degree. The Indian people should pay for recovery because they were partially responsible for it happening by electing a government that allowed this to happen.
As an American, I am certainly not willing to pay in taxes to help them fix this. I mean, I’m not a stock owner of DOW and never was, and never owned mutual funds with DOW chemicals as a component, and I never was part of that decision making process, and not part not because of my negligence but because it just didn’t involve me. Some entrepreneur decided to get the money, goto India, bring a risky technology there, and…, and I don’t know what happened between then and the accident,…, but those involved in the process should pay for it!
If they’re all dead and gone, then certainly the current government should pass legislations to prevent it from happening, followed by technological and financial aid to help clean up the mess. It should be powered by a charity, paid by people who have vested interest in India.
Heck, if such a corporation existed, the “Help Bhopal India To Recover” for profit corporation, I may even by stocks in it.
But I am really sorry this happened to India, and I’m sure it’s happening in China too, but with less visibility…

Apologies

Posted on by under Uncategorized

I feel that I want to write in my blog following my last post apologizing to Jewish people who suffered greatly from Holocaust. I don’t mean to suggest that malice towards the Jewish people during the Holocaust was an inconvenient side effect of failure in communication.

Also, I will apologize to Chinese people who think that I am writing about the cultural revolution and saying that those in power arranged for atrocities to achieve personal or political interest counter to the interest of the Chinese people. I’m sure that was… well, I won’t characterize it here, but to say that I’m not talking about that at all.
However, my confusion and questions about 911 and post-911 decade remains. I mean, the terrorist may have some how smuggled state of the art explosive material onto the plain before boarding, that would explain why they found that in the debris…
Sadly, to me, at this moment, that seems less likely than a government inside job–having watched the movie.
Mmmm.

memories of 911

Posted on by under Uncategorized

I just watched a documentary called “Loose Change 9/11” The movie claims that many of the things about the 9-11 attack on the United States seem to be faked by a certain group of Industrialists who are lead by the Bush family. The movie claims that explosives were used to stage the various crashes and tower… It claims to find evidence of very powerful state of the art military grade explosives through out the towers, and shows shots of explosions happening on several floors as the tower falls to that floor.

There’re somethings that are contradictory about the movie though. If the forces behind the conspiracy that lasted centuries were so powerful, how was it that he is able to make this movie? How is it that they didn’t arrange for his and his crews’ death? (As the movie claims that witnesses of foul play were killed after they came public)
What would be even more fascinating to me, and if some centuries later this is proven, would be for the conspiracy to have occurred not by direct arrangement.
How can that be?
Well, in America, as I am sure is the case else where in the world, superstition are reinforced. Many times I would observe irrational behavior of individuals in power, and could not explain it for many years after (even after all the facts come to light and all parties that were to benefit from the secrecy of the knowledge of the act have fully gained the benefit of the secrecy) The irrational behaviors could still not be explained.
Many times, large companies operate based on “communication skills” that are quite often emphasized in modern corporate world. These skills tend to be unclear, unnecessary, and really really mean things that people say and do to each other in corporations to get their point across. But most importantly, on big important decisions, unclear mumble is the key feature. And if communication skills are taught using these very same communication skills, then would we not certainly get unclear communication on top of unclear language?
I have a feeling that because of a great emphasis on this type of “communication skills”, that somebody, or the whole military/government/large corporate powers may have inadvertently developed a completely different identity–a massive blob of intelligent sub-agents, who will transmit messages they were given, and act on messages that they receive. But these message’s original intent were not transmitted (intentionally for purpose of secrecy).
So, this massive chaotic intelligence built on top of units and organizational units of humans, groups, teams, departments, corporations, squads, platoons, companies, battalions, regiments, corps, armies, … all these hierarchical groups operating in total darkness of true intent of the message and tasks that they were given.
I don’t really believe that money is so powerful that it could move a person, or even a whole group of (democratically elected, no less, professionally trained, no less, highly educated with the knowledges of human civilization, no less) people to do something like this.
However, I can bring myself to believe that this would happen due to lack of global intelligence on the part of each of the units participating in this system. I can believe that each units were eventually given orders to put explosives in place (if that were really the case), and followed the orders as soldiers are trained to do, but that the orders or given by misleading communications to different levels of the military, and that large orders broke down into small orders, and at each level, the order were transmuted in this unclear communication that we have developed…
This idea came from an algorithm in the Machine Learning community called “belief propagation” which will lead to a global solution when each node of a Baye’s Net send it’s current belief to the other nodes it is connected to. Because each unit receiving orders are receiving unclear orders that they can only pass on what they believe was the intent of the order. And through this kind of message passing algorithm, 911 was arranged, executed, and covered up.
I mean, really, if we’re to believe 9-11 is a self-inflicted terrorism act, are we to believe that the Holocaust was all a hoax too? But certainly I absolute can believe that the human system of the time eventually exploded due to lack of global situation awareness by each individual participant and unit of participants, and that caused holocaust, and that the Germans, Japanese, and Italian people are still people, just as Americans are still people even after 9-11 and all the wars that it caused.
…and all the economic problems America suffered, and probably continues to suffer today.. All that, due to a maladjustment in our human system…

Naive Bloomier Filter

Posted on by under Uncategorized

I’ve been playing with the Naive implementation of the Bloomier Filter recently. The Bloomier Filter is just a stack of several Bloom Filters on each level. (Well the Naive version described in the first few pages of the paper is any ways) Is it me or is it that I just realized that there is a huge savings that can be had by allocating the second layer to have size FPR*N where FPR is the guaranteed False Positive Rate and N is the total number of items we inserted. So, but how small is the set of items that may make it to the next layer? Well, the expected value can be calculated as FPR*N. I guess if I am smarter, I would go ahead and calculate the variance of this RV, and compute how much variance to the overall expected false positives I add by making this second layer just a fraction of the first layer….

I guess he probably says this in the paper some where, but alas, took me a day to figure out.
Actually in reality it is very rare for me to break out of the first layer unless I construct a really really obnoxious case where I tell the BloomFilter that I plan to put in 100 items, and that I want 0.00001 FPR, and then put in 100k items, then it explodes and generates extra false positives.
This is such an interesting data structure. Have you had success with this algorithm?

An experiment in Obamamania

Posted on by under Uncategorized

Okay, I’m running out of cash. I admit it. I’ve been spending like a crazy person since my purchase of a house here in the slums of Redwood City. My bank accounts are on the verge of depletion and I’ve been paying 10% APR on my credit cards for several months now.

Why I Spend so Much?
It’s kind of surprising how you expect to spend no money on the house other than the closing cost on the Good Faith Estimate, but then later, once the house closes, the moving costs, the blinders… ugh, where did all that money go? all I can recall is buying some blinders for the house, and yet my monthly expenditure, aside from the mortgage payments and utilities, has gone up by at least $1000/month.

The Blog Entry
Anyway, I decided to try to take advantage of President Barack Obama’s Credit Card reform. According to this article on creditcards.com, when the credit cards will receive your monthly payment, it will take the “minimum payment” and do as it please with it, and the remaining money in the payment will be required to go towards paying off the portion of the debt that has the highest interest!!

Let’s say that I have a credit card that I’ve used to make “auto-payment” of about $500/month to utilities, phones, tv, and the likes. These auto-payments occur every month, and I have arranged for the credit card company to withdraw that exact amount from my bank account monthly. If I do not pay this amount on time, my interest rate is something like 12% APR charged monthly at 1% a month.

Now, I have also received an offer of $5000 12-months 0% APR Promo-check from the same credit card company.

Prior to this legislation, my monthly auto-payment to this credit card will go towards paying off the $5000 0% APR debt, and the $500 I spend every month will accumulate and charge interest until the $5000 is paid off. So the payment/balance looks like this for the first few months

Month 0% APR balance 12% APR balance Payment Total Balance Interest Charged this Month
1 $4500 $500 $500 $5000 0
2 $4000 $1000 $500 $5000 $500 *.01=$5
3 $3500 $1500 $500 $5000 $1000 *.01=$10
4 $3000 $2000 $500 $5000 $1500 *.01=$15

Today, under the new rule, the same charge/payment that I make monthly would produce a table like this:

Month 0% APR balance 12% APR balance Payment Total Balance Interest Charged this Month
1 $5000 $500 $500 $5000 0
2 $5000 $500 $500 $5000 0
3 $5000 $500 $500 $5000 0
4 $5000 $500 $500 $5000 0

So that 0% apr promotional charge is a real 0% APR.

Now, I don’t believe my eyes when I read this “policy update” from the bank. So I found the legislation. Here’s a snippet of the relevant portion:

And here are some links to the text of the bill passed:here, here, and here. 111th Congress, H.R.627;

I guess even after researching into the bill and reading the text of it, I am still sceptical. Why has the Obama administration not spend any marketing money on the passage of this bill? It would surely stimulate the economy by making more people take advantage of the true 0% APR? There must be a catch that I haven’t seen yet. I’ll probably get slammed by a loss of job, bodily injuries, numerous car accidents and traffic tickets, tree falling on my house, etc, for taking advantage of this 0% APR. I mean it’s happened before that some thing random and rare happens every single time I used one of these 0% APR checks to interrupt my monthly payment. It’s almost like the efficient market is efficient in taking my money in super-natural ways.

But despite my misgivings about the free market, the whole credit card industry and Obama brand of hopus-believous, I have taken one of those checks on the credit card I describe above and cashed it.

Call me stupid.

Call me gullible.

But having hope is right? I hope it’s right…

12% interest is worth the price for even 1% chance of discovering a change that we’ve hoped for and can believe in.

The Greatest Magic Show on Earth

Posted on by under Uncategorized

According to the Christian Science Monitor, “The Chinese traffic jam that has lasted for over two weeks has vanished overnight.” The article cites both MSNBC and French press AFP as source.

More supprisingly than the magic of 60 miles of traffic disappearing is the fact that western media did not make a glib remark about how it’s another communist PR ploy, and how they mobilized local farmers and burried the 60 miles of cars to fool the western media into believe the incredible powers of the Chinese communist party.

I mean, the article does go into detail about how bad the traffic is in China, which is completely fair. I just cannot get used to the neutral tone of this report.

The Mechanisms of the Hack

Posted on by under Uncategorized

I am attempting to demonstrate to every person who has basic knowledge of the computer that it is fairly easy for another employee at my company to accomplish these acts remotely:

  • Screen shot of my computer at work.
  • Take a video recording of my screen at work.
  • Take a video of me or anything else that is currently in front of my laptop with the built-in camera.
  • Make an audio recording from my laptop’s microphone.
  • Send timed key-stroke sequences to my computer as if I pressed those keys.
  • Send mouse movements to my computer as if I moved the mouse.
  • Interact with application on my screen as if they are sitting in front of my computer.

There are many over the counter software that facilitate this on a modern computer. If you do not believe me, go ahead and download these software and try them for free. You will need two computers on the same network to see how you can manipulate a remote computer.

  • On the Mac, Viola is an application that allow for recording of the screen.
  • On the Mac, Command-Shift-3, and Command-Shift-3 will result in a picture of the entire screen or a selected window, respectively to be saved to a file on the desktop. 
  • On Window machines, the [Print Screen] key and [Alt]-[Print Screen] will result in a picture of the the entire screen or a selected window, respectively, to be saved to a file.
  • On most OS’s VNC is a program that allows you to see and control a computer remotely.
  • On most OS’s Synergy is a program that allows you to control mouse or keyboard from a remote computer system.
  • On XWindow systems, XTEST from XAutomation package has a command called xte which produces key stroke and mouse movement ad specified time without a person actually moving the mouse or keyboard
All I wish to demonstrate in this blog entry is that you, any of you who knows how to open a browser window and read a blog entry, can go to these websites, download a program, and read another person’s email, send email as another person, or fake typo’s while he is typing.
Next time I or anybody else speak to you and say: “hey, you know? I think xyz at work has been spying on my email.” or “My work has been sabotaged by my cube-mate.” That you do not dismiss him as an escapee from an insane asylum.
This issue must be dealt with openly and rationally.

re-repigged

Posted on by under Uncategorized

So, here’s the solution to the question in my previous post.

The most obvious way is to perform a partial aggregate and then merge the results.

A = group TABLE by (f1,f2,f3, ((long)(random()*100));
B = foreach A generate FLATTEN(group), SUM(f4) as sf4, MIN(f5) as mf5, MAX(f6) as mf6;
C = group B by (group::f1, group::f2, group::f3);
D = foreach C generate FLATTEN(group), SUM(f4) as sum_of_f4, MIN(mf5) as min_of_f5, MAX(mf6) as max_of_f6;

simple and effective. Does not require extra programming in java at all. This algorithm can be applied to all associative reducing functions. But associativity is not a necessary condition. The Algebraic functions in Pig actually allows for more general implementation where by an operation is divided into two stages. The first stage is required to produce tuples, which the second stage then processes.

You will probably never meet a problem that produces a bag larger than one compute node can handle. I mean I work for one of the fastest growing internet eCommerce servicing companies and our processing never even come close to seeing this problem.

But, in case in some distant future, when you encounter bags in computation that overflow one or even two compute nodes, and even at that time Pig/Hadoop still does not have a built in mechanism to deal with this automatically. So the above is an workaround.

Hope this helps you in your work or play when I’m long gone and dead.