I’ve been thinking about autoactivations recently. This is one of those great innovations that stood up to the test of time, it still works after a lot of debugging and exposure to new data and models.

I find that I have been referring to autoactivations as pre-activation because they occur to deep neural nets before the parameters are actually mixed with input data (or previous layers activations) but if you look at the two expressions:

1. To pre-activate a parameter means to apply nonlinearity before it is used. e.g. preheating the oven, the suffix is a verb and happens before something else.
2. But a pre-activation is actually an adjective meaning before any activations. e.g. pre-trial motions. It’s suffix is a noun and becomes the subject to be preceded.

And actually similar problem applies to ante- prefix. So, to avoid confusion, we should probably refer to autoactivations as foreactivations and to foreactivate the layer. This prefix also means before and it works both for nouns: foresight, foreknowledge, forethought, forerunner, foreword, foreman, and also works for verbs: forecast, foreshadow, foredone, foreshorten, forewarn, forestall, foredoom. In each case the suffix is always the prior thing before but never preceded by another.

So, let us all try out foreactivations and related approaches. The speed up in training will surely be a good thing for humanity, at least, we won’t be consuming as much energy training models without foreactivations.

😀

I just read a rather alarming article about Asian Men harassing Asian women for dating and marrying white and other non-minority races. Admittedly this is alarming because I may have expressed jealousy towards some white kids having more tofu from Chinese girls than I. And also, some of them seem to know more about Chinese culture than I. I think casting aspersion on the motives of Asian women becoming embroiled with non-minority races has certainly crossed my mind a million times if not materialized into blog entry. (It will before this entery publishes)

Personally, I don’t feel it unethical or illogical to have these thoughts. I’m not terribly ashamed of them either. But by the same token, a lot of the envy come from seeing some very very very successful friends and acquaintances make Asian-women-non-asian-men relationship work. There are really nice best-of-both-worlds marriages starting with TWO ceremonies(parties)… Two homes on different continents to take new family members to… Two successful cultures, religions, languages, And those beautifully exotic hybridized kids… Love them to pieces! So much so that the feeling really hurts some times.

But that is not to say harassment is right. Asian women probably suffered more than non-Asian women throughout history. But today they deserve the respect of a fellow human being. We, Asian men should never harass Asian women for their independently made decisions!! And incidents of harassment should of course be treated with urgency.

Asian men do have a hard time in America, as the female author of the harassment piece notes. Although it sounds a little bit belittling, these weaklings who cannot compete reproductively resorts to emailed harassment of lost mates. I mean, god, as an Asian men, reading this piece makes me hate Asian mankind even more than I already do, having all my inferiorities and inferiority complexities. It really doesn’t make me respect Asian women more when she publicly writes these belittling pieces about a group of people whom she chose not to associate with.

Yes, just like women have to fart and poop on rare occasions, I believe non-minority women are capable of racist behavior too. (Not speaking from personal experience, of course, every women I know are not racists, but in the large there must be enough racism to make the pairing challenging) Therefore the celebration of those few successful Asian-men-white-women relationships should not be disparaged. The lack of celebration for Asian-women-non-asian-men relationships is absolutely no reason to diminish the success of those who ventured and succeeded.

But I am a bigger person.

I think a rational concern, if I were elder to an Asian girl, would be the balance of power. Yes yes yes, there’s love, but a little balance never hurts. Perhaps one thing that Asians don’t understand about each other is what we think of each other’s power in a relationship. Who decides where to live? Who decides what to eat? Who decides what to drive? Who decides when to have kids? Who decides how to spend money, Who decides …

Asian men may imagine that in mixed relationship, the girl lacks power, that she give up rights and freedoms, those she would enjoy in a racially equal relationship, for … whatever it is that she does it for with the non-minority, maybe better sex? Asian women, probably think they have all the power in the relationship. I mean if she doesn’t feel safe, she probably wouldn’t enter into that relationship (there, I am a bigger person, I respect rationality of girls when it comes to love)

The disparate perspectives probably causes all the angst of the article. So, IMMHO, can Asian girls prove they have access to their rights and privileges? Can the girls show us, and the world that they are in an equal partnership? And Asian men who can’t get Asian women, can you try white girls, I’m guessing they can be quite nice to you if you find your way around surrounding racism and into her heart. My guess is that women of all races are feeling and thinking rational beings.

Ugh, random rambling, hope this note will still make sense when it sees the light of day

I’ve been watching British police story Bodyguard on Netflix.

Did anybody notice that the British show is much much much more diverse than any American police/politics drama. I mean, yeah, sure, I noticed in past viewings that black people in Europe, when caught on camera are very happy and well respected. But this show… It has random Asian on the bomb squad! Mixed races of all kinds at all levels of the government. Police force had more women than men in powerful places… And it all fit naturally–and it may just be British English on my American ears, but it all seemed so proper. (As opposed to American shows in which female leadership seems forced… With few exceptions of course.) Maybe it’s because of their Queen that their entire culture has builtin affordance for women in power.

The bomb squad is a job, in American shows, usually reserved for red-blooded white Americans plus one African American. The Bodyguard shows us a vision of truly diversely integrated society.

I suppose the British empire did have a few more years of experience with multiculturalism than America. Whether it is enslavement or equality and rights creation, they have always been on the forefront of not just behind Americans.

I guess the only grudge I can have for this show is there is not an Asian villain. Sure the bad guy looks German, I guess it’s a running joke about Brits’ (or is it the Englishes’) feeling about Germans… But if the European/British organized crime suddenly had a Mexican or Singaporean syndicate… Wouldn’t that be something?

Perhaps the grass is greener on this side when viewed from the other side?

It occurred to me that this whole death of Jamal Khashoggi’s killing might be a red herring to block news of Harvard discrimination from surfacing. I happen to follow the latter news and it has all but disappeared from my news feeds since the journalists death.

If there is any thing we know about Harvard, is that it hosts a whole lot of foreign donors and famous people. According to news, possibility of future donation is a very influential consideration for admission to Harvard. If this setup is ruined, it could affect future Saudi princes or nobilities from being admitted.

Plausible explanation, imho, of the sloppy way this is being handled by everyone in power as well as many famous people. For example the first related news was Jamie Dillon, CEO of Chase, the largest bank, refusing to attend Saudi investment conference. Mr. Dillon of course has his MBA from Harvard. Uber CEO, Dara Khosrowshahi, Brown Alumni, also has Ivy League connection. The only other CEO mentioned in headline news is Richard Branson, who obviously didn’t go to school. Trump who is down playing this seem to be unaffected by his Penn connection. In the case of Branson and Trump, these guys are so rich and powerful that they may not suffer from influence of Harvard.

Yeeesh….

Read an article about NIPS considering changing it’s name because it is slang for either many anything Japanese people or a nipple visible through clothing.

This reminds me of a conversation I had with someone explaining what a drag suit is, or rather what it isn’t. Drag suit increase water resistance and makes it harder for a swimmer to swim. And this is for normal atheistic swimmers, not a special-interest niche of people.

But I didn’t mind explaining it at all.

For people who grew up with NIPS and not nip or nips, there is no confusion.

Also, the review and data gathered to decide to keep the name was interesting exercise in data science, imho. The committee lamented that half of the opinionated women wants to change and half do not. (Opinionated in that they expressed a non-neutral preference in the official survey.) It would be a displeasure to half of the women both changing and not changing the name. The feminist infection I got from a youthful encounter is rolling my eyes and chewing the word “men!!!”

…

And honestly, of all the people that I’ve ever met who knew of NIPS, CS/mathy folks, many of hem leaders of this field, they didn’t seem the inappropriate kind. To these people, there are far more amusing artifacts of nature, and corollarily, hacks to nature, that render these kind of jokes and computer hacking kind of uninteresting. But admittedly, with diversity come these kind of cultural inconveniences–and by diversity, here, I mean in addition to there being women, the diversity of professional cultures and standards of behavior. When you bring doctors and lawyers and biologists and statisticians and business people, and pliticians, and judges(federal and appeals and…) and drug companies and weapons companies and social network companies to a academic conference… this little problem with a slang, albeit very sensitive, should seems rather small and inconsequential in the face of the more material challenged AI faces.

Let’s hope that problem is addressed systematically as humanity grows.

And CS will survive this! I mean considering it survived explaining why the Neural Information Processing Systems was dominated by Tree based methods, boosting, kernel methods, Bayesian methods for so long a time, and only now do we finally have anything remotely Neural. Nobody minded explaining then and no body should worry now.

Reading an article in the WSJ about the lawsuit against Harvard regarding discrimination against Chinese students. One thought comes to mind, this likability score they use as part of the judgement for admission is admittedly and objectively existent sentiment that affects success and future donation to the school.

Chinese people have things that are extremely not likable. Trust me, I would know, being one and living among them. Not any more so than other races, but one can definitely see the problem. And I’ll abbreviate here and all Chinese-like Asian cultures I’ll just abbreviate and call everyone Chinese for this post.

I mean com’on let’s start with their starchy ricy food tradition, one can’t possibly do well and have the longevity to pay back the school on a diet like that! For a person on a modern diet, being offered a Chinese meal is practically an affront to one’s dignity and sanity. I cannot imagine Harvard ever wanting to recruit Chinese people or people who eat like that. Look at the Indians, they eat long grain rice that doesn’t give you diabetes and slow-burns longer.

Chinese vanity for humbleness is incredibly destructive. Why be humble? why sandbag? when the competition is so fierce, why spend any brain-energy on holding back? No body really knows how strong everyone are because of this extra layer of defense. This age-old tradition is unlikely to go away, and contributes little to the endowment or prestige of an institution.

Chineese problem solving, if one were to sample from the entire US Chinese population, suffered from Communist indoctrination. Really, the things you don’t like are not all traditionally Chinese, a lot of it is modern imported thoughts from Marx, Lenin, Engels, and many others… Chinese people have this social expectation that other people behave well. The Chinese governments have arms–may, tentacles–that can exert power to enforce a certain level social nicety. I remember the indoctrination by all authority figures that one should yield to elderly when taking seats on busses, one should not spit, not be greedy or lazy, one should be honest, and be loyal to the country which will provide you with a nice society to live in. (And maybe some other managed aspects of life that I did not personally experience like conditions of marriage, pregnancy+abortions, and death…)

The same is accomplished in our developed world by non-profits through advocacy, and through operational culture, which, in this case, seems encoded in the admission process of Harvard and similar schools. There is no explicit funded authority whose job it is to make the world a nice place to live. This difference in expectation of others and the government, the entitlement to reciprocation equaling to one’s own regulated participation in society, is clearly a political perspective shared by many Chinese people–the unconscious bias of Chinese. The Chinese demands a society that wants everyone to be nice if they themselves are nice.

There is no authority, except the admission authority in the universe of Universities, they are funded and empowered to choose the members of their prestigious society. Perhaps they fear what would happen to these delusional people when they find out that the world really doesn’t work that way. Society wants what it wants, not what you do or what you want. In some sense, this bias against inclusion of Chinese students is kindest choice, for everyone.

To me, the ideal outcome is this lawsuit is the establishment of unfair discrimination, but with no punishment for these institutions. All we want is to improve life for future generations and we, realistically, expect no greatness for the generation that did and still do that.

I realize, perhaps this last idea is too Chinese. Why the fuck would we seek no damages if the school discriminated? Have I caught a bug in my brain? This pessimism I feel for the possibility of progress in the society is troubling and definitely Unamerican, imho, but is it Chinese to seek no reparation?

In the meantime, though, while American Chinese figure out these things from within, perhaps we should have our separate peace? Let sleeping dogs lie. And if the institution decides to fake it until betterment is made to happen, that would be fine with me.

Just skimmed a public draft of the last chapter of Machine Learning Yearning book by Andrew Ng. I know a lot of people out there might be joking that it’s more like Machine Learning Yawning. But in reality, the execution of these seemingly simple ideas is probably incredibly difficult.

Consider the last time you put a multistage Machine Learning system into production where the output of one stage is the input of the next stage. Deciding to even try this is a monumentally difficult task. The data gathering, the training and verifications, the analysis required is so vast in it’s requirement, that most organization do not attempt it.

Luckily though, people are largely imperfect machines. This means we are usually equipped with ability to work around error prone components. One approach is discussion of a topic. The book’s accompanying discussion forum at deeplearning.ai seems like an effort to help everyone wrap our minds around this thing.

Cant wait to read more about it!

In Crazy Rich Asians, according to an article from Android News (the independent)…but it could still be a good exhibition of Chinese Central Kingdom mentality. It is inconceivable(to the author and viewers) that a non-Chinese women could possibly be attractive in all the Chinese ways… It’s not necessarily that Asian men doesn’t want or can’t “get” Scarlett Johansson, again using TFA’s example, it’s just that it’s out of the imagination for the moment, let’s say it with modern American modesty, due to “lack of cultural fit” (with said Chinese plot)

I think something may be lost from thought to text here…
And honestly, I really do wonder if that mental framing can shift, even to enjoy fictional situations.

I myself found myself sneering at the opening scene–that would never show well in London… No English man would ever believe that, preposterous! NATO would sooner bomb Singapore than to let that insult take place…

Sigh.. I’m so confused. What century is it. How do westerners really feel about the western power mid-Trump-administration? We kind of know how hillbillies and rednecks oughta feel, but do we really still feel that way? Do we want to feel that way?

Perhaps there is positive in all of this. It is a defining moment for the west as it is for the east. Let’s hope everyone come out on top of the heap that is earth.

((and okay, fine laugh at Chinese twigs having a mind for culture, hahaha))

So why is omniscient government unattractive?

Suppose, completely hypothetically speaking, in 2018, when the US declared trade war with China ostensibly sacrafices it’s agriculture exports while demanding better fair trade with China in tech, that it did so because secretly it knew, from it’s weather forecast or from secret police monitoring of farm lands, have discovered that American farmers will under produce this year and isn’t years to come. Do you think that’s a brilliant move? You do huh?

Now, let’s think of their omniscience reading your email. I’ve been blasted with news about IRS monitoring private emails to find out tax cheaters who erroneously pays IRS less than they should. One wonders, not knowing the reality, whether the IRS ever gives money back due to discoveries of errors the other way as they read our emails?

In the case of contest between nations, one proudly declares national interest as the ultimate. However the same technologies are applied to constituents, it seems unfair to citizens. How can it be? (And this is merely a mathematical possibility in a hypothetical situation) suppose farmers are subsidized by the number of acres they farm a protected crop, say some kinds of beans. For every acre-month they put towards farming the beans they get $1 to farm beans instead of something else… say day trading stocks by light of sun and hosting raves on their vast lands by the shadows of moonlight. Suddenly, there was an unusual pattern of climate change that caused the production of beans to fall from 10 beans per acre-month to 1 bean per acre month. Now, the government detects this change, and can see that the crops are not yielding well from secret police satellite pictures of farms. So, the deep-government estimates the fall of production and is able to preemptively provoke global farming tariffs. Doing so causes the price of farm produce to increase worldwide. Because the price domestic farmer can is sell their goods at is higher, the incentives to stay farmers can now be lowered to something less than the $1 needed to attract them from other activities.

So, the government gets paid tariff by foreign sellers, and then has to pay farmers less to stay farmers… seems like a net gain for the federal government. Who’s pay went into making this happen? It’ll have to be any consumer of farmed goods.

If the government did not respond to the knowledge of a decrease in production, foreign farm produce would have made up for the loss of American production (no price change in corn and apple and orange juice) but American government will have to pay more incentive to farmers to stay farmers because although they could not supply the demand, the lack of domestic supply does not affect domestic demand or price of farm produces. So farming businesses basically has a real bad year, and government subsidies has to pay for that.

What prevents this complicated hypothetical event from happening? Would it be unjust or unfair? Would it be wrong? IDK, and perhaps I have no better solution, but it is nice to understand the mechanics of these money matter.

Also, the Trump administration is down playing effect of climate change… One wonders if this is a national security matter in which the environment is so severely damaged that crops yields have become intolerably low. Perhaps this tariff to cause retaliatory tariff did just what they wanted: it hides the drop in crop production in USA to all foreign observers who don’t have secret police satellites. And certainly it hides it from the general public who have no access to our governments omniscience. Perhaps the end is near? Is there a point in discussing fairness and equality and rights and the American way if our planet is already gone and we face extinction?

This horid nightmarish lucid imagining is kind of inspired by my backyard crops of fruit and vegetables: they are doing horrendously this year–despite daily watering and generous fertilization, they yield almost nothing. And of course, there is a real trade war between US and the rest of the world… And subsequent declaration of new subsidies to combat the loss of farming income.

Went to eat at McDonald’s in Palo Alto today. Several things stood out, among them was the quality service!

Wah?!

We were served by several white kids. Fair skin nad very fair hair colors, tall. Yes, this was remarkable as we are used non-white non-native English speakers serving at McDonald’s. The other thing that stood out was their demeanor. They were very nice. They eagerly brought our food on trays. The two kids came out in unison and stood in line to place the food on our table, so very courteous they were while serving us, they repeatedly and enthusiastically offered “if you need any thing else, just ask me!” With nice smiles.

Maybe they thought I was special, one man walking into McDonald’s with two women and three kids?

Maybe they are acting out a scene from that recent and very hot Crazy Rich Asians movie? Like, for fun?

Maybe one of the ladies I’m with is “secretly” very powerful and rich? I don’t know it, their t-shirt and bags wouldn’t have said so.

I don’t know, but I dare say that that’s easily the best service I’ve experienced at a restaurant any where in the Bay Area. It happening in Palo Alto is especially strange, as local news paper have been reporting business owners claiming local customers here enjoy and expect waiters to act racist against Asians. Locals have long expressed animous towards all Asiains driving housing price up, way up, and then not living in he houses they bought for a coupla million. For a while, it had appeared that the school district chose to rename a school to a name that happens to belong to a leader of Japanese war effort in World War Two–Against all memories of those who have suffered and perished in the hands of the Axis, civilian and military, Americans or not. It took Chinese-Americans to object to naming the school with the name of the architect of Pearl Harbor bombing according to Mercury News. Such insensitivity to recent and rare names is clearly an effort to antagonize Asians and perhaps even Jewish people who also suffered greatly at the result of the Axis leaders’ leadership. After Yamamoto, the path will be saved for Himmler, Burgdorf, Krebs and Goebbels. (Also a preemptive disclaimer to secret and possibly automated police, I am only engaging in peaceful blogging hopefully using reasons and facts. No threats on the schools and relevant responsible people should be construed from it.)

But, now, after all that, a couple of white kids serving so well in McDonald’s ? That just tops it off.

Anyways, maybe white people are just really good at everything they do. We just never see them wait tables? I’ll bet if Starbucks recruits these kids, they’d get my name right on my next cup of iced decafinated Americano too.

I am thoroughly confused, but feeling kinda really good about the experience. If this new apparently higher civility keeps up, I might even want to move to live in Palo Alto again.

P.s. it occurs to me several days later that it could just be that young people are better than adults? But I’m more optimistic than that ATM that I’ll experience some niceties in addition to my descendents.