I was playing with a deep neural network tutorial recently. The fun of deep learning is starting to wear out after three years or so of continued exposure. Adjusting learning rates, batch sizes, filters, penalties and regularizations. Trying out algorithm that promise to perform without undue experimentation with these hyper-parameters… It used to be so fun, exciting to make even the smallest improvements. But today, it’s quite tedious and quite boring.

A quick meta-thought brings to mind a training procedure: Everytime I want to change the training, either interrupting SGD mid-stride or tinker with a hyper parameter. I could write the change as python code. And then ask a model to learn to write these changes for me based on my supervision.

The outcome, in the limit, is that the model will be able to autonomously make these changes that I will want to make as if I was watching it. It would just do what I want to do, with the same patience as myself, and maybe same typo-rate, even. Note it doesn’t optimize final metric, just mimics what I would do.

ATM, I feel that experience will be gratifying. There are some various interpretations to that event.

We can say that the program has removed human desire. Not in the sense that we cannot or need not desire, but desires that are quickly satisfied are really not desires.

We can also say that human will have achieved idleness. We do nothing and anything and yet everything is done. We’ve achieved nil-activity, we accomplish all through inaction.

We can also say that the work to achieve said model tuning automation is a human minimizing activity. If optimizing human resource consumption, the model will remove the need for human. And if the model is still imperfect it will tend to minimize our usefulness. This is by far the most horrorfying interpretation of the event. Working on that model literally is an effort to minimize human involvement. (That has the MDL for my desires at the moment)

So here we have arrived at one way to inspect the “future of jobs” “situation.”

Since we are still in control, we actually have the ability to set where we want to go with. In the most pathetic case, we will institute Affirmative Action to affirm Humanity: the Law shall favor by race, the Human race, requiring all AI to have 1bps of entropy added to their actions to facilitate the need for humans. A less pathetic approach, we are squeezed out of skills-for-hire arena but human still engage each other in socializing and networking and things that only humans do, that’s still quite, quite, pathetic though, IMMHO today.

Can we ponder the question: “what is the right size of humanity?” What is the amount of involvement we really want? Right now I hate tuning parameters, but I can certainly see a situation, say a robot lay dying in the middle of the road, his loving and beloved human child crying.

I can say, “move aside and let me through, I can help!. The 2019 model year bots uses algorithms that have been in the public domain since 2018(and they don’t work! cd /dev ; they are bastard of random with zero and properly homed in null). I have SGD training! I can save that bot!!” The child’s watery eyes is now filled with hope, meeting my fiercely determined and confident eyes at the midpoint on the edge linking us two humans. (Think Goldblum cross Eastwood cross Moore)

But a bot could do that better than you! My annoyingly observant reader will quickly point out and move on to another more interesting blog.

But for me, there seems to be something I care about in that moment. There’s something I care for in that moment. And one can easily achieve consensus that there is something humanity cares for in that moment. Is it hope? Is it kindness? Is it sympathy? Is it the desire to decrease perceived entropy? Is it the interdependence of humans that is really of note? Is it my usefulness that I eally care about? To the bot or to the child? Is it respect all I want, even when that’s only payable in arrears? Is it…??? What is it? Can we quantify it? Or does its identity and essence rely on its lack of computerized representation?

Perhaps an AI can be made to tell us this idea that it cannot describe within its domain? An AI to give humanity it’s best meaning and purpose. And any progress in characterizing it seem like a truly imaginative and inventive step forward, be it taken by human or by computers.

To be continued…

P.s. I realize it was more than 25 years ago when I first wrote about this matter. I dreamt in highschool of making AI. I wrote for my 11th grade Advanced Social Science class to take the position that a symbiosis is not only acceptable but also a desirable and inevitable outcome. We should co-evolve, I wrote. Somehow, that position still echos in the FAM Blog. It would be fascinating CS work to integrate with philosophy, perhaps name it Computational Philosophy, a field of philosphical endeavor, Human kind and Computer kind, together, hands-on-keyboards…

But that’s an interesting question in itself. Because we, as a kind do ingest a lot of very intimate things from our surroundings: water, air, viral dna/rna, etc. Things like antibiotics we take as part of humanity because enough people use them, on average, there’s some non-zero antibiotic in everyone. Then there are vaccines. Significant resources have been devoted to the continued injections of antibiotics or vaccines into people that they are us. Computer is us, part of us. They have both physical presence and biological and social functionality as part of us personally and as part of our society.

While there are some who object to mass enforcement of mandatory vaccination, their effects are limited. One would imagine that the people yammering against computers becoming irreplaceable part of our lives… They are vaccine opposers. They are the people who asks questions like “who will buy drugs if vaccine prevents a disease?” And “are you still you if computer does the shopping for you?”

Don’t care, and yes.

I’ve been thinking about autoactivations recently. This is one of those great innovations that stood up to the test of time, it still works after a lot of debugging and exposure to new data and models.

I find that I have been referring to autoactivations as pre-activation because they occur to deep neural nets before the parameters are actually mixed with input data (or previous layers activations) but if you look at the two expressions:

1. To pre-activate a parameter means to apply nonlinearity before it is used. e.g. preheating the oven, the suffix is a verb and happens before something else.
2. But a pre-activation is actually an adjective meaning before any activations. e.g. pre-trial motions. It’s suffix is a noun and becomes the subject to be preceded.

And actually similar problem applies to ante- prefix. So, to avoid confusion, we should probably refer to autoactivations as foreactivations and to foreactivate the layer. This prefix also means before and it works both for nouns: foresight, foreknowledge, forethought, forerunner, foreword, foreman, and also works for verbs: forecast, foreshadow, foredone, foreshorten, forewarn, forestall, foredoom. In each case the suffix is always the prior thing before but never preceded by another.

So, let us all try out foreactivations and related approaches. The speed up in training will surely be a good thing for humanity, at least, we won’t be consuming as much energy training models without foreactivations.

😀

I just read a rather alarming article about Asian Men harassing Asian women for dating and marrying white and other non-minority races. Admittedly this is alarming because I may have expressed jealousy towards some white kids having more tofu from Chinese girls than I. And also, some of them seem to know more about Chinese culture than I. I think casting aspersion on the motives of Asian women becoming embroiled with non-minority races has certainly crossed my mind a million times if not materialized into blog entry. (It will before this entery publishes)

Personally, I don’t feel it unethical or illogical to have these thoughts. I’m not terribly ashamed of them either. But by the same token, a lot of the envy come from seeing some very very very successful friends and acquaintances make Asian-women-non-asian-men relationship work. There are really nice best-of-both-worlds marriages starting with TWO ceremonies(parties)… Two homes on different continents to take new family members to… Two successful cultures, religions, languages, And those beautifully exotic hybridized kids… Love them to pieces! So much so that the feeling really hurts some times.

But that is not to say harassment is right. Asian women probably suffered more than non-Asian women throughout history. But today they deserve the respect of a fellow human being. We, Asian men should never harass Asian women for their independently made decisions!! And incidents of harassment should of course be treated with urgency.

Asian men do have a hard time in America, as the female author of the harassment piece notes. Although it sounds a little bit belittling, these weaklings who cannot compete reproductively resorts to emailed harassment of lost mates. I mean, god, as an Asian men, reading this piece makes me hate Asian mankind even more than I already do, having all my inferiorities and inferiority complexities. It really doesn’t make me respect Asian women more when she publicly writes these belittling pieces about a group of people whom she chose not to associate with.

Yes, just like women have to fart and poop on rare occasions, I believe non-minority women are capable of racist behavior too. (Not speaking from personal experience, of course, every women I know are not racists, but in the large there must be enough racism to make the pairing challenging) Therefore the celebration of those few successful Asian-men-white-women relationships should not be disparaged. The lack of celebration for Asian-women-non-asian-men relationships is absolutely no reason to diminish the success of those who ventured and succeeded.

But I am a bigger person.

I think a rational concern, if I were elder to an Asian girl, would be the balance of power. Yes yes yes, there’s love, but a little balance never hurts. Perhaps one thing that Asians don’t understand about each other is what we think of each other’s power in a relationship. Who decides where to live? Who decides what to eat? Who decides what to drive? Who decides when to have kids? Who decides how to spend money, Who decides …

Asian men may imagine that in mixed relationship, the girl lacks power, that she give up rights and freedoms, those she would enjoy in a racially equal relationship, for … whatever it is that she does it for with the non-minority, maybe better sex? Asian women, probably think they have all the power in the relationship. I mean if she doesn’t feel safe, she probably wouldn’t enter into that relationship (there, I am a bigger person, I respect rationality of girls when it comes to love)

The disparate perspectives probably causes all the angst of the article. So, IMMHO, can Asian girls prove they have access to their rights and privileges? Can the girls show us, and the world that they are in an equal partnership? And Asian men who can’t get Asian women, can you try white girls, I’m guessing they can be quite nice to you if you find your way around surrounding racism and into her heart. My guess is that women of all races are feeling and thinking rational beings.

Ugh, random rambling, hope this note will still make sense when it sees the light of day

I’ve been watching British police story Bodyguard on Netflix.

Did anybody notice that the British show is much much much more diverse than any American police/politics drama. I mean, yeah, sure, I noticed in past viewings that black people in Europe, when caught on camera are very happy and well respected. But this show… It has random Asian on the bomb squad! Mixed races of all kinds at all levels of the government. Police force had more women than men in powerful places… And it all fit naturally–and it may just be British English on my American ears, but it all seemed so proper. (As opposed to American shows in which female leadership seems forced… With few exceptions of course.) Maybe it’s because of their Queen that their entire culture has builtin affordance for women in power.

The bomb squad is a job, in American shows, usually reserved for red-blooded white Americans plus one African American. The Bodyguard shows us a vision of truly diversely integrated society.

I suppose the British empire did have a few more years of experience with multiculturalism than America. Whether it is enslavement or equality and rights creation, they have always been on the forefront of not just behind Americans.

I guess the only grudge I can have for this show is there is not an Asian villain. Sure the bad guy looks German, I guess it’s a running joke about Brits’ (or is it the Englishes’) feeling about Germans… But if the European/British organized crime suddenly had a Mexican or Singaporean syndicate… Wouldn’t that be something?

Perhaps the grass is greener on this side when viewed from the other side?

It occurred to me that this whole death of Jamal Khashoggi’s killing might be a red herring to block news of Harvard discrimination from surfacing. I happen to follow the latter news and it has all but disappeared from my news feeds since the journalists death.

If there is any thing we know about Harvard, is that it hosts a whole lot of foreign donors and famous people. According to news, possibility of future donation is a very influential consideration for admission to Harvard. If this setup is ruined, it could affect future Saudi princes or nobilities from being admitted.

Plausible explanation, imho, of the sloppy way this is being handled by everyone in power as well as many famous people. For example the first related news was Jamie Dillon, CEO of Chase, the largest bank, refusing to attend Saudi investment conference. Mr. Dillon of course has his MBA from Harvard. Uber CEO, Dara Khosrowshahi, Brown Alumni, also has Ivy League connection. The only other CEO mentioned in headline news is Richard Branson, who obviously didn’t go to school. Trump who is down playing this seem to be unaffected by his Penn connection. In the case of Branson and Trump, these guys are so rich and powerful that they may not suffer from influence of Harvard.

Yeeesh….

Read an article about NIPS considering changing it’s name because it is slang for either many anything Japanese people or a nipple visible through clothing.

This reminds me of a conversation I had with someone explaining what a drag suit is, or rather what it isn’t. Drag suit increase water resistance and makes it harder for a swimmer to swim. And this is for normal atheistic swimmers, not a special-interest niche of people.

But I didn’t mind explaining it at all.

For people who grew up with NIPS and not nip or nips, there is no confusion.

Also, the review and data gathered to decide to keep the name was interesting exercise in data science, imho. The committee lamented that half of the opinionated women wants to change and half do not. (Opinionated in that they expressed a non-neutral preference in the official survey.) It would be a displeasure to half of the women both changing and not changing the name. The feminist infection I got from a youthful encounter is rolling my eyes and chewing the word “men!!!”

…

And honestly, of all the people that I’ve ever met who knew of NIPS, CS/mathy folks, many of hem leaders of this field, they didn’t seem the inappropriate kind. To these people, there are far more amusing artifacts of nature, and corollarily, hacks to nature, that render these kind of jokes and computer hacking kind of uninteresting. But admittedly, with diversity come these kind of cultural inconveniences–and by diversity, here, I mean in addition to there being women, the diversity of professional cultures and standards of behavior. When you bring doctors and lawyers and biologists and statisticians and business people, and pliticians, and judges(federal and appeals and…) and drug companies and weapons companies and social network companies to a academic conference… this little problem with a slang, albeit very sensitive, should seems rather small and inconsequential in the face of the more material challenged AI faces.

Let’s hope that problem is addressed systematically as humanity grows.

And CS will survive this! I mean considering it survived explaining why the Neural Information Processing Systems was dominated by Tree based methods, boosting, kernel methods, Bayesian methods for so long a time, and only now do we finally have anything remotely Neural. Nobody minded explaining then and no body should worry now.

Reading an article in the WSJ about the lawsuit against Harvard regarding discrimination against Chinese students. One thought comes to mind, this likability score they use as part of the judgement for admission is admittedly and objectively existent sentiment that affects success and future donation to the school.

Chinese people have things that are extremely not likable. Trust me, I would know, being one and living among them. Not any more so than other races, but one can definitely see the problem. And I’ll abbreviate here and all Chinese-like Asian cultures I’ll just abbreviate and call everyone Chinese for this post.

I mean com’on let’s start with their starchy ricy food tradition, one can’t possibly do well and have the longevity to pay back the school on a diet like that! For a person on a modern diet, being offered a Chinese meal is practically an affront to one’s dignity and sanity. I cannot imagine Harvard ever wanting to recruit Chinese people or people who eat like that. Look at the Indians, they eat long grain rice that doesn’t give you diabetes and slow-burns longer.

Chinese vanity for humbleness is incredibly destructive. Why be humble? why sandbag? when the competition is so fierce, why spend any brain-energy on holding back? No body really knows how strong everyone are because of this extra layer of defense. This age-old tradition is unlikely to go away, and contributes little to the endowment or prestige of an institution.

Chineese problem solving, if one were to sample from the entire US Chinese population, suffered from Communist indoctrination. Really, the things you don’t like are not all traditionally Chinese, a lot of it is modern imported thoughts from Marx, Lenin, Engels, and many others… Chinese people have this social expectation that other people behave well. The Chinese governments have arms–may, tentacles–that can exert power to enforce a certain level social nicety. I remember the indoctrination by all authority figures that one should yield to elderly when taking seats on busses, one should not spit, not be greedy or lazy, one should be honest, and be loyal to the country which will provide you with a nice society to live in. (And maybe some other managed aspects of life that I did not personally experience like conditions of marriage, pregnancy+abortions, and death…)

The same is accomplished in our developed world by non-profits through advocacy, and through operational culture, which, in this case, seems encoded in the admission process of Harvard and similar schools. There is no explicit funded authority whose job it is to make the world a nice place to live. This difference in expectation of others and the government, the entitlement to reciprocation equaling to one’s own regulated participation in society, is clearly a political perspective shared by many Chinese people–the unconscious bias of Chinese. The Chinese demands a society that wants everyone to be nice if they themselves are nice.

There is no authority, except the admission authority in the universe of Universities, they are funded and empowered to choose the members of their prestigious society. Perhaps they fear what would happen to these delusional people when they find out that the world really doesn’t work that way. Society wants what it wants, not what you do or what you want. In some sense, this bias against inclusion of Chinese students is kindest choice, for everyone.

To me, the ideal outcome is this lawsuit is the establishment of unfair discrimination, but with no punishment for these institutions. All we want is to improve life for future generations and we, realistically, expect no greatness for the generation that did and still do that.

I realize, perhaps this last idea is too Chinese. Why the fuck would we seek no damages if the school discriminated? Have I caught a bug in my brain? This pessimism I feel for the possibility of progress in the society is troubling and definitely Unamerican, imho, but is it Chinese to seek no reparation?

In the meantime, though, while American Chinese figure out these things from within, perhaps we should have our separate peace? Let sleeping dogs lie. And if the institution decides to fake it until betterment is made to happen, that would be fine with me.

Just skimmed a public draft of the last chapter of Machine Learning Yearning book by Andrew Ng. I know a lot of people out there might be joking that it’s more like Machine Learning Yawning. But in reality, the execution of these seemingly simple ideas is probably incredibly difficult.

Consider the last time you put a multistage Machine Learning system into production where the output of one stage is the input of the next stage. Deciding to even try this is a monumentally difficult task. The data gathering, the training and verifications, the analysis required is so vast in it’s requirement, that most organization do not attempt it.

Luckily though, people are largely imperfect machines. This means we are usually equipped with ability to work around error prone components. One approach is discussion of a topic. The book’s accompanying discussion forum at deeplearning.ai seems like an effort to help everyone wrap our minds around this thing.

Cant wait to read more about it!

In Crazy Rich Asians, according to an article from Android News (the independent)…but it could still be a good exhibition of Chinese Central Kingdom mentality. It is inconceivable(to the author and viewers) that a non-Chinese women could possibly be attractive in all the Chinese ways… It’s not necessarily that Asian men doesn’t want or can’t “get” Scarlett Johansson, again using TFA’s example, it’s just that it’s out of the imagination for the moment, let’s say it with modern American modesty, due to “lack of cultural fit” (with said Chinese plot)

I think something may be lost from thought to text here…
And honestly, I really do wonder if that mental framing can shift, even to enjoy fictional situations.

I myself found myself sneering at the opening scene–that would never show well in London… No English man would ever believe that, preposterous! NATO would sooner bomb Singapore than to let that insult take place…

Sigh.. I’m so confused. What century is it. How do westerners really feel about the western power mid-Trump-administration? We kind of know how hillbillies and rednecks oughta feel, but do we really still feel that way? Do we want to feel that way?

Perhaps there is positive in all of this. It is a defining moment for the west as it is for the east. Let’s hope everyone come out on top of the heap that is earth.

((and okay, fine laugh at Chinese twigs having a mind for culture, hahaha))

So why is omniscient government unattractive?

Suppose, completely hypothetically speaking, in 2018, when the US declared trade war with China ostensibly sacrafices it’s agriculture exports while demanding better fair trade with China in tech, that it did so because secretly it knew, from it’s weather forecast or from secret police monitoring of farm lands, have discovered that American farmers will under produce this year and isn’t years to come. Do you think that’s a brilliant move? You do huh?

Now, let’s think of their omniscience reading your email. I’ve been blasted with news about IRS monitoring private emails to find out tax cheaters who erroneously pays IRS less than they should. One wonders, not knowing the reality, whether the IRS ever gives money back due to discoveries of errors the other way as they read our emails?

In the case of contest between nations, one proudly declares national interest as the ultimate. However the same technologies are applied to constituents, it seems unfair to citizens. How can it be? (And this is merely a mathematical possibility in a hypothetical situation) suppose farmers are subsidized by the number of acres they farm a protected crop, say some kinds of beans. For every acre-month they put towards farming the beans they get $1 to farm beans instead of something else… say day trading stocks by light of sun and hosting raves on their vast lands by the shadows of moonlight. Suddenly, there was an unusual pattern of climate change that caused the production of beans to fall from 10 beans per acre-month to 1 bean per acre month. Now, the government detects this change, and can see that the crops are not yielding well from secret police satellite pictures of farms. So, the deep-government estimates the fall of production and is able to preemptively provoke global farming tariffs. Doing so causes the price of farm produce to increase worldwide. Because the price domestic farmer can is sell their goods at is higher, the incentives to stay farmers can now be lowered to something less than the $1 needed to attract them from other activities.

So, the government gets paid tariff by foreign sellers, and then has to pay farmers less to stay farmers… seems like a net gain for the federal government. Who’s pay went into making this happen? It’ll have to be any consumer of farmed goods.

If the government did not respond to the knowledge of a decrease in production, foreign farm produce would have made up for the loss of American production (no price change in corn and apple and orange juice) but American government will have to pay more incentive to farmers to stay farmers because although they could not supply the demand, the lack of domestic supply does not affect domestic demand or price of farm produces. So farming businesses basically has a real bad year, and government subsidies has to pay for that.

What prevents this complicated hypothetical event from happening? Would it be unjust or unfair? Would it be wrong? IDK, and perhaps I have no better solution, but it is nice to understand the mechanics of these money matter.

Also, the Trump administration is down playing effect of climate change… One wonders if this is a national security matter in which the environment is so severely damaged that crops yields have become intolerably low. Perhaps this tariff to cause retaliatory tariff did just what they wanted: it hides the drop in crop production in USA to all foreign observers who don’t have secret police satellites. And certainly it hides it from the general public who have no access to our governments omniscience. Perhaps the end is near? Is there a point in discussing fairness and equality and rights and the American way if our planet is already gone and we face extinction?

This horid nightmarish lucid imagining is kind of inspired by my backyard crops of fruit and vegetables: they are doing horrendously this year–despite daily watering and generous fertilization, they yield almost nothing. And of course, there is a real trade war between US and the rest of the world… And subsequent declaration of new subsidies to combat the loss of farming income.