I told a blind woman: hey you are stomping into a shit puddle and splashing me with turd juice!

She says back after some deliberation: who are you? Jesus Christ? You are just as blind as me,you shouldn’t judge me. Truly you probably put that puddle there to set a trap for me.

I say to her: wait!! What?? You are blind not anosmic! Why are we even having this conversation? Why would I do that to you!?!? Why am I even at this camp!? And now you are just shitting an pissing into the puddle standing upright, is this incontinence camp too?

She says: you are just jealous that I thought of it first…

…

Oh no, now you are douching yourself with it.. I cannot watch this any more…

…
(oh wait that’s actually kind of smart, because now I, a man, cannot, within my imagination, copy the idea you presently practice.(sorry, despite present absurdities, I still have a little residual sense of wonderment about the products of people conditions and processes))

And the day continues this way, and day after day..

Seriously this seriously happened, come and smell me. (Now a few years later, the oder should be palpable still)

Ps, as always, I whole vehemently deny and disavow any connection with or even similarity to real world people, things or events of past, present or future.

Several months ago I wrote about the Deep Universal Regressor. Hopefully you have discovered that it is full of bugs, fashioned typically for the late 2010’s, following Interweb AI blogging conventions of the day. It isn’t really because I’m time traveling and writing this from the far past or far future, just that this is the way we wrote in those days.

One most notable one is if you tried to regress using my suggestion, you found that it does not extrapolate well. After some inspection, one realizes the problem. One way to fix it is to use a decision list to make progressively smaller estimates:

The are linear activations from previous layers calculated from .

But of course, this approach also has some fatal flaws that needs to be fixed. Some effort has to be made to ensure the right term will be adjusted at the right moment–it needs regularization. It also needs some experimentation around what is.

After that, the is the limit.

Ps, it’s kind of interesting that this range-preserving combination approach is slightly more general version of attention mechanism. Recall attention is applied as softmax combination of items of consideration. The softmax is just a positively weighted average, which in turn is a convex combination.

P.p.s. On the reverse side of this approach, perhaps we can apply activation in an earlier place to create similar effect. This approach is often taken to restrict parameter in a range, for example using to produce a dense layer that applies only positive multiples to inputs. The approach I have tried with good success is the exponentiation activation. At first blush, you might judge this absurd. When we take a step, would the change in not be the same as the change in ? There is no contribution to complexity of the model in this parameter activation. If you look at it as and then compute the output of the neuron as , the derivative of x with respect to A is linear in X. What difference does it make how you arrive at ?

But I have observed that the application of this activation makes linear regression converge faster under SGD. Looking at it more specifically is the parameter’s value is its own derivative. In other parlance the sensitivity of the derivative with respect to a parameter is linear in size of the neuron’s outputs. If A was just raw tensors, this derivative would be the constant . Another consideration is that perhaps all we did was to initialize A using a log-normal or log-uniform distribution. Lastly, the effect of exponentiation maybe through the effect of increased or decreased learning rates. For this blogs purpose, I tried to implement a few of these alternative explanations directly with linear model: using log-normal initialization to a simple dense layer, trying various learning rates. These attempts did not produce the same quickened convergence as exponentiating the raw parameters.

These meandering is of course a middle way between many. For example, if we’re not raw tensors but the output of some tensor function of the input, and we further normalize: this would be an attention mechanism like those used in transformers. Although the derivative of this is neither linear or constant.

Here is a list of parameter activations and their detivatives. Expressing in terms of A brings us a view of the scale of the gradient.

		in
W	1

Clearly, its hard to boost the progress of training anymore this way.

Ppps ah, yes, astute you are, I have assumed positive data. The negative of can be concatenated as parameter or learned using a second layer or even a simple softened sign parameter. That should be exercise for reader to adapt to their own problems.

Pppps, This works so well, I want to name it, let’s call it autoactivation. In the tradition of autoencoders and similar to the auto- in autoimmune diseases, autoactivation means that the parameters of the network activate themselves, they are autoactivating. By the same convention, self-attention models are autoattentive.

Has anybody given a serious go at using log-normal initializer for deep neural network parameters? Seems likely to make sense. Additionally, one can also do batch- and layer- log-normalization on layer activations.

I wonder how the world will turn out one day. Once long ago, I was in the company of some intellectual human discussing political ethics… And it occured to me that I was struggling to understand a lot of what was being said.

What will happen one day when our political practices advance to a place when most people cannot hold the concepts in their head due to cognitive limits? What will happen if truth, happiness and right and wrong are not comprehensible by most people? These incomprehensible thoughts about things that are fundamentally human(good/happy/free…), let’s call them truer truths…

It’s kind of weird that from intellectually enlightened discussion come a thought of the possibility that we cannot comprehend what is right or good. But from the conception of such truer truths we deduce knowledge of truer truths. It would seem, if there be gods who comprehend these truer truths, lEr him slip a hint of their existence, like black hole leaking radiation.

This of course also come home to the origins of the discussion with companions, what if AI became more cognitively advanced than humans in that they know more about the truer truth that us? It is imaginable today that we have enough computers, under control of a single entity, to computer more than human brain in some task. The AI may be closer to Gods of religion than any of us.

Scary stuff.

Let’s come back down to earth. What if I cannot understand what you say is right? Am I in the deficit to all that is good and mighty for not understanding you? There is a slight chance that you are making it up and that you don’t actually make any sense at all for real. What if we have a constitutional amendment that says “gdjiDkdberish” ? Or a law that says “only non-alien citizens can live and non-aliens can understand e=mc^2” or “affirmative action credits shall be calculated by the suface derivative of the Mao-Zhou balance equations of said population as measured by Kim’s protocol.”

Hey, I mean, it’s not like I’m inventing a human thought here. Among us who are subjects to it, how many really understands the US tax code? In theory, that’s something we chose to do out of our free will. Like really, how are any thing we have today different than these examples above?

And the other side of this argument, how simple do our value system have to be for it to be practical? How universal do our value systems have to be for it to be right? For example, can we demand that citizens can remember to follow the laws? Can we require people to have enough math skills to pay taxes? Can we require them to be literate enough to read candidate names to participate in an election? Can we require citizens to read election candidates write-ups and pass a test before casting his ballot?

What are the faculties, cognitive or otherwise, do we request of our citizens to access to their rights?

…

Been busy for a few weeks, so I m still making my way through the war on normal people by 2020 presidential hopeful Andrew Yang. Although his prospects dim as president Trump succeeds in North Korea negotiations, his arguments are still very refreshing.

The one thing I wonder, in the zealous disclosure of all these cool economic data and progressive thoughts on how the govenrment should help its people pursue happiness, I feel that in order for this to work, he has to explain the American freedoms of his system. In essence an America UBI imposes a financial action on each person to contribute to other Americans or companies. Some could argue that this deprives them of a freedom to not spend money that way.

I really love his idea for more wholestic medicine valuing quality over quantity. This definitely has roots in Chinese culture and teachings. Once Chinese, one never free one’s self of the feeling that the whole needs more servicing than it gets and the parts less. But it would seem to be a common complaint against thst philosophy that lacking money driven competition, healthcare just cannot be as good as it is today because there wouldn’t be as many people trying as hard to do better than other people. In some sense, people can argue that his idea for reformed medical system is unamerican. Fuethermore it is unrealistic in 2020. But let it be said that I do think human health is a basic human right worthy of national protection, possibly more so than speech, press, assembly, religion, politics, property and money.

I think he will need to publish more books to explain what is planned for the 2020 term.

The Amerian exchange program or citizenship trip he proposes is a horrifying idea: all graduating highschool seniors are scheduled to go on a trip to take part in other parts of America. So… Lets see as recent as the 2010’s Taiwan had a “national” draft that forces every capable man(i.e. highschool grads) to take part in military. So, most Taiwanese man(by “citizenship”) you meet for the next decade, probably all served in Taiwanese millitary for 1-2 years. Indeed, this is well known in the US as its famed computer scientist Feng–hsiung Hsu, maker of DeepBlue which beat human champion in chess, mentioned that he felt liberated to have, at begining of his computer career, come to study at Carnegie Mellon University from a very boring Taiwan draft. Yang’s dad probably had to take part in it too… or not since he seems of a borderline age when Taiwan wasn’t a thing yet.

Thats not the scary part. The sending school-aged kids to an unfamiliar environment so they can learn about fellow citizens was actually a fate suffered by many PRC Chinese. In the 1950’s and 1960’s, Communist China engineered a social movement called 上山下乡，according to wikipedia, to:

decrease difference between industry and agriculture, rural and urban, and physical and mental laborers. All educated kids we sent away from home to live in the worst and most unfamiliar places they could find.

This, btw, is one of the reasons why Chinese think China is still very ver behind other countries in technology and science, their youths were sent to waste time and many died of malnutrition or other dangers. In any case, Yang’s proposal sounds eerily similar in means and ends to PRC’s 上山下乡 movment. Maybe he shares some Hunan ancestry with Mao?

Of course, it is en vogue to copy PRC China. For one example, silicon valley have in recent years adopted this word of a “lead” to mean a role in a company that demands more responsibility and more expertise and of course more political alignment. A non-lead, is a less responsible and lesser expert and may question the plan more than their leads. This in particular is not a word used else where until perhapd very recently replacing or superseding words like “manager”, “director”, “partner”, “president”, and “executive”. Or modifiers like “special agent in charge,” “managing partner,” “lead investigator.” I guess if you search the IMDb you may find a “situation lead” in military context where it is used as a noun. What happened to a good old American title like “boss,” “chief,” “honcho,” or “jefe?” But the “lead” concept is the foundamental management role in PRC China since its early days mid last century. They have roles called 领导，literally “lead and guide”. While there are still leveling, to be a 领导, very much demands political alignment and responsibility, and in particular more than those that you lead. So, it would appear, at least superficially, that copying communist PRC isn’t a real deal breaker. (Even if it was copied from military origins in America, it has been the public knowledge and concern that many of the the hugely successful Chinese enterprises that grew from startup were run by ex-military persons possibly using skills they acquired in the military)

Good effort so far, lets see if the third half of the book is any better in these aspects.

Just had a cystoscope (yeah look that one up), all normal, cancer scare’s over. Last one I’ll need to have now that they’ve established a baseline for me. Went to Gott’s roadside at town&country and had me a Impossible Burger(were half of you thinking dr. Who’s Impossible girl?)

Wow, I’ve been vegetarian for around 20 years, many of those were vegan years. This Burger does taste like a hamburger. I am really surprised that I remember the texture of the the veins and fat popping as I bite into the Burger. The slight smell of heme is definitely a big hint!

The meal was very enjoyable. The cook on it could probably used a bit more sear, jalapeño would hit the spot for me personally, and maybe more burger, as I find myself wanting another one after eating the one I had. The bread is too heavy, or rather he patty could probably be a little fattier so that it doesn’t crumb as much. The fat would also help with the sear. The hint of red is too uniform. I think a little bit more with-in-party variety could do this Berger some good.

I’d definitely order another one for the taste of it. The website says it is made out of coconut oil and potato, so it’ll mostly be for fun I guess.

Next step for me, being in the unfortunate disposition of having only CS training is probably to try to find a way to buy some of their stock. All their jobs (presumably companies by steeply discounted burgers) are non-computer work. Sigh…

I didn’t completely understand the Catholic and Jewish dialogs. I don’t know that Catholics are shun here?! My former Employer Jack Dorsey isn’t shy about his and his family’s afinity for Catholics. Indeed many employees found him quoting from the Popes speeches directly or indirectly when directing his company. I just am completely missing the joke here or I just don’t get this genere thst also includes Beavis and Butthead?

Also, this whole Asia copying America thing. Yes it happens so much and so blatently that its disgusting. But its really not that odd. Copying is a very normal thing to do, I mean your genes do it many times a day… Its part of nature like peeing pooping and bad comedy. Alas, it can be pointed out, while I still have this memory, in 2005, I was working at Yahoo! A bunch of execs just returned from a visit to Asia, mostly because of the typhoon their EA scheduled them to fly into, but also because of the awesome tech they saw. “The Taxi driver has two phones! One for nav and messaging and the other one to making calls… And theres this crazy micro-blogging thing thst every Korean is doing right now, especially young people whom we want to target” The exec blurted out in a single breath…

And sure enough, twitter was founded in the same year to do the same thing. Admittedly, nobody here admits to cooying Asia(or funding an idea that was successful in Asia), but one craze did mature in Asia before the other in America. And of course in Silicone Valley we know that today’s Chinese weibo is a copy of twitter or facebook… But its actually named, literally, a micro-blog. And they’ve had that before we did. Facebook, too, the social network has a broadcasting self-publishing and self-promotional functions that is basically you writing a short text and showing it to a lot of people. It all satisfy some kind of human need, and Asian human needs are just same as American human needs. I mean I’m writting a blog, aren’t I?

(And there are other similar examples, including moocs, escooters, all female executive teams…, etc.)

But I did not know about the horse manure problem of 1894. That was quite a jewel in the rough. Some writer sure sounds inspired…

But with so much other weird things like attacking people who have weird machine related sexual orientation… Thats so rude. We haven’t really established thst his robot was… errr… inappropriately handled. It goes to motive that if no wrong is done, why would he launch an ddos? This whole subplot is just an excuse to shine a derissive light on those with agalmatophilia.

You people make me sick!

Ps ohhh, okay, wait, there was this thing where many thousands of cases of Catholic priests sexually mollesting kids(and mostly male homosexual abuse as seen on TV) and the then holy see hid the fact and kept priests priests… Yeah i guess thats pretty nasty. although I don’t know that Silicone valley is more disgusted by this than people and companies elsewhere.

You know, I abhor Trump’s attack on naturalized citizens who hold opposing political views against him. I do not think it is very presidential or even legal for him to ask American citizens to leave the country for exercising their political voice and powers.

But as a naturalized citizen, I can’t help but cheer him on towards acquisition of 51st State of America. Like who gives a shit about the Mexican wall if we get another state. Americans should carve Greenland coast into a profile of Trumps face to celebrate his presidency if he pulls off a real deal. The New millennial manifest destiny! We can conquer the whole world with American ideals for human rights and our economical political system–by buying it!

Of course, it might very quickly melt or get washed away due to global warming.

I have to admit that the American voluptuary in me is in conflict with my rational scientific self. Both feel nationalistic pride but Trump sure knows how to tug at the American voluptuary’s heart.

Let’s make America Greater! Again!!

“Don’t eat our dogs!!”

“You hit my dog!!”

“Our dog lives with babies”

She managed to yell out in between grandma’s shouts of outrage against a freely roaming dog in a public space. ( we did not hit her dog! Nor did we intend to eat it)

A little dog approached my daughter recently. Trump was president, people running over other people with cars in an act of terror around the globe is still a fresh memory from days prior) the dog kind of knocked her on her back on this beach. She was scared, the dog kept on advancing at her torso and head, Perhaps just a couple of feet away.

After some many arms waving at the dog and yelling, the owner finally called the dog away from us from about an eighth of a mile away.

I raise my iphone, in the now universal gesture of insult: I have taken a picture of you, your rude acts are recorded for ever.

She responds in kind, raising her camera and records back at us and continue to barrage us with insults and demanding insinuations. (Man! Apple/google/smartphone-makers makes this society so much more civil… these could have been middle fingers or guns being raised into air)

I raise my phone to report her to the police: “she is approaching me now” I say into the mic, “I feel threatened and she is shouting at us”

I walk away hastily retreating towards the Pacific Ocean… hoping she wouldn’t follow. She follows for another 10 feet or so approaching me. (Her dog still off leash btw, she is now between me and my daughter, grandma between daughter and dog)

She backs off a bit on hearing this conversation with the police.

After a few more rounds of raising phone to take picture. They drive off in their Toyota.

Where are they driving off to? Could their gang drive up next and shoot us right on the Del Mar beach? Maybe they’ll suddenly drive their car into the beach and just wipe these dog eating chinks out!! Maybe they called for backup in way of local police who will surely love to talk these hippy sf Bay Area Asians down about how dogs are part of the family here and are treated as equals when it comes to right to use the beach.

Let’s go! I said with a grim biting tone (think Clint Eastwood)

“But why!???!!” My daughter flashed me the biggest surprise face of a broad grin (she’s on her sand toys which we have been wanting to exercise on real sand for half an age), grandma says “no, let’s stay here, let the kids play!”

“But I don’t feel safe anymore. We had a shouting fight with locals… we don’t know the rules here, this is wild wild west terra infirma, let’s get out of here” still thinking of that determined, biting authoritative tone of Eastwood, but obviously the words don’t fit the tone no more… and nobody followed me. So much for Eastwood of the family…

Well actually their backup came in way of four unspayed and unleashed German Sheppard catching balls.

We, the whole family this time, made ourselves very scare very quickly. This no country for dog fearing people.

We retrieve to the safety of a Starbucks 10 minutes from the beach in our minivan. Driving very carefully to avoid attention, checking for tails or police sirens. It seemed a distant but not nonexistent possibility that she called the cops on us, based on her accusations on the beach. One can’t be too careful.

The daughter catches the wifi two blocks away and starts watching kids show on Amazon streaming… I’m thinking she has recovered fully from that traumatic encounter… grandma is silent, deep in thought… reviewing her long and laborious journeys to America and then to California, no doubt trying to find anything similar to this kind of encounter (where she didn’t win a fight against a bully by KO)

Inside Starbucks that familiar and reassuringly familiar coffee roast smell fills my nostrils… mommy snacking with her daughter, an elderly man in his 50’s or 60’s is working on his partial differential equations homework(erf?!), gamer gaming, kid listening to music… safety at last!

I use my gold card to buy a coffee for the long journey home.

Phewwww… one disastrous situation averted…

Seriously if I actually called the cops, I do not know if the result will be favorable to us. All parks and beeches requiring leash on dogs where we live means nothing here. Dogs are “us” and dog eating Asians are not “us” as far as I can imagine in this little vacationing town. It wouldn’t be the first time we hear that from a person of officer or a public person of government either…

I did not feel the safety and comfort of an American citizen during this vacation on American soil… and I don’t blame dogs or their owners… really, I feel a strong urge to figure this relationship problem with dogs if we’re to do well in America.

There’s this thread of analysis on how bad Facebook make people feel. When he sees his friends vacation photos pop up during late night debugging in the NOC, or when his friends paper is published in Nature or Science when he’s stuck writing a QA spec, or experiencing a fire sale of his own company and seeing his friends companies were funded in an up round by yet another friend who’s a partner in the funding VC, both of whom posts braggy posts to their wall on Facebook… sucks to be the watchers on Facebook.

The depressive effects of social media has also leeched into moocs. As leading moocs sites like Ng’s Coursera, Thrun’s Udafity, and edX, etc. have taken hold and continued their growth. More and more courses of higher and higher production value are being disseminated online. Recently I saw a video of highly popular Harvard course. And wow! A student, an Asian kid, at Harvard, had the audacity to get up in a lecture hall of hundreds and challenged the professor in his assertion. Professor handwaves him off, but before relinquishing the mic, the kid gets one shot that stumps the prof for a good 15 seconds! The sheer intellectual liveliness of this exchange, and a lot more for the rest of the lecture. It makes me wish I was 20 years younger and went to Harvard and chose that major… I might even nudge my kids towards that path. (Of course, the Harvard professor says the Asian kid’s English name wrong, and everyone else’s name was spoke correctly… but… still… the diversity and tolerance in the class is commendable imho… The episode has high production value highlighting the cultural and intellectual diversity at Harvard that other universities maybe hard pressed to generate)

In the end, after the fun I had watching someone else debate the professor… I am alone, in my room, in front of a browser… at the end, this moocs gives me great regret and sadness. Watching younger smarter people enjoy themselves in a place and time that I never have or will experience in reality. Knowing that there is much more beyond what I can see, the learning and mentorship that happens in person… It’s School Porn!!

“But you’ve had your chance” you might say to me. Think of your wonderful time at CMU and the massive fun you’ve had… and yes, I did indeed already experience the wonderful joy of learning at an all-around-best university, college and department. It is perhaps because I know of how wonderful it can be that I am upset at these teasers of moocs…

I hope the leaders of this mooocs industry maps out more clearly a path forward. Can a moocs student meet the profs and other students live? Is there a path from all the undergraduate courses towards graduate study? Is there a path from moocs to social-economic advancement? What is the plan? I mean at least say that you have an AI with big data to help the students to map out their longer term learning outcome. State the expected educational outcome–beyond: “look ma, I’m taking a Harvard class”

Admittedly, at the birth of a technology and industry, nothing is really clear. But what is clear to me is that these courses are not as intense as real classroom. The information seem to be maybe about half to a third of a typical course with the same name. Of course I may be biased because I attended a good university and that the average university level course may be equivalent to moocs course. If this were the case, maybe we do need to more seriously invest in American education infrastructure from daycares through higher education. Because other countries are working hard on their kids. How is a future American to compete with an immigrant if they grow up being taught to achieve only these moocs?

ps it is with great optimism that I write this, I’ve scheduled it for publication a long time after its writing that viewer may feel it is outdated comment.

pps as of this writing, there is great grief among Chinese students that Harvard discriminated against them and admits relatively fewer than fair portion of qualified applicants. Several actions are ongoing.

ppps there are micro-degrees on moocs right now. Perhaps the idea is continue pursuing the complete education at a distance idea. If this is true, then certainly a lot of work is to be done, these courses can hardly help most normal students to gain the knowledge taught in the full course

pppps agreed, a motivated student could learn everything today, but a normal average kid cannot imho.

ppppps The founders teach, as examplar courses in their field of expertiese. They teach very high quality survey courses that are vocationally oriented. It enables a person to learn and become kore and more fluent in working with their technology. The same is not true for most moocs courses though.