You know, all you hackers and crackers, and breakers, whether you work for the government, a corporation, my parents, my wife or my kids. I will always win this one in my mind that you are crap of humanity when you take my personal information and reveal it to other people. You are all scum of the earth when you take what I intend to keep private to myself and present it to other people with or without my knowledge and consent. You can hack my bank account and all my brokerage accounts and all my social network accounts and maybe even my kids and other families online accounts. But you know you will always be wrong and evil and the worst humanity ever produced in my mind. I don’t care if all your people does it this way. Were it not wrong for me to do so, I would hate you with the fervor of religious zealots hating blasphemers, homosexuals and competing religions or believes. You should suffer the consequences of having categorically immoral behavior.

But hopefully you will choose to do otherwise in the future.

Just want to note down: first time that I have not been asked for an ID while buying alcohol… in almost 40 years… I don’t feel old, but I distinctly feel the difference in perception of me, now that I am old…

Hey, this is to help neighbor celebrate 70th, so, I don’t feel that bad yet, perhaps after the beer this will sink in more.

I told a blind woman: hey you are stomping into a shit puddle and splashing me with turd juice!

She says back after some deliberation: who are you? Jesus Christ? You are just as blind as me,you shouldn’t judge me. Truly you probably put that puddle there to set a trap for me.

I say to her: wait!! What?? You are blind not anosmic! Why are we even having this conversation? Why would I do that to you!?!? Why am I even at this camp!? And now you are just shitting an pissing into the puddle standing upright, is this incontinence camp too?

She says: you are just jealous that I thought of it first…

…

Oh no, now you are douching yourself with it.. I cannot watch this any more…

…
(oh wait that’s actually kind of smart, because now I, a man, cannot, within my imagination, copy the idea you presently practice.(sorry, despite present absurdities, I still have a little residual sense of wonderment about the products of people conditions and processes))

And the day continues this way, and day after day..

Seriously this seriously happened, come and smell me. (Now a few years later, the oder should be palpable still)

Ps, as always, I whole vehemently deny and disavow any connection with or even similarity to real world people, things or events of past, present or future.

Several months ago I wrote about the Deep Universal Regressor. Hopefully you have discovered that it is full of bugs, fashioned typically for the late 2010’s, following Interweb AI blogging conventions of the day. It isn’t really because I’m time traveling and writing this from the far past or far future, just that this is the way we wrote in those days.

One most notable one is if you tried to regress using my suggestion, you found that it does not extrapolate well. After some inspection, one realizes the problem. One way to fix it is to use a decision list to make progressively smaller estimates:

The are linear activations from previous layers calculated from .

But of course, this approach also has some fatal flaws that needs to be fixed. Some effort has to be made to ensure the right term will be adjusted at the right moment–it needs regularization. It also needs some experimentation around what is.

After that, the is the limit.

Ps, it’s kind of interesting that this range-preserving combination approach is slightly more general version of attention mechanism. Recall attention is applied as softmax combination of items of consideration. The softmax is just a positively weighted average, which in turn is a convex combination.

P.p.s. On the reverse side of this approach, perhaps we can apply activation in an earlier place to create similar effect. This approach is often taken to restrict parameter in a range, for example using to produce a dense layer that applies only positive multiples to inputs. The approach I have tried with good success is the exponentiation activation. At first blush, you might judge this absurd. When we take a step, would the change in not be the same as the change in ? There is no contribution to complexity of the model in this parameter activation. If you look at it as and then compute the output of the neuron as , the derivative of x with respect to A is linear in X. What difference does it make how you arrive at ?

But I have observed that the application of this activation makes linear regression converge faster under SGD. Looking at it more specifically is the parameter’s value is its own derivative. In other parlance the sensitivity of the derivative with respect to a parameter is linear in size of the neuron’s outputs. If A was just raw tensors, this derivative would be the constant . Another consideration is that perhaps all we did was to initialize A using a log-normal or log-uniform distribution. Lastly, the effect of exponentiation maybe through the effect of increased or decreased learning rates. For this blogs purpose, I tried to implement a few of these alternative explanations directly with linear model: using log-normal initialization to a simple dense layer, trying various learning rates. These attempts did not produce the same quickened convergence as exponentiating the raw parameters.

These meandering is of course a middle way between many. For example, if we’re not raw tensors but the output of some tensor function of the input, and we further normalize: this would be an attention mechanism like those used in transformers. Although the derivative of this is neither linear or constant.

Here is a list of parameter activations and their detivatives. Expressing in terms of A brings us a view of the scale of the gradient.

		in
W	1

Clearly, its hard to boost the progress of training anymore this way.

Ppps ah, yes, astute you are, I have assumed positive data. The negative of can be concatenated as parameter or learned using a second layer or even a simple softened sign parameter. That should be exercise for reader to adapt to their own problems.

Pppps, This works so well, I want to name it, let’s call it autoactivation. In the tradition of autoencoders and similar to the auto- in autoimmune diseases, autoactivation means that the parameters of the network activate themselves, they are autoactivating. By the same convention, self-attention models are autoattentive.

Has anybody given a serious go at using log-normal initializer for deep neural network parameters? Seems likely to make sense. Additionally, one can also do batch- and layer- log-normalization on layer activations.

I wonder how the world will turn out one day. Once long ago, I was in the company of some intellectual human discussing political ethics… And it occured to me that I was struggling to understand a lot of what was being said.

What will happen one day when our political practices advance to a place when most people cannot hold the concepts in their head due to cognitive limits? What will happen if truth, happiness and right and wrong are not comprehensible by most people? These incomprehensible thoughts about things that are fundamentally human(good/happy/free…), let’s call them truer truths…

It’s kind of weird that from intellectually enlightened discussion come a thought of the possibility that we cannot comprehend what is right or good. But from the conception of such truer truths we deduce knowledge of truer truths. It would seem, if there be gods who comprehend these truer truths, lEr him slip a hint of their existence, like black hole leaking radiation.

This of course also come home to the origins of the discussion with companions, what if AI became more cognitively advanced than humans in that they know more about the truer truth that us? It is imaginable today that we have enough computers, under control of a single entity, to computer more than human brain in some task. The AI may be closer to Gods of religion than any of us.

Scary stuff.

Let’s come back down to earth. What if I cannot understand what you say is right? Am I in the deficit to all that is good and mighty for not understanding you? There is a slight chance that you are making it up and that you don’t actually make any sense at all for real. What if we have a constitutional amendment that says “gdjiDkdberish” ? Or a law that says “only non-alien citizens can live and non-aliens can understand e=mc^2” or “affirmative action credits shall be calculated by the suface derivative of the Mao-Zhou balance equations of said population as measured by Kim’s protocol.”

Hey, I mean, it’s not like I’m inventing a human thought here. Among us who are subjects to it, how many really understands the US tax code? In theory, that’s something we chose to do out of our free will. Like really, how are any thing we have today different than these examples above?

And the other side of this argument, how simple do our value system have to be for it to be practical? How universal do our value systems have to be for it to be right? For example, can we demand that citizens can remember to follow the laws? Can we require people to have enough math skills to pay taxes? Can we require them to be literate enough to read candidate names to participate in an election? Can we require citizens to read election candidates write-ups and pass a test before casting his ballot?

What are the faculties, cognitive or otherwise, do we request of our citizens to access to their rights?

…

Been busy for a few weeks, so I m still making my way through the war on normal people by 2020 presidential hopeful Andrew Yang. Although his prospects dim as president Trump succeeds in North Korea negotiations, his arguments are still very refreshing.

The one thing I wonder, in the zealous disclosure of all these cool economic data and progressive thoughts on how the govenrment should help its people pursue happiness, I feel that in order for this to work, he has to explain the American freedoms of his system. In essence an America UBI imposes a financial action on each person to contribute to other Americans or companies. Some could argue that this deprives them of a freedom to not spend money that way.

I really love his idea for more wholestic medicine valuing quality over quantity. This definitely has roots in Chinese culture and teachings. Once Chinese, one never free one’s self of the feeling that the whole needs more servicing than it gets and the parts less. But it would seem to be a common complaint against thst philosophy that lacking money driven competition, healthcare just cannot be as good as it is today because there wouldn’t be as many people trying as hard to do better than other people. In some sense, people can argue that his idea for reformed medical system is unamerican. Fuethermore it is unrealistic in 2020. But let it be said that I do think human health is a basic human right worthy of national protection, possibly more so than speech, press, assembly, religion, politics, property and money.

I think he will need to publish more books to explain what is planned for the 2020 term.

The Amerian exchange program or citizenship trip he proposes is a horrifying idea: all graduating highschool seniors are scheduled to go on a trip to take part in other parts of America. So… Lets see as recent as the 2010’s Taiwan had a “national” draft that forces every capable man(i.e. highschool grads) to take part in military. So, most Taiwanese man(by “citizenship”) you meet for the next decade, probably all served in Taiwanese millitary for 1-2 years. Indeed, this is well known in the US as its famed computer scientist Feng–hsiung Hsu, maker of DeepBlue which beat human champion in chess, mentioned that he felt liberated to have, at begining of his computer career, come to study at Carnegie Mellon University from a very boring Taiwan draft. Yang’s dad probably had to take part in it too… or not since he seems of a borderline age when Taiwan wasn’t a thing yet.

Thats not the scary part. The sending school-aged kids to an unfamiliar environment so they can learn about fellow citizens was actually a fate suffered by many PRC Chinese. In the 1950’s and 1960’s, Communist China engineered a social movement called 上山下乡，according to wikipedia, to:

decrease difference between industry and agriculture, rural and urban, and physical and mental laborers. All educated kids we sent away from home to live in the worst and most unfamiliar places they could find.

This, btw, is one of the reasons why Chinese think China is still very ver behind other countries in technology and science, their youths were sent to waste time and many died of malnutrition or other dangers. In any case, Yang’s proposal sounds eerily similar in means and ends to PRC’s 上山下乡 movment. Maybe he shares some Hunan ancestry with Mao?

Of course, it is en vogue to copy PRC China. For one example, silicon valley have in recent years adopted this word of a “lead” to mean a role in a company that demands more responsibility and more expertise and of course more political alignment. A non-lead, is a less responsible and lesser expert and may question the plan more than their leads. This in particular is not a word used else where until perhapd very recently replacing or superseding words like “manager”, “director”, “partner”, “president”, and “executive”. Or modifiers like “special agent in charge,” “managing partner,” “lead investigator.” I guess if you search the IMDb you may find a “situation lead” in military context where it is used as a noun. What happened to a good old American title like “boss,” “chief,” “honcho,” or “jefe?” But the “lead” concept is the foundamental management role in PRC China since its early days mid last century. They have roles called 领导，literally “lead and guide”. While there are still leveling, to be a 领导, very much demands political alignment and responsibility, and in particular more than those that you lead. So, it would appear, at least superficially, that copying communist PRC isn’t a real deal breaker. (Even if it was copied from military origins in America, it has been the public knowledge and concern that many of the the hugely successful Chinese enterprises that grew from startup were run by ex-military persons possibly using skills they acquired in the military)

Good effort so far, lets see if the third half of the book is any better in these aspects.

Just had a cystoscope (yeah look that one up), all normal, cancer scare’s over. Last one I’ll need to have now that they’ve established a baseline for me. Went to Gott’s roadside at town&country and had me a Impossible Burger(were half of you thinking dr. Who’s Impossible girl?)

Wow, I’ve been vegetarian for around 20 years, many of those were vegan years. This Burger does taste like a hamburger. I am really surprised that I remember the texture of the the veins and fat popping as I bite into the Burger. The slight smell of heme is definitely a big hint!

The meal was very enjoyable. The cook on it could probably used a bit more sear, jalapeño would hit the spot for me personally, and maybe more burger, as I find myself wanting another one after eating the one I had. The bread is too heavy, or rather he patty could probably be a little fattier so that it doesn’t crumb as much. The fat would also help with the sear. The hint of red is too uniform. I think a little bit more with-in-party variety could do this Berger some good.

I’d definitely order another one for the taste of it. The website says it is made out of coconut oil and potato, so it’ll mostly be for fun I guess.

Next step for me, being in the unfortunate disposition of having only CS training is probably to try to find a way to buy some of their stock. All their jobs (presumably companies by steeply discounted burgers) are non-computer work. Sigh…

I didn’t completely understand the Catholic and Jewish dialogs. I don’t know that Catholics are shun here?! My former Employer Jack Dorsey isn’t shy about his and his family’s afinity for Catholics. Indeed many employees found him quoting from the Popes speeches directly or indirectly when directing his company. I just am completely missing the joke here or I just don’t get this genere thst also includes Beavis and Butthead?

Also, this whole Asia copying America thing. Yes it happens so much and so blatently that its disgusting. But its really not that odd. Copying is a very normal thing to do, I mean your genes do it many times a day… Its part of nature like peeing pooping and bad comedy. Alas, it can be pointed out, while I still have this memory, in 2005, I was working at Yahoo! A bunch of execs just returned from a visit to Asia, mostly because of the typhoon their EA scheduled them to fly into, but also because of the awesome tech they saw. “The Taxi driver has two phones! One for nav and messaging and the other one to making calls… And theres this crazy micro-blogging thing thst every Korean is doing right now, especially young people whom we want to target” The exec blurted out in a single breath…

And sure enough, twitter was founded in the same year to do the same thing. Admittedly, nobody here admits to cooying Asia(or funding an idea that was successful in Asia), but one craze did mature in Asia before the other in America. And of course in Silicone Valley we know that today’s Chinese weibo is a copy of twitter or facebook… But its actually named, literally, a micro-blog. And they’ve had that before we did. Facebook, too, the social network has a broadcasting self-publishing and self-promotional functions that is basically you writing a short text and showing it to a lot of people. It all satisfy some kind of human need, and Asian human needs are just same as American human needs. I mean I’m writting a blog, aren’t I?

(And there are other similar examples, including moocs, escooters, all female executive teams…, etc.)

But I did not know about the horse manure problem of 1894. That was quite a jewel in the rough. Some writer sure sounds inspired…

But with so much other weird things like attacking people who have weird machine related sexual orientation… Thats so rude. We haven’t really established thst his robot was… errr… inappropriately handled. It goes to motive that if no wrong is done, why would he launch an ddos? This whole subplot is just an excuse to shine a derissive light on those with agalmatophilia.

You people make me sick!

Ps ohhh, okay, wait, there was this thing where many thousands of cases of Catholic priests sexually mollesting kids(and mostly male homosexual abuse as seen on TV) and the then holy see hid the fact and kept priests priests… Yeah i guess thats pretty nasty. although I don’t know that Silicone valley is more disgusted by this than people and companies elsewhere.

You know, I abhor Trump’s attack on naturalized citizens who hold opposing political views against him. I do not think it is very presidential or even legal for him to ask American citizens to leave the country for exercising their political voice and powers.

But as a naturalized citizen, I can’t help but cheer him on towards acquisition of 51st State of America. Like who gives a shit about the Mexican wall if we get another state. Americans should carve Greenland coast into a profile of Trumps face to celebrate his presidency if he pulls off a real deal. The New millennial manifest destiny! We can conquer the whole world with American ideals for human rights and our economical political system–by buying it!

Of course, it might very quickly melt or get washed away due to global warming.

I have to admit that the American voluptuary in me is in conflict with my rational scientific self. Both feel nationalistic pride but Trump sure knows how to tug at the American voluptuary’s heart.

Let’s make America Greater! Again!!