I just saw a poignant document-drama on Netflix about the Panama papers, Meryl Streep, Gary Oldman, Antonio a Andreas, Steven Sodernberg. What a power packed short episode on a serious matter!

The issues seems conflated and leaves way too much for speculation. The Chinese episode, again, the Sichuan accent here wasn’t very convincing, is pure speculation. The senior leaders of China, if we are to believe American propaganda, operate for the country–if they bought a house on the Rivera, it was probably following the party’s direction. If a plastic factory was installed, it was probably the party’s intention. Okay, yes, Bo Xilai apparently headed the party for Chongqing. But given how much monitoring happens in the world, I feel that his actions were probably not that extraordinary for people of his peers.

That is of course pure speculation from me.

But to mix it into a story about American taxes… it’s just not right. American public companies has stringent reporting requirements. Yes, everyone probably cheats a little tax here too, but in the case of Amazon, it seems believable for them to not pay corporate tax some years. Amazon is famous for breaking the PE ratio because it had very high stock market capitalization but zero profit for many many years. It earned no money because it invested in growths for decades. Okay, so that may be my speculation too… But have you ever seen Ex-Amazon employees? They are caffeine driven fiends that work crazy long hours at neck braking speeds. Their core values

* customer obsession

* frugality

* bias for action

* ownership

* high bar for talent

* innovation

(And more on their leadership page.)

It is commonly agreed upon that they live to their mottos.

For all those in Hollowood who wants to attack something, can you think of all the poor creatures who slaved under Bezos all those years to finally get your organic face peel, and, what was that expression used in WSJ, the rainbow fart fuel you guzzle, how they got it to you next day or today?

That took real work! That took a lot of real hard work and sacrifice to get it there. That took a lot of money investment to make it happen. And according to US laws, you can carry forward losses to deduct from earnings before computing taxes. I mean it used to be you can carry-back losses too, but not any more since 2018.

For other examples, you may find a lot of unicorns like Ubar, Luft, Teslo,…, Swuare, etc. etc. that is well in debt and losses but is committed to eventually making a kick-ass products or service. They’ll get to deduct their accrued losses too when they get there.

I can’t find myself hating them for making that deduction. I do find myself hating that I can’t deduct enough of my own misspent money, but that’s just not how our law works. If I buy an ice cream and it tastes like vomit, and I lose half a day of work, I cannot personally deduct that from my income. But if a company does it and loses half a day of work day, they get a lot of deductions.

The show, however does not clarify that this was the problem they are complaining about. It also doesn’t complain about expensive luxury office items like… new office buildings, which can be seen as extravagant and gratuitous, are also completely deducted from income. Amazon, and all these other growing and innovating companies are not bad, this show is bad.

Gary Oldman always does creeps so well. And Streep, of course. But this talent seem wasted on a wrong cause imho… or a good cause but hidden in a lot of awfulness.

Perhaps the team will work together and produce a sequel that explains the position a bit more. I do enjoy the onscreen drama.

Ps here is an idea for a new drama: the intricate story of the stories told by new content coming online at streaming media companies like Netflix, Amazon Prime streaming, Disney+, AppleTV, etc., to defame each other. That would be fun to watch.

I saw the tweet from Jack Dorsey live, yesterday–read it as the alerts came in. Twitter will disallow all political candidate and issues advertising. It chose to suffer this sacrifice to its income because it is felt that paid targeting gives political agents too much power to fool individual voters.

According to the news, Facebook has chosen to allow all political ads (for a few), in the name of freedom of speech. (And honestly, freedom of capitalism where the operations of capital is brought to bear in all organized activities)

I feel the righteousness of both sides. Hyper-targeting individuals, at their own stages of life, work, and living, at all times and places, it can overwhelm each of our senses so as to render our collective votes useless to the democracy. Scientifically, this theory stands. Rationally, the removal of organized political advertising means the discourse on twitter will be more democratic. It is a more real-time version of voting in some sense. This serves as a booster the value of each tweet as opinion of an entity and not effort of a political motivation. It corresponds to the source of what we value. It also boosts trustworthiness of the company possibly leading to more faithful users and promoters.

But refusing to sell a valuable commodity that you have an infinite supply of is very anti-free-market and anti-capitalism. What is this? A church state? The operation of business has to submit to the higher-than-thou self righteousness ?! That is so very un-American!! Who should decide what is a political issue and what is not? If climate watch group posts a new research, and advertises their finding, is that political or scientific? The detail of this scheme is devilish in that someone gets to make the call regarding whether a tweet is political or apolitical. This is a thinly vailed censorship with vast latitude for manipulation by twitter!

So, to organize these thoughts we can list the active constraints in this system

1. Capitalism: Money should be able to buy or manipulate everything under human control. How money is used is the freedom bestowed by Capitalism on its constituents. In an ideally implemented market economy, money works just right automatically. Any restrictions to the power and flow of money detracts from Capitalism with free markets.
2. Democracy: right of speech is necessary for the efficacious function of democracy. The purpose of speech as it pertains to democracy lie in the ability of speech to influence an audience, without audience and an affect, speech is irrelevant. (An extension of this speech-right-redux to right-to-be-heard might be the right to be found by internet search.) Reduction of the right to be heard detracts from Democracy. The increase of total paid political speech is effectively a decrease in everyone’s right-to-be-heard, and for all democratic intents and purposes paid political advertising is equivalent to buying away everyone else’s freedom to speech.
3. Technology: we should try to grow and leverage our technology as much as we can. History has shown that invention and application of new technology repeatedly drove social progress in very significant ways. But history also shows that technologies destroy, consequently we have also chosen to set boundaries on our technology to ensure individual and social wellness: cars have speed limit and you “can’t buy a higher speed-limit”. Highly focused targeting and damaging machination like laser pointers are disallowed in most circumstances–and you can’t buy a permit for that either.
4. Balancing Acts: the above constraints are balanced when interacting: One can only cast one vote, cannot buy votes no matter how much money or enthusiasm you have for a candidate or issue. Can’t lie in courts, can’t buy the right to lie in court. Cannot hack voting machines even if you have the technology and the money to do so.

I feel like the goalie at a world cups final overtime shootout–don’t know which company’s political advertisement policy to dive towards on this one.

But I will confirm that the fact that these two media giants feel enough freedom to take these divergent paths is great testament to American society. Our capitalism did not overwhelm every CEO to make the same decision when faced with trade off between those constraints. Our governments permits both paths to be taken. Our technology allows us to both implement and monitor the effects of these choices. The wide latitude they have to pursue that which is righteous is absolutely freaking awesome! That these transpired this week gives proof that America is greater!

Funny thing happened to me today. It’s like that time I found out about integer division // by googling during an interview. This time, it was not an interview, but still a hair raising surprise to me. So the following with numpy arrays:

>>> a = 16796160.0

>>> np.array([a,], ‘float32’) == (a-1)

array([ True])

>>> np.array([a,], ‘float32’) == (a+1)

array([ True])

Cool, huh?

The problem is 32-bit float has about 7 significant digits for integers and this number has 8 digits, the least significant is not kept. Even though numpy came out of the box float64 by default, some tool, somewhere among the Great Python Data-science Stack, the default for float is 32 bits.

The data we deal with now counts in the tens of millions and you cannot count it using float.

I’m observing this unadulterated joy of a young child doing exactly what you shout for him not to do:

Don’t touch that

Don’t eat that

Don’t jump on the bed when wet from shower

Don’t mess up the dress/shirt/hair…

Don’t scream before bedtime.

.

.

.

The more you say, the more serious you sound, the more urgency in your voice, the more joy and giggling and the more persistence.

I wonder if the pervs watching and mucking with my emails and calendars and every single thing I do are like this. The more I say that clandestine spying and interfering with my personal affairs are unfair trespasses on my human rights, the more joy and vigor their actions brings them.

Seriously, stop it.

No means no for men too.

So, it seems that in a system of all radically rational agents, ie give me freedom or give me death, fairness is an essential condition of its existence. If particular rational agents within a society feels that it is unfair to them, the available actions, to modern enlightened human being as agents, include opposing the unfair system. Ideally, a social system needs to have existential support from all constituents for it to be “universal.” If it’s constituents oppose it, it must not be universal. Non-universality of a system in this minimal way is sure sign of under-endowment as a virtuous and desirable system.

So, there. I now feel, after many years of wondering, that fairness must be a very important aspect of any pan-human endeavors, as a practical, spiritual and philosophical matter.

If you are an AGI/deep learning worrier, here is some good news: the seems are starting to show.

By some strange coincidences of cavities and countermeasures, my family have taken to watching videos to combat boredom while brushing two minutes. After a series of experimentation, we discovered the best way to watch a tiny screen is to watch its reflection on the vanity mirrors–the phone or iPad facing away from us. The sound quality is also admirable having bounced off of the same surface.

I discover that I’m able to unlock the iPad using FaceID in reverse. This is iPadOS 13.1.3 circa q4 2019. This seems to be an artifact of ConvNet training that renders the network insensitive to reflections. (Most naively if you imagine the trainer showed the network each image but each time modified the image by adding splotches or shifting or rotating or reflecting) This was popular in the 2010’s because it builds a more robust network that can more reliably detect objects.

While there might be a chance that it is necessary to unlock in the mirror to defuse a bomb in a Sci-Fi movie, the more likely opinion comes from security purists: you can fool someone to look into a mirror and unlock their iPhone or iPad. The most likely thing that happened was that reflection locking was introduced inadvertently applying sota DL algorithms.

This is a very exciting field with a lot of problems to solve! Let’s see if this bug/feature is fixed/justified respectively by the time you see this.

(Interview question)

A man enters a public bathroom that does not have paper towel and whose door opens inward into the bathroom and the levered lock latches behind him. The lack of paper towel, in addition to other abundant evidences, suggests that most users do not wash their hands. What can he do to escape the room?

.

.

.

.

.

An idea below

.

.

.

.

.

One suggestion is for him to take a generous portion of liquid soap typically found in such a bathroom into his less favored hand. Then he shall lather the soap onto the lever generously so as to substantially coat all surfaces of the lock lever or otherwise where hand contact is required to open the door. He the opens the door expecting the slippery soap to provide some disinfecting and deodorizing benefit from matters on the door’s lock lever. Upon absconsion, he is then at liberty to wash his dirty soapy hand in a more sanitary bathroom, or just to rub and rinse the soap off at a drinking fountain nearby.

* Extra credit for addressing environmental issues. For example, the side effect of this exit is that he has left the inside bathroom door lock lever soapy. But in the grand scheme of things, this does no great injury to any. If a mindless person opens the door, he surely benefits from the soap-cleaner handle. This person can also just rinse the soap off at the water fountains.

* Extra credit for suggesting that humanity really should have invented a popularizable foot operated or a touch free bathroom door by now.

* Extra credit for discussions of building codes and suggesting that modification of the law, in an orderly fashion of course, be a means of progress forward.

* Extra credit for discussions of starting a grassroots movement to perform this act which will surely cause “the authorities” to take measure and buy paper towels instead of more soap. But maybe use recycled material for that.

* No points off for someone who wants to coast through after some other people dirties themselves to exit.

Some months ago I wrote of a discovery regarding the training of deep regressors using SGD. I have since come to realize the benefit of exponentiating the raw parameters before using them is reasonable, sometimes. It would appear that for approximate second order optimizers, like Adam that I used instead of the SGD that I thought I used, the exponential has the effect of modulating the variance aspect of the optimization. The signal to noise ratio of gradients of for identically distributed A in the two case where and will vary but is largely dependent on A. If A is small, the SNR for is stronger. If A is large, then SNR is stronger for .

The combination of methods would tend to follow gradient more eagerly for neuronal activation less dependent on input than larger dependence on neuronal activation. This appear counter to my originally documented intuition that the larger the dependence neuronal output has on neuronal input the larger the gradient step–mainly due to use of approximate second order SGD. My modification, as you would expect, allows the gradients to move more freely even if they are not normally distributed. The exponentiation of weights is akin to log-transform that we used in linear regression analysis, it lets us use linear methods that rely on normality of error on some systems with non-normally distributed, often heavy tailed, errors.

Therefore although my success with this method stems more from the fortuitous conditioning of my problem than it does from the universality of the approach, it can make sense for a very large, albeit non-universal, set of problems. Subsequently any exponent is equivalent to the corresponding inverse power transform of power .

Stay tuned, more to come as I remove more bugs from the experiments.

Just reviewing the convention of the the trade: “lower layer” in deep network means “closer to input” and “higher layer” means “closer to the output.”

Sometimes early stopping helps to regularize models, other times it seem to have numerical properties. When using learning rate decayers like linear, polynomial or cosine, they allow the rate to be very close to zero for a while. It seems that sometimes this will underflow the change to only some parameters while other parameter changes remain nonzero, The result is that an inaccurate gradient is applied and the model drops in performance. It is detectable of course, but one can probably just snapshot models and choose an earlier model when performance starts going south.

…

Ah, okay, and some time passes, and I found a slew of papers from several years ago pointing out the problem is with the used in Adam, it looks like an underflow because it fell below kudos to the people who found this obvious problem.

P.s. and extra kudos for the people who actually just “fixed it” for me by lowering the value inside their software package.