最近在上网课多在youtube 上，次数多了说话时多少会提到这个邮图必耳，尊称为邮老师。

甲:你最近网球发球加快了好多呀！？在那儿学到的新姿势？

乙：那当然是从邮老师那儿学的啰。

😆

Did we just observe, like, the biggest crossover in Trek history? It’s really hard to tel a joke:

Picard: Picard to Rios, Picard to Rios, please come in. Are you quite done drooling over ancient doctors and all the while living with The Flight Attendant ??

Sheesh! It’s like, given the time travel nature of the current Picard-verse, the story seems quite plausibly connected….

Btw, Picard s2 is FAMx0.4; so far I have watched every episode more than twice on average. The show moves very slowly. Ao a second and third viewing helps to answer that nagging question:”what did I miss?” when the end credit comes on.

For attention mechanisms such as softmax have an interesting behavior when trained with weight decay. It seems that when a parameter used to produce the signal line becomes useless, as evidenced by reduction in the attention to it, it’s gradient tends toward zero. Ultimately the weight control will shrink this variable to zero. The parameter for the attention line for associated with this unimportant feature will continue to decrease into negative. However weight decay would actually reduce this believe that the feature is unimportant as the signal line decrease in amplitude.

In short, the bad feature gets to be relearned from fresh and the model can update its believe about the usefulness of the signal line.

One immediate idea following this view of the attention mechanisms is to update weight decay to decay towards an ideal “initial naive state” that may not be zero. Or one may alter weight decay to gain noise as the value of the parameter approaches the naive state. As an example, the bias term of softmax numerators may target shrinkage to -7, Shrinking to 0 causes the exponentiated result to bias towards 1, where as shrinking to a bias of -7 permits the probability of some classes to decrease to almost zero(1e-3). This reduction results in less label smoothing implicit in softmaxes with bias trained with weight decay. The same can be done during exploration stage we could weight decay the bias towards 1.0 or 2.0 forcing the model to reconsider and explore probabilities more.

My iPad just sped up 25%-40% after upgrading iOS 15.4.1 We’re starting to lose games because they, presumably, now run at their designed performance.

Notably in the news, this version fixes a security hole which allows arbitrary remote execution of code on the device…. So… the mind thinketh perhaps, in all of my memory with this iPad, someone was earning cents constantly mining some crypto coin?

The additional Apple warning about battery consumption increasing seems to indicate that perhaps it wasn’t malware slowing down the iPad. Apple might have simply removed the battery life preserver that slows down the cpu in order to use extend battery life. Maybe this is even liked tot he rumor that Biden may invoke the defense production act to make more raw material for making batteries. If that happens, the battery life may not be such a huge problem.

Anyways, a lot of speculations and lost virtual lives. Feeling kind of embarrassed to be self concerned at a moment when Russia invasion of Ukraine rages on.

Peace!