Mini-batch size and pipeline effects

I have the joy of instructing a child in the techniques of eating recently. It occurs to me that I have not really ever described the chewing process. It took some years for me to finally point out the deficiencies of their eating technique.

First issue may have been caused by our own pedantic habits. We insist to the child that she swallow a bite fully before starting next bite. That’s how we were taught. But after years of struggling with slow eating, I finally decided that perhaps they should put more food into their mouth while the last bite is incomplete. Truth be told that is what I do. The food at the end of chewing is tasteless. Adding more unchewed food in the front helps supplant our taste sensations to stimulate saliva production which in turn helps the food go downs. Our teeth are arranged for staged chewing. Front teeth’s cuts and rear molars grind the food. So pipelining the chewing seems natural.

A second problem with chew-swallow-more procedure is that sometimes the mouth end up with insufficient amount food. The mouth operates inefficiently as the food decreases. This is both in terms of bandwidth (that it is chewing less for every tightening of muscles), and in terms of per-chew efficiency. At the limit where our teeth are close together, the jaw muscles are not terribly strong. At least not mine. The chewing is most forceful when there is at least a quarter inch to an inch of food mass in the mouth. The effect of small bite is that with each tightening of the jaw muscles, less food is being chewed and the food is being chewed less effectively than a fuller mouthful. A similar problem arise when you use batch normalization, the original batch normalization where the normalization stats are literally computed on the mini-batch, if mini-batch size is is too small, the variance will be too high to be very useful.

What a strange thing to have to think about. I have always remembered the opposite directions: small bites, swallow before eating more. But in reality and in reasoning, it seems my de facto strategy of medium amount of food and pipelined chewing is most effective.

I suppose many people figure that out through experimentation, some may even do research in this subject. I wonder what else I did or did not figure out about myself as I grew and grew and grew?

It is, however, pure joy and peace, to be part of childrens’ growing process. Even though often we are made to examine the nature of our realities…, and the letters of our spelling.

Does DS9 know about this?

Wait, what? The Dominion is involved in vote tempering in America ???? Quick, some body raise !

Actually, wait, what would the TLD be for Starfleet? It has that militaristic organization, but its driven by directives and principal that goes beyond serve and protect. Maybe they’ll have a TLD called .fed to service federation activities. Idk, there are paradigm shifts that maybe present say analogies just can’t describe.

It’s difficult to imagine a pursuit more noble than space exploration and exploitation as promoted by my most favorite TV show Star Trek. like would there need to be a TLD for FED? Are there any thing else I the future other than the Federation? Maybe the TLD will be .sf for Starfleet.

Thankfully, this is not yet a full blown problem yet today. The evil genius know as Elon Misk still does not have a definitive advent age on space exploration and occupation. But there are news appearing with titles like “Psyche, an asteroid believed to be worth more than $10,000 quadrillion is observed through the Hubble in new study.”

Don’t anybody tell The Chine$e, lest Trump tweeter storm America about its, what if they get a head start on these “rare” earth metals?

Also, did they consider inflation when they did that calculation? Were the metal really become available, their cost would drop so far that it wouldn’t be worth that much. This might be a thing that economist have a name for, but the intrinsic value of that asteroid seems vastly different from its market value due to plentiful ness.

The point being, what do we do when we find out our measly existence and value system is absurdly moronic when a technological shift suddenly changes everything…

Do we drop everything and go digging for precious metal? Do we give up monogamy because… we’ll just because it sucks. Do we fire the president and vote a Misklike person to be our eternal leader? That’s not completely absurd, for $10,000 quadrillion, nothing is absurd. We could reproduce profusely and prolifically until our bodies build a bridge to the stars and a wall against aliens. We could impeach the entire congress and only build rockets and fly, fly, fly!!!

Again, we’re obviously not there yet, but it feels tantalizingly close.

What shall we do then? Will our children live fruitful lives that we may comprehend? Where in the world is our world going ?

I have to get better 二锅头。Who knew these cheap Chinese products can all have same name and be so different in effects…

Ahhh, shoot, I just got it. “Black alert!”

I hope it doesn’t get misinterpreted as … in the context of BLM, … and worst if it is glorified for violence.

And I wonder why in those those thousands of years, now referring to Star Trek Discovery universe, why they haven’t figured out a stable way to make changes using time travel. It feels like there can be multiple equilibria that one can choose from and that achieving a different equilibrium can both be “good for everyone” and “immutable change” that cannot be changed by another time travel. I would be disappointed if that’s a physical impossibility.

Again, I need better …

I Voted in the 2020 Election

I just voted in the 2020 election. Now I have rushed to Starbucks for that ceremoniously celebratory coffee. Democratic capitalism is so great!

Hey, did anybody find the radio buttons on this years voting app non-intuitive? After making a selection, you must in select it (by tapping the already-selected box again) before you can select another one. If you want to update a write-in choice, you must unselect, re-select, and retype the choice. This is kind of counter-intuitive. Most people are used to radio button groups that can unselect.

A related issue is the non-standard keyboard that is arranged alphabetically onscreen. Why not at least let me choose the QWERTY keyboard? This abnormal element of tablet UI surely adds to tech anxiety everyone has developed circa 2020 about stepping up to a new tablet.

Another thing kind of bugging me is the enormous screens in use. I mean I sure feel like this is special, the voting screen is even larger than the screen inside Tesla cars. It seems so long. But for especially engineered screens, it has horrible glare due to the angle and ceiling lights at my voting location. My tax well spent, I think…

Volunteers seem to have underrepresented minorities. I don’t know if white people just have more enthusiasm for democracy or what. Sure there’re those Asian kids trying to get community service hours. And sure there is a Hispanic guard. But election officials in my town were white. This is something we minorities have to try harder at. Yes, work is long and hard, pay is lousy and mouths-to-feed are many, relatively speaking, but you have gotta make the time for election! Minorities, please volunteer !!

And those are my only gripes. Otherwise I am immensely grateful for my safe and uncoerced participation in this sacred event in persons. It was very exciting to see people of all kinds standing in line, six feet apart, in this brisk November morning, to vote. I have voted many times before, but voting in person is definitely very different.


Why is 42 the answer?

Maybe the question is from a maitre’d at a restaurant…

“It’ll just be for(4) two(2) tonight.”

“It’ll just be 42 of us tonight.”

Never doubted the ingenuity of British literature.