Mini-batch size and pipeline effects

I have the joy of instructing a child in the techniques of eating recently. It occurs to me that I have not really ever described the chewing process. It took some years for me to finally point out the deficiencies of their eating technique.

First issue may have been caused by our own pedantic habits. We insist to the child that she swallow a bite fully before starting next bite. That’s how we were taught. But after years of struggling with slow eating, I finally decided that perhaps they should put more food into their mouth while the last bite is incomplete. Truth be told that is what I do. The food at the end of chewing is tasteless. Adding more unchewed food in the front helps supplant our taste sensations to stimulate saliva production which in turn helps the food go downs. Our teeth are arranged for staged chewing. Front teeth’s cuts and rear molars grind the food. So pipelining the chewing seems natural.

A second problem with chew-swallow-more procedure is that sometimes the mouth end up with insufficient amount food. The mouth operates inefficiently as the food decreases. This is both in terms of bandwidth (that it is chewing less for every tightening of muscles), and in terms of per-chew efficiency. At the limit where our teeth are close together, the jaw muscles are not terribly strong. At least not mine. The chewing is most forceful when there is at least a quarter inch to an inch of food mass in the mouth. The effect of small bite is that with each tightening of the jaw muscles, less food is being chewed and the food is being chewed less effectively than a fuller mouthful. A similar problem arise when you use batch normalization, the original batch normalization where the normalization stats are literally computed on the mini-batch, if mini-batch size is is too small, the variance will be too high to be very useful.

What a strange thing to have to think about. I have always remembered the opposite directions: small bites, swallow before eating more. But in reality and in reasoning, it seems my de facto strategy of medium amount of food and pipelined chewing is most effective.

I suppose many people figure that out through experimentation, some may even do research in this subject. I wonder what else I did or did not figure out about myself as I grew and grew and grew?

It is, however, pure joy and peace, to be part of childrens’ growing process. Even though often we are made to examine the nature of our realities…, and the letters of our spelling.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s