Mind, Mathematics & Feature Selection

Gustav Fechner (1801-87) discovered that the way humans perceive the intensity of sensory inputs is logarithmically proportional to the absolute magnitude of stimulus as measured by non-living measuring devices.

That is, intensity of light, sound, and heat is perceived in log scales. Additionally, it has been shown that people, who are not trained in school mathematics, consider “3” to be the mid-point of 1 and 9, which is a correct answer if the numbers are innately imagined in log scale.

In a 2012 paper titled “A framework for Bayesian optimality of psychophysical laws”, researchers showed that formulating a Bayesian framework for the scaling of perception and find logarithmic and related scalings are optimal under expected relative error fidelity.

There must be strong evolutionary reasons for sensory perceptions happening at log scales. A predator who is originally three meters away, moving closer to you by two meter has completely different meaning that a predator who was originally 30 meters away and had moved by the same distance.

So log scales are very common in nature. It seems therefore that using log scales on feature vectors during doing machine learning would be a very “natural” thing to do. They may be particularly helpful with perceptual models.

Hence we use log scales as a common tool when modelling a new phenomena using machine learning. We would be interested in your experiences… do you find log scaling an useful exercise?

Image Credit: http://www.flickr.com/photos/chop/4329524896/in/photolist-7AzWUY-7Awbkn/

Share on Facebook Tweet
comments powered by Disqus