The sigmoid or activation function and their uses

From Wikipedia:

“Many natural processes, including those of complex system learning curves, exhibit a progression from small beginnings that accelerates and approaches a climax over time. When a detailed description is lacking, a sigmoid function is often used. A sigmoid curve is produced by a mathematical function having an “S” shape. Often, sigmoid function refers to the special case of the logistic function shown at right and defined by the formula”

When you are faced to a mathematical problem, where is necessary to find a output value, (given a continue input ) that behave like near to Boolean, for example, in simulated neural networks or in very range of problem, could be necessary to use this function and adapt it to our needs.

As was previously stated, the graph of sigmoid function has “S” shape, but could be necessary to change the shape of it “s” or their position.

In the previous formula, we have added a and b.

If you change a, for instance, a=10, you will get a short “S”, if you let a=20, “S” will be more short.

If you change b, it will let you move your S in the graph, allowing let it not centred on zero. For example, if you input range (domain of function) is between 0.0 and 1.0, you could need the following function:

Where y=0.5, when x=0.5.

You can see it graph using Google calculator here

I found more information about this function on Dr. Mark Humphrys webpage

Regards,

Álvaro