The sigmoid or activation function and their uses

From Wikipedia:

“Many natural processes, including those of complex system learning curves, exhibit a progression from small beginnings that accelerates and approaches a climax over time. When a detailed description is lacking, a sigmoid function is often used. A sigmoid curve is produced by a mathematical function having an “S” shape. Often, sigmoid function refers to the special case of the logistic function shown at right and defined by the formula”

When you are faced to a mathematical problem, where is necessary to find a output value, (given a continue input ) that behave like near to Boolean, for example, in simulated neural networks or in very range of problem, could be necessary to use this function and adapt it to our needs.

As was previously stated, the graph of sigmoid function has “S” shape, but could be necessary to change the shape of it “s” or their position.

In the previous formula, we have added a and b.

If you change a, for instance, a=10, you will get a short “S”, if you let a=20, “S” will be more short.

If you change b, it will let you move your S in the graph, allowing let it not centred on zero. For example, if you input range (domain of function) is between 0.0 and 1.0, you could need the following function:

Where y=0.5, when x=0.5.

You can see it graph using Google calculator here

I found more information about this function on Dr. Mark Humphrys webpage




2 responses to “The sigmoid or activation function and their uses

  • Katia Absher

    I have to tell you that it’s hard to find your posts
    in google, i found this one on 21 spot, you should build some quality
    backlinks in order to rank your page, i
    know how to help you, just search in google – k2 seo tips

  • Latisha

    I read a lot of interesting posts here. Probably you spend a lot of time
    writing, i know how to save you a lot of time, there is an online tool that creates high quality, SEO friendly posts
    in minutes, just search in google – laranitas free content source

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: