Keras is a popular deep learning library that simplifies the process of building neural networks. One of the key components in a neural network is the activation function. It helps introduce non-linearity to the network, allowing it to learn complex patterns and make accurate predictions.

In this blog post, we will explore various activation functions available in Keras and showcase how to use them in Python.

1. Sigmoid Activation function

The sigmoid activation function is commonly used in binary classification problems. It squashes the input between 0 and 1, making it suitable for predicting probabilities.

import keras
from keras.layers import Dense

# Create a dense layer with sigmoid activation function
model.add(Dense(units=10, activation='sigmoid'))

2. ReLU (Rectified Linear Unit) Activation function

ReLU is a widely used activation function in deep neural networks. It is defined as f(x) = max(0, x). ReLU helps alleviate the vanishing gradient problem and provides faster convergence.

import keras
from keras.layers import Dense

# Create a dense layer with ReLU activation function
model.add(Dense(units=10, activation='relu'))

3. Tanh (Hyperbolic Tangent) Activation function

The hyperbolic tangent activation function tanh(x) is similar to the sigmoid function, but it squashes the input between -1 and 1. It is commonly used in recurrent neural networks.

import keras
from keras.layers import Dense

# Create a dense layer with tanh activation function
model.add(Dense(units=10, activation='tanh'))

4. Softmax Activation function

The softmax activation function is primarily used in multi-class classification problems. It normalizes the outputs into a probability distribution, making it easier to interpret the final predictions.

import keras
from keras.layers import Dense

# Create a dense layer with softmax activation function
model.add(Dense(units=10, activation='softmax'))

5. Custom Activation function

Keras also allows you to define your own custom activation functions. You can define a function that takes the tensor as an input and returns the transformed tensor.

import keras
from keras.layers import Activation

# Define a custom activation function using the Keras backend
def custom_activation(x):
    return keras.backend.square(x)  # Square the input tensor

# Create a dense layer with custom activation function
model.add(Dense(units=10))
model.add(Activation(custom_activation))

In conclusion, Keras provides a wide range of activation functions to choose from, each suitable for different types of problems. Understanding the characteristics and appropriate use cases of these functions can greatly improve the performance of your neural networks.

Remember to experiment with different activation functions to find the one that performs best for your specific task.

Happy coding!