AI with Java Netbeans 16, Ant

**Dietmar** · March 31, 2023

Here is a first try with Backpropagation for to learn the "AND" function.

Nice, works :cheerleader: !

Only Java Standard Bibliothek is used

Dietmar

To train a program to learn the AND function, we can use a neural network. A neural network is a type of machine learning model that consists of multiple layers of interconnected nodes (neurons) that can learn to recognize patterns in data.

For this particular problem, we can create a neural network with two input nodes, one output node, and one hidden layer with a variable number of neurons. The two input nodes correspond to the two binary inputs of the AND function, and the output node corresponds to the output of the function.

To train the neural network, we need to provide it with a set of input-output pairs (also known as training examples) for the AND function. For example, one input-output pair could be (0,0) -> 0, meaning that when the input is (0,0), the output should be 0. We can create a set of four such input-output pairs for the AND function using the truth table above.

We can then randomly initialize the weights and biases of the neural network, and use a training algorithm (such as stochastic gradient descent) to adjust the weights and biases based on the training examples. The goal is to minimize the difference between the output of the neural network and the desired output for each training example.

As the neural network trains, it learns to recognize the patterns in the input-output pairs and adjusts its weights and biases to better predict the output for each input. Once the training process is complete, the neural network should be able to correctly predict the output of the AND function for any given input.

In summary, the program learns the AND function by using a neural network with two input nodes, one output node, and one hidden layer with a variable number of neurons. It is trained using a set of input-output pairs for the AND function and a training algorithm that adjusts the weights and biases of the neural network based on the training examples. The neural network learns to recognize patterns in the input-output pairs and adjusts its weights and biases to better predict the output for each input.

package neuralnetwork;

import java.util.Arrays;

public class NeuralNetwork {
    private int numInputNodes;
    private int numHiddenNodes;
    private int numOutputNodes;
    private int numHiddenLayers;

    private double[][] inputHiddenWeights;
    private double[][] hiddenOutputWeights;
    private double[] hiddenBias;
    private double[] outputBias;

    public NeuralNetwork(int numInputNodes, int numHiddenNodes, int numOutputNodes, int numHiddenLayers) {
        this.numInputNodes = numInputNodes;
        this.numHiddenNodes = numHiddenNodes;
        this.numOutputNodes = numOutputNodes;
        this.numHiddenLayers = numHiddenLayers;

        // initialize weights and biases randomly
        inputHiddenWeights = new double[numInputNodes][numHiddenNodes];
        hiddenOutputWeights = new double[numHiddenNodes][numOutputNodes];
        hiddenBias = new double[numHiddenNodes];
        outputBias = new double[numOutputNodes];

        for (double[] row : inputHiddenWeights) {
            Arrays.fill(row, Math.random());
        }

        for (double[] row : hiddenOutputWeights) {
            Arrays.fill(row, Math.random());
        }

        for (int i = 0; i < numHiddenNodes; i++) {
            hiddenBias[i] = Math.random();
        }

        for (int i = 0; i < numOutputNodes; i++) {
            outputBias[i] = Math.random();
        }
    }

    public double sigmoid(double x) {
        return 1 / (1 + Math.exp(-x));
    }

    public double sigmoidDerivative(double x) {
        double fx = sigmoid(x);
        return fx * (1 - fx);
    }

    public double[] forwardPropagation(double[] input) {
        // calculate activations for first hidden layer
        double[] hiddenActivations = new double[numHiddenNodes];

        for (int j = 0; j < numHiddenNodes; j++) {
            double sum = 0;

            for (int i = 0; i < numInputNodes; i++) {
                sum += input[i] * inputHiddenWeights[i][j];
            }

            hiddenActivations[j] = sigmoid(sum + hiddenBias[j]);
        }

        // calculate activations for subsequent hidden layers
        for (int layer = 1; layer < numHiddenLayers; layer++) {
            double[] nextHiddenActivations = new double[numHiddenNodes];

            for (int j = 0; j < numHiddenNodes; j++) {
                double sum = 0;

                for (int i = 0; i < numHiddenNodes; i++) {
                    sum += hiddenActivations[i] * inputHiddenWeights[i][j];
                }

                nextHiddenActivations[j] = sigmoid(sum + hiddenBias[j]);
            }

            hiddenActivations = nextHiddenActivations;
        }

        // calculate output layer activations
        double[] outputActivations = new double[numOutputNodes];

        for (int j = 0; j < numOutputNodes; j++) {
            double sum = 0;

            for (int i = 0; i < numHiddenNodes; i++) {
                sum += hiddenActivations[i] * hiddenOutputWeights[i][j];
            }

            outputActivations[j] = sigmoid(sum + outputBias[j]);
        }

        return outputActivations;
    }

    

public void backPropagation(double[] input, double[] targetOutput, double learningRate) {
    // perform forward propagation to get activations for all layers
    double[] hiddenActivations = new double[numHiddenNodes];

    for (int j = 0; j < numHiddenNodes; j++) {
        double sum = 0;

        for (int i = 0; i < numInputNodes; i++) {
            sum += input[i] * inputHiddenWeights[i][j];
        }

        hiddenActivations[j] = sigmoid(sum + hiddenBias[j]);
    }

    double[] outputActivations = new double[numOutputNodes];

    for (int j = 0; j < numOutputNodes; j++) {
        double sum = 0;

        for (int i = 0; i < numHiddenNodes; i++) {
            sum += hiddenActivations[i] * hiddenOutputWeights[i][j];
        }

        outputActivations[j] = sigmoid(sum + outputBias[j]);
    }

    // calculate output layer error
    double[] outputErrors = new double[numOutputNodes];

    for (int j = 0; j < numOutputNodes; j++) {
        double outputActivation = outputActivations[j];
        double targetOutputValue = targetOutput[j];

        outputErrors[j] = outputActivation * (1 - outputActivation) * (targetOutputValue - outputActivation);
    }

    // calculate hidden layer errors
    double[] hiddenErrors = new double[numHiddenNodes];

    for (int j = 0; j < numHiddenNodes; j++) {
        double hiddenActivation = hiddenActivations[j];
        double sum = 0;

        for (int k = 0; k < numOutputNodes; k++) {
            double outputError = outputErrors[k];
            double weight = hiddenOutputWeights[j][k];
            sum += outputError * weight;
        }

        hiddenErrors[j] = hiddenActivation * (1 - hiddenActivation) * sum;
    }

    // update weights and biases for output layer
    for (int j = 0; j < numOutputNodes; j++) {
        double outputError = outputErrors[j];

        for (int i = 0; i < numHiddenNodes; i++) {
            double hiddenActivation = hiddenActivations[i];
            double delta = learningRate * outputError * hiddenActivation;
            hiddenOutputWeights[i][j] += delta;
        }

        outputBias[j] += learningRate * outputError;
    }

    // update weights and biases for hidden layer
    for (int j = 0; j < numHiddenNodes; j++) {
        double hiddenError = hiddenErrors[j];

        for (int i = 0; i < numInputNodes; i++) {
            double inputActivation = input[i];
            double delta = learningRate * hiddenError * inputActivation;
            inputHiddenWeights[i][j] += delta;
        }

        hiddenBias[j] += learningRate * hiddenError;
    }
}


public static void main(String[] args) {
    // create neural network with 2 input nodes, 2 hidden nodes, and 1 output node
    NeuralNetwork nn = new NeuralNetwork(2, 2, 1, 1);

    // define input and target output for AND function
    double[][] input = {{0, 0}, {0, 1}, {1, 0}, {1, 1}};
    double[][] targetOutput = {{0}, {0}, {0}, {1}};

    // train network using backpropagation
    for (int epoch = 0; epoch < 100000; epoch++) {
        for (int i = 0; i < input.length; i++) {
            nn.backPropagation(input[i], targetOutput[i], 0.1);
        }
    }

    // test network with some inputs
    double[] testInput1 = {0, 0};
    double[] testInput2 = {0, 1};
    double[] testInput3 = {1, 0};
    double[] testInput4 = {1, 1};

    System.out.println("0 AND 0 = " + nn.forwardPropagation(testInput1)[0]);
    System.out.println("0 AND 1 = " + nn.forwardPropagation(testInput2)[0]);
    System.out.println("1 AND 0 = " + nn.forwardPropagation(testInput3)[0]);
    System.out.println("1 AND 1 = " + nn.forwardPropagation(testInput4)[0]);
}
}

Edited March 31, 2023 by Dietmar

**Mark-XP** · March 31, 2023

VERY nice job, @Dietmar! I'm honestly in awe of how you get deeply into the matter... stark!

This last example is extremely useful in regards of how the A"I" is working: let the trainig run loop for only 10.000 times (not 100.000) and the output is absolutely astonishing (at least for me):

0 AND 0 = 0.002703633461159949
0 AND 1 = 0.028244357868107424
1 AND 0 = 0.02999418942102172
1 AND 1 = 0.9603305464776584

For (0 and 0) it is 10 times more secure about the result than for (1 and 0). I derive 2 things: First, it's learning capabilities are terrific, intimidating.

And second: it hasn't understood the essence of the matter at all.

Edited March 31, 2023 by Mark-XP
typo

**Dietmar** · March 31, 2023

@Mark-XP

Oh, this is such crazy hard work.

But here is next step for the Network with more than 1 Hidden Layer.

It also gives out ALL the weights and Bias, so you can calculate it by hand (much more crazy^^).

I did. I calculate and check all by hand, brrr.. The program works correct.

But I notice some things, that need to be upgraded in the next version of this program.

1.) Until now, a single Neuron gets the same Weights on all of its arms. This should be changed.

2.) The Hidden Layers are treated all as the same. So no different numbers of Neurons in each Hidden Layer is allowed.

3.) Here we get for the Weights and the Bias values from 0...1. But this should be changed for generality to values from -1...+1.

In my thinking, only 1.) and 3.) need to be changed

Dietmar

package neuralnetwork;


import java.util.Arrays;

public class NeuralNetwork {
    private int numInputNodes;
    private int numHiddenNodes;
    private int numOutputNodes;
    private int numHiddenLayers;

    private double[][] inputHiddenWeights;
    private double[][] hiddenOutputWeights;
    private double[] hiddenBias;
    private double[] outputBias;

    public NeuralNetwork(int numInputNodes, int numHiddenNodes, int numOutputNodes, int numHiddenLayers) {
        this.numInputNodes = numInputNodes;
        this.numHiddenNodes = numHiddenNodes;
        this.numOutputNodes = numOutputNodes;
        this.numHiddenLayers = numHiddenLayers;

        
        // initialize weights and biases randomly
inputHiddenWeights = new double[numHiddenNodes][numHiddenNodes];
hiddenOutputWeights = new double[numHiddenNodes][numOutputNodes];
hiddenBias = new double[numHiddenNodes];
outputBias = new double[numOutputNodes];

for (double[] row : inputHiddenWeights) {
    Arrays.fill(row, Math.random());
}

for (double[] row : hiddenOutputWeights) {
    Arrays.fill(row, Math.random());
}

for (int i = 0; i < numHiddenNodes; i++) {
    hiddenBias[i] = Math.random();
}

for (int i = 0; i < numOutputNodes; i++) {
    outputBias[i] = Math.random();
}
        
    }       
        

    public double sigmoid(double x) {
        return 1 / (1 + Math.exp(-x));
    }

   
    public double[] forwardPropagation(double[] input) {
    // calculate activations for first hidden layer
    double[] hiddenActivations = new double[numHiddenNodes];

    for (int j = 0; j < numHiddenNodes; j++) {
        double sum = 0;

        for (int i = 0; i < numInputNodes; i++) {
            sum += input[i] * inputHiddenWeights[i][j];
        }

        hiddenActivations[j] = sigmoid(sum + hiddenBias[j]);
    }

    // calculate activations for subsequent hidden layers
    for (int layer = 1; layer < numHiddenLayers; layer++) {
        double[] nextHiddenActivations = new double[numHiddenNodes];

        for (int j = 0; j < numHiddenNodes; j++) {
            double sum = 0;

            for (int i = 0; i < numHiddenNodes; i++) {
                sum += hiddenActivations[i] * inputHiddenWeights[i][j];
            }

            nextHiddenActivations[j] = sigmoid(sum + hiddenBias[j]);
        }

        hiddenActivations = nextHiddenActivations;
    }

    // calculate output layer activations
    double[] outputActivations = new double[numOutputNodes];

    for (int j = 0; j < numOutputNodes; j++) {
        double sum = 0;

        for (int i = 0; i < numHiddenNodes; i++) {
            sum += hiddenActivations[i] * hiddenOutputWeights[i][j];
        }

        outputActivations[j] = sigmoid(sum + outputBias[j]);
    }

    return outputActivations;
}

    
    
 public void printWeightsAndBiases() {
    System.out.println("Input-hidden weights:");
    for (int i = 0; i < numInputNodes; i++) {
        System.out.println(Arrays.toString(inputHiddenWeights[i]));
    }
    System.out.println("Hidden-output weights:");
    for (int i = 0; i < numHiddenNodes; i++) {
        System.out.println(Arrays.toString(hiddenOutputWeights[i]));
    }
    System.out.println("Hidden bias:");
    System.out.println(Arrays.toString(hiddenBias));
    System.out.println("Output bias:");
    System.out.println(Arrays.toString(outputBias));
}

public static void main(String[] args) {
    // create neural network with 2 input nodes, 4 hidden nodes, 1 output node, and 2 hidden layers
    NeuralNetwork nn = new NeuralNetwork(2, 4, 1, 2);

    // test forward propagation with input [0.5, 0.7]
    double[] input = {0.5, 0.7};
    double[] output = nn.forwardPropagation(input);

    System.out.println(Arrays.toString(output)); // print output

    // print weights and biases
    nn.printWeightsAndBiases();
}
}

Edited April 1, 2023 by Dietmar

**Dietmar** · March 31, 2023

What can I say now about Artificial Intelligence?

Each calculation is primitive as much as possible. Nature shows in last 4.5 Billion years,

that this concept with Hidden Layers is more sucessfull than any other. It is strict sequential.

But parts can run at the same time. Interesting here is, that those artificial Neurons always fire.

So, not all good from natural Neuron is still in compi as today, 31 March 2023.

When you know Gauß "Methode der kleinsten Quadrate" and Matrix Multiplication, thats all.

But: With enough computing power, no more invention is needed.

Those machines can recognice every pattern, if there is one.

For example I noticed, that in the Prime Numbers are more stored in, than only its divisibilty.

So, in principle we already build a Super Intelligence, which is more capable than any human will ever be.

Only the enormous power consumption gives as at the moment a chance to survive:

10.000.000.000 more power need any cpu, compared with Human brain.

This problem will only be solved with Quantum Computers. The same Algorithm are there already.

Until now, a machine with Intelligence of normal human person, needs all US power plants

Dietmar

Edited March 31, 2023 by Dietmar

**Mark-XP** · March 31, 2023

5 hours ago, Dietmar said:

But: With enough computing power, no more invention is needed.

Those machines can recognice every pattern, if there is one.

Yes @Dietmar, you're absolutely right!

Did you know that today most new pharaceutical ingredients/agents for medicines are found by AI? The same for primes or chess: when the spaces of possibilities are sheer endless and the solutions very rare, the machine will beat humans.

You probably know that Stephen Hawking warned about the situation when AI will begin to improve itself? A realy dystopian idea...

Edited March 31, 2023 by Mark-XP
link added

**Dietmar** · April 1, 2023

@Mark-XP

Next try. here is the output for this example

package neuralnetwork;
import java.util.Arrays;
import java.util.Random;

public class NeuralNetwork {
    private int numInputNodes;
    private int numHiddenNodes1;
    private int numHiddenNodes2;
    private int numOutputNodes;
    private double[][] weights1;
    private double[][] weights2;
    private double[] bias1;
    private double[] bias2;

    public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) {
        this.numInputNodes = numInputNodes;
        this.numHiddenNodes1 = numHiddenNodes1;
        this.numHiddenNodes2 = numHiddenNodes2;
        this.numOutputNodes = numOutputNodes;

        Random rand = new Random();
        this.weights1 = new double[numInputNodes][numHiddenNodes1];
        for (int i = 0; i < numInputNodes; i++) {
            for (int j = 0; j < numHiddenNodes1; j++) {
                this.weights1[i][j] = rand.nextDouble() * 2 - 1;
            }
        }

        this.weights2 = new double[numHiddenNodes1][numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes1; i++) {
            for (int j = 0; j < numHiddenNodes2; j++) {
                this.weights2[i][j] = rand.nextDouble() * 2 - 1;
            }
        }

        this.bias1 = new double[numHiddenNodes1];
        for (int i = 0; i < numHiddenNodes1; i++) {
            this.bias1[i] = rand.nextDouble() * 2 - 1;
        }

        this.bias2 = new double[numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes2; i++) {
            this.bias2[i] = rand.nextDouble() * 2 - 1;
        }
    }

    
    
    public double[] feedForward(double[] inputs) {
    double[] hidden1 = new double[numHiddenNodes1];
    double[] hidden2 = new double[numHiddenNodes2];
    double[] outputs = new double[numOutputNodes];

    // Calculate outputs of hidden layer 1
    for (int j = 0; j < numHiddenNodes1; j++) {
        double sum = 0;
        for (int i = 0; i < numInputNodes; i++) {
            sum += inputs[i] * weights1[i][j];
        }
        sum += bias1[j];
        hidden1[j] = sigmoid(sum);
    }

    // Calculate outputs of hidden layer 2
    for (int j = 0; j < numHiddenNodes2; j++) {
        double sum = 0;
        for (int i = 0; i < numHiddenNodes1; i++) {
            sum += hidden1[i] * weights2[i][j];
        }
        sum += bias2[j];
        hidden2[j] = sigmoid(sum);
    }

    // Calculate outputs
    for (int j = 0; j < numOutputNodes; j++) {
        double sum = 0;
        for (int i = 0; i < numHiddenNodes2; i++) {
            sum += hidden2[i] * weights2[i][j];
        }
        outputs[j] = sigmoid(sum);
    }

    return outputs;
}
    
    

    private double sigmoid(double x) {
        return 1.0 / (1.0 + Math.exp(-x));
    }

    
public double[][] getWeights1() {
    return this.weights1;
}

public double[][] getWeights2() {
    return this.weights2;
}

public double[] getBias1() {
    return this.bias1;
}

public double[] getBias2() {
    return this.bias2;
}

public static void main(String[] args) {
    // Create a new neural network with 2 input neurons, 2 hidden neurons per layer, and 1 output neuron
    NeuralNetwork nn = new NeuralNetwork(2, 2, 2, 1);

    // Set the input values
    double[] inputs = {0.5, 0.7};

    // Calculate the output
    double[] outputs = nn.feedForward(inputs);

    // Print the weights, biases, and output
    System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1()));
    System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2()));
    System.out.println("Bias 1: " + Arrays.toString(nn.getBias1()));
    System.out.println("Bias 2: " + Arrays.toString(nn.getBias2()));
    System.out.println("Output: " + Arrays.toString(outputs));
}
}

INPUT LAYER:
Neuron 1 (input1) = 0.5
Neuron 2 (input2) = 0.7

HIDDEN LAYER 1:
Neuron 1:
    - Weight 1 (from input1) = -0.7027589339310366
    - Weight 2 (from input2) = 0.33105745845013817
    - Bias = 0.13582257112693045
Neuron 2:
    - Weight 1 (from input1) = -0.4714845612294083
    - Weight 2 (from input2) = -0.05963021907204609
    - Bias = 0.2444519533561199

HIDDEN LAYER 2:
Neuron 1:
    - Weight 1 (from hidden1 neuron 1) = -0.3751207787477896
    - Weight 2 (from hidden1 neuron 2) = 0.1042063242730766
    - Bias = 0.8879381568571372
Neuron 2:
    - Weight 1 (from hidden1 neuron 1) = 0.08391533816837345
    - Weight 2 (from hidden1 neuron 2) = -0.05156610086577956
    - Bias = -0.38783790451084085

OUTPUT LAYER:
Neuron 1:
    - Weight 1 (from hidden2 neuron 1) = 0.6365834130026754
    - Weight 2 (from hidden2 neuron 2) = -0.40315098335735446
    - Bias = 0.42197805928249526

OUTPUT:
[0.32523829093880324]

**Dietmar** · April 1, 2023

And the next one

package neuralnetwork;

import java.util.Arrays;
import java.util.Random;

public class NeuralNetwork {
    private int numInputNodes;
    private int numHiddenNodes1;
    private int numHiddenNodes2;
    private int numOutputNodes;
    private double[][] weights1;
    private double[][] weights2;
    private double[][] weights3;
    private double[] bias1;
    private double[] bias2;
    private double[] bias3;

    public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) {
        this.numInputNodes = numInputNodes;
        this.numHiddenNodes1 = numHiddenNodes1;
        this.numHiddenNodes2 = numHiddenNodes2;
        this.numOutputNodes = numOutputNodes;

        Random rand = new Random();
        this.weights1 = new double[numInputNodes][numHiddenNodes1];
        for (int i = 0; i < numInputNodes; i++) {
            for (int j = 0; j < numHiddenNodes1; j++) {
                this.weights1[i][j] = rand.nextDouble() * 2 - 1;
            }
        }

        this.weights2 = new double[numHiddenNodes1][numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes1; i++) {
            for (int j = 0; j < numHiddenNodes2; j++) {
                this.weights2[i][j] = rand.nextDouble() * 2 - 1;
            }
        }
        
        this.weights3 = new double[numHiddenNodes2][numOutputNodes];
for (int i = 0; i < numHiddenNodes2; i++) {
    for (int j = 0; j < numOutputNodes; j++) {
        this.weights3[i][j] = rand.nextDouble() * 2 - 1;
    }
}

        this.bias1 = new double[numHiddenNodes1];
        for (int i = 0; i < numHiddenNodes1; i++) {
            this.bias1[i] = rand.nextDouble() * 2 - 1;
        }

        this.bias2 = new double[numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes2; i++) {
            this.bias2[i] = rand.nextDouble() * 2 - 1;
        }

        this.bias3 = new double[numOutputNodes];
        for (int i = 0; i < numOutputNodes; i++) {
            this.bias3[i] = rand.nextDouble() * 2 - 1;
        }
    }

    public double[] feedForward(double[] inputs) {
        double[] hidden1 = new double[numHiddenNodes1];
        double[] hidden2 = new double[numHiddenNodes2];
        double[] outputs = new double[numOutputNodes];

        // Calculate outputs of hidden layer 1
        for (int j = 0; j < numHiddenNodes1; j++) {
            double sum = 0;
            for (int i = 0; i < numInputNodes; i++) {
                sum += inputs[i] * weights1[i][j];
            }
            sum += bias1[j];
            hidden1[j] = sigmoid(sum);
        }

        // Calculate outputs of hidden layer 2
        for (int j = 0; j < numHiddenNodes2; j++) {
            double sum = 0;
            for (int i = 0; i < numHiddenNodes1; i++) {
                sum += hidden1[i] * weights2[i][j];
            }
            sum += bias2[j];
            hidden2[j] = sigmoid(sum);
        }

        

// Calculate outputs
for (int j = 0; j < numOutputNodes; j++) {
    double sum = 0;
    for (int i = 0; i < numHiddenNodes2; i++) {
        sum += hidden2[i] * weights3[i][j];
    }
    sum += bias3[j];
    outputs[j] = sigmoid(sum);
}

return outputs;
}

private double sigmoid(double x) {
return 1.0 / (1.0 + Math.exp(-x));
}

public double[][] getWeights1() {
return this.weights1;
}

public double[][] getWeights2() {
return this.weights2;
}

public double[][] getWeights3() {
return this.weights3;
}

public double[] getBias1() {
return this.bias1;
}

public double[] getBias2() {
return this.bias2;
}
public double[] getBias3() {
return this.bias3;

}

public static void main(String[] args) {
// Create a new neural network with 3 input neurons, 4 hidden neurons per layer, and 1 output neuron
NeuralNetwork nn = new NeuralNetwork(2, 2,2, 1);



// Set the input values
double[] inputs = {0.5, 0.7};

// Calculate the output
double[] outputs = nn.feedForward(inputs);

// Print the weights, biases, and output
System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1()));
System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2()));
System.out.println("Weights 3: " + Arrays.deepToString(nn.getWeights3()));
System.out.println("Bias 1: " + Arrays.toString(nn.getBias1()));
System.out.println("Bias 2: " + Arrays.toString(nn.getBias2()));
System.out.println("Bias 3: " + Arrays.toString(nn.getBias3()));
System.out.println("Output: " + Arrays.toString(outputs));

}
}

run:
Weights 1: [[0.09303725391486473, 0.2273131996623583], [-0.20910441351577114, -0.2548189285854723]]
Weights 2: [[0.662298013491458, 0.724727889610413], [0.31748519498752814, 0.7246618090408234]]
Weights 3: [[0.7929154750701544], [-0.3285913401755325]]
Bias 1: [0.7367533853940502, 0.7135902655585928]
Bias 2: [-0.03510499932477007, -0.005652189644535399]
Bias 3: [-0.9887913773443431]
Output: [0.32910816325148523]
BUILD SUCCESSFUL (total time: 0 seconds)

Edited April 1, 2023 by Dietmar

**Mark-XP** · April 1, 2023

@Dietmar, all the weights are initialised randomly, hence the output seem to be somewhat Gauss distributed with center around the average of the both inputs. Is this a deeper investigation of how a NN works (2 hidden layers x 2 nodes, recalculated (forwardfed) utilizing sigmoid function?

**Dietmar** · April 1, 2023

@Mark-XP

I just correct my second example.

Crazy hard work. I thought, that the values are really random in Java, but who knows.

I make a try with other random

Dietmar

**Dietmar** · April 1, 2023

@Mark-XP

I changed the random function in my program.

Is this now better

Dietmar

package neuralnetwork;


import java.util.Arrays;
import java.util.concurrent.ThreadLocalRandom;

public class NeuralNetwork {
    private int numInputNodes;
    private int numHiddenNodes1;
    private int numHiddenNodes2;
    private int numOutputNodes;
    private double[][] weights1;
    private double[][] weights2;
    private double[][] weights3;
    private double[] bias1;
    private double[] bias2;
    private double[] bias3;

    public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) {
        this.numInputNodes = numInputNodes;
        this.numHiddenNodes1 = numHiddenNodes1;
        this.numHiddenNodes2 = numHiddenNodes2;
        this.numOutputNodes = numOutputNodes;

        this.weights1 = new double[numInputNodes][numHiddenNodes1];
        for (int i = 0; i < numInputNodes; i++) {
            for (int j = 0; j < numHiddenNodes1; j++) {
                this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1);
            }
        }

        this.weights2 = new double[numHiddenNodes1][numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes1; i++) {
            for (int j = 0; j < numHiddenNodes2; j++) {
                this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1);
            }
        }

        this.weights3 = new double[numHiddenNodes2][numOutputNodes];
        for (int i = 0; i < numHiddenNodes2; i++) {
            for (int j = 0; j < numOutputNodes; j++) {
                this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1);
            }
        }

        this.bias1 = new double[numHiddenNodes1];
        for (int i = 0; i < numHiddenNodes1; i++) {
            this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1);
        }

        this.bias2 = new double[numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes2; i++) {
            this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1);
        }

        this.bias3 = new double[numOutputNodes];
        for (int i = 0; i < numOutputNodes; i++) {
            this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1);
        }
    }

  






    public double[] feedForward(double[] inputs) {
        double[] hidden1 = new double[numHiddenNodes1];
        double[] hidden2 = new double[numHiddenNodes2];
        double[] outputs = new double[numOutputNodes];

        // Calculate outputs of hidden layer 1
        for (int j = 0; j < numHiddenNodes1; j++) {
            double sum = 0;
            for (int i = 0; i < numInputNodes; i++) {
                sum += inputs[i] * weights1[i][j];
            }
            sum += bias1[j];
            hidden1[j] = sigmoid(sum);
        }

        // Calculate outputs of hidden layer 2
        for (int j = 0; j < numHiddenNodes2; j++) {
            double sum = 0;
            for (int i = 0; i < numHiddenNodes1; i++) {
                sum += hidden1[i] * weights2[i][j];
            }
            sum += bias2[j];
            hidden2[j] = sigmoid(sum);
        }

        

// Calculate outputs
for (int j = 0; j < numOutputNodes; j++) {
    double sum = 0;
    for (int i = 0; i < numHiddenNodes2; i++) {
        sum += hidden2[i] * weights3[i][j];
    }
    sum += bias3[j];
    outputs[j] = sigmoid(sum);
}

return outputs;
}

private double sigmoid(double x) {
return 1.0 / (1.0 + Math.exp(-x));
}

public double[][] getWeights1() {
return this.weights1;
}

public double[][] getWeights2() {
return this.weights2;
}

public double[][] getWeights3() {
return this.weights3;
}

public double[] getBias1() {
return this.bias1;
}

public double[] getBias2() {
return this.bias2;
}
public double[] getBias3() {
return this.bias3;

}

public static void main(String[] args) {
// Create a new neural network with 3 input neurons, 4 hidden neurons per layer, and 1 output neuron
NeuralNetwork nn = new NeuralNetwork(2, 2,2, 1);



// Set the input values
double[] inputs = {0.5, 0.7};

// Calculate the output
double[] outputs = nn.feedForward(inputs);

// Print the weights, biases, and output
System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1()));
System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2()));
System.out.println("Weights 3: " + Arrays.deepToString(nn.getWeights3()));
System.out.println("Bias 1: " + Arrays.toString(nn.getBias1()));
System.out.println("Bias 2: " + Arrays.toString(nn.getBias2()));
System.out.println("Bias 3: " + Arrays.toString(nn.getBias3()));
System.out.println("Output: " + Arrays.toString(outputs));

}
}

**Dietmar** · April 1, 2023

@Mark-XP

Now I use Sigmoid ==> Tanh.

For the very first time I see negative values for the Output.

For me it is strange, why Sigmoid does not work correct

Dietmar

package neuralnetwork;

import java.util.Arrays;
import java.util.concurrent.ThreadLocalRandom;

public class NeuralNetwork {
    private int numInputNodes;
    private int numHiddenNodes1;
    private int numHiddenNodes2;
    private int numOutputNodes;
    private double[][] weights1;
    private double[][] weights2;
    private double[][] weights3;
    private double[] bias1;
    private double[] bias2;
    private double[] bias3;

    public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) {
        this.numInputNodes = numInputNodes;
        this.numHiddenNodes1 = numHiddenNodes1;
        this.numHiddenNodes2 = numHiddenNodes2;
        this.numOutputNodes = numOutputNodes;

        this.weights1 = new double[numInputNodes][numHiddenNodes1];
        for (int i = 0; i < numInputNodes; i++) {
            for (int j = 0; j < numHiddenNodes1; j++) {
                this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1);
            }
        }

        this.weights2 = new double[numHiddenNodes1][numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes1; i++) {
            for (int j = 0; j < numHiddenNodes2; j++) {
                this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1);
            }
        }

        this.weights3 = new double[numHiddenNodes2][numOutputNodes];
        for (int i = 0; i < numHiddenNodes2; i++) {
            for (int j = 0; j < numOutputNodes; j++) {
                this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1);
            }
        }

        this.bias1 = new double[numHiddenNodes1];
        for (int i = 0; i < numHiddenNodes1; i++) {
            this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1);
        }

        this.bias2 = new double[numHiddenNodes2];
        for (int i = 0; i < numHiddenNodes2; i++) {
            this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1);
        }

        this.bias3 = new double[numOutputNodes];
        for (int i = 0; i < numOutputNodes; i++) {
            this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1);
        }
    }

    public double[] feedForward(double[] inputs) {
        double[] hidden1 = new double[numHiddenNodes1];
        double[] hidden2 = new double[numHiddenNodes2];
        double[] outputs = new double[numOutputNodes];

        // Calculate outputs of hidden layer 1
        for (int j = 0; j < numHiddenNodes1; j++) {
            double sum = 0;
            for (int i = 0; i < numInputNodes; i++) {
                sum += inputs[i] * weights1[i][j];
            }
            sum += bias1[j];
            hidden1[j] = Math.tanh(sum);
        }

        

        // Calculate outputs of hidden layer 2
        for (int j = 0; j < numHiddenNodes2; j++) {
            double sum = 0;
            for (int i = 0; i < numHiddenNodes1; i++) {
                sum += hidden1[i] * weights2[i][j];
            }
            sum += bias2[j];
            hidden2[j] = Math.tanh(sum);;
        }

        

// Calculate outputs
for (int j = 0; j < numOutputNodes; j++) {
    double sum = 0;
    for (int i = 0; i < numHiddenNodes2; i++) {
        sum += hidden2[i] * weights3[i][j];
    }
    sum += bias3[j];
    outputs[j] = Math.tanh(sum);;
}

return outputs;
}



public double[][] getWeights1() {
return this.weights1;
}

public double[][] getWeights2() {
return this.weights2;
}

public double[][] getWeights3() {
return this.weights3;
}

public double[] getBias1() {
return this.bias1;
}

public double[] getBias2() {
return this.bias2;
}
public double[] getBias3() {
return this.bias3;

}

public static void main(String[] args) {
// Create a new neural network with 2 input neurons, 2 Hidden Layer, 2 neurons per Hidden Layer, and 1 output neuron
NeuralNetwork nn = new NeuralNetwork(2, 2,2, 1);



// Set the input values
double[] inputs = {0.5, 0.7};

// Calculate the output
double[] outputs = nn.feedForward(inputs);

// Print the weights, biases, and output
System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1()));
System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2()));
System.out.println("Weights 3: " + Arrays.deepToString(nn.getWeights3()));
System.out.println("Bias 1: " + Arrays.toString(nn.getBias1()));
System.out.println("Bias 2: " + Arrays.toString(nn.getBias2()));
System.out.println("Bias 3: " + Arrays.toString(nn.getBias3()));
System.out.println("Output: " + Arrays.toString(outputs));

}
}

run:
Weights 1: [[-0.591580065902009, -0.9453068657325465], [-0.8451442791495898, 0.15755068403508]]
Weights 2: [[-0.03842387110007617, 0.47422479845076615], [-0.602792967964829, 0.5721493318277344]]
Weights 3: [[-0.42546222327748273], [0.6576758395859805]]
Bias 1: [-0.9988761406206685, -0.45190281545054556]
Bias 2: [0.9550748640633424, -0.3193549269165996]
Bias 3: [0.27808333998786816]
Output: [-0.5632479238220491]
BUILD SUCCESSFUL (total time: 0 seconds)

Edited April 1, 2023 by Dietmar

**Mark-XP** · April 1, 2023

27 minutes ago, Dietmar said:

@Mark-XP

Now I use Sigmoid ==> Tanh.

For the very first time I see negative values.

For me it is strange, why Sigmoid does not work correct

@Dietmar This seems rather obvious since Tanh is negative for a nagative input (tanh "=" 2 x sigmoid - 1).

Edit: since i don't know what you wanna achieve, i cannot consider it correct or wrong :rolleyes:

Edited April 1, 2023 by Mark-XP

**Dietmar** · April 1, 2023

@Mark-XP

Have you tested my last program with Tanh?

Until now, it works only for 2 Hidden Layers.

But anything else you can change

Dietmar

**Dietmar** · April 1, 2023

@Mark-XP

What I want is very easy:

I want a maximal stable Neural Network for BIG tasks

Dietmar

**Mark-XP** · April 1, 2023

26 minutes ago, Dietmar said:

@Mark-XP Have you tested my last program with Tanh?

@Dietmar Yes: now the output (in several runs) seems more equal distributed (between -1 and 1). Tanh is flatter than sigmoid, hence outputs seem to be more ,scattered'.

Hence, for a stable NN i would prefer a sigmoid TransferFunction (rather than Tanh). (at my currently low knowledge state about NNs)

Edited April 1, 2023 by Mark-XP

Sign In

AI with Java Netbeans 16, Ant

Recommended Posts

Dietmar

Mark-XP

Dietmar

Dietmar

Mark-XP

Dietmar

Dietmar

Mark-XP

Dietmar

Dietmar

Dietmar

Mark-XP

Dietmar

Dietmar

Mark-XP

Please sign in to comment

Recently Browsing 0 members

Activity

Browse