Dietmar

The Prime Program produces a file "input_log.txt". This file is stored. Now I write another program, for to put correct information about all no primes and primes into this file. It works. All the numbers in this file are now correct identified as primes or no prime from the prim program above. To teach the prim program, you have to answer with "0" for no prime and "1" for prime. Neural Networks can find pattern. I check with even and not even numbers, works. Here is the generator program for a filled "input_log.txt" until 1000 with correct information, cool Dietmar package generateinputlog; import java.io.BufferedWriter; import java.io.FileWriter; import java.io.IOException; public class GenerateInputLog { public static void main(String[] args) throws IOException { String logFileName = "input_log.txt"; BufferedWriter writer = new BufferedWriter(new FileWriter(logFileName)); for (int i = 0; i <= 1000; i++) { boolean isPrime = isPrime(i); int output = isPrime ? 1 : 0; writer.write(i + "," + output + "\n"); } writer.close(); } private static boolean isPrime(int num) { if (num <= 1) { return false; } for (int i = 2; i <= Math.sqrt(num); i++) { if (num % i == 0) { return false; } } return true; } }

@Mark-XP I write a new Prime Number program with Neuroph. It is the best until now. But it cant find primes. If there is anything, that Neural Networks can found, that it is a pattern. So, even with really BIG learning, no pattern can be found in Primes via a Neural network Dietmar package geradeungeradenetzwerk; import java.io.BufferedWriter; import java.io.FileWriter; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Paths; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.Scanner; import org.neuroph.core.NeuralNetwork; import org.neuroph.core.data.DataSet; import org.neuroph.core.data.DataSetRow; import org.neuroph.nnet.MultiLayerPerceptron; import org.neuroph.nnet.learning.BackPropagation; public class GeradeUngeradeNetzwerk { public static void main(String[] args) throws IOException { String logFileName = "input_log.txt"; Map<Integer, Double> userInputMap = new HashMap<>(); // Load input-output pairs from log file DataSet dataset = new DataSet(1, 1); if (Files.exists(Paths.get(logFileName))) { List<String> lines = Files.readAllLines(Paths.get(logFileName)); for (String line : lines) { String[] parts = line.split(","); int input = Integer.parseInt(parts[0]); double output = Double.parseDouble(parts[1]); double[] inputArr = new double[] { input }; double[] outputArr = new double[] { output }; dataset.add(new DataSetRow(inputArr, outputArr)); userInputMap.put(input, output); } } // Train neural network on input-output pairs MultiLayerPerceptron neuralNet = new MultiLayerPerceptron(1, 300, 300,300,300,300,300,300, 1); BackPropagation learningRule = neuralNet.getLearningRule(); learningRule.setMaxError(0.1); learningRule.setMaxIterations(1000); neuralNet.learn(dataset); // Use trained neural network to classify new integers as prime or not prime Scanner scanner = new Scanner(System.in); while (true) { System.out.print("Enter an integer (or 'exit' to quit): "); String inputStr = scanner.nextLine(); if (inputStr.equals("exit")) { break; } int input; try { input = Integer.parseInt(inputStr); } catch (NumberFormatException e) { System.out.println("Invalid input. Please enter an integer."); continue; } if(userInputMap.containsKey(input)){ double result = userInputMap.get(input); if (result >= 0.5) { System.out.println(input + " is prime (according to the neural network)"); } else { System.out.println(input + " is not prime (according to the neural network)"); } } else{ double[] inputArr = new double[] { input }; neuralNet.setInput(inputArr); neuralNet.calculate(); double[] output = neuralNet.getOutput(); double result = output[0]; if (result >= 0.5) { System.out.println(input + " is prime (according to the neural network)"); } else { System.out.println(input + " is not prime (according to the neural network)"); } // Ask user if result is correct and store input-output pair in log file System.out.print("Is this result correct? (y/n): "); String answer = scanner.nextLine().toLowerCase(); if (answer.equals("y")) { double[] outputArr = new double[]{result}; dataset.add(new DataSetRow(inputArr, outputArr)); BufferedWriter writer = new BufferedWriter(new FileWriter(logFileName, true)); writer.write(input + "," + result + "\n"); writer.close(); userInputMap.put(input, result); } else if (answer.equals("n")) { // Change output to the correct result and add input-output pair to log file System.out.print("What is the correct result? "); String correctResultStr = scanner.nextLine(); try { double correctResult = Double.parseDouble(correctResultStr); double[] outputArr = new double[]{correctResult}; dataset.add(new DataSetRow(inputArr, outputArr)); BufferedWriter writer = new BufferedWriter(new FileWriter(logFileName, true)); writer.write(input + "," + correctResult + "\n"); writer.close(); userInputMap.put(input, correctResult); } catch (NumberFormatException e) { System.out.println("Invalid input, the result was not recorded"); } } } }}}

@Mark-XP Here is a new version. It shows all my helpless (or the helpless of Ai). First I use the Input 0 0 0. Program calculates all Bias and Weights, so that it reaches about 0.5. Then, after 20 Iterations, the last Bias and Values are stored. Now, without any Backpropagation, I use this set of Bias and weights, for the new Input 0 0 1. And now, the crazy things happen: After each run, the output for the 0 0 0 is about 0.5. But the output for the 0 0 1 changes at lot with each run. ChatGPT tells me, to take the Middle of all the used Bias and Weights. But I think, this is helpless as much as possible. So, I have no idea to really train my Network for primes Dietmar package neuralnetwork; import java.util.Arrays; import java.util.concurrent.ThreadLocalRandom; import java.util.Scanner; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[][] weights3; private double[] bias1; private double[] bias2; private double[] bias3; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights3 = new double[numHiddenNodes2][numOutputNodes]; for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias3 = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = Math.tanh(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = Math.tanh(sum);; } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights3[i][j]; } sum += bias3[j]; outputs[j] = Math.tanh(sum);; } return outputs; } public void setWeights1(double[][] weights) { this.weights1 = weights; } public void setBias1(double[] bias) { this.bias1 = bias; } public void setWeights2(double[][] weights) { this.weights2 = weights; } public void setBias2(double[] bias) { this.bias2 = bias; } public void setWeights3(double[][] weights) { this.weights3 = weights; } public void setBias3(double[] bias) { this.bias3 = bias; } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[][] getWeights3() { return this.weights3; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public double[] getBias3() { return this.bias3; } // Backward Propagation public void backPropagate(double[] inputs, double[] expectedOutputs, double learningRate) { // Feed forward to get outputs double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = feedForward(inputs); // Calculate error in output layer double[] outputErrors = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { outputErrors[i] = expectedOutputs[i] - outputs[i]; } // Calculate error in hidden layer 2 double[] hidden2Errors = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { double error = 0; for (int j = 0; j < numOutputNodes; j++) { error += outputErrors[j] * weights3[i][j]; } hidden2Errors[i] = (1 - Math.pow(Math.tanh(hidden2[i]), 2)) * error; } // Calculate error in hidden layer 1 double[] hidden1Errors = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = (1 - Math.pow(Math.tanh(hidden1[i]), 2)) * error; } // Update weights and biases in output layer for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { double delta = outputErrors[j] * tanhDerivative(outputs[j]) * hidden2[i]; weights3[i][j] += learningRate * delta; } } for (int i = 0; i < numOutputNodes; i++) { bias3[i] += learningRate * outputErrors[i] * tanhDerivative(outputs[i]); } // Calculate error in hidden layer 1 for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = error * tanhDerivative(hidden1[i]); } // Update weights and biases in hidden layer 2 for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { double delta = hidden2Errors[j] * tanhDerivative(hidden2[j]) * hidden1[i]; weights2[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes2; i++) { bias2[i] += learningRate * hidden2Errors[i] * tanhDerivative(hidden2[i]); } // Update weights and biases in hidden layer 1 for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { double delta = hidden1Errors[j] * tanhDerivative(hidden1[j]) * inputs[i]; weights1[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes1; i++) { bias1[i] += learningRate * hidden1Errors[i] * tanhDerivative(hidden1[i]); } } // Helper method to calculate the derivative of the hyperbolic tangent function private double tanhDerivative(double x) { double tanh = Math.tanh(x); return 1 - tanh * tanh; } public static void main(String[] args) { NeuralNetwork nn = new NeuralNetwork(3, 4, 4, 1); double[] inputs = {0, 0, 0}; double[] expectedOutputs = {0.5}; for (int i = 0; i < 20; i++) { nn.backPropagate(inputs, expectedOutputs, 0.1); double[] outputs = nn.feedForward(inputs); System.out.println("\nIteration " + (i+1)); System.out.println("Weights1:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println("Bias1:"); System.out.println(Arrays.toString(nn.getBias1())); System.out.println("Weights2:"); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println("Bias2:"); System.out.println(Arrays.toString(nn.getBias2())); System.out.println("Weights3:"); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println("Bias3:"); System.out.println(Arrays.toString(nn.getBias3())); System.out.println("Output:"); System.out.println(Arrays.toString(outputs)); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } Scanner scanner = new Scanner(System.in); System.out.println("\nEnter any key to continue:"); scanner.nextLine(); System.out.println("Initial weights and biases:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println(Arrays.toString(nn.getBias1())); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println(Arrays.toString(nn.getBias2())); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println(Arrays.toString(nn.getBias3())); while (true) { System.out.println("\nEnter 'q' to quit or any other key to continue:"); String input = scanner.nextLine(); if (input.equals("q")) { break; } // Change weight and bias values nn.getWeights1()[1][2] += 0.5; nn.getBias2()[1] -= 0.5; nn.getWeights3()[0][0] *= 2; // Output modified weights and biases System.out.println("\nModified weights and biases:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println(Arrays.toString(nn.getBias1())); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println(Arrays.toString(nn.getBias2())); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println(Arrays.toString(nn.getBias3())); // Feed-forward with new inputs using the last trained weights and biases double[] newInputs = {0, 0, 1}; nn.setWeights1(nn.getWeights1()); nn.setBias1(nn.getBias1()); nn.setWeights2(nn.getWeights2()); nn.setBias2(nn.getBias2()); nn.setWeights3(nn.getWeights3()); nn.setBias3(nn.getBias3()); double[] newOutputs = nn.feedForward(newInputs); System.out.println("New output:"); System.out.println(Arrays.toString(newOutputs)); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } } }

@Mark-XP I notice 2 things: Always the Wights and Bias are different, during different runs. And there appear Weights and Bias >1 and < -1 ??? Chatgpt tells, that this is normal. I would say, it is a big mistake. I really start to wonder, what an Ai is really learning. It can learn facts. But then? As long as there are 2 possible ways, with complete different Weights and Bias for to reach the same result, the Ai has no chance and takes randomally any way. Any "understanding" in such a situation is impossible, Dietmar PS: Those values >1 or <-1 are produced from the Backpropagate algorithmus, brr.. So I think, much more theoretical examination is necessary for Neural Networks.

@Mark-XP A very interesting thing is, to compare all the Bias and Weights after training. I mean, to compare the last Bias and Weights between 2 different runs of the program. As you can see, the values change stetig after some iterations in one and the same run. But I think, that the Neural Network only "understands" the problem, IF in each run at the end the weights are ALL nearly identic. If not, there exist many diffferent solutions for the same(!) task and the Neural Network has no idea, which way to choose Dietmar

@Mark-XP Here is another example. As you can see, it runs also stable with 1000 Neurons in each of the 2 Hidden Layers. Just now, it is more stable than Neuroph. And I also understand, for what the BIG BIG amount of memory is needed: ALL the Bias and Weights and Outputs have to be stored for each iteration. Only then, the network can learn Dietmar PS: Question is, if after training most of the used memory can be set free because only the last Bias, Weights and Outputs have to be stored. But at this example you can see, how much resources during training are really needed. Gigabyte. package neuralnetwork; import java.util.Arrays; import java.util.concurrent.ThreadLocalRandom; import java.util.Scanner; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[][] weights3; private double[] bias1; private double[] bias2; private double[] bias3; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights3 = new double[numHiddenNodes2][numOutputNodes]; for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias3 = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = Math.tanh(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = Math.tanh(sum);; } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights3[i][j]; } sum += bias3[j]; outputs[j] = Math.tanh(sum);; } return outputs; } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[][] getWeights3() { return this.weights3; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public double[] getBias3() { return this.bias3; } // Backward Propagation public void backPropagate(double[] inputs, double[] expectedOutputs, double learningRate) { // Feed forward to get outputs double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = feedForward(inputs); // Calculate error in output layer double[] outputErrors = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { outputErrors[i] = expectedOutputs[i] - outputs[i]; } // Calculate error in hidden layer 2 double[] hidden2Errors = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { double error = 0; for (int j = 0; j < numOutputNodes; j++) { error += outputErrors[j] * weights3[i][j]; } hidden2Errors[i] = (1 - Math.pow(Math.tanh(hidden2[i]), 2)) * error; } // Calculate error in hidden layer 1 double[] hidden1Errors = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = (1 - Math.pow(Math.tanh(hidden1[i]), 2)) * error; } // Update weights and biases in output layer for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { double delta = outputErrors[j] * tanhDerivative(outputs[j]) * hidden2[i]; weights3[i][j] += learningRate * delta; } } for (int i = 0; i < numOutputNodes; i++) { bias3[i] += learningRate * outputErrors[i] * tanhDerivative(outputs[i]); } // Calculate error in hidden layer 1 for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = error * tanhDerivative(hidden1[i]); } // Update weights and biases in hidden layer 2 for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { double delta = hidden2Errors[j] * tanhDerivative(hidden2[j]) * hidden1[i]; weights2[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes2; i++) { bias2[i] += learningRate * hidden2Errors[i] * tanhDerivative(hidden2[i]); } // Update weights and biases in hidden layer 1 for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { double delta = hidden1Errors[j] * tanhDerivative(hidden1[j]) * inputs[i]; weights1[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes1; i++) { bias1[i] += learningRate * hidden1Errors[i] * tanhDerivative(hidden1[i]); } } // Helper method to calculate the derivative of the hyperbolic tangent function private double tanhDerivative(double x) { double tanh = Math.tanh(x); return 1 - tanh * tanh; } public static void main(String[] args) { NeuralNetwork nn = new NeuralNetwork(3, 1000, 1000, 1); double[] inputs = {7, 3, 9}; double[] expectedOutputs = {0.5}; for (int i = 0; i < 20; i++) { nn.backPropagate(inputs, expectedOutputs, 0.00001); double[] outputs = nn.feedForward(inputs); System.out.println("\nIteration " + (i+1)); System.out.println("Weights1:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println("Bias1:"); System.out.println(Arrays.toString(nn.getBias1())); System.out.println("Weights2:"); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println("Bias2:"); System.out.println(Arrays.toString(nn.getBias2())); System.out.println("Weights3:"); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println("Bias3:"); System.out.println(Arrays.toString(nn.getBias3())); System.out.println("Output:"); System.out.println(Arrays.toString(outputs)); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } Scanner scanner = new Scanner(System.in); System.out.println("\nEnter any key to continue:"); scanner.nextLine(); System.out.println("Initial weights and biases:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println(Arrays.toString(nn.getBias1())); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println(Arrays.toString(nn.getBias2())); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println(Arrays.toString(nn.getBias3())); while (true) { System.out.println("\nEnter 'q' to quit or any other key to continue:"); String input = scanner.nextLine(); if (input.equals("q")) { break; } // Change weight and bias values nn.getWeights1()[1][2] += 0.5; nn.getBias2()[1] -= 0.5; nn.getWeights3()[0][0] *= 2; // Output modified weights and biases System.out.println("\nModified weights and biases:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println(Arrays.toString(nn.getBias1())); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println(Arrays.toString(nn.getBias2())); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println(Arrays.toString(nn.getBias3())); // Calculate and output new output with modified weights and biases double[] outputs = nn.feedForward(inputs); System.out.println("New output:"); System.out.println(Arrays.toString(outputs)); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } } }

@Mark-XP Here is the next one, this is really nice !!! You can see, how the program "learns", what a prime is Dietmar package neuralnetwork; import java.util.Arrays; import java.util.concurrent.ThreadLocalRandom; import java.util.Scanner; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[][] weights3; private double[] bias1; private double[] bias2; private double[] bias3; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights3 = new double[numHiddenNodes2][numOutputNodes]; for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias3 = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = Math.tanh(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = Math.tanh(sum);; } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights3[i][j]; } sum += bias3[j]; outputs[j] = Math.tanh(sum);; } return outputs; } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[][] getWeights3() { return this.weights3; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public double[] getBias3() { return this.bias3; } // Backward Propagation public void backPropagate(double[] inputs, double[] expectedOutputs, double learningRate) { // Feed forward to get outputs double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = feedForward(inputs); // Calculate error in output layer double[] outputErrors = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { outputErrors[i] = expectedOutputs[i] - outputs[i]; } // Calculate error in hidden layer 2 double[] hidden2Errors = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { double error = 0; for (int j = 0; j < numOutputNodes; j++) { error += outputErrors[j] * weights3[i][j]; } hidden2Errors[i] = (1 - Math.pow(Math.tanh(hidden2[i]), 2)) * error; } // Calculate error in hidden layer 1 double[] hidden1Errors = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = (1 - Math.pow(Math.tanh(hidden1[i]), 2)) * error; } // Update weights and biases in output layer for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { double delta = outputErrors[j] * tanhDerivative(outputs[j]) * hidden2[i]; weights3[i][j] += learningRate * delta; } } for (int i = 0; i < numOutputNodes; i++) { bias3[i] += learningRate * outputErrors[i] * tanhDerivative(outputs[i]); } // Calculate error in hidden layer 1 for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = error * tanhDerivative(hidden1[i]); } // Update weights and biases in hidden layer 2 for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { double delta = hidden2Errors[j] * tanhDerivative(hidden2[j]) * hidden1[i]; weights2[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes2; i++) { bias2[i] += learningRate * hidden2Errors[i] * tanhDerivative(hidden2[i]); } // Update weights and biases in hidden layer 1 for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { double delta = hidden1Errors[j] * tanhDerivative(hidden1[j]) * inputs[i]; weights1[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes1; i++) { bias1[i] += learningRate * hidden1Errors[i] * tanhDerivative(hidden1[i]); } } // Helper method to calculate the derivative of the hyperbolic tangent function private double tanhDerivative(double x) { double tanh = Math.tanh(x); return 1 - tanh * tanh; } public static void main(String[] args) { NeuralNetwork nn = new NeuralNetwork(3, 2, 2, 1); double[] inputs = {0, 0, 7}; double[] expectedOutputs = {0.5}; for (int i = 0; i < 20; i++) { nn.backPropagate(inputs, expectedOutputs, 0.2); double[] outputs = nn.feedForward(inputs); System.out.println("\nIteration " + (i+1)); System.out.println("Weights1:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println("Bias1:"); System.out.println(Arrays.toString(nn.getBias1())); System.out.println("Weights2:"); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println("Bias2:"); System.out.println(Arrays.toString(nn.getBias2())); System.out.println("Weights3:"); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println("Bias3:"); System.out.println(Arrays.toString(nn.getBias3())); System.out.println("Output:"); System.out.println(Arrays.toString(outputs)); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } Scanner scanner = new Scanner(System.in); System.out.println("\nEnter any key to continue:"); scanner.nextLine(); System.out.println("Initial weights and biases:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println(Arrays.toString(nn.getBias1())); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println(Arrays.toString(nn.getBias2())); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println(Arrays.toString(nn.getBias3())); while (true) { System.out.println("\nEnter 'q' to quit or any other key to continue:"); String input = scanner.nextLine(); if (input.equals("q")) { break; } // Change weight and bias values nn.getWeights1()[1][2] += 0.5; nn.getBias2()[1] -= 0.5; nn.getWeights3()[0][0] *= 2; // Output modified weights and biases System.out.println("\nModified weights and biases:"); System.out.println(Arrays.deepToString(nn.getWeights1())); System.out.println(Arrays.toString(nn.getBias1())); System.out.println(Arrays.deepToString(nn.getWeights2())); System.out.println(Arrays.toString(nn.getBias2())); System.out.println(Arrays.deepToString(nn.getWeights3())); System.out.println(Arrays.toString(nn.getBias3())); // Calculate and output new output with modified weights and biases double[] outputs = nn.feedForward(inputs); System.out.println("New output:"); System.out.println(Arrays.toString(outputs)); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } } }

@Mark-XP This is my first try, using the network with Tanh from above, for to train it for Prime numbers. This is really crazy hard work. A Prime number is found, when the output is near to +0.5. This I have to teach by hand until now, if the input is a prime or not. When the output is near to -0.5 it is called "no prime". First I start with 3 Input Neurons, so that I can check primes until 9 9 9. I use 0 0 7 here. To separate the number in its decimales is important, for not to run in problems with normalisation because of Tanh for big numbers Dietmar package neuralnetwork; import java.util.Arrays; import java.util.concurrent.ThreadLocalRandom; import java.util.Scanner; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[][] weights3; private double[] bias1; private double[] bias2; private double[] bias3; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights3 = new double[numHiddenNodes2][numOutputNodes]; for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias3 = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = Math.tanh(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = Math.tanh(sum);; } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights3[i][j]; } sum += bias3[j]; outputs[j] = Math.tanh(sum);; } return outputs; } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[][] getWeights3() { return this.weights3; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public double[] getBias3() { return this.bias3; } // Backward Propagation public void backPropagate(double[] inputs, double[] expectedOutputs, double learningRate) { // Feed forward to get outputs double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = feedForward(inputs); // Calculate error in output layer double[] outputErrors = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { outputErrors[i] = expectedOutputs[i] - outputs[i]; } // Calculate error in hidden layer 2 double[] hidden2Errors = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { double error = 0; for (int j = 0; j < numOutputNodes; j++) { error += outputErrors[j] * weights3[i][j]; } hidden2Errors[i] = (1 - Math.pow(Math.tanh(hidden2[i]), 2)) * error; } // Calculate error in hidden layer 1 double[] hidden1Errors = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = (1 - Math.pow(Math.tanh(hidden1[i]), 2)) * error; } // Update weights and biases in output layer for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { double delta = outputErrors[j] * tanhDerivative(outputs[j]) * hidden2[i]; weights3[i][j] += learningRate * delta; } } for (int i = 0; i < numOutputNodes; i++) { bias3[i] += learningRate * outputErrors[i] * tanhDerivative(outputs[i]); } // Calculate error in hidden layer 1 for (int i = 0; i < numHiddenNodes1; i++) { double error = 0; for (int j = 0; j < numHiddenNodes2; j++) { error += hidden2Errors[j] * weights2[i][j]; } hidden1Errors[i] = error * tanhDerivative(hidden1[i]); } // Update weights and biases in hidden layer 2 for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { double delta = hidden2Errors[j] * tanhDerivative(hidden2[j]) * hidden1[i]; weights2[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes2; i++) { bias2[i] += learningRate * hidden2Errors[i] * tanhDerivative(hidden2[i]); } // Update weights and biases in hidden layer 1 for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { double delta = hidden1Errors[j] * tanhDerivative(hidden1[j]) * inputs[i]; weights1[i][j] += learningRate * delta; } } for (int i = 0; i < numHiddenNodes1; i++) { bias1[i] += learningRate * hidden1Errors[i] * tanhDerivative(hidden1[i]); } } // Helper method to calculate the derivative of the hyperbolic tangent function private double tanhDerivative(double x) { double tanh = Math.tanh(x); return 1 - tanh * tanh; } public static void main(String[] args) { NeuralNetwork nn = new NeuralNetwork(3, 4, 5, 1); double[] inputs = {0, 0, 7}; double[] expectedOutputs = {0.5}; for (int i = 0; i < 10000; i++) { nn.backPropagate(inputs, expectedOutputs, 0.01); } double[] outputs = nn.feedForward(inputs); System.out.println(Arrays.toString(outputs)); } }

@Mark-XP This is Tanh, which I use now Dietmar

@Mark-XP Here, the Neurons always fire. So, it is not possible with only positiv values, to make a neuron shut its mouth. But with negativ values it can be done. One Neuron just can delete to zero the output of another. When you look for pattern, intensive as much as possible, you may need this. Not important, if your input is always positiv Dietmar

@Mark-XP What I want is very easy: I want a maximal stable Neural Network for BIG tasks Dietmar

@Mark-XP Have you tested my last program with Tanh? Until now, it works only for 2 Hidden Layers. But anything else you can change Dietmar

@Mark-XP Now I use Sigmoid ==> Tanh. For the very first time I see negative values for the Output. For me it is strange, why Sigmoid does not work correct Dietmar package neuralnetwork; import java.util.Arrays; import java.util.concurrent.ThreadLocalRandom; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[][] weights3; private double[] bias1; private double[] bias2; private double[] bias3; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights3 = new double[numHiddenNodes2][numOutputNodes]; for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias3 = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = Math.tanh(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = Math.tanh(sum);; } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights3[i][j]; } sum += bias3[j]; outputs[j] = Math.tanh(sum);; } return outputs; } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[][] getWeights3() { return this.weights3; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public double[] getBias3() { return this.bias3; } public static void main(String[] args) { // Create a new neural network with 2 input neurons, 2 Hidden Layer, 2 neurons per Hidden Layer, and 1 output neuron NeuralNetwork nn = new NeuralNetwork(2, 2,2, 1); // Set the input values double[] inputs = {0.5, 0.7}; // Calculate the output double[] outputs = nn.feedForward(inputs); // Print the weights, biases, and output System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1())); System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2())); System.out.println("Weights 3: " + Arrays.deepToString(nn.getWeights3())); System.out.println("Bias 1: " + Arrays.toString(nn.getBias1())); System.out.println("Bias 2: " + Arrays.toString(nn.getBias2())); System.out.println("Bias 3: " + Arrays.toString(nn.getBias3())); System.out.println("Output: " + Arrays.toString(outputs)); } } run: Weights 1: [[-0.591580065902009, -0.9453068657325465], [-0.8451442791495898, 0.15755068403508]] Weights 2: [[-0.03842387110007617, 0.47422479845076615], [-0.602792967964829, 0.5721493318277344]] Weights 3: [[-0.42546222327748273], [0.6576758395859805]] Bias 1: [-0.9988761406206685, -0.45190281545054556] Bias 2: [0.9550748640633424, -0.3193549269165996] Bias 3: [0.27808333998786816] Output: [-0.5632479238220491] BUILD SUCCESSFUL (total time: 0 seconds)

@Mark-XP I changed the random function in my program. Is this now better Dietmar package neuralnetwork; import java.util.Arrays; import java.util.concurrent.ThreadLocalRandom; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[][] weights3; private double[] bias1; private double[] bias2; private double[] bias3; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.weights3 = new double[numHiddenNodes2][numOutputNodes]; for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { this.weights3[i][j] = ThreadLocalRandom.current().nextDouble(-1, 1); } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } this.bias3 = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { this.bias3[i] = ThreadLocalRandom.current().nextDouble(-1, 1); } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = sigmoid(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = sigmoid(sum); } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights3[i][j]; } sum += bias3[j]; outputs[j] = sigmoid(sum); } return outputs; } private double sigmoid(double x) { return 1.0 / (1.0 + Math.exp(-x)); } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[][] getWeights3() { return this.weights3; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public double[] getBias3() { return this.bias3; } public static void main(String[] args) { // Create a new neural network with 3 input neurons, 4 hidden neurons per layer, and 1 output neuron NeuralNetwork nn = new NeuralNetwork(2, 2,2, 1); // Set the input values double[] inputs = {0.5, 0.7}; // Calculate the output double[] outputs = nn.feedForward(inputs); // Print the weights, biases, and output System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1())); System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2())); System.out.println("Weights 3: " + Arrays.deepToString(nn.getWeights3())); System.out.println("Bias 1: " + Arrays.toString(nn.getBias1())); System.out.println("Bias 2: " + Arrays.toString(nn.getBias2())); System.out.println("Bias 3: " + Arrays.toString(nn.getBias3())); System.out.println("Output: " + Arrays.toString(outputs)); } }

@Mark-XP I just correct my second example. Crazy hard work. I thought, that the values are really random in Java, but who knows. I make a try with other random Dietmar

And the next one package neuralnetwork; import java.util.Arrays; import java.util.Random; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[][] weights3; private double[] bias1; private double[] bias2; private double[] bias3; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; Random rand = new Random(); this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = rand.nextDouble() * 2 - 1; } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = rand.nextDouble() * 2 - 1; } } this.weights3 = new double[numHiddenNodes2][numOutputNodes]; for (int i = 0; i < numHiddenNodes2; i++) { for (int j = 0; j < numOutputNodes; j++) { this.weights3[i][j] = rand.nextDouble() * 2 - 1; } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = rand.nextDouble() * 2 - 1; } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = rand.nextDouble() * 2 - 1; } this.bias3 = new double[numOutputNodes]; for (int i = 0; i < numOutputNodes; i++) { this.bias3[i] = rand.nextDouble() * 2 - 1; } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = sigmoid(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = sigmoid(sum); } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights3[i][j]; } sum += bias3[j]; outputs[j] = sigmoid(sum); } return outputs; } private double sigmoid(double x) { return 1.0 / (1.0 + Math.exp(-x)); } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[][] getWeights3() { return this.weights3; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public double[] getBias3() { return this.bias3; } public static void main(String[] args) { // Create a new neural network with 3 input neurons, 4 hidden neurons per layer, and 1 output neuron NeuralNetwork nn = new NeuralNetwork(2, 2,2, 1); // Set the input values double[] inputs = {0.5, 0.7}; // Calculate the output double[] outputs = nn.feedForward(inputs); // Print the weights, biases, and output System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1())); System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2())); System.out.println("Weights 3: " + Arrays.deepToString(nn.getWeights3())); System.out.println("Bias 1: " + Arrays.toString(nn.getBias1())); System.out.println("Bias 2: " + Arrays.toString(nn.getBias2())); System.out.println("Bias 3: " + Arrays.toString(nn.getBias3())); System.out.println("Output: " + Arrays.toString(outputs)); } } run: Weights 1: [[0.09303725391486473, 0.2273131996623583], [-0.20910441351577114, -0.2548189285854723]] Weights 2: [[0.662298013491458, 0.724727889610413], [0.31748519498752814, 0.7246618090408234]] Weights 3: [[0.7929154750701544], [-0.3285913401755325]] Bias 1: [0.7367533853940502, 0.7135902655585928] Bias 2: [-0.03510499932477007, -0.005652189644535399] Bias 3: [-0.9887913773443431] Output: [0.32910816325148523] BUILD SUCCESSFUL (total time: 0 seconds)

@Mark-XP Next try. here is the output for this example package neuralnetwork; import java.util.Arrays; import java.util.Random; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes1; private int numHiddenNodes2; private int numOutputNodes; private double[][] weights1; private double[][] weights2; private double[] bias1; private double[] bias2; public NeuralNetwork(int numInputNodes, int numHiddenNodes1, int numHiddenNodes2, int numOutputNodes) { this.numInputNodes = numInputNodes; this.numHiddenNodes1 = numHiddenNodes1; this.numHiddenNodes2 = numHiddenNodes2; this.numOutputNodes = numOutputNodes; Random rand = new Random(); this.weights1 = new double[numInputNodes][numHiddenNodes1]; for (int i = 0; i < numInputNodes; i++) { for (int j = 0; j < numHiddenNodes1; j++) { this.weights1[i][j] = rand.nextDouble() * 2 - 1; } } this.weights2 = new double[numHiddenNodes1][numHiddenNodes2]; for (int i = 0; i < numHiddenNodes1; i++) { for (int j = 0; j < numHiddenNodes2; j++) { this.weights2[i][j] = rand.nextDouble() * 2 - 1; } } this.bias1 = new double[numHiddenNodes1]; for (int i = 0; i < numHiddenNodes1; i++) { this.bias1[i] = rand.nextDouble() * 2 - 1; } this.bias2 = new double[numHiddenNodes2]; for (int i = 0; i < numHiddenNodes2; i++) { this.bias2[i] = rand.nextDouble() * 2 - 1; } } public double[] feedForward(double[] inputs) { double[] hidden1 = new double[numHiddenNodes1]; double[] hidden2 = new double[numHiddenNodes2]; double[] outputs = new double[numOutputNodes]; // Calculate outputs of hidden layer 1 for (int j = 0; j < numHiddenNodes1; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += inputs[i] * weights1[i][j]; } sum += bias1[j]; hidden1[j] = sigmoid(sum); } // Calculate outputs of hidden layer 2 for (int j = 0; j < numHiddenNodes2; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes1; i++) { sum += hidden1[i] * weights2[i][j]; } sum += bias2[j]; hidden2[j] = sigmoid(sum); } // Calculate outputs for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes2; i++) { sum += hidden2[i] * weights2[i][j]; } outputs[j] = sigmoid(sum); } return outputs; } private double sigmoid(double x) { return 1.0 / (1.0 + Math.exp(-x)); } public double[][] getWeights1() { return this.weights1; } public double[][] getWeights2() { return this.weights2; } public double[] getBias1() { return this.bias1; } public double[] getBias2() { return this.bias2; } public static void main(String[] args) { // Create a new neural network with 2 input neurons, 2 hidden neurons per layer, and 1 output neuron NeuralNetwork nn = new NeuralNetwork(2, 2, 2, 1); // Set the input values double[] inputs = {0.5, 0.7}; // Calculate the output double[] outputs = nn.feedForward(inputs); // Print the weights, biases, and output System.out.println("Weights 1: " + Arrays.deepToString(nn.getWeights1())); System.out.println("Weights 2: " + Arrays.deepToString(nn.getWeights2())); System.out.println("Bias 1: " + Arrays.toString(nn.getBias1())); System.out.println("Bias 2: " + Arrays.toString(nn.getBias2())); System.out.println("Output: " + Arrays.toString(outputs)); } } INPUT LAYER: Neuron 1 (input1) = 0.5 Neuron 2 (input2) = 0.7 HIDDEN LAYER 1: Neuron 1: - Weight 1 (from input1) = -0.7027589339310366 - Weight 2 (from input2) = 0.33105745845013817 - Bias = 0.13582257112693045 Neuron 2: - Weight 1 (from input1) = -0.4714845612294083 - Weight 2 (from input2) = -0.05963021907204609 - Bias = 0.2444519533561199 HIDDEN LAYER 2: Neuron 1: - Weight 1 (from hidden1 neuron 1) = -0.3751207787477896 - Weight 2 (from hidden1 neuron 2) = 0.1042063242730766 - Bias = 0.8879381568571372 Neuron 2: - Weight 1 (from hidden1 neuron 1) = 0.08391533816837345 - Weight 2 (from hidden1 neuron 2) = -0.05156610086577956 - Bias = -0.38783790451084085 OUTPUT LAYER: Neuron 1: - Weight 1 (from hidden2 neuron 1) = 0.6365834130026754 - Weight 2 (from hidden2 neuron 2) = -0.40315098335735446 - Bias = 0.42197805928249526 OUTPUT: [0.32523829093880324]

@George King This attempt will not work. The i219 is very much different from i218, i217.. The only way is, to use the Win7 driver. But for this you need ndis6. I made a try with Vista Longhorn 5048. I has ndis6. There was only one missed dependency Dietmar

What can I say now about Artificial Intelligence? Each calculation is primitive as much as possible. Nature shows in last 4.5 Billion years, that this concept with Hidden Layers is more sucessfull than any other. It is strict sequential. But parts can run at the same time. Interesting here is, that those artificial Neurons always fire. So, not all good from natural Neuron is still in compi as today, 31 March 2023. When you know Gauß "Methode der kleinsten Quadrate" and Matrix Multiplication, thats all. But: With enough computing power, no more invention is needed. Those machines can recognice every pattern, if there is one. For example I noticed, that in the Prime Numbers are more stored in, than only its divisibilty. So, in principle we already build a Super Intelligence, which is more capable than any human will ever be. Only the enormous power consumption gives as at the moment a chance to survive: 10.000.000.000 more power need any cpu, compared with Human brain. This problem will only be solved with Quantum Computers. The same Algorithm are there already. Until now, a machine with Intelligence of normal human person, needs all US power plants Dietmar

@Mark-XP Oh, this is such crazy hard work. But here is next step for the Network with more than 1 Hidden Layer. It also gives out ALL the weights and Bias, so you can calculate it by hand (much more crazy^^). I did. I calculate and check all by hand, brrr.. The program works correct. But I notice some things, that need to be upgraded in the next version of this program. 1.) Until now, a single Neuron gets the same Weights on all of its arms. This should be changed. 2.) The Hidden Layers are treated all as the same. So no different numbers of Neurons in each Hidden Layer is allowed. 3.) Here we get for the Weights and the Bias values from 0...1. But this should be changed for generality to values from -1...+1. In my thinking, only 1.) and 3.) need to be changed Dietmar package neuralnetwork; import java.util.Arrays; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes; private int numOutputNodes; private int numHiddenLayers; private double[][] inputHiddenWeights; private double[][] hiddenOutputWeights; private double[] hiddenBias; private double[] outputBias; public NeuralNetwork(int numInputNodes, int numHiddenNodes, int numOutputNodes, int numHiddenLayers) { this.numInputNodes = numInputNodes; this.numHiddenNodes = numHiddenNodes; this.numOutputNodes = numOutputNodes; this.numHiddenLayers = numHiddenLayers; // initialize weights and biases randomly inputHiddenWeights = new double[numHiddenNodes][numHiddenNodes]; hiddenOutputWeights = new double[numHiddenNodes][numOutputNodes]; hiddenBias = new double[numHiddenNodes]; outputBias = new double[numOutputNodes]; for (double[] row : inputHiddenWeights) { Arrays.fill(row, Math.random()); } for (double[] row : hiddenOutputWeights) { Arrays.fill(row, Math.random()); } for (int i = 0; i < numHiddenNodes; i++) { hiddenBias[i] = Math.random(); } for (int i = 0; i < numOutputNodes; i++) { outputBias[i] = Math.random(); } } public double sigmoid(double x) { return 1 / (1 + Math.exp(-x)); } public double[] forwardPropagation(double[] input) { // calculate activations for first hidden layer double[] hiddenActivations = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += input[i] * inputHiddenWeights[i][j]; } hiddenActivations[j] = sigmoid(sum + hiddenBias[j]); } // calculate activations for subsequent hidden layers for (int layer = 1; layer < numHiddenLayers; layer++) { double[] nextHiddenActivations = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes; i++) { sum += hiddenActivations[i] * inputHiddenWeights[i][j]; } nextHiddenActivations[j] = sigmoid(sum + hiddenBias[j]); } hiddenActivations = nextHiddenActivations; } // calculate output layer activations double[] outputActivations = new double[numOutputNodes]; for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes; i++) { sum += hiddenActivations[i] * hiddenOutputWeights[i][j]; } outputActivations[j] = sigmoid(sum + outputBias[j]); } return outputActivations; } public void printWeightsAndBiases() { System.out.println("Input-hidden weights:"); for (int i = 0; i < numInputNodes; i++) { System.out.println(Arrays.toString(inputHiddenWeights[i])); } System.out.println("Hidden-output weights:"); for (int i = 0; i < numHiddenNodes; i++) { System.out.println(Arrays.toString(hiddenOutputWeights[i])); } System.out.println("Hidden bias:"); System.out.println(Arrays.toString(hiddenBias)); System.out.println("Output bias:"); System.out.println(Arrays.toString(outputBias)); } public static void main(String[] args) { // create neural network with 2 input nodes, 4 hidden nodes, 1 output node, and 2 hidden layers NeuralNetwork nn = new NeuralNetwork(2, 4, 1, 2); // test forward propagation with input [0.5, 0.7] double[] input = {0.5, 0.7}; double[] output = nn.forwardPropagation(input); System.out.println(Arrays.toString(output)); // print output // print weights and biases nn.printWeightsAndBiases(); } }

@Mark-XP Here is a first try with Backpropagation for to learn the "AND" function. Nice, works! Only Java Standard Bibliothek is used Dietmar To train a program to learn the AND function, we can use a neural network. A neural network is a type of machine learning model that consists of multiple layers of interconnected nodes (neurons) that can learn to recognize patterns in data. For this particular problem, we can create a neural network with two input nodes, one output node, and one hidden layer with a variable number of neurons. The two input nodes correspond to the two binary inputs of the AND function, and the output node corresponds to the output of the function. To train the neural network, we need to provide it with a set of input-output pairs (also known as training examples) for the AND function. For example, one input-output pair could be (0,0) -> 0, meaning that when the input is (0,0), the output should be 0. We can create a set of four such input-output pairs for the AND function using the truth table above. We can then randomly initialize the weights and biases of the neural network, and use a training algorithm (such as stochastic gradient descent) to adjust the weights and biases based on the training examples. The goal is to minimize the difference between the output of the neural network and the desired output for each training example. As the neural network trains, it learns to recognize the patterns in the input-output pairs and adjusts its weights and biases to better predict the output for each input. Once the training process is complete, the neural network should be able to correctly predict the output of the AND function for any given input. In summary, the program learns the AND function by using a neural network with two input nodes, one output node, and one hidden layer with a variable number of neurons. It is trained using a set of input-output pairs for the AND function and a training algorithm that adjusts the weights and biases of the neural network based on the training examples. The neural network learns to recognize patterns in the input-output pairs and adjusts its weights and biases to better predict the output for each input. package neuralnetwork; import java.util.Arrays; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes; private int numOutputNodes; private int numHiddenLayers; private double[][] inputHiddenWeights; private double[][] hiddenOutputWeights; private double[] hiddenBias; private double[] outputBias; public NeuralNetwork(int numInputNodes, int numHiddenNodes, int numOutputNodes, int numHiddenLayers) { this.numInputNodes = numInputNodes; this.numHiddenNodes = numHiddenNodes; this.numOutputNodes = numOutputNodes; this.numHiddenLayers = numHiddenLayers; // initialize weights and biases randomly inputHiddenWeights = new double[numInputNodes][numHiddenNodes]; hiddenOutputWeights = new double[numHiddenNodes][numOutputNodes]; hiddenBias = new double[numHiddenNodes]; outputBias = new double[numOutputNodes]; for (double[] row : inputHiddenWeights) { Arrays.fill(row, Math.random()); } for (double[] row : hiddenOutputWeights) { Arrays.fill(row, Math.random()); } for (int i = 0; i < numHiddenNodes; i++) { hiddenBias[i] = Math.random(); } for (int i = 0; i < numOutputNodes; i++) { outputBias[i] = Math.random(); } } public double sigmoid(double x) { return 1 / (1 + Math.exp(-x)); } public double sigmoidDerivative(double x) { double fx = sigmoid(x); return fx * (1 - fx); } public double[] forwardPropagation(double[] input) { // calculate activations for first hidden layer double[] hiddenActivations = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += input[i] * inputHiddenWeights[i][j]; } hiddenActivations[j] = sigmoid(sum + hiddenBias[j]); } // calculate activations for subsequent hidden layers for (int layer = 1; layer < numHiddenLayers; layer++) { double[] nextHiddenActivations = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes; i++) { sum += hiddenActivations[i] * inputHiddenWeights[i][j]; } nextHiddenActivations[j] = sigmoid(sum + hiddenBias[j]); } hiddenActivations = nextHiddenActivations; } // calculate output layer activations double[] outputActivations = new double[numOutputNodes]; for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes; i++) { sum += hiddenActivations[i] * hiddenOutputWeights[i][j]; } outputActivations[j] = sigmoid(sum + outputBias[j]); } return outputActivations; } public void backPropagation(double[] input, double[] targetOutput, double learningRate) { // perform forward propagation to get activations for all layers double[] hiddenActivations = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += input[i] * inputHiddenWeights[i][j]; } hiddenActivations[j] = sigmoid(sum + hiddenBias[j]); } double[] outputActivations = new double[numOutputNodes]; for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes; i++) { sum += hiddenActivations[i] * hiddenOutputWeights[i][j]; } outputActivations[j] = sigmoid(sum + outputBias[j]); } // calculate output layer error double[] outputErrors = new double[numOutputNodes]; for (int j = 0; j < numOutputNodes; j++) { double outputActivation = outputActivations[j]; double targetOutputValue = targetOutput[j]; outputErrors[j] = outputActivation * (1 - outputActivation) * (targetOutputValue - outputActivation); } // calculate hidden layer errors double[] hiddenErrors = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double hiddenActivation = hiddenActivations[j]; double sum = 0; for (int k = 0; k < numOutputNodes; k++) { double outputError = outputErrors[k]; double weight = hiddenOutputWeights[j][k]; sum += outputError * weight; } hiddenErrors[j] = hiddenActivation * (1 - hiddenActivation) * sum; } // update weights and biases for output layer for (int j = 0; j < numOutputNodes; j++) { double outputError = outputErrors[j]; for (int i = 0; i < numHiddenNodes; i++) { double hiddenActivation = hiddenActivations[i]; double delta = learningRate * outputError * hiddenActivation; hiddenOutputWeights[i][j] += delta; } outputBias[j] += learningRate * outputError; } // update weights and biases for hidden layer for (int j = 0; j < numHiddenNodes; j++) { double hiddenError = hiddenErrors[j]; for (int i = 0; i < numInputNodes; i++) { double inputActivation = input[i]; double delta = learningRate * hiddenError * inputActivation; inputHiddenWeights[i][j] += delta; } hiddenBias[j] += learningRate * hiddenError; } } public static void main(String[] args) { // create neural network with 2 input nodes, 2 hidden nodes, and 1 output node NeuralNetwork nn = new NeuralNetwork(2, 2, 1, 1); // define input and target output for AND function double[][] input = {{0, 0}, {0, 1}, {1, 0}, {1, 1}}; double[][] targetOutput = {{0}, {0}, {0}, {1}}; // train network using backpropagation for (int epoch = 0; epoch < 100000; epoch++) { for (int i = 0; i < input.length; i++) { nn.backPropagation(input[i], targetOutput[i], 0.1); } } // test network with some inputs double[] testInput1 = {0, 0}; double[] testInput2 = {0, 1}; double[] testInput3 = {1, 0}; double[] testInput4 = {1, 1}; System.out.println("0 AND 0 = " + nn.forwardPropagation(testInput1)[0]); System.out.println("0 AND 1 = " + nn.forwardPropagation(testInput2)[0]); System.out.println("1 AND 0 = " + nn.forwardPropagation(testInput3)[0]); System.out.println("1 AND 1 = " + nn.forwardPropagation(testInput4)[0]); } }

@Mark-XP Hi, here is the Neural Network with Java from Scratch. What a crazy hard job. It works using only the Java Standard Bibliothek. This program is written in Java, which is a programming language. It's a neural network program, which means it's designed to learn and recognize patterns in data. Neural networks are used in many applications such as image recognition, natural language processing, and recommendation systems. The program starts by defining the NeuralNetwork class, which contains the following variables: numInputNodes: the number of input nodes in the neural network. numHiddenNodes: the number of hidden nodes in the neural network. numOutputNodes: the number of output nodes in the neural network. numHiddenLayers: the number of hidden layers in the neural network. inputHiddenWeights: a matrix of weights between the input layer and the first hidden layer. hiddenOutputWeights: a matrix of weights between the last hidden layer and the output layer. hiddenBias: an array of biases for the hidden nodes. outputBias: an array of biases for the output nodes. The constructor of the NeuralNetwork class takes four arguments: numInputNodes, numHiddenNodes, numOutputNodes, and numHiddenLayers. These arguments define the structure of the neural network. The constructor initializes the inputHiddenWeights, hiddenOutputWeights, hiddenBias, and outputBias variables with random values between 0 and 1. The program then defines a sigmoid function, which is a mathematical function used in neural networks to convert any input value into a value between 0 and 1. The sigmoid function is used to calculate the activation level of each node in the neural network. The forwardPropagation method takes an input array and returns an output array. The input array represents the input data that the neural network is trying to recognize. The forwardPropagation method calculates the activation level of each node in the neural network and returns the activation level of the output nodes as the result. The forwardPropagation method starts by calculating the activation level of the nodes in the first hidden layer. It does this by multiplying the input values by the weights between the input layer and the first hidden layer, adding the biases for each hidden node, and then applying the sigmoid function to the result. This gives the activation level of each node in the first hidden layer. The forwardPropagation method then calculates the activation level of the nodes in the subsequent hidden layers in the same way as the first hidden layer. It does this by multiplying the activation levels of the nodes in the previous hidden layer by the weights between the previous hidden layer and the current hidden layer, adding the biases for each hidden node, and then applying the sigmoid function to the result. Finally, the forwardPropagation method calculates the activation level of the output nodes by multiplying the activation levels of the nodes in the last hidden layer by the weights between the last hidden layer and the output layer, adding the biases for each output node, and then applying the sigmoid function to the result. The main method of the program creates an instance of the NeuralNetwork class with 2 input nodes, 4 hidden nodes, 1 output node, and 1 hidden layer. It then tests the forwardPropagation method with an input array of [0.5, 0.7]. The output of the forwardPropagation method is printed to the console using the Arrays.toString method. The output represents the activation level of the output node(s) for the given input. I hope this explanation helps! Let me know if you have any more questions. Dietmar package neuralnetwork; import java.util.Arrays; public class NeuralNetwork { private int numInputNodes; private int numHiddenNodes; private int numOutputNodes; private int numHiddenLayers; private double[][] inputHiddenWeights; private double[][] hiddenOutputWeights; private double[] hiddenBias; private double[] outputBias; public NeuralNetwork(int numInputNodes, int numHiddenNodes, int numOutputNodes, int numHiddenLayers) { this.numInputNodes = numInputNodes; this.numHiddenNodes = numHiddenNodes; this.numOutputNodes = numOutputNodes; this.numHiddenLayers = numHiddenLayers; // initialize weights and biases randomly inputHiddenWeights = new double[numInputNodes][numHiddenNodes]; hiddenOutputWeights = new double[numHiddenNodes][numOutputNodes]; hiddenBias = new double[numHiddenNodes]; outputBias = new double[numOutputNodes]; for (double[] row : inputHiddenWeights) { Arrays.fill(row, Math.random()); } for (double[] row : hiddenOutputWeights) { Arrays.fill(row, Math.random()); } for (int i = 0; i < numHiddenNodes; i++) { hiddenBias[i] = Math.random(); } for (int i = 0; i < numOutputNodes; i++) { outputBias[i] = Math.random(); } } public double sigmoid(double x) { return 1 / (1 + Math.exp(-x)); } public double[] forwardPropagation(double[] input) { // calculate activations for first hidden layer double[] hiddenActivations = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double sum = 0; for (int i = 0; i < numInputNodes; i++) { sum += input[i] * inputHiddenWeights[i][j]; } hiddenActivations[j] = sigmoid(sum + hiddenBias[j]); } // calculate activations for subsequent hidden layers for (int layer = 1; layer < numHiddenLayers; layer++) { double[] nextHiddenActivations = new double[numHiddenNodes]; for (int j = 0; j < numHiddenNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes; i++) { sum += hiddenActivations[i] * inputHiddenWeights[i][j]; } nextHiddenActivations[j] = sigmoid(sum + hiddenBias[j]); } hiddenActivations = nextHiddenActivations; } // calculate output layer activations double[] outputActivations = new double[numOutputNodes]; for (int j = 0; j < numOutputNodes; j++) { double sum = 0; for (int i = 0; i < numHiddenNodes; i++) { sum += hiddenActivations[i] * hiddenOutputWeights[i][j]; } outputActivations[j] = sigmoid(sum + outputBias[j]); } return outputActivations; } public static void main(String[] args) { // create neural network with 2 input nodes, 4 hidden nodes, 1 output node, and 1 hidden layer NeuralNetwork nn = new NeuralNetwork(2, 4, 1, 1); // test forward propagation with input [0.5, 0.7] double[] input = {0.5, 0.7}; double[] output = nn.forwardPropagation(input); System.out.println(Arrays.toString(output)); // print output } }

@Mark-XP I make a try with Neuroph 2.96. Hangs at exact the same place. So, it is a loong standing bug in Neuroph Dietmar PS: I try to use only the original Java Bibliothek. Hard job, to implement everything by hand for a working Neural Network, but I try.

@Mark-XP During search for Prime Numbers, the Hund, Katze Program crashes again, means it is hanging when setting all the weights new after an "n". So, there is a bug in Neuroph 2.98, which depends not whether it is bit32 or bit64, win10 or XP SP3, or the Java version Dietmar

@Mark-XP Today I make a new try with the "tiere" program. I install Netbeans 17 bit64 on win10 bit64 with latest Java. And voila, no crash now at all with the Neuroph Neural Network Bibliothek from Belgrad. So, I was right, that it is a resources problem. The program uses more than 1Gbyte ram in a session with training with 100 different names for dogs and cats. I run it on the Asrock z690 Extreme board with 12900k cpu and 32 Gbyte ram. Now, this nice program can be used also for looking for primes. Now it is fast. It is a fantastic program, I think nearly everything can be shown with this program, that has to do with Ai Dietmar