TensorFlow Initialize Global Variables: Initialize TensorFlow Variables That Depend On Other TensorFlow Variables

First, we import TensorFlow as tf.

import tensorflow as tf

Then we print the TensorFlow version that we are using.

print(tf.__version__)

We are using TensorFlow 1.0.1.

We start by building the computational graph.

That is, we define the variables and computations that will take place.

However, it’s important to note that nothing is evaluated at this time.

We are going to create two TensorFlow variables that will hold random numbers.

So here is random_tensor_var_one:

random_tensor_var_one = tf.Variable(tf.random_uniform([2, 3, 4], minval=0, maxval=10, dtype=tf.int32, seed=None, name=None))

We use the tf.Variable functionality, tf.random_uniform.

The tensor is going to be a shape 2x3x4, min value 0, max value 10, data type is int32.

Next, we define a second TensorFlow variable the same way:

random_tensor_var_two = tf.Variable(tf.random_uniform([2, 3, 4], minval=0, maxval=10, dtype=tf.int32, seed=None, name=None))

The Python variable that we assign it to is random_tensor_var_two.

Next, we’re going to print the two variables.

First, we print random_tensor_var_one:

print(random_tensor_var_one)

And we see that it is a tensor but we don’t see any values because it hasn’t been initialized yet.

Then we print random_tensor_var_two:

print(random_tensor_var_two)

And you can see that for both variables, they are TensorFlow variables.

However, because they haven’t been initialized, the print functionality just tells us what it is.

It doesn’t tell us the values.

So random_tensor_add_result is going to create a TensorFlow variable using the tf.add functionality.

random_tensor_add_result = tf.Variable(tf.add(random_tensor_var_one, random_tensor_var_two))

That’s going to add our random_tensor_var_one plus random_tensor_var_two.

Then we can print the tensor:

print(random_tensor_add_result)

And see that it is a variable but again, because we have not initialized it or evaluated it, it does not hold any values.

Now that we have created and defined all of the operations of the computational graph, it’s time to run the computational graph we have built.

sess = tf.Session()

So we launch the graph in a session.

Then we initialize all the global variables in the graph.

sess.run(tf.global_variables_initializer())

Whoops!

So we get a TensorFlow error.

The error that we get is "FailedPreconditionError (see above for traceback): Attempting to use uninitialized value Variable".

What happened here is that we have three TensorFlow variables – random_tensor_var_one, random_tensor_var_two, and random_tensor_add_result.

The trouble is that we do not know the order of how the global variable initializer initializes variables.

That is, TensorFlow initializes the variables in a non-deterministic order.

That means the global variable initializer can exhibit different behaviors on different runs, so we have no way to predict which variables are going to be initialized before other variables.

So when the random_tensor_add_result tries to get initialized, adding random_tensor_var_one plus random_tensor_var_two, either one or both of the random_tensor_var_one or random_tensor_var_two may not have been initialized yet, so it throws the error.

To solve this, we have to define operations that will use the initialized values of TensorFlow tensors or variables in a specific way that tells the global variable initializer to use the initialized values and not to try to use the TensorFlow objects before they are initialized.

The way we’re going to do this is by using the TensorFlow initialized_value functionality.

Let’s quit the session and try again.

exit()

We start our Python interpreter again.

# Command line
$ python

We import TensorFlow.

import tensorflow as tf

We define our two TensorFlow variables again.

random_tensor_var_one first:

random_tensor_var_one = tf.Variable(tf.random_uniform([2, 3, 4], minval=0, maxval=10, dtype=tf.int32, seed=None, name=None))

And then random_tensor_var_two:

random_tensor_var_two = tf.Variable(tf.random_uniform([2, 3, 4], minval=0, maxval=10, dtype=tf.int32, seed=None, name=None))

Then we define the random_tensor_add_result variable.

random_tensor_add_result = tf.Variable(tf.add(random_tensor_var_one.initialized_value(), random_tensor_var_two.initialized_value()))

Specifically, we want to make sure that we are using the .initialized_value() functionality for each of them which tells the global variable initializer to wait until random_tensor_var_one and random_tensor_var_two have been initialized before it tries to do the tf.add functionality.

So that way, random_tensor_add_result is using initialized values for the addition of both tensors.

Now that we have created and defined all of the operations of the computational graph, it’s time to run the computational graph we have built.

sess = tf.Session()

So we launch the graph in a session.

Now we initialize all the global variables in the graph:

sess.run(tf.global_variables_initializer())

And bingo!

Notice that we didn’t get any error.

So now that we’ve done this, we can print out our three TensorFlow variables to see what we got.

First, we print out random_tensor_var_one.

print(sess.run(random_tensor_var_one))

Then we print out random_tensor_var_two.

print(sess.run(random_tensor_var_two))

And now we print the addition of random_tensor_var_one plus random_tensor_var_two.

print(sess.run(random_tensor_add_result))

We can look at the last line of each tensor to do a visual doublecheck.

So 3 + 3 is 6, 9 + 2 is 11, 6 + 7 is 13, 7 + 7 is 14.

It worked and it didn’t throw an error.

The last thing we do is we close the TensorFlow session to release the TensorFlow resources we used within this session as they’re no longer required.

sess.close()

That is how you can initialize TensorFlow variables that depend on other TensorFlow variables by using the TensorFlow initialized_value functionality.