Create TensorFlow Name Scopes For TensorBoard

In this video, we’re going to use TensorFlow name scopes to group graph notes together in the TensorBoard web service so that your graph visualization is legible.

First, we import TensorFlow as tf.

import tensorflow as tf

Then we print out the version of TensorFlow we are using.

print(tf.__version__)

We are using TensorFlow 1.8.0.

The example we’re going to create in this video is to add four named TensorFlow scalars together using the TensorFlow add operation.

First, we’ll add two scalars together using TensorFlow’s add operation.

Then we’ll add another two scalars together.

Finally, we’ll add the two results.

Let’s get started with our first name scope.

with tf.name_scope('scalar_set_one') as scope:

So we say with tf.name_scope.

We’re going to set the name to 'scalar_set_one' as scope.

Now that we have the scope, let’s define our first constant scalar.

with tf.name_scope('scalar_set_one') as scope:
    tf_constant_one = tf.constant(10, name="ten")

So we use four spaces, then we use the tf.constant.

We give it the number 10, we give it the name of ten, and we assign it to the Python variable tf_constant_one.

Next, we define our second scalar the same way, four spaces in front of the Python variable, we use tf.constant, we give it the value of 20, the name of twenty, and we assign it to tf_constant_two.

with tf.name_scope('scalar_set_one') as scope:
    tf_constant_one = tf.constant(10, name="ten")
    tf_constant_two = tf.constant(20, name="twenty")

Finally, we do our tf.add with tf_constant_one, tf_constant_two, and we give it the name scalar_ten_plus_twenty.

with tf.name_scope('scalar_set_one') as scope:
    tf_constant_one = tf.constant(10, name="ten")
    tf_constant_two = tf.constant(20, name="twenty")
    scalar_sum_one = tf.add(tf_constant_one, tf_constant_two, name="scalar_ten_plus_twenty")

We assign that to the Python variable scalar_sum_one, again leaving four spaces.

Note that we leave four spaces between the start of the line where our code is to make sure Python knows that we are still under the name scope code block.

Next, let’s hit return on an empty line to signal to Python that we are finished with this code block.

Let’s now print the Python variable tf_constant_one.

print(tf_constant_one)

We see that it’s a TensorFlow tensor, we see that the name scope is scalar_set_one, which is what we defined up here, and we see the variable name which is ten.

We can see that the shape is empty since it’s a scalar, and we can see the data type is int32.

Remember that TensorFlow first builds the graph and then later evaluates the graph.

And since we’re still in the building the graph stage, both of the TensorFlow constant scalars haven’t been evaluated in a TensorFlow session yet, so they are still uninitialized.

Next, let’s define the second scope.

with tf.name_scope('scalar_set_two') as scope:

So we say with tf.name_scope.

We’re going to assign the name 'scalar_set_two' and we say as scope.

Now that we have the scope, we define our third constant scalar.

with tf.name_scope('scalar_set_two') as scope:
    tf_constant_three = tf.constant(30, name="thirty")

So tf.constant, we give it the value of 30, we name it the string thirty, and we assign it to the Python variable tf_constant_three, again leaving four spaces in front.

Next, we define our fourth constant scalar.

with tf.name_scope('scalar_set_two') as scope:
    tf_constant_three = tf.constant(30, name="thirty")
    tf_constant_four = tf.constant(40, name="fourty")

So we use tf.constant, we give it the value of 40, the name is the string forty, and we assign it to the Python variable tf_constant_four.

Then finally, we do our second scalar addition, so tf.add, tf_constant_three plus tf_constant_four, and we give the name of the result as scalar_thirty_plus_forty, and we assign it to the Python variable scalar_sum_two.

with tf.name_scope('scalar_set_two') as scope:
    tf_constant_three = tf.constant(30, name="thirty")
    tf_constant_four = tf.constant(40, name="fourty")
    scalar_sum_two = tf.add(tf_constant_three, tf_constant_four, name="scalar_thirty_plus_fourty")

Then we hit return on an empty line to signal to Python that we are finished with this second code block.

Finally, let’s create a final sum that adds the two scalar sums already done.

scalar_sum_sum = tf.add(scalar_sum_one, scalar_sum_two)

So we do tf.add and we have our scalar_sum_one Python variable from the first name scope, then we have our scalar_sum_two from our second name scope.

Then the result of this addition is going to be assigned to the Python variable scalar_sum_sum.

Now that we’ve created our TensorFlow graph, it’s time to run the computational graph.

So we launch the graph in a session.

sess = tf.Session()

Then we initialize all the global variables in the graph.

sess.run(tf.global_variables_initializer())

The next thing we want to do is send our TensorFlow graph to TensorBoard so that we can visualize the graph.

The way we’ll do this is to use TensorFlow’s summary FileWriter to create a protocol buffer that serializes the structured data of our graph so that TensorBoard can later create a visual representation.

tf_tensorboard_writer = tf.summary.FileWriter('./graphs', sess.graph)

So we do tf.summary.FileWriter.

We’re going to assign this to the directory graphs.

We want to pass in our session graph, and we are assigning the FileWriter to the tf_tensorboard_writer Python variable.

Now that we’re done with our computation, let’s close the FileWriter.

tf_tensorboard_writer.close()

Let’s also close the TensorFlow session to release the TensorFlow resources we used within the session.

sess.close()

Next, let’s close Python so we can start TensorBoard from the command line.

exit()

First, let’s check to see if there’s anything in the graph’s directory.

ls graphs

We see one event protobuf that we just created.

Now, we can use the TensorBoard command line utility to specify our logdir and have it start the TensorBoard web service for us.

tensorboard --logdir="./graphs"

So again, we’re in the graph’s directory and we’re going to start the TensorBoard service.

We can see that we’re using TensorBoard, the version is 1.8.0, and it’s going to host the service at this web address.

Because I am on a Mac, I’m going to use localhost.

So I tap in localhost:6006, and we can see TensorBoard loaded.

We can see our two name scopes.

We have scalar_set_one and scalar_set_two.

This is why TensorFlow name scopes are extremely useful when you build larger graphs.

Because by using name scoping definitions and operations, we are able to group graph notes together to make it easier to see what was going on.

Let’s first click in the scalar_set_one name scope.

Move it back so we can see it.

When we do that, inside we can see our graph of ten plus twenty and our operation of the addition.

And you can see that when I hover over where it goes, it goes to this addition.

So we can double click and close it.

Then we can click into the scalar_set_two name scope and we see our other two notes.

So thirty and forty go into the scalar addition operation and that then later goes, feeds into this add operation.

So we can see the arrow goes into the add operation.

And we double click on the name scope to close it.

Perfect - We were able to use TensorFlow name scopes to group graph notes together in the TensorBoard web service so that our graph visualization is legible.