Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
1217 Discussions

How to disable this waring message in Tensorflow 2.8 training?

CLi37
Beginner
1,471 Views
2022-05-30 07:50:10.642587: W tensorflow/core/grappler/optimizers/data/auto_shard.cc:776] AUTO sharding policy will apply DATA sharding policy as it failed to apply FILE sharding policy because of the following reason: Did not find a shardable source, walked to a node which is not a dataset: name: "FlatMapDataset/_2"
op: "FlatMapDataset"
input: "TensorDataset/_1"
attr {
  key: "Targuments"
  value {
    list {
    }
  }
}
attr {
  key: "_cardinality"
  value {
    i: -2
  }
}
attr {
  key: "f"
  value {
    func {
      name: "__inference_Dataset_flat_map_flat_map_fn_1048"
    }
  }
}
attr {
  key: "metadata"
  value {
    s: "\n\020FlatMapDataset:1"
  }
}
attr {
  key: "output_shapes"
  value {
    list {
      shape {
        dim {
          size: -1
        }
        dim {
          size: -1
        }
        dim {
          size: -1
        }
        dim {
          size: -1
        }
      }
      shape {
        dim {
          size: -1
        }
        dim {
          size: -1
        }
      }
    }
  }
}
attr {
  key: "output_types"
  value {
    list {
      type: DT_FLOAT
      type: DT_INT64
    }
  }
}
experimental_type {
  type_id: TFT_PRODUCT
  args {
    type_id: TFT_DATASET
    args {
      type_id: TFT_PRODUCT
      args {
        type_id: TFT_TENSOR
        args {
          type_id: TFT_FLOAT
        }
      }
      args {
        type_id: TFT_TENSOR
        args {
          type_id: TFT_INT64
        }
      }
    }
  }
  args {
    type_id: TFT_DATASET
    args {
      type_id: TFT_PRODUCT
      args {
        type_id: TFT_TENSOR
        args {
          type_id: TFT_FLOAT
        }
      }
      args {
        type_id: TFT_TENSOR
        args {
          type_id: TFT_INT64
        }
      }
    }
  }
}
. Consider either turning off auto-sharding or switching the auto_shard_policy to DATA to shard this dataset. You can do this by creating a new `tf.data.Options()` object then setting `options.experimental_distribute.auto_shard_policy = AutoShardPolicy.DATA` before applying the options object to the dataset via `dataset.with_options(options)`.
2022-05-30 07:50:10.707679: W tensorflow/core/framework/dataset.cc:768] Input of GeneratorDatasetOp::Dataset will not be optimized because the dataset does not implement the AsGraphDefInternal() method needed to apply optimizations.
0 Kudos
1 Solution
Hari_B_Intel
Moderator
1,376 Views

Hi CLi37, 

 

Thank you for your patience and understanding, after further identifying the issue you face, it seems that you are trying to train some sample models on Intel® DevCloud for edge?

For your information, currently, the Intel® DevCloud for edge platform is not designed to train the model and its purpose is to simulate the OpenVINO inference engine on different hardware and generate the results for comparison. 

 

Regarding the issue you are facing, you can refer to this Stack Overflow discussion here

GeneratorDatasetOp:Dataset will not be optimized because the dataset does not implement the AsGraphD... 

 

Hope this information helps

 

Thank you

 

View solution in original post

3 Replies
VaradJ_Intel
Moderator
1,451 Views

Hi,

 

Thanks for posting in Intel Communities.

 

Can you share the Intel DevCloud that you are using? Is it Intel DevCloud for oneAPI or FPGA or Edge?

 

Thank You.


Hari_B_Intel
Moderator
1,424 Views

Hi CLi37

we might need some time to further investigate this issue. please allow us some time to investigate it.


Thank you


Hari_B_Intel
Moderator
1,377 Views

Hi CLi37, 

 

Thank you for your patience and understanding, after further identifying the issue you face, it seems that you are trying to train some sample models on Intel® DevCloud for edge?

For your information, currently, the Intel® DevCloud for edge platform is not designed to train the model and its purpose is to simulate the OpenVINO inference engine on different hardware and generate the results for comparison. 

 

Regarding the issue you are facing, you can refer to this Stack Overflow discussion here

GeneratorDatasetOp:Dataset will not be optimized because the dataset does not implement the AsGraphD... 

 

Hope this information helps

 

Thank you

 

Reply