Deprecation of Tensorflow CudnnRNNV3 on conversion to IR Format

[Xanph]

OS: Ubuntu 24.04.2 LTS

OpenVINO Version: 2025.0.0

CUDA Version: V12.5.82

CuDNN Version: 9.8.0

Tensorflow 2 Version: 2.19.0

Keras: 3.9.0

I'm attempting to convert my Keras model into the IR format. To do so I ran the following code:

model_name = f"{version}_best_motionx_model"

model_path = f'models/best_keras/{model_name}.keras'

model = keras.models.load_model(model_path)

# Save the model in the SavedModel format

saved_model_dir = f'models/tensorflow/{model_name}_tf'

model.export(saved_model_dir, format='tf_saved_model')

# Convert the SavedModel to OpenVINO IR format with multiple outputs

ir_model = ov.convert_model(

saved_model_dir, 

)

# Save the converted IR model

output_dir = "models/intermediate_representation"

ov.save_model(ir_model, f"{output_dir}/ir_motionX{version}.xml")

On the ov.convert_model call, I got this error saying that a translator cannot be found for CudnnRNNV3. On looking into this in the Tensorflow docs, it appears to be deprecated.

OpConversionFailure:

---------------------------------------------------------------------------
OpConversionFailure Traceback (most recent call last)
Cell In[3], line 15
12 model.export(saved_model_dir, format='tf_saved_model')
14 # Convert the SavedModel to OpenVINO IR format with multiple outputs
---> 15 ir_model = ov.convert_model(
16 saved_model_dir, 
17 )
19 # Save the converted IR model
20 output_dir = "models/intermediate_representation"

File /media/xanph/Data A/Development/motionx/motionx/lib/python3.12/site-packages/openvino/tools/ovc/convert.py:105, in convert_model(input_model, input, output, example_input, extension, verbose, share_weights)
103 logger_state = get_logger_state()
104 cli_parser = get_all_cli_parser()
--> 105 ov_model, _ = _convert(cli_parser, params, True)
106 restore_logger_state(logger_state)
107 return ov_model

File /media/xanph/Data A/Development/motionx/motionx/lib/python3.12/site-packages/openvino/tools/ovc/convert_impl.py:565, in _convert(cli_parser, args, python_api_used)
563 send_conversion_result('fail')
564 if python_api_used:
--> 565 raise e
566 else:
567 return None, argv

File /media/xanph/Data A/Development/motionx/motionx/lib/python3.12/site-packages/openvino/tools/ovc/convert_impl.py:505, in _convert(cli_parser, args, python_api_used)
499 if argv.framework is None and get_pytorch_decoder_for_model_on_disk(argv, args):
500 # try to load a model from disk as TorchScript or ExportedProgram
501 # TorchScriptPythonDecoder or TorchFXPythonDecoder object will be assigned to argv.input_model
502 # saved TorchScript and ExportedModel model can be passed to both ovc tool and Python convert_model
503 pytorch_model_on_disk = True
--> 505 ov_model = driver(argv, {"conversion_parameters": non_default_params})
507 if pytorch_model_on_disk:
508 # release memory allocated for temporal object
509 del argv.input_model

File /media/xanph/Data A/Development/motionx/motionx/lib/python3.12/site-packages/openvino/tools/ovc/convert_impl.py:249, in driver(argv, non_default_params)
245 def driver(argv: argparse.Namespace, non_default_params: dict):
246 # Log dictionary with non-default cli parameters where complex classes are excluded.
247 log.debug(str(non_default_params))
--> 249 ov_model = moc_emit_ir(prepare_ir(argv), argv)
251 return ov_model

File /media/xanph/Data A/Development/motionx/motionx/lib/python3.12/site-packages/openvino/tools/ovc/convert_impl.py:195, in prepare_ir(argv)
193 for extension in filtered_extensions(argv.extension):
194 moc_front_end.add_extension(extension)
--> 195 ov_model = moc_pipeline(argv, moc_front_end)
196 return ov_model
198 if not argv.input_model:

File /media/xanph/Data A/Development/motionx/motionx/lib/python3.12/site-packages/openvino/tools/ovc/moc_frontend/pipeline.py:293, in moc_pipeline(argv, moc_front_end)
289 input_model.set_partial_shape(place, ov_shape)
291 input_model.set_tensor_value(place, value)
--> 293 ov_model = moc_front_end.convert(input_model)
295 return ov_model

File /media/xanph/Data A/Development/motionx/motionx/lib/python3.12/site-packages/openvino/frontend/frontend.py:18, in FrontEnd.convert(self, model)
17 def convert(self, model: Union[Model, InputModel]) -> Model:
---> 18 converted_model = super().convert(model)
19 if isinstance(model, InputModel):
20 return Model(converted_model)

OpConversionFailure: Check 'is_conversion_successful' failed at src/frontends/tensorflow/src/frontend.cpp:478:
FrontEnd API failed with OpConversionFailure:
[TensorFlow Frontend] Internal error, no translator found for operation(s): CudnnRNNV3
To facilitate the conversion of unsupported operations, refer to Frontend Extension documentation: https://docs.openvino.ai/latest/openvino_docs_Extensibility_UG_Frontend_Extensions.html

For context, I was able to run this conversion back in November. Since then I think the only thing I have changed is moving to Keras 3 and Tensorflow 2.19.0

What do you recommend I do?

Many thanks

[Peh_Intel · Accepted solution]

Hi Xanph,

Thanks for reaching out to us.

I would suggest you to downgrade TensorFlow version to 2.16.1 and convert the Keras model into the IR format again.

Regards,

Peh

[Xanph]

Hi Peh,

Thank you for that - it worked.

Does OpenVINO support TF 2.17? I ask this because I need to get CUDA 12.3 to work on Ubuntu 24.04, which currently seems to be a bit tricky to support.

Many thanks

[Xanph]

Hi @Peh_Intel,

Are there any plans to support CudnnRNNV3 as part of the conversion process? Or have there been better ways to get around this, apart from remaining on Tensorflow version 2.16.1?

I'm happy to share model architecture directly with you.

Thank you.

Kind regards,

Xanph

[Peh_Intel]

Hi Xanph,

CudnnRNNV3 operation is deprecated and removed in future TensorFlow versions after 2.16.1. While converting model into OpenVINO IR, the conversion rely on translating framework ops into their own IR. Hence, if using later TensorFlow version, which CudnnRNNV3 operation is removed, the translator fail to map it and cause the conversion to fail.

Regards,

Peh

[Xanph]

Hi @Peh_Intel ,

I sent a reply yesterday, but either I got signed out when I posted it here, or it got deleted.

It's been few months since I last worked on this project that is using openvino, and when I tried your fix to downgrade to TF 2.16.1, it worked. But since then I ended up with TF 2.17.0 which then seemed to cause the problems again.

After downgrading back to 2.16.1, and still experiencing the error. I reverted to a clean slate on all packages, including CUDA and cuDNN. I wrote this test script to help narrow down to the root source of my problem:

#!/usr/bin/env python3
"""
Test script to verify LSTM conversion compatibility
Tests CUDA setup, TensorFlow GPU detection, and OpenVINO LSTM conversion
"""

import os
import numpy as np
import tensorflow as tf

def test_cuda_setup():
    """Test CUDA and GPU availability"""
    print("=" * 50)
    print("CUDA AND GPU SETUP TEST")
    print("=" * 50)
    
    # Check CUDA paths
    cuda_home = os.environ.get('CUDA_HOME', '/usr/local/cuda')
    print(f"CUDA_HOME: {cuda_home}")
    print(f"CUDA path exists: {os.path.exists(cuda_home)}")
    
    # Check CUDA version
    if os.path.exists('/usr/local/cuda/version.json'):
        import json
        with open('/usr/local/cuda/version.json', 'r') as f:
            cuda_version = json.load(f)
        print(f"CUDA version: {cuda_version.get('cuda', {}).get('version', 'Unknown')}")
    
    # Check TensorFlow GPU detection
    gpus = tf.config.list_physical_devices('GPU')
    print(f"GPUs detected by TensorFlow: {len(gpus)}")
    
    for i, gpu in enumerate(gpus):
        print(f"  GPU {i}: {gpu}")
        # Try to set memory growth
        try:
            tf.config.experimental.set_memory_growth(gpu, True)
            print(f"    Memory growth enabled")
        except Exception as e:
            print(f"    Memory growth failed: {e}")
    
    return len(gpus) > 0

def test_tensorflow_versions():
    """Test TensorFlow and related package versions"""
    print("\n" + "=" * 50)
    print("PACKAGE VERSIONS")
    print("=" * 50)
    
    print(f"TensorFlow version: {tf.__version__}")
    try:
        print(f"Keras version: {tf.keras.__version__}")
    except AttributeError:
        try:
            import keras
            print(f"Keras version: {keras.__version__}")
        except:
            print("Keras version: Unable to detect")
    
    try:
        import openvino as ov
        print(f"OpenVINO version: {ov.__version__}")
    except ImportError:
        print("OpenVINO not installed")
    
    try:
        import nncf
        print(f"NNCF version: {nncf.__version__}")
    except ImportError:
        print("NNCF not installed")
    
    print(f"NumPy version: {np.__version__}")

def test_lstm_model_creation():
    """Test LSTM model creation and training"""
    print("\n" + "=" * 50)
    print("LSTM MODEL CREATION TEST")
    print("=" * 50)
    
    try:
        # Create LSTM model identical to production model
        model = tf.keras.Sequential([
            tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True), input_shape=(50, 64)),
            tf.keras.layers.Dense(1, activation='sigmoid')
        ], name="test_lstm_model")
        
        print("✅ LSTM model created successfully")
        print(f"Model input shape: {model.input_shape}")
        print(f"Model output shape: {model.output_shape}")
        
        # Compile model
        model.compile(
            optimizer='adam',
            loss='binary_crossentropy',
            metrics=['accuracy']
        )
        print("✅ Model compiled successfully")
        
        # Create dummy data to match Bidirectional LSTM output shape
        X = np.random.random((100, 50, 64)).astype(np.float32)
        y = np.random.randint(0, 2, (100, 50, 1)).astype(np.float32)  # Match (batch, sequence, features)
        
        print(f"Training data shape: {X.shape}")
        print(f"Training labels shape: {y.shape}")
        
        # Train model briefly
        print("Training model for 1 epoch...")
        history = model.fit(X, y, epochs=1, batch_size=32, verbose=1)
        
        print("✅ Model training successful")
        print(f"Final loss: {history.history['loss'][-1]:.4f}")
        print(f"Final accuracy: {history.history['accuracy'][-1]:.4f}")
        
        return model
        
    except Exception as e:
        print(f"❌ LSTM model creation/training failed: {e}")
        return None

def test_model_saving(model):
    """Test model saving in different formats"""
    print("\n" + "=" * 50)
    print("MODEL SAVING TEST")
    print("=" * 50)
    
    if model is None:
        print("❌ No model to save")
        return None, None
    
    keras_path = None
    savedmodel_path = None
    
    # Test Keras format saving
    try:
        keras_path = 'test_lstm_model.keras'
        model.save(keras_path)
        print(f"✅ Model saved in Keras format: {keras_path}")
    except Exception as e:
        print(f"❌ Keras format saving failed: {e}")
    
    # Test SavedModel format saving
    try:
        savedmodel_path = 'test_lstm_savedmodel'
        model.export(savedmodel_path, format='tf_saved_model')
        print(f"✅ Model exported in SavedModel format: {savedmodel_path}")
    except Exception as e:
        print(f"❌ SavedModel format saving failed: {e}")
    
    return keras_path, savedmodel_path

def test_openvino_conversion(keras_path, savedmodel_path):
    """Test OpenVINO conversion"""
    print("\n" + "=" * 50)
    print("OPENVINO CONVERSION TEST")
    print("=" * 50)
    
    try:
        import openvino as ov
        print(f"Using OpenVINO version: {ov.__version__}")
    except ImportError:
        print("❌ OpenVINO not available")
        return False
    
    conversion_successful = False
    
    # Test conversion from SavedModel (preferred)
    if savedmodel_path and os.path.exists(savedmodel_path):
        print(f"\nTesting conversion from SavedModel: {savedmodel_path}")
        try:
            ov_model = ov.convert_model(savedmodel_path)
            output_path = 'test_lstm_openvino.xml'
            ov.save_model(ov_model, output_path)
            print(f"🎉 SUCCESS! SavedModel → OpenVINO conversion successful!")
            print(f"✅ Converted model saved: {output_path}")
            conversion_successful = True
        except Exception as e:
            print(f"❌ SavedModel conversion failed: {e}")
            if 'CudnnRNNV3' in str(e):
                print("   → CudnnRNNV3 error detected")
            elif 'translator' in str(e):
                print("   → Missing translator for LSTM operations")
    
    # Test conversion from Keras format (fallback)
    if not conversion_successful and keras_path and os.path.exists(keras_path):
        print(f"\nTesting conversion from Keras format: {keras_path}")
        try:
            ov_model = ov.convert_model(keras_path)
            output_path = 'test_lstm_openvino_keras.xml'
            ov.save_model(ov_model, output_path)
            print(f"🎉 SUCCESS! Keras → OpenVINO conversion successful!")
            print(f"✅ Converted model saved: {output_path}")
            conversion_successful = True
        except Exception as e:
            print(f"❌ Keras conversion failed: {e}")
            if 'CudnnRNNV3' in str(e):
                print("   → CudnnRNNV3 error detected")
            elif 'Cannot recognize input model' in str(e):
                print("   → OpenVINO cannot read Keras format")
    
    return conversion_successful

def cleanup_test_files():
    """Clean up test files"""
    print("\n" + "=" * 50)
    print("CLEANUP")
    print("=" * 50)
    
    test_files = [
        'test_lstm_model.keras',
        'test_lstm_savedmodel',
        'test_lstm_openvino.xml',
        'test_lstm_openvino.bin',
        'test_lstm_openvino_keras.xml',
        'test_lstm_openvino_keras.bin'
    ]
    
    for file_path in test_files:
        try:
            if os.path.exists(file_path):
                if os.path.isdir(file_path):
                    import shutil
                    shutil.rmtree(file_path)
                else:
                    os.remove(file_path)
                print(f"✅ Removed: {file_path}")
        except Exception as e:
            print(f"❌ Failed to remove {file_path}: {e}")

def main():
    """Run all tests"""
    print("LSTM CONVERSION COMPATIBILITY TEST")
    print("Testing CUDA, TensorFlow, and OpenVINO LSTM conversion")
    print("=" * 70)
    
    # Test 1: CUDA and GPU setup
    gpu_available = test_cuda_setup()
    
    # Test 2: Package versions
    test_tensorflow_versions()
    
    # Test 3: LSTM model creation and training
    model = test_lstm_model_creation()
    
    # Test 4: Model saving
    keras_path, savedmodel_path = test_model_saving(model)
    
    # Test 5: OpenVINO conversion
    conversion_success = test_openvino_conversion(keras_path, savedmodel_path)
    
    # Summary
    print("\n" + "=" * 70)
    print("TEST SUMMARY")
    print("=" * 70)
    print(f"GPU Available: {'✅' if gpu_available else '❌'}")
    print(f"LSTM Model Creation: {'✅' if model is not None else '❌'}")
    print(f"Model Saving: {'✅' if keras_path or savedmodel_path else '❌'}")
    print(f"OpenVINO Conversion: {'✅' if conversion_success else '❌'}")
    
    if conversion_success:
        print("\n🎉 ALL TESTS PASSED! LSTM conversion is working correctly.")
        print("You can now proceed with retraining your production model.")
    else:
        print("\n❌ LSTM conversion failed. Package compatibility issue detected.")
        print("Consider trying different package versions or alternative approaches.")
    
    # Cleanup
    cleanup_test_files()
    
    return conversion_success

if __name__ == "__main__":
    main()

Here's a revised summary of the problem:

Unable to convert TensorFlow models containing Bidirectional(LSTM(..., return_sequences=True)) to OpenVINO IR format. Previously worked 4 months ago, but now fails across multiple
package combinations.

Test Environment

- Hardware: NVIDIA RTX 3070 + RTX 3060 Ti, CUDA 12.3.0, cuDNN 8.9.0
- OS: Ubuntu 24.04
- Model Architecture: Bidirectional(LSTM(32, return_sequences=True), input_shape=(50, 64))

Testing results

Tensorflow	Keras	Error Type	Error Message
2.16.1	3.11.3	CudnnRNNV3	no translator found for operation(s): CudnnRNNV3
2.16.1	3.11.3	CudnnRNNV3	no translator found for operation(s): CudnnRNNV3
2.16.1	3.11.3	CudnnRNNV3	no translator found for operation(s): CudnnRNNV3
2.16.0rc0	3.11.3	CudnnRNNV3	no translator found for operation(s): CudnnRNNV3
2.17.0	3.11.3	CudnnRNNV3	no translator found for operation(s): CudnnRNNV3
2.16.1	3.0.2	CudnnRNN	no translator found for operation(s): CudnnRNN
2.16.1	3.0.2	Concat Error	Concat Parameter axis 2 out of tensor rank range [0, 0]
2.16.1	3.0.2	ReverseSequence	Cannot get length of dynamic dimension in ReverseSequence

I'm not too sure where to go from here or how I can force my LSTM layers to use OpenVINO supported operations.

Best regards,

Xanph

[Xanph]

It seems that adding `recurrent_dropout=0.1` to the LSTM layer has forced TF to rely on the standard TF/Keras LSTM operation, avoiding use of cuDNN. When the CuDNN criteria aren't met, it falls back to the "generic GPU kernel" or standard implementation.

So my test model now looks like this:

        model = tf.keras.Sequential([
            tf.keras.layers.Bidirectional(
                tf.keras.layers.LSTM(32, return_sequences=True, recurrent_dropout=0.1), 
                input_shape=(50, 64)
            ),
            tf.keras.layers.Dense(1, activation='sigmoid')
        ], name="test_lstm_model")

Tensorflow cuDNN requirements: https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTM

I've chosen to use recurrent dropout as the only downside would be slower training, and may improve accuracy by a small margin

Going to now run a test train on this to see what the impact would look like for the production model (this will take several days). @Peh_Intel if you have any more insight, that would be really useful.

Many thanks and in the meantime, have a nice weekend

[Peh_Intel]

Hi Xanph,

Yes, OpenVINO have fully validated the support for TensorFlow version from 1.15.5 to 2.17. This information can be obtained from the System Requirements under DL framework versions of Operating systems and developer environment.

Regards,

Peh

[Peh_Intel]

Hi Xanph,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question. 

Regards,

Peh