Depth post-processing in с#

[PPark7]

Hi,

I'm working with a D435 camera with Visual Studio using C#.

I'm wondering if there is any example out there of how to turn on the post-processing filters via code? How to use the threshold_filter filter?

I tried using the cs-tutorial-3-processing example (librealsense-2.18.0). If you change the value of the decimate.Options [Option.FilterMagnitude] file .Value gives an error or some kind of interference. Code below.

Thanks in advance!

public partial class MainWindow : Window { private Pipeline pipeline = new Pipeline(); private Colorizer colorizer = new Colorizer(); private Align align = new Align(Stream.Color); private DecimationFilter decimate = new DecimationFilter(); private SpatialFilter spatial = new SpatialFilter(); private TemporalFilter temp = new TemporalFilter(); private CustomProcessingBlock block; private CancellationTokenSource tokenSource = new CancellationTokenSource(); private bool isPostProcessing;   static Action<VideoFrame> UpdateImage(Image img) { var wbmp = img.Source as WriteableBitmap; return new Action<VideoFrame>(frame => { using (frame) { var rect = new Int32Rect(0, 0, frame.Width, frame.Height); wbmp.WritePixels(rect, frame.Data, frame.Stride * frame.Height, frame.Stride); } }); }   public MainWindow() { InitializeComponent();   try { var cfg = new Config(); cfg.EnableStream(Stream.Depth, 640, 480); cfg.EnableStream(Stream.Color, Format.Rgb8); var pp = pipeline.Start(cfg);   // Allocate bitmaps for rendring. // Since the sample aligns the depth frames to the color frames, both of the images will have the color resolution using (var p = pp.GetStream(Stream.Color) as VideoStreamProfile) { imgColor.Source = new WriteableBitmap(p.Width, p.Height, 96d, 96d, PixelFormats.Rgb24, null); imgDepth.Source = new WriteableBitmap(p.Width, p.Height, 96d, 96d, PixelFormats.Rgb24, null); } var updateColor = UpdateImage(imgColor); var updateDepth = UpdateImage(imgDepth); // Create custom processing block // For demonstration purposes it will: // a. Get a frameset // b. Run post-processing on the depth frame // c. Combine the result back into a frameset // Processing blocks are inherently thread-safe and play well with // other API primitives such as frame-queues, // and can be used to encapsulate advanced operations. // All invokations are, however, synchronious so the high-level threading model // is up to the developer block = new CustomProcessingBlock((f, src) => { // We create a FrameReleaser object that would track // all newly allocated .NET frames, and ensure deterministic finalization // at the end of scope. using (var releaser = new FramesReleaser()) { var frames = FrameSet.FromFrame(f).DisposeWith(releaser);   FrameSet processedFrames;   if (isPostProcessing) { processedFrames = frames; processedFrames = processedFrames.ApplyFilter(decimate).DisposeWith(releaser); //.ApplyFilter(spatial).DisposeWith(releaser) //.ApplyFilter(temp).DisposeWith(releaser) processedFrames = processedFrames.ApplyFilter(align).DisposeWith(releaser); processedFrames = processedFrames.ApplyFilter(colorizer).DisposeWith(releaser); } else { processedFrames = frames.ApplyFilter(align).DisposeWith(releaser) .ApplyFilter(colorizer).DisposeWith(releaser); } var colorFrame = processedFrames.ColorFrame.DisposeWith(releaser); var colorizedDepth = processedFrames[Stream.Depth, Format.Rgb8].DisposeWith(releaser);   // Combine the frames into a single result var res = src.AllocateCompositeFrame(colorizedDepth, colorFrame).DisposeWith(releaser); // Send it to the next processing stage src.FramesReady(res); } });   // Register to results of processing via a callback: block.Start(f => { using (var frames = FrameSet.FromFrame(f)) { var colorFrame = frames.ColorFrame.DisposeWith(frames); var colorizedDepth = frames[Stream.Depth, Format.Rgb8].DisposeWith(frames);   Dispatcher.Invoke(DispatcherPriority.Render, updateDepth, colorizedDepth); Dispatcher.Invoke(DispatcherPriority.Render, updateColor, colorFrame); } });   var token = tokenSource.Token;   var t = Task.Factory.StartNew(() => { while (!token.IsCancellationRequested) { using (var frames = pipeline.WaitForFrames()) { // Invoke custom processing block block.ProcessFrames(frames); } } }, token); } catch (Exception ex) { MessageBox.Show(ex.Message); Application.Current.Shutdown(); } }   private void control_Closing(object sender, System.ComponentModel.CancelEventArgs e) { tokenSource.Cancel(); }   private void button_Click(object sender, RoutedEventArgs e) { int val = Convert.ToInt16(trDecimation.EditValue.ToString()); decimate.Options[Option.FilterMagnitude].Value = val; }   private void trDecimation_EditValueChanged(object sender, DevExpress.Xpf.Editors.EditValueChangedEventArgs e) { int val = Convert.ToInt16(trDecimation.EditValue.ToString()); decimate.Options[Option.FilterMagnitude].Value = val; }   private void chPostProcessing_Click(object sender, RoutedEventArgs e) { isPostProcessing = (bool)chPostProcessing.IsChecked; } }

[MartyG · Accepted solution]

There was a question in December 2018 over on the RealSense GitHub website about converting samples from C++ to C#. Dorodnic the RealSense SDK Manager agreed that converting examples such as rs-measure and re-align to C# would be a beneficial idea for future SDK releases.

In the meantime, a member of the RealSense community made an attempt to convert the Measure example (which also uses the decimation filter) to C#. they posted their adapted script, and Dorodnic offered advice on changing the decimation filter code for that example.

https://github.com/IntelRealSense/librealsense/issues/2809#issuecomment-444444247

At the very bottom of the page, the community member posts a link to download a zip file of their completed C# conversion of Measure.