IFX TOKENIZE execution error

[NFKirkby]

Hi

Just downloaded and installed IFX 2025.0.0 Build 20241008 and noticed SPLIT and TOKENIZE intrinsic procedures in "Whats new". So I copied and pasted Intel's example programs from the documentation into a minimal test example and they worked as advertised. Excellent.

Then I copied and pasted very similar code into a test/examples procedure in a much, much bigger code, it compiles fine but it does not run and  I get the attached ( I am using snipping tool so you can see line numbers etc)

where the code referred to is (again forgive the screen grab)

1) Now, I have spent too much time already trying to get a minimal reproducer, and I have failed, so some general advice about how to get a reproducer would be appreciated.

2) The error message "severe (408): fort: (8): Attempt to fetch from allocatable variable TOKENS when it is not allocated" really worries me because you can see I have explicitly double checked that TOKENS is not allocated before the only call to TOKENIZE that involves the allocatable array TOKENS. And it is the job of TOKENIZE to allocate TOKENS, not "fetch" from it.

3) What is it about a big program ( roughly 0.3 M lines of source code ) that could cause this to fail, when a little example works fine? The debug executable file is 155 MB, I have lots of memory available, I would be very surprised if the attempt to allocate TOKENS was running out of memory, heap or stack.

I have

1) Recompiled the complete code in 2025.0.0 and there were no new warnings or errors of any sort.

2) I have run a random assortment of test procedures on the original code and all else seems to be working normally.

3) I am running on Windows 11, and Visual Studio 17.10.5

Thoughts?

[Steve_Lionel]

The steps I typically go through to cut down a big example:

1. find the minimal number of compiler options needed to reproduce.
2. Drop optimization level as far as you can while still reproducing. You may find that only one source needs some option to show the problem
3. Remove as much code after the problem point as you can

Admittedly, with a brand new feature there may be bugs triggered by something else in your code. If nothing else, make the full test case available to Intel support.

[NFKirkby]

OK, thanks, working on it.