According to the help documentation: "Upon completion of a call to TOKENIZE (set, string, tokens [, separator]), tokens is allocated with a lower bound of one and a size equal to the number of tokens in string, and a character length equal to the length of the longest token in string". However, the following program yields the tokens "my" and "st":
program tokenize_test
implicit none
integer :: n
character(:), dimension(:), allocatable :: tokens
call tokenize(string="my string", set=" ", tokens=tokens)
do n=1, size(tokens)
print '(A, I0, 3A)', "tokens(", n, ") = '", tokens(n), "'"
end do
end program
Is this a bug, or do I miss something here? If the string is changed to "my string theory", the result is correct. It seems tokenize is using the length of the first token, when the string contains two tokens. I have tested on ifx 2025.2.1 on linux and ifx 2025.1.1 on Windows.
