Re: Speed of code

JNichols · ‎09-12-2021

The code is a beast to read if I put the c1 and c2 values directly into the read(iline1, but would it speed up the code. I just think this makes more sense for someone looking at the code.

The last time I did this, I used separate format numbers for the different length numbers, but that appears to be a waste of code. Am I likely to encounter problems?

 do k = 1, 1000000
        READ (sRT, '(A)', IOSTAT=IOstatus)iline1    ! Read line
        IF (IOstatus < 0)  EXIT           ! if end-of-file reached, exit
        Num = Num + 1               ! no, have one more value
        count = 1
        do 120 j = 1, 127

            comma = iline1(j:j)
            !write(*,'(A)')comma
            if(comma .eq. ',') then
                bits(count) = j
                if(count .eq. 1) then
                    ! write(*,*)k,count,j
                    read(iline1(1:bits(count)),*)number(k)
                end if
                if(count .eq. 4)then
                    !write(*,*)iline1((bits(count-1)+1):(bits(count)-1))
                    c1 = bits(count-1)+1
                    c2 = bits(count)-1
                    read(iline1(c1:c2),*)X(k)
                end if
                if(count .eq. 5)then
                    !write(*,*)iline1((bits(count-1)+1):(bits(count)-1))
                    c1 = bits(count-1)+1
                    c2 = bits(count)-1
                    read(iline1(c1:c2),*)Y(k)
                end if
                if(count .eq. 6)then
                    !write(*,*)iline1((bits(count-1)+1):(bits(count)-1))
                    c1 = bits(count-1)+1
                    c2 = bits(count)-1
                    read(iline1(c1:c2),*)Z(k)
                end if
                count = count+1
            end if

120 end do



    end do

Steve_Lionel · ‎09-12-2021

I'll throw in my usual caution against relying on list-directed input for such things, but if you are sure you aren't going to have anything other than numbers in the fields, it's OK. You're spending a lot of time going in and out of the formatted I/O system, which will swamp anything else you do here. I gather you are reading a comma-list of numbers - why not rely on list-directed treating comma as a delimiter and just read the three values in a row? Or are they not always present?

JNichols · ‎09-12-2021

2000,09/10/2021,20:18:34.112803,-0.018999940374028,0.00819236910148201,1.01052165544384,,,

This is a sample, the last two can have or not have data.

The other problem I run into is the Europeans can use , for . and then a csv file is an unholy mess. I have a routine to fix this.

The program does 8 minutes of data in 11 seconds, so it is not going to stop the NUC. Each file is 1 million lines.

Will the / and : not play havoc with the input?

Steve_Lionel · ‎09-12-2021

The / and ; will definitely create problems for list-directed input. / will be treated as end-of-record, and ; will trigger an error. If you can bypass those first few fields you could use list-directed on the rest of the line. As for comma as radix point, Fortran can handle either but you have to open the file with the appropriate DECIMAL= value.

GVautier · ‎09-13-2021

Why not use select case and exit when count is 6?

        count = 1
        start = 1
        do 120 j = 1, 127

            !write(*,'(A)')iline1(j:j)
            if(iline1(j:j).eq. ',') then
                select case(count)
                case(1)
                    ! write(*,*)k,count,j
                    read(iline1(start:j-1)),*)number(k)
                case(4)
                    !write(*,*)iline1(start:j-1)
                    read(iline1(start:j-1),*)X(k)
                case(5)
                    !write(*,*)iline1(start:j-1)
                    read(iline1(start:j-1),*)Y(k)
                case(6)
                    !write(*,*)iline1(start:j-1)
                    read(iline1(start:j-1),*)Z(k)
                    exit
                end select
                count = count+1
                start=j+1
            end if

120     end do

If "bits" array is not used elsewhere it is not necessary just keep the index j of the start of a value.

JNichols · ‎09-13-2021

Thanks, brilliant, I keep forgetting about case in Fortran. I use it a lot in C#.

I looked in the Microsoft Fortran Version 5, book manual, still very useful, and it has Select Case as a MS Fortran extension.

When did it become a language standard?

JNichols · ‎09-13-2021

The other interesting part of the solution is to look for the peaks, I stumbled across this idea, somewhere, and it works nicely in Fortran.

 do 130 k = 1,num

        xa = x(k) - mean
        if(k .gt. 2) then
            x3 = x(k)
            x2 = x(k-1)
            x1 = x(k-2)
            if(x3.gt. x2 .and. x2.gt.x1) then
                zeros(k) = 0
            else if(x3.lt. x2 .and. x2.lt.x1) then
                zeros(k) = 0
            else if(x3.lt. x2 .and. x2.gt.x1) then
                zeros(k) = 1
                deltax(k-1) = x2 - xlast
                xlast = x2
            else if(x3.gt. x2 .and. x2.lt.x1) then
                zeros(k) = -1
                deltax(k-1) = x2 - xlast
                xlast = x2
            elseif(x3 .eq. x2 .or. x2 .eq. x1)then
                zeros(k) = 0
            endif

200         format(i6,' , ',F14.8,' , ',i6)
        endif
130 end do

I thought there would be four different sketches, but I did not realize I would occasionally get two consecutive values the same, maybe 6 or seven times per million readings, so I had to add the last one.

The interesting challenge is you cannot determine the sets of descending peaks stepping forward, you get trapped in a local minima, you have to do it in reverse.

Steve_Lionel · ‎09-13-2021

SELECT CASE was added in Fortran 90.

lEdge · ‎09-13-2021

Ha. To be honest I had a dream again. And I was hoping somebody was going to ask this question. A lot of programs lost that model. I would honestly suggest more break lines than you have. Because you need some pauses between precision processes. I know intel has this 'ready' technology to take care of 100% stability. But, I would suggest it anyway. There are a lot of problems with operating systems now days.

A perfect example I finally discovered in Intel NUC6I7KYK. It doesn't like to wake up after software sleep button for Windows 8.1 Embedded Pro. It likes that sleep function from USB keyboard button. It would hang up the USB in between S3 states.

JNichols · ‎09-13-2021

I am just checking, but are you suggesting more exit statements?

I have NUCS all over the world, some at remote locations. The hardest thing to do is keep the operating system operating 24/7/365. Sooner or later something interferes and they die. If they are behind a wifi you cannot send a wake on lan, because the wifi does not support it because the people who wrote the standard did not think it was necessary. How times have changed.

I have two hung in Iowa at the moment.

lEdge · ‎09-13-2021

Okay. I see what you mean. In your case it would seem better like return. And if I recall correctly wait may have a length of time. That's what I mean.

I'm not entirely sure about this c for excel, but it may also work in time off 1,000,000 to 2,000,000. No pressure how you do it.