Max10 sometimes randomly stops operation, depending on settings in Signal Tap Logic Analyzer

KS_AX · ‎06-30-2020

Hello everyone,

I ran into some unexpected behavior regarding the operation of my design on the MAX10 FPGA 10M08SAM153I7G. Now, first I have to say that I am quite new to Quartus/FPGA/Verilog and I'm trying to learn it by myself, so please have merci if I got something completely wrong.

In some configurations, a part of my design stops working randomly between 5 ms to 50 ms after powerup of the FPGA. My design consists of a PLL and multiple instances coded in Verilog. I was able to narrow the problem down to one specific instant. (please see the code below.)

//////////////////////////////////////////////////////////////////////////////////////////
// module sampleDataIn                                                                  //
//////////////////////////////////////////////////////////////////////////////////////////
// Detektiert Präambel (011110) und normiert counter (cnt) auf ihre fallende Flanke.    //
// Sampled 25 bits (channel und data) aus bitstream (bs) synchron auf die Bitmitte      //
// Decodiert channel und data mit 4/5-Bit Code und gibt diese als output aus.           //
//////////////////////////////////////////////////////////////////////////////////////////

module sampleDataIn (
    input   clk, bs,                            // Clock, bitstream,
    output  reg             OutputEn  = 1'b0,   // Output ist aktiviert und Stabil
    output  reg     [15:0]  data,               // aktuell ausgegebene Daten, gemessen im letzten Frame
    output  reg             sample    = 1'b0,   // Output Sample Flag, Synchron zur Bitmitte
    output  reg             negedgePra = 1'b0,  // Output Puls für negative Flanke der Präambel
    output  reg     [3:0]   channel);           // aktuell ausgegebener channel, gemessen im letzten Frame


parameter 	PRA  = 3'b000,  //Detektiere Präambel, warte bis bs==1 für mind 8µs
			WAIT = 3'b001,  //Warte bis bs==0, detektiere fallende Flanke Präambel (bs=bitstream)
            FLA  = 3'b010,  //Fallende Flanke Präambel detektiert, normiere jetzt cnt, setze cnt=0
			SAM	 = 3'b011,  //Sample 25 bit aus bs, cnt[4:0]==5'b10011 ist sampleClk (5'b10000 = 500KHz, 5'b00011=offset für mittiges, bitsynchrones sampling)
            DEC  = 3'b100,  //Decodiere 4/5-Bit Codierung mit function [3:0] decode45bit und aktiviere output von channel, OutputEn=0
            OUT  = 3'b101;  //Aktiviere Output von Data, bitbereich abhängig von channel, OutputEn=0

reg [9:0]   cnt         = 10'b0;
reg         temp        = 1'b0;
reg [4:0]   bitN        = 5'b0;
reg [4:0]   channel5b   = 5'b0;
reg [19:0]  data5b      = 20'b0;
reg [15:0]  data4b      = 16'b0;
reg [2:0] 	state       = PRA, next;

function [3:0] decode45bit; //Decodiere 4/5-Bit Codierung
// confidential
endfunction

always @(posedge clk) begin
    state <= next;
    case (state)
    PRA:    begin //Detektiere Präambel, warte bis bs==1 für mind 8µs
                if (bs==1'b0)   cnt <= 10'b0;
                else            cnt <= cnt + 1'b1;
            end
    FLA:    begin   cnt <= 10'd0;       end //Fallende Flanke Präambel detektiert, normiere jetzt cnt, setze cnt=0
    SAM:    begin
                cnt <= cnt + 1'b1;
                if (bitN!=5'b0) begin   // Überspringe 0-tes Bit, letztes Bit von Präambel
                    if (bitN <= 5'd5)   channel5b[5-bitN] <= temp; // Schreibe temporäres Bit in korrektes Register, abhängig von Bitzähler bitN
                    else                data5b[25-bitN]   <= temp;
                end
            end
    DEC:    begin   // Decodiere 4/5-Bit Codierung mit function [3:0] decode45bit und aktiviere output von channel, OutputEn=0
                cnt <= 10'b0;
                channel          <= decode45bit(channel5b);
                data4b[3:0]      <= decode45bit(data5b[4:0]);
                data4b[7:4]      <= decode45bit(data5b[9:5]);
                data4b[11:8]     <= decode45bit(data5b[14:10]);
                data4b[15:12]    <= decode45bit(data5b[19:15]);
            end
    OUT:    begin
                data <= data4b;
            end
    endcase
end

always @(*) begin
	next        = 3'bx;
    temp        = 1'bx;
    bitN        = 5'b0; // Zahler für BitNummer des channels
    sample      = 1'b0; // Sample Pulse als Output
    OutputEn    = 1'b1; // Immer OutputEn ausser in DEC und OUT
    negedgePra  = 1'b0; // Negedge Detektionspuls als Output

    case (state)
    PRA:    begin //Detektiere Präambel, warte bis bs==1 für mind 8µs
                if (cnt>10'd128)                    next = WAIT;
                else                                next = PRA;
            end
    WAIT:    begin //Warte bis bs==0, detektiere fallende Flanke Präambel (bs=bitstream)
                //if (rst_n==1'b0)                    next = PRA; // Reset detektiert
                if (bs==1'b1)                       next = WAIT;
                else                                next = FLA;
            end
    FLA:    begin  //Fallende Flanke Präambel detektiert, normiere jetzt cnt mit cnt=0
                                                    next = SAM;
            negedgePra = 1'b1;
            end
    SAM:    begin //Sample 25 bit aus bs, cnt[4:0]==5'b10011 ist sampleClk
                //if (rst_n==1'b0)                    next = PRA; // Reset detektiert
                if (cnt > 10'b1100110100)           next = DEC; // alle 25 bit gesampled, gehe zu Decoding
                else begin
                                                    next = SAM;
                    if (cnt>=10'b0000110000) begin
                        if (cnt[4:0]==5'b10011) begin
                            //cnt[4:0]==5'b10011 ist sampleClk (5'b10000 = 500KHz, 5'b00011=offset für mittiges, bitsynchrones sampling)
                            bitN = cnt[9:5]; // bitN = cnt[9:5] ist Zahler für BitNummer dieses channels, 25 Bit pro channel
                            temp = bs;  // schreibe aktuellen gesampleten Wert in temporäres Register
                            sample = 1'b1; // Sample Pulse Output
                        end
                    end
                end
            end
    DEC:    begin                                   next = OUT;
            OutputEn = 1'b0; //Setze OutputEn-flag auf 0
            end
    OUT:    begin                                   next = PRA;
            OutputEn = 1'b0; //Setze OutputEn-flag auf 0
            end
    endcase
end

endmodule

The instant is clocked with the PLL. It detects a preambel in the bitstream, synchronizes to its falling edge and samples the following data-bits bit-centralized. The code simulates and compiles fine. During operation, the FSM sometimes stops after the state OUT and does not proceed to the state PRA. The time when it stops after startup is completely random and not reproducible. In the time between startup and unexpected stopping the design works completely fine. The State Mashine Viewer shows the expected correct behavior of the FSM.

The strange thing is that, if I use the Signal Tap Logic Analyzer (STLA) to monitor some signals on the FPGA during operation, it sometimes works completely fine. However, if I change some settings in the STLA (sample depth, observated signals) the design again shows the faulty behavior. I could not find any specific pattern regarding the influence of the settings in the STLA on the faulty behavior.

During compilation, the following warnings caught my eye:

Warning (332060): Node: sampleDataIn:inst4|state.DEC was determined to be a clock but was found without an associated clock assignment.
	Info (13166): Register MemLin:inst3|InsAdr[4] is being clocked by sampleDataIn:inst4|state.DEC
Warning (332060): Node: WD:inst2|Wdog1:inst15|lpm_counter:LPM_COUNTER_component|cntr_kcj:auto_generated|counter_reg_bit[0] was determined to be a clock but was found without an associated clock assignment.
	Info (13166): Register WD:inst2|Wdog2:inst16|lpm_counter:LPM_COUNTER_component|cntr_7ih:auto_generated|counter_reg_bit[0] is being clocked by WD:inst2|Wdog1:inst15|lpm_counter:LPM_COUNTER_component|cntr_kcj:auto_generated|counter_reg_bit[0]
Warning (332060): Node: sampleDataIn:inst4|cnt[0] was determined to be a clock but was found without an associated clock assignment.
	Info (13166): Register WD:inst2|Wdog1:inst15|lpm_counter:LPM_COUNTER_component|cntr_kcj:auto_generated|counter_reg_bit[0] is being clocked by sampleDataIn:inst4|cnt[0]
Warning (332060): Node: MemLin:inst3|state.READ was determined to be a clock but was found without an associated clock assignment.
	Info (13166): Register SPI_LAN:inst10|dataTemp[6] is being clocked by MemLin:inst3|state.READ
Warning (332060): Node: WD:inst2|Wdog2:inst16|lpm_counter:LPM_COUNTER_component|cntr_7ih:auto_generated|counter_reg_bit[8] was determined to be a clock but was found without an associated clock assignment.
	Info (13166): Register WD:inst2|inst14 is being clocked by WD:inst2|Wdog2:inst16|lpm_counter:LPM_COUNTER_component|cntr_7ih:auto_generated|counter_reg_bit[8]

It seems, that Quartus detects some signals / state variables as clocks. Yesterday it detected 5 more clocks without an assignment, those were in fact clocks in the rest of my design and I constrained them in the SDC file. (Also attached). I also uploaded the full compilation report.

My theory is that I am running in some timing issues. Depending on the settings in the STLA the fitter routes my design differently which affects the timing. Now, if my design is at the aproximate threshhold of working/not working regarding the timing, those small timing changes make all the difference and it randomly stops (or not).

My questions are the following:

- Could it be a timing issue regarding the clocks?

- is Quartus correct when detecting state variables as clocks? If yes, how do I constrain them correctly? if no, how do I discard them as clocks?

- Are there any other warnings in the compilation report that could lead to this behavior? There are a few warnings but I don't think they are significant for this problem

Now, as I said in the beginning, I am quite new to Quartus/FPGA/Verilog and I'm trying to learn it by myself for a few months now so please be not so hard on me.

I am very glad about any tips you can give me, I searched extensively and could not find a similar problem.

Thank you all a lot,

best wishes,

KS_AX

sstrell · ‎06-30-2020

It is very likely that this is a timing issue. Looking quickly at your compilation report, your hold slack values are really small compared to your relatively gigantic setup slack values.

Looking at your .sdc file, is there a structure in your design that is performing divide by 2 on those 4 clocks after the PLL? If not, you don't need those 4 constraints because derive_pll_clocks already constrains the outputs of the PLL. Adding those extra constraints, assuming your PLL is already doing a divide by 2, means those clocks are now divide by 4 with respect to the input reference clock to the PLL. Or are these output clocks from the device? I'm guessing not because they're not targeting output ports of the device.

You're also missing set_input_delay and set_output_delay constraints for the I/O in your design, so your design is not fully constrained for timing. Check the unconstrained paths report to see what you need to constrain.

As for the warnings, counter_reg_bit, InsAdr, and MemLin are not shown in your code so it's not clear why those signals are being seen as clocks.

Finally, I don't know if it's part of the issue, but you have a lot of comparator logic in your SAM next state logic (3 levels at most). You might want to do those comparator checks in separate processes to simplify the next state logic.

Hope this helps a bit.

#iwork4intel

IDeyn · ‎07-06-2020

Hi KS_AX!

Well, if I understand correctly, your design takes a data from external port (bitstream).

Is it so?

If it is, when the issue is because you use comparison on bitstream value right inside your FSM.

It could be then a metastability issue. Try to synchronize bs port using two registers.

Metastability is the main reason I suppose why FSM can stop.

Hope it helps.

--

Best regards,

Ivan