When using the curie (intel_pmt) saving functionality - fails to iterate through all committed nodes

[mo_bri]

When using this code from the examples

CuriePME.beginSaveMode();

Intel_PMT::neuronData neuronData;

int count = 1;

while (uint16_t nCount = CuriePME.iterateNeuronsToSave(neuronData))

{

// iterate over the network and save the data.

if (nCount == CuriePME.noMatch)

{

Serial.print("Hit 0x7FFF");

break;

}

else

{

//save the cat, min inf, inf, context

//then save 128 uint8 byte vector

}

etc.

etc.

The nCount that comes back when the committed nodes are finished (ie I have 10 committed nodes, on the 11th iteration on the above - the first non-committed neuron), the noMatch constant of 0x7FFF never gets hit, the number that comes back for category/context etc is 65535 (0xFFFF).

Finished Saving Cat Neuron:65535

Saving node# :406 NN cat # 65535Saved this info from iterate func in lib for cat :

65535

context:65535 influence:65535 minInfluence:2 category:65535

Vector:255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255

--------------

So is the noMatch wrong, should it be 0xFFFF NOT 0x7FFF ?

Marcus

[idata]

Hi Marcuso,

Thank you for your interest in the Intel® Curie Module.I've doing some research to find the code you're using, and while I found the GitHub for PMT (this one I believe https://github.com/01org/Intel-Pattern-Matching-Technology https://github.com/01org/Intel-Pattern-Matching-Technology) I didn't find the code you're using. Could you please provide the link to that example? If you could provide the specific steps you're using to run these examples I would also appreciate, in case I need to reproduce your scenario.Also, I believe you've been handed some links about Pattern Learning and Recognition with the Curie module in this other thread /thread/109712 https://communities.intel.com/thread/109712, so I would like to emphasize on reading/checking those documents as they contain some really good information.Regards,-Pablo

[mo_bri]

OK I have fixed this.

Be careful when using the save and restore functions as they require you know about their implementation (terrible API design), its like new'ing data in a function then returning it, and expecting the caller to delete it. Anyway enough analogies.

These save and restore functions use an internal index, when you call the save or restore function it moves the index along.

In the CuriePME.cpp we have our global instance to the NN created :

# include "CuriePME.h"

Intel_PMT CuriePME;

So when we save or restore on this instance of the curiepme class we normally first initialise everything, by doing ONE of these these :

CuriePME.beginSaveMode()

CuriePME.beginRestoreMode()

Then when we are finished we call ONE of these :

CuriePM.endSaveMode()

CuriePM.endRestoreMode()

When in save or restore mode (i.e. after calling begin and before end), the problem is when calling any other function that moves the internal next neuron pointer,it will interfere with the save or restore.

Basically, these functions share an internal pionter with each other AND the readNeuron function !! Look how bad this is :

uint16_t Intel_PMT::readNeuron( int32_t neuronID, neuronData& data_array)

{

uint16_t dummy = 0;

// range check the ID - technically, this should be an error.

if( neuronID < firstNeuronID )

neuronID = firstNeuronID;

if(neuronID > lastNeuronID )

neuronID = lastNeuronID;

// use the beginSaveMode method

beginSaveMode();

//iterate over n elements in order to reach the one we want.

for( int i = 0; i < (neuronID -1); i++)

{

dummy = regRead16( CAT );

}

// retrieve the data using the iterateToSave method

iterateNeuronsToSave( data_array);

//restore the network to how we found it.

endSaveMode();

return 0;

}

SO - I was performing a save of the whole network to disk, based on how many committed neurons there were. As I was saving each neuron in my save function, I was printing each node that came back from the iterateNeuronsToSave call. And to verify this was correct I was also getting the nodes data with a readNeruon call, then comparing the two. Obviously they were the same, and I was happy. But on calling the readNeuron in the middile of a beginsave and endsave, it was messing up the atomic save function.

If you have an API that relies on implementation details being known by the user then a comment isn't really enough in the code, you need an excepion (not in arduinio) or error to be passed back.

Why not have a static bool in the code, then turns on (true) when you call begin on save or restore, if you then call anything but the iterate or matching end ie if you call a read, or another begin then you throw an error.

You have code that is atomic ie a single un-interruptable series of instructions, that can be interferred with. Of course the best thing is to have a single function that returns ALL the neurons to save ie all committed neurons. Then it really is atomic, as you can get rid of the begin and end on the APi aswell.

Marcus.

[mo_bri]

OK I have fixed this.

Be careful when using the save and restore functions as they require you know about their implementation (terrible API design), its like new'ing data in a function then returning it, and expecting the caller to delete it. Anyway enough analogies.

These save and restore functions use an internal index, when you call the save or restore function it moves the index along.

In the CuriePME.cpp we have our global instance to the NN created :

# include "CuriePME.h"

Intel_PMT CuriePME;

So when we save or restore on this instance of the curiepme class we normally first initialise everything, by doing ONE of these these :

CuriePME.beginSaveMode()

CuriePME.beginRestoreMode()

Then when we are finished we call ONE of these :

CuriePM.endSaveMode()

CuriePM.endRestoreMode()

When in save or restore mode (i.e. after calling begin and before end), the problem is when calling any other function that moves the internal next neuron pointer,it will interfere with the save or restore.

Basically, these functions share an internal pionter with each other AND the readNeuron function !! Look how bad this is :

uint16_t Intel_PMT::readNeuron( int32_t neuronID, neuronData& data_array)

{

uint16_t dummy = 0;

// range check the ID - technically, this should be an error.

if( neuronID < firstNeuronID )

neuronID = firstNeuronID;

if(neuronID > lastNeuronID )

neuronID = lastNeuronID;

// use the beginSaveMode method

beginSaveMode();

//iterate over n elements in order to reach the one we want.

for( int i = 0; i < (neuronID -1); i++)

{

dummy = regRead16( CAT );

}

// retrieve the data using the iterateToSave method

iterateNeuronsToSave( data_array);

//restore the network to how we found it.

endSaveMode();

return 0;

}

SO - I was performing a save of the whole network to disk, based on how many committed neurons there were. As I was saving each neuron in my save function, I was printing each node that came back from the iterateNeuronsToSave call. And to verify this was correct I was also getting the nodes data with a readNeruon call, then comparing the two. Obviously they were the same, and I was happy. But on calling the readNeuron in the middile of a beginsave and endsave, it was messing up the atomic save function.

If you have an API that relies on implementation details being known by the user then a comment isn't really enough in the code, you need an excepion (not in arduinio) or error to be passed back.

Why not have a static bool in the code, then turns on (true) when you call begin on save or restore, if you then call anything but the iterate or matching end ie if you call a read, or another begin then you throw an error.

You have code that is atomic ie a single un-interruptable series of instructions, that can be interferred with. Of course the best thing is to have a single function that returns ALL the neurons to save ie all committed neurons. Then it really is atomic, as you can get rid of the begin and end on the APi aswell.

Marcus.

[idata]

Hi Marcus,

Thank you very much for the detailed explanation. This will definitely help some other users in the future.Regards,-Pablo

[Lawrence_B_Intel]

The reasoning behind the API was to deal with the very limited memory available on the Arduino board. There's simply not enough memory available to a sketch to save/restore a full network's worth of knowledge. The idea was to put the network into the save/restore mode with the opening call, then iterate over the nodes one at a time and save/restore each individually by whatever means the user desires (serial, print out, flash, etc) then make the closing call to restore the network to the learn/classify mode.

It is a little tricky, admittedly and not the cleanest API possible. Reading/writing certain registers in the network have side effects and move the internal machine pointers around.