Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Novice
1,675 Views

Send data from Galileo to dataserver

Jump to solution

Hi I have a project send data from Galileo board to data server, I researched around but I can't find the best solution for my project. Anyone can help me with the idea.

I use Galileo talk to GPS sensor through a serial pin by Arduino sketch. the result will pass to python script then send to data server. My question is how python script can get result from Arduino sketch, is there any function or we should write to a text file?

Second, my data server is just a work station, the sql is created by MS access, is it applicable to send data to this work station

Tags (1)

Accepted Solutions
Highlighted
Honored Contributor I
14 Views

Hello FrankBot,

I use Galileo talk to GPS sensor through a serial pin by Arduino sketch. the result will pass to python script then send to data server. My question is how python script can get result from Arduino sketch, is there any function or we should write to a text file?

 

I implemented something similar to what you describe, and it worked. I do not know if it is the best solution, however. If the python script is used only for sending the data, then maybe it would be better to send the data from the sketch and avoid python, actually. Anyhow, what I've implemented could be described as:

a) Sketch gets and writes data to a file

b) Sketch signals (via a file creation) the python script data is available

c) Python gets data and deletes file (so it is not read twice by mistake)

i.e. I used files as buffer and for "signals"... not very elegant, but worked. I can post some code, if you want.

HTH,

Fernando.

View solution in original post

0 Kudos
27 Replies
Highlighted
Honored Contributor I
15 Views

Hello FrankBot,

I use Galileo talk to GPS sensor through a serial pin by Arduino sketch. the result will pass to python script then send to data server. My question is how python script can get result from Arduino sketch, is there any function or we should write to a text file?

 

I implemented something similar to what you describe, and it worked. I do not know if it is the best solution, however. If the python script is used only for sending the data, then maybe it would be better to send the data from the sketch and avoid python, actually. Anyhow, what I've implemented could be described as:

a) Sketch gets and writes data to a file

b) Sketch signals (via a file creation) the python script data is available

c) Python gets data and deletes file (so it is not read twice by mistake)

i.e. I used files as buffer and for "signals"... not very elegant, but worked. I can post some code, if you want.

HTH,

Fernando.

View solution in original post

0 Kudos
Highlighted
Novice
14 Views

Hi FGT, thanks for your reply

I'm very excited if you share your project code with me. that's really what I'm looking for.

I still have 1 concern that need your advice since I've never worked with data server. Can we send data to table in data server by text file?

Frank

0 Kudos
Highlighted
Honored Contributor I
14 Views

Hi Frank,

FrankBot wrote:

I'm very excited if you share your project code with me. that's really what I'm looking for.

A sketch code snippet adapted from something I've made in the paste would be:

-----------------------------------------------------

// Set up everything

...

void loop()

{

/* Files (named pipes) for cgi process comm. */

FILE *fpin, *fpout;

char line[50];

int i;

...

// Get data and put data into "line"

...

// Store data in file

system("umask 000; touch /tmp/files/ardu2python");

fpout = fopen("/tmp/files/ardu2python", "w");

fputs(line, fpout);

fclose(fpout);

// "Signal" the python process

system("umask 000; touch /tmp/files/gopython");

...

}

-----------------------------------------------------

In the python code:

-----------------------------------------------------

# Wait for the file /tmp/files/gopython to exist

...

# Read/process/send the data in the file /tmp/files/ardu2python

...

# Delete files /tmp/files/gopython and /tmp/files/ardu2python

...

-----------------------------------------------------

If you think there would be race conditions, then add another file to signal sketch to go on running

About:

"I still have 1 concern that need your advice since I've never worked with data server. Can we send data to table in data server by text file?"

I do not understand what you mean/need...

HTH,

Fernando.

0 Kudos
Highlighted
Novice
14 Views

Hi Fernado,

Thanks for your reply, those are very helpful.

The other question means the format of record file, I chose to write in csv, similar like txt but can easily send to database by python.

Regards

Frank

0 Kudos
Highlighted
Honored Contributor I
14 Views

The other question means the format of record file, I chose to write in csv, similar like txt but can easily send to database by python.

Then, I think you have figured it out, right?

Fernando.

0 Kudos
Highlighted
Novice
14 Views

Right, Fernado

Frank

0 Kudos
Highlighted
Novice
14 Views

Hi Fernando,

I have written data to csv file. so what protocol that galileo support to send file to server without installing new library? Can I use SCP command? I had stastic server IP already

0 Kudos
Highlighted
Honored Contributor I
14 Views

I have written data to csv file. so what protocol that galileo support to send file to server without installing new library? Can I use SCP command? I had stastic server IP already

Hi Frank,

I think scp should be available, I do not have a Galileo at hand to take a look. Take into account that the server should have ssh server.

Fernando.

0 Kudos
Highlighted
Novice
14 Views

Hi Fernando,

SCP is support SSH, and my Gallileo is not always running PuTTy. can it use SCP in system command by python? I have had a server already, what I need to do on server to send data through SCP? Can you give me more detail information on tranfer file between galileo and Server? I am not good at networking

I read about FTP and my server was installed FTP. Do you know how to use FTP with Galileo?

0 Kudos
Highlighted
Honored Contributor I
14 Views

SCP is support SSH, and my Gallileo is not always running PuTTy. can it use SCP in system command by python? I have had a server already, what I need to do on server to send data through SCP? Can you give me more detail information on tranfer file between galileo and Server? I am not good at networking

I read about FTP and my server was installed FTP. Do you know how to use FTP with Galileo?

Hi Frank,

Both scp and ssh are client programs, which at the server side need sshd or something like it: the ssh server. ssh and scp are unrelated to PuTTY which is another way of running a ssh client, actually.

At the server side, you need only the ssh or ftp server, if you actually are able to do ssh to the server, you can do scp to the same server.

At the client side, you need scp or ftp client, respectively (scp if the ssh server is running at the server side or ftp if the ftp server is running at the server side).

I think that ssh client, scp, and ftp client are already installed in Galileo. Just type those commands at the terminal prompt and see if they work.

I do not fully understand your setting, would you describe a little bit more?

Fernando.

0 Kudos
Highlighted
Novice
14 Views

HI Fernado,

I am not aware galileo have ssh scp and ftp inside. Just though we need putty then ssh board to use those cmdline. Silly me...

IT gives me FTP server with stastic IP, acc, pass, port. We still don't know how to interact between board and server

as you said, would I write system(...) in sketch instead of python, board will perform faster right?

would you mind demo cmdline for FTP transfer a file to server?

0 Kudos
Highlighted
Honored Contributor I
14 Views

as you said, would I write system(...) in sketch instead of python, board will perform faster right?

I do not understand this... but put it aside for a while.

IT gives me FTP server with stastic IP, acc, pass, port. We still don't know how to interact between board and server

...

would you mind demo cmdline for FTP transfer a file to server?

Try first a Galileo terminal command line like

scp

and replace with the static IP IT gives you, it something like usr or passwd appears at the terminal, then everything is fine with the network, server, client, etc. Once this is fine, we can move forward, I think.

Fernando.

0 Kudos
Highlighted
Novice
14 Views

Hi,

Do i need to install anything in server for use scp? Or FTP support SCP also, the server is window server.

moving forward. Do I need to use terminal? I can write command line in arduino sketch either python... System(scp...)

highly appreciated your spending time for responding and explaining

0 Kudos
Highlighted
Honored Contributor I
14 Views

 

Do i need to install anything in server for use scp? Or FTP support SCP also, the server is window server.

Hmm... my mistate, I didn't see IT gave you ftp server, not ssh server... ftp does not provide scp support...

moving forward. Do I need to use terminal? I can write command line in arduino sketch either python... System(scp...)

The terminal is not strictly needed, but let us try and solve one issue at a time... do you have a Galileo terminal? It is not terrible if you don't, it makes things simpler for me... so maybe I'll have to think some alternative way.

highly appreciated your spending time for responding and explaining

You are welcome. I learn a lot too while trying to solve issues like this one.

Fernando.

0 Kudos
Highlighted
Novice
14 Views

I have a galileo board, and it shall run alone in the vehicle while upload data to the server. That's why I try not to write cmdline through putty. I am also looking for 1 line cmd scp or ftp Frank

0 Kudos
Highlighted
Honored Contributor I
14 Views

FrankBot wrote:

I have a galileo board, and it shall run alone in the vehicle while upload data to the server. That's why I try not to write cmdline through putty. I am also looking for 1 line cmd scp or ftp Frank

My idea is to experiment with the command line before figuring out the final setup without command line (no matter if in a vehicle or any other environment) that's why I need you tell me whether you are able to use a terminal or not.. are you?

I'm curious about how the Galileo shall run alone in the vehicle... How do you plan to power the Galileo?

Fernando.

0 Kudos
Highlighted
Novice
14 Views

Terninal you mean PC connect with galileo? If it is, I cannot use

i implant in the bus which have power socket, futhermore I use power bank if need.

0 Kudos
Highlighted
Honored Contributor I
14 Views

i implant in the bus which have power socket, futhermore I use power bank if need.

What do you mean by "i implant in the bus"? What bus?

Would you send a picture about how you are running programs/sketches in Galileo? What power bank are you using?

0 Kudos
Highlighted
Community Manager
14 Views

Honestly I would reassess the approach and do something like send data from python reading your GPS sensor directly to a web app and store it in a database.

Since you are on Windows I would install a WAMP stack https://bitnami.com/stack/wamp WAMP Stack Cloud Hosting, WAMP Stack Hosting - Installers and VM .

Add code that would store data by the hash of the MAC address with a timestamp.

First create a database to store the GPS data in MySQL.

CREATE database galileoData;

use galileoData;

CREATE TABLE `gpsData` (

`id` int(11) NOT NULL AUTO_INCREMENT,

`date` datetime DEFAULT NULL,

`coordinates` varchar(8) DEFAULT NULL,

`sensor` int(11) DEFAULT NULL,

PRIMARY KEY (`id`),

KEY `id_idx` (`sensor`),

CONSTRAINT `id` FOREIGN KEY (`sensor`) REFERENCES `sensors` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION

) ENGINE=InnoDB AUTO_INCREMENT=71 DEFAULT CHARSET=utf8

 

CREATE TABLE `sensors` (

`id` int(11) NOT NULL AUTO_INCREMENT,

`hash` varchar(45) NOT NULL,

`owner` varchar(256) NOT NULL,

PRIMARY KEY (`id`)

) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8

Create an MD5 of the MAC address of your galileo to create a unique id.

ifconfig wlp1s0 | awk ' /HWaddr/ { print $5 } ' | md5sum

Then on PHP create a page to register your galileo

connect($db);

switch($s) {

default;

case 0: echo " Add a new galileo Sensor

galileo Hash

galileo Owner Email

";

break;

case 1:

if (strlen($h) != 32) {

echo "Error. Invalid MD5 Hash ";

exit;

}

$galileoHash=mysqli_real_escape_string($db, $h);

$galileoOwner=mysqli_real_escape_string($db, $e);

if ($stmt = mysqli_prepare($db, "INSERT INTO sensors set id='', hash=?, owner=?")) {

mysqli_stmt_bind_param($stmt, "ss", $galileoHash, $galileoOwner);

mysqli_stmt_execute($stmt);

mysqli_stmt_close($stmt);

echo "Success! Added a new galileo " . htmlspecialchars($h) . " with owner " . htmlspecialchars($e) . " ";

} else {

echo "Error. Try again ";

}

break;

And another page to insert the data.

//Check for a valid hash

if (strlen($h) != 32) {

echo "Error. Invalid MD5 Hash ";

exit;

}

//Verify a valid hash

$galileHash=mysqli_real_escape_string($db, $h);

$galileOwner=mysqli_real_escape_string($db, $e);

if ($stmt = mysqli_prepare($db, "select id from sensors where hash=? and owner=?")) {<...

0 Kudos