Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
New Contributor II
3,496 Views

clone eMMC image to SD Card or USB Thumb Drive

Looking for reference documentation discussing the process of replicating my development system environment and image to a secondary storage device (SD Card or USB Thumb Drive) to allow for quick upgrade to newly birthed systems.

Can this be performed in the BIOS?

I have an OSTRO image in eMMC, that I need to backup/save to secondary storage. And then I shall flash my working Ubuntu image from the USB Thumb drive into the eMMC.

Tags (1)
0 Kudos
23 Replies
Highlighted
New Contributor II
178 Views

I have tried the following with no success, possibly due to FAT32 file limitations of 4GB (Ostro does not recognize SD Card as NTFS or exFat).

SD Card 16GB - mmcblk1

root@intel-corei7-64:~# lsblk

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

mmcblk0rpmb 179:40 0 4M 0 disk

mmcblk0boot0 179:8 0 4M 1 disk

mmcblk0boot1 179:16 0 4M 1 disk

mmcblk0 179:0 0 14.7G 0 disk

|-mmcblk0p1 179:1 0 15M 0 part

|-mmcblk0p2 179:2 0 15M 0 part

`-mmcblk0p3 179:3 0 14.7G 0 part /

mmcblk1 179:48 0 14.4G 0 disk

`-mmcblk1p1 179:49 0 14.4G 0 part

mmcblk0gp0 179:24 0 4M 0 disk

mmcblk0gp3 179:32 0 4M 0 disk

mount /dev/mmcblk1p1 /mnt

dd if=/dev/mmcblk0 of=/mnt/joule_backup.img

root@intel-corei7-64:/dev# dd if=/dev/mmcblk0 of=/mnt/joule_backup.img

dd: writing '/mnt/joule_backup.img': File too large

8388609+0 records in

8388607+1 records out

root@intel-corei7-64:/dev#

0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

Thanks for your interest in the Intel® Joule Platform.

 

Thanks for all the information provided, I believe that the message you are getting is due to FAT32 file limitation of 4GB as you mentioned above. We would like to suggest you to try to compress the mmcblk0 files, take a look at this link: http://www.debianhelp.co.uk/ddcommand.htm http://www.debianhelp.co.uk/ddcommand.htm, maybe that information could help you with this.

Hope this information helps, if you have any questions or updates, let us know, please.

Regards,

 

-Yermi

 

0 Kudos
Highlighted
New Contributor II
178 Views

Updated Joule BIOS to latest BIOS

Performed Clone to SD Card:

Formatted SD Card on Win7 Fat32 block size 4K

Ostro image to SD Card Clone (time ~13 minutes

  1. mount /dev/mmcblk1p1 /mnt
  2. dd if=/dev/mmcblk0 | http://www.debianhelp.co.uk/ddcommand.htm gzip > /mnt/joule_image.gz

root@intel-corei7-64:~# mount /dev/mmcblk1p1 /mnt

root@intel-corei7-64:~# dd if=/dev/mmcblk0 | gzip > /mnt/joule_image.gz

30760960+0 records in

30760960+0 records out

root@intel-corei7-64:~#

root@intel-corei7-64:~# ls -l /mnt

-rwxr-xr-x 1 root root 816643768 Nov 16 15:18 joule_image.gz

 

Restore SD Card to eMMC memory

  1. Mount /dev/mmcblk1p1 /mnt
  2. gzip -dc /mnt/joule_image.gz | dd of=/dev/mmcblk0

command line output errors did occur, which I was not able to capture.

The Joule system eventually rebooted, and the display output was continuous. Powered the Joule off and then on.

Error

 

0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

 

 

Thanks for all details provided, please let us investigate and as soon as we find helpful information we'll let you know. We'll appreciate your patience.

 

 

Regards,

 

-Yermi

 

0 Kudos
Highlighted
New Contributor II
178 Views

Thanks.

As a side note, I followed the online Intel document to flash the on-board Joule eMMC using a USB and or SD Card (with the "default/golden" Ostro image), and was unsuccessful. Was using the latest Joule BIOS.

Have you successfully performed this procedure?

0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

 

 

Regarding to clone the image you can also format the drive (SD card or USB drive) as another file system. Moreover, we would like to know which error you are getting when flashing the Joule.

 

 

We'll be waiting your reply.

 

 

Regards,

 

-Yermi

 

0 Kudos
Highlighted
New Contributor II
178 Views

Yermi,

thanks for responding, but you did not answer my question above:

"Have you successfully performed this procedure?"

Please answer yes or no, and if yes, please provide additional information regarding process.

Note:

I cannot tell the system display was updated rather quickly, and I was unable to read and capture the failure.

Also, were you capable of successfully performing the following (yes/no answer would be beneficial):

1) Cloning eMMC (Ostro image and file system with user files and environments) saved/flashed to SD Card test # 1

2) Restore/clone the SD Card test 1 image back into eMMC, after erasing eMMC

3) Cloning eMMC (Ubuntu image and file system with user files and environments) saved/flashed to SD Card test # 2

4) Restore/clone the SD Card test 2 image back into eMMC, after erasing eMMC

We'll be waiting your reply.

0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

 

 

I would like to apologize for not having answered your question previously. Currently, I have not successfully performed this procedure, however, I'm trying to perform the 1 and 2 steps you have mentioned and I'll post my outcome as soon as possible.

 

 

Regards,

 

-Yermi

 

Highlighted
Community Manager
178 Views

Hi Floydg,

 

I have been working on performing successfully this procedure and I would like to share my results:

 

Cloning eMMC image (Ostro)
  1. Mount the SD card

mkdir /mnt/mmc

 

mount /dev/mmcblk1p1 /mnt/mmc

2. Backup the image

  1. First I tried to backup the eMMC using a SD card formatted in FAT32 and used this command
dd if=/dev/mmcblk0 of=/mnt/mmc/backup

I have got the same issue that you had got previously. Message: File too large.

 

b. Also, I confirmed that this issue is because the SD card is formatted as FAT and the file limit for FAT is ~4GB.

c. So, I ran the command that I had suggested you.

dd if=/dev/mmcblk0 | gzip > /mnt/mmc/backup

And I was able to clone the image, please check the picture below.

 

d. Then, I checked for the file in the SD card, see the following picture

 

Note: I tried to use the SD card formatted in NTFS and exFat, but I got these error messages:

 

NFTS

 

 

exFAT

 

 

Restore SD card to eMMC
  1. Mount SD card
  2. Restore the image
    1. I ran this command:
gzip –dc /mnt/mmc/backup | dd of=/dev/mmcblk0

When the procedure finished (I haven't got any error message) I got the following output:

 

 

I rebooted the Joule and it works well. I hope this information helps.

 

Regards,

 

-Yermi
0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

 

 

I would like to know if you have any updates or questions. Were you able to clone and restore the image?

 

 

Regards,

 

-Yermi

 

0 Kudos
Highlighted
New Contributor II
178 Views

Yermi,

I performed the following:

  1. mkdir /mnt/mmc
  2. mount /dev/mmcblk1p1 /mnt/mmc
  3. dd if=/dev/mmcblk0 | gzip > /mnt/mmc/backup

Error - gzip: short file

backup file created is 4661248 Bytes

0 Kudos
Highlighted
New Contributor II
178 Views

In regards to the gzip error above (short file), the SD Card maybe corrupted.

I formatted as FAT with Ubuntu, and Ostro still gave the same error when attempting to perform a "clone" - dd if=/dev/mmcblk0 | gzip > /mnt/mmc/backup

Inserted a "new" SD Card into the Joule, at "time 0" performed a clone to SD Card

intel_powerclamp: Start idle injection to reduce power

Intel_powerclamp: Stop forced idle

...

Intel_powerclamp: Start idle injection to reduce power

Intel_powerclamp: Stop forced idle

system display crashed at 10 mins

serial debug port operational.

dmesg:

[ 432.009995] mmc0: new high speed SDHC card at address 1234

[ 432.021530] mmcblk1: mmc0:1234 SA16G 14.4 GiB

[ 432.030240] mmcblk1: p1

[ 453.882691] FAT-fs (mmcblk1p1): Directory bread(block 68) failed

[ 453.894391] FAT-fs (mmcblk1p1): Directory bread(block 69) failed

[ 453.902791] FAT-fs (mmcblk1p1): Directory bread(block 70) failed

[ 453.911282] FAT-fs (mmcblk1p1): Directory bread(block 71) failed

[ 453.919659] FAT-fs (mmcblk1p1): Directory bread(block 72) failed

[ 453.928015] FAT-fs (mmcblk1p1): Directory bread(block 73) failed

[ 453.936362] FAT-fs (mmcblk1p1): Directory bread(block 74) failed

[ 453.944728] FAT-fs (mmcblk1p1): Directory bread(block 75) failed

[ 453.952980] FAT-fs (mmcblk1p1): Directory bread(block 76) failed

[ 453.961168] FAT-fs (mmcblk1p1): Directory bread(block 77) failed

[ 984.729400] intel_powerclamp: Start idle injection to reduce power

[ 988.744045] intel_powerclamp: Stop forced idle injection

[ 1008.800161] intel_powerclamp: Start idle injection to reduce power

[ 1012.817128] intel_powerclamp: Stop forced idle injection

[ 1032.880370] intel_powerclamp: Start idle injection to reduce power

[ 1036.896753] intel_powerclamp: Stop forced idle injection

root@intel-corei7-64:~#

0 Kudos
Highlighted
New Contributor II
178 Views

Win7 format Fat32 of 16GB SD Card (Full format - not quick)

re attempted clone (~35 mins to complete):

Output from dmesg:

root@intel-corei7-64:~# dmesg | tail

[ 1044.700565] i915 0000:00:02.0: [drm:broxton_ddi_phy_verify_state [i915]] DDI

PHY state mismatch

[ 1657.636559] i915 0000:00:02.0: [drm:broxton_ddi_phy_verify_state [i915]] DDI

PHY state mismatch

[ 2121.990164] CPU2: Package temperature above threshold, cpu clock throttled (t

otal events = 252)

[ 2121.990247] CPU3: Package temperature above threshold, cpu clock throttled (t

otal events = 252)

[ 2121.990260] CPU0: Package temperature above threshold, cpu clock throttled (t

otal events = 252)

[ 2121.990262] CPU1: Package temperature above threshold, cpu clock throttled (t

otal events = 252)

[ 2121.991270] CPU3: Package temperature/speed normal

[ 2121.991283] CPU0: Package temperature/speed normal

[ 2121.991285] CPU1: Package temperature/speed normal

[ 2122.045381] CPU2: Package temperature/speed normal

root@intel-corei7-64:~#

root@intel-corei7-64:~# ls -l /mnt/mmc

-rw-r--r-- 1 root root 12801561740 Nov 30 18:28 backupOstro

The Joule I am testing with, for the clone restore from SD Card, has a panic error (crash) while booting Ostro, since the eMMC image is bad. Shall test clone restore, after I get a good image installed or running Ostro from USB Drive.

Booted from USB Thumb drive (Ostro).

image is not recognized on SD Card after mounting: ls -l /mnt/mmc

FAT-fs (mmcblk101):Directory bread(block 8192) failed

...

FAT-fs (mmcblk101):Directory bread(block 8201) failed

Updated BIOS

attempted to clone to SD Card (dd if=/dev/mmcblk0 | gzip > /mnt/mcc/backupOstro): unsuccessful:

...

[ 980.674802] i915 0000:00:02.0: [drm:broxton_ddi_phy_verify_state [i915]] DDI

PHY state mismatch

[ 1688.944724] CPU3: Package temperature above threshold, cpu clock throttled (t

otal events = 29)

[ 1688.944809] CPU2: Package temperature above threshold, cpu clock throttled (t

otal events = 29)

[ 1688.944822] CPU0: Package temperature above threshold, cpu clock throttled (t

otal events = 29)

[ 1688.944825] CPU1: Package temperature above threshold, cpu clock throttled (t

otal events = 29)

[ 1688.945820] CPU2: Package temperature/speed normal

[ 1688.945833] CPU0: Package temperature/speed normal

[ 1688.945835] CPU1: Package temperature/speed normal

[ 1689.010168] CPU3: Package temperature/speed normal

gzip: short write

Yermi:

The 16GB of eMMC Flash is only ~17% used. Why does the entire 4GB USB drive "sdb1" fill up if the backup file is compressed?

gzip -9 "best" was included in the operation, and still ran out of space!

Yermi the Ostro image you tested may not have been as large as my development file system. Did you use a "best case" clean/minimal default Ostro install?

Filesystem Size Used Available Use% Mounted on

devtmpfs 1.9G 0 1.9G 0% /dev

/dev/disk/by-partuuid/12345678-9abc-def0-0fed-cba987654320

14.3G 2.3G 11.3G 17% /

tmpfs 1.9G 0 1.9G 0% /dev/shm

tmpfs 1.9G 25.0M 1.8G 1% /run

tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup

tmpfs 1.9G 0 1.9G 0% /tmp

tmpfs 1.9G 8.0K 1.9G 0% /var/volatile

tmpfs 382.9M 0 382.9M 0% /run/user/0

/dev/sda1 1.9G 1.9G 0 100% /mnt/sda1

/dev/sdb1 3.7G 3.7G 33.3M 99% /mnt/sdb1

root@intel-corei7-64:/#

0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

 

Thanks for all details provided, regarding your questions, I'm not sure why the 4GB USB is filling up, have you copied the backupOstro file, which you compressed above, into your USB drive?

 

As a note, I have noticed that in my Joule the eMMC is 100% used, look the following screenshot:

 

 

Regarding your last question, I'm using this image (see screenshot below) and just ran the instruction I have posted above.

 

 

I'll be waiting for your reply as well as any updates or questions you could have.

 

Regards,

 

-Yermi
0 Kudos
Highlighted
New Contributor II
178 Views

Yermi,

Are you not curious, or concerned that your system states 0% available for eMMC and 100% full, with no free space.

Something is strange with your system or the df command. Mine states 17% full, which sounds feasible.

I am using the same Yocto image as captured above in your note: (4.4.15)

Typically the backupOstro file appears damaged, and I have tried both SD Card and USB Thumb drive for the clone operation.

What size is your compressed image for your clone saved to SD Card?

0 Kudos
Highlighted
178 Views

It was not clear to me when reading through these posts whether the posters were trying to clone the image of a running system, or running off a secondary system (eg a LiveUSB). In my experience, while trying to clone and restore into a running system SOMETIMES works, if the filesystem gets updated while the copying is taking place the resulting image often gets corrupted. So it is better to run your OS off a different file system than the one you are cloning so you are sure it is not changing underneath you. Ditto for restoring a system on top of a running system... it's akin to DIY brain surgery. Typically if you are LUCKY the system crashes while you are doing it and you can get it running again after a few reboots...

Second, it was not clear to me whether people updated their BIOS first or not, or what versions they were running. There have been some fixes which will affect this process.

On other UEFI Intel systems (eg the UP Board) I have successfully used Clonezilla to make and restore disk images. Clonezilla is basically an instance of Linux on a LiveUSB that automatically runs a script that does the dd etc for you. I have not tried Clonezilla on the Joule but IF the bios is updated so eg a regular Ubuntu install works, then there is a chance it will work. Otherwise you can do the same steps manually, with appropriate fixes (eg if you are restoring an Ubuntu image, for instance, you may have to do the same fixes to the EFI needed for a regular install). Note that for Joule you probably need (at least) the Xenial version of Clonezilla to get a kernel that will run on the Goldmont architecture.

At any rate, I plan to take stab at this.

BTW: You should probably just format your backup media as ext3 or whatever first to get around the 4GB limitation of FAT32. There is no reason to look at it on a system that can't do ext3 anyway.

0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

 

 

I have checked another Joule and it has 15% used (2.1GB), I don't have the last file that I was clone, so I'll try to clone this image and as soon as I get it, I'll post the size of the file I have got.

 

 

Additionally, I would like to suggest you to take account the McCool's suggestion, it could help you.

 

 

Regards,

 

-Yermi

 

0 Kudos
Highlighted
Community Manager
178 Views

Hi Floydg,

 

 

Thank you very much for your patience.

 

 

I have been working on it, however I want to let you know that I'll continue working since I have getting some issues during the backup procedure. I'll appreciate your patience during the meantime.

 

 

Regards,

 

-Yermi

 

0 Kudos
Highlighted
New Contributor II
178 Views

Any status?

0 Kudos