DSE II Remote Farm SSH Setup

[nick-f]

We are trying to use DSE with a farm of Linux servers, via SSH.  The DSE client will run on Windows 10.

I have got the SSH connection working, but DSE fails at the next stage. Here is the output from DSE:

Info: Starting compile design only exploration
Info: buildpc2: Connected Successfully
Error: Wait for workers to phone home have timed out for 1 workers.

When I look on the Linux server, DSE had created some folders (dse/, dse/dse1, and des/dse1/dse1_base) in the folder which I allocated as the IWD (initial working directory). But it doesn't seem to get any further than that.

Here are the ssh_farm_args we are using:

ssh_farm_args=farm_os=linux`hostname=buildpc2`iwd=/home/linuxbuild/FPGA/dsebuild`jobs_per_host=1`local_remote_path_map="\\buildpc2\FPGA\dseshare";/home/linuxbuild/FPGA/dseshare`private_key=C:\Users\Nick\Documents\dse.ppk`quartus_rootdir=/home/linuxbuild/intelFPGA_pro/21.3/quartus`ssh_client=C:\Program Files\PuTTY\plink.exe`user=linuxbuild

The local_remote_path_map is mapping a shared drive on the Linux machine, which is accessible from the Windows machine. We are not sure if this is correct (the documentation is not at all clear on this).

We would be glad of any help with this, as we have spent many hours trying to get this to work

Nick.

[Nurina]

Hi,

Thank you for using Intel Communities.

I'm consulting our internal team about your issue and I will let you know of any solutions/updates.

Regards,

Nurina

[Nurina]

Hi,

Try quartus_dse -help, you will be able to see the following online help related to SSH:

SSH Example:

SSH Prerequisites:

Before running an ssh remote compilation, be sure you can ssh to

the remote host without having to enter a password or private

key passphrase. Check the following:

1. Confirm the public ssh key pair that corresponds to

the private ssh key you want to use is appended to the

user's ~/.ssh/authorized_keys2 file on the remote host.

2. Windows PuTTY users (skip if connecting from Linux):

PuTTY uses a different ssh key file format called *.ppk.

A Linux ssh private key (id_rsa) file needs to be converted

to the *.ppk format using puttygen.

Run puttygen and click on its Conversions->Import key menu option.

Then press "Save private key" to enter a filename for the ppk file.

3. Run the ssh key agent and register the private ssh key

with the key agent program. This is a one-time setup to decrypt

private ssh keys and make them accessible to the ssh client.

Note: This step needs to be repeated every time the machine is rebooted.

Linux example adding private key to key agent:

> ssh-add <path to private ssh key>

Windows PuTTY example adding private key to key agent:

> pageant <path to private ssh key>.ppk

4. Remote connect to host and confirm connection can

be completed without having to enter password or passphrase.

Linux:

> ssh -i <private ssh key file> -l <user name> hostname

Windows PuTTY:

> plink -i <private key *.ppk file> -l <user name> hostname

Linux SSH Example 1: Launch an ssh Remote Compilation To a Remote Linux Host.

> quartus_dse --launcher ssh --num-seeds 2

  --ssh-farm-args='hostname=machine`private_key=/data/abc/pkey`quartus_rootdir=/tools/acds/quartus/14.1/'

  one_reg

Windows SSH Example 2a: Using PuTTY's ssh Client to Launch

an ssh Remote Compilation to a Remote Linux Host:

Start the PuTTY ssh key agent

> pageant c:\mykeys\my_pkey.ppk

The path to plink.exe and the private key

should use the '/' character and can not contain any spaces.

The ssh-farm-args arguments should be on one line.

It is split for readability here.

> quartus_dse --num-seeds 2 --launcher ssh

  --ssh-farm-args=ssh_client=c:/bin/plink.exe`

hostname=machine`

private_key=c:/mykeys/my_pkey.ppk`

quartus_rootdir=/tools/acds/quartus`

local_remote_path_map=S:\;/`

farm_os=linux one_reg

Windows SSH Example 2b: Windows Local Host Using PuTTY's plink Client to

Launch to Windows Running a Cygwin sshd Server:

 Paths that are to be used by the remote host need to

 use /cygdrive/. References to the shared S:\ path are

 mapped to /cygdrive/s on the remote Windows host.

> pageant c:\mykeys\my_pkey.ppk

> quartus_dse --launcher ssh

  --ssh-farm-args=farm_os=windows`

hostname=machine`

iwd=/cygdrive/c/build`

private_key=c:/mykeys/my_pkey.ppk`

quartus_rootdir=/cygdrive/c/acds/quartus`

ssh_client=c:/bin/plink`

local_remote_path_map=S:\;/cygdrive/s/ one_reg

[nick-f]

Hi NurinaW, and thanks for the reply. There was some useful info there, that would have helped me earlier.

But in fact I already had the SSH link working (you can see the "Connected successfully" message that I pasted).

I have now solved my issue. From the DSE GUI it was necessary to set the address of the host machine (in my case, the windows machine running the DSE client). This is hidden under the Explorations tab, so I didn't see it. In the .dse file it looks like:

    host=<ip address of my machine>

Another thing that confused me is the local_remote_path_map.  DSE won't run without this, but it seem we can put any folder names in there and they didn't seem to get used for anything. Can anyone explain what this is for?

Finally, I was hoping to also use Windows as my server OS, but your example still seems to require Cygwin. We want to stop using Cygwin here, and instead use tools such as openSSH that are built into Windows now. Has anybody got this working?

Thanks,

Nick

[Nurina]

Hello Nick,

Apologies for the late response. locate_remote_path_map is used to map between local and remote host, this is required since you are using a remote farm setup. You may refer to this video: https://www.youtube.com/watch?v=WdO5VOQkl7s

As for your concern about using Windows as your server OS, I will have to consult our internal team about that. I will let you know of any updates from them.

Regards,

Nurina

[Nurina]

Hi Nick,

We are still investigating this issue, we will let you know of any updates.

Regards,

Nurina

[Nurina]

Hi Nick,

Internal team is still investigating the problem. I will let you know of any updates.

Regards,

Nurina

[Nurina]

Hi Nick,

Here is what I received from the internal team:

=================================================================

DSE has not been validated with Windows's OpenSSH server. Customer can try these steps to get started and familiar w/ Microsoft OpenSSH.

quartus_dse -h has additional online documentation. We will enhance the GUI w/ additional information on SSH once this has been validated.

1) First step to getting DSE to work w/ OpenSSH is to install OpenSSH. The openssd server service should be installed on remote host. Microsoft has detailed instructions below. 

https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview

https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse

2) Setup SSH key based authentication. Run the ssh-keygen -t ed25519 executable on remote host.  This generates key using Ed25519 algorithm. (ssh-keygen -t rsa -b 2048 is alternative). When prompted file to save key, enter C:\Users\<username>\.ssh\<key_filename> When prompted about passphrase, leave blank and press enter. (If you use a passphrase, you will need to run a ssh-agent and use ssh-add <private key>. For now keep it simple and run w/o passphrase.) ssh-keygen will create two files, c:\Users\<username>\.ssh\<key_filename> and <key_filename>.pub.  The <key_filename> is the private key that DSE can use to connect.

https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement

3)  Customer should confirmed from one host you can connect to the remote host using the ssh executable that is part of OpenSSH. Run: ssh  -i C:\Users\<username>\.ssh\<key_filename > -l <username> <remote host or IP address>

If connecting to remote host for first time, it may ask about fingerprint. Accept.

More details here: https://man.openbsd.org/ssh

3) Once customer has confirmed it works we can try using DSE to run remote compile. 

Start quartus_dsew GUI. Open Project. Click Setup. Select Remote: SSH. 

Specify hostname of remote machine or remote machine's IP address. 

Specify the private_key: C:\Users\<username>\.ssh\<key_filename>.

Specify the Quartus Root Path on the remote machine: Example: C:\acds\quartus

Specify Farm Operating System: windows. 

Local to Remote Path Mapping. c:\;c:\. This field should not be marked required if going connecting from Windows to Windows and there is no shared drive. Initial Working Directory. Specify the directory the remote machine should start in: C:\.

=================================================================

Regards,

Nurina

[Nurina]

Hi Nick,

We did not receive any response to the previous answer provided, thus I will put this case to close pending. Please post a response in the next 15 days to allow me to continue to support you. After 15 days, this thread will be transitioned to community support. The community users will be able to help you with your follow-up questions.

Regards,

Nurina

P/S: If you like my comment, feel free to give Kudos. If my comment solved your problem, feel free to accept my comment as solution.

[nick-f]

Hi Nurina,

Thank your for your reply, I am grateful for Intel's attention on this request. Sorry for the delay, I have been away for a few days.

In fact I had already followed the steps in your last post, for DSE with Windows's OpenSSH server.

I had basically used exactly the same steps to set up the server on openSSH, but recently I have also tried using PuTTY on the client side, just in case it made any difference.  I don't believe it does make a difference, and I am continuing to use PuTTY on the client side right now.

The DSE client is correctly making an SSH connection to openSSH running on the Windows server. The problem happens after that.....

The first problem is that the default shell on Windows openSSH is a Windows command shell. It clearly does not understand the incoming remote job from DSE.

So I tried changing the default openSSH shell on the Windows server to be a BASH shell. There are instructions here:  

https://www.hanselman.com/blog/the-easy-way-how-to-ssh-into-bash-and-wsl2-on-windows-10-from-an-external-machine

In HKEY_LOCAL_MACHINE\SOFTWARE\OpenSSH i have set DefaultShell to C:\Windows\System32\bash.exe

But this still returns errors. It looks like some basic problems to do with path formats. Here is the transcript from DSE:

Info: Job 1 : quartus_sh --ipc_json --flow compile vproc.qsf
Info: Explorer server: http://10.60.10.190:50817 started.
Info: Starting seed sweep only exploration
Info: buildpc1: Connected Successfully
Info: buildpc1: /bin/bash: line 6: export: `Files/PuTTY/:/mnt/c/Program': not a valid identifier
Info: buildpc1: /bin/bash: line 6: export: `Files/Git/cmd:/mnt/c/Program': not a valid identifier
Info: buildpc1: /bin/bash: line 6: export: `Files/TortoiseGit/bin:/mnt/c/Windows/system32/config/systemprofile/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/Cyviz/AppData/Local/Microsoft/WindowsApps:/mnt/c/intelFPGA_pro/21.3/questa_fse/win64:/mnt/c/intelFPGA_pro/21.4/questa_fse/win64': not a valid identifier
Info: buildpc1: /bin/bash: line 10: cd: C:FPGA: No such file or directory
Warning: Worker for revision: dse1_1 failed to register with exploration server. It may have terminated abnormally: Terminated
Error: Completed exploration. Results: 0 passed; 1 failed; 0 stopped.

I should say that I am no expert with this stuff, so I am quite possibly missing something obvious!

Thanks again and regards,

Nick