Intel Fortran debugger extension for vscode

[michaelkonecny]

This is a message explaining why I think debugging Fortran binaries in vscode on Windows is needed and what can be done to allow it.

The only way to debug Intel Fortran programs on Windows at the moment is using Visual Studio with the Intel Fortran bundle. While I understand that Visual Studio has been *the tool* for low-level programming on Windows for a very long time, the times, they inevitably are a-changing. Microsoft has invested a lot of effort into Visual Studio Code (vscode) which is by now a much better IDE for the majority of tasks for a number of reasons.
Aside from that, I have to admit I tend to dislike Visual Studio since the moment I had to start working with it 6 years ago, even before vscode was as good as it it now. I'll try to elaborate on the reasons below.

Reasons I dislike Visual Studio
Disclaimer: this is going to sound like a lot of complaining. I feel like all the arguments are valid, though. The problems I describe often appearing simultaneously, which poses real workflow issues both to me and everyone in a small Fortran team in the company I work for.

Clumsy build configuration
Example: A compiler has an option called /Qsave. Why do I have to remember it is also called Data->Local Variable Storage->ALL variables SAVE in Visual Studio?
This makes building projects for both Windows and Linux especially cumbersome.

The VS's build configuration GUI also makes it very hard to see which options have default values set and which have been changed.
It is also very hard to see whether the options are the same in different configurations/platforms.
(I realize I can choose All Configurations and then go through all the options to check. I still have to go through all the sections, though.)

I just want to be able to type all the compiler options I want in a single line in a makefile and be done.

Projects stop building for no good reason (the famous "The operation could not be completed" error)
This error randomly appears in the build output window. No further explanation, I have never found the cause. VS restart is needed each time. Sometimes, I have to restart VS dozens of times in a day because of this.

VS randomly freezes when editing fortran files
I'm suspecting this is an IntelliSense issue. This doesn't happen often, but is extra annoying when it does, as unsaved changes get lost.

Fortran editor slows down to an almost unusable speed after an hour of work, VS restart needed
Before becoming unusable, the editor gradually gets slower, which is annoying on its own. A person wants a responsive editor, not having to wait for a character they typed to appear in a second or more.

Supplied IntelliSense is very basic, doesn't offer much help
I realize this can be improved slightly by checking more options in Tools -> Options -> Text Editor -> Fortran -> Advanced. Even then, the advisor isn't very smart compared to others.
fortls, which can be added to vscode, provides a much better service, despite it being a one-man project and being buggy.
LFortran also has plans (https://gitlab.com/lfortran/lfortran/-/issues/12) on adding a language server interface running on top of their parser (which already works), which will probably soon become the best option.

There are three people in our dev team and all of us experience all of the above issues - this shows it's not just me or just my machine.

Some might say I could go and try to resolve/work around most of these issues. But apart from that taking forever, this isn't a systematic solution. Some of the issues above are inherent to Visual Studio and its poor architecture. vscode feels to be a much better design.

What is currently available in vscode:
- one can edit fortran files just fine with the help of fortls
I actually very much prefer this to writing code in Visual Studio. I only do debugging there.
- one can build projects, using Make or any similar tool
- one can partially debug fortran binaries as well using the cppvsdbg debug configuration type (see attached launch.json (just rename it to remove the .txt extension (a limitation of this forum)) for an example).
Breakpoints can be added after setting "debug.allowBreakpointsEverywhere": true, in settings.json and they do work.
Scalars, strings and user-defined types can be inspected in the debugger. Arrays (one of the most important features of Fortran), however, cannot be viewed.

What is missing:
The main thing that seems to be missing in vscode is the inspection of Fortran arrays in the debugger.
This would, IMO, make vscode a completely usable IDE for most of the tasks one needs.

What I'm trying to plea for here is that Intel supply a vscode debugger extension (https://code.visualstudio.com/api/extension-guides/debugger-extension) which would add fortran arrays debugging and allow breakpoints in .f90 files.

Something could perhaps be achieved by just using a .natvis file (https://docs.microsoft.com/en-us/visualstudio/debugger/create-custom-views-of-native-objects?view=vs-2022), which would a bit of a hack. A proper debugging extension would most certainly be a lot better, though.
If Intel doesn't have the resources available to do this, or it simply doesn't fit their strategy (which I think it should), maybe they can at least provide the information necessary for someone in the Fortran community to write the extensions on their own?
The main thing needed is the knowledge of how and where are arrays and their metadata stored in memory.

There seems to be a bit of a renaissance period going on around Fortran.
I'm a big fan of Ondrej Certik's https://fortran-lang.org/ project, for example.
This is to say Fortran is by no means a dead language.

We need to bring it to the 21st century, though, even for Windows users.

I'd like to hear reactions from other Windows users - would you also welcome this? Please give a thumbs up/leave a comment or a suggestion, so there's an idea of how many people want this - as well as a reaction from Intel - how can we make this a reality?

Thank you!
Michael Konecny

[FortranFan]

@michaelkonecny ,

I agree with you wholeheartedly on the idea of Intel Software team providing debugging support with Intel Fortran in VS Code.

Also, Intel Software team would do well to enhance the integration of Intel Fortran in Visual Studio IDE.

The rest of your long note and complaints and how-to suggestions in particular appear unnecessary, perhaps they are even a great distraction from and unfruitful toward your desired outcomes.  

Re: "how can we make this a reality?," I suggest

1. If you haven't already, start with your company procuring paid support subscription for yourself and your fellow developers, then
2. seek out a named account rep toward your subscription, and
3. you work through this account rep to arrange a (virtual) meeting with person(s) who have product stewardship of Intel Fortran (IFORT and IFX) as part of their role on the Intel Software team side, and
4. you engage with such person(s) to request debugging support with Intel Fortran in VS Code and ask how that can be achieved in a timeline acceptable to you and your company.

You can use my Nom de Plume as another customer who seconds your request (The Intel folk will hopefully make the customer connection with my id).

[michaelkonecny]

Thank you.

The reason I included the how-tos is to illustrate that the debugger extension is the only thing Intel really has to work on.
I wanted this to be clear, in order for them not to create a big impossible job of it. We don't need a full-fledged solution with all the bells and whistles one can think of. Just make debugging possible. That's all.
Everything else is already there or can be provided by the community.

Feels like it should be a no-brainer for Intel considering how much such a small thing would improve workflow on Windows.

[Barbara_P_Intel]

A Feature Request (FR) has been filed for VS Code and Fortran debugging.  Are there any other goodies anyone would like to see added besides "Fortran arrays debugging and allow breakpoints in .f90 files?"  

I will add them to the FR.

[michaelkonecny]

Thank you @Barbara_P_Intel . This filled me with some optimism.

Meanwhile, in a discussion we're having about this over on fortran-lang discourse , Giannis Nikiteas remembered an issue on vscode-cpptools's github, where this had already been discussed:

https://github.com/microsoft/vscode-cpptools/issues/8479
A developer there points towards a way this should be implemented (perhaps by creating an Expression Evaluator as a Concord Extension).
Maybe you could add this to the FR to aid your developers?

It would be great if this was an open source project, so the community could build on it further. Is this in any way possible? Can you add this suggestion to the FR?

Is there a way to follow the FR?
I'd like to know when anything new is going on.

Also, is there a way I could become a beta tester for this?

I'd love to be able to provide constructive feedback.

[Barbara_P_Intel]

Unfortunately, there's no way for you to track the FR.  I added the links you provided to the FR.

I also said you'd like to be a beta tester.  Thank you!

[Steve_Lionel]

I'm going to throw my two cents in, and that's all it's worth right now, having been "out of the game" for five+ years...

What you're asking is a MUCH heavier load than you imagine, though I can see how in the long run it could be beneficial. The tight integration with VS is seen as a benefit to many users, and Intel is able to leverage Microsoft's doing most of the debugger work, only calling out to the Fortran expression evaluator when needed. The debug object format is, I think. proprietary to Microsoft and any more general debug support would probably require a parallel (in the "on the side" sense) implementation. Depending on open-source software and trying to support that is also problematic.

Maybe this has changed since 2016, but at that time, all of the VS integration work was being handled by contractors in Russia. You can imagine how problematic that is now. I'm not aware that Intel developed any "internal" institutional knowledge about the VS and debugger interface in the meantime, though maybe that would be forced to change now. Even if it did, catering to what is really a fringe use would not be considered a good use of limited developer resources.

I find your examples of issues somewhat amusing and perhaps misguided. I understand you are used to makefiles, but few developers mix makefiles and building in VS. The Intel documentation does a good job of cross-referencing options to project properties, and one can do VS builds from a batch script - many do. Others use makefiles to build and use VS only for debugging (which allows you to skip all the build issues.)

As for build errors - the most common one has to do with the Manifest Tool, which can be skipped for most Fortran projects.

Now, if Microsoft decides to stop development on VS and switch entirely to VS Code, that would be something else - it would be very hard for a few years on Intel. But I don't see that happening.

This is a long way of saying "don't hold your breath". Like pretty much the entire industry, Intel doesn't have pools of experienced developers just twiddling their thumbs waiting for new projects to work on - there was a major loss of institutional knowledge resulting from the "reduction in force" of 2016 that affected far more than Fortran. I'm pleasantly astonished with how much they have managed to accomplish with Fortran over the past five years.

[michaelkonecny]

Thank you, Steve, for taking the time to chime in. You are a legend.

What I was mainly trying to achieve by describing the issues is to convey the feeling I have from working with Visual Studio. Everything seems at least twice as complicated as it has to be.
Another way of describing the difference between VS and vscode would be to say it's like driving a 25 year old car vs. a 5 year old car. I prefer the newer one.

I think calling vscode a "fringe" use could have been apt several years ago, but as of this year, it's already the 3rd most popular IDE and has a number of downloads at about 40 % of that of VS (https://pypl.github.io/IDE.html).
I don't think Microsoft is planning on replacing VS with vscode just yet, just because of the sheer volume of functionality in VS, but vscode has definitely become a full-fledged IDE, a completely valid best choice for a lot of uses (writing Fortran code included, IMHO).

Nevertheless, thank you for pointing out Manifest Tool could be causing the "operation could not be completed" error. I will try disabling that, if it helps, that would already be a huge improvement.

As for the vscode extension, from what I gather, it might be enough to also only implement an Expression Evaluator as a Concord extension, Concord being the debugger engine for vscode. So the situation seems to be quite similar to what you've described for VS. There's probably no need to do a parallel on the side implementation of a whole debugger. I haven't studied this in detail, but this is the sense I get.

I realise Intel's resources aren't limitless and that the logistics regarding Russian workforce make things complicated. Working with VS was bugging me so much, though, that I decided I must somehow try to push this in the direction I think is right.

I will probably eventually iterate to a state with VS which I'll find satisfactory - mainly by getting rid of the operation could not be completed error and by switching to makefiles instead of the VS project config.
I will probably still use vscode for writing new code, just because it feels so much nicer, faster, more practical.

I wish I didn't have to, though. Wish I could just use my favourite editor. It would make life nicer for all the Fortran newbies, as well.

[Steve_Lionel]

I use VS Code (to build firmware for one of my 3D printers), but don't find it materially better than Visual Studio. The reason it is widely used is that it is a good, free IDE with easy extensibility by open-source packages. Really, a lot depends on what you are used to. I see Mac users exclaim how "easy to use" and "intuitive" MacOS is, but I find it maddeningly opaque and frustrating, largely because I haven't lived in that environment. 

[michaelkonecny]

I admit my reservations towards VS may be, in a part, subjective. The thing is, my overall feeling is that it's "always something" with VS, whereas vscode just works as I expect. If VS worked correctly, then I would agree it's just a matter of preference.

I would also think a lot of users who are coming to Fortran because of performance are likely to be coming from python (as did I), so vscode could be more likely to feel natural.

But this is splitting hairs... the main reason still is the number of annoyances one has to work through in VS.
Hopefully I can resolve all of them, if I do, I think I'll publish a cheat sheet or something so everybody doesn't have to work it out on their own...

[JNichols]

I find that VC does not work as I expect and am quite happy with VS.  We all come from different backgrounds.   

[FortranFan]

With Visual Studio as well as Visual Studio Code, I have tried to spend a fair amount of time going through what's increasingly the typical developer workflow toward engineering and scientifically based application and tool/solution development in industry, albeit using simple (<500 LOC) examples toward root-finding, quadrature, etc., some with C++20 and others C++17 .  Visual Studio Code offers distinct advantages compared to Visual Studio, particularly with IntelliSense and the IDE.  This is especially the case if one keeps in mind the very near future in terms of where things are going and all the stuff the developers are asked to do and what they need to do to more productive.

Attention @Barbara_P_Intel , Intel would do well to offer something that enables better debugging experience with Visual Studio Code, that will really boost what they have embarked upon with Intel oneAPI.  The potential benefits for Fortranners is huge.

[Barbara_P_Intel]

Understood. This thread is part of the Feature Request for the Intel team to read.

[FortranFan]

A renewed search for Visual Studio Code of late on a personal device has led to the following link from Microsoft getting highlighted for me with the use of Outlook in Edge (it suggests again the standard refrain: someone or other is always following you on the digital trail of bread crumbs you leave online):

https://docs.microsoft.com/en-us/shows/vs-code-livestreams/getting-started-with-rust-in-vs-code?ocid=AID3031635&utm_issue=April2022

Nonetheless, that brings up a question: when, if ever, will there be a livestream online called "Getting started with Fortran in VS Code" that covers, "Come learn how to get started with programming in Fortran in Visual Studio Code and set up your development environment. We'll cover setting up the Fortran-analyzer extension, using code completion to write code faster, how to refactor Fortran code, running tests, and debugging."?

[michaelkonecny]

This poll over at fortran-lang Discourse is probably worth noting: Code Editors/IDEs used for Fortran ,
vscode being the most popular among 106 users.

[Jacob_Williams]

+1 for VSCode integration. The main reasons are:

* It's free.

* It's cross platform.

* That's where the developers are. Nobody nowadays wants to use a different editor for different languages.

Even though oneAPI is now free, there is still a barrier to use it because of the Visual Studio connection on Windows. There needs to be a single way to use it on all platforms that is totally free and easy to set up.

[Steve_Lionel]

Visual Studio is free for just about everyone. VS Code doesn't include components that Intel Fortran depends on - VS is more than just an IDE.

[Jacob_Williams]

Ah, but Visual Studio is not free, in general, in an enterprise environment (last time I checked). The free one has some restrictions (open source, or educational use, etc.) Also there is no Linux version (and not really a Mac version either). 

Ideally i want oneAPI as an installable extension in VSCode. So download VSCode (free), type 'oneAPI' in the extension dialog, click install, and I now have everything ready to go. And it works the same on all platforms. That would be a great user experience!

[FortranFan]

@Steve_Lionel wrote:
Visual Studio is free for just about everyone. VS Code doesn't include components that Intel Fortran depends on - VS is more than just an IDE.

The fact of the matter is the integration of Intel Fortran with Visual Studio is practically in a "broken" state.

Sure if one is coming to the use of VS IDE from legacy FORTRAN perspective and the days of tons of PRINT statements as the debugging approach and with little to no build systems or the use of manual scripts masquerading as build tools, the integration with VS to enable the debug facility and the project/solution configuration, etc. is a huuuuge step up.  However many powers-that-be tell me such users are now in a rapidly diminishing minority, especially in the enterprise domain.

On the other hand, there is a growing number of users who need to support and maintain Fortran codebases but who are extremely savvy with VS specifically and IDEs generally and they know build systems inside out and are adept at CI workflow and so forth and who have remarkable expertise and experience with many other languages and their ecosystems.  These users find the integration of Intel Fortran with Visual Studio to be absolutely pathetic, the editor is useless for them as it freezes often, they then investigate and they find not only has the Intel Fortran integration with VS NOT had any meaningful improvement since early to mid-2000s, things break far too often with Intel Parallel Studio/oneAPI updates and the issues don't get fixed for years. 

The only hope is if Intel can make something - say a decent Fortran debugging adapter - available publicly and others can take over from there to offer Fortran extensions with VSCode.

[Jacob_Williams]

Agreed. The "intellisense" components of Intel's VS integration are really bad. It's just not usable or in any way what we expect nowadays from a code editor. VSCode with a couple of the opensource Modern Fortran plugins is a vastly better editing experience. Once you use that you cannot go back to Intel+VS. All we are missing is the debugger.

[JNichols]

I have to say I never thought I would be arguing with Jacob Williams, but I am going to argue. 

The beauty of human development is that different paths lead to different places, but in the end the streams may cross.  

I have tried VS Code.  My personal opinion is that it is a dog of a system.  Luckily, I am entitled to my opinion.  I like Visual Studio,  I appreciate the comments about enterprise editions,  as I appreciate my friend's comments about MS SQL Server on Tuesday night,  my friend was passionate about the fact that MS SQL Server was worse than a dog.   I had quietly moved her to MySQL several years ago, I should have deleted the MS SQL Server.

I also appreciate the thoughts that one might use LINUX.  I have programmed in LINUX quite a bit, and it is wonderful for some things, but I have had terrible problems with some features that just work in Windows.  Deploying  some remote sensors with cell phones is a beast of a problem with LINUX.   Some really good LINUX people could not solve one little critical communication problem. 

I had a jolly good chuckle last week after some one in another country told me we can solve this problem with LINUX and I know they are going to waste a lot of money on the problem.  But one cannot tell them.

A lot of stuff is now driven by high school students with RPi's.  At least most of their stuff is in the open domain.  Try and find a program in the open domain that reads acceleration and other data from a ST accelerometer.  Beside the one I just posted. 

In terms of mac's, my daughter wanted a mac, I bought her a top of the line machine, it sits in her bedroom and she uses a core i7 - 4200 I think,  for most of her work.   She was not born when I bought that machine. 

Humans are humans and saying something is best is a waste of effort.  

There are many computer languages, but as we all know if you want to dream of speed and Tom Cruise's new movie, then you dream in Fortran.  

My final opinion is anyone who would code in Python has some interesting problems.   But that is a personal comment.