It either asks for the missing source file, and then allows me to mix source and assembly code together,
or it doesn't ask me for source code at all, and I can only view the assembler.
How do I specify the source code manually, if it doesn't prompt me automatically?
Might it be that the object files, where you aren't asked for a source file, do not contain debug information? (e.g. because they were not compiled with -g or because they are stripped) There is an option that let's you choose if you want to be warned about this or VTune should simply show the assembly code. I suspect that the first variant is turned on for you.