"Intel offers the entry-level Intel Quartus Prime Lite Edition and ModelSim - Intel FPGA Starter Edition software. This software supports selected devices, provides limited feature support, and does not require license files."
I am not suere about the installer of Quartus Lite but the installers of the Standard and Pro versions come with both Modelsim Free and Paid; you have to specifically install the Free version. Furthermore, Modelsim Free Edition has a line limit of 10,000; maybe you are running into that limit?
By any chance, is it failing when you re-run it after a successful run? I've had issues with some Modelsim sub-processes not getting properly terminating when running on Linux (which then gives weird behavior when you then re-run).