This is one of my (many) soapbox issues. I believe a lot more software should employ databases -- especially scientific software.
The major impediments have been, in no particular order:
Organizational: databases are owned by the DBAs, code by the developers. It is often impossible to use a database for purely organizational reasons.
Cost: in the bad old world, database engines were prohibitively expensive so there were both development barriers and deployment barriers.
Familiarity: for the above reasons, most programmers have little experience with database mechanics and practices. Business (and now web) application programmer excepted, of course.
Portability: the libraries that provide the interface tend to be tightly bound to both the development tools and the database engine family.
The last one is still an issue. Plus you have to learn SQL, "a mistake carried through to perfection" [Datamation cover story].
On the plus side, you get a degree of parallelism for free, you enable multiple independently developed tools to share overlapping data collections, and your burden for documenting the data format (and writing the parser) is much reduced. If your application calls for interactive editing of the data using something other than a text editor you can save a vast amount of development and maintenance time by leveraging the database toolset's table editor -- or by sloughing the job off to a DB frontend programmer.
Functionally, you can leverage SQL to enable data selection operations without having to provide UI elements in your program plus logic in your data file parser to support every combination of criteria.
-swn
