I don't see what is inflexible about the standard prefetching techniques, as they extend to any size of data blocks. Maybe you could give an example, and specify an architecture.
On P4/Xeon architectures, hardware prefetch is interrupted when crossing a page boundary. That has no great effect on performance, if all data from each cache line are used. There would be no apparent reason for using software prefetch, unless you had to disable hardware prefetch. Effect of software prefetch on hardware prefetch varies between models.
On Itanium, software prefetch is engaged automatically at -O3. You could check the code generated by the compiler to see how it is done. The objective is to issue prefetch instructions several hundred clock cycles before the data access. You could place your data blocks so that they fit within a single page, if you had a reason to do so. At the instruction level, you have a choice of cache level.
Prefetch intrinsics are written up in the Intel compiler documentation, and the instructions are described in the chip architecture documents. If those details are more than you care to know, perhaps the default techniques are sufficient.
