"Is there any other way to do what I want?" But what do you want? What specific reason do you have for not wanting to keep the parents around? Are you certain that this is not premature optimisation?
You might make the tasks children of a long-lived task, but the dependence on a shared reference count might harm performance. You might let a parent take on the assignment of one of the children before joining with the rest of them, which seems generally a better choice (go have a look at parallel_for). The absolute best choice probably depends on the exact circumstances, but finding it may not be where you should direct your attention.