You might be interested in this method of weighting neural networks in 2 different spaces to get layer-wise fully connected networks with only a few weights per neuron and potentially some other benefits.
I haven't read through your blog since it's not available by visiting the link submitted, but I think that weighting ANN in 2 different spaces is not efficient or not really possible. Thanks.
