updated by @sarah8: 04/23/15 08:38:07PM
This is a novice answer, so please take it with a grain of salt, or better yet, a truffle.
Skipping the step of grinding the nibs into chocolate liquor prior to further processing could affect the particle size or shape of the finished chocolate.
I've read that you can add nibs directly to some machines and the chocolate will come out just fine. However, the processing time will generally be shorter if you work with chocolate liquor. - The particles are already much smaller.
I've read that there is a point at which each cocoa solid particle is coated in a layer of fat. This coating allows the chocolate to glide smoothly and gives it a nice mouthfeel. If the particles are too small, then the surface area for a given amount of cocoa butter is too great and there isn't enough cocoa butter to coat the particles.
See Industrial Chocolate Manufacture and Use Chapter 7
If the cocoa butter is not sufficiently coating the cocoa, sugar, and ?milk? particles, then it would make sense that it cannot form a uniform tempered structure and thus crumble easily.
Again, I'm still a novice and chocolate is complexso I'm sure there is more going on.
I'm not sure what you mean by liquoring. Do you just mean that the chocolate maker who makes the chocolate you use does not pre-grind his nibs before further refining? If so, I don't think that should cause the problems you're having, as that's what myself and many other makers do, and the resulting chocolate tempers fine.